support/download: change format of archives generated from git
authorYann E. MORIN <yann.morin.1998@free.fr>
Mon, 25 Mar 2019 21:48:12 +0000 (22:48 +0100)
committerYann E. MORIN <yann.morin.1998@free.fr>
Sun, 10 Jan 2021 21:06:58 +0000 (22:06 +0100)
commit5b95a5dc27c0d8002c00bda1c867ddea9218087e
tree557ec673614fbbfd4ec94b87c55021bd39c3d214
parentcbe95b1a455bbefcaa90a08cf3dd1a590630921e
support/download: change format of archives generated from git

Switch to using the tarball helper, that can generate reproducible
archives whatever the tar version >= 1.27.

However, those archives are not identical to the previous ones generated
in the (now-broken) gnu format.

To avoid any clashing between old and new archives, and new and old
Buildroot versions, we need to name the new generated archives
differently from the existing ones.

So, we bump the git-specific format-version to -br1.

The %ci date  has been supported by git back to 1.6.0, released August
2008); it is not strictly ISO8601, but is still accepted as a PAX date
header. The strict ISO8601 placeholder, %cI, was only introduced with
2.2.0, release in November 2014, so too recent to be widely available.

As the format and the names of the archives changes, we need to update
all the hash files with the new names and hashes.

Of all the bootloaders that have a git download method, vexpress-firmware
is the only one to have a hash. Others have no hash files, or they have
explicitly set BR_NO_CHECK_HASH_FOR.

For the packages, linux-headers is the special snowflake, as the git
download is only for custom git tree, so it is excluded from the hash
verification with BR_NO_CHECK_HASH_FOR.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vincent Fazio <vfazio@xes-inc.com>
    ---8<------8<------8<------8<---
    #!/bin/sh
    # Find and download all packages using git as backend.
    # Manually fix hashes for affected packages.

    # Packages that only have a host variant
    HOST_ONLY='imx-mkimage|mxsldr|netsurf-buildsystem|opkg-utils|prelink-cross|qoriq-rcw|vboot-utils'

    # Packages that have a non-git main _SOURCE, and/or which
    # have BR_NO_CHECK_HASH_FOR for the git _SOURCE
    NOT_GIT='aufs|aufs-util|xenomai|linux-headers'

    export BR2_DL_DIR=$(pwd)/temp-dl-dir

    make defconfig
    make $( git grep -l -E 'SITE_METHOD[[:space:]]*:?=[[:space:]]*git\>|_SITE[[:space:]]*:?=[[:space:]]*git:' \
                boot/vexpress-firmware/ package/ \
            |sed -r -e 's,.*/([^/]+)\.mk,\1,' \
            |sed -r -e '/^('"${NOT_GIT}"')$/d;' \
                    -e 's/^('"${HOST_ONLY}"')/host-\1/;' \
                    -e 's/$/-legal-info/;'
          )

    ---8<------8<------8<------8<---
89 files changed:
boot/vexpress-firmware/vexpress-firmware.hash
package/aer-inject/aer-inject.hash
package/am33x-cm3/am33x-cm3.hash
package/armbian-firmware/armbian-firmware.hash
package/at/at.hash
package/azure-iot-sdk-c/azure-iot-sdk-c.hash
package/bayer2rgb-neon/bayer2rgb-neon.hash
package/brickd/brickd.hash
package/c-capnproto/c-capnproto.hash
package/dbus-triggerd/dbus-triggerd.hash
package/dt-utils/dt-utils.hash
package/dtv-scan-tables/dtv-scan-tables.hash
package/edid-decode/edid-decode.hash
package/ell/ell.hash
package/flashbench/flashbench.hash
package/fmc/fmc.hash
package/fmlib/fmlib.hash
package/freescale-imx/imx-alsa-plugins/imx-alsa-plugins.hash
package/freescale-imx/imx-lib/imx-lib.hash
package/google-breakpad/google-breakpad.hash
package/gstreamer1/gst1-interpipe/gst1-interpipe.hash
package/gstreamer1/gst1-shark/gst1-shark.hash
package/imx-mkimage/imx-mkimage.hash
package/iwd/iwd.hash
package/kvm-unit-tests/kvm-unit-tests.hash
package/kvmtool/kvmtool.hash
package/libbroadvoice/libbroadvoice.hash
package/libcamera/libcamera.hash
package/libclc/libclc.hash
package/libg7221/libg7221.hash
package/libilbc/libilbc.hash
package/libsilk/libsilk.hash
package/libsvgtiny/libsvgtiny.hash
package/libuci/libuci.hash
package/libyuv/libyuv.hash
package/linux-firmware/linux-firmware.hash
package/linux-syscall-support/linux-syscall-support.hash
package/minicom/minicom.hash
package/mmc-utils/mmc-utils.hash
package/net-tools/net-tools.hash
package/netcat-openbsd/netcat-openbsd.hash
package/netsurf-buildsystem/netsurf-buildsystem.hash
package/odhcp6c/odhcp6c.hash
package/open62541/open62541.hash
package/opkg-utils/opkg-utils.hash
package/piglit/piglit.hash
package/pkg-download.mk
package/prelink-cross/prelink-cross.hash
package/psplash/psplash.hash
package/qoriq-rcw/qoriq-rcw.hash
package/qt-webkit-kiosk/qt-webkit-kiosk.hash
package/qt5/qt5coap/qt5coap.hash
package/qt5/qt5knx/qt5knx.hash
package/qt5/qt5mqtt/qt5mqtt.hash
package/rtc-tools/rtc-tools.hash
package/rtmpdump/rtmpdump.hash
package/speexdsp/speexdsp.hash
package/stm32flash/stm32flash.hash
package/ti-sgx-demos/ti-sgx-demos.hash
package/ti-sgx-km/ti-sgx-km.hash
package/ti-sgx-um/ti-sgx-um.hash
package/trace-cmd/trace-cmd.hash
package/tremor/tremor.hash
package/ubus/ubus.hash
package/uclibc-ng-test/uclibc-ng-test.hash
package/uemacs/uemacs.hash
package/uhttpd/uhttpd.hash
package/ustream-ssl/ustream-ssl.hash
package/vboot-utils/vboot-utils.hash
package/wilink-bt-firmware/wilink-bt-firmware.hash
package/x11r7/xdriver_xf86-video-imx-viv/xdriver_xf86-video-imx-viv.hash
package/x11r7/xdriver_xf86-video-intel/xdriver_xf86-video-intel.hash
support/download/git
support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash
support/testing/tests/download/br2-external/git-hash/package/good/good.hash
support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-branch-head/git-partial-sha1-branch-head.hash
support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-branch/git-partial-sha1-reachable-by-branch.hash
support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-reachable-by-tag/git-partial-sha1-reachable-by-tag.hash
support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-itself/git-partial-sha1-tag-itself.hash
support/testing/tests/download/br2-external/git-refs/package/git-partial-sha1-tag-points-to/git-partial-sha1-tag-points-to.hash
support/testing/tests/download/br2-external/git-refs/package/git-sha1-branch-head/git-sha1-branch-head.hash
support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-branch/git-sha1-reachable-by-branch.hash
support/testing/tests/download/br2-external/git-refs/package/git-sha1-reachable-by-tag/git-sha1-reachable-by-tag.hash
support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-itself/git-sha1-tag-itself.hash
support/testing/tests/download/br2-external/git-refs/package/git-sha1-tag-points-to/git-sha1-tag-points-to.hash
support/testing/tests/download/br2-external/git-refs/package/git-submodule-disabled/git-submodule-disabled.hash
support/testing/tests/download/br2-external/git-refs/package/git-submodule-enabled/git-submodule-enabled.hash
support/testing/tests/download/br2-external/git-refs/package/git-tag/git-tag.hash
support/testing/tests/download/br2-external/git-refs/package/git-wrong-content/git-wrong-content.hash