Compare commits

..

85 Commits
main ... main

Author SHA1 Message Date
Jochen Sprickerhof 98b3c7f2cd
m-a-b-q: replace test_installed by dpkg-checkbuilddeps 2 weeks ago
Johannes Schauer Marin Rodrigues 8e2f62d08c
release 1.5.2 1 month ago
Johannes Schauer Marin Rodrigues 8130f1cef0
README.md: add new authors 1 month ago
Johannes Schauer Marin Rodrigues 6262c1921c
tests/check-against-debootstrap-dist: allow for different installation order of passwd and systemd 1 month ago
Johannes Schauer Marin Rodrigues 500b0d2512
mmdebstrap: fix perltidy formatting 1 month ago
Chris Hofstaedtler 37678c4fb5
tests/check-against-debootstrap-dist: allow /var/log/{faillog,lastlog} to be absent 1 month ago
Johannes Schauer Marin Rodrigues 134330d786
mmdebstrap-autopkgtest-build-qemu: pipe tarball to mke2fs and make bit-by-bit reproducible 1 month ago
Colin Watson bda207af63
mmdebstrap-autopkgtest-build-qemu: selects wrong kernel package for armhf
Package: mmdebstrap
Version: 1.5.0-2
Severity: normal
Tags: patch

With --architecture=armhf, mmdebstrap-autopkgtest-build-qemu fails with
"E: Unable to locate package linux-image-armhf".  The following patch
fixes that.

i386 has a similar problem, and I included a fix for that in this patch
too, though I haven't tested that.

After this, armhf still fails with:

  arm-linux-gnueabihf-objdump: /usr/lib/systemd/boot/efi/linuxarm.efi.stub: file format not recognized
  failed to discover the alignment of the efi stub

... but that's a separate problem.
1 month ago
Johannes Schauer Marin Rodrigues d0c30c70bd
tests/debootstrap: dump $tmpdir/debootstrap/debootstrap.log on failure 1 month ago
Johannes Schauer Marin Rodrigues 1a4bb39aad
check if libarchive is available for ext4 format 1 month ago
Johannes Schauer Marin Rodrigues dfeb21cfe5
tests/empty-sources.list: only write out /etc/apt/sources.list if it doesn't exist yet
In the mmdebstrap autpkgtest, debian/tests/sourcesfilter will write out
/etc/apt/sources.list before the test's setup-hook is run. Thus, the
test's setup-hook will overwrite the contents of the sources.list that
debian/tests/sourcesfilter set up, which will end up pulling in the
wrong packages.

Thus, only write out our own /etc/apt/sources.list if it does not yet
exist. If it does exist, nothing needs to be done.
2 months ago
Jochen Sprickerhof d0add325d3
Mark oldstable as bullseye_or_later 2 months ago
Johannes Schauer Marin Rodrigues d4149bb4db
coverage.sh: add missing newline at end of curl output 2 months ago
Johannes Schauer Marin Rodrigues 61509691a8
tests: base-files now ships merged-/usr symlinks 2 months ago
Johannes Schauer Marin Rodrigues 3969727cc0
tests: bug #1031276 got fixed 2 months ago
Johannes Schauer Marin Rodrigues 6818d2ed40
release 1.5.1 2 months ago
Johannes Schauer Marin Rodrigues 9f8172bbc0
Switch variant apt from 'apt-get dist-upgrade' to apt patterns
The dist-upgrade method will fail to install essential packages but
still exit successfully without reporting an error, see #1072218
2 months ago
Johannes Schauer Marin Rodrigues 767e8a8bfb
run_qemu.sh: allow setting MMDEBSTRAP_TESTS_DEBUG and add auto-detection for amlogic a311d bananapi 2 months ago
Johannes Schauer Marin Rodrigues abc66c5dc7
increase additional number of blocks from 1.1 to 1.2 times the computed number 2 months ago
Johannes Schauer Marin Rodrigues f3ea5f2676
add flock on temporary directory in /tmp
Since systemd 256~rc3-3, /tmp is regularly cleaned up, removing files
older than 10 days. Since a rootfs contains files with timestamps
potentially much older than that, we exclude our temporary directory by
adding an exclusive lock on it which will stop systemd-tmpfiles from
cleaning up anything in it.

Thanks: Peter Pentchev <roam@ringlet.net>
2 months ago
Johannes Schauer Marin Rodrigues 5f491f2955
make_mirror.sh: force systemd to not mount /tmp as tmpfs
Since systemd 256~rc3-3, /tmp is mounted as tmpfs by default. This
breaks our tests because /tmp is mounted with nodev which makes it
impossible for debootstrap to mknod.
2 months ago
Johannes Schauer Marin Rodrigues 821c2e1328
In unshare mode, make all mounts private recursively
This emulates what unshare(1) does by default or by passing
--propagation=private explicitly. Mounting and unmounting filesystems
will affect mounts outside the namespace which are marked as shared (see
last column of `findmnt -o+PROPAGATION`). Since mmdebstrap's goal is to
isolate the mounts in the new namespace, we perform the equivalent of

    mount(NULL, "/", MS_REC | MS_PRIVATE, NULL);

from util-linux/sys-utils/unshare.c:set_propagation() which is in shell:

    mount --make-rprivate /

See mount_namespaces(7) for details. Without setting this, unmounting
/sys (and its sub-mounts) in unshare mode as root user will also unmount
the sub-mounts of /sys on the outside of the namespace. This breaks
tests/unshare-as-root-user which will fail to shut down with the following
errors in the log:

[FAILED] Failed unmounting mnt.mount - /mnt.
[FAILED] Failed unmounting run-lock.mount - Legacy Locks Directory /run/lock.
[...]
[  OK  ] Reached target poweroff.target - System Power Off.

Afterwards it will stall indefinitely. Stopping mmdebstrap from messing
with the /sys mounts on the outside stops this behaviour and allows to
cleanly shut down the virtual machine.

Thanks: Helmut Grohne
2 months ago
Johannes Schauer Marin Rodrigues 84f80673f4
Revert "attempt diagnosing skip-tar-in-mknod failure 'file changed as we read it' using auditd"
This reverts commit 726fc38d1d.
2 months ago
Johannes Schauer Marin Rodrigues e3eafd0009
release 1.5.0 2 months ago
Johannes Schauer Marin Rodrigues eed6a86480
mmdebstrap-autopkgtest-build-qemu: document https://github.com/util-linux/util-linux/issues/2981 2 months ago
Johannes Schauer Marin Rodrigues 5a06c67aaa
document that --cache-dir is also not a supported debootstrap option 2 months ago
Johannes Schauer Marin Rodrigues d26afd110f
add more docs to the tar-out special hook 2 months ago
Johannes Schauer Marin Rodrigues 4ad8245a14
tests/missing-dev-sys-proc-inside-the-chroot: work around for bug #1071078 2 months ago
Johannes Schauer Marin Rodrigues 409686048b
add --format=ext4 3 months ago
Jochen Sprickerhof 4a294f05bd
Add test if dpkg-dev is installed
Needed for dpkg-architecture.
3 months ago
Jochen Sprickerhof 8c0ddc3266
mmdebstrap-autopkgtest-build-qemu: use mount --no-mtab
Otherwise it fails with:

umount: /tmp/mmdebstrap.Tw9G7ZLL4J/mnt: filesystem was unmounted, but failed to update userspace mount table.
E: setup failed: E: command failed: umount --lazy "$1/mnt"

Also umount mnt/dev.
3 months ago
Johannes Schauer Marin Rodrigues 1e68ffd2c4
tests: changelog.Debian.$foreign_arch.gz files are not always present 4 months ago
Johannes Schauer Marin Rodrigues 726fc38d1d
attempt diagnosing skip-tar-in-mknod failure 'file changed as we read it' using auditd 4 months ago
Johannes Schauer Marin Rodrigues ae09a50f9d
document unshare --map-auto --map-user=65536 --map-group=65536 --keep-caps trick 4 months ago
Johannes Schauer Marin Rodrigues 9726836ac4
mmdebstrap-autopkgtest-build-qemu: add documentation of some unshare magic 4 months ago
Johannes Schauer Marin Rodrigues cdf6959a41
make_mirror.sh: retry apt with verbose output 4 months ago
Johannes Schauer Marin Rodrigues 1cf0d87a60
hooks/file-mirror-automount/setup00.sh: prefix warning with W: 4 months ago
Johannes Schauer Marin Rodrigues 0973de1530
hooks/copy-host-apt-sources-and-preferences/setup00.sh: document with comment on top 4 months ago
Johannes Schauer Marin Rodrigues d883fa13bb
hooks/maybe-merged-usr: prepare for the time when usr-is-merged exists only as a virtual package 5 months ago
Max-Julian Pogner 286cecc21b
follow adduser's changes in example: --gecos => --comment
Considered References:
https://www.debian.org/releases/bookworm/amd64/release-notes/ch-information.en.html#adduser-changes
/usr/share/doc/adduser/NEWS.Debian.gz (from adduser v3.134)
https://manpages.debian.org/bookworm/adduser/adduser.8.en.html
6 months ago
Johannes Schauer Marin Rodrigues 113532b3e1
refactor worker function to remove code duplication
Thanks: Guillem Jover <guillem@debian.org>
6 months ago
Johannes Schauer Marin Rodrigues d244f4f1de
release 1.4.3 6 months ago
Johannes Schauer Marin Rodrigues 81589889f9
check for dpkg-dev being installed for dpkg-architecture when doing foreign fakechroot 6 months ago
Johannes Schauer Marin Rodrigues 35cd477fea
Take hard links into account when computing disk usage based on dpkg-gencontrol.pl
Thanks: Guillem Jover <guillem@debian.org>, Sven Joachim <svenjoac@gmx.de>
6 months ago
Johannes Schauer Marin Rodrigues a7586e55d1
coverage.txt: exclude create-foreign-tarball from arm64 in fakechroot mode because usrmerge postinst under fakechroot wants to copy /lib/ld-linux-x86-64.so.2 (which does not exist) instead of /lib64/ld-linux-x86-64.so.2 6 months ago
Johannes Schauer Marin Rodrigues 65c27a55b3
cleanup start-stop-daemon without root prefix when performing a pivot-root 6 months ago
Johannes Schauer Marin Rodrigues 59c9c399c6
remove leftover debugging output 6 months ago
Johannes Schauer Marin Rodrigues e661b79749
mmdebstrap-autopkgtest-build-qemu: add reasons for image being raw and not qcow2 6 months ago
Johannes Schauer Marin Rodrigues 4bcd6fa015
rename install-libmagic-mgc-on-arm64 test and allow running it on both amd64 and arm64 6 months ago
Johannes Schauer Marin Rodrigues b54564a84d
release 1.4.2 6 months ago
Johannes Schauer Marin Rodrigues ba6e9af9a2
README.md: update list of contributors 6 months ago
Francesco Poli 8410dc6636
mmdebstrap-autopkgtest-build-qemu: fix octal mode computation 6 months ago
Johannes Schauer Marin Rodrigues 3e233e10df
mmdebstrap-autopkgtest-build-qemu: make the hostname 'host' as it is done by autopkgtest-build-qemu as it is expected by sbuild-qemu-update and sbuild-qemu-boot 6 months ago
Johannes Schauer Marin Rodrigues 79ef2e3437
tests/remove-start-stop-daemon-and-policy-rc-d-in-hook: remove /usr/sbin/start-stop-daemon and not /sbin/start-stop-daemon 6 months ago
Johannes Schauer Marin Rodrigues 2e7a3ae8b7
remove docs concerning qemu-user-static 6 months ago
Johannes Schauer Marin Rodrigues cc831fc276
remove clean-up of qemu-user-static as it is not copied into the chroot anymore 6 months ago
Johannes Schauer Marin Rodrigues 366d2ffbec
rename create-arm64-tarball test and allow running it on both amd64 and arm64 6 months ago
Johannes Schauer Marin Rodrigues a8583eb39b
fix documentation of buildd variant to only include essential, apt and build-essential 6 months ago
Johannes Schauer Marin Rodrigues ac2aba5074
fix spelling enviroment -> environment 6 months ago
Johannes Schauer Marin Rodrigues 134fc15634
rename include-libmagic-mgc-arm64 tests and allow running them on both amd64 and arm64 6 months ago
Johannes Schauer Marin Rodrigues 4d72f617d9
dpkg 1.22.3 moved start-stop-daemon from /sbin to /usr/sbin, see #1059982 6 months ago
Johannes Schauer Marin Rodrigues ae5bddb2aa
coverage.sh: anticipate more variations for CMD 6 months ago
Johannes Schauer Marin Rodrigues 1c669e8f86
tests/chrootless-fakeroot: exclude /var/log/journal and /etc/credstore* from tarballs instead of trying to fix them up 6 months ago
Johannes Schauer Marin Rodrigues 4c87024356
release 1.4.1 6 months ago
Johannes Schauer Marin Rodrigues 2f768b07dc
coverage.sh: check for pod2man errors 6 months ago
Johannes Schauer Marin Rodrigues 4ca0556cd2
mmdebstrap-autopkgtest-build-qemu: usability and man page improvements
- explicitly instruct to add --boot=efi to autopkgtest-virt-qemu
 - add example how to run autopkgtest with --boot=efi
 - document image location requirements giving unshare restrictions
 - check if foreign arch is configured
 - instruct how to add a foreign architecture
 - check that the unshared user is able to access the image location
 - suggest to install qemu-system-* packages if they are missing
 - suggest to install packages containing EDK II OVMF UEFI firmware
6 months ago
Johannes Schauer Marin Rodrigues d9f9c64ac2
do not fail during cleanup if /etc/apt/apt.conf.d/00mmdebstrap got removed, only warn 6 months ago
Johannes Schauer Marin Rodrigues dd94ee3b84
read files passed as --aptopt and --dpkgopt outside the unshared namespace to avoid permission issues 6 months ago
Johannes Schauer Marin Rodrigues 99d2579e0b
document that the required and minbase variants do not explicitly install apt 6 months ago
Johannes Schauer Marin Rodrigues 610058d105
document how SUITE influences the selection of essential packages 6 months ago
Johannes Schauer Marin Rodrigues 2ff8f6142d
document how to run chrootless mode wrapped inside mmdebstrap 6 months ago
Johannes Schauer Marin Rodrigues 417d958a14
document how to remove a directory created with unshare mode 7 months ago
Johannes Schauer Marin Rodrigues 8674e11c71
allow for /etc/resolv.conf and /etc/hostname to already exist inside the chroot without warning about it 7 months ago
Johannes Schauer Marin Rodrigues daa886264b
reword the first few paragraphs
Thanks: Raphaël Hertzog
7 months ago
Johannes Schauer Marin Rodrigues d157ba2b9a
only print short --help output if wrong args are passed 7 months ago
Johannes Schauer Marin Rodrigues 884a04b18a
make_mirror.sh: disable networking 7 months ago
Johannes Schauer Marin Rodrigues 014a9c30a5
tests/check-against-debootstrap-dist: systemd 255 dropped split-/usr support 7 months ago
Johannes Schauer Marin Rodrigues 90fe7941bb
tests/check-against-debootstrap-dist: debootstrap installs prio:required packages in oldstable and stable for the buildd profile 7 months ago
Johannes Schauer Marin Rodrigues 428ee78121
disallow running chrootless as root without fakeroot unless --skip=check/chrootless is used 7 months ago
Johannes Schauer Marin Rodrigues ae6dcc001d
tests/install-busybox-based-sub-essential-system: busybox 1:1.36.1-6 moved to /usr 7 months ago
Johannes Schauer Marin Rodrigues b4ba78897b
make_mirror.sh: add newline at the end of /etc/hosts so that appending writes to the next line and not the current 7 months ago
Johannes Schauer Marin Rodrigues 69954515e7
tests/chrootless-fakeroot: also fix permissions and extended attributes for /etc/credstore 7 months ago
Johannes Schauer Marin Rodrigues 136cbdf0f1
run_qemu.sh: replace storing the pid and kill it a trap by using 'setpriv --pdeathsig TERM' 7 months ago
Johannes Schauer Marin Rodrigues 87edb1c2d1
coverage.sh: also run shellcheck on mmdebstrap-autopkgtest-build-qemu 7 months ago
Johannes Schauer Marin Rodrigues 0de9e19ca4
make_mirror.sh: explicitly install passwd since systemd 254.4-1 doesn't pull it in anymore 7 months ago

@ -1,3 +1,39 @@
1.5.2 (2024-06-26)
------------------
- mmdebstrap-autopkgtest-build-qemu produces bit-by-bit reproducible output
1.5.1 (2024-06-03)
------------------
- in root and unshare mode, run 'mount --make-rprivate /' before bind-mounting
- switch apt variant from using 'apt-get dist-upgrade' to apt patterns
1.5.0 (2024-05-14)
------------------
- add --format=ext4
1.4.3 (2024-02-01)
------------------
- take hard links into account when computing disk usage
1.4.2 (2024-01-29)
------------------
- allow for start-stop-daemon to be in either /sbin or /usr/sbin
- mmdebstrap-autopkgtest-build-qemu: fix octal mode computation and hostname
1.4.1 (2024-01-09)
------------------
- set DPkg::Chroot-Directory in APT_CONFIG to simplify calling apt in hooks
- disallow running chrootless as root without fakeroot unless
--skip=check/chrootless is used
- only print short --help output if wrong args are passed
- read files passed as --aptopt and --dpkgopt outside the unshared namespace
1.4.0 (2023-10-24)
------------------

@ -158,15 +158,19 @@ Contributors
- Johannes Schauer Marin Rodrigues (main author)
- Helmut Grohne
- Jochen Sprickerhof
- Gioele Barabucci
- Benjamin Drung
- Jochen Sprickerhof
- Josh Triplett
- Konstantin Demin
- Chris Hofstaedtler
- Colin Watson
- David Kalnischkies
- Emilio Pozuelo Monfort
- Francesco Poli
- Jakub Wilk
- Joe Groocock
- Max-Julian Pogner
- Nicolas Vigier
- Raul Tambre
- Steve Dodd

@ -34,7 +34,7 @@ all_variants = [
"standard",
]
default_format = "auto"
all_formats = ["auto", "directory", "tar", "squashfs", "ext2", "null"]
all_formats = ["auto", "directory", "tar", "squashfs", "ext2", "ext4", "null"]
mirror = os.getenv("mirror", "http://127.0.0.1/debian")
hostarch = subprocess.check_output(["dpkg", "--print-architecture"]).decode().strip()

@ -2,11 +2,21 @@
set -eu
if [ -e ./mmdebstrap ]; then
# by default, use the mmdebstrap executable in the current directory together
# with perl Devel::Cover but allow to overwrite this
: "${CMD:=perl -MDevel::Cover=-silent,-nogcov ./mmdebstrap}"
case "$CMD" in
"mmdebstrap "*|mmdebstrap|*" mmdebstrap"|*" mmdebstrap "*)
MMSCRIPT="$(command -v mmdebstrap 2>/dev/null)";;
*) MMSCRIPT=./mmdebstrap;;
esac
if [ -e "$MMSCRIPT" ]; then
TMPFILE=$(mktemp)
perltidy < ./mmdebstrap > "$TMPFILE"
perltidy < "$MMSCRIPT" > "$TMPFILE"
ret=0
diff -u ./mmdebstrap "$TMPFILE" || ret=$?
diff -u "$MMSCRIPT" "$TMPFILE" || ret=$?
if [ "$ret" -ne 0 ]; then
echo "perltidy failed" >&2
rm "$TMPFILE"
@ -14,12 +24,14 @@ if [ -e ./mmdebstrap ]; then
fi
rm "$TMPFILE"
if [ "$(sed -e '/^__END__$/,$d' ./mmdebstrap | wc --max-line-length)" -gt 79 ]; then
if [ "$(sed -e '/^__END__$/,$d' "$MMSCRIPT" | wc --max-line-length)" -gt 79 ]; then
echo "exceeded maximum line length of 79 characters" >&2
exit 1
fi
perlcritic --severity 4 --verbose 8 ./mmdebstrap
perlcritic --severity 4 --verbose 8 "$MMSCRIPT"
pod2man "$MMSCRIPT" >/dev/null
fi
for f in tarfilter coverage.py caching_proxy.py; do
@ -27,7 +39,7 @@ for f in tarfilter coverage.py caching_proxy.py; do
black --check "./$f"
done
shellcheck --exclude=SC2016 coverage.sh make_mirror.sh run_null.sh run_qemu.sh gpgvnoexpkeysig hooks/*/*.sh
shellcheck --exclude=SC2016 coverage.sh make_mirror.sh run_null.sh run_qemu.sh gpgvnoexpkeysig mmdebstrap-autopkgtest-build-qemu hooks/*/*.sh
mirrordir="./shared/cache/debian"
@ -63,9 +75,6 @@ export LC_ALL=C.UTF-8
: "${HAVE_BINFMT:=yes}"
# by default, use the mmdebstrap executable in the current directory together
# with perl Devel::Cover but allow to overwrite this
: "${CMD:=perl -MDevel::Cover=-silent,-nogcov ./mmdebstrap}"
mirror="http://127.0.0.1/debian"
export HAVE_QEMU HAVE_BINFMT RUN_MA_SAME_TESTS DEFAULT_DIST SOURCE_DATE_EPOCH CMD mirror
@ -97,7 +106,7 @@ fi
# check if the wiki has to be updated with pod2markdown output
if [ "${DEBEMAIL:-}" = "josch@debian.org" ]; then
bash -exc "diff -u <(curl --silent https://gitlab.mister-muffin.de/josch/mmdebstrap/wiki/raw/Home | dos2unix) <(pod2markdown < mmdebstrap)" || :
bash -exc "diff -u <(curl --silent https://gitlab.mister-muffin.de/josch/mmdebstrap/wiki/raw/Home | dos2unix; echo) <(pod2markdown < mmdebstrap)" || :
fi
rm -f shared/test.sh shared/tar1.txt shared/tar2.txt shared/pkglist.txt shared/doc-debian.tar.list shared/mmdebstrap shared/tarfilter shared/proxysolver

@ -59,21 +59,21 @@ Needs-QEMU: true
Test: mmdebstrap
Needs-Root: true
Modes: root
Formats: tar squashfs ext2
Formats: tar squashfs ext2 ext4
Variants: essential apt minbase buildd - standard
Skip-If:
variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558
mode == "fakechroot" and variant in ["-", "standard"] # no extended attributes
variant == "standard" and hostarch in ["armel", "armhf", "mipsel"] # #1031276
variant == "standard" and dist in ["oldstable", "stable"] and hostarch in ["armel", "armhf", "mipsel"] # #1031276
Test: check-for-bit-by-bit-identical-format-output
Modes: unshare fakechroot
Formats: tar squashfs ext2
Formats: tar squashfs ext2 ext4
Variants: essential apt minbase buildd - standard
Skip-If:
variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558
mode == "fakechroot" and variant in ["-", "standard"] # no extended attributes
variant == "standard" and hostarch in ["armel", "armhf", "mipsel"] # #1031276
variant == "standard" and dist in ["oldstable", "stable"] and hostarch in ["armel", "armhf", "mipsel"] # #1031276
Test: tarfilter-idshift
Needs-QEMU: true
@ -194,18 +194,18 @@ Skip-If:
hostarch != "amd64"
not run_ma_same_tests
Test: include-libmagic-mgc-arm64
Test: include-foreign-libmagic-mgc
Needs-Root: true
Needs-APT-Config: true
Skip-If:
hostarch != "amd64"
hostarch not in ["amd64", "arm64"]
not run_ma_same_tests
Test: include-libmagic-mgc-arm64-with-multiple-arch-options
Test: include-foreign-libmagic-mgc-with-multiple-arch-options
Needs-Root: true
Needs-APT-Config: true
Skip-If:
hostarch != "amd64"
hostarch not in ["amd64", "arm64"]
not run_ma_same_tests
Test: aptopt
@ -273,17 +273,13 @@ Test: debootstrap-no-op-options
Needs-Root: true
Test: verbose
Variants: - standard
Variants: standard
Skip-If:
variant == "-" and hostarch not in ["armel", "armhf", "mipsel"] # #1031276
variant == "standard" and hostarch in ["armel", "armhf", "mipsel"] # #1031276
variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558
Test: debug
Variants: - standard
Variants: standard
Skip-If:
variant == "-" and hostarch not in ["armel", "armhf", "mipsel"] # #1031276
variant == "standard" and hostarch in ["armel", "armhf", "mipsel"] # #1031276
variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558
Test: quiet
@ -365,21 +361,22 @@ Variants: custom
Modes: chrootless
Needs-APT-Config: true
Test: install-libmagic-mgc-on-arm64
Test: install-libmagic-mgc-on-foreign
Variants: custom
Modes: chrootless
Skip-If:
hostarch != "amd64"
hostarch not in ["amd64", "arm64"]
not have_binfmt
Test: install-busybox-based-sub-essential-system
Needs-Root: true
Test: create-arm64-tarball
Test: create-foreign-tarball
Modes: root unshare fakechroot
Skip-If:
hostarch != "amd64"
hostarch not in ["amd64", "arm64"]
mode == "fakechroot" and not run_ma_same_tests
mode == "fakechroot" and hostarch == "arm64" # usrmerge postinst under fakechroot wants to copy /lib/ld-linux-x86-64.so.2 (which does not exist) instead of /lib64/ld-linux-x86-64.so.2
not have_binfmt
Test: no-sbin-in-path

@ -1,4 +1,13 @@
#!/bin/sh
#
# This script makes sure that the apt sources.list and preferences from outside
# the chroot also exist inside the chroot by *appending* them to any existing
# files. If you do not want to keep the original content, add another setup
# hook before this one which cleans up the files you don't want to keep.
#
# If instead of copying sources.list verbatim you want to mangle its contents,
# consider using python-apt for that. An example can be found in the Debian
# packaging of mmdebstrap in ./debian/tests/sourcesfilter
set -eu

@ -15,7 +15,7 @@ env APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get indextargets --no-release-info -
| while read -r path; do
mkdir -p "$rootdir/run/mmdebstrap"
if [ ! -d "/$path" ]; then
echo "/$path is not an existing directory" >&2
echo "W: /$path is not an existing directory" >&2
continue
fi
case $MMDEBSTRAP_MODE in

@ -15,6 +15,10 @@ case "$ver" in
echo "usr-is-merged package from src:usrmerge installed -- not running merged-usr essential hook" >&2
exit 0
;;
'not-installed ')
echo "usr-is-merged was not installed in a previous hook -- not running merged-usr essential hook" >&2
exit 0
;;
*)
echo "unexpected situation for package usr-is-merged: $ver" >&2
exit 1

@ -4,12 +4,22 @@ set -eu
env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get update --error-on=any
# if the usr-is-merged package cannot be installed with apt, do nothing
if ! env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged > /dev/null 2>&1; then
echo "no package called usr-is-merged found -- not running merged-usr extract hook" >&2
exit 0
if env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged > /dev/null 2>&1; then
# if apt-cache exited successfully, then usr-is-merged exists either as
# a real or virtual package
if env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged 2>/dev/null | grep -q "Package: usr-is-merged"; then
echo "usr-is-merged found -- running merged-usr extract hook" >&2
else
# The usr-is-merged must be virtual, so assume that nothing
# has to be done. This is the case with Debian Trixie or later
# or with Ubuntu Lunar or later
echo "usr-is-merged found but not real -- not running merged-usr extract hook" >&2
exit 0
fi
else
echo "package usr-is-merged found -- running merged-usr extract hook" >&2
# if the usr-is-merged package cannot be installed with apt, do nothing
echo "no package providing usr-is-merged found -- not running merged-usr extract hook" >&2
exit 0
fi
# resolve the script path using several methods in order:

@ -4,12 +4,22 @@ set -eu
env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get update --error-on=any
# if the usr-is-merged package cannot be installed with apt, do nothing
if ! env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged > /dev/null 2>&1; then
echo "no package called usr-is-merged found -- not running merged-usr setup hook" >&2
exit 0
if env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged > /dev/null 2>&1; then
# if apt-cache exited successfully, then usr-is-merged exists either as
# a real or virtual package
if env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged 2>/dev/null | grep -q "Package: usr-is-merged"; then
echo "usr-is-merged found -- running merged-usr setup hook" >&2
else
# The usr-is-merged must be virtual, so assume that nothing
# has to be done. This is the case with Debian Trixie or later
# or with Ubuntu Lunar or later
echo "usr-is-merged found but not real -- not running merged-usr setup hook" >&2
exit 0
fi
else
echo "package usr-is-merged found -- running merged-usr setup hook" >&2
# if the usr-is-merged package cannot be installed with apt, do nothing
echo "no package providing usr-is-merged found -- not running merged-usr setup hook" >&2
exit 0
fi
# resolve the script path using several methods in order:

@ -33,7 +33,7 @@ deletecache() {
done
# deleting artifacts from test "mmdebstrap"
for variant in essential apt minbase buildd - standard; do
for format in tar ext2 squashfs; do
for format in tar ext2 ext4 squashfs; do
if [ -e "$dir/mmdebstrap-$dist-$variant.$format" ]; then
# attempt to delete for all dists because DEFAULT_DIST might've been different the last time
rm "$dir/mmdebstrap-$dist-$variant.$format"
@ -236,7 +236,11 @@ END
esac
# shellcheck disable=SC2086
APT_CONFIG="$rootdir/etc/apt/apt.conf" apt-get --yes install $pkgs
APT_CONFIG="$rootdir/etc/apt/apt.conf" apt-get --yes install $pkgs \
|| APT_CONFIG="$rootdir/etc/apt/apt.conf" apt-get --yes install \
-oDebug::pkgProblemResolver=true -oDebug::pkgDepCache::Marker=1 \
-oDebug::pkgDepCache::AutoInstall=1 \
$pkgs
rm "$rootdir/var/cache/apt/archives/lock"
rmdir "$rootdir/var/cache/apt/archives/partial"
@ -449,7 +453,7 @@ if [ "$HAVE_QEMU" = "yes" ]; then
tmpdir="$(mktemp -d)"
trap 'kill "$PROXYPID" || :;cleanuptmpdir; cleanup_newcachedir' EXIT INT TERM
pkgs=perl-doc,systemd-sysv,perl,arch-test,fakechroot,fakeroot,mount,uidmap,qemu-user-static,qemu-user,dpkg-dev,mini-httpd,libdevel-cover-perl,libtemplate-perl,debootstrap,procps,apt-cudf,aspcud,python3,libcap2-bin,gpg,debootstrap,distro-info-data,iproute2,ubuntu-keyring,apt-utils,squashfs-tools-ng,genext2fs,linux-image-generic
pkgs=perl-doc,systemd-sysv,perl,arch-test,fakechroot,fakeroot,mount,uidmap,qemu-user-static,qemu-user,dpkg-dev,mini-httpd,libdevel-cover-perl,libtemplate-perl,debootstrap,procps,apt-cudf,aspcud,python3,libcap2-bin,gpg,debootstrap,distro-info-data,iproute2,ubuntu-keyring,apt-utils,squashfs-tools-ng,genext2fs,linux-image-generic,passwd,e2fsprogs,uuid-runtime
if [ ! -e ./mmdebstrap ]; then
pkgs="$pkgs,mmdebstrap"
fi
@ -493,6 +497,8 @@ mount -t 9p -o trans=virtio,access=any,msize=128k mmdebstrap /mnt
# need to restart mini-httpd because we mounted different content into www-root
systemctl restart mini-httpd
ip link set enp0s1 down || :
handler () {
while IFS= read -r line || [ -n "$line" ]; do
printf "%s %s: %s\n" "$(date -u -d "0 $(date +%s.%3N) seconds - $2 seconds" +"%T.%3N")" "$1" "$line"
@ -530,7 +536,8 @@ END
fi
# set PATH to pick up the correct mmdebstrap variant
env PATH="$(dirname "$(realpath --canonicalize-existing "$CMD")"):$PATH" \
debvm-create --skip=usrmerge --size="$DISK_SIZE" --release="$DEFAULT_DIST" \
debvm-create --skip=usrmerge,systemdnetwork \
--size="$DISK_SIZE" --release="$DEFAULT_DIST" \
--output="$newcachedir/debian-$DEFAULT_DIST.ext4" -- \
--architectures="$arches" --include="$pkgs" \
--setup-hook='echo "Acquire::http::Proxy \"http://127.0.0.1:8080/\";" > "$1/etc/apt/apt.conf.d/00proxy"' \
@ -541,8 +548,9 @@ END
--customize-hook='touch "$1/mmdebstrap-testenv"' \
--customize-hook='copy-in "'"$tmpdir"'/mmdebstrap.service" /etc/systemd/system/' \
--customize-hook='copy-in "'"$tmpdir"'/worker.sh" /' \
--customize-hook='printf 127.0.0.1 localhost > "$1/etc/hosts"' \
--customize-hook='echo 127.0.0.1 localhost > "$1/etc/hosts"' \
--customize-hook='printf "START=1\nDAEMON_OPTS=\"-h 127.0.0.1 -p 80 -u nobody -dd /mnt/cache -i /var/run/mini-httpd.pid -T UTF-8\"\n" > "$1/etc/default/mini-httpd"' \
--customize-hook='touch "$1/etc/systemd/system/tmp.mount"' \
"$mirror"
kill $PROXYPID

File diff suppressed because it is too large Load Diff

@ -29,8 +29,11 @@ B<mmdebstrap-autopkgtest-build-qemu> [I<OPTIONS>] B<--boot>=B<efi> I<RELEASE> I<
B<mmdebstrap-autopkgtest-build-qemu> is a mostly compatible drop-in replacement
for L<autopkgtest-build-qemu(1)> with two main differences: Firstly, it uses
L<mmdebstrap(1)> instead of L<vmdb2(1)> and thus is able to create QEMU disk
images without requiring superuser privileges. Secondly, it uses
L<systemd-boot(7)> and thus only supports booting via EFI.
images without requiring superuser privileges and with bit-by-bit reproducible
output. Secondly, it uses L<systemd-boot(7)> and thus only supports booting via
EFI. For architectures for which L<autopkgtest-virt-qemu(1)> does not default
to EFI booting you must pass B<--boot=efi> when invoking the autopkgtest virt
backend.
=head1 POSITIONAL PARAMETERS
@ -96,9 +99,23 @@ Passes an additional B<--keyring> parameter to B<mmdebstrap>.
=head1 EXAMPLES
$ mmdebstrap-autopkgtest-build-qemu --boot=efi stable /path/to/debian-stable-i386.img i386
Make sure, that F</path/to/debian-unstable.img> is a path that the unshared
user has access to. This can be done by ensuring world-execute permissions on
all path components or by creating the image in a world-readable directory like
/tmp before copying it into its final location.
$ mmdebstrap-autopkgtest-build-qemu --boot=efi unstable /path/to/debian-unstable.img
$ mmdebstrap-autopkgtest-build-qemu --boot=efi --arch=amd64 unstable /path/to/debian-unstable.img
[...]
$ autopkgtest mypackage -- qemu --boot=efi --dpkg-architecture=amd64 /path/to/debian-unstable.img
Make sure to add B<--boot=efi> to both the B<mmdebstrap-autopkgtest-build-qemu>
as well as the B<autopkgtest-virt-qemu> invocation.
Create bit-by-bit reproducible images from a given snapshot.d.o timestamp.
SOURCE_DATE_EPOCH=1612543740 mmdebstrap-autopkgtest-build-qemu --boot=efi \
--mirror=http://snapshot.debian.org/archive/debian/20210205T164900Z/ \
unstable /path/to/debian-unstable.img
=head1 SEE ALSO
@ -223,37 +240,60 @@ test "$BOOT" = efi ||
case "$ARCHITECTURE" in
amd64)
EFIIMG=bootx64.efi
QEMUARCH=x86_64
VMFPKG=ovmf
LINUXIMAGE=linux-image-amd64
;;
arm64)
EFIIMG=bootaa64.efi
QEMUARCH=aarch64
VMFPKG=qemu-efi-aarch64
LINUXIMAGE=linux-image-arm64
;;
armhf)
EFIIMG=bootarm.efi
QEMUARCH=arm
VMFPKG=qemu-efi-arm
LINUXIMAGE=linux-image-armmp
;;
i386)
EFIIMG=bootia32.efi
QEMUARCH=i386
VMFPKG=ovmf-ia32
LINUXIMAGE=linux-image-686-pae
;;
riscv64)
EFIIMG=bootriscv64.efi
QEMUARCH=riscv64
VMFPKG=
LINUXIMAGE=linux-image-riscv64
;;
*)
die "unsupported architecture"
die "unsupported architecture: $ARCHITECTURE"
;;
esac
if test "$(dpkg-query -f '${db:Status-Status}' -W binutils-multiarch)" = installed; then
GNU_PREFIX=
BINUTILS=
else
GNU_ARCHITECTURE="$(dpkg-architecture "-a$ARCHITECTURE" -qDEB_HOST_GNU_TYPE)"
GNU_PREFIX="$GNU_ARCHITECTURE-"
GNU_SUFFIX="-$(echo "$GNU_ARCHITECTURE" | tr _ -)"
test "$(dpkg-query -f '${db:Status-Status}' -W "binutils$GNU_SUFFIX")" = installed ||
die "please install binutils$GNU_SUFFIX or binutils-multiarch"
BINUTILS=", binutils$GNU_SUFFIX | binutils-multiarch"
fi
for pkg in autopkgtest dosfstools e2fsprogs fdisk mount mtools passwd "systemd-boot-efi:$ARCHITECTURE" uidmap; do
test "$(dpkg-query -f '${db:Status-Status}' -W "$pkg")" = installed ||
die "please install $pkg"
done
arches=" $(dpkg --print-architecture) $(dpkg --print-foreign-architectures | tr '\n' ' ') "
case $arches in
*" $ARCHITECTURE "*) : ;; # nothing to do
*) die "enable $ARCHITECTURE by running: sudo dpkg --add-architecture $ARCHITECTURE && sudo apt update" ;;
esac
test "$(dpkg-query -f '${db:Status-Status}' -W "dpkg-dev")" = installed ||
die "please install dpkg-dev"
dpkg-checkbuilddeps -d "autopkgtest, dosfstools, e2fsprogs, fdisk, mount, mtools, passwd, uidmap, libarchive13, systemd-boot-efi:$ARCHITECTURE $BINUTILS" /dev/null ||
die "please install the required packages listed above"
BOOTSTUB="/usr/lib/systemd/boot/efi/linux${EFIIMG#boot}.stub"
@ -270,26 +310,38 @@ WORKDIR=$(mktemp -d)
FAT_OFFSET_SECTORS=$((1024*2))
FAT_SIZE_SECTORS=$((1024*254))
# Make the image writeable to the first subgid. mmdebstrap will map this gid to
# the root group. unshare instead will map the current gid to 0 and the first
# subgid to 1. Therefore mmdebstrap will be able to write to the image.
rm -f "$IMAGE"
: >"$IMAGE"
unshare -U -r --map-groups=auto chown 0:1 "$IMAGE"
chmod 0660 "$IMAGE"
# The image is raw and not in qcow2 format because:
# - faster run-time as the "qemu-image convert" step is not needed
# - image can be used independent of qemu tooling
# - modifying the image just with "mount" instead of requiring qemu-nbd
# - sparse images make the file just as small as with qcow2
# - trim support is more difficult on qcow2
# - snapshots and overlays work just as well with raw images
# - users who prefer qcow2 get to choose to run it themselves with their own
# custom options like compression
set -- \
--mode=unshare \
--format=tar \
--variant=important \
--architecture="$ARCHITECTURE"
test "$RELEASE" = jessie &&
case $MIRROR in http://snapshot.debian.org/archive/*|https://snapshot.debian.org/archive/*)
set -- "$@" --aptopt='Acquire::Check-Valid-Until "false"';;
esac
EXT_FEATURES=
if test "$RELEASE" = jessie; then
set -- "$@" --keyring=/usr/share/keyrings/debian-archive-removed-keys.gpg
set -- "$@" --aptopt='Apt::Key::gpgvcommand "/usr/libexec/mmdebstrap/gpgvnoexpkeysig"'
set -- "$@" --hook-dir=/usr/share/mmdebstrap/hooks/jessie-or-older
EXT_FEATURES="^metadata_csum,^metadata_csum_seed,^orphan_file"
fi
set -- "$@" \
"--include=init,linux-image-$ARCHITECTURE,python3" \
'--customize-hook=echo autopkgtestvm >"$1/etc/hostname"' \
'--customize-hook=echo 127.0.0.1 localhost autopkgtestvm >"$1/etc/hosts"' \
"--include=init,$LINUXIMAGE,python3" \
'--customize-hook=echo host >"$1/etc/hostname"' \
'--customize-hook=echo 127.0.0.1 localhost host >"$1/etc/hosts"' \
'--customize-hook=passwd --root "$1" --delete root' \
'--customize-hook=useradd --root "$1" --home-dir /home/user --create-home user' \
'--customize-hook=passwd --root "$1" --delete user' \
@ -302,26 +354,35 @@ if test -n "$SCRIPT"; then
'--customize-hook=rm -f "$1/userscript"'
fi
EXT4_OFFSET_BYTES=$(( (FAT_OFFSET_SECTORS + FAT_SIZE_SECTORS) * 512))
EXT4_OPTIONS="offset=$EXT4_OFFSET_BYTES,assume_storage_prezeroed=1"
set -- "$@" \
"--customize-hook=download vmlinuz '$WORKDIR/kernel'" \
"--customize-hook=download initrd.img '$WORKDIR/initrd'" \
'--customize-hook=mount --bind "$1" "$1/mnt"' \
'--customize-hook=mount --bind "$1/mnt/mnt" "$1/mnt/dev"' \
'--customize-hook=/sbin/mkfs.ext4 -d "$1/mnt" -L autopkgtestvm -E '"'$EXT4_OPTIONS' '$IMAGE' '$SIZE'" \
'--customize-hook=umount --lazy "$1/mnt"' \
"$RELEASE" \
/dev/null
-
test -n "$MIRROR" && set -- "$@" "$MIRROR"
test -n "$KEYRING" && set -- "$@" "--keyring=$KEYRING"
echo "mmdebstrap $*"
mmdebstrap "$@" || die "mmdebstrap failed"
echo "+ mmdebstrap $*" >&2
# https://github.com/koalaman/shellcheck/issues/2555
# shellcheck disable=SC3040
set -o pipefail
mmdebstrap "$@" | {
set -- -t ext4 -L autopkgtestvm -d -
if test -n "$EXT_FEATURES"; then
set -- "$@" -O "$EXT_FEATURES"
fi
EXTOPTS="offset=$(( (FAT_OFFSET_SECTORS + FAT_SIZE_SECTORS) * 512))"
if test -n "${SOURCE_DATE_EPOCH:-}"; then
uuid="$(uuidgen --sha1 --namespace="$(uuidgen --sha1 --namespace='@dns' --name mister-muffin.de)" --name "$SOURCE_DATE_EPOCH")"
set -- "$@" -U "$uuid"
EXTOPTS="$EXTOPTS,hash_seed=$uuid"
fi
set -- "$@" -E "$EXTOPTS" "$IMAGE" "$SIZE"
echo "+ mke2fs $*" >&2
/sbin/mke2fs "$@"
}
unshare -U -r --map-groups=auto chown 0:0 "$IMAGE"
chmod "$(printf %o "$(( 0666 - 0$(umask) ))")" "$IMAGE"
echo "root=LABEL=autopkgtestvm rw console=ttyS0" > "$WORKDIR/cmdline"
@ -387,3 +448,13 @@ start=$((FAT_OFFSET_SECTORS + FAT_SIZE_SECTORS)), type=0FC63DAF-8483-4772-8E79-3
EOF
dd if="$WORKDIR/fat" of="$IMAGE" conv=notrunc,sparse bs=512 "seek=$FAT_OFFSET_SECTORS" status=none
if test "$(dpkg --print-architecture)" != "$ARCHITECTURE" && test "$(dpkg-query -f '${db:Status-Status}' -W "qemu-system-$QEMUARCH")" != installed; then
echo "I: you might need to install a package providing qemu-system-$QEMUARCH to use this image with autopkgtest-virt-qemu" >&2
fi
if test -n "$VMFPKG" && test "$(dpkg-query -f '${db:Status-Status}' -W "$VMFPKG")" != installed; then
echo "I: you might need to install $VMFPKG to use this image with autopkgtest-virt-qemu" >&2
fi
echo "I: SUCCESS! Your new image can be found here: $IMAGE" >&2
echo "I: Don't forget to pass --boot=efi when running autopkgtest-virt-qemu with this image" >&2

@ -4,15 +4,13 @@ set -eu
: "${DEFAULT_DIST:=unstable}"
: "${cachedir:=./shared/cache}"
: "${MMDEBSTRAP_TESTS_DEBUG:=no}"
tmpdir="$(mktemp -d)"
cleanup() {
rv=$?
rm -f "$tmpdir/log"
[ -e "$tmpdir" ] && rmdir "$tmpdir"
if [ -n "${TAIL_PID:-}" ]; then
kill "$TAIL_PID"
fi
if [ -e shared/output.txt ]; then
res="$(cat shared/exitstatus.txt)"
if [ "$res" != "0" ]; then
@ -30,22 +28,44 @@ if [ -e shared/output.txt ]; then
rm shared/output.txt
fi
touch shared/output.txt
tail -f shared/output.txt &
TAIL_PID=$!
# to connect to serial use:
# minicom -D 'unix#/tmp/ttyS0'
#
# or this (quit with ctrl+q):
# socat stdin,raw,echo=0,escape=0x11 unix-connect:/tmp/ttyS0
ret=0
timeout --foreground 40m debvm-run --image="$(realpath "$cachedir")/debian-$DEFAULT_DIST.ext4" -- \
-m 4G -snapshot \
setpriv --pdeathsig TERM tail -f shared/output.txt &
set -- timeout --foreground 40m \
debvm-run --image="$(realpath "$cachedir")/debian-$DEFAULT_DIST.ext4" \
--
cpuname=$(lscpu | awk '/Model name:/ {print $3}' | tr '\n' '+')
ncpu=$(lscpu | awk '/Core\(s\) per socket:/ {print $4}' | tr '\n' '+')
if [ "$cpuname" = "Cortex-A53+Cortex-A73+" ] && [ "$ncpu" = "2+4+" ]; then
# crude detection of the big.LITTLE heterogeneous setup of cores on the
# amlogic a311d bananapi
#
# https://lists.nongnu.org/archive/html/qemu-devel/2020-10/msg08494.html
# https://gitlab.com/qemu-project/qemu/-/issues/239
# https://segments.zhan.science/posts/kvm_on_pinehone_pro/#trouble-with-heterogeneous-architecture
set -- taskset --cpu-list 2,3,4,5 "$@" -smp 4
fi
set -- "$@" -nic none -m 4G -snapshot
if [ "$MMDEBSTRAP_TESTS_DEBUG" = "no" ]; then
# to connect to serial use:
# minicom -D 'unix#/tmp/ttyS0'
# or this (quit with ctrl+q):
# socat stdin,raw,echo=0,escape=0x11 unix-connect:/tmp/ttyS0
set -- "$@" \
-monitor unix:/tmp/monitor,server,nowait \
-serial unix:/tmp/ttyS0,server,nowait \
-serial unix:/tmp/ttyS1,server,nowait \
-virtfs local,id=mmdebstrap,path="$(pwd)/shared",security_model=none,mount_tag=mmdebstrap \
>"$tmpdir/log" 2>&1 || ret=$?
-serial unix:/tmp/ttyS1,server,nowait
fi
set -- "$@" -virtfs local,id=mmdebstrap,path="$(pwd)/shared",security_model=none,mount_tag=mmdebstrap
ret=0
if [ "$MMDEBSTRAP_TESTS_DEBUG" = "no" ]; then
"$@" >"$tmpdir/log" 2>&1 || ret=$?
else
"$@" 2>&1 | tee "$tmpdir/log" || ret=$?
fi
if [ "$ret" -ne 0 ]; then
cat "$tmpdir/log"
exit $ret

@ -5,4 +5,4 @@ trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
{{ CMD }} --mode={{ MODE }} --variant=essential \
--include '?or(?exact-name(dummy-does-not-exist),?exact-name(apt))' \
{{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
tar -tf /tmp/debian-chroot.tar | sort | grep -v ./var/lib/apt/extended_states | diff -u tar1.txt -
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -

@ -6,4 +6,4 @@ trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
--include '?narrow(?archive(^{{ DIST }}$),?essential)' \
--include apt \
{{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -
{ tar -tf /tmp/debian-chroot.tar; echo ./var/lib/apt/extended_states; } | sort | diff -u tar1.txt -

@ -6,6 +6,6 @@ trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
--include "$(tr '\n' ',' < pkglist.txt)" \
--aptopt='APT::Solver "aspcud"' \
{{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
tar -tf /tmp/debian-chroot.tar | sort \
{ tar -tf /tmp/debian-chroot.tar; echo ./var/lib/apt/extended_states; } | sort \
| grep -v '^./etc/apt/apt.conf.d/99mmdebstrap$' \
| diff -u tar1.txt -

@ -13,6 +13,7 @@ echo "SOURCE_DATE_EPOCH=$SOURCE_DATE_EPOCH"
{{ CMD }} --variant={{ VARIANT }} --mode={{ MODE }} \
--essential-hook='[ {{ DIST }} = oldstable ] && [ {{ VARIANT }} = - ] && echo _apt:*:100:65534::/nonexistent:/usr/sbin/nologin >> "$1"/etc/passwd || :' \
"$(if [ {{ DIST }} = oldstable ]; then echo --merged-usr; else echo --hook-dir=./hooks/merged-usr; fi)" \
"$(case {{ DIST }} in oldstable) echo --include=e2fsprogs,mount,tzdata,gcc-9-base;; stable) echo --include=e2fsprogs,mount,tzdata;; *) echo --include=base-files ;; esac )" \
{{ DIST }} /tmp/debian-{{ DIST }}-mm.tar {{ MIRROR }}
mkdir /tmp/debian-{{ DIST }}-mm
@ -69,6 +70,8 @@ rm /tmp/debian-{{ DIST }}-debootstrap/var/cache/debconf/templates.dat-old \
/tmp/debian-{{ DIST }}-mm/var/cache/debconf/templates.dat-old
rm /tmp/debian-{{ DIST }}-debootstrap/var/lib/dpkg/status-old \
/tmp/debian-{{ DIST }}-mm/var/lib/dpkg/status-old
rm -f /tmp/debian-{{ DIST }}-debootstrap/var/lib/dpkg/diversions-old \
/tmp/debian-{{ DIST }}-mm/var/lib/dpkg/diversions-old
# remove dpkg files
rm /tmp/debian-{{ DIST }}-debootstrap/var/lib/dpkg/available
rm /tmp/debian-{{ DIST }}-debootstrap/var/lib/dpkg/cmethopt
@ -154,33 +157,72 @@ if [ "{{ VARIANT }}" = "-" ] && [ "{{ DIST}}" = oldstable ]; then
fi
for log in faillog lastlog; do
if ! cmp /tmp/debian-{{ DIST }}-debootstrap/var/log/$log /tmp/debian-{{ DIST }}-mm/var/log/$log >&2;then
f1="/tmp/debian-{{ DIST }}-debootstrap/var/log/$log"
f2="/tmp/debian-{{ DIST }}-mm/var/log/$log"
# skip cmp if file is absent in both chroots
if [ ! -e "$f1" ] && [ ! -e "$f2" ]; then
continue
fi
if ! cmp "$f1" "$f2" >&2;then
# if the files differ, make sure they are all zeroes
cmp -n "$(stat -c %s "/tmp/debian-{{ DIST }}-debootstrap/var/log/$log")" "/tmp/debian-{{ DIST }}-debootstrap/var/log/$log" /dev/zero >&2
cmp -n "$(stat -c %s "/tmp/debian-{{ DIST }}-mm/var/log/$log")" "/tmp/debian-{{ DIST }}-mm/var/log/$log" /dev/zero >&2
cmp -n "$(stat -c %s "$f1")" "$f1" /dev/zero >&2
cmp -n "$(stat -c %s "$f2")" "$f2" /dev/zero >&2
# then delete them
rm /tmp/debian-{{ DIST }}-debootstrap/var/log/$log /tmp/debian-{{ DIST }}-mm/var/log/$log
rm "$f1" "$f2"
fi
done
# the order in which systemd and cron get installed differ and thus the order
# of lines in /etc/group and /etc/gshadow differs
if [ "{{ VARIANT }}" = "-" ]; then
# the order in which systemd and cron get installed differ and thus the order
# of lines in /etc/group and /etc/gshadow differs
for f in group group- gshadow gshadow-; do
for d in mm debootstrap; do
sort /tmp/debian-{{ DIST }}-$d/etc/$f > /tmp/debian-{{ DIST }}-$d/etc/$f.bak
mv /tmp/debian-{{ DIST }}-$d/etc/$f.bak /tmp/debian-{{ DIST }}-$d/etc/$f
done
done
# the order in which systemd and passwd get installed differ and thus
# the order of lines in /etc/shadow and /etc/shadow- differs
for f in shadow shadow-; do
for d in mm debootstrap; do
sort /tmp/debian-{{ DIST }}-$d/etc/$f > /tmp/debian-{{ DIST }}-$d/etc/$f.bak
mv /tmp/debian-{{ DIST }}-$d/etc/$f.bak /tmp/debian-{{ DIST }}-$d/etc/$f
done
done
# and since the order was different, ignore the *- files
for f in shadow- passwd-; do
for d in mm debootstrap; do
rm /tmp/debian-{{ DIST }}-$d/etc/$f
done
done
fi
# since debootstrap 1.0.133 there is no tzdata in the buildd variant and thus
# debootstrap creates its own /etc/localtime
if [ "{{ VARIANT }}" = "buildd" ]; then
if [ "{{ VARIANT }}" = "buildd" ] && [ "{{ DIST }}" != "stable" ] && [ "{{ DIST }}" != "oldstable" ]; then
[ "$(readlink /tmp/debian-{{ DIST }}-debootstrap/etc/localtime)" = /usr/share/zoneinfo/UTC ]
rm /tmp/debian-{{ DIST }}-debootstrap/etc/localtime
fi
# starting with systemd 255 upstream dropped splitusr support and depending on
# the installation order, symlink targets are prefixed with /usr or not
# See #1060000 and #1054137
case {{ DIST }} in testing|unstable)
for f in multi-user.target.wants/e2scrub_reap.service timers.target.wants/apt-daily-upgrade.timer timers.target.wants/apt-daily.timer timers.target.wants/e2scrub_all.timer; do
for d in mm debootstrap; do
[ -L "/tmp/debian-{{ DIST }}-$d/etc/systemd/system/$f" ] || continue
oldlink="$(readlink "/tmp/debian-{{ DIST }}-$d/etc/systemd/system/$f")"
case $oldlink in
/usr/*) : ;;
/*) oldlink="/usr$oldlink" ;;
*) echo unexpected >&2; exit 1 ;;
esac
ln -sf "$oldlink" "/tmp/debian-{{ DIST }}-$d/etc/systemd/system/$f"
done
done
;;
esac
# check if the file content differs
diff --unified --no-dereference --recursive /tmp/debian-{{ DIST }}-debootstrap /tmp/debian-{{ DIST }}-mm >&2
@ -189,8 +231,9 @@ diff --unified --no-dereference --recursive /tmp/debian-{{ DIST }}-debootstrap /
find /tmp/debian-{{ DIST }}-debootstrap /tmp/debian-{{ DIST }}-mm -type d -print0 | xargs -0 touch --date="@{{ SOURCE_DATE_EPOCH }}"
# debootstrap never ran apt -- fixing permissions
for d in ./var/lib/apt/lists/partial ./var/cache/apt/archives/partial; do
chroot /tmp/debian-{{ DIST }}-debootstrap chmod 0700 $d
chroot /tmp/debian-{{ DIST }}-debootstrap chown "$(id -u _apt):root" $d
unmergedPATH="$PATH$(if [ "{{ DIST }}" = oldstable ]; then echo :/bin:/sbin; fi)"
PATH="$unmergedPATH" chroot /tmp/debian-{{ DIST }}-debootstrap chmod 0700 $d
PATH="$unmergedPATH" chroot /tmp/debian-{{ DIST }}-debootstrap chown "$(id -u _apt):root" $d
done
tar -C /tmp/debian-{{ DIST }}-debootstrap --numeric-owner --sort=name --clamp-mtime --mtime="$(date --utc --date=@{{ SOURCE_DATE_EPOCH }} --iso-8601=seconds)" -cf /tmp/root1.tar .
tar -C /tmp/debian-{{ DIST }}-mm --numeric-owner --sort=name --clamp-mtime --mtime="$(date --utc --date=@{{ SOURCE_DATE_EPOCH }} --iso-8601=seconds)" -cf /tmp/root2.tar .

@ -8,7 +8,7 @@ trap "rm -f /tmp/chrootless.tar /tmp/root.tar" EXIT INT TERM
for INCLUDE in '' 'apt' 'apt,build-essential' 'systemd-sysv'; do
for MODE in root chrootless; do
{{ CMD }} --mode=$MODE --variant={{ VARIANT }} --hook-dir=./hooks/merged-usr \
${INCLUDE:+--include="$INCLUDE"} \
${INCLUDE:+--include="$INCLUDE"} --skip=check/chrootless \
{{ DIST }} "/tmp/$MODE.tar" {{ MIRROR }}
done
cmp /tmp/root.tar /tmp/chrootless.tar || diffoscope /tmp/root.tar /tmp/chrootless.tar

@ -18,18 +18,26 @@ if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && [ "{{ MODE }}" != "auto
prefix="runuser -u ${SUDO_USER:-user} --"
fi
MMTARFILTER=
[ -x /usr/bin/mmtarfilter ] && MMTARFILTER=/usr/bin/mmtarfilter
[ -x ./tarfilter ] && MMTARFILTER=./tarfilter
# we need --hook-dir=./hooks/merged-usr because usrmerge does not understand
# DPKG_ROOT
# permissions drwxr-sr-x and extended attributes of ./var/log/journal/ cannot
# be preserved under fakeroot
# this applies to 'z' lines in files in /usr/lib/tmpfiles.d/
for INCLUDE in '' 'apt' 'apt,build-essential' 'systemd-sysv'; do
{{ CMD }} --variant={{ VARIANT }} --hook-dir=./hooks/merged-usr \
--customize-hook='if [ -d "$1"/var/log/journal ]; then rmdir "$1"/var/log/journal; mkdir --mode=2755 "$1"/var/log/journal; chroot "$1" chown root:systemd-journal /var/log/journal; fi' \
${INCLUDE:+--include="$INCLUDE"} \
{{ DIST }} /tmp/root.tar {{ MIRROR }}
{{ DIST }} - {{ MIRROR }} \
| "$MMTARFILTER" --path-exclude="/var/log/journal" --path-exclude="/etc/credstore*" \
>/tmp/root.tar
$prefix fakeroot {{ CMD }} --mode={{ MODE }} --variant={{ VARIANT }} --hook-dir=./hooks/merged-usr \
${INCLUDE:+--include="$INCLUDE"} \
{{ DIST }} /tmp/chrootless.tar {{ MIRROR }}
{{ DIST }} - {{ MIRROR }} \
| "$MMTARFILTER" --path-exclude="/var/log/journal" --path-exclude="/etc/credstore*" \
> /tmp/chrootless.tar
cmp /tmp/root.tar /tmp/chrootless.tar || diffoscope /tmp/root.tar /tmp/chrootless.tar
rm /tmp/chrootless.tar /tmp/root.tar
done

@ -45,7 +45,7 @@ for INCLUDE in '' 'apt' 'systemd-sysv'; do
arch-test "$arch" && exit 1
{{ CMD }} --mode=chrootless --architecture="$arch" --variant={{ VARIANT }} \
--hook-dir=./hooks/merged-usr ${INCLUDE:+--include="$INCLUDE"} \
{{ DIST }} "/tmp/chrootless.tar" {{ MIRROR }}
--skip=check/chrootless {{ DIST }} "/tmp/chrootless.tar" {{ MIRROR }}
# when creating a foreign architecture chroot, the tarballs are not
# bit-by-bit identical but contain a few remaining differences:
#

@ -1,65 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
prefix=
if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && [ "{{ MODE }}" != "auto" ]; then
if ! id "${SUDO_USER:-user}" >/dev/null 2>&1; then
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir "/home/${SUDO_USER:-user}" --create-home "${SUDO_USER:-user}"
fi
prefix="runuser -u ${SUDO_USER:-user} --"
fi
[ "{{ MODE }}" = "fakechroot" ] && prefix="$prefix fakechroot fakeroot"
$prefix {{ CMD }} --mode={{ MODE }} --variant=apt --architectures=arm64 {{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
# we ignore differences between architectures by ignoring some files
# and renaming others
{ tar -tf /tmp/debian-chroot.tar \
| grep -v '^\./usr/lib/ld-linux-aarch64\.so\.1$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/ld-linux-aarch64\.so\.1$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/asm-generic/int-ll64\.ph$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/asm-generic/types\.ph$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/asm-generic/unistd\.ph$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/asm/sigcontext\.ph$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/asm/sve_context\.ph$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/asm/types\.ph$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/bits/procfs-extra\.ph$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/bits/procfs-id\.ph$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/bits/procfs-prregset\.ph$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/bits/procfs\.ph$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/gnu/stubs-lp64\.ph$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/linux/types\.ph$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/sys/procfs\.ph$' \
| grep -v '^\./usr/lib/aarch64-linux-gnu/perl/5\.[0-9]\+\.0/sys/user\.ph$' \
| grep -v '^\./usr/share/doc/[^/]\+/changelog\(\.Debian\)\?\.arm64\.gz$' \
| sed 's/aarch64-linux-gnu/x86_64-linux-gnu/' \
| sed 's/arm64/amd64/';
} | sort > tar2.txt
{ < tar1.txt \
grep -v '^\./usr/bin/i386$' \
| grep -v '^\./usr/bin/x86_64$' \
| grep -v '^\./lib32$' \
| grep -v '^\./lib64$' \
| grep -v '^\./libx32$' \
| grep -v '^\./usr/lib32/$' \
| grep -v '^\./usr/libx32/$' \
| grep -v '^\./usr/lib64/$' \
| grep -v '^\./usr/lib64/ld-linux-x86-64\.so\.2$' \
| grep -v '^\./usr/lib/x86_64-linux-gnu/ld-linux-x86-64\.so\.2$' \
| grep -v '^\./usr/lib/x86_64-linux-gnu/libmvec\.so\.1$' \
| grep -v '^\./usr/lib/x86_64-linux-gnu/perl/5\.[0-9]\+\.0/asm/posix_types_32\.ph$' \
| grep -v '^\./usr/lib/x86_64-linux-gnu/perl/5\.[0-9]\+\.0/asm/posix_types_64\.ph$' \
| grep -v '^\./usr/lib/x86_64-linux-gnu/perl/5\.[0-9]\+\.0/asm/posix_types_x32\.ph$' \
| grep -v '^\./usr/lib/x86_64-linux-gnu/perl/5\.[0-9]\+\.0/asm/unistd_32\.ph$' \
| grep -v '^\./usr/lib/x86_64-linux-gnu/perl/5\.[0-9]\+\.0/asm/unistd_64\.ph$' \
| grep -v '^\./usr/lib/x86_64-linux-gnu/perl/5\.[0-9]\+\.0/asm/unistd_x32\.ph$' \
| grep -v '^\./usr/lib/x86_64-linux-gnu/perl/5\.[0-9]\+\.0/gnu/stubs-64\.ph$' \
| grep -v '^\./usr/share/doc/[^/]\+/changelog\(\.Debian\)\?\.amd64\.gz$' \
| grep -v '^\./usr/share/man/man8/i386\.8\.gz$' \
| grep -v '^\./usr/share/man/man8/x86_64\.8\.gz$';
} | sort | diff -u - tar2.txt >&2
rm /tmp/debian-chroot.tar

@ -0,0 +1,77 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
prefix=
if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && [ "{{ MODE }}" != "auto" ]; then
if ! id "${SUDO_USER:-user}" >/dev/null 2>&1; then
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir "/home/${SUDO_USER:-user}" --create-home "${SUDO_USER:-user}"
fi
prefix="runuser -u ${SUDO_USER:-user} --"
fi
case "$(dpkg --print-architecture)" in
arm64)
native_arch=arm64
native_gnu=aarch64-linux-gnu
foreign_arch=amd64
foreign_gnu=x86_64-linux-gnu
;;
amd64)
native_arch=amd64
native_gnu=x86_64-linux-gnu
foreign_arch=arm64
foreign_gnu=aarch64-linux-gnu
;;
*)
echo "unsupported native architecture" >&2
exit 1
;;
esac
[ "{{ MODE }}" = "fakechroot" ] && prefix="$prefix fakechroot fakeroot"
$prefix {{ CMD }} --mode={{ MODE }} --variant=apt --architectures="$foreign_arch" \
{{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
# we ignore differences between architectures by ignoring some files
# and renaming others
{ tar -tf /tmp/debian-chroot.tar \
| grep -v '^\./usr/bin/i386$' \
| grep -v '^\./usr/bin/x86_64$' \
| grep -v '^\./lib64$' \
| grep -v '^\./usr/lib64/$' \
| grep -v '^\./usr/lib64/ld-linux-x86-64\.so\.2$' \
| grep -v '^\./usr/lib/ld-linux-aarch64\.so\.1$' \
| grep -v "^\\./usr/lib/$foreign_gnu/ld-linux-aarch64\\.so\\.1$" \
| grep -v "^\\./usr/lib/$foreign_gnu/ld-linux-x86-64\\.so\\.2$" \
| grep -v "^\\./usr/lib/$foreign_gnu/perl/5\\.[0-9][.0-9]\\+/.*\\.ph$" \
| grep -v "^\\./usr/lib/$foreign_gnu/libmvec\\.so\\.1$" \
| grep -v "^\\./usr/share/doc/[^/]\\+/changelog\\(\\.Debian\\)\\?\\.$foreign_arch\\.gz$" \
| grep -v '^\./usr/share/man/man8/i386\.8\.gz$' \
| grep -v '^\./usr/share/man/man8/x86_64\.8\.gz$' \
| sed "s/$foreign_gnu/$native_gnu/" \
| sed "s/$foreign_arch/$native_arch/";
} | sort > /tmp/tar2.txt
{ < tar1.txt \
grep -v '^\./usr/bin/i386$' \
| grep -v '^\./usr/bin/x86_64$' \
| grep -v '^\./lib32$' \
| grep -v '^\./lib64$' \
| grep -v '^\./libx32$' \
| grep -v '^\./usr/lib32/$' \
| grep -v '^\./usr/libx32/$' \
| grep -v '^\./usr/lib64/$' \
| grep -v '^\./usr/lib64/ld-linux-x86-64\.so\.2$' \
| grep -v '^\./usr/lib/ld-linux-aarch64\.so\.1$' \
| grep -v "^\\./usr/lib/$native_gnu/ld-linux-x86-64\\.so\\.2$" \
| grep -v "^\\./usr/lib/$native_gnu/ld-linux-aarch64\\.so\\.1$" \
| grep -v "^\\./usr/lib/$native_gnu/libmvec\\.so\\.1$" \
| grep -v "^\\./usr/lib/$native_gnu/perl/5\\.[0-9][.0-9]\\+/.*\\.ph$" \
| grep -v "^\\./usr/share/doc/[^/]\\+/changelog\\(\\.Debian\\)\\?\\.$native_arch\\.gz$" \
| grep -v '^\./usr/share/man/man8/i386\.8\.gz$' \
| grep -v '^\./usr/share/man/man8/x86_64\.8\.gz$';
} | sort | diff -u - /tmp/tar2.txt >&2
rm /tmp/debian-chroot.tar /tmp/tar2.txt

@ -5,6 +5,12 @@ export SOURCE_DATE_EPOCH={{ SOURCE_DATE_EPOCH }}
tmpdir="$(mktemp -d)"
chmod 755 "$tmpdir"
debootstrap "$([ "{{ DIST }}" = oldstable ] && echo --no-merged-usr || echo --merged-usr)" --variant={{ VARIANT }} {{ DIST }} "$tmpdir" {{ MIRROR }}
ret=0
debootstrap "$([ "{{ DIST }}" = oldstable ] && echo --no-merged-usr || echo --merged-usr)" --variant={{ VARIANT }} {{ DIST }} "$tmpdir" {{ MIRROR }} || ret=$?
if [ "$ret" -ne 0 ]; then
echo "E: debootstrap failed, dumping $tmpdir/debootstrap/debootstrap.log"
cat "$tmpdir/debootstrap/debootstrap.log"
exit 1
fi
tar --sort=name --mtime=@$SOURCE_DATE_EPOCH --clamp-mtime --numeric-owner --one-file-system --xattrs -C "$tmpdir" -c . > "./cache/debian-{{ DIST }}-{{ VARIANT }}.tar"
rm -r "$tmpdir"

@ -7,9 +7,7 @@ trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
# we use variant standard in verbose mode to see the maximum number of packages
# that was chosen in case of USE_HOST_APT_CONFIG=yes
# we use variant important on arches where variant standard is not bit-by-bit
# reproducible due to #1031276
case {{ VARIANT }} in standard|-) : ;; *) exit 1;; esac
case {{ VARIANT }} in standard) : ;; *) exit 1;; esac
{{ CMD }} --variant={{ VARIANT }} --debug {{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}

@ -37,7 +37,6 @@ elfheader="$elfheader\\001\\000"
tar -C /tmp/debian-chroot --one-file-system -c . \
| tar -t \
| sort \
| grep -v '^\./var/lib/dpkg/diversions\(-old\)\?$' \
| diff -u tar1.txt -
rm /tmp/checkeatmydata.sh
rm -r /tmp/debian-chroot

@ -3,6 +3,6 @@ set -eu
export LC_ALL=C.UTF-8
trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
printf '' | {{ CMD }} --mode={{ MODE }} --variant=apt \
--setup-hook='echo "deb {{ MIRROR }} {{ DIST }} main" > "$1"/etc/apt/sources.list' \
--setup-hook='test -e "$1"/etc/apt/sources.list || echo "deb {{ MIRROR }} {{ DIST }} main" > "$1"/etc/apt/sources.list' \
{{ DIST }} /tmp/debian-chroot.tar -
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -

@ -17,5 +17,4 @@ tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort \
| grep -v '^./usr/share/lintian/overrides/tzdata' \
| grep -v '^./usr/share/zoneinfo' \
| grep -v '^./var/lib/dpkg/info/tzdata.' \
| grep -v '^./var/lib/apt/extended_states$' \
| diff -u tar1.txt -

@ -6,7 +6,6 @@ trap "rm -rf /tmp/debian-chroot" EXIT INT TERM
rm /tmp/debian-chroot/usr/share/doc-base/doc-debian.debian-*
rm -r /tmp/debian-chroot/usr/share/doc/debian
rm -r /tmp/debian-chroot/usr/share/doc/doc-debian
rm /tmp/debian-chroot/var/lib/apt/extended_states
rm /tmp/debian-chroot/var/lib/dpkg/info/doc-debian.list
rm /tmp/debian-chroot/var/lib/dpkg/info/doc-debian.md5sums
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort | diff -u tar1.txt -

@ -6,20 +6,38 @@
# - installs only few files
# - doesn't change its name regularly (like gcc-*-base)
case "$(dpkg --print-architecture)" in
arm64)
native_arch=arm64
foreign_arch=amd64
;;
amd64)
native_arch=amd64
foreign_arch=arm64
;;
*)
echo "unsupported native architecture" >&2
exit 1
;;
esac
set -eu
export LC_ALL=C.UTF-8
{{ CMD }} --mode=root --variant=apt --architectures=amd64,arm64 --include=libmagic-mgc:arm64 {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
{ echo "amd64"; echo "arm64"; } | cmp /tmp/debian-chroot/var/lib/dpkg/arch -
rm /tmp/debian-chroot/var/lib/apt/extended_states
rm /tmp/debian-chroot/var/lib/dpkg/info/libmagic-mgc.list
rm /tmp/debian-chroot/var/lib/dpkg/info/libmagic-mgc.md5sums
{{ CMD }} --mode=root --variant=apt \
--architectures="$native_arch,$foreign_arch" \
--include="libmagic-mgc:$foreign_arch" \
{{ DIST }} /tmp/debian-chroot {{ MIRROR }}
{ echo "$native_arch"; echo "$foreign_arch"; } | cmp /tmp/debian-chroot/var/lib/dpkg/arch -
rm /tmp/debian-chroot/usr/lib/file/magic.mgc
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/README.Debian
rm -f /tmp/debian-chroot/usr/share/doc/libmagic-mgc/"changelog.Debian.$foreign_arch.gz"
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/changelog.Debian.gz
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/changelog.gz
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/copyright
rm /tmp/debian-chroot/usr/share/file/magic.mgc
rm /tmp/debian-chroot/usr/share/misc/magic.mgc
rm /tmp/debian-chroot/var/lib/dpkg/info/libmagic-mgc.list
rm /tmp/debian-chroot/var/lib/dpkg/info/libmagic-mgc.md5sums
rmdir /tmp/debian-chroot/usr/share/doc/libmagic-mgc/
rmdir /tmp/debian-chroot/usr/share/file/magic/
rmdir /tmp/debian-chroot/usr/share/file/

@ -0,0 +1,47 @@
#!/bin/sh
#
# to test foreign architecture package installation we choose a package which
# - is not part of the native installation set
# - does not have any dependencies
# - installs only few files
# - doesn't change its name regularly (like gcc-*-base)
case "$(dpkg --print-architecture)" in
arm64)
native_arch=arm64
foreign_arch=amd64
;;
amd64)
native_arch=amd64
foreign_arch=arm64
;;
*)
echo "unsupported native architecture" >&2
exit 1
;;
esac
set -eu
export LC_ALL=C.UTF-8
{{ CMD }} --mode=root --variant=apt \
--architectures="$native_arch" \
--architectures="$foreign_arch" \
--include="libmagic-mgc:$foreign_arch" \
{{ DIST }} /tmp/debian-chroot {{ MIRROR }}
{ echo "$native_arch"; echo "$foreign_arch"; } | cmp /tmp/debian-chroot/var/lib/dpkg/arch -
rm /tmp/debian-chroot/usr/lib/file/magic.mgc
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/README.Debian
rm -f /tmp/debian-chroot/usr/share/doc/libmagic-mgc/"changelog.Debian.$foreign_arch.gz"
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/changelog.Debian.gz
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/changelog.gz
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/copyright
rm /tmp/debian-chroot/usr/share/file/magic.mgc
rm /tmp/debian-chroot/usr/share/misc/magic.mgc
rm /tmp/debian-chroot/var/lib/dpkg/info/libmagic-mgc.list
rm /tmp/debian-chroot/var/lib/dpkg/info/libmagic-mgc.md5sums
rmdir /tmp/debian-chroot/usr/share/doc/libmagic-mgc/
rmdir /tmp/debian-chroot/usr/share/file/magic/
rmdir /tmp/debian-chroot/usr/share/file/
rmdir /tmp/debian-chroot/usr/lib/file/
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort | diff -u tar1.txt -
rm -r /tmp/debian-chroot

@ -1,21 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
{{ CMD }} --mode=root --variant=apt --architectures=amd64 --architectures=arm64 --include=libmagic-mgc:arm64 {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
{ echo "amd64"; echo "arm64"; } | cmp /tmp/debian-chroot/var/lib/dpkg/arch -
rm /tmp/debian-chroot/var/lib/apt/extended_states
rm /tmp/debian-chroot/var/lib/dpkg/info/libmagic-mgc.list
rm /tmp/debian-chroot/var/lib/dpkg/info/libmagic-mgc.md5sums
rm /tmp/debian-chroot/usr/lib/file/magic.mgc
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/README.Debian
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/changelog.Debian.gz
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/changelog.gz
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/copyright
rm /tmp/debian-chroot/usr/share/file/magic.mgc
rm /tmp/debian-chroot/usr/share/misc/magic.mgc
rmdir /tmp/debian-chroot/usr/share/doc/libmagic-mgc/
rmdir /tmp/debian-chroot/usr/share/file/magic/
rmdir /tmp/debian-chroot/usr/share/file/
rmdir /tmp/debian-chroot/usr/lib/file/
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort | diff -u tar1.txt -
rm -r /tmp/debian-chroot

@ -22,12 +22,17 @@ chroot /tmp/debian-chroot dpkg-query -f '${binary:Package}\n' -W \
rm /tmp/expected
for cmd in echo cat sed grep; do
test -L /tmp/debian-chroot/bin/$cmd
test "$(readlink /tmp/debian-chroot/bin/$cmd)" = "/bin/busybox"
test "$(readlink /tmp/debian-chroot/bin/$cmd)" = "/usr/bin/busybox"
done
for cmd in sort tee; do
test -L /tmp/debian-chroot/usr/bin/$cmd
test "$(readlink /tmp/debian-chroot/usr/bin/$cmd)" = "/bin/busybox"
test "$(readlink /tmp/debian-chroot/usr/bin/$cmd)" = "/usr/bin/busybox"
done
# if /bin or /sbin are not symlinks, add /bin and /sbin to PATH
if [ ! -L /tmp/debian-chroot/bin ] || [ ! -L /tmp/debian-chroot/sbin ]; then
export PATH="$PATH:/sbin:/bin"
fi
chroot /tmp/debian-chroot echo foobar \
| chroot /tmp/debian-chroot cat \
| chroot /tmp/debian-chroot sort \

@ -17,13 +17,27 @@ if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && [ "{{ MODE }}" != "auto
prefix="runuser -u ${SUDO_USER:-user} --"
fi
$prefix {{ CMD }} --mode={{ MODE }} --variant={{ VARIANT }} --architectures=arm64 --include=libmagic-mgc {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
case "$(dpkg --print-architecture)" in
arm64)
foreign_arch=amd64
;;
amd64)
foreign_arch=arm64
;;
*)
echo "unsupported native architecture" >&2
exit 1
;;
esac
$prefix {{ CMD }} --mode={{ MODE }} --variant={{ VARIANT }} --architectures="$foreign_arch" --include=libmagic-mgc {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
# delete contents of libmagic-mgc
rm /tmp/debian-chroot/usr/lib/file/magic.mgc
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/README.Debian
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/changelog.Debian.gz
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/changelog.gz
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/copyright
rm -f /tmp/debian-chroot/usr/share/doc/libmagic-mgc/"changelog.Debian.$foreign_arch.gz"
rm /tmp/debian-chroot/usr/share/file/magic.mgc
rm /tmp/debian-chroot/usr/share/misc/magic.mgc
# delete real files

@ -30,6 +30,7 @@ filter() {
--path-exclude=/var/cache/debconf/templates.dat-old \
--path-exclude=/var/lib/dpkg/available \
--path-exclude=/var/lib/dpkg/diversions \
--path-exclude=/var/lib/dpkg/diversions-old \
--path-exclude=/var/lib/dpkg/cmethopt \
--path-exclude=/var/lib/dpkg/status-old \
--path-exclude=/var/lib/shells.state

@ -16,6 +16,7 @@ I: running apt-get update...
I: downloading packages with apt...
I: extracting archives...
I: installing essential packages...
I: installing remaining packages inside the chroot...
I: cleaning package lists and apt cache...
LOG
tail --lines=1 /tmp/log | grep '^I: success in .* seconds$'

@ -17,4 +17,8 @@ if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && [ "{{ MODE }}" != "auto
prefix="runuser -u ${SUDO_USER:-user} --"
fi
$prefix {{ CMD }} --mode={{ MODE }} --variant={{ VARIANT }} --include=dpkg,dash,diffutils,coreutils,libc-bin,sed {{ DIST }} /dev/null {{ MIRROR }}
# do not install base-files, so that /proc, /sys and /dev are missing
$prefix {{ CMD }} --mode={{ MODE }} --variant={{ VARIANT }} \
--setup-hook='for dir in bin lib lib32 lib64 libo32 libx32 sbin; do ln -s "usr/$dir" "$1/$dir"; done' \
--include=dpkg,dash,diffutils,coreutils,libc-bin,sed \
{{ DIST }} /dev/null {{ MIRROR }}

@ -5,15 +5,26 @@ export SOURCE_DATE_EPOCH={{ SOURCE_DATE_EPOCH }}
[ "$(id -u)" -eq 0 ]
[ {{ MODE }} = "root" ]
case {{ FORMAT }} in tar|squashfs|ext2) : ;; *) exit 1;; esac
case {{ FORMAT }} in tar|squashfs|ext2|ext4) : ;; *) exit 1;; esac
{{ CMD }} --mode={{ MODE }} --variant={{ VARIANT }} {{ DIST }} ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.{{ FORMAT }} {{ MIRROR }}
{{ CMD }} --mode={{ MODE }} --variant={{ VARIANT }} {{ DIST }} /tmp/mmdebstrap-{{ DIST }}-{{ VARIANT }}.{{ FORMAT }} {{ MIRROR }}
# creating an ext4 image on a 9p filesystem produces different results compared
# to creating it on a tmpfs or ext4 fs because 9p does not support discards and
# even when running with -E nodiscard, the number of written bytes will differ
# https://lore.kernel.org/linux-ext4/171484520952.2626447.2160419274451668597@localhost/T/#t
mv /tmp/mmdebstrap-{{ DIST }}-{{ VARIANT }}.{{ FORMAT }} ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.{{ FORMAT }}
if [ "{{ FORMAT }}" = tar ]; then
printf 'ustar ' | cmp --bytes=6 --ignore-initial=257:0 ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.tar -
elif [ "{{ FORMAT }}" = squashfs ]; then
printf 'hsqs' | cmp --bytes=4 ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.squashfs -
elif [ "{{ FORMAT }}" = ext2 ]; then
printf '\123\357' | cmp --bytes=2 --ignore-initial=1080:0 ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.ext2 -
printf '\000\000\000\000\000\000\000\000\000\000\000\000' | cmp --bytes=12 --ignore-initial=1116:0 ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.ext2 -
elif [ "{{ FORMAT }}" = ext4 ]; then
printf '\123\357' | cmp --bytes=2 --ignore-initial=1080:0 ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.ext4 -
printf '\074\020\000\000\302\042\000\000\153\004\000\000' | cmp --bytes=12 --ignore-initial=1116:0 ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.ext4 -
[ "$(/sbin/blkid --match-tag UUID --output value ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.ext4)" = "$(uuidgen --sha1 --namespace="$(uuidgen --sha1 --namespace='@dns' --name mister-muffin.de)" --name $SOURCE_DATE_EPOCH)" ]
else
echo "unknown format: {{ FORMAT }}" >&2
exit 1

@ -11,7 +11,6 @@ rm /tmp/debian-chroot/etc/localtime
rm /tmp/debian-chroot/etc/timezone
rm -r /tmp/debian-chroot/usr/share/doc/tzdata
rm -r /tmp/debian-chroot/usr/share/zoneinfo
rm /tmp/debian-chroot/var/lib/apt/extended_states
for p in doc-debian tzdata; do
for f in list md5sums config postinst postrm templates preinst prerm; do
[ -e "/tmp/debian-chroot/var/lib/dpkg/info/$p.$f" ] || continue

@ -6,4 +6,4 @@ trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
--essential-hook='APT_CONFIG=$MMDEBSTRAP_APT_CONFIG apt-get update' \
--essential-hook='APT_CONFIG=$MMDEBSTRAP_APT_CONFIG apt-get --yes install apt' \
{{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
tar -tf /tmp/debian-chroot.tar | sort | grep -v ./var/lib/apt/extended_states | diff -u tar1.txt -
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -

@ -17,19 +17,19 @@ if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && [ "{{ MODE }}" != "auto
fi
$prefix {{ CMD }} --mode={{ MODE }} --variant=apt \
--include=mount \
--include=mount,perl \
{{ DIST }} /tmp/chroot1.tar {{ MIRROR }}
if [ {{ MODE }} = "unshare" ]; then
# calling pivot_root in root mode does not work for mysterious reasons:
# pivot_root: failed to change root from `.' to `mnt': Invalid argument
$prefix {{ CMD }} --mode={{ MODE }} --variant=apt --include=mount \
$prefix {{ CMD }} --mode={{ MODE }} --variant=apt --include=mount,perl \
--customize-hook='mkdir -p "$1/mnt" "$1/oldroot"' \
--customize-hook='[ ! -e /usr/bin/mmdebstrap ] || cp -aT /usr/bin/mmdebstrap "$1/usr/bin/mmdebstrap"' \
--customize-hook='[ ! -e ./mmdebstrap ] || cp -aT ./mmdebstrap "$1/mnt/mmdebstrap"' \
--customize-hook='mount -o rbind "$1" /mnt && cd /mnt && /sbin/pivot_root . oldroot' \
--customize-hook='unshare -U echo nested unprivileged unshare' \
--customize-hook='env --chdir=/mnt {{ CMD }} --mode=unshare --variant=apt --include=mount {{ DIST }} /tmp/chroot3.tar {{ MIRROR }}' \
--customize-hook='env --chdir=/mnt {{ CMD }} --mode=unshare --variant=apt --include=mount,perl {{ DIST }} /tmp/chroot3.tar {{ MIRROR }}' \
--customize-hook='copy-out /tmp/chroot3.tar /tmp' \
--customize-hook='rm -f "/usr/bin/mmdebstrap" "/mnt/mmdebstrap"' \
--customize-hook='umount -l oldroot sys' \
@ -41,11 +41,11 @@ if [ {{ MODE }} = "unshare" ]; then
rm /tmp/chroot2.tar /tmp/chroot3.tar
fi
$prefix {{ CMD }} --mode={{ MODE }} --variant=apt --include=mount \
$prefix {{ CMD }} --mode={{ MODE }} --variant=apt --include=mount,perl \
--customize-hook='mkdir -p "$1/mnt"' \
--customize-hook='[ ! -e /usr/bin/mmdebstrap ] || cp -aT /usr/bin/mmdebstrap "$1/usr/bin/mmdebstrap"' \
--customize-hook='[ ! -e ./mmdebstrap ] || cp -aT ./mmdebstrap "$1/mnt/mmdebstrap"' \
--chrooted-customize-hook='env --chdir=/mnt {{ CMD }} --mode=unshare --variant=apt --include=mount {{ DIST }} /tmp/chroot3.tar {{ MIRROR }}' \
--chrooted-customize-hook='env --chdir=/mnt {{ CMD }} --mode=unshare --variant=apt --include=mount,perl {{ DIST }} /tmp/chroot3.tar {{ MIRROR }}' \
--customize-hook='copy-out /tmp/chroot3.tar /tmp' \
--customize-hook='rm -f "$1/usr/bin/mmdebstrap" "$1/mnt/mmdebstrap"' \
{{ DIST }} /tmp/chroot2.tar {{ MIRROR }}

@ -2,5 +2,7 @@
set -eu
export LC_ALL=C.UTF-8
trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
{{ CMD }} --mode={{ MODE }} --variant=apt --customize-hook='rm "$1/usr/sbin/policy-rc.d"; rm "$1/sbin/start-stop-daemon"' {{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
{{ CMD }} --mode={{ MODE }} --variant=apt \
--customize-hook='rm "$1/usr/sbin/policy-rc.d"; rm "$1/usr/sbin/start-stop-daemon"' \
{{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -

@ -7,9 +7,7 @@ trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
# we use variant standard in verbose mode to see the maximum number of packages
# that was chosen in case of USE_HOST_APT_CONFIG=yes
# we use variant important on arches where variant standard is not bit-by-bit
# reproducible due to #1031276
case {{ VARIANT }} in standard|-) : ;; *) exit 1;; esac
case {{ VARIANT }} in standard) : ;; *) exit 1;; esac
{{ CMD }} --variant={{ VARIANT }} --verbose {{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}

Loading…
Cancel
Save