No description
Find a file
Johannes Schauer Marin Rodrigues d0568a2b9e
Wait for (reap) potential zombies and otherwise long-running background processes
Otherwise they might hog resources like /dev/null which can then not be
unmounted resulting in their mountpoints (the regular files) not being
removable and then the removal of device nodes in run_cleanup (if
mmdebstrap is run with --skip=output/dev) will fail.

Another potential solution would be to run each hook and apt invocation
in its own process namespace but this would require to remount /proc and
this in turn would require a new mount namespace as well but we'd like
to keep the mount namespace across multiple hooks...
2024-08-18 22:07:52 +02:00
examples/twb finalize mmdebstrap-autopkgtest-build-qemu 2022-03-06 10:16:11 +01:00
hooks hooks/file-mirror-automount/setup00.sh: prefix warning with W: 2024-03-23 22:37:26 +01:00
tests Wait for (reap) potential zombies and otherwise long-running background processes 2024-08-18 22:07:52 +02:00
.gitignore Instead of requiring root, use qemu 2018-11-21 00:21:43 +01:00
.mailmap .mailmap: add botched email due to https://salsa.debian.org/debian/devscripts/-/merge_requests/323 2023-02-01 18:39:26 +01:00
.perltidyrc format code with perltidy 2020-01-08 17:46:41 +01:00
caching_proxy.py caching_proxy.py: add comment about not using shutil.copyfileobj() 2023-06-14 07:34:51 +02:00
CHANGELOG.md release 1.5.2 2024-06-26 12:52:56 +02:00
coverage.py add --format=ext4 2024-05-12 18:38:47 +02:00
coverage.sh coverage.sh: add missing newline at end of curl output 2024-06-11 15:49:48 +02:00
coverage.txt Wait for (reap) potential zombies and otherwise long-running background processes 2024-08-18 22:07:52 +02:00
gpgvnoexpkeysig Reword gpgvnoexpkeysig public domain claim 2023-03-23 17:45:41 +01:00
ldconfig.fakechroot ldconfig.fakechroot: do not ignore it, if ldconfig was already called with -r 2023-01-23 07:20:13 +01:00
make_mirror.sh make_mirror.sh: force systemd to not mount /tmp as tmpfs 2024-06-02 08:44:28 +02:00
mmdebstrap Wait for (reap) potential zombies and otherwise long-running background processes 2024-08-18 22:07:52 +02:00
mmdebstrap-autopkgtest-build-qemu m-a-b-q: replace test_installed by dpkg-checkbuilddeps 2024-07-12 09:49:39 +02:00
proxysolver add my name to several scripts 2021-09-16 16:24:16 +02:00
README.md README.md: add new authors 2024-06-26 07:36:40 +02:00
run_null.sh run_null.sh: use file descriptors instead of temporary files to get the exit status of the first part of a pipeline 2023-01-16 23:16:11 +01:00
run_qemu.sh run_qemu.sh: allow setting MMDEBSTRAP_TESTS_DEBUG and add auto-detection for amlogic a311d bananapi 2024-06-03 08:04:03 +02:00
tarfilter tarfilter: add --type-exclude option 2023-10-23 10:26:47 +02:00

mmdebstrap

An alternative to debootstrap which uses apt internally and is thus able to use more than one mirror and resolve more complex dependencies.

Usage

Use like debootstrap:

sudo mmdebstrap unstable ./unstable-chroot

Without superuser privileges:

mmdebstrap unstable unstable-chroot.tar

With complex apt options:

cat /etc/apt/sources.list | mmdebstrap > unstable-chroot.tar

For the full documentation use:

pod2man ./mmdebstrap | man -l -

Or read a HTML version of the man page in either of these locations:

The sales pitch in comparison to debootstrap

Summary:

  • more than one mirror possible
  • security and updates mirror included for Debian stable chroots
  • twice as fast
  • chroot with apt in 11 seconds
  • gzipped tarball with apt is 27M small
  • bit-by-bit reproducible output
  • unprivileged operation using Linux user namespaces or fakechroot
  • can operate on filesystems mounted with nodev
  • foreign architecture chroots with qemu-user
  • variant installing only Essential:yes packages and dependencies
  • temporary chroots by redirecting to /dev/null
  • chroots without apt inside (for chroot from buildinfo file with debootsnap)

The author believes that a chroot of a Debian stable release should include the latest packages including security fixes by default. This has been a wontfix with debootstrap since 2009 (See #543819 and #762222). Since mmdebstrap uses apt internally, support for multiple mirrors comes for free and stable or oldstable chroots will include security and updates mirrors.

A side-effect of using apt is being twice as fast as debootstrap. The timings were carried out on a laptop with an Intel Core i5-5200U, using a mirror on localhost and a tmpfs.

variant mmdebstrap debootstrap
essential 9.52 s n.a
apt 10.98 s n.a
minbase 13.54 s 26.37 s
buildd 21.31 s 34.85 s
- 23.01 s 48.83 s

Apt considers itself an Essential: yes package. This feature allows one to create a chroot containing just the Essential: yes packages and apt (and their hard dependencies) in just 11 seconds.

If desired, a most minimal chroot with just the Essential: yes packages and their hard dependencies can be created with a gzipped tarball size of just 34M. By using dpkg's --path-exclude option to exclude documentation, even smaller gzipped tarballs of 21M in size are possible. If apt is included, the result is a gzipped tarball of only 27M.

These small sizes are also achieved because apt caches and other cruft is stripped from the chroot. This also makes the result bit-by-bit reproducible if the $SOURCE_DATE_EPOCH environment variable is set.

The author believes, that it should not be necessary to have superuser privileges to create a file (the chroot tarball) in one's home directory. Thus, mmdebstrap provides multiple options to create a chroot tarball with the right permissions without superuser privileges. This avoids a whole class of bugs like #921815. Depending on what is available, it uses either Linux user namespaces or fakechroot. Debootstrap supports fakechroot but will not create a tarball with the right permissions by itself. Support for Linux user namespaces is missing (see #829134).

When creating a chroot tarball with debootstrap, the temporary chroot directory cannot be on a filesystem that has been mounted with nodev. In unprivileged mode, mknod is never used, which means that /tmp can be used as a temporary directory location even if if it's mounted with nodev as a security measure.

If the chroot architecture cannot be executed by the current machine, qemu-user is used to allow one to create a foreign architecture chroot.

Limitations in comparison to debootstrap

Debootstrap supports creating a Debian chroot on non-Debian systems but mmdebstrap requires apt and is thus limited to Debian and derivatives. This means that mmdebstrap can never fully replace debootstrap and debootstrap will continue to be relevant in situations where you want to create a Debian chroot from a platform without apt and dpkg.

There is no SCRIPT argument.

The following options, don't exist: --second-stage, --exclude, --resolve-deps, --force-check-gpg, --merged-usr and --no-merged-usr.

The quirks from debootstrap are needed to create chroots of Debian unstable from snapshot.d.o before timestamp 20141107T220431Z or Debian 8 (Jessie) or later.

Tests

The script coverage.sh runs mmdebstrap in all kind of scenarios to execute all code paths of the script. It verifies its output in each scenario and displays the results gathered with Devel::Cover. It also compares the output of mmdebstrap with debootstrap in several scenarios. To run the testsuite, run:

./make_mirror.sh
CMD=./mmdebstrap ./coverage.sh

To also generate perl Devel::Cover data, omit the CMD environment variable. But that will also take a lot longer.

The make_mirror.sh script will be a no-op if nothing changed in Debian unstable. You don't need to run make_mirror.sh before every invocation of coverage.sh. When you make changes to make_mirror.sh and want to regenerate the cache, run:

touch -d yesterday shared/cache/debian/dists/unstable/Release

The script coverage.sh does not need an active internet connection by default. An online connection is only needed by the make_mirror.sh script which fills a local cache with a few minimal Debian mirror copies.

By default, coverage.sh will skip running a single test which tries creating a Ubuntu Focal chroot. To not skip that test, run coverage.sh with the environment variable ONLINE=yes.

If a test fails you can run individual tests by executing coverage.py with the test name and optionally limit it to a specific distribution like so:

CMD=./mmdebstrap ./coverage.py --dist unstable check-against-debootstrap-dist

Bugs

mmdebstrap has bugs. Report them here: https://gitlab.mister-muffin.de/josch/mmdebstrap/issues

Contributors

  • Johannes Schauer Marin Rodrigues (main author)
  • Helmut Grohne
  • Jochen Sprickerhof
  • Gioele Barabucci
  • Benjamin Drung
  • Josh Triplett
  • Konstantin Demin
  • Chris Hofstaedtler
  • Colin Watson
  • David Kalnischkies
  • Emilio Pozuelo Monfort
  • Francesco Poli
  • Jakub Wilk
  • Joe Groocock
  • Max-Julian Pogner
  • Nicolas Vigier
  • Raul Tambre
  • Steve Dodd
  • Trent W. Buck
  • Vagrant Cascadian