Compare commits

..

6 commits

Author SHA1 Message Date
6baf4151f9 Set MMDEBSTRAP_VERBOSITY to inform hooks of verbosity level
Hooks might want to be more or less verbose depending on how verbose
mmdebstrap is configured to be.
2022-05-13 11:26:08 +02:00
c2cb442899 Let apt decide unpack order instead of sorting filenames
Now that the deb files can reside in different places sorting them leads
to subtil differences in the order and hence the created chroot. apts
unpack order on the other hand might not be a good order (but why would
one sorted from a to z be one?), but it is far more stable as it is
independent on the filenames.
2022-05-13 11:26:08 +02:00
317bfea7e5 Support file:// mirrors the same way copy:// is supported
As long as you can make it so that the same path to the deb file works
inside and outside of the chroot using file:// as a mirror is no longer
a problem with the previous work.
2022-05-13 11:26:08 +02:00
3ceebe7e07 Ask apt to give us the deb filenames directly
Guessing filenames is boring. What if we could ask apt to tell us which
debs it downloaded (or found lying around elsewhere) directly? Turns out
we can rather easily avoiding a bunch of guesswork.
2022-05-13 11:26:08 +02:00
75e5a14e6d Factor out downloading packages with apt 2022-05-13 11:26:08 +02:00
ee142d52a5 Replace EDSP with EIPP usage obsoleting proxysolver
EIPP stands for "External Installation Planner Protocol" and is rather
similar to EDSP but with the clear advantage that we can extract the
information we need more easily as we can tell apt to write the file for
us rather than playing solver-in-the-middle and the problem space is
much smaller meaning less data for apt to generate and to pass through
our hands.

The idea here is simply that every package which doesn't have a Status
field in EIPP has the uninstalled status and the only reason its is part
of the EIPP request is that we want to change this by installing it.
That could be verified via the Install header at the start of the
request, but this commit doesn't implement that.

Note that this means we need "more" than the download-only mode can
provide: Either a simulation or "the real deal". Except we modify the
later to be a fancy no op.
2022-05-13 11:26:08 +02:00
156 changed files with 6081 additions and 10136 deletions

View file

@ -1,6 +1,2 @@
Johannes Schauer Marin Rodrigues <josch@mister-muffin.de>
Johannes Schauer Marin Rodrigues <josch@mister-muffin.de> <j.schauer@email.de>
Johannes Schauer Marin Rodrigues <josch@mister-muffin.de> <josch@debian.org>
Johannes Schauer Marin Rodrigues <josch@mister-muffin.de> <Johannes Schauer Marin Rodrigues josch@debian.org>
Helmut Grohne <helmut@subdivi.de> <helmut.grohne@intenta.de>
Benjamin Drung <benjamin.drung@ionos.com> <benjamin.drung@cloud.ionos.com>

View file

@ -1,188 +1,3 @@
1.5.4 (2024-10-28)
------------------
- do not generate apt sources.list entry if SUITE is empty
1.5.3 (2024-09-13)
------------------
- tidy up any zombie processes
- chrootless hurd-i386
- add --skip=cleanup/reproducible/machine-id
- m-a-b-q: replace test_installed by dpkg-checkbuilddeps
1.5.2 (2024-06-26)
------------------
- mmdebstrap-autopkgtest-build-qemu produces bit-by-bit reproducible output
1.5.1 (2024-06-03)
------------------
- in root and unshare mode, run 'mount --make-rprivate /' before bind-mounting
- switch apt variant from using 'apt-get dist-upgrade' to apt patterns
1.5.0 (2024-05-14)
------------------
- add --format=ext4
1.4.3 (2024-02-01)
------------------
- take hard links into account when computing disk usage
1.4.2 (2024-01-29)
------------------
- allow for start-stop-daemon to be in either /sbin or /usr/sbin
- mmdebstrap-autopkgtest-build-qemu: fix octal mode computation and hostname
1.4.1 (2024-01-09)
------------------
- set DPkg::Chroot-Directory in APT_CONFIG to simplify calling apt in hooks
- disallow running chrootless as root without fakeroot unless
--skip=check/chrootless is used
- only print short --help output if wrong args are passed
- read files passed as --aptopt and --dpkgopt outside the unshared namespace
1.4.0 (2023-10-24)
------------------
- add mmdebstrap-autopkgtest-build-qemu
- export container=mmdebstrap-unshare env variable in unshare-mode hooks
- add new skip options: output/dev, output/mknod, tar-in/mknod,
copy-in/mknod, sync-in/mknod
- stop copying qemu-$arch-static binary into the chroot
- tarfilter: add --type-exclude option
- set MMDEBSTRAP_FORMAT in hooks
- do not install priority:required in buildd variant following debootstrap
1.3.8 (2023-08-20)
------------------
- hooks/merged-usr: implement post-merging as debootstrap does
- exclude ./lost+found from tarball
1.3.7 (2023-06-21)
------------------
- add hooks/copy-host-apt-sources-and-preferences
1.3.6 (2023-06-16)
------------------
- bugfix release
1.3.5 (2023-03-20)
------------------
- bugfix release
1.3.4 (2023-03-16)
------------------
- more safeguards before automatically choosing unshare mode
1.3.3 (2023-02-19)
------------------
- testsuite improvements
1.3.2 (2023-02-16)
------------------
- unshare mode works in privileged docker containers
1.3.1 (2023-01-20)
------------------
- bugfix release
1.3.0 (2023-01-16)
------------------
- add hooks/maybe-jessie-or-older and hooks/maybe-merged-usr
- add --skip=check/signed-by
- hooks/jessie-or-older: split into two individual hook files
- skip running apt-get update if we are very sure that it was already run
- be more verbose when 'apt-get update' failed
- warn if a hook is named like one but not executable and if a hook is
executable but not named like one
- to find signed-by value, run gpg on the individual keys to print better
error messages in case it fails (gpg doesn't give an indication which file
it was unable to read) and print progress bar
- allow empty sources.list entries
1.2.5 (2023-01-04)
------------------
- bugfix release
1.2.4 (2022-12-23)
------------------
- bugfix release
- add jessie-or-older extract hook
1.2.3 (2022-11-16)
------------------
- use Text::ParseWords::shellwords instead of spawning a new shell
- mount and unmount once, instead for each run_chroot() call
1.2.2 (2022-10-27)
------------------
- allow /etc/apt/trusted.gpg.d/ not to exist
- always create /var/lib/dpkg/arch to make foreign architecture chrootless
tarballs bit-by-bit identical
- write an empty /etc/machine-id instead of writing 'uninitialized'
- only print progress bars on interactive terminals that are wide enough
1.2.1 (2022-09-08)
------------------
- bugfix release
1.2.0 (2022-09-05)
------------------
- remove proot mode
- error out if stdout is an interactive terminal
- replace taridshift by tarfilter --idshift
- tarfilter: add --transform option
- multiple --skip options can be separated by comma or whitespace
- also cleanup the contents of /run
- support apt patterns and paths with commas and whitespace in --include
- hooks: store the values of the --include option in MMDEBSTRAP_INCLUDE
- add new --skip options: chroot/start-stop-daemon, chroot/policy-rc.d
chroot/mount, chroot/mount/dev, chroot/mount/proc, chroot/mount/sys,
cleanup/run
1.1.0 (2022-07-26)
----------------
- mount a new /dev/pts instance into the chroot to make posix_openpt work
- adjust merged-/usr hook to work the same way as debootstrap
- add no-merged-usr hook
1.0.1 (2022-05-29)
------------------
- bugfix release
1.0.0 (2022-05-28)
------------------
- all documented interfaces are now considered stable
- allow file:// mirrors
- /var/cache/apt/archives/ is now allowed to contain *.deb packages
- add file-mirror-automount hook-dir
- set $MMDEBSTRAP_VERBOSITY in hooks
- rewrite coverage with multiple individual and skippable shell scripts
0.8.6 (2022-03-25)
------------------

View file

@ -23,11 +23,6 @@ For the full documentation use:
pod2man ./mmdebstrap | man -l -
Or read a HTML version of the man page in either of these locations:
- https://gitlab.mister-muffin.de/josch/mmdebstrap/wiki
- https://manpages.debian.org/unstable/mmdebstrap/mmdebstrap.1.en.html
The sales pitch in comparison to debootstrap
--------------------------------------------
@ -39,7 +34,7 @@ Summary:
- chroot with apt in 11 seconds
- gzipped tarball with apt is 27M small
- bit-by-bit reproducible output
- unprivileged operation using Linux user namespaces or fakechroot
- unprivileged operation using Linux user namespaces, fakechroot or proot
- can operate on filesystems mounted with nodev
- foreign architecture chroots with qemu-user
- variant installing only Essential:yes packages and dependencies
@ -83,9 +78,9 @@ privileges to create a file (the chroot tarball) in one's home directory.
Thus, mmdebstrap provides multiple options to create a chroot tarball with the
right permissions **without superuser privileges**. This avoids a whole class
of bugs like #921815. Depending on what is available, it uses either Linux user
namespaces or fakechroot. Debootstrap supports fakechroot but will not
namespaces, fakechroot or proot. Debootstrap supports fakechroot but will not
create a tarball with the right permissions by itself. Support for Linux user
namespaces is missing (see #829134).
namespaces and proot is missing (see bugs #829134 and #698347, respectively).
When creating a chroot tarball with debootstrap, the temporary chroot directory
cannot be on a filesystem that has been mounted with nodev. In unprivileged
@ -142,11 +137,6 @@ By default, `coverage.sh` will skip running a single test which tries creating
a Ubuntu Focal chroot. To not skip that test, run `coverage.sh` with the
environment variable `ONLINE=yes`.
If a test fails you can run individual tests by executing `coverage.py` with
the test name and optionally limit it to a specific distribution like so:
CMD=./mmdebstrap ./coverage.py --dist unstable check-against-debootstrap-dist
Bugs
====
@ -158,21 +148,10 @@ Contributors
- Johannes Schauer Marin Rodrigues (main author)
- Helmut Grohne
- Jochen Sprickerhof
- Gioele Barabucci
- Benjamin Drung
- Steve Dodd
- Josh Triplett
- Konstantin Demin
- Chris Hofstaedtler
- Colin Watson
- David Kalnischkies
- Emilio Pozuelo Monfort
- Francesco Poli
- Jakub Wilk
- Joe Groocock
- Max-Julian Pogner
- Nicolas Vigier
- Raul Tambre
- Steve Dodd
- Trent W. Buck
- Vagrant Cascadian
- Gioele Barabucci

View file

@ -1,118 +0,0 @@
#!/usr/bin/env python3
import sys
import os
import time
import http.client
import http.server
from io import StringIO
import pathlib
import urllib.parse
oldcachedir = None
newcachedir = None
readonly = False
class ProxyRequestHandler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
assert int(self.headers.get("Content-Length", 0)) == 0
assert self.headers["Host"]
pathprefix = "http://" + self.headers["Host"] + "/"
assert self.path.startswith(pathprefix)
sanitizedpath = urllib.parse.unquote(self.path.removeprefix(pathprefix))
oldpath = oldcachedir / sanitizedpath
newpath = newcachedir / sanitizedpath
if not readonly:
newpath.parent.mkdir(parents=True, exist_ok=True)
# just send back to client
if newpath.exists():
print(f"proxy cached: {self.path}", file=sys.stderr)
self.wfile.write(b"HTTP/1.1 200 OK\r\n")
self.send_header("Content-Length", newpath.stat().st_size)
self.end_headers()
with newpath.open(mode="rb") as new:
while True:
buf = new.read(64 * 1024) # same as shutil uses
if not buf:
break
self.wfile.write(buf)
self.wfile.flush()
return
if readonly:
newpath = pathlib.Path("/dev/null")
# copy from oldpath to newpath and send back to client
# Only take files from the old cache if they are .deb files or Packages
# files in the by-hash directory as only those are unique by their path
# name. Other files like InRelease files have to be downloaded afresh.
if oldpath.exists() and (
oldpath.suffix == ".deb" or "by-hash" in oldpath.parts
):
print(f"proxy cached: {self.path}", file=sys.stderr)
self.wfile.write(b"HTTP/1.1 200 OK\r\n")
self.send_header("Content-Length", oldpath.stat().st_size)
self.end_headers()
with oldpath.open(mode="rb") as old, newpath.open(mode="wb") as new:
# we are not using shutil.copyfileobj() because we want to
# write to two file objects simultaneously
while True:
buf = old.read(64 * 1024) # same as shutil uses
if not buf:
break
self.wfile.write(buf)
new.write(buf)
self.wfile.flush()
return
# download fresh copy
try:
print(f"\rproxy download: {self.path}", file=sys.stderr)
conn = http.client.HTTPConnection(self.headers["Host"], timeout=5)
conn.request("GET", self.path, None, dict(self.headers))
res = conn.getresponse()
assert (res.status, res.reason) == (200, "OK"), (res.status, res.reason)
self.wfile.write(b"HTTP/1.1 200 OK\r\n")
for k, v in res.getheaders():
# do not allow a persistent connection
if k == "connection":
continue
self.send_header(k, v)
self.end_headers()
with newpath.open(mode="wb") as f:
# we are not using shutil.copyfileobj() because we want to
# write to two file objects simultaneously and throttle the
# writing speed to 1024 kB/s
while True:
buf = res.read(64 * 1024) # same as shutil uses
if not buf:
break
self.wfile.write(buf)
f.write(buf)
time.sleep(64 / 1024) # 1024 kB/s
self.wfile.flush()
except Exception as e:
self.send_error(502)
def main():
global oldcachedir, newcachedir, readonly
if sys.argv[1] == "--readonly":
readonly = True
oldcachedir = pathlib.Path(sys.argv[2])
newcachedir = pathlib.Path(sys.argv[3])
else:
oldcachedir = pathlib.Path(sys.argv[1])
newcachedir = pathlib.Path(sys.argv[2])
print(f"starting caching proxy for {newcachedir}", file=sys.stderr)
httpd = http.server.ThreadingHTTPServer(
server_address=("", 8080), RequestHandlerClass=ProxyRequestHandler
)
httpd.serve_forever()
if __name__ == "__main__":
main()

View file

@ -1,474 +0,0 @@
#!/usr/bin/env python3
from debian.deb822 import Deb822, Release
import email.utils
import os
import sys
import shutil
import subprocess
import argparse
import time
from datetime import timedelta
from collections import defaultdict
from itertools import product
have_qemu = os.getenv("HAVE_QEMU", "yes") == "yes"
have_binfmt = os.getenv("HAVE_BINFMT", "yes") == "yes"
run_ma_same_tests = os.getenv("RUN_MA_SAME_TESTS", "yes") == "yes"
use_host_apt_config = os.getenv("USE_HOST_APT_CONFIG", "no") == "yes"
cmd = os.getenv("CMD", "./mmdebstrap")
default_dist = os.getenv("DEFAULT_DIST", "unstable")
all_dists = ["oldstable", "stable", "testing", "unstable"]
default_mode = "auto"
all_modes = ["auto", "root", "unshare", "fakechroot", "chrootless"]
default_variant = "apt"
all_variants = [
"extract",
"custom",
"essential",
"apt",
"minbase",
"buildd",
"-",
"standard",
]
default_format = "auto"
all_formats = ["auto", "directory", "tar", "squashfs", "ext2", "ext4", "null"]
mirror = os.getenv("mirror", "http://127.0.0.1/debian")
hostarch = subprocess.check_output(["dpkg", "--print-architecture"]).decode().strip()
release_path = f"./shared/cache/debian/dists/{default_dist}/InRelease"
if not os.path.exists(release_path):
print("path doesn't exist:", release_path, file=sys.stderr)
print("run ./make_mirror.sh first", file=sys.stderr)
exit(1)
if os.getenv("SOURCE_DATE_EPOCH") is not None:
s_d_e = os.getenv("SOURCE_DATE_EPOCH")
else:
with open(release_path) as f:
rel = Release(f)
s_d_e = str(email.utils.mktime_tz(email.utils.parsedate_tz(rel["Date"])))
separator = (
"------------------------------------------------------------------------------"
)
def skip(condition, dist, mode, variant, fmt):
if not condition:
return ""
for line in condition.splitlines():
if not line:
continue
if eval(line):
return line.strip()
return ""
def parse_config(confname):
config_dict = defaultdict(dict)
config_order = list()
all_vals = {
"Dists": all_dists,
"Modes": all_modes,
"Variants": all_variants,
"Formats": all_formats,
}
with open(confname) as f:
for test in Deb822.iter_paragraphs(f):
if "Test" not in test.keys():
print("Test without name", file=sys.stderr)
exit(1)
name = test["Test"]
config_order.append(name)
for k in test.keys():
v = test[k]
if k not in [
"Test",
"Dists",
"Modes",
"Variants",
"Formats",
"Skip-If",
"Needs-QEMU",
"Needs-Root",
"Needs-APT-Config",
]:
print(f"Unknown field name {k} in test {name}")
exit(1)
if k in all_vals.keys():
if v == "default":
print(
f"Setting {k} to default in Test {name} is redundant",
file=sys.stderr,
)
exit(1)
if v == "any":
v = all_vals[k]
else:
# else, split the value by whitespace
v = v.split()
for i in v:
if i not in all_vals[k]:
print(
f"{i} is not a valid value for {k}", file=sys.stderr
)
exit(1)
config_dict[name][k] = v
return config_order, config_dict
def format_test(num, total, name, dist, mode, variant, fmt, config_dict):
ret = f"({num}/{total}) {name}"
if len(config_dict[name].get("Dists", [])) > 1:
ret += f" --dist={dist}"
if len(config_dict[name].get("Modes", [])) > 1:
ret += f" --mode={mode}"
if len(config_dict[name].get("Variants", [])) > 1:
ret += f" --variant={variant}"
if len(config_dict[name].get("Formats", [])) > 1:
ret += f" --format={fmt}"
return ret
def print_time_per_test(time_per_test, name="test"):
print(
f"average time per {name}:",
sum(time_per_test.values(), start=timedelta()) / len(time_per_test),
file=sys.stderr,
)
print(
f"median time per {name}:",
sorted(time_per_test.values())[len(time_per_test) // 2],
file=sys.stderr,
)
head_tail_num = 10
print(f"{head_tail_num} fastests {name}s:", file=sys.stderr)
for k, v in sorted(time_per_test.items(), key=lambda i: i[1])[
: min(head_tail_num, len(time_per_test))
]:
print(f" {k}: {v}", file=sys.stderr)
print(f"{head_tail_num} slowest {name}s:", file=sys.stderr)
for k, v in sorted(time_per_test.items(), key=lambda i: i[1], reverse=True)[
: min(head_tail_num, len(time_per_test))
]:
print(f" {k}: {v}", file=sys.stderr)
def main():
parser = argparse.ArgumentParser()
parser.add_argument("test", nargs="*", help="only run these tests")
parser.add_argument(
"-x",
"--exitfirst",
action="store_const",
dest="maxfail",
const=1,
help="exit instantly on first error or failed test.",
)
parser.add_argument(
"--maxfail",
metavar="num",
action="store",
type=int,
dest="maxfail",
default=0,
help="exit after first num failures or errors.",
)
parser.add_argument(
"--mode",
metavar="mode",
help=f"only run tests with this mode (Default = {default_mode})",
)
parser.add_argument(
"--dist",
metavar="dist",
help=f"only run tests with this dist (Default = {default_dist})",
)
parser.add_argument(
"--variant",
metavar="variant",
help=f"only run tests with this variant (Default = {default_variant})",
)
parser.add_argument(
"--format",
metavar="format",
help=f"only run tests with this format (Default = {default_format})",
)
parser.add_argument(
"--skip", metavar="test", action="append", help="skip this test"
)
args = parser.parse_args()
# copy over files from git or as distributed
for git, dist, target in [
("./mmdebstrap", "/usr/bin/mmdebstrap", "mmdebstrap"),
("./tarfilter", "/usr/bin/mmtarfilter", "tarfilter"),
(
"./proxysolver",
"/usr/lib/apt/solvers/mmdebstrap-dump-solution",
"proxysolver",
),
(
"./ldconfig.fakechroot",
"/usr/libexec/mmdebstrap/ldconfig.fakechroot",
"ldconfig.fakechroot",
),
]:
if os.path.exists(git):
shutil.copy(git, f"shared/{target}")
else:
shutil.copy(dist, f"shared/{target}")
# copy over hooks from git or as distributed
if os.path.exists("hooks"):
shutil.copytree("hooks", "shared/hooks", dirs_exist_ok=True)
else:
shutil.copytree(
"/usr/share/mmdebstrap/hooks", "shared/hooks", dirs_exist_ok=True
)
# parse coverage.txt
config_order, config_dict = parse_config("coverage.txt")
indirbutnotcovered = set(
[d for d in os.listdir("tests") if not d.startswith(".")]
) - set(config_order)
if indirbutnotcovered:
print(
"test(s) missing from coverage.txt: %s"
% (", ".join(sorted(indirbutnotcovered))),
file=sys.stderr,
)
exit(1)
coveredbutnotindir = set(config_order) - set(
[d for d in os.listdir("tests") if not d.startswith(".")]
)
if coveredbutnotindir:
print(
"test(s) missing from ./tests: %s"
% (", ".join(sorted(coveredbutnotindir))),
file=sys.stderr,
)
exit(1)
# produce the list of tests using the cartesian product of all allowed
# dists, modes, variants and formats of a given test
tests = []
for name in config_order:
test = config_dict[name]
for dist, mode, variant, fmt in product(
test.get("Dists", [default_dist]),
test.get("Modes", [default_mode]),
test.get("Variants", [default_variant]),
test.get("Formats", [default_format]),
):
skipreason = skip(test.get("Skip-If"), dist, mode, variant, fmt)
if skipreason:
tt = ("skip", skipreason)
elif (
test.get("Needs-APT-Config", "false") == "true" and use_host_apt_config
):
tt = ("skip", "test cannot use host apt config")
elif have_qemu:
tt = "qemu"
elif test.get("Needs-QEMU", "false") == "true":
tt = ("skip", "test needs QEMU")
elif test.get("Needs-Root", "false") == "true":
tt = "sudo"
elif mode == "root":
tt = "sudo"
else:
tt = "null"
tests.append((tt, name, dist, mode, variant, fmt))
torun = []
num_tests = len(tests)
if args.test:
# check if all given tests are either a valid name or a valid number
for test in args.test:
if test in [name for (_, name, _, _, _, _) in tests]:
continue
if not test.isdigit():
print(f"cannot find test named {test}", file=sys.stderr)
exit(1)
if int(test) >= len(tests) or int(test) <= 0 or str(int(test)) != test:
print(f"test number {test} doesn't exist", file=sys.stderr)
exit(1)
for i, (_, name, _, _, _, _) in enumerate(tests):
# if either the number or test name matches, then we use this test,
# otherwise we skip it
if name in args.test:
torun.append(i)
if str(i + 1) in args.test:
torun.append(i)
num_tests = len(torun)
starttime = time.time()
skipped = defaultdict(list)
failed = []
num_success = 0
num_finished = 0
time_per_test = {}
acc_time_per_test = defaultdict(list)
for i, (test, name, dist, mode, variant, fmt) in enumerate(tests):
if torun and i not in torun:
continue
print(separator, file=sys.stderr)
print("(%d/%d) %s" % (i + 1, len(tests), name), file=sys.stderr)
print("dist: %s" % dist, file=sys.stderr)
print("mode: %s" % mode, file=sys.stderr)
print("variant: %s" % variant, file=sys.stderr)
print("format: %s" % fmt, file=sys.stderr)
if num_finished > 0:
currenttime = time.time()
timeleft = timedelta(
seconds=int(
(num_tests - num_finished)
* (currenttime - starttime)
/ num_finished
)
)
print("time left: %s" % timeleft, file=sys.stderr)
if failed:
print("failed: %d" % len(failed), file=sys.stderr)
num_finished += 1
with open("tests/" + name) as fin, open("shared/test.sh", "w") as fout:
for line in fin:
line = line.replace("{{ CMD }}", cmd)
line = line.replace("{{ SOURCE_DATE_EPOCH }}", s_d_e)
line = line.replace("{{ DIST }}", dist)
line = line.replace("{{ MIRROR }}", mirror)
line = line.replace("{{ MODE }}", mode)
line = line.replace("{{ VARIANT }}", variant)
line = line.replace("{{ FORMAT }}", fmt)
line = line.replace("{{ HOSTARCH }}", hostarch)
fout.write(line)
# ignore:
# SC2016 Expressions don't expand in single quotes, use double quotes for that.
# SC2050 This expression is constant. Did you forget the $ on a variable?
# SC2194 This word is constant. Did you forget the $ on a variable?
shellcheck = subprocess.run(
[
"shellcheck",
"--exclude=SC2050,SC2194,SC2016",
"-f",
"gcc",
"shared/test.sh",
],
check=False,
stdout=subprocess.PIPE,
).stdout.decode()
shfmt = subprocess.run(
[
"shfmt",
"--posix",
"--binary-next-line",
"--case-indent",
"--indent",
"2",
"--simplify",
"-d",
"shared/test.sh",
],
check=False,
stdout=subprocess.PIPE,
).stdout.decode()
argv = None
match test:
case "qemu":
argv = ["./run_qemu.sh"]
case "sudo":
argv = ["./run_null.sh", "SUDO"]
case "null":
argv = ["./run_null.sh"]
case ("skip", reason):
skipped[reason].append(
format_test(
i + 1, len(tests), name, dist, mode, variant, fmt, config_dict
)
)
print(f"skipped because of {reason}", file=sys.stderr)
continue
print(separator, file=sys.stderr)
if args.skip and name in args.skip:
print(f"skipping because of --skip={name}", file=sys.stderr)
continue
if args.dist and args.dist != dist:
print(f"skipping because of --dist={args.dist}", file=sys.stderr)
continue
if args.mode and args.mode != mode:
print(f"skipping because of --mode={args.mode}", file=sys.stderr)
continue
if args.variant and args.variant != variant:
print(f"skipping because of --variant={args.variant}", file=sys.stderr)
continue
if args.format and args.format != fmt:
print(f"skipping because of --format={args.format}", file=sys.stderr)
continue
before = time.time()
proc = subprocess.Popen(argv)
try:
proc.wait()
except KeyboardInterrupt:
proc.terminate()
proc.wait()
break
after = time.time()
walltime = timedelta(seconds=int(after - before))
formated_test_name = format_test(
i + 1, len(tests), name, dist, mode, variant, fmt, config_dict
)
time_per_test[formated_test_name] = walltime
acc_time_per_test[name].append(walltime)
print(separator, file=sys.stderr)
print(f"duration: {walltime}", file=sys.stderr)
if proc.returncode != 0 or shellcheck != "" or shfmt != "":
if shellcheck != "":
print(shellcheck)
if shfmt != "":
print(shfmt)
failed.append(formated_test_name)
print("result: FAILURE", file=sys.stderr)
else:
print("result: SUCCESS", file=sys.stderr)
num_success += 1
if args.maxfail and len(failed) >= args.maxfail:
break
print(separator, file=sys.stderr)
print(
"successfully ran %d tests" % num_success,
file=sys.stderr,
)
if skipped:
print("skipped %d:" % sum([len(v) for v in skipped.values()]), file=sys.stderr)
for reason, l in skipped.items():
print(f"skipped because of {reason}:", file=sys.stderr)
for t in l:
print(f" {t}", file=sys.stderr)
if len(time_per_test) > 1:
print_time_per_test(time_per_test)
if len(acc_time_per_test) > 1:
print_time_per_test(
{
f"{len(v)}x {k}": sum(v, start=timedelta())
for k, v in acc_time_per_test.items()
},
"accumulated test",
)
if failed:
print("failed %d:" % len(failed), file=sys.stderr)
for f in failed:
print(f, file=sys.stderr)
currenttime = time.time()
walltime = timedelta(seconds=int(currenttime - starttime))
print(f"total runtime: {walltime}", file=sys.stderr)
if failed:
exit(1)
if __name__ == "__main__":
main()

File diff suppressed because it is too large Load diff

View file

@ -1,437 +0,0 @@
Test: debootstrap
Dists: any
Variants: minbase buildd -
Needs-Root: true
Needs-APT-Config: true
Test: check-against-debootstrap-dist
Dists: any
Variants: minbase buildd -
Needs-Root: true
Needs-APT-Config: true
Test: as-debootstrap-unshare-wrapper
Modes: unshare
Needs-Root: true
Variants: minbase -
Needs-APT-Config: true
Test: help
Test: man
Test: version
Test: create-directory
Needs-Root: true
Test: unshare-as-root-user
Needs-Root: true
Test: dist-using-codename
Dists: any
Needs-APT-Config: true
Test: fail-without-etc-subuid
Needs-QEMU: true
Test: fail-without-username-in-etc-subuid
Needs-QEMU: true
Test: unshare-as-root-user-inside-chroot
Needs-Root: true
Needs-APT-Config: true
Test: root-mode-inside-chroot
Needs-Root: true
Needs-APT-Config: true
Test: root-mode-inside-unshare-chroot
Modes: unshare
Needs-APT-Config: true
Test: root-without-cap-sys-admin
Needs-Root: true
Test: mount-is-missing
Needs-QEMU: true
Test: mmdebstrap
Needs-Root: true
Modes: root
Formats: tar squashfs ext2 ext4
Variants: essential apt minbase buildd - standard
Skip-If:
variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558
mode == "fakechroot" and variant in ["-", "standard"] # no extended attributes
variant == "standard" and dist in ["oldstable", "stable"] and hostarch in ["armel", "armhf", "mipsel"] # #1031276
Test: check-for-bit-by-bit-identical-format-output
Modes: unshare fakechroot
Formats: tar squashfs ext2 ext4
Variants: essential apt minbase buildd - standard
Skip-If:
variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558
mode == "fakechroot" and variant in ["-", "standard"] # no extended attributes
variant == "standard" and dist in ["oldstable", "stable"] and hostarch in ["armel", "armhf", "mipsel"] # #1031276
Test: tarfilter-idshift
Needs-QEMU: true
Test: progress-bars-on-fake-tty
Test: debug-output-on-fake-tty
Test: existing-empty-directory
Needs-Root: true
Test: existing-directory-with-lost-found
Needs-Root: true
Test: fail-installing-to-non-empty-lost-found
Test: fail-installing-to-non-empty-target-directory
Test: missing-device-nodes-outside-the-chroot
Needs-QEMU: true
Test: missing-dev-sys-proc-inside-the-chroot
Modes: unshare
Variants: custom
Test: chroot-directory-not-accessible-by-apt-user
Needs-Root: true
Test: cwd-directory-not-accessible-by-unshared-user
Needs-Root: true
Modes: unshare
Test: create-gzip-compressed-tarball
Test: custom-tmpdir
Needs-Root: true
Modes: unshare
Test: xz-compressed-tarball
Test: directory-ending-in-tar
Modes: root
Needs-Root: true
Test: auto-mode-without-unshare-capabilities
Needs-QEMU: true
Test: fail-with-missing-lz4
Test: fail-with-path-with-quotes
Test: create-tarball-with-tmp-mounted-nodev
Needs-QEMU: true
Test: read-from-stdin-write-to-stdout
Test: supply-components-manually
Modes: root
Needs-Root: true
Needs-APT-Config: true
Test: stable-default-mirror
Needs-QEMU: true
Test: pass-distribution-but-implicitly-write-to-stdout
Needs-QEMU: true
Test: aspcud-apt-solver
Test: mirror-is-stdin
Test: copy-mirror
Needs-QEMU: true
Test: file-mirror
Needs-QEMU: true
Test: file-mirror-automount-hook
Modes: root unshare fakechroot
Needs-QEMU: true
Test: mirror-is-deb
Test: mirror-is-real-file
Needs-APT-Config: true
Test: deb822-1-2
Modes: root
Needs-Root: true
Needs-APT-Config: true
Test: deb822-2-2
Modes: root
Needs-Root: true
Needs-APT-Config: true
Test: automatic-mirror-from-suite
Needs-QEMU: true
Test: invalid-mirror
Needs-APT-Config: true
Test: fail-installing-to-root
Modes: root
Needs-Root: true
Test: fail-installing-to-existing-file
Modes: root
Needs-Root: true
Test: arm64-without-qemu-support
Needs-QEMU: true
Skip-If: hostarch != "amd64"
Test: i386-which-can-be-executed-without-qemu
Needs-QEMU: true
Skip-If:
hostarch != "amd64"
not run_ma_same_tests
Test: include-foreign-libmagic-mgc
Needs-Root: true
Needs-APT-Config: true
Skip-If:
hostarch not in ["amd64", "arm64"]
not run_ma_same_tests
Test: include-foreign-libmagic-mgc-with-multiple-arch-options
Needs-Root: true
Needs-APT-Config: true
Skip-If:
hostarch not in ["amd64", "arm64"]
not run_ma_same_tests
Test: aptopt
Needs-Root: true
Test: keyring
Needs-QEMU: true
Test: keyring-overwrites
Needs-Root: true
Needs-APT-Config: true
Test: signed-by-without-host-keys
Needs-QEMU: true
Test: ascii-armored-keys
Needs-QEMU: true
Test: signed-by-with-host-keys
Needs-Root: true
Needs-APT-Config: true
Test: dpkgopt
Needs-Root: true
Test: include
Needs-Root: true
Test: multiple-include
Needs-Root: true
Test: include-with-multiple-apt-sources
Needs-Root: true
Test: essential-hook
Needs-Root: true
Test: customize-hook
Needs-Root: true
Test: failing-customize-hook
Needs-Root: true
Test: sigint-during-customize-hook
Needs-Root: true
Test: hook-directory
Needs-Root: true
Test: eatmydata-via-hook-dir
Needs-Root: true
Test: special-hooks-using-helpers
Needs-Root: true
Needs-APT-Config: true
Test: special-hooks-using-helpers-and-env-vars
Needs-Root: true
Needs-APT-Config: true
Test: special-hooks-with-mode-mode
Modes: root unshare fakechroot
Test: debootstrap-no-op-options
Needs-Root: true
Test: verbose
Variants: standard
Skip-If:
variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558
Test: debug
Variants: standard
Skip-If:
variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558
Test: quiet
Needs-Root: true
Test: logfile
Needs-Root: true
Needs-APT-Config: true
Test: without-etc-resolv-conf-and-etc-hostname
Needs-QEMU: true
Test: preserve-mode-of-etc-resolv-conf-and-etc-hostname
Modes: root
Needs-QEMU: true
Test: not-having-to-install-apt-in-include-because-a-hook-did-it-before
Test: remove-start-stop-daemon-and-policy-rc-d-in-hook
Test: skip-start-stop-daemon-policy-rc
Test: skip-mount
Modes: unshare
Test: compare-output-with-pre-seeded-var-cache-apt-archives
Needs-QEMU: true
Variants: any
Skip-If:
variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558
Test: create-directory-dry-run
Modes: root
Test: create-tarball-dry-run
Variants: any
Modes: any
Test: unpack-doc-debian
Modes: root fakechroot
Variants: extract
Needs-APT-Config: true
Test: install-doc-debian
Modes: chrootless
Variants: custom
Needs-APT-Config: true
Test: chrootless
Variants: essential
Modes: chrootless
Needs-Root: true
Skip-If:
dist == "oldstable"
Test: chrootless-fakeroot
Variants: essential
Modes: chrootless
Skip-If:
dist == "oldstable"
hostarch in ["i386", "armel", "armhf", "mipsel"] # #1023286
Test: chrootless-foreign
Variants: essential
Modes: chrootless
Skip-If:
dist == "oldstable"
hostarch not in ["amd64", "arm64"]
not run_ma_same_tests
Needs-QEMU: true
Test: install-doc-debian-and-output-tarball
Variants: custom
Modes: chrootless
Needs-APT-Config: true
Test: install-doc-debian-and-test-hooks
Variants: custom
Modes: chrootless
Needs-APT-Config: true
Test: install-libmagic-mgc-on-foreign
Variants: custom
Modes: chrootless
Skip-If:
hostarch not in ["amd64", "arm64"]
not have_binfmt
Test: install-busybox-based-sub-essential-system
Needs-Root: true
Test: create-foreign-tarball
Modes: root unshare fakechroot
Skip-If:
hostarch not in ["amd64", "arm64"]
mode == "fakechroot" and not run_ma_same_tests
mode == "fakechroot" and hostarch == "arm64" # usrmerge postinst under fakechroot wants to copy /lib/ld-linux-x86-64.so.2 (which does not exist) instead of /lib64/ld-linux-x86-64.so.2
not have_binfmt
Test: no-sbin-in-path
Modes: fakechroot
Test: dev-ptmx
Modes: root unshare
Test: error-if-stdout-is-tty
Test: variant-custom-timeout
Test: include-deb-file
Modes: root unshare fakechroot
Needs-APT-Config: true
Test: unshare-include-deb
Modes: unshare
Test: pivot_root
Modes: root unshare
Needs-APT-Config: true
Test: jessie-or-older
Needs-Root: true
Modes: root unshare fakechroot
Variants: essential apt minbase
Skip-If: mode == "fakechroot" and hostarch in ["i386", "armel", "armhf", "mipsel"] # #1023286
Test: apt-patterns
Test: apt-patterns-custom
Test: empty-sources.list
Test: merged-fakechroot-inside-unmerged-chroot
Needs-Root: true
Needs-APT-Config: true
Skip-If:
hostarch in ["i386", "armel", "armhf", "mipsel"] # #1023286
dist in ["testing", "unstable"] # #1053671
Test: auto-mode-as-normal-user
Modes: auto
Test: skip-output-dev
Modes: root unshare
Test: skip-output-mknod
Modes: root unshare
Test: skip-tar-in-mknod
Modes: unshare
Test: zombie-reaping
Modes: unshare
Test: empty-suite

View file

@ -1,7 +1,6 @@
#!/bin/sh
#
# No copyright is claimed. This code is in the public domain; do with
# it what you wish.
# This script is in the public domain
#
# Author: Johannes Schauer Marin Rodrigues <josch@mister-muffin.de>
#
@ -28,23 +27,23 @@
set -eu
find_gpgv_status_fd() {
while [ "$#" -gt 0 ]; do
if [ "$1" = '--status-fd' ]; then
echo "$2"
return 0
fi
shift
done
# default fd is stdout
echo 1
while [ "$#" -gt 0 ]; do
if [ "$1" = '--status-fd' ]; then
echo "$2"
return 0
fi
shift
done
# default fd is stdout
echo 1
}
GPGSTATUSFD="$(find_gpgv_status_fd "$@")"
case $GPGSTATUSFD in
'' | *[!0-9]*)
echo "invalid --status-fd argument" >&2
exit 1
;;
''|*[!0-9]*)
echo "invalid --status-fd argument" >&2
exit 1
;;
esac
# we need eval because we cannot redirect a variable fd

View file

@ -1,10 +1,6 @@
#!/bin/sh
set -eu
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
set -x
fi
set -exu
rootdir="$1"

View file

@ -1,10 +1,6 @@
#!/bin/sh
set -eu
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
set -x
fi
set -exu
rootdir="$1"

View file

@ -1,44 +0,0 @@
#!/usr/bin/perl
#
# This script makes sure that all packages that are installed both locally as
# well as inside the chroot have the same version.
#
# It is implemented in Perl because there are no associative arrays in POSIX
# shell.
use strict;
use warnings;
sub get_pkgs {
my $root = shift;
my %pkgs = ();
open(my $fh, '-|', 'dpkg-query', "--root=$root", '--showformat',
'${binary:Package}=${Version}\n', '--show')
// die "cannot exec dpkg-query";
while (my $line = <$fh>) {
my ($pkg, $ver) = split(/=/, $line, 2);
$pkgs{$pkg} = $ver;
}
close $fh;
if ($? != 0) { die "failed to run dpkg-query" }
return %pkgs;
}
my %pkgs_local = get_pkgs('/');
my %pkgs_chroot = get_pkgs($ARGV[0]);
my @diff = ();
foreach my $pkg (keys %pkgs_chroot) {
next unless exists $pkgs_local{$pkg};
if ($pkgs_local{$pkg} ne $pkgs_chroot{$pkg}) {
push @diff, $pkg;
}
}
if (scalar @diff > 0) {
print STDERR "E: packages from the host and the chroot differ:\n";
foreach my $pkg (@diff) {
print STDERR "E: $pkg $pkgs_local{$pkg} $pkgs_chroot{$pkg}\n";
}
exit 1;
}

View file

@ -1,55 +0,0 @@
#!/bin/sh
#
# This script makes sure that the apt sources.list and preferences from outside
# the chroot also exist inside the chroot by *appending* them to any existing
# files. If you do not want to keep the original content, add another setup
# hook before this one which cleans up the files you don't want to keep.
#
# If instead of copying sources.list verbatim you want to mangle its contents,
# consider using python-apt for that. An example can be found in the Debian
# packaging of mmdebstrap in ./debian/tests/sourcesfilter
set -eu
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
set -x
fi
if [ -n "${MMDEBSTRAP_SUITE:-}" ]; then
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 1 ]; then
echo "W: using a non-empty suite name $MMDEBSTRAP_SUITE does not make sense with this hook and might select the wrong Essential:yes package set" >&2
fi
fi
rootdir="$1"
SOURCELIST="/etc/apt/sources.list"
eval "$(apt-config shell SOURCELIST Dir::Etc::SourceList/f)"
SOURCEPARTS="/etc/apt/sources.d/"
eval "$(apt-config shell SOURCEPARTS Dir::Etc::SourceParts/d)"
PREFERENCES="/etc/apt/preferences"
eval "$(apt-config shell PREFERENCES Dir::Etc::Preferences/f)"
PREFERENCESPARTS="/etc/apt/preferences.d/"
eval "$(apt-config shell PREFERENCESPARTS Dir::Etc::PreferencesParts/d)"
for f in "$SOURCELIST" \
"$SOURCEPARTS"/*.list \
"$SOURCEPARTS"/*.sources \
"$PREFERENCES" \
"$PREFERENCESPARTS"/*; do
[ -e "$f" ] || continue
mkdir --parents "$(dirname "$rootdir/$f")"
if [ -e "$rootdir/$f" ]; then
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 2 ]; then
echo "I: $f already exists in chroot, appending..." >&2
fi
# Add extra newline between old content and new content.
# This is required in case of deb822 files.
echo >> "$rootdir/$f"
fi
cat "$f" >> "$rootdir/$f"
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
echo "D: contents of $f inside the chroot:" >&2
cat "$rootdir/$f" >&2
fi
done

View file

@ -1,10 +1,6 @@
#!/bin/sh
set -eu
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
set -x
fi
set -exu
rootdir="$1"
@ -13,14 +9,14 @@ if [ -e "$rootdir/var/lib/dpkg/arch" ]; then
else
chrootarch=$(dpkg --print-architecture)
fi
libdir="/usr/lib/$(dpkg-architecture -a "$chrootarch" -q DEB_HOST_MULTIARCH)"
libdir="/usr/lib/$(dpkg-architecture -a $chrootarch -q DEB_HOST_MULTIARCH)"
# if eatmydata was actually installed properly, then we are not removing
# anything here
if ! chroot "$rootdir" dpkg-query --show eatmydata; then
if ! chroot "$rootdir" dpkg-query --list eatmydata; then
rm "$rootdir/usr/bin/eatmydata"
fi
if ! chroot "$rootdir" dpkg-query --show libeatmydata1; then
if ! chroot "$rootdir" dpkg-query --list libeatmydata1; then
rm "$rootdir$libdir"/libeatmydata.so*
fi

View file

@ -1,10 +1,6 @@
#!/bin/sh
set -eu
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
set -x
fi
set -exu
rootdir="$1"
@ -14,10 +10,8 @@ else
chrootarch=$(dpkg --print-architecture)
fi
trusted=
eval "$(apt-config shell trusted Dir::Etc::trusted/f)"
trustedparts=
eval "$(apt-config shell trustedparts Dir::Etc::trustedparts/d)"
eval $(apt-config shell trusted Dir::Etc::trusted/f)
eval $(apt-config shell trustedparts Dir::Etc::trustedparts/d)
tmpfile=$(mktemp --tmpdir="$rootdir/tmp")
cat << END > "$tmpfile"
Apt::Architecture "$chrootarch";
@ -31,8 +25,8 @@ END
# nothing will be printed for them
tmpdir=$(mktemp --directory --tmpdir="$rootdir/tmp")
env --chdir="$tmpdir" APT_CONFIG="$tmpfile" apt-get download --print-uris eatmydata libeatmydata1 \
| sed -ne "s/^'\([^']\+\)'\s\+\(\S\+\)\s\+\([0-9]\+\)\s\+\(SHA256:[a-f0-9]\+\)$/\1 \2 \3 \4/p" \
| while read -r uri fname size hash; do
| sed -ne "s/^'\([^']\+\)'\s\+\([^\s]\+\)\s\+\([0-9]\+\)\s\+\(SHA256:[a-f0-9]\+\)$/\1 \2 \3 \4/p" \
| while read uri fname size hash; do
echo "processing $fname" >&2
if [ -e "$tmpdir/$fname" ]; then
echo "$tmpdir/$fname already exists" >&2
@ -47,7 +41,7 @@ env --chdir="$tmpdir" APT_CONFIG="$tmpfile" apt-get download --print-uris eatmyd
| tar --directory="$rootdir/usr/bin" --strip-components=3 --extract --verbose ./usr/bin/eatmydata
;;
libeatmydata1_*_$chrootarch.deb)
libdir="/usr/lib/$(dpkg-architecture -a "$chrootarch" -q DEB_HOST_MULTIARCH)"
libdir="/usr/lib/$(dpkg-architecture -a $chrootarch -q DEB_HOST_MULTIARCH)"
mkdir -p "$rootdir$libdir"
dpkg-deb --fsys-tarfile "$tmpdir/$fname" \
| tar --directory="$rootdir$libdir" --strip-components=4 --extract --verbose --wildcards ".$libdir/libeatmydata.so*"

View file

@ -1,41 +0,0 @@
#!/bin/sh
#
# shellcheck disable=SC2086
set -eu
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
set -x
fi
rootdir="$1"
if [ ! -e "$rootdir/run/mmdebstrap/file-mirror-automount" ]; then
exit 0
fi
xargsopts="--null --no-run-if-empty -I {} --max-args=1"
case $MMDEBSTRAP_MODE in
root|unshare)
echo "unmounting the following mountpoints:" >&2 ;;
*)
echo "removing the following directories:" >&2 ;;
esac
< "$rootdir/run/mmdebstrap/file-mirror-automount" \
xargs $xargsopts echo " $rootdir/{}"
case $MMDEBSTRAP_MODE in
root|unshare)
< "$rootdir/run/mmdebstrap/file-mirror-automount" \
xargs $xargsopts umount "$rootdir/{}"
;;
*)
< "$rootdir/run/mmdebstrap/file-mirror-automount" \
xargs $xargsopts rm -r "$rootdir/{}"
;;
esac
rm "$rootdir/run/mmdebstrap/file-mirror-automount"
rmdir --ignore-fail-on-non-empty "$rootdir/run/mmdebstrap"

View file

@ -1,73 +0,0 @@
#!/bin/sh
set -eu
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
set -x
fi
rootdir="$1"
# process all configured apt repositories
env APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get indextargets --no-release-info --format '$(REPO_URI)' \
| sed -ne 's/^file:\/\+//p' \
| sort -u \
| while read -r path; do
mkdir -p "$rootdir/run/mmdebstrap"
if [ ! -d "/$path" ]; then
echo "W: /$path is not an existing directory" >&2
continue
fi
case $MMDEBSTRAP_MODE in
root|unshare)
echo "bind-mounting /$path into the chroot" >&2
mkdir -p "$rootdir/$path"
mount -o ro,bind "/$path" "$rootdir/$path"
;;
*)
echo "copying /$path into the chroot" >&2
mkdir -p "$rootdir/$path"
"$MMDEBSTRAP_ARGV0" --hook-helper "$rootdir" "$MMDEBSTRAP_MODE" "$MMDEBSTRAP_HOOK" env "$MMDEBSTRAP_VERBOSITY" sync-in "/$path" "/$path" <&"$MMDEBSTRAP_HOOKSOCK" >&"$MMDEBSTRAP_HOOKSOCK"
;;
esac
printf '/%s\0' "$path" >> "$rootdir/run/mmdebstrap/file-mirror-automount"
done
# process all files given via --include
set -f # turn off pathname expansion
IFS=',' # split by comma
for pkg in $MMDEBSTRAP_INCLUDE; do
set +f; unset IFS
case $pkg in
./*|../*|/*) : ;; # we are interested in this case
*) continue ;; # not a file
esac
# undo escaping
pkg="$(printf '%s' "$pkg" | sed 's/%2C/,/g; s/%25/%/g')"
# check for existance
if [ ! -f "$pkg" ]; then
echo "$pkg does not exist" >&2
continue
fi
# make path absolute
pkg="$(realpath "$pkg")"
case "$pkg" in
/*) : ;;
*) echo "path for $pkg is not absolute" >&2; continue;;
esac
mkdir -p "$rootdir/run/mmdebstrap"
mkdir -p "$rootdir/$(dirname "$pkg")"
case $MMDEBSTRAP_MODE in
root|unshare)
echo "bind-mounting $pkg into the chroot" >&2
touch "$rootdir/$pkg"
mount -o bind "$pkg" "$rootdir/$pkg"
;;
*)
echo "copying $pkg into the chroot" >&2
"$MMDEBSTRAP_ARGV0" --hook-helper "$rootdir" "$MMDEBSTRAP_MODE" "$MMDEBSTRAP_HOOK" env "$MMDEBSTRAP_VERBOSITY" upload "$pkg" "$pkg" <&"$MMDEBSTRAP_HOOKSOCK" >&"$MMDEBSTRAP_HOOKSOCK"
;;
esac
printf '/%s\0' "$pkg" >> "$rootdir/run/mmdebstrap/file-mirror-automount"
done
set +f; unset IFS

View file

@ -1,16 +0,0 @@
#!/bin/sh
set -eu
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
set -x
fi
TARGET="$1"
# not needed since dpkg 1.17.11
for f in available diversions cmethopt; do
if [ ! -e "$TARGET/var/lib/dpkg/$f" ]; then
touch "$TARGET/var/lib/dpkg/$f"
fi
done

View file

@ -1,47 +0,0 @@
#!/bin/sh
#
# needed until init 1.33 which pre-depends on systemd-sysv
# starting with init 1.34, init is not Essential:yes anymore
#
# jessie has init 1.22
set -eu
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
set -x
fi
TARGET="$1"
if [ -z "${MMDEBSTRAP_ESSENTIAL+x}" ]; then
MMDEBSTRAP_ESSENTIAL=
for f in "$TARGET/var/cache/apt/archives/"*.deb; do
[ -f "$f" ] || continue
f="${f#"$TARGET"}"
MMDEBSTRAP_ESSENTIAL="$MMDEBSTRAP_ESSENTIAL $f"
done
fi
fname_base_passwd=
fname_base_files=
fname_dpkg=
for pkg in $MMDEBSTRAP_ESSENTIAL; do
pkgname=$(dpkg-deb --show --showformat='${Package}' "$TARGET/$pkg")
# shellcheck disable=SC2034
case $pkgname in
base-passwd) fname_base_passwd=$pkg;;
base-files) fname_base_files=$pkg;;
dpkg) fname_dpkg=$pkg;;
esac
done
for var in base_passwd base_files dpkg; do
eval 'val=$fname_'"$var"
[ -z "$val" ] && continue
chroot "$TARGET" dpkg --install --force-depends "$val"
done
# shellcheck disable=SC2086
chroot "$TARGET" dpkg --unpack --force-depends $MMDEBSTRAP_ESSENTIAL
chroot "$TARGET" dpkg --configure --pending

View file

@ -1,37 +0,0 @@
#!/bin/sh
set -eu
# we need to check the version of dpkg
# since at this point packages are just extracted but not installed, we cannot use dpkg-query
# since we want to support chrootless, we cannot run dpkg --version inside the chroot
# to avoid this hook depending on dpkg-dev being installed, we do not parse the extracted changelog with dpkg-parsechangelog
# we also want to avoid parsing the changelog because /usr/share/doc might've been added to dpkg --path-exclude
# instead, we just ask apt about the latest version of dpkg it knows of
# this should only fail in situations where there are multiple versions of dpkg in different suites
ver=$(env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions dpkg 2>/dev/null | sed -ne 's/^Version: \(.*\)$/\1/p' || printf '')
if [ -z "$ver" ]; then
echo "no package called dpkg can be installed -- not running jessie-or-older extract00 hook" >&2
exit 0
fi
if dpkg --compare-versions "$ver" ge 1.17.11; then
echo "dpkg version $ver is >= 1.17.11 -- not running jessie-or-older extract00 hook" >&2
exit 0
else
echo "dpkg version $ver is << 1.17.11 -- running jessie-or-older extract00 hook" >&2
fi
# resolve the script path using several methods in order:
# 1. using dirname -- "$0"
# 2. using ./hooks
# 3. using /usr/share/mmdebstrap/hooks/
for p in "$(dirname -- "$0")/.." ./hooks /usr/share/mmdebstrap/hooks; do
if [ -x "$p/jessie-or-older/extract00.sh" ] && [ -x "$p/jessie-or-older/extract01.sh" ]; then
"$p/jessie-or-older/extract00.sh" "$1"
exit 0
fi
done
echo "cannot find jessie-or-older hook anywhere" >&2
exit 1

View file

@ -1,57 +0,0 @@
#!/bin/sh
set -eu
# The jessie-or-older extract01 hook has to be run up to the point where the
# Essential:yes field was removed from the init package (with
# init-system-helpers 1.34). Since the essential packages have only been
# extracted but not installed, we cannot use dpkg-query to find out its
# version. Since /usr/share/doc might be missing due to dpkg --path-exclude, we
# also cannot check whether /usr/share/doc/init/copyright exists. There also
# was a time (before init-system-helpers 1.20) where there was no init package
# at all where we also want to apply this hook. So we just ask apt about the
# candidate version for init-system-helpers. This should only fail in
# situations where there are multiple versions of init-system-helpers in
# different suites.
ver=$(env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions init-system-helpers 2>/dev/null | sed -ne 's/^Version: \(.*\)$/\1/p' || printf '')
if [ -z "$ver" ]; then
# there is no package called init-system-helpers, so either:
# - this is so old that init-system-helpers didn't exist yet
# - we are in a future where init-system-helpers doesn't exist anymore
# - something strange is going on
# we should only call the hook in the first case
ver=$(env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions base-files 2>/dev/null | sed -ne 's/^Version: \(.*\)$/\1/p' || printf '')
if [ -z "$ver" ]; then
echo "neither init-system-helpers nor base-files can be installed -- not running jessie-or-older extract01 hook" >&2
exit 0
fi
# Jessie is Debian 8
if dpkg --compare-versions "$ver" ge 8; then
echo "there is no init-system-helpers but base-files version $ver is >= 8 -- not running jessie-or-older extract01 hook" >&2
exit 0
else
echo "there is no init-system-helpers but base-files version $ver is << 8 -- running jessie-or-older extract01 hook" >&2
fi
else
if dpkg --compare-versions "$ver" ge 1.34; then
echo "init-system-helpers version $ver is >= 1.34 -- not running jessie-or-older extract01 hook" >&2
exit 0
else
echo "init-system-helpers version $ver is << 1.34 -- running jessie-or-older extract01 hook" >&2
fi
fi
# resolve the script path using several methods in order:
# 1. using dirname -- "$0"
# 2. using ./hooks
# 3. using /usr/share/mmdebstrap/hooks/
for p in "$(dirname -- "$0")/.." ./hooks /usr/share/mmdebstrap/hooks; do
if [ -x "$p/jessie-or-older/extract00.sh" ] && [ -x "$p/jessie-or-older/extract01.sh" ]; then
"$p/jessie-or-older/extract01.sh" "$1"
exit 0
fi
done
echo "cannot find jessie-or-older hook anywhere" >&2
exit 1

View file

@ -1,40 +0,0 @@
#!/bin/sh
set -eu
ver=$(dpkg-query --root="$1" -f '${db:Status-Status} ${Source} ${Version}' --show usr-is-merged 2>/dev/null || printf '')
case "$ver" in
'')
echo "no package called usr-is-merged is installed -- not running merged-usr essential hook" >&2
exit 0
;;
'installed mmdebstrap-dummy-usr-is-merged 1')
echo "dummy usr-is-merged package installed -- running merged-usr essential hook" >&2
;;
'installed usrmerge '*)
echo "usr-is-merged package from src:usrmerge installed -- not running merged-usr essential hook" >&2
exit 0
;;
'not-installed ')
echo "usr-is-merged was not installed in a previous hook -- not running merged-usr essential hook" >&2
exit 0
;;
*)
echo "unexpected situation for package usr-is-merged: $ver" >&2
exit 1
;;
esac
# resolve the script path using several methods in order:
# 1. using dirname -- "$0"
# 2. using ./hooks
# 3. using /usr/share/mmdebstrap/hooks/
for p in "$(dirname -- "$0")/.." ./hooks /usr/share/mmdebstrap/hooks; do
if [ -x "$p/merged-usr/setup00.sh" ] && [ -x "$p/merged-usr/extract00.sh" ] && [ -x "$p/merged-usr/essential00.sh" ]; then
"$p/merged-usr/essential00.sh" "$1"
exit 0
fi
done
echo "cannot find merged-usr hook anywhere" >&2
exit 1

View file

@ -1,37 +0,0 @@
#!/bin/sh
set -eu
env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get update --error-on=any
if env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged > /dev/null 2>&1; then
# if apt-cache exited successfully, then usr-is-merged exists either as
# a real or virtual package
if env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged 2>/dev/null | grep -q "Package: usr-is-merged"; then
echo "usr-is-merged found -- running merged-usr extract hook" >&2
else
# The usr-is-merged must be virtual, so assume that nothing
# has to be done. This is the case with Debian Trixie or later
# or with Ubuntu Lunar or later
echo "usr-is-merged found but not real -- not running merged-usr extract hook" >&2
exit 0
fi
else
# if the usr-is-merged package cannot be installed with apt, do nothing
echo "no package providing usr-is-merged found -- not running merged-usr extract hook" >&2
exit 0
fi
# resolve the script path using several methods in order:
# 1. using dirname -- "$0"
# 2. using ./hooks
# 3. using /usr/share/mmdebstrap/hooks/
for p in "$(dirname -- "$0")/.." ./hooks /usr/share/mmdebstrap/hooks; do
if [ -x "$p/merged-usr/setup00.sh" ] && [ -x "$p/merged-usr/extract00.sh" ] && [ -x "$p/merged-usr/essential00.sh" ]; then
"$p/merged-usr/extract00.sh" "$1"
exit 0
fi
done
echo "cannot find merged-usr hook anywhere" >&2
exit 1

View file

@ -1,37 +0,0 @@
#!/bin/sh
set -eu
env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get update --error-on=any
if env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged > /dev/null 2>&1; then
# if apt-cache exited successfully, then usr-is-merged exists either as
# a real or virtual package
if env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged 2>/dev/null | grep -q "Package: usr-is-merged"; then
echo "usr-is-merged found -- running merged-usr setup hook" >&2
else
# The usr-is-merged must be virtual, so assume that nothing
# has to be done. This is the case with Debian Trixie or later
# or with Ubuntu Lunar or later
echo "usr-is-merged found but not real -- not running merged-usr setup hook" >&2
exit 0
fi
else
# if the usr-is-merged package cannot be installed with apt, do nothing
echo "no package providing usr-is-merged found -- not running merged-usr setup hook" >&2
exit 0
fi
# resolve the script path using several methods in order:
# 1. using dirname -- "$0"
# 2. using ./hooks
# 3. using /usr/share/mmdebstrap/hooks/
for p in "$(dirname -- "$0")/.." ./hooks /usr/share/mmdebstrap/hooks; do
if [ -x "$p/merged-usr/setup00.sh" ] && [ -x "$p/merged-usr/extract00.sh" ] && [ -x "$p/merged-usr/essential00.sh" ]; then
"$p/merged-usr/setup00.sh" "$1"
exit 0
fi
done
echo "cannot find merged-usr hook anywhere" >&2
exit 1

View file

@ -1,28 +0,0 @@
#!/bin/sh
set -eu
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
set -x
fi
TARGET="$1"
if [ "${MMDEBSTRAP_MODE:-}" = "chrootless" ]; then
APT_CONFIG=$MMDEBSTRAP_APT_CONFIG apt-get --yes install \
-oDPkg::Chroot-Directory= \
-oDPkg::Options::=--force-not-root \
-oDPkg::Options::=--force-script-chrootless \
-oDPkg::Options::=--root="$TARGET" \
-oDPkg::Options::=--log="$TARGET/var/log/dpkg.log" \
usr-is-merged
export DPKG_ROOT="$TARGET"
dpkg-query --showformat '${db:Status-Status}\n' --show usr-is-merged | grep -q '^installed$'
dpkg-query --showformat '${Source}\n' --show usr-is-merged | grep -q '^usrmerge$'
dpkg --compare-versions "1" "lt" "$(dpkg-query --showformat '${Version}\n' --show usr-is-merged)"
else
APT_CONFIG=$MMDEBSTRAP_APT_CONFIG apt-get --yes install usr-is-merged
chroot "$TARGET" dpkg-query --showformat '${db:Status-Status}\n' --show usr-is-merged | grep -q '^installed$'
chroot "$TARGET" dpkg-query --showformat '${Source}\n' --show usr-is-merged | grep -q '^usrmerge$'
dpkg --compare-versions "1" "lt" "$(chroot "$TARGET" dpkg-query --showformat '${Version}\n' --show usr-is-merged)"
fi

View file

@ -1,85 +0,0 @@
#!/bin/sh
set -eu
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
set -x
fi
TARGET="$1"
# can_usrmerge_symlink() and can_usrmerge_symlink() are
# Copyright 2023 Helmut Grohne <helmut@subdivi.de>
# and part of the debootstrap source in /usr/share/debootstrap/functions
# https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/96
# https://bugs.debian.org/104989
can_usrmerge_symlink() {
# Absolute symlinks can be relocated without problems.
test "${2#/}" = "$2" || return 0
while :; do
if test "${2#/}" != "$2"; then
# Handle double-slashes.
set -- "$1" "${2#/}"
elif test "${2#./}" != "$2"; then
# Handle ./ inside a link target.
set -- "$1" "${2#./}"
elif test "$2" = ..; then
# A parent directory symlink is ok if it does not
# cross the top level directory.
test "${1%/*/*}" != "$1" -a -n "${1%/*/*}"
return $?
elif test "${2#../}" != "$2"; then
# Symbolic link crossing / cannot be moved safely.
# This is prohibited by Debian Policy 10.5.
test "${1%/*/*}" = "$1" -o -z "${1%/*/*}" && return 1
set -- "${1%/*}" "${2#../}"
else
# Consider the symlink ok if its target does not
# contain a parent directory. When we fail here,
# the link target is non-minimal and doesn't happen
# in the archive.
test "${2#*/../}" = "$2"
return $?
fi
done
}
merge_usr_entry() {
# shellcheck disable=SC3043
local entry canon
canon="$TARGET/usr/${1#"$TARGET/"}"
test -h "$canon" &&
error 1 USRMERGEFAIL "cannot move %s as its destination exists as a symlink" "${1#"$TARGET"}"
if ! test -e "$canon"; then
mv "$1" "$canon"
return 0
fi
test -d "$1" ||
error 1 USRMERGEFAIL "cannot move non-directory %s as its destination exists" "${1#"$TARGET"}"
test -d "$canon" ||
error 1 USRMERGEFAIL "cannot move directory %s as its destination is not a directory" "${1#"$TARGET"}"
for entry in "$1/"* "$1/."*; do
# Some shells return . and .. on dot globs.
test "${entry%/.}" != "${entry%/..}" && continue
if test -h "$entry" && ! can_usrmerge_symlink "${entry#"$TARGET"}" "$(readlink "$entry")"; then
error 1 USRMERGEFAIL "cannot move relative symlink crossing top-level directory" "${entry#"$TARGET"}"
fi
# Ignore glob match failures
if test "${entry%'/*'}" != "${entry%'/.*'}" && ! test -e "$entry"; then
continue
fi
merge_usr_entry "$entry"
done
rmdir "$1"
}
# This is list includes all possible multilib directories. It must be
# updated when new multilib directories are being added. Hopefully,
# all new architectures use multiarch instead, so we never get to
# update this.
for dir in bin lib lib32 lib64 libo32 libx32 sbin; do
test -h "$TARGET/$dir" && continue
test -e "$TARGET/$dir" || continue
merge_usr_entry "$TARGET/$dir"
ln -s "usr/$dir" "$TARGET/$dir"
done

View file

@ -39,41 +39,43 @@
# out merged-/usr is bad from the dpkg point-of-view and completely opposite of
# the vision with which in mind I wrote mmdebstrap.
set -eu
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
set -x
fi
set -exu
TARGET="$1"
# now install an empty "usr-is-merged" package to avoid installing the
# usrmerge package on this system even after init-system-helpers starts
# depending on "usrmerge | usr-is-merged".
#
# This package will not end up in the final chroot because the essential
# hook replaces it with the actual usr-is-merged package from src:usrmerge.
if [ -e "$TARGET/var/lib/dpkg/arch" ]; then
ARCH=$(head -1 "$TARGET/var/lib/dpkg/arch")
else
ARCH=$(dpkg --print-architecture)
fi
tmpdir=$(mktemp --directory --tmpdir="$TARGET/tmp")
mkdir -p "$tmpdir/usr-is-merged/DEBIAN"
if [ -e /usr/share/debootstrap/functions ]; then
. /usr/share/debootstrap/functions
doing_variant () { [ $1 != "buildd" ]; }
MERGED_USR="yes"
# until https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/48 gets merged
link_dir=""
setup_merged_usr
else
link_dir=""
case $ARCH in
hurd-*) exit 0;;
amd64) link_dir="lib32 lib64 libx32" ;;
i386) link_dir="lib64 libx32" ;;
mips|mipsel) link_dir="lib32 lib64" ;;
mips64*|mipsn32*) link_dir="lib32 lib64 libo32" ;;
powerpc) link_dir="lib64" ;;
ppc64) link_dir="lib32 lib64" ;;
ppc64el) link_dir="lib64" ;;
s390x) link_dir="lib32" ;;
sparc) link_dir="lib64" ;;
sparc64) link_dir="lib32 lib64" ;;
x32) link_dir="lib32 lib64 libx32" ;;
esac
link_dir="bin sbin lib $link_dir"
cat << END > "$tmpdir/usr-is-merged/DEBIAN/control"
Package: usr-is-merged
Priority: optional
Section: oldlibs
Maintainer: Johannes Schauer Marin Rodrigues <josch@debian.org>
Architecture: all
Multi-Arch: foreign
Source: mmdebstrap-dummy-usr-is-merged
Version: 1
Description: dummy package created by mmdebstrap merged-usr setup hook
This package was generated and installed by the mmdebstrap merged-usr
setup hook at /usr/share/mmdebstrap/hooks/merged-usr.
.
If this package is installed in the final chroot, then this is a bug
in mmdebstrap. Please report: https://gitlab.mister-muffin.de/josch/mmdebstrap
END
dpkg-deb --build "$tmpdir/usr-is-merged" "$tmpdir/usr-is-merged.deb"
dpkg --root="$TARGET" --log="$TARGET/var/log/dpkg.log" --install "$tmpdir/usr-is-merged.deb"
rm "$tmpdir/usr-is-merged.deb" "$tmpdir/usr-is-merged/DEBIAN/control"
rmdir "$tmpdir/usr-is-merged/DEBIAN" "$tmpdir/usr-is-merged" "$tmpdir"
for dir in $link_dir; do
ln -s usr/"$dir" "$TARGET/$dir"
mkdir -p "$TARGET/usr/$dir"
done
fi

View file

@ -1 +0,0 @@
../merged-usr/essential00.sh

View file

@ -1,54 +0,0 @@
#!/bin/sh
#
# mmdebstrap does have a --no-merged-usr option but only as a no-op for
# debootstrap compatibility
#
# Using this hook script, you can emulate what debootstrap does to set up
# a system without merged-/usr even after the essential init-system-helpers
# package added a dependency on "usrmerge | usr-is-merged". By installing
# a dummy usr-is-merged package, it avoids pulling in the dependencies of
# the usrmerge package.
set -eu
if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then
set -x
fi
TARGET="$1"
echo "Warning: starting with Debian 12 (Bookworm), systems without merged-/usr are not supported anymore" >&2
echo "Warning: starting with Debian 13 (Trixie), merged-/usr symlinks are shipped by packages in the essential-set making this hook ineffective" >&2
echo "this system will not be supported in the future" > "$TARGET/etc/unsupported-skip-usrmerge-conversion"
# now install an empty "usr-is-merged" package to avoid installing the
# usrmerge package on this system even after init-system-helpers starts
# depending on "usrmerge | usr-is-merged".
#
# This package will not end up in the final chroot because the essential
# hook replaces it with the actual usr-is-merged package from src:usrmerge.
tmpdir=$(mktemp --directory --tmpdir="$TARGET/tmp")
mkdir -p "$tmpdir/usr-is-merged/DEBIAN"
cat << END > "$tmpdir/usr-is-merged/DEBIAN/control"
Package: usr-is-merged
Priority: optional
Section: oldlibs
Maintainer: Johannes Schauer Marin Rodrigues <josch@debian.org>
Architecture: all
Multi-Arch: foreign
Source: mmdebstrap-dummy-usr-is-merged
Version: 1
Description: dummy package created by mmdebstrap no-merged-usr setup hook
This package was generated and installed by the mmdebstrap no-merged-usr
setup hook at /usr/share/mmdebstrap/hooks/no-merged-usr.
.
If this package is installed in the final chroot, then this is a bug
in mmdebstrap. Please report: https://gitlab.mister-muffin.de/josch/mmdebstrap
END
dpkg-deb --build "$tmpdir/usr-is-merged" "$tmpdir/usr-is-merged.deb"
dpkg --root="$TARGET" --log="$TARGET/var/log/dpkg.log" --install "$tmpdir/usr-is-merged.deb"
rm "$tmpdir/usr-is-merged.deb" "$tmpdir/usr-is-merged/DEBIAN/control"
rmdir "$tmpdir/usr-is-merged/DEBIAN" "$tmpdir/usr-is-merged" "$tmpdir"

View file

@ -106,24 +106,10 @@ def main():
for d in get_libdirs(chroot, [chroot / "etc" / "ld.so.conf"]):
make_relative(d)
rootarg = chroot
argv = sys.argv[1:]
for arg in sys.argv[1:]:
if arg == "-r":
rootarg = None
elif rootarg is None:
argpath = Path(arg)
if argpath.is_absolute():
rootarg = chroot / argpath.relative_to("/")
else:
rootarg = Path.cwd() / argpath
if rootarg is None:
rootarg = chroot
# we add any additional arguments before "-r" such that any other "-r"
# option will be overwritten by the one we set
subprocess.check_call(
[chroot / "sbin" / "ldconfig"] + sys.argv[1:] + ["-r", rootarg]
[chroot / "sbin" / "ldconfig"] + sys.argv[1:] + ["-r", chroot]
)

File diff suppressed because it is too large Load diff

4919
mmdebstrap

File diff suppressed because it is too large Load diff

View file

@ -1,460 +1,255 @@
#!/bin/sh
# Copyright 2023 Johannes Schauer Marin Rodrigues <josch@debian.org>
# Copyright 2023 Helmut Grohne <helmut@subdivi.de>
# SPDX-License-Identifier: MIT
# We generally use single quotes to avoid variable expansion:
# shellcheck disable=SC2016
# Replacement for autopkgtest-build-qemu and vmdb2 for all architectures
# supporting EFI booting (amd64, arm64, armhf, i386, riscv64).
# For use as replacement for autopkgtest-build-qemu and vmdb2 on ppc64el which
# neither supports extlinux nor efi booting there is an unmaintained script
# which uses grub instead to boot:
#
# https://gitlab.mister-muffin.de/josch/mmdebstrap/src/commit/
# e523741610a4ed8579642bfc755956f64c847ef3/mmdebstrap-autopkgtest-build-qemu
: <<'POD2MAN'
=head1 NAME
mmdebstrap-autopkgtest-build-qemu - autopkgtest-build-qemu without vmdb2 but mmdebstrap and EFI boot
=head1 SYNOPSIS
B<mmdebstrap-autopkgtest-build-qemu> [I<OPTIONS>] B<--boot>=B<efi> I<RELEASE> I<IMAGE>
=head1 DESCRIPTION
B<mmdebstrap-autopkgtest-build-qemu> is a mostly compatible drop-in replacement
for L<autopkgtest-build-qemu(1)> with two main differences: Firstly, it uses
L<mmdebstrap(1)> instead of L<vmdb2(1)> and thus is able to create QEMU disk
images without requiring superuser privileges and with bit-by-bit reproducible
output. Secondly, it uses L<systemd-boot(7)> and thus only supports booting via
EFI. For architectures for which L<autopkgtest-virt-qemu(1)> does not default
to EFI booting you must pass B<--boot=efi> when invoking the autopkgtest virt
backend.
=head1 POSITIONAL PARAMETERS
=over 8
=item I<RELEASE>
The release to download from the I<MIRROR>. This parameter is required.
=item I<IMAGE>
The file to write, in raw format. This parameter is required.
=back
=head1 OPTIONS
=over 8
=item B<--mirror>=I<MIRROR>
Specify which distribution to install. It defaults to
http://deb.debian.org/debian (i.e. Debian), but you can pass a mirror of any
Debian derivative.
=item B<--architecture>=I<ARCHITECTURE>
Set the architecture for the virtual machine image, specified as a L<dpkg(1)>
architecture. If omitted, the host architecture is assumed.
B<--arch>=I<ARCH> is an alias for this option.
=item B<--script>=I<SCRIPT>
Specifies a user script that will be called with the root filesystem of the
image as its first parameter. This script can them make any necesssary
modifications to the root filesystem.
The script must be a POSIX shell script, and should not depend on bash-specific
features. This script will be executed inside a L<chroot(1)> call in the
virtual machine root filesystem.
=item B<--size>=I<SIZE>
Specifies the image size for the virtual machine, defaulting to 25G.
=item B<--apt-proxy>=I<PROXY>
Specify an apt proxy to use in the virtual machine. By default, if you have
an apt proxy configured on the host, the virtual machine will automatically use
this, otherwise there is no default.
=item B<--boot>=B<efi>, B<--efi>
Select the way the generated image will expect to be booted. Unless you
explicitly select --boot=efi, operation will fail.
=item B<--keyring>=I<KEYRING>
Passes an additional B<--keyring> parameter to B<mmdebstrap>.
=back
=head1 EXAMPLES
Make sure, that F</path/to/debian-unstable.img> is a path that the unshared
user has access to. This can be done by ensuring world-execute permissions on
all path components or by creating the image in a world-readable directory like
/tmp before copying it into its final location.
$ mmdebstrap-autopkgtest-build-qemu --boot=efi --arch=amd64 unstable /path/to/debian-unstable.img
[...]
$ autopkgtest mypackage -- qemu --boot=efi --dpkg-architecture=amd64 /path/to/debian-unstable.img
Make sure to add B<--boot=efi> to both the B<mmdebstrap-autopkgtest-build-qemu>
as well as the B<autopkgtest-virt-qemu> invocation.
Create bit-by-bit reproducible images from a given snapshot.d.o timestamp.
SOURCE_DATE_EPOCH=1612543740 mmdebstrap-autopkgtest-build-qemu --boot=efi \
--mirror=http://snapshot.debian.org/archive/debian/20210205T164900Z/ \
unstable /path/to/debian-unstable.img
=head1 SEE ALSO
L<autopkgtest-build-qemu(1)>, L<autopkgtest-virt-qemu(1)>, L<mmdebstrap(1)>, L<autopkgtest(1)>
=cut
POD2MAN
# © 2022 Johannes Schauer Marin Rodrigues <josch@mister-muffin.de>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to
# deal in the Software without restriction, including without limitation the
# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
# sell copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# The software is provided "as is", without warranty of any kind, express or
# implied, including but not limited to the warranties of merchantability,
# fitness for a particular purpose and noninfringement. In no event shall the
# authors or copyright holders be liable for any claim, damages or other
# liability, whether in an action of contract, tort or otherwise, arising
# from, out of or in connection with the software or the use or other dealings
# in the software.
set -eu
die() {
echo "$*" 1>&2
exit 1
}
# This script creates debian-$RELEASE.qcow2 in the current directory which can
# then be used by the autopkgtest qemu backend.
#
# Thanks to Francesco Poli for providing ideas and testing this.
#
# Thanks to Lars Wirzenius of vmdb2 where the grub and efi magic comes from.
#
# Only the native architecture is supported because guestfish doesn't support
# foreign architectures.
usage() {
die "usage: $0 [--architecture=|--apt-proxy=|--keyring=|--mirror=|--script=|--size=] --boot=efi <RELEASE> <IMAGE>"
}
usage_error() {
echo "error: $*" 1>&2
usage
echo "Usage: $0 [--size=SIZE] [--boot=BOOT] RELEASE IMAGE" >&2
echo >&2
echo "RELEASE is a Debian release like unstable" >&2
echo "IMAGE will be stored in qcow2 format" >&2
echo "SIZE is 25G by default" >&2
echo "BOOT is either auto (the default), bios, efi or ieee1275" >&2
}
BOOT=auto
ARCHITECTURE=$(dpkg --print-architecture)
IMAGE=
MIRROR=
KEYRING=
RELEASE=
SIZE=25G
SCRIPT=
nativearch="$(dpkg --print-architecture)"
# consumed by setup-testbed
export AUTOPKGTEST_BUILD_QEMU=1
opt_boot() {
BOOT="$1"
}
opt_architecture() {
ARCHITECTURE="$1"
}
opt_arch() {
ARCHITECTURE="$1"
}
opt_apt_proxy() {
# consumed by setup-testbed
export AUTOPKGTEST_APT_PROXY="$1"
# consumed by mmdebstrap
if test "$1" = DIRECT; then
unset http_proxy
else
export http_proxy="$1"
fi
}
opt_keyring() {
KEYRING="$1"
}
opt_mirror() {
# consumed by setup-testbed
export MIRROR="$1"
}
opt_script() {
test -f "$1" || die "passed script '$1' does not refer to a file"
SCRIPT="$1"
}
opt_size() {
SIZE="$1"
}
positional=1
positional_1() {
# consumed by setup-testbed
export RELEASE="$1"
}
positional_2() {
IMAGE="$1"
}
positional_3() { opt_mirror "$@"; }
positional_4() { opt_architecture "$@"; }
positional_5() { opt_script "$@"; }
positional_6() { opt_size "$@"; }
positional_7() {
die "too many positional options"
}
while test "$#" -gt 0; do
case "$1" in
--architecture=* | --arch=* | --boot=* | --keyring=* | --mirror=* | --script=* | --size=*)
optname="${1%%=*}"
"opt_${optname#--}" "${1#*=}"
;;
--apt-proxy=*)
opt_apt_proxy "${1#*=}"
;;
--architecture | --arch | --boot | --keyring | --mirror | --script | --size)
test "$#" -ge 2 || usage_error "missing argument for $1"
"opt_${1#--}" "$2"
shift
;;
--apt-proxy)
test "$#" -ge 2 || usage_error "missing argument for $1"
opt_apt_proxy "$2"
shift
;;
--efi)
opt_boot efi
;;
--*)
usage_error "unrecognized argument $1"
;;
*)
"positional_$positional" "$1"
positional=$((positional + 1))
;;
esac
shift
done
test -z "$RELEASE" -o -z "$IMAGE" && usage_error "missing positional arguments"
test "$BOOT" = efi \
|| die "this tool does not support boot modes other than efi"
case "$ARCHITECTURE" in
amd64)
EFIIMG=bootx64.efi
QEMUARCH=x86_64
VMFPKG=ovmf
LINUXIMAGE=linux-image-amd64
;;
arm64)
EFIIMG=bootaa64.efi
QEMUARCH=aarch64
VMFPKG=qemu-efi-aarch64
LINUXIMAGE=linux-image-arm64
;;
armhf)
EFIIMG=bootarm.efi
QEMUARCH=arm
VMFPKG=qemu-efi-arm
LINUXIMAGE=linux-image-armmp
;;
i386)
EFIIMG=bootia32.efi
QEMUARCH=i386
VMFPKG=ovmf-ia32
LINUXIMAGE=linux-image-686-pae
;;
riscv64)
EFIIMG=bootriscv64.efi
QEMUARCH=riscv64
VMFPKG=
LINUXIMAGE=linux-image-riscv64
;;
*)
die "unsupported architecture: $ARCHITECTURE"
;;
esac
if test "$(dpkg-query -f '${db:Status-Status}' -W binutils-multiarch)" = installed; then
GNU_PREFIX=
BINUTILS=
SIZE="25G" # default from autopkgtest-build-qemu
BOOT="auto"
if [ "$#" -lt 2 ]; then
echo "Error: Insufficient number of arguments" >&2
usage
exit 1
elif [ "$#" -eq 2 ]; then
RELEASE=$1
IMAGE=$2
else
GNU_ARCHITECTURE="$(dpkg-architecture "-a$ARCHITECTURE" -qDEB_HOST_GNU_TYPE)"
GNU_PREFIX="$GNU_ARCHITECTURE-"
GNU_SUFFIX="-$(echo "$GNU_ARCHITECTURE" | tr _ -)"
BINUTILS=", binutils$GNU_SUFFIX | binutils-multiarch"
# parse options
OPTS=$(getopt -n "$0" -o h --long size:,boot:,architecture:,help -- "$@")
if [ "$?" -ne 0 ]; then
echo "Error: Cannot parse arguments" >&2
usage
exit 1
fi
eval set -- "$OPTS"
while true; do
case "$1" in
--size) SIZE="$2"; shift 2; continue;;
--boot) BOOT="$2"; shift 2; continue;;
--help) usage; exit 1;;
--architecture)
echo "Error: cannot (yet) create foreign architecture images" >&2
exit 1
;;
--) shift; break;;
*)
echo "Error: unknown option $1" >&2
usage
exit 1
;;
esac
done
RELEASE=$1
IMAGE=$2
fi
arches=" $(dpkg --print-architecture) $(dpkg --print-foreign-architectures | tr '\n' ' ') "
case $arches in
*" $ARCHITECTURE "*) : ;; # nothing to do
*) die "enable $ARCHITECTURE by running: sudo dpkg --add-architecture $ARCHITECTURE && sudo apt update" ;;
# By default with --boot=auto (the default), bios boot is chosen for
# amd64 and i386. Compare /usr/share/autopkgtest/lib/autopkgtest_qemu.py
# But in practice, amd64 and i386 also support efi boot. But then
# autopkgtest-virt-qemu has to be run with --boot=efi
case "$BOOT" in
auto)
case "$nativearch" in
amd64|i386) BOOT=bios;;
armhf|arm64) BOOT=efi;;
ppc64el) BOOT=ieee1275;;
esac
;;
bios)
case "$nativearch" in amd64|i386);;
*)
echo "bios booting only possible on amd64 and i386" >&2
exit 1
;;
esac
;;
efi)
case "$nativearch" in amd64|i386|armhf|arm64);;
*)
echo "efi booting only possible on amd64, i386, armhf and arm64" >&2
exit 1
;;
esac
;;
ieee1275)
if [ "$nativearch" != "ppc64el" ]; then
echo "ieee1275 booting only possible on ppc64el" >&2
exit 1
fi
;;
*)
echo "invalid value for --boot" >&2;;
esac
test "$(dpkg-query -f '${db:Status-Status}' -W "dpkg-dev")" = installed \
|| die "please install dpkg-dev"
dpkg-checkbuilddeps -d "autopkgtest, dosfstools, e2fsprogs, fdisk, mount, mtools, passwd, uidmap, libarchive13, systemd-boot-efi:$ARCHITECTURE $BINUTILS" /dev/null \
|| die "please install the required packages listed above"
BOOTSTUB="/usr/lib/systemd/boot/efi/linux${EFIIMG#boot}.stub"
WORKDIR=
cleanup() {
test -n "$WORKDIR" && rm -Rf "$WORKDIR"
}
trap cleanup EXIT INT TERM QUIT
WORKDIR=$(mktemp -d)
FAT_OFFSET_SECTORS=$((1024 * 2))
FAT_SIZE_SECTORS=$((1024 * 254))
# The image is raw and not in qcow2 format because:
# - faster run-time as the "qemu-image convert" step is not needed
# - image can be used independent of qemu tooling
# - modifying the image just with "mount" instead of requiring qemu-nbd
# - sparse images make the file just as small as with qcow2
# - trim support is more difficult on qcow2
# - snapshots and overlays work just as well with raw images
# - users who prefer qcow2 get to choose to run it themselves with their own
# custom options like compression
set -- \
--mode=unshare \
--format=tar \
--variant=important \
--architecture="$ARCHITECTURE"
case $MIRROR in http://snapshot.debian.org/archive/* | https://snapshot.debian.org/archive/*)
set -- "$@" --aptopt='Acquire::Check-Valid-Until "false"'
;;
case "$nativearch" in
amd64)
[ $BOOT = bios ] || [ $BOOT = efi ]
if [ $BOOT = bios ]; then
include="linux-image-amd64 grub-pc"
grub_target="i386-pc"
elif [ $BOOT = efi ]; then
include="linux-image-amd64 grub-efi"
grub_target="x86_64-efi"
fi
;;
arm64)
[ $BOOT = efi ]
include="linux-image-arm64 grub-efi"
grub_target="arm64-efi"
;;
armhf)
[ $BOOT = efi ]
include="linux-image-armmp-lpae grub-efi"
grub_target="arm-efi"
;;
i386)
[ $BOOT = bios ] || [ $BOOT = efi ]
if [ $BOOT = bios ]; then
include="linux-image-686-pae grub-pc"
grub_target="i386-efi"
elif [ $BOOT = efi ]; then
include="linux-image-686-pae grub-efi"
grub_target="i386-efi"
fi
;;
ppc64el)
[ $BOOT = ieee1275 ]
include="linux-image-powerpc64le grub-ieee1275"
grub_target="powerpc-ieee1275"
;;
*)
echo "architecture $nativearch not yet supported" >&2
exit 1
;;
esac
EXT_FEATURES=
if test "$RELEASE" = jessie; then
set -- "$@" --keyring=/usr/share/keyrings/debian-archive-removed-keys.gpg
set -- "$@" --aptopt='Apt::Key::gpgvcommand "/usr/libexec/mmdebstrap/gpgvnoexpkeysig"'
set -- "$@" --hook-dir=/usr/share/mmdebstrap/hooks/jessie-or-older
EXT_FEATURES="^metadata_csum,^metadata_csum_seed,^orphan_file"
case "$nativearch" in
arm64|armhf) serial="loglevel=3 console=tty0 console=ttyAMA0,115200n8" ;;
ppc64el) serial="loglevel=3 console=tty0 console=hvc0,115200n8" ;;
*) serial="loglevel=3 console=tty0 console=ttyS0,115200n8" ;;
esac
if ! command -v guestfish >/dev/null; then
echo "Error: requires guestfish being installed" >&2
exit 1
fi
set -- "$@" \
"--include=init,$LINUXIMAGE,python3" \
'--customize-hook=echo host >"$1/etc/hostname"' \
'--customize-hook=echo 127.0.0.1 localhost host >"$1/etc/hosts"' \
'--customize-hook=passwd --root "$1" --delete root' \
'--customize-hook=useradd --root "$1" --home-dir /home/user --create-home user' \
'--customize-hook=passwd --root "$1" --delete user' \
'--customize-hook=/usr/share/autopkgtest/setup-commands/setup-testbed'
if test -n "$SCRIPT"; then
set -- "$@" \
"--customize-hook=upload '$SCRIPT' /userscript" \
"--chrooted-customize-hook=sh /userscript" \
'--customize-hook=rm -f "$1/userscript"'
if [ ! -e /usr/share/autopkgtest/setup-commands/setup-testbed ]; then
echo "Error: requires autopkgtest being installed" >&2
exit 1
fi
set -- "$@" \
"--customize-hook=download vmlinuz '$WORKDIR/kernel'" \
"--customize-hook=download initrd.img '$WORKDIR/initrd'" \
"$RELEASE" \
-
test -n "$MIRROR" && set -- "$@" "$MIRROR"
test -n "$KEYRING" && set -- "$@" "--keyring=$KEYRING"
echo "+ mmdebstrap $*" >&2
# https://github.com/koalaman/shellcheck/issues/2555
# shellcheck disable=SC3040
set -o pipefail
mmdebstrap "$@" | {
set -- -t ext4 -L autopkgtestvm -d -
if test -n "$EXT_FEATURES"; then
set -- "$@" -O "$EXT_FEATURES"
fi
EXTOPTS="offset=$(((FAT_OFFSET_SECTORS + FAT_SIZE_SECTORS) * 512))"
if test -n "${SOURCE_DATE_EPOCH-}"; then
uuid="$(uuidgen --sha1 --namespace="$(uuidgen --sha1 --namespace='@dns' --name mister-muffin.de)" --name "$SOURCE_DATE_EPOCH")"
set -- "$@" -U "$uuid"
EXTOPTS="$EXTOPTS,hash_seed=$uuid"
fi
set -- "$@" -E "$EXTOPTS" "$IMAGE" "$SIZE"
echo "+ mke2fs $*" >&2
/sbin/mke2fs "$@"
run_mmdebstrap() {
mmdebstrap --variant=important --include="$include" \
--customize-hook='chroot "$1" passwd --delete root' \
--customize-hook='chroot "$1" useradd --home-dir /home/user --create-home user' \
--customize-hook='chroot "$1" passwd --delete user' \
--customize-hook='echo host > "$1/etc/hostname"' \
--customize-hook='echo "127.0.0.1 localhost host" > "$1/etc/hosts"' \
--customize-hook='env AUTOPKGTEST_BUILD_QEMU=1 /usr/share/autopkgtest/setup-commands/setup-testbed "$1"' \
"$RELEASE" -
}
echo "root=LABEL=autopkgtestvm rw console=ttyS0" >"$WORKDIR/cmdline"
align_size() {
echo "$((($1) + ($2) - 1 - (($1) + ($2) - 1) % ($2)))"
guestfish_bios() {
guestfish -- \
disk-create "$IMAGE" qcow2 "$SIZE" : \
add-drive "$IMAGE" format:qcow2 : \
launch : \
part-disk /dev/sda mbr : \
part-set-bootable /dev/sda 1 true : \
mkfs ext4 /dev/sda1 : mount /dev/sda1 / : \
tar-in - / xattrs:true : \
command "sh -c 'echo UUID=\$(blkid -c /dev/null -o value -s UUID /dev/sda1) / ext4 errors=remount-ro 0 1 > /etc/fstab'" : \
command "update-initramfs -u" : \
command "grub-mkconfig -o /boot/grub/grub.cfg" : \
command "grub-install /dev/sda --target=$grub_target --no-nvram --force-extra-removable --no-floppy --modules=part_gpt --grub-mkdevicemap=/boot/grub/device.map" : \
sync : umount / : shutdown
}
alignment=$("${GNU_PREFIX}objdump" -p "$BOOTSTUB" | sed 's/^SectionAlignment\s\+\([0-9]\)/0x/;t;d')
test -z "$alignment" && die "failed to discover the alignment of the efi stub"
echo "determined efi vma alignment as $alignment"
test "$RELEASE" = jessie -a "$((alignment))" -lt "$((1024 * 1024))" && {
echo "increasing efi vma alignment for jessie"
alignment=$((1024 * 1024))
guestfish_efi() {
guestfish -- \
disk-create "$IMAGE" qcow2 "$SIZE" : \
add-drive "$IMAGE" format:qcow2 : \
launch : \
part-init /dev/sda gpt : \
part-add /dev/sda primary 8192 262144 : \
part-add /dev/sda primary 262145 -34 : \
part-set-gpt-type /dev/sda 1 C12A7328-F81F-11D2-BA4B-00A0C93EC93B : \
mkfs ext4 /dev/sda2 : mount /dev/sda2 / : \
tar-in - / xattrs:true : \
mkdir-p /boot/efi : \
mkfs vfat /dev/sda1 : mount /dev/sda1 /boot/efi : \
command "sh -c 'echo UUID=\$(blkid -c /dev/null -o value -s UUID /dev/sda2) / ext4 errors=remount-ro 0 1 > /etc/fstab'" : \
command "sh -c 'echo UUID=\$(blkid -c /dev/null -o value -s UUID /dev/sda1) /boot/efi vfat errors=remount-ro 0 2 >> /etc/fstab'" : \
command "sed -i 's/^GRUB_CMDLINE_LINUX_DEFAULT=/GRUB_CMDLINE_LINUX_DEFAULT=\"biosdevname=0 net.ifnames=0 consoleblank=0 rw $serial\"/' /etc/default/grub" : \
command "update-initramfs -u" : \
command "grub-mkconfig -o /boot/grub/grub.cfg" : \
command "grub-install /dev/sda --target=$grub_target --no-nvram --force-extra-removable --no-floppy --modules=part_gpt --grub-mkdevicemap=/boot/grub/device.map" : \
sync : umount /boot/efi : umount / : shutdown
}
lastoffset=0
# shellcheck disable=SC2034 # unused variables serve documentation
lastoffset="$("${GNU_PREFIX}objdump" -h "$BOOTSTUB" \
| while read -r idx name size vma lma fileoff algn behind; do
test -z "$behind" -a "${algn#"2**"}" != "$algn" || continue
offset=$((0x$vma + 0x$size))
test "$offset" -gt "$lastoffset" || continue
lastoffset="$offset"
echo "$lastoffset"
done | tail -n1)"
lastoffset=$(align_size "$lastoffset" "$alignment")
echo "determined minimum efi vma offset as $lastoffset"
cmdline_size="$(stat -Lc%s "$WORKDIR/cmdline")"
cmdline_size="$(align_size "$cmdline_size" "$alignment")"
linux_size="$(stat -Lc%s "$WORKDIR/kernel")"
linux_size="$(align_size "$linux_size" "$alignment")"
cmdline_offset="$lastoffset"
linux_offset=$((cmdline_offset + cmdline_size))
initrd_offset=$((linux_offset + linux_size))
guestfish_ieee1275() {
guestfish -- \
disk-create "$IMAGE" qcow2 "$SIZE" : \
add-drive "$IMAGE" format:qcow2 : \
launch : \
part-init /dev/sda gpt : \
part-add /dev/sda primary 8192 20480 : \
part-add /dev/sda primary 20481 -34 : \
part-set-gpt-type /dev/sda 1 9E1A2D38-C612-4316-AA26-8B49521E5A8B : \
mkfs ext4 /dev/sda2 : mount /dev/sda2 / : \
tar-in - / xattrs:true : \
command "sh -c 'echo UUID=\$(blkid -c /dev/null -o value -s UUID /dev/sda2) / ext4 errors=remount-ro 0 1 > /etc/fstab'" : \
command "sed -i 's/^GRUB_CMDLINE_LINUX_DEFAULT=/GRUB_CMDLINE_LINUX_DEFAULT=\"biosdevname=0 net.ifnames=0 consoleblank=0 rw $serial\"/' /etc/default/grub" : \
command "update-initramfs -u" : \
command "grub-mkconfig -o /boot/grub/grub.cfg" : \
command "grub-install /dev/sda --target=$grub_target --no-nvram --force-extra-removable --no-floppy --modules=part_gpt --grub-mkdevicemap=/boot/grub/device.map" : \
sync : umount / : shutdown
}
SOURCE_DATE_EPOCH=0 \
"${GNU_PREFIX}objcopy" \
--enable-deterministic-archives \
--add-section .cmdline="$WORKDIR/cmdline" \
--change-section-vma .cmdline="$(printf 0x%x "$cmdline_offset")" \
--add-section .linux="$WORKDIR/kernel" \
--change-section-vma .linux="$(printf 0x%x "$linux_offset")" \
--add-section .initrd="$WORKDIR/initrd" \
--change-section-vma .initrd="$(printf 0x%x "$initrd_offset")" \
"$BOOTSTUB" "$WORKDIR/efiimg"
case "$BOOT" in
bios) run_mmdebstrap | guestfish_bios;;
efi) run_mmdebstrap | guestfish_efi;;
ieee1275) run_mmdebstrap | guestfish_ieee1275;;
esac
rm -f "$WORKDIR/kernel" "$WORKDIR/initrd"
truncate -s "$((FAT_SIZE_SECTORS * 512))" "$WORKDIR/fat"
/sbin/mkfs.fat -F 32 --invariant "$WORKDIR/fat"
mmd -i "$WORKDIR/fat" EFI EFI/BOOT
mcopy -i "$WORKDIR/fat" "$WORKDIR/efiimg" "::EFI/BOOT/$EFIIMG"
rm -f "$WORKDIR/efiimg"
truncate --size="+$((34 * 512))" "$IMAGE"
/sbin/sfdisk "$IMAGE" <<EOF
label: gpt
unit: sectors
start=$FAT_OFFSET_SECTORS, size=$FAT_SIZE_SECTORS, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B
start=$((FAT_OFFSET_SECTORS + FAT_SIZE_SECTORS)), type=0FC63DAF-8483-4772-8E79-3D69D8477DE4
EOF
dd if="$WORKDIR/fat" of="$IMAGE" conv=notrunc,sparse bs=512 "seek=$FAT_OFFSET_SECTORS" status=none
if test "$(dpkg --print-architecture)" != "$ARCHITECTURE" && test "$(dpkg-query -f '${db:Status-Status}' -W "qemu-system-$QEMUARCH")" != installed; then
echo "I: you might need to install a package providing qemu-system-$QEMUARCH to use this image with autopkgtest-virt-qemu" >&2
fi
if test -n "$VMFPKG" && test "$(dpkg-query -f '${db:Status-Status}' -W "$VMFPKG")" != installed; then
echo "I: you might need to install $VMFPKG to use this image with autopkgtest-virt-qemu" >&2
fi
echo "I: SUCCESS! Your new image can be found here: $IMAGE" >&2
echo "I: Don't forget to pass --boot=efi when running autopkgtest-virt-qemu with this image" >&2
echo "Success! The image is stored as $IMAGE" >&2

View file

@ -1,56 +0,0 @@
#!/usr/bin/env python3
#
# This script is in the public domain
#
# Author: Johannes Schauer Marin Rodrigues <josch@mister-muffin.de>
#
# thin layer around /usr/lib/apt/solvers/apt, so that we can capture the solver
# result
#
# we set Debug::EDSP::WriteSolution=yes so that Install stanzas also come with
# Package and Version fields. That way, we do not also have to parse the EDSP
# request and spend time matching ID numbers
import subprocess
import sys
import os
import getpass
if not os.path.exists("/usr/lib/apt/solvers/apt"):
print(
"""Error: ERR_NO_SOLVER
Message: The external apt solver doesn't exist. You must install the apt-utils package.
"""
)
exit()
fname = os.environ.get("APT_EDSP_DUMP_FILENAME")
if fname is None:
print(
"""Error: ERR_NO_FILENAME
Message: You have to set the environment variable APT_EDSP_DUMP_FILENAME
to a valid filename to store the dump of EDSP solver input in.
For example with: export APT_EDSP_DUMP_FILENAME=/tmp/dump.edsp
"""
)
exit()
try:
with open(fname, "w") as f:
with subprocess.Popen(
["/usr/lib/apt/solvers/apt", "-oDebug::EDSP::WriteSolution=yes"],
stdin=sys.stdin.fileno(),
stdout=subprocess.PIPE,
bufsize=0, # unbuffered
text=True, # open in text mode
) as p:
for line in p.stdout:
print(line, end="")
f.write(line)
except (FileNotFoundError, PermissionError) as e:
print(
"""Error: ERR_CREATE_FILE
Message: Writing EDSP solver input to file '%s' failed as it couldn't be created!
"""
% fname
)

View file

@ -4,46 +4,27 @@ set -eu
SUDO=
while [ "$#" -gt 0 ]; do
key="$1"
case "$key" in
SUDO)
SUDO=sudo
;;
*)
echo "Unknown argument: $key"
exit 1
;;
esac
shift
key="$1"
case "$key" in
SUDO)
SUDO=sudo
;;
*)
echo "Unknown argument: $key"
exit 1
;;
esac
shift
done
# - Run command with fds 3 and 4 closed so that whatever test.sh does it
# cannot interfere with these.
# - Both stdin and stderr of test.sh are written to stdout
# - Write exit status of test.sh to fd 3
# - Write stdout to shared/output.txt as well as to fd 4
# - Redirect fd 3 to stdout
# - Read fd 3 and let the group exit with that value
# - Redirect fd 4 to stdout
ret=0
{
{
{
{
ret=0
(
exec 3>&- 4>&-
env --chdir=./shared $SUDO sh -x ./test.sh 2>&1
) || ret=$?
echo $ret >&3
} | tee shared/output.txt >&4
} 3>&1
} | {
read -r xs
exit "$xs"
}
} 4>&1 || ret=$?
if [ "$ret" -ne 0 ]; then
echo "test.sh failed"
exit 1
# subshell so that we can cd without effecting the rest
(
set +e
cd ./shared;
$SUDO sh -x ./test.sh;
echo $?;
) 2>&1 | tee shared/result.txt | head --lines=-1
if [ "$(tail --lines=1 shared/result.txt)" -ne 0 ]; then
echo "test.sh failed"
exit 1
fi

View file

@ -4,69 +4,45 @@ set -eu
: "${DEFAULT_DIST:=unstable}"
: "${cachedir:=./shared/cache}"
: "${MMDEBSTRAP_TESTS_DEBUG:=no}"
tmpdir="$(mktemp -d)"
cleanup() {
rv=$?
rm -f "$tmpdir/log"
[ -e "$tmpdir" ] && rmdir "$tmpdir"
if [ -e shared/output.txt ]; then
res="$(cat shared/exitstatus.txt)"
if [ "$res" != "0" ]; then
# this might possibly overwrite another non-zero rv
rv=1
fi
fi
exit $rv
rv=$?
rm -f "$tmpdir/debian-$DEFAULT_DIST-overlay.qcow"
rm -f "$tmpdir/log"
[ -e "$tmpdir" ] && rmdir "$tmpdir"
if [ -e shared/result.txt ]; then
head --lines=-1 shared/result.txt
res="$(tail --lines=1 shared/result.txt)"
rm shared/result.txt
if [ "$res" != "0" ]; then
# this might possibly overwrite another non-zero rv
rv=1
fi
fi
exit $rv
}
trap cleanup INT TERM EXIT
echo 1 >shared/exitstatus.txt
if [ -e shared/output.txt ]; then
rm shared/output.txt
fi
touch shared/output.txt
setpriv --pdeathsig TERM tail -f shared/output.txt &
set -- timeout --foreground 40m \
debvm-run --image="$(realpath "$cachedir")/debian-$DEFAULT_DIST.ext4" \
--
cpuname=$(lscpu | awk '/Model name:/ {print $3}' | tr '\n' '+')
ncpu=$(lscpu | awk '/Core\(s\) per socket:/ {print $4}' | tr '\n' '+')
if [ "$cpuname" = "Cortex-A53+Cortex-A73+" ] && [ "$ncpu" = "2+4+" ]; then
# crude detection of the big.LITTLE heterogeneous setup of cores on the
# amlogic a311d bananapi
#
# https://lists.nongnu.org/archive/html/qemu-devel/2020-10/msg08494.html
# https://gitlab.com/qemu-project/qemu/-/issues/239
# https://segments.zhan.science/posts/kvm_on_pinehone_pro/#trouble-with-heterogeneous-architecture
set -- taskset --cpu-list 2,3,4,5 "$@" -smp 4
fi
set -- "$@" -nic none -m 4G -snapshot
if [ "$MMDEBSTRAP_TESTS_DEBUG" = "no" ]; then
# to connect to serial use:
# minicom -D 'unix#/tmp/ttyS0'
# or this (quit with ctrl+q):
# socat stdin,raw,echo=0,escape=0x11 unix-connect:/tmp/ttyS0
set -- "$@" \
-monitor unix:/tmp/monitor,server,nowait \
-serial unix:/tmp/ttyS0,server,nowait \
-serial unix:/tmp/ttyS1,server,nowait
fi
set -- "$@" -virtfs local,id=mmdebstrap,path="$(pwd)/shared",security_model=none,mount_tag=mmdebstrap
# the path to debian-$DEFAULT_DIST.qcow must be absolute or otherwise qemu will
# look for the path relative to debian-$DEFAULT_DIST-overlay.qcow
qemu-img create -f qcow2 -b "$(realpath $cachedir)/debian-$DEFAULT_DIST.qcow" -F qcow2 "$tmpdir/debian-$DEFAULT_DIST-overlay.qcow"
# to connect to serial use:
# minicom -D 'unix#/tmp/ttyS0'
ret=0
if [ "$MMDEBSTRAP_TESTS_DEBUG" = "no" ]; then
"$@" >"$tmpdir/log" 2>&1 || ret=$?
else
"$@" 2>&1 | tee "$tmpdir/log" || ret=$?
fi
timeout 20m qemu-system-x86_64 \
-no-user-config \
-M accel=kvm:tcg -m 1G -nographic \
-object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 \
-monitor unix:/tmp/monitor,server,nowait \
-serial unix:/tmp/ttyS0,server,nowait \
-serial unix:/tmp/ttyS1,server,nowait \
-net nic,model=virtio -net user \
-virtfs local,id=mmdebstrap,path="$(pwd)/shared",security_model=none,mount_tag=mmdebstrap \
-drive file="$tmpdir/debian-$DEFAULT_DIST-overlay.qcow",cache=unsafe,index=0,if=virtio \
>"$tmpdir/log" 2>&1 || ret=$?
if [ "$ret" -ne 0 ]; then
cat "$tmpdir/log"
exit $ret
cat "$tmpdir/log"
exit $ret
fi

171
tarfilter
View file

@ -43,80 +43,20 @@ class PaxFilterAction(argparse.Action):
setattr(namespace, "paxfilter", items)
class TypeFilterAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
items = getattr(namespace, "typefilter", [])
match values:
case "REGTYPE" | "0":
items.append(tarfile.REGTYPE)
case "LNKTYPE" | "1":
items.append(tarfile.LNKTYPE)
case "SYMTYPE" | "2":
items.append(tarfile.SYMTYPE)
case "CHRTYPE" | "3":
items.append(tarfile.CHRTYPE)
case "BLKTYPE" | "4":
items.append(tarfile.BLKTYPE)
case "DIRTYPE" | "5":
items.append(tarfile.DIRTYPE)
case "FIFOTYPE" | "6":
items.append(tarfile.FIFOTYPE)
case _:
raise ValueError("invalid type: %s" % values)
setattr(namespace, "typefilter", items)
class TransformAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
items = getattr(namespace, "trans", [])
# This function mimics what src/transform.c from tar does
if not values.startswith("s"):
raise ValueError("regex must start with an 's'")
if len(values) <= 4:
# minimum regex: s/x//
raise ValueError("invalid regex (too short)")
d = values[1]
if values.startswith(f"s{d}{d}"):
raise ValueError("empty regex")
values = values.removeprefix(f"s{d}")
flags = 0
if values.endswith(f"{d}i"):
# trailing flags
flags = re.IGNORECASE
values = values.removesuffix(f"{d}i")
# This regex only finds non-empty tokens.
# Finding empty tokens would require a variable length look-behind
# or \K in order to find escaped delimiters which is not supported by
# the python re module.
tokens = re.findall(rf"(?:\\[\\{d}]|[^{d}])+", values)
match len(tokens):
case 0:
raise ValueError("invalid regex: not enough terms")
case 1:
repl = ""
case 2:
repl = tokens[1]
case _:
raise ValueError("invalid regex: too many terms: %s" % tokens)
items.append((re.compile(tokens[0], flags), repl))
setattr(namespace, "trans", items)
def main():
parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter,
description="""\
Filters a tarball on standard input by the same rules as the dpkg --path-exclude
and --path-include options and writes resulting tarball to standard output. See
dpkg(1) for information on how these two options work in detail. To reuse the
exact same semantics as used by dpkg, paths must be given as /path and not as
./path even though they might be stored as such in the tarball.
dpkg(1) for information on how these two options work in detail. Since this is
meant for filtering tarballs storing a rootfs, notice that paths must be given
as /path and not as ./path even though they might be stored as such in the
tarball.
Secondly, filter out unwanted pax extended headers using --pax-exclude and
--pax-include. This is useful in cases where a tool only accepts certain xattr
prefixes. For example tar2sqfs only supports SCHILY.xattr.user.*,
SCHILY.xattr.trusted.* and SCHILY.xattr.security.* but not
SCHILY.xattr.system.posix_acl_default.*.
Similarly, filter out unwanted pax extended headers. This is useful in cases
where a tool only accepts certain xattr prefixes. For example tar2sqfs only
supports SCHILY.xattr.user.*, SCHILY.xattr.trusted.* and
SCHILY.xattr.security.* but not SCHILY.xattr.system.posix_acl_default.*.
Both types of options use Unix shell-style wildcards:
@ -125,85 +65,45 @@ Both types of options use Unix shell-style wildcards:
[seq] matches any character in seq
[!seq] matches any character not in seq
Thirdly, filter out files matching a specific tar archive member type using
--type-exclude. Valid type names are REGTYPE (regular file), LNKTYPE
(hardlink), SYMTYPE (symlink), CHRTYPE (character special), BLKTYPE (block
special), DIRTYPE (directory), FIFOTYPE (fifo) or their tar format flag value
(0-6, respectively).
Fourthly, transform the path of tar members using a sed expression just as with
GNU tar --transform.
Fifthly, strip leading directory components off of tar members. Just as with
Thirdly, strip leading directory components off of tar members. Just as with
GNU tar --strip-components, tar members that have less or equal components in
their path are not passed through.
Lastly, shift user id and group id of each entry by the value given by the
--idshift argument. The resulting uid or gid must not be negative.
""",
"""
)
parser.add_argument(
"--path-exclude",
metavar="pattern",
action=PathFilterAction,
help="Exclude path matching the given shell pattern. "
"This option can be specified multiple times.",
help="Exclude path matching the given shell pattern.",
)
parser.add_argument(
"--path-include",
metavar="pattern",
action=PathFilterAction,
help="Re-include a pattern after a previous exclusion. "
"This option can be specified multiple times.",
help="Re-include a pattern after a previous exclusion.",
)
parser.add_argument(
"--pax-exclude",
metavar="pattern",
action=PaxFilterAction,
help="Exclude pax header matching the given globbing pattern. "
"This option can be specified multiple times.",
help="Exclude pax header matching the given globbing pattern.",
)
parser.add_argument(
"--pax-include",
metavar="pattern",
action=PaxFilterAction,
help="Re-include a pax header after a previous exclusion. "
"This option can be specified multiple times.",
)
parser.add_argument(
"--type-exclude",
metavar="type",
action=TypeFilterAction,
help="Exclude certain member types by their type. Choose types either "
"by their name (REGTYPE, LNKTYPE, SYMTYPE, CHRTYPE, BLKTYPE, DIRTYPE, "
"FIFOTYPE) or by their tar format flag values (0-6, respectively). "
"This option can be specified multiple times.",
)
parser.add_argument(
"--transform",
"--xform",
metavar="EXPRESSION",
action=TransformAction,
help="Use sed replace EXPRESSION to transform file names. "
"This option can be specified multiple times.",
help="Re-include a pax header after a previous exclusion.",
)
parser.add_argument(
"--strip-components",
metavar="NUMBER",
metavar="number",
type=int,
help="Strip NUMBER leading components from file names",
)
parser.add_argument(
"--idshift",
metavar="NUM",
type=int,
help="Integer value by which to shift the uid and gid of each entry",
)
args = parser.parse_args()
if (
not hasattr(args, "pathfilter")
and not hasattr(args, "paxfilter")
and not hasattr(args, "typefilter")
and not hasattr(args, "strip_components")
):
from shutil import copyfileobj
@ -218,22 +118,19 @@ Lastly, shift user id and group id of each entry by the value given by the
skip = False
if not hasattr(args, "pathfilter"):
return False
# normalize path and make it absolute by stripping off all leading
# dots and slashes and then prepending a slash
name = "/" + member.name.lstrip("./")
for t, r in args.pathfilter:
if r.match(name) is not None:
for (t, r) in args.pathfilter:
if r.match(member.name[1:]) is not None:
if t == "path_include":
skip = False
else:
skip = True
if skip and (member.isdir() or member.issym()):
for t, r in args.pathfilter:
for (t, r) in args.pathfilter:
if t != "path_include":
continue
prefix = prefix_prog.sub(r"\1", r.pattern)
prefix = prefix.rstrip("/")
if name.startswith(prefix):
if member.name[1:].startswith(prefix):
return False
return skip
@ -241,7 +138,7 @@ Lastly, shift user id and group id of each entry by the value given by the
if not hasattr(args, "paxfilter"):
return False
skip = False
for t, r in args.paxfilter:
for (t, r) in args.paxfilter:
if r.match(header) is None:
continue
if t == "pax_include":
@ -250,28 +147,16 @@ Lastly, shift user id and group id of each entry by the value given by the
skip = True
return skip
def type_filter_should_skip(member):
if not hasattr(args, "typefilter"):
return False
for t in args.typefilter:
if member.type == t:
return True
return False
# starting with Python 3.8, the default format became PAX_FORMAT but we
# are still explicit here in case of future changes.
# starting with Python 3.8, the default format became PAX_FORMAT, so this
# is only for compatibility with older versions of Python 3
with tarfile.open(fileobj=sys.stdin.buffer, mode="r|*") as in_tar, tarfile.open(
fileobj=sys.stdout.buffer, mode="w|", format=tarfile.PAX_FORMAT
) as out_tar:
for member in in_tar:
if path_filter_should_skip(member):
continue
if type_filter_should_skip(member):
continue
if args.strip_components:
comps = member.name.split("/")
# just as with GNU tar, archive members with less or equal
# number of components are not passed through at all
if len(comps) <= args.strip_components:
continue
member.name = "/".join(comps[args.strip_components :])
@ -280,18 +165,6 @@ Lastly, shift user id and group id of each entry by the value given by the
for k, v in member.pax_headers.items()
if not pax_filter_should_skip(k)
}
if args.idshift:
if args.idshift < 0 and -args.idshift > member.uid:
print("uid cannot be negative", file=sys.stderr)
exit(1)
if args.idshift < 0 and -args.idshift > member.gid:
print("gid cannot be negative", file=sys.stderr)
exit(1)
member.uid += args.idshift
member.gid += args.idshift
if hasattr(args, "trans"):
for r, s in args.trans:
member.name = r.sub(s, member.name)
if member.isfile():
with in_tar.extractfile(member) as file:
out_tar.addfile(member, file)

67
taridshift Executable file
View file

@ -0,0 +1,67 @@
#!/usr/bin/env python3
#
# This script is in the public domain
#
# Author: Johannes Schauer Marin Rodrigues <josch@mister-muffin.de>
#
# This script accepts a tarball on standard input and prints a tarball on
# standard output with the same contents but all uid and gid ownership
# information shifted by the value given as first command line argument.
#
# A tool like this should be written in C but libarchive has issues:
# https://github.com/libarchive/libarchive/issues/587
# https://github.com/libarchive/libarchive/pull/1288/ (needs 3.4.1)
# Should these issues get fixed, then a good template is tarfilter.c in the
# examples directory of libarchive.
#
# We are not using Perl either, because Archive::Tar slurps the whole tarball
# into memory.
#
# We could also use Go but meh...
# https://stackoverflow.com/a/59542307/784669
import tarfile
import sys
import argparse
def main():
parser = argparse.ArgumentParser(
description="""\
Accepts a tarball on standard input and prints a tarball on standard output
with the same contents but all uid and gid ownership information shifted by the
value given as first command line argument.
"""
)
parser.add_argument(
"idshift",
metavar="NUM",
type=int,
help="Integer value by which to shift the uid and gid of each entry",
)
args = parser.parse_args()
# starting with Python 3.8, the default format became PAX_FORMAT, so this
# is only for compatibility with older versions of Python 3
with tarfile.open(fileobj=sys.stdin.buffer, mode="r|*") as in_tar, tarfile.open(
fileobj=sys.stdout.buffer, mode="w|", format=tarfile.PAX_FORMAT
) as out_tar:
for member in in_tar:
if args.idshift < 0 and -args.idshift > member.uid:
print("uid cannot be negative", file=sys.stderr)
exit(1)
if args.idshift < 0 and -args.idshift > member.gid:
print("gid cannot be negative", file=sys.stderr)
exit(1)
member.uid += args.idshift
member.gid += args.idshift
if member.isfile():
with in_tar.extractfile(member) as file:
out_tar.addfile(member, file)
else:
out_tar.addfile(member)
if __name__ == "__main__":
main()

View file

@ -1,8 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
{{ CMD }} --mode={{ MODE }} --variant=essential \
--include '?or(?exact-name(dummy-does-not-exist),?exact-name(apt))' \
{{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -

View file

@ -1,12 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
{{ CMD }} --mode={{ MODE }} --variant=custom \
--include '?narrow(?archive(^{{ DIST }}$),?essential)' \
--include apt \
{{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
{
tar -tf /tmp/debian-chroot.tar
echo ./var/lib/apt/extended_states
} | sort | diff -u tar1.txt -

View file

@ -1,9 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -rf /tmp/debian-chroot; rm -f /tmp/config" EXIT INT TERM
echo 'Acquire::Languages "none";' >/tmp/config
{{ CMD }} --mode=root --variant=apt --aptopt='Acquire::Check-Valid-Until "false"' --aptopt=/tmp/config {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
printf 'Acquire::Check-Valid-Until "false";\nAcquire::Languages "none";\n' | cmp /tmp/debian-chroot/etc/apt/apt.conf.d/99mmdebstrap -
rm /tmp/debian-chroot/etc/apt/apt.conf.d/99mmdebstrap
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort | diff -u tar1.txt -

View file

@ -1,18 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
apt-get remove --yes qemu-user-binfmt binfmt-support qemu-user
# the following is not necessary anymore since systemd-binfmt
# successfully disables support upon removal of qemu-user with
# the upload of src:systemd 251.2-4: https://bugs.debian.org/1012163
#echo 0 > /proc/sys/fs/binfmt_misc/qemu-aarch64
ret=0
{{ CMD }} --mode={{ MODE }} --variant=apt --architectures=arm64 {{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }} || ret=$?
if [ "$ret" = 0 ]; then
echo expected failure but got exit $ret >&2
exit 1
fi

View file

@ -1,133 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
export SOURCE_DATE_EPOCH={{ SOURCE_DATE_EPOCH }}
prefix=
if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && [ "{{ MODE }}" != "auto" ]; then
if ! id "${SUDO_USER:-user}" >/dev/null 2>&1; then
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir "/home/${SUDO_USER:-user}" --create-home "${SUDO_USER:-user}"
fi
prefix="runuser -u ${SUDO_USER:-user} --"
fi
# debootstrap uses apt-config to figure out whether the system running it has
# any proxies configured and then runs the binary to set the http_proxy
# environment variable. This will fail if debootstrap is run in a linux user
# namespace because auto-apt-proxy will see /tmp/.auto-apt-proxy-0 as being
# owned by the user "nobody" and group "nogroup" and fail with:
# insecure cache dir /tmp/.auto-apt-proxy-0. Must be owned by UID 0 and have permissions 700
# We cannot overwrite a configuration item using the APT_CONFIG environment
# variable, so instead we use it to set the Dir configuration option
# to /dev/null to force all apt settings to their defaults.
# There is currently no better way to disable this behavior. See also:
# https://bugs.debian.org/1031105
# https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/90
AUTOPROXY=
eval "$(apt-config shell AUTOPROXY Acquire::http::Proxy-Auto-Detect)"
if [ -n "$AUTOPROXY" ] && [ -x "$AUTOPROXY" ] && [ -e /tmp/.auto-apt-proxy-0 ]; then
TMP_APT_CONFIG=$(mktemp)
echo 'Dir "/dev/null";' >"$TMP_APT_CONFIG"
chmod 644 "$TMP_APT_CONFIG"
fi
$prefix {{ CMD }} --variant=custom --mode={{ MODE }} \
--setup-hook='env '"${AUTOPROXY:+APT_CONFIG='$TMP_APT_CONFIG'}"' debootstrap --variant={{ VARIANT }} unstable "$1" {{ MIRROR }}' \
- /tmp/debian-mm.tar {{ MIRROR }}
if [ -n "$AUTOPROXY" ] && [ -x "$AUTOPROXY" ] && [ -e /tmp/.auto-apt-proxy-0 ]; then
rm "$TMP_APT_CONFIG"
fi
mkdir /tmp/debian-mm
tar --xattrs --xattrs-include='*' -C /tmp/debian-mm -xf /tmp/debian-mm.tar
mkdir /tmp/debian-debootstrap
tar --xattrs --xattrs-include='*' -C /tmp/debian-debootstrap -xf "cache/debian-unstable-{{ VARIANT }}.tar"
# diff cannot compare device nodes, so we use tar to do that for us and then
# delete the directory
tar -C /tmp/debian-debootstrap -cf dev1.tar ./dev
tar -C /tmp/debian-mm -cf dev2.tar ./dev
cmp dev1.tar dev2.tar >&2
rm dev1.tar dev2.tar
rm -r /tmp/debian-debootstrap/dev /tmp/debian-mm/dev
# remove downloaded deb packages
rm /tmp/debian-debootstrap/var/cache/apt/archives/*.deb
# remove aux-cache
rm /tmp/debian-debootstrap/var/cache/ldconfig/aux-cache
# remove logs
rm /tmp/debian-debootstrap/var/log/dpkg.log \
/tmp/debian-debootstrap/var/log/bootstrap.log \
/tmp/debian-debootstrap/var/log/alternatives.log \
/tmp/debian-mm/var/log/bootstrap.log
# clear out /run except for /run/lock
find /tmp/debian-debootstrap/run/ -mindepth 1 -maxdepth 1 ! -name lock -print0 | xargs --no-run-if-empty -0 rm -r
# debootstrap doesn't clean apt
rm /tmp/debian-debootstrap/var/lib/apt/lists/127.0.0.1_debian_dists_unstable_main_binary-{{ HOSTARCH }}_Packages \
/tmp/debian-debootstrap/var/lib/apt/lists/127.0.0.1_debian_dists_unstable_InRelease \
/tmp/debian-debootstrap/var/lib/apt/lists/127.0.0.1_debian_dists_unstable_Release \
/tmp/debian-debootstrap/var/lib/apt/lists/127.0.0.1_debian_dists_unstable_Release.gpg
if [ -e /tmp/debian-debootstrap/etc/machine-id ]; then
rm /tmp/debian-debootstrap/etc/machine-id /tmp/debian-mm/etc/machine-id
fi
rm /tmp/debian-mm/var/cache/apt/archives/lock
rm /tmp/debian-mm/var/lib/apt/lists/lock
rm /tmp/debian-mm/var/lib/dpkg/arch
# also needed for users that are created by systemd-sysusers before systemd 252
# https://github.com/systemd/systemd/pull/24534
for f in shadow shadow-; do
if [ ! -e /tmp/debian-debootstrap/etc/$f ]; then
continue
fi
if ! cmp /tmp/debian-debootstrap/etc/$f /tmp/debian-mm/etc/$f >&2; then
echo patching /etc/$f >&2
awk -v FS=: -v OFS=: -v SDE={{ SOURCE_DATE_EPOCH }} '{ print $1,$2,int(SDE/60/60/24),$4,$5,$6,$7,$8,$9 }' </tmp/debian-mm/etc/$f >/tmp/debian-mm/etc/$f.bak
cat /tmp/debian-mm/etc/$f.bak >/tmp/debian-mm/etc/$f
rm /tmp/debian-mm/etc/$f.bak
else
echo no difference for /etc/$f >&2
fi
done
# isc-dhcp-client postinst doesn't create this file in debootstrap run with
# unshared wrapper. The responsible postinst snippet was automatically added
# by dh_apparmor since isc-dhcp-client 4.4.3-P1-1.1
if [ -e /tmp/debian-debootstrap/etc/apparmor.d/local/sbin.dhclient ] && [ ! -s /tmp/debian-debootstrap/etc/apparmor.d/local/sbin.dhclient ]; then
echo /sbin/setcap >/tmp/debian-debootstrap/etc/apparmor.d/local/sbin.dhclient
fi
# check if the file content differs
diff --unified --no-dereference --recursive /tmp/debian-debootstrap /tmp/debian-mm >&2
# check permissions, ownership, symlink targets, modification times using tar
# mtimes of directories created by mmdebstrap will differ, thus we equalize them first
for d in etc/apt/preferences.d/ etc/apt/sources.list.d/ etc/dpkg/dpkg.cfg.d/ var/log/apt/; do
touch --date="@{{ SOURCE_DATE_EPOCH }}" /tmp/debian-debootstrap/$d /tmp/debian-mm/$d
done
# debootstrap never ran apt -- fixing permissions
for d in ./var/lib/apt/lists/partial ./var/cache/apt/archives/partial; do
chroot /tmp/debian-debootstrap chmod 0700 $d
chroot /tmp/debian-debootstrap chown _apt:root $d
done
tar -C /tmp/debian-debootstrap --numeric-owner --xattrs --xattrs-include='*' --sort=name --clamp-mtime --mtime="$(date --utc --date=@{{ SOURCE_DATE_EPOCH }} --iso-8601=seconds)" -cf /tmp/root1.tar .
tar -C /tmp/debian-mm --numeric-owner --xattrs --xattrs-include='*' --sort=name --clamp-mtime --mtime="$(date --utc --date=@{{ SOURCE_DATE_EPOCH }} --iso-8601=seconds)" -cf /tmp/root2.tar .
tar --full-time --verbose -tf /tmp/root1.tar >/tmp/root1.tar.list
tar --full-time --verbose -tf /tmp/root2.tar >/tmp/root2.tar.list
# despite SOURCE_DATE_EPOCH and --clamp-mtime, the timestamps in the tarball
# will slightly differ from each other in the sub-second precision (last
# decimals) so the tarballs will not be identical, so we use diff to compare
# content and tar to compare attributes
diff -u /tmp/root1.tar.list /tmp/root2.tar.list >&2
rm /tmp/root1.tar /tmp/root2.tar /tmp/root1.tar.list /tmp/root2.tar.list
rm /tmp/debian-mm.tar
rm -r /tmp/debian-debootstrap /tmp/debian-mm

View file

@ -1,21 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
for f in /etc/apt/trusted.gpg.d/*.gpg /etc/apt/trusted.gpg.d/*.asc; do
[ -e "$f" ] || continue
rm "$f"
done
rmdir /etc/apt/trusted.gpg.d
mkdir /etc/apt/trusted.gpg.d
for f in /usr/share/keyrings/*.gpg; do
name=$(basename "$f" .gpg)
gpg --no-default-keyring --keyring="/usr/share/keyrings/$name.gpg" --armor --output="/etc/apt/trusted.gpg.d/$name.asc" --export
rm "/usr/share/keyrings/$name.gpg"
done
{{ CMD }} --mode=root --variant=apt {{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -
rm -r /tmp/debian-chroot.tar

View file

@ -1,14 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
{{ CMD }} --mode={{ MODE }} --variant=custom \
--include "$(tr '\n' ',' <pkglist.txt)" \
--aptopt='APT::Solver "aspcud"' \
{{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
{
tar -tf /tmp/debian-chroot.tar
echo ./var/lib/apt/extended_states
} | sort \
| grep -v '^./etc/apt/apt.conf.d/99mmdebstrap$' \
| diff -u tar1.txt -

View file

@ -1,22 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -f /tmp/debian-chroot.tar.gz" EXIT INT TERM
[ {{ MODE }} = "auto" ]
prefix=
if [ "$(id -u)" -eq 0 ]; then
if ! id "${SUDO_USER:-user}" >/dev/null 2>&1; then
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir "/home/${SUDO_USER:-user}" --create-home "${SUDO_USER:-user}"
fi
prefix="runuser -u ${SUDO_USER:-user} --"
fi
$prefix {{ CMD }} --mode={{ MODE }} --variant=apt {{ DIST }} /tmp/debian-chroot.tar.gz {{ MIRROR }}
tar -tf /tmp/debian-chroot.tar.gz | sort | diff -u tar1.txt -

View file

@ -1,14 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir /home/user --create-home user
if [ -e /proc/sys/kernel/unprivileged_userns_clone ] && [ "$(sysctl -n kernel.unprivileged_userns_clone)" = "1" ]; then
sysctl -w kernel.unprivileged_userns_clone=0
fi
runuser -u user -- {{ CMD }} --mode=auto --variant=apt {{ DIST }} /tmp/debian-chroot.tar.gz {{ MIRROR }}
tar -tf /tmp/debian-chroot.tar.gz | sort | diff -u tar1.txt -
rm /tmp/debian-chroot.tar.gz

View file

@ -1,14 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
cat <<HOSTS >>/etc/hosts
127.0.0.1 deb.debian.org
127.0.0.1 security.debian.org
HOSTS
{{ CMD }} --mode={{ MODE }} --variant=apt {{ DIST }} /tmp/debian-chroot.tar
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -
rm /tmp/debian-chroot.tar

View file

@ -1,257 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
export SOURCE_DATE_EPOCH={{ SOURCE_DATE_EPOCH }}
echo "SOURCE_DATE_EPOCH=$SOURCE_DATE_EPOCH"
# we create the apt user ourselves or otherwise its uid/gid will differ
# compared to the one chosen in debootstrap because of different installation
# order in comparison to the systemd users
# https://bugs.debian.org/969631
# we cannot use useradd because passwd is not Essential:yes
{{ CMD }} --variant={{ VARIANT }} --mode={{ MODE }} \
--essential-hook='[ {{ DIST }} = oldstable ] && [ {{ VARIANT }} = - ] && echo _apt:*:100:65534::/nonexistent:/usr/sbin/nologin >> "$1"/etc/passwd || :' \
"$(if [ {{ DIST }} = oldstable ]; then echo --merged-usr; else echo --hook-dir=./hooks/merged-usr; fi)" \
"$(case {{ DIST }} in oldstable) echo --include=e2fsprogs,mount,tzdata,gcc-9-base ;; stable) echo --include=e2fsprogs,mount,tzdata ;; *) echo --include=base-files ;; esac)" \
{{ DIST }} /tmp/debian-{{ DIST }}-mm.tar {{ MIRROR }}
mkdir /tmp/debian-{{ DIST }}-mm
tar --xattrs --xattrs-include='*' -C /tmp/debian-{{ DIST }}-mm -xf /tmp/debian-{{ DIST }}-mm.tar
rm /tmp/debian-{{ DIST }}-mm.tar
mkdir /tmp/debian-{{ DIST }}-debootstrap
tar --xattrs --xattrs-include='*' -C /tmp/debian-{{ DIST }}-debootstrap -xf "cache/debian-{{ DIST }}-{{ VARIANT }}.tar"
# diff cannot compare device nodes, so we use tar to do that for us and then
# delete the directory
tar -C /tmp/debian-{{ DIST }}-debootstrap -cf /tmp/dev1.tar ./dev
tar -C /tmp/debian-{{ DIST }}-mm -cf /tmp/dev2.tar ./dev
ret=0
cmp /tmp/dev1.tar /tmp/dev2.tar >&2 || ret=$?
if [ "$ret" -ne 0 ]; then
if type diffoscope >/dev/null; then
diffoscope /tmp/dev1.tar /tmp/dev2.tar
exit 1
else
echo "no diffoscope installed" >&2
fi
if type base64 >/dev/null; then
base64 /tmp/dev1.tar
base64 /tmp/dev2.tar
exit 1
else
echo "no base64 installed" >&2
fi
if type xxd >/dev/null; then
xxd /tmp/dev1.tar
xxd /tmp/dev2.tar
exit 1
else
echo "no xxd installed" >&2
fi
exit 1
fi
rm /tmp/dev1.tar /tmp/dev2.tar
rm -r /tmp/debian-{{ DIST }}-debootstrap/dev /tmp/debian-{{ DIST }}-mm/dev
# remove downloaded deb packages
rm /tmp/debian-{{ DIST }}-debootstrap/var/cache/apt/archives/*.deb
# remove aux-cache
rm /tmp/debian-{{ DIST }}-debootstrap/var/cache/ldconfig/aux-cache
# remove logs
rm /tmp/debian-{{ DIST }}-debootstrap/var/log/dpkg.log \
/tmp/debian-{{ DIST }}-debootstrap/var/log/bootstrap.log \
/tmp/debian-{{ DIST }}-debootstrap/var/log/alternatives.log
# remove *-old files
rm /tmp/debian-{{ DIST }}-debootstrap/var/cache/debconf/config.dat-old \
/tmp/debian-{{ DIST }}-mm/var/cache/debconf/config.dat-old
rm /tmp/debian-{{ DIST }}-debootstrap/var/cache/debconf/templates.dat-old \
/tmp/debian-{{ DIST }}-mm/var/cache/debconf/templates.dat-old
rm /tmp/debian-{{ DIST }}-debootstrap/var/lib/dpkg/status-old \
/tmp/debian-{{ DIST }}-mm/var/lib/dpkg/status-old
rm -f /tmp/debian-{{ DIST }}-debootstrap/var/lib/dpkg/diversions-old \
/tmp/debian-{{ DIST }}-mm/var/lib/dpkg/diversions-old
# remove dpkg files
rm /tmp/debian-{{ DIST }}-debootstrap/var/lib/dpkg/available
rm /tmp/debian-{{ DIST }}-debootstrap/var/lib/dpkg/cmethopt
# remove /var/lib/dpkg/arch
rm /tmp/debian-{{ DIST }}-mm/var/lib/dpkg/arch
# since we installed packages directly from the .deb files, Priorities differ
# thus we first check for equality and then remove the files
chroot /tmp/debian-{{ DIST }}-debootstrap dpkg --list >/tmp/dpkg1
chroot /tmp/debian-{{ DIST }}-mm dpkg --list >/tmp/dpkg2
diff -u /tmp/dpkg1 /tmp/dpkg2 >&2
rm /tmp/dpkg1 /tmp/dpkg2
grep -v '^Priority: ' /tmp/debian-{{ DIST }}-debootstrap/var/lib/dpkg/status >/tmp/status1
grep -v '^Priority: ' /tmp/debian-{{ DIST }}-mm/var/lib/dpkg/status >/tmp/status2
diff -u /tmp/status1 /tmp/status2 >&2
rm /tmp/status1 /tmp/status2
rm /tmp/debian-{{ DIST }}-debootstrap/var/lib/dpkg/status /tmp/debian-{{ DIST }}-mm/var/lib/dpkg/status
# debootstrap exposes the hosts's kernel version
if [ -e /tmp/debian-{{ DIST }}-debootstrap/etc/apt/apt.conf.d/01autoremove-kernels ]; then
rm /tmp/debian-{{ DIST }}-debootstrap/etc/apt/apt.conf.d/01autoremove-kernels
fi
if [ -e /tmp/debian-{{ DIST }}-mm/etc/apt/apt.conf.d/01autoremove-kernels ]; then
rm /tmp/debian-{{ DIST }}-mm/etc/apt/apt.conf.d/01autoremove-kernels
fi
# clear out /run except for /run/lock
find /tmp/debian-{{ DIST }}-debootstrap/run/ -mindepth 1 -maxdepth 1 ! -name lock -print0 | xargs --no-run-if-empty -0 rm -r
# debootstrap doesn't clean apt
rm /tmp/debian-{{ DIST }}-debootstrap/var/lib/apt/lists/127.0.0.1_debian_dists_{{ DIST }}_main_binary-{{ HOSTARCH }}_Packages \
/tmp/debian-{{ DIST }}-debootstrap/var/lib/apt/lists/127.0.0.1_debian_dists_{{ DIST }}_InRelease \
/tmp/debian-{{ DIST }}-debootstrap/var/lib/apt/lists/127.0.0.1_debian_dists_{{ DIST }}_Release \
/tmp/debian-{{ DIST }}-debootstrap/var/lib/apt/lists/127.0.0.1_debian_dists_{{ DIST }}_Release.gpg
if [ "{{ VARIANT }}" = "-" ]; then
rm /tmp/debian-{{ DIST }}-debootstrap/etc/machine-id
rm /tmp/debian-{{ DIST }}-mm/etc/machine-id
rm /tmp/debian-{{ DIST }}-debootstrap/var/lib/systemd/catalog/database
rm /tmp/debian-{{ DIST }}-mm/var/lib/systemd/catalog/database
case {{ DIST }} in oldstable | stable)
cap=$(chroot /tmp/debian-{{ DIST }}-debootstrap /sbin/getcap /bin/ping)
expected="/bin/ping cap_net_raw=ep"
if [ "$cap" != "$expected" ]; then
echo "expected bin/ping to have capabilities $expected" >&2
echo "but debootstrap produced: $cap" >&2
exit 1
fi
cap=$(chroot /tmp/debian-{{ DIST }}-mm /sbin/getcap /bin/ping)
if [ "$cap" != "$expected" ]; then
echo "expected bin/ping to have capabilities $expected" >&2
echo "but mmdebstrap produced: $cap" >&2
exit 1
fi
;;
esac
fi
rm /tmp/debian-{{ DIST }}-mm/var/cache/apt/archives/lock
rm /tmp/debian-{{ DIST }}-mm/var/lib/apt/extended_states
rm /tmp/debian-{{ DIST }}-mm/var/lib/apt/lists/lock
# the list of shells might be sorted wrongly
# /var/lib/dpkg/triggers/File might be sorted wrongly
for f in "/var/lib/dpkg/triggers/File" "/etc/shells"; do
f1="/tmp/debian-{{ DIST }}-debootstrap/$f"
f2="/tmp/debian-{{ DIST }}-mm/$f"
# both chroots must have the file
if [ ! -e "$f1" ] || [ ! -e "$f2" ]; then
continue
fi
# the file must be different
if cmp "$f1" "$f2" >&2; then
continue
fi
# then sort both
sort -o "$f1" "$f1"
sort -o "$f2" "$f2"
done
# Because of unreproducible uids (#969631) we created the _apt user ourselves
# and because passwd is not Essential:yes we didn't use useradd. But newer
# versions of adduser and shadow will create a different /etc/shadow
if [ "{{ VARIANT }}" = "-" ] && [ "{{ DIST}}" = oldstable ]; then
for f in shadow shadow-; do
if grep -q '^_apt:!:' /tmp/debian-{{ DIST }}-debootstrap/etc/$f; then
sed -i 's/^_apt:\*:\([^:]\+\):0:99999:7:::$/_apt:!:\1::::::/' /tmp/debian-{{ DIST }}-mm/etc/$f
fi
done
fi
for log in faillog lastlog; do
f1="/tmp/debian-{{ DIST }}-debootstrap/var/log/$log"
f2="/tmp/debian-{{ DIST }}-mm/var/log/$log"
# skip cmp if file is absent in both chroots
if [ ! -e "$f1" ] && [ ! -e "$f2" ]; then
continue
fi
if ! cmp "$f1" "$f2" >&2; then
# if the files differ, make sure they are all zeroes
cmp -n "$(stat -c %s "$f1")" "$f1" /dev/zero >&2
cmp -n "$(stat -c %s "$f2")" "$f2" /dev/zero >&2
# then delete them
rm "$f1" "$f2"
fi
done
if [ "{{ VARIANT }}" = "-" ]; then
# the order in which systemd and cron get installed differ and thus the order
# of lines in /etc/group and /etc/gshadow differs
for f in group group- gshadow gshadow-; do
for d in mm debootstrap; do
sort /tmp/debian-{{ DIST }}-$d/etc/$f >/tmp/debian-{{ DIST }}-$d/etc/$f.bak
mv /tmp/debian-{{ DIST }}-$d/etc/$f.bak /tmp/debian-{{ DIST }}-$d/etc/$f
done
done
# the order in which systemd and passwd get installed differ and thus
# the order of lines in /etc/shadow and /etc/shadow- differs
for f in shadow shadow-; do
for d in mm debootstrap; do
sort /tmp/debian-{{ DIST }}-$d/etc/$f >/tmp/debian-{{ DIST }}-$d/etc/$f.bak
mv /tmp/debian-{{ DIST }}-$d/etc/$f.bak /tmp/debian-{{ DIST }}-$d/etc/$f
done
done
# and since the order was different, ignore the *- files
for f in shadow- passwd-; do
for d in mm debootstrap; do
rm /tmp/debian-{{ DIST }}-$d/etc/$f
done
done
fi
# since debootstrap 1.0.133 there is no tzdata in the buildd variant and thus
# debootstrap creates its own /etc/localtime
if [ "{{ VARIANT }}" = "buildd" ] && [ "{{ DIST }}" != "stable" ] && [ "{{ DIST }}" != "oldstable" ]; then
[ "$(readlink /tmp/debian-{{ DIST }}-debootstrap/etc/localtime)" = /usr/share/zoneinfo/UTC ]
rm /tmp/debian-{{ DIST }}-debootstrap/etc/localtime
fi
# starting with systemd 255 upstream dropped splitusr support and depending on
# the installation order, symlink targets are prefixed with /usr or not
# See #1060000 and #1054137
case {{ DIST }} in testing | unstable)
for f in multi-user.target.wants/e2scrub_reap.service timers.target.wants/apt-daily-upgrade.timer timers.target.wants/apt-daily.timer timers.target.wants/e2scrub_all.timer; do
for d in mm debootstrap; do
[ -L "/tmp/debian-{{ DIST }}-$d/etc/systemd/system/$f" ] || continue
oldlink="$(readlink "/tmp/debian-{{ DIST }}-$d/etc/systemd/system/$f")"
case $oldlink in
/usr/*) : ;;
/*) oldlink="/usr$oldlink" ;;
*)
echo unexpected >&2
exit 1
;;
esac
ln -sf "$oldlink" "/tmp/debian-{{ DIST }}-$d/etc/systemd/system/$f"
done
done
;;
esac
# check if the file content differs
diff --unified --no-dereference --recursive /tmp/debian-{{ DIST }}-debootstrap /tmp/debian-{{ DIST }}-mm >&2
# check permissions, ownership, symlink targets, modification times using tar
# directory mtimes will differ, thus we equalize them first
find /tmp/debian-{{ DIST }}-debootstrap /tmp/debian-{{ DIST }}-mm -type d -print0 | xargs -0 touch --date="@{{ SOURCE_DATE_EPOCH }}"
# debootstrap never ran apt -- fixing permissions
for d in ./var/lib/apt/lists/partial ./var/cache/apt/archives/partial; do
unmergedPATH="$PATH$(if [ "{{ DIST }}" = oldstable ]; then echo :/bin:/sbin; fi)"
PATH="$unmergedPATH" chroot /tmp/debian-{{ DIST }}-debootstrap chmod 0700 $d
PATH="$unmergedPATH" chroot /tmp/debian-{{ DIST }}-debootstrap chown "$(id -u _apt):root" $d
done
tar -C /tmp/debian-{{ DIST }}-debootstrap --numeric-owner --sort=name --clamp-mtime --mtime="$(date --utc --date=@{{ SOURCE_DATE_EPOCH }} --iso-8601=seconds)" -cf /tmp/root1.tar .
tar -C /tmp/debian-{{ DIST }}-mm --numeric-owner --sort=name --clamp-mtime --mtime="$(date --utc --date=@{{ SOURCE_DATE_EPOCH }} --iso-8601=seconds)" -cf /tmp/root2.tar .
tar --full-time --verbose -tf /tmp/root1.tar >/tmp/root1.tar.list
tar --full-time --verbose -tf /tmp/root2.tar >/tmp/root2.tar.list
diff -u /tmp/root1.tar.list /tmp/root2.tar.list >&2
rm /tmp/root1.tar /tmp/root2.tar /tmp/root1.tar.list /tmp/root2.tar.list
# check if file properties (permissions, ownership, symlink names, modification time) differ
#
# we cannot use this (yet) because it cannot cope with paths that have [ or @ in them
#fmtree -c -p /tmp/debian-{{ DIST }}-debootstrap -k flags,gid,link,mode,size,time,uid | sudo fmtree -p /tmp/debian-{{ DIST }}-mm
rm -r /tmp/debian-{{ DIST }}-debootstrap /tmp/debian-{{ DIST }}-mm

View file

@ -1,28 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
export SOURCE_DATE_EPOCH={{ SOURCE_DATE_EPOCH }}
trap "rm -f /tmp/debian-chroot-{{ MODE }}.{{ FORMAT }}" EXIT INT TERM
case {{ MODE }} in unshare | fakechroot) : ;; *) exit 1 ;; esac
prefix=
if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && [ "{{ MODE }}" != "auto" ]; then
if ! id "${SUDO_USER:-user}" >/dev/null 2>&1; then
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir "/home/${SUDO_USER:-user}" --create-home "${SUDO_USER:-user}"
fi
prefix="runuser -u ${SUDO_USER:-user} --"
fi
$prefix {{ CMD }} --mode={{ MODE }} --variant={{ VARIANT }} {{ DIST }} /tmp/debian-chroot-{{ MODE }}.{{ FORMAT }} {{ MIRROR }}
cmp ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.{{ FORMAT }} /tmp/debian-chroot-{{ MODE }}.{{ FORMAT }} \
|| diffoscope ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.{{ FORMAT }} /tmp/debian-chroot-{{ MODE }}.{{ FORMAT }}
# we cannot test chrootless mode here, because mmdebstrap relies on the
# usrmerge package to set up merged-/usr and that doesn't work in chrootless
# mode

View file

@ -1,8 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -rf /tmp/debian-chroot" EXIT INT TERM
mkdir /tmp/debian-chroot
chmod 700 /tmp/debian-chroot
{{ CMD }} --mode=root --variant=apt {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort | diff -u tar1.txt -

View file

@ -1,16 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
export SOURCE_DATE_EPOCH={{ SOURCE_DATE_EPOCH }}
trap "rm -f /tmp/chrootless.tar /tmp/root.tar" EXIT INT TERM
# we need --hook-dir=./hooks/merged-usr because usrmerge does not understand
# DPKG_ROOT
for INCLUDE in '' 'apt' 'apt,build-essential' 'systemd-sysv'; do
for MODE in root chrootless; do
{{ CMD }} --mode=$MODE --variant={{ VARIANT }} --hook-dir=./hooks/merged-usr \
${INCLUDE:+--include="$INCLUDE"} --skip=check/chrootless \
{{ DIST }} "/tmp/$MODE.tar" {{ MIRROR }}
done
cmp /tmp/root.tar /tmp/chrootless.tar || diffoscope /tmp/root.tar /tmp/chrootless.tar
rm /tmp/chrootless.tar /tmp/root.tar
done

View file

@ -1,43 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
export SOURCE_DATE_EPOCH={{ SOURCE_DATE_EPOCH }}
trap "rm -f /tmp/chrootless.tar /tmp/root.tar" EXIT INT TERM
[ {{ MODE }} = chrootless ]
prefix=
if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && [ "{{ MODE }}" != "auto" ]; then
if ! id "${SUDO_USER:-user}" >/dev/null 2>&1; then
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir "/home/${SUDO_USER:-user}" --create-home "${SUDO_USER:-user}"
fi
prefix="runuser -u ${SUDO_USER:-user} --"
fi
MMTARFILTER=
[ -x /usr/bin/mmtarfilter ] && MMTARFILTER=/usr/bin/mmtarfilter
[ -x ./tarfilter ] && MMTARFILTER=./tarfilter
# we need --hook-dir=./hooks/merged-usr because usrmerge does not understand
# DPKG_ROOT
# permissions drwxr-sr-x and extended attributes of ./var/log/journal/ cannot
# be preserved under fakeroot
# this applies to 'z' lines in files in /usr/lib/tmpfiles.d/
for INCLUDE in '' 'apt' 'apt,build-essential' 'systemd-sysv'; do
{{ CMD }} --variant={{ VARIANT }} --hook-dir=./hooks/merged-usr \
${INCLUDE:+--include="$INCLUDE"} \
{{ DIST }} - {{ MIRROR }} \
| "$MMTARFILTER" --path-exclude="/var/log/journal" --path-exclude="/etc/credstore*" \
>/tmp/root.tar
$prefix fakeroot {{ CMD }} --mode={{ MODE }} --variant={{ VARIANT }} --hook-dir=./hooks/merged-usr \
${INCLUDE:+--include="$INCLUDE"} \
{{ DIST }} - {{ MIRROR }} \
| "$MMTARFILTER" --path-exclude="/var/log/journal" --path-exclude="/etc/credstore*" \
>/tmp/chrootless.tar
cmp /tmp/root.tar /tmp/chrootless.tar || diffoscope /tmp/root.tar /tmp/chrootless.tar
rm /tmp/chrootless.tar /tmp/root.tar
done

View file

@ -1,66 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
export SOURCE_DATE_EPOCH={{ SOURCE_DATE_EPOCH }}
trap "rm -f /tmp/chrootless.tar /tmp/root.tar" EXIT INT TERM
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
deb2qemu() {
case "$1" in
amd64) echo x86_64 ;;
arm64) echo aarch64 ;;
armel | armhf) echo arm ;;
ppc64el) echo ppc64le ;;
*) echo "$1" ;;
esac
}
if [ "$(dpkg --print-architecture)" = "arm64" ]; then
arch=amd64
else
arch=arm64
fi
[ "$(id -u)" -eq 0 ]
[ -e "/proc/sys/fs/binfmt_misc/qemu-$(deb2qemu "$arch")" ]
# we need --hook-dir=./hooks/merged-usr because usrmerge does not understand
# DPKG_ROOT
#
# dpkg is unable to install architecture arch:all packages with a
# dependency on an arch:any package (perl-modules-5.34 in this case)
# inside foreign architecture chrootless chroots, because dpkg will use
# its own architecture as the native architecture, see #825385 and #1020533
# So we are not testing the installation of apt,build-essential here.
for INCLUDE in '' 'apt' 'systemd-sysv'; do
echo 1 >"/proc/sys/fs/binfmt_misc/qemu-$(deb2qemu "$arch")"
arch-test "$arch"
{{ CMD }} --mode=root --architecture="$arch" --variant={{ VARIANT }} \
--hook-dir=./hooks/merged-usr ${INCLUDE:+--include="$INCLUDE"} \
{{ DIST }} "/tmp/root.tar" {{ MIRROR }}
echo 0 >"/proc/sys/fs/binfmt_misc/qemu-$(deb2qemu "$arch")"
arch-test "$arch" && exit 1
{{ CMD }} --mode=chrootless --architecture="$arch" --variant={{ VARIANT }} \
--hook-dir=./hooks/merged-usr ${INCLUDE:+--include="$INCLUDE"} \
--skip=check/chrootless {{ DIST }} "/tmp/chrootless.tar" {{ MIRROR }}
# when creating a foreign architecture chroot, the tarballs are not
# bit-by-bit identical but contain a few remaining differences:
#
# * /etc/ld.so.cache -- hard problem, must be solved in glibc upstream
# * /var/lib/dpkg/triggers -- #990712
# * /var/cache/debconf/*.dat-old -- needs investigation
for tar in root chrootless; do
./tarfilter <"/tmp/$tar.tar" \
--path-exclude=/var/cache/debconf/config.dat-old \
--path-exclude=/var/cache/debconf/templates.dat-old \
--path-exclude=/etc/ld.so.cache \
--path-exclude=/var/lib/dpkg/triggers/File \
--path-exclude=/var/lib/dpkg/triggers/ldconfig \
>"/tmp/$tar.tar.tmp"
mv "/tmp/$tar.tar.tmp" "/tmp/$tar.tar"
done
cmp /tmp/root.tar /tmp/chrootless.tar || diffoscope /tmp/root.tar /tmp/chrootless.tar
rm /tmp/chrootless.tar /tmp/root.tar
done

View file

@ -1,44 +0,0 @@
#!/bin/sh
#
# test that the user can drop archives into /var/cache/apt/archives as well as
# into /var/cache/apt/archives/partial
set -eu
export LC_ALL=C.UTF-8
export SOURCE_DATE_EPOCH={{ SOURCE_DATE_EPOCH }}
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test requires the cache directory to be mounted on /mnt and should only be run inside a container" >&2
exit 1
fi
tmpdir=$(mktemp -d)
trap 'rm -f "$tmpdir"/*.deb /tmp/orig.tar /tmp/test1.tar /tmp/test2.tar; rmdir "$tmpdir"' EXIT INT TERM
include="--include=doc-debian"
if [ "{{ VARIANT }}" = "custom" ]; then
include="$include,base-files,base-passwd,coreutils,dash,diffutils,dpkg,libc-bin,sed"
fi
{{ CMD }} $include --mode={{ MODE }} --variant={{ VARIANT }} \
--setup-hook='mkdir -p "$1"/var/cache/apt/archives/partial' \
--setup-hook='touch "$1"/var/cache/apt/archives/lock' \
--setup-hook='chmod 0640 "$1"/var/cache/apt/archives/lock' \
{{ DIST }} - {{ MIRROR }} >/tmp/orig.tar
# somehow, when trying to create a tarball from the 9p mount, tar throws the
# following error: tar: ./doc-debian_6.4_all.deb: File shrank by 132942 bytes; padding with zeros
# to reproduce, try: tar --directory /mnt/cache/debian/pool/main/d/doc-debian/ --create --file - . | tar --directory /tmp/ --extract --file -
# this will be different:
# md5sum /mnt/cache/debian/pool/main/d/doc-debian/*.deb /tmp/*.deb
# another reason to copy the files into a new directory is, that we can use shell globs
cp /mnt/cache/debian/pool/main/b/busybox/busybox_*"_{{ HOSTARCH }}.deb" /mnt/cache/debian/pool/main/a/apt/apt_*"_{{ HOSTARCH }}.deb" "$tmpdir"
{{ CMD }} $include --mode={{ MODE }} --variant={{ VARIANT }} \
--setup-hook='mkdir -p "$1"/var/cache/apt/archives/partial' \
--setup-hook='sync-in "'"$tmpdir"'" /var/cache/apt/archives/partial' \
{{ DIST }} - {{ MIRROR }} >/tmp/test1.tar
cmp /tmp/orig.tar /tmp/test1.tar
{{ CMD }} $include --mode={{ MODE }} --variant={{ VARIANT }} \
--customize-hook='touch "$1"/var/cache/apt/archives/partial' \
--setup-hook='mkdir -p "$1"/var/cache/apt/archives/' \
--setup-hook='sync-in "'"$tmpdir"'" /var/cache/apt/archives/' \
--setup-hook='chmod 0755 "$1"/var/cache/apt/archives/' \
--customize-hook='find "'"$tmpdir"'" -type f -exec md5sum "{}" \; | sed "s|"'"$tmpdir"'"|$1/var/cache/apt/archives|" | md5sum --check' \
{{ DIST }} - {{ MIRROR }} >/tmp/test2.tar
cmp /tmp/orig.tar /tmp/test2.tar

View file

@ -1,10 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test requires the cache directory to be mounted on /mnt and should only be run inside a container" >&2
exit 1
fi
{{ CMD }} --mode={{ MODE }} --variant=apt {{ DIST }} /tmp/debian-chroot.tar "deb copy:///mnt/cache/debian {{ DIST }} main"
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -
rm /tmp/debian-chroot.tar

View file

@ -1,9 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -rf /tmp/debian-chroot" EXIT INT TERM
{{ CMD }} --mode=root --variant=apt {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
chroot /tmp/debian-chroot dpkg-query --showformat '${binary:Package}\n' --show >pkglist.txt
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort >tar1.txt

View file

@ -1,29 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
{{ CMD }} --mode={{ MODE }} --dry-run --variant=apt \
--setup-hook="exit 1" \
--essential-hook="exit 1" \
--customize-hook="exit 1" \
{{ DIST }} /tmp/debian-chroot {{ MIRROR }}
rm /tmp/debian-chroot/dev/console
rm /tmp/debian-chroot/dev/fd
rm /tmp/debian-chroot/dev/full
rm /tmp/debian-chroot/dev/null
rm /tmp/debian-chroot/dev/ptmx
rm /tmp/debian-chroot/dev/random
rm /tmp/debian-chroot/dev/stderr
rm /tmp/debian-chroot/dev/stdin
rm /tmp/debian-chroot/dev/stdout
rm /tmp/debian-chroot/dev/tty
rm /tmp/debian-chroot/dev/urandom
rm /tmp/debian-chroot/dev/zero
rm /tmp/debian-chroot/etc/apt/sources.list
rm /tmp/debian-chroot/etc/fstab
rm /tmp/debian-chroot/etc/hostname
rm /tmp/debian-chroot/etc/resolv.conf
rm /tmp/debian-chroot/var/lib/apt/lists/lock
rm /tmp/debian-chroot/var/lib/dpkg/status
rm /tmp/debian-chroot/var/lib/dpkg/arch
# the rest should be empty directories that we can rmdir recursively
find /tmp/debian-chroot -depth -print0 | xargs -0 rmdir

View file

@ -1,78 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
prefix=
if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && [ "{{ MODE }}" != "auto" ]; then
if ! id "${SUDO_USER:-user}" >/dev/null 2>&1; then
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir "/home/${SUDO_USER:-user}" --create-home "${SUDO_USER:-user}"
fi
prefix="runuser -u ${SUDO_USER:-user} --"
fi
case "$(dpkg --print-architecture)" in
arm64)
native_arch=arm64
native_gnu=aarch64-linux-gnu
foreign_arch=amd64
foreign_gnu=x86_64-linux-gnu
;;
amd64)
native_arch=amd64
native_gnu=x86_64-linux-gnu
foreign_arch=arm64
foreign_gnu=aarch64-linux-gnu
;;
*)
echo "unsupported native architecture" >&2
exit 1
;;
esac
[ "{{ MODE }}" = "fakechroot" ] && prefix="$prefix fakechroot fakeroot"
$prefix {{ CMD }} --mode={{ MODE }} --variant=apt --architectures="$foreign_arch" \
{{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
# we ignore differences between architectures by ignoring some files
# and renaming others
{
tar -tf /tmp/debian-chroot.tar \
| grep -v '^\./usr/bin/i386$' \
| grep -v '^\./usr/bin/x86_64$' \
| grep -v '^\./lib64$' \
| grep -v '^\./usr/lib64/$' \
| grep -v '^\./usr/lib64/ld-linux-x86-64\.so\.2$' \
| grep -v '^\./usr/lib/ld-linux-aarch64\.so\.1$' \
| grep -v "^\\./usr/lib/$foreign_gnu/ld-linux-aarch64\\.so\\.1$" \
| grep -v "^\\./usr/lib/$foreign_gnu/ld-linux-x86-64\\.so\\.2$" \
| grep -v "^\\./usr/lib/$foreign_gnu/perl/5\\.[0-9][.0-9]\\+/.*\\.ph$" \
| grep -v "^\\./usr/lib/$foreign_gnu/libmvec\\.so\\.1$" \
| grep -v "^\\./usr/share/doc/[^/]\\+/changelog\\(\\.Debian\\)\\?\\.$foreign_arch\\.gz$" \
| grep -v '^\./usr/share/man/man8/i386\.8\.gz$' \
| grep -v '^\./usr/share/man/man8/x86_64\.8\.gz$' \
| sed "s/$foreign_gnu/$native_gnu/" \
| sed "s/$foreign_arch/$native_arch/"
} | sort >/tmp/tar2.txt
{
grep <tar1.txt -v '^\./usr/bin/i386$' \
| grep -v '^\./usr/bin/x86_64$' \
| grep -v '^\./lib32$' \
| grep -v '^\./lib64$' \
| grep -v '^\./libx32$' \
| grep -v '^\./usr/lib32/$' \
| grep -v '^\./usr/libx32/$' \
| grep -v '^\./usr/lib64/$' \
| grep -v '^\./usr/lib64/ld-linux-x86-64\.so\.2$' \
| grep -v '^\./usr/lib/ld-linux-aarch64\.so\.1$' \
| grep -v "^\\./usr/lib/$native_gnu/ld-linux-x86-64\\.so\\.2$" \
| grep -v "^\\./usr/lib/$native_gnu/ld-linux-aarch64\\.so\\.1$" \
| grep -v "^\\./usr/lib/$native_gnu/libmvec\\.so\\.1$" \
| grep -v "^\\./usr/lib/$native_gnu/perl/5\\.[0-9][.0-9]\\+/.*\\.ph$" \
| grep -v "^\\./usr/share/doc/[^/]\\+/changelog\\(\\.Debian\\)\\?\\.$native_arch\\.gz$" \
| grep -v '^\./usr/share/man/man8/i386\.8\.gz$' \
| grep -v '^\./usr/share/man/man8/x86_64\.8\.gz$'
} | sort | diff -u - /tmp/tar2.txt >&2
rm /tmp/debian-chroot.tar /tmp/tar2.txt

View file

@ -1,20 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
prefix=
if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && [ "{{ MODE }}" != "auto" ]; then
if ! id "${SUDO_USER:-user}" >/dev/null 2>&1; then
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir "/home/${SUDO_USER:-user}" --create-home "${SUDO_USER:-user}"
fi
prefix="runuser -u ${SUDO_USER:-user} --"
fi
$prefix {{ CMD }} --mode={{ MODE }} --variant=apt {{ DIST }} /tmp/debian-chroot.tar.gz {{ MIRROR }}
printf '\037\213\010' | cmp --bytes=3 /tmp/debian-chroot.tar.gz -
tar -tf /tmp/debian-chroot.tar.gz | sort | diff -u tar1.txt -
rm /tmp/debian-chroot.tar.gz

View file

@ -1,27 +0,0 @@
#!/bin/sh
#
# we are testing all variants here because with 0.7.5 we had a bug:
# mmdebstrap sid /dev/null --simulate ==> E: cannot read /var/cache/apt/archives/
set -eu
export LC_ALL=C.UTF-8
prefix=
include=,
if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != root ] && [ "{{ MODE }}" != auto ]; then
if ! id "${SUDO_USER:-user}" >/dev/null 2>&1; then
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir "/home/${SUDO_USER:-user}" --create-home "${SUDO_USER:-user}"
fi
prefix="runuser -u ${SUDO_USER:-user} --"
if [ "{{ VARIANT }}" = extract ] || [ "{{ VARIANT }}" = custom ]; then
include="$(tr '\n' ',' <pkglist.txt)"
fi
fi
$prefix {{ CMD }} --mode={{ MODE }} --include="$include" --dry-run --variant={{ VARIANT }} {{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
if [ -e /tmp/debian-chroot.tar ]; then
echo "/tmp/debian-chroot.tar must not be created with --dry-run" >&2
exit 1
fi

View file

@ -1,12 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
mount -t tmpfs -o nodev,nosuid,size=400M tmpfs /tmp
# use --customize-hook to exercise the mounting/unmounting code of block devices in root mode
{{ CMD }} --mode=root --variant=apt --customize-hook='mount | grep /dev/full' --customize-hook='test "$(echo foo | tee /dev/full 2>&1 1>/dev/null)" = "tee: /dev/full: No space left on device"' {{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -
rm /tmp/debian-chroot.tar

View file

@ -1,33 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
[ "$(id -u)" -eq 0 ]
[ {{ MODE }} = "unshare" ]
if ! id "${SUDO_USER:-user}" >/dev/null 2>&1; then
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir "/home/${SUDO_USER:-user}" --create-home "${SUDO_USER:-user}"
fi
prefix="runuser -u ${SUDO_USER:-user} --"
# https://www.etalabs.net/sh_tricks.html
quote() { printf %s\\n "$1" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/'/"; }
homedir=$($prefix sh -c 'cd && pwd')
# apt:test/integration/test-apt-key
TMPDIR_ADD='This is fü$$ing cràzy, $(apt -v)$!'
$prefix mkdir "$homedir/$TMPDIR_ADD"
# make sure the unshared user can traverse into the TMPDIR
chmod 711 "$homedir"
# set permissions and sticky bit like the real /tmp
chmod 1777 "$homedir/$TMPDIR_ADD"
$prefix env TMPDIR="$homedir/$TMPDIR_ADD" {{ CMD }} --mode={{ MODE }} --variant=apt \
--setup-hook='case "$1" in '"$(quote "$homedir/$TMPDIR_ADD/mmdebstrap.")"'??????????) exit 0;; *) echo "$1"; exit 1;; esac' \
{{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -
# use rmdir as a quick check that nothing is remaining in TMPDIR
$prefix rmdir "$homedir/$TMPDIR_ADD"
rm /tmp/debian-chroot.tar

View file

@ -1,16 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -rf /tmp/debian-chroot; rm -f /tmp/customize.sh" EXIT INT TERM
cat <<'SCRIPT' >/tmp/customize.sh
#!/bin/sh
chroot "$1" whoami > "$1/output2"
chroot "$1" pwd >> "$1/output2"
SCRIPT
chmod +x /tmp/customize.sh
{{ CMD }} --mode=root --variant=apt --customize-hook='chroot "$1" sh -c "whoami; pwd" > "$1/output1"' --customize-hook=/tmp/customize.sh {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
printf "root\n/\n" | cmp /tmp/debian-chroot/output1
printf "root\n/\n" | cmp /tmp/debian-chroot/output2
rm /tmp/debian-chroot/output1
rm /tmp/debian-chroot/output2
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort | diff -u tar1.txt -

View file

@ -1,30 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
[ "$(id -u)" -eq 0 ]
[ {{ MODE }} = "unshare" ]
if ! id "${SUDO_USER:-user}" >/dev/null 2>&1; then
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir "/home/${SUDO_USER:-user}" --create-home "${SUDO_USER:-user}"
fi
prefix="runuser -u ${SUDO_USER:-user} --"
mkdir /tmp/debian-chroot
chmod 700 /tmp/debian-chroot
chown "${SUDO_USER:-user}:${SUDO_USER:-user}" /tmp/debian-chroot
set -- env --chdir=/tmp/debian-chroot
if [ "{{ CMD }}" = "./mmdebstrap" ]; then
set -- "$@" "$(realpath --canonicalize-existing ./mmdebstrap)"
elif [ "{{ CMD }}" = "perl -MDevel::Cover=-silent,-nogcov ./mmdebstrap" ]; then
set -- "$@" perl -MDevel::Cover=-silent,-nogcov "$(realpath --canonicalize-existing ./mmdebstrap)"
else
set -- "$@" {{ CMD }}
fi
$prefix "$@" --mode={{ MODE }} --variant=apt {{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -
rm /tmp/debian-chroot.tar

View file

@ -1,45 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -rf /tmp/debian-chroot; rm -f /tmp/sources.list /tmp/deb822.sources" EXIT INT TERM
cat <<SOURCES >/tmp/deb822.sources
Types: deb
URIs: {{ MIRROR }}1
Suites: {{ DIST }}
Components: main
SOURCES
echo "deb {{ MIRROR }}2 {{ DIST }} main" >/tmp/sources.list
echo "deb {{ MIRROR }}3 {{ DIST }} main" \
| {{ CMD }} --mode={{ MODE }} --variant=apt {{ DIST }} \
/tmp/debian-chroot \
/tmp/deb822.sources \
{{ MIRROR }}4 \
- \
"deb {{ MIRROR }}5 {{ DIST }} main" \
{{ MIRROR }}6 \
/tmp/sources.list
test ! -e /tmp/debian-chroot/etc/apt/sources.list
cat <<SOURCES | cmp /tmp/debian-chroot/etc/apt/sources.list.d/0000deb822.sources -
Types: deb
URIs: {{ MIRROR }}1
Suites: {{ DIST }}
Components: main
SOURCES
cat <<SOURCES | cmp /tmp/debian-chroot/etc/apt/sources.list.d/0001main.list -
deb {{ MIRROR }}4 {{ DIST }} main
deb {{ MIRROR }}3 {{ DIST }} main
deb {{ MIRROR }}5 {{ DIST }} main
deb {{ MIRROR }}6 {{ DIST }} main
SOURCES
echo "deb {{ MIRROR }}2 {{ DIST }} main" | cmp /tmp/debian-chroot/etc/apt/sources.list.d/0002sources.list -
tar -C /tmp/debian-chroot --one-file-system -c . \
| {
tar -t \
| grep -v "^./etc/apt/sources.list.d/0000deb822.sources$" \
| grep -v "^./etc/apt/sources.list.d/0001main.list$" \
| grep -v "^./etc/apt/sources.list.d/0002sources.list"
printf "./etc/apt/sources.list\n"
} | sort | diff -u tar1.txt -

View file

@ -1,44 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -rf /tmp/debian-chroot; rm -f /tmp/sources /tmp/deb822" EXIT INT TERM
cat <<SOURCES >/tmp/deb822
Types: deb
URIs: {{ MIRROR }}1
Suites: {{ DIST }}
Components: main
SOURCES
echo "deb {{ MIRROR }}2 {{ DIST }} main" >/tmp/sources
cat <<SOURCES | {{ CMD }} --mode={{ MODE }} --variant=apt {{ DIST }} \
/tmp/debian-chroot \
/tmp/deb822 \
- \
/tmp/sources
Types: deb
URIs: {{ MIRROR }}3
Suites: {{ DIST }}
Components: main
SOURCES
test ! -e /tmp/debian-chroot/etc/apt/sources.list
ls -lha /tmp/debian-chroot/etc/apt/sources.list.d/
cat <<SOURCES | cmp /tmp/debian-chroot/etc/apt/sources.list.d/0000deb822.sources -
Types: deb
URIs: {{ MIRROR }}1
Suites: {{ DIST }}
Components: main
SOURCES
cat <<SOURCES | cmp /tmp/debian-chroot/etc/apt/sources.list.d/0001main.sources -
Types: deb
URIs: {{ MIRROR }}3
Suites: {{ DIST }}
Components: main
SOURCES
echo "deb {{ MIRROR }}2 {{ DIST }} main" | cmp /tmp/debian-chroot/etc/apt/sources.list.d/0002sources.list -
tar -C /tmp/debian-chroot --one-file-system -c . \
| {
tar -t \
| grep -v "^./etc/apt/sources.list.d/0000deb822.sources$" \
| grep -v "^./etc/apt/sources.list.d/0001main.sources$" \
| grep -v "^./etc/apt/sources.list.d/0002sources.list$"
printf "./etc/apt/sources.list\n"
} | sort | diff -u tar1.txt -

View file

@ -1,16 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
export SOURCE_DATE_EPOCH={{ SOURCE_DATE_EPOCH }}
tmpdir="$(mktemp -d)"
chmod 755 "$tmpdir"
ret=0
debootstrap "$([ "{{ DIST }}" = oldstable ] && echo --no-merged-usr || echo --merged-usr)" --variant={{ VARIANT }} {{ DIST }} "$tmpdir" {{ MIRROR }} || ret=$?
if [ "$ret" -ne 0 ]; then
echo "E: debootstrap failed, dumping $tmpdir/debootstrap/debootstrap.log"
cat "$tmpdir/debootstrap/debootstrap.log"
exit 1
fi
tar --sort=name --mtime=@$SOURCE_DATE_EPOCH --clamp-mtime --numeric-owner --one-file-system --xattrs -C "$tmpdir" -c . >"./cache/debian-{{ DIST }}-{{ VARIANT }}.tar"
rm -r "$tmpdir"

View file

@ -1,6 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
{{ CMD }} --mode=root --variant=apt --resolve-deps --merged-usr --no-merged-usr --force-check-gpg {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort | diff -u tar1.txt -
rm -r /tmp/debian-chroot

View file

@ -1,15 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
export SOURCE_DATE_EPOCH={{ SOURCE_DATE_EPOCH }}
trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
# we use variant standard in verbose mode to see the maximum number of packages
# that was chosen in case of USE_HOST_APT_CONFIG=yes
case {{ VARIANT }} in standard) : ;; *) exit 1 ;; esac
{{ CMD }} --variant={{ VARIANT }} --debug {{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
cmp ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.tar /tmp/debian-chroot.tar \
|| diffoscope ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.tar /tmp/debian-chroot.tar

View file

@ -1,6 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
script -qfc "{{ CMD }} --mode={{ MODE }} --debug --variant=apt {{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}" /dev/null
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -

View file

@ -1,149 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
if [ {{ MODE }} != unshare ] && [ {{ MODE }} != root ]; then
echo "test requires root or unshare mode" >&2
exit 1
fi
prefix=
if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && [ "{{ MODE }}" != "auto" ]; then
if ! id "${SUDO_USER:-user}" >/dev/null 2>&1; then
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir "/home/${SUDO_USER:-user}" --create-home "${SUDO_USER:-user}"
fi
prefix="runuser -u ${SUDO_USER:-user} --"
fi
# this mimics what apt does in apt-pkg/deb/dpkgpm.cc/pkgDPkgPM::StartPtyMagic()
cat >/tmp/test.c <<'END'
#define _GNU_SOURCE
#include <stdlib.h>
#include <fcntl.h>
#include <termios.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/ioctl.h>
#include <signal.h>
int main() {
int ret;
int fd = posix_openpt(O_RDWR | O_NOCTTY);
if (fd < 0) {
perror("posix_openpt");
return 1;
}
char buf[64]; // 64 is used by apt
ret = ptsname_r(fd, buf, sizeof(buf));
if (ret != 0) {
perror("ptsname_r");
return 1;
}
ret = grantpt(fd);
if (ret == -1) {
perror("grantpt");
return 1;
}
struct termios origtt;
ret = tcgetattr(STDIN_FILENO, &origtt);
if (ret != 0) {
perror("tcgetattr1");
return 1;
}
struct termios tt;
ret = tcgetattr(STDOUT_FILENO, &tt);
if (ret != 0) {
perror("tcgetattr2");
return 1;
}
struct winsize win;
ret = ioctl(STDOUT_FILENO, TIOCGWINSZ, &win);
if (ret < 0) {
perror("ioctl stdout TIOCGWINSZ");
return 1;
}
ret = ioctl(fd, TIOCSWINSZ, &win);
if (ret < 0) {
perror("ioctl fd TIOCGWINSZ");
return 1;
}
ret = tcsetattr(fd, TCSANOW, &tt);
if (ret != 0) {
perror("tcsetattr1");
return 1;
}
cfmakeraw(&tt);
tt.c_lflag &= ~ECHO;
tt.c_lflag |= ISIG;
sigset_t sigmask;
sigset_t sigmask_old;
ret = sigemptyset(&sigmask);
if (ret != 0) {
perror("sigemptyset");
return 1;
}
ret = sigaddset(&sigmask, SIGTTOU);
if (ret != 0) {
perror("sigaddset");
return 1;
}
ret = sigprocmask(SIG_BLOCK,&sigmask, &sigmask_old);
if (ret != 0) {
perror("sigprocmask1");
return 1;
}
ret = tcsetattr(STDIN_FILENO, TCSAFLUSH, &tt);
if (ret != 0) {
perror("tcsetattr2");
return 1;
}
ret = sigprocmask(SIG_BLOCK,&sigmask_old, NULL);
if (ret != 0) {
perror("sigprocmask2");
return 1;
}
ret = tcsetattr(STDIN_FILENO, TCSAFLUSH, &origtt);
if (ret != 0) {
perror("tcsetattr3");
return 1;
}
return 0;
}
END
# use script to create a fake tty
# run all tests as root and as a normal user (the latter requires ptmxmode=666)
script -qfec "$prefix {{ CMD }} --mode={{ MODE }} --variant=apt \
--include=gcc,libc6-dev,python3,passwd \
--customize-hook='chroot \"\$1\" useradd --home-dir /home/user --create-home user' \
--customize-hook='chroot \"\$1\" python3 -c \"import pty; print(pty.openpty())\"' \
--customize-hook='chroot \"\$1\" runuser -u user -- python3 -c \"import pty; print(pty.openpty())\"' \
--customize-hook='chroot \"\$1\" script -c \"echo foobar\"' \
--customize-hook='chroot \"\$1\" runuser -u user -- env --chdir=/home/user script -c \"echo foobar\"' \
--customize-hook='chroot \"\$1\" apt-get install --yes doc-debian 2>&1 | tee \"\$1\"/tmp/log' \
--customize-hook=\"copy-in /tmp/test.c /tmp\" \
--customize-hook='chroot \"\$1\" gcc /tmp/test.c -o /tmp/test' \
--customize-hook='chroot \"\$1\" /tmp/test' \
--customize-hook='chroot \"\$1\" runuser -u user -- /tmp/test' \
--customize-hook='rm \"\$1\"/tmp/test \"\$1\"/tmp/test.c' \
--customize-hook=\"copy-out /tmp/log /tmp\" \
{{ DIST }} /dev/null {{ MIRROR }}" /dev/null
fail=0
[ -r /tmp/log ] || fail=1
grep '^E:' /tmp/log && fail=1
grep 'Can not write log' /tmp/log && fail=1
grep 'posix_openpt' /tmp/log && fail=1
grep 'No such file or directory' /tmp/log && fail=1
if [ $fail -eq 1 ]; then
echo "apt failed to write log:" >&2
cat /tmp/log >&2
exit 1
fi
rm /tmp/test.c /tmp/log

View file

@ -1,12 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
[ "$(whoami)" = "root" ]
trap "rm -rf /tmp/debian-chroot.tar" EXIT INT TERM
{{ CMD }} --mode={{ MODE }} --variant=apt --format=directory {{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
ftype=$(stat -c %F /tmp/debian-chroot.tar)
if [ "$ftype" != directory ]; then
echo "expected directory but got: $ftype" >&2
exit 1
fi
tar -C /tmp/debian-chroot.tar --one-file-system -c . | tar -t | sort | diff -u tar1.txt -

View file

@ -1,13 +0,0 @@
#!/bin/sh
#
# make sure that using codenames works https://bugs.debian.org/1003191
set -eu
export LC_ALL=C.UTF-8
trap "rm -f InRelease; rm -rf /tmp/debian-chroot.tar /tmp/expected" EXIT INT TERM
/usr/lib/apt/apt-helper download-file "{{ MIRROR }}/dists/{{ DIST }}/InRelease" InRelease
codename=$(awk '/^Codename: / { print $2; }' InRelease)
{{ CMD }} --mode={{ MODE }} --variant=apt "$codename" /tmp/debian-chroot.tar {{ MIRROR }}
echo "deb {{ MIRROR }} $codename main" >/tmp/expected
tar --to-stdout --extract --file /tmp/debian-chroot.tar ./etc/apt/sources.list \
| diff -u /tmp/expected -

View file

@ -1,14 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -rf /tmp/debian-chroot; rm -f /tmp/config" EXIT INT TERM
echo no-pager >/tmp/config
{{ CMD }} --mode=root --variant=apt --dpkgopt="path-exclude=/usr/share/doc/*" --dpkgopt=/tmp/config --dpkgopt="path-include=/usr/share/doc/dpkg/copyright" {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
printf 'path-exclude=/usr/share/doc/*\nno-pager\npath-include=/usr/share/doc/dpkg/copyright\n' | cmp /tmp/debian-chroot/etc/dpkg/dpkg.cfg.d/99mmdebstrap -
rm /tmp/debian-chroot/etc/dpkg/dpkg.cfg.d/99mmdebstrap
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort >tar2.txt
{
grep -v '^./usr/share/doc/.' tar1.txt
echo ./usr/share/doc/dpkg/
echo ./usr/share/doc/dpkg/copyright
} | sort | diff -u - tar2.txt

View file

@ -1,48 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
cat <<SCRIPT >/tmp/checkeatmydata.sh
#!/bin/sh
set -exu
cat << EOF | diff - "\$1"/usr/bin/dpkg
#!/bin/sh
exec /usr/bin/eatmydata /usr/bin/dpkg.distrib "\\\$@"
EOF
[ -e "\$1"/usr/bin/eatmydata ]
SCRIPT
chmod +x /tmp/checkeatmydata.sh
# first four bytes: magic
elfheader='\177ELF'
# fifth byte: bits
case "$(dpkg-architecture -qDEB_HOST_ARCH_BITS)" in
32) elfheader="$elfheader\\001" ;;
64) elfheader="$elfheader\\002" ;;
*)
echo "bits not supported"
exit 1
;;
esac
# sixth byte: endian
case "$(dpkg-architecture -qDEB_HOST_ARCH_ENDIAN)" in
little) elfheader="$elfheader\\001" ;;
big) elfheader="$elfheader\\002" ;;
*)
echo "endian not supported"
exit 1
;;
esac
# seventh and eigth byte: elf version (1) and abi (unset)
elfheader="$elfheader\\001\\000"
{{ CMD }} --mode=root --variant=apt \
--customize-hook=/tmp/checkeatmydata.sh \
--essential-hook=/tmp/checkeatmydata.sh \
--extract-hook='printf "'"$elfheader"'" | cmp --bytes=8 - "$1"/usr/bin/dpkg' \
--hook-dir=./hooks/eatmydata \
--customize-hook='printf "'"$elfheader"'" | cmp --bytes=8 - "$1"/usr/bin/dpkg' \
{{ DIST }} /tmp/debian-chroot {{ MIRROR }}
tar -C /tmp/debian-chroot --one-file-system -c . \
| tar -t \
| sort \
| diff -u tar1.txt -
rm /tmp/checkeatmydata.sh
rm -r /tmp/debian-chroot

View file

@ -1,8 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
printf '' | {{ CMD }} --mode={{ MODE }} --variant=apt \
--setup-hook='test -e "$1"/etc/apt/sources.list || echo "deb {{ MIRROR }} {{ DIST }} main" > "$1"/etc/apt/sources.list' \
{{ DIST }} /tmp/debian-chroot.tar -
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -

View file

@ -1,13 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
export SOURCE_DATE_EPOCH={{ SOURCE_DATE_EPOCH }}
trap "rm -f /tmp/debian-chroot.tar" EXIT INT TERM
{{ CMD }} --variant={{ VARIANT }} --verbose \
--setup-hook='echo deb {{ MIRROR }} {{ DIST }} main >> "$1"/etc/apt/sources.list' \
'' /tmp/debian-chroot.tar
cmp ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.tar /tmp/debian-chroot.tar \
|| diffoscope ./cache/mmdebstrap-{{ DIST }}-{{ VARIANT }}.tar /tmp/debian-chroot.tar

View file

@ -1,12 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
ret=0
script -qfec "{{ CMD }} --mode={{ MODE }} --variant=apt {{ DIST }} - {{ MIRROR }}" /dev/null || ret=$?
if [ "$ret" = 0 ]; then
echo expected failure but got exit $ret >&2
exit 1
fi

View file

@ -1,20 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -rf /tmp/debian-chroot; rm -f /tmp/essential.sh" EXIT INT TERM
cat <<'SCRIPT' >/tmp/essential.sh
#!/bin/sh
echo tzdata tzdata/Zones/Europe select Berlin | chroot "$1" debconf-set-selections
SCRIPT
chmod +x /tmp/essential.sh
{{ CMD }} --mode=root --variant=apt --include=tzdata --essential-hook='echo tzdata tzdata/Areas select Europe | chroot "$1" debconf-set-selections' --essential-hook=/tmp/essential.sh {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
[ "$(readlink /tmp/debian-chroot/etc/localtime)" = "/usr/share/zoneinfo/Europe/Berlin" ]
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort \
| grep -v '^./etc/localtime' \
| grep -v '^./etc/timezone' \
| grep -v '^./usr/sbin/tzconfig' \
| grep -v '^./usr/share/doc/tzdata' \
| grep -v '^./usr/share/lintian/overrides/tzdata' \
| grep -v '^./usr/share/zoneinfo' \
| grep -v '^./var/lib/dpkg/info/tzdata.' \
| diff -u tar1.txt -

View file

@ -1,9 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -rf /tmp/debian-chroot" EXIT INT TERM
mkdir /tmp/debian-chroot
mkdir /tmp/debian-chroot/lost+found
{{ CMD }} --mode=root --variant=apt {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
rmdir /tmp/debian-chroot/lost+found
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort | diff -u tar1.txt -

View file

@ -1,7 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -rf /tmp/debian-chroot" EXIT INT TERM
mkdir /tmp/debian-chroot
{{ CMD }} --mode=root --variant=apt {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort | diff -u tar1.txt -

View file

@ -1,13 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -f /tmp/exists" EXIT INT TERM
touch /tmp/exists
ret=0
{{ CMD }} --mode={{ MODE }} --variant=apt {{ DIST }} /tmp/exists {{ MIRROR }} || ret=$?
if [ "$ret" = 0 ]; then
echo expected failure but got exit $ret >&2
exit 1
fi

View file

@ -1,13 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm /tmp/debian-chroot/lost+found/exists; rmdir /tmp/debian-chroot/lost+found /tmp/debian-chroot" EXIT INT TERM
mkdir /tmp/debian-chroot
mkdir /tmp/debian-chroot/lost+found
touch /tmp/debian-chroot/lost+found/exists
ret=0
{{ CMD }} --mode={{ MODE }} --variant=apt {{ DIST }} /tmp/debian-chroot {{ MIRROR }} || ret=$?
if [ "$ret" = 0 ]; then
echo expected failure but got exit $ret >&2
exit 1
fi

View file

@ -1,13 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rmdir /tmp/debian-chroot/lost+found; rm /tmp/debian-chroot/exists; rmdir /tmp/debian-chroot" EXIT INT TERM
mkdir /tmp/debian-chroot
mkdir /tmp/debian-chroot/lost+found
touch /tmp/debian-chroot/exists
ret=0
{{ CMD }} --mode={{ MODE }} --variant=apt {{ DIST }} /tmp/debian-chroot {{ MIRROR }} || ret=$?
if [ "$ret" = 0 ]; then
echo expected failure but got exit $ret >&2
exit 1
fi

View file

@ -1,9 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
ret=0
{{ CMD }} --mode={{ MODE }} --variant=apt {{ DIST }} / {{ MIRROR }} || ret=$?
if [ "$ret" = 0 ]; then
echo expected failure but got exit $ret >&2
exit 1
fi

View file

@ -1,9 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
ret=0
{{ CMD }} --mode={{ MODE }} --variant=apt {{ DIST }} /tmp/debian-chroot.tar.lz4 {{ MIRROR }} || ret=$?
if [ "$ret" = 0 ]; then
echo expected failure but got exit $ret >&2
exit 1
fi

View file

@ -1,12 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap 'rm -rf /tmp/quoted\"path' EXIT INT TERM
ret=0
{{ CMD }} --mode={{ MODE }} --variant=apt {{ DIST }} /tmp/quoted\"path {{ MIRROR }} || ret=$?
if [ "$ret" = 0 ]; then
echo expected failure but got exit $ret >&2
exit 1
fi

View file

@ -1,16 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir /home/user --create-home user
rm /etc/subuid
ret=0
runuser -u user -- {{ CMD }} --mode=unshare --variant=apt {{ DIST }} /tmp/debian-chroot {{ MIRROR }} || ret=$?
if [ "$ret" = 0 ]; then
echo expected failure but got exit $ret >&2
exit 1
fi
[ ! -e /tmp/debian-chroot ]

View file

@ -1,17 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir /home/user --create-home user
awk -F: '$1!="user"' /etc/subuid >/etc/subuid.tmp
mv /etc/subuid.tmp /etc/subuid
ret=0
runuser -u user -- {{ CMD }} --mode=unshare --variant=apt {{ DIST }} /tmp/debian-chroot {{ MIRROR }} || ret=$?
if [ "$ret" = 0 ]; then
echo expected failure but got exit $ret >&2
exit 1
fi
[ ! -e /tmp/debian-chroot ]

View file

@ -1,10 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
ret=0
{{ CMD }} --mode=root --variant=apt --customize-hook='chroot "$1" sh -c "exit 1"' {{ DIST }} /tmp/debian-chroot {{ MIRROR }} || ret=$?
rm -r /tmp/debian-chroot
if [ "$ret" = 0 ]; then
echo expected failure but got exit $ret >&2
exit 1
fi

View file

@ -1,13 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test requires the cache directory to be mounted on /mnt and should only be run inside a container" >&2
exit 1
fi
{{ CMD }} --mode={{ MODE }} --variant=apt \
--setup-hook='mkdir -p "$1"/mnt/cache/debian; mount -o ro,bind /mnt/cache/debian "$1"/mnt/cache/debian' \
--customize-hook='umount "$1"/mnt/cache/debian; rmdir "$1"/mnt/cache/debian "$1"/mnt/cache' \
{{ DIST }} /tmp/debian-chroot.tar "deb file:///mnt/cache/debian {{ DIST }} main"
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -
rm /tmp/debian-chroot.tar

View file

@ -1,20 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test requires the cache directory to be mounted on /mnt and should only be run inside a container" >&2
exit 1
fi
if [ "$(id -u)" -eq 0 ] && ! id -u user >/dev/null 2>&1; then
useradd --home-dir /home/user --create-home user
fi
prefix=
[ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && prefix="runuser -u user --"
[ "{{ MODE }}" = "fakechroot" ] && prefix="$prefix fakechroot fakeroot"
$prefix {{ CMD }} --mode={{ MODE }} --variant=apt \
--hook-dir=./hooks/file-mirror-automount \
--customize-hook='[ ! -e "$1"/mnt/cache/debian/ ] || rmdir "$1"/mnt/cache/debian/' \
--customize-hook='rmdir "$1"/mnt/cache' \
{{ DIST }} /tmp/debian-chroot.tar "deb file:///mnt/cache/debian {{ DIST }} main"
tar -tf /tmp/debian-chroot.tar | sort | diff -u tar1.txt -
rm /tmp/debian-chroot.tar

View file

@ -1,6 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
# we redirect to /dev/null instead of using --quiet to not cause a broken pipe
# when grep exits before mmdebstrap was able to write all its output
{{ CMD }} --help | grep --fixed-strings 'mmdebstrap [OPTION...] [SUITE [TARGET [MIRROR...]]]' >/dev/null

View file

@ -1,49 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
for h in hookA hookB; do
mkdir /tmp/$h
for s in setup extract essential customize; do
cat <<SCRIPT >/tmp/$h/${s}00.sh
#!/bin/sh
echo $h/${s}00 >> "\$1/$s"
SCRIPT
chmod +x /tmp/$h/${s}00.sh
cat <<SCRIPT >/tmp/$h/${s}01.sh
echo $h/${s}01 >> "\$1/$s"
SCRIPT
chmod +x /tmp/$h/${s}01.sh
done
done
{{ CMD }} --mode=root --variant=apt \
--setup-hook='echo cliA/setup >> "$1"/setup' \
--extract-hook='echo cliA/extract >> "$1"/extract' \
--essential-hook='echo cliA/essential >> "$1"/essential' \
--customize-hook='echo cliA/customize >> "$1"/customize' \
--hook-dir=/tmp/hookA \
--setup-hook='echo cliB/setup >> "$1"/setup' \
--extract-hook='echo cliB/extract >> "$1"/extract' \
--essential-hook='echo cliB/essential >> "$1"/essential' \
--customize-hook='echo cliB/customize >> "$1"/customize' \
--hook-dir=/tmp/hookB \
--setup-hook='echo cliC/setup >> "$1"/setup' \
--extract-hook='echo cliC/extract >> "$1"/extract' \
--essential-hook='echo cliC/essential >> "$1"/essential' \
--customize-hook='echo cliC/customize >> "$1"/customize' \
{{ DIST }} /tmp/debian-chroot {{ MIRROR }}
printf "cliA/setup\nhookA/setup00\nhookA/setup01\ncliB/setup\nhookB/setup00\nhookB/setup01\ncliC/setup\n" | diff -u - /tmp/debian-chroot/setup
printf "cliA/extract\nhookA/extract00\nhookA/extract01\ncliB/extract\nhookB/extract00\nhookB/extract01\ncliC/extract\n" | diff -u - /tmp/debian-chroot/extract
printf "cliA/essential\nhookA/essential00\nhookA/essential01\ncliB/essential\nhookB/essential00\nhookB/essential01\ncliC/essential\n" | diff -u - /tmp/debian-chroot/essential
printf "cliA/customize\nhookA/customize00\nhookA/customize01\ncliB/customize\nhookB/customize00\nhookB/customize01\ncliC/customize\n" | diff -u - /tmp/debian-chroot/customize
for s in setup extract essential customize; do
rm /tmp/debian-chroot/$s
done
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort | diff -u tar1.txt -
for h in hookA hookB; do
for s in setup extract essential customize; do
rm /tmp/$h/${s}00.sh
rm /tmp/$h/${s}01.sh
done
rmdir /tmp/$h
done
rm -r /tmp/debian-chroot

View file

@ -1,42 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
# remove qemu just to be sure
apt-get remove --yes qemu-user-binfmt binfmt-support qemu-user
{{ CMD }} --mode={{ MODE }} --variant=apt --architectures=i386 {{ DIST }} /tmp/debian-chroot.tar {{ MIRROR }}
# we ignore differences between architectures by ignoring some files
# and renaming others
{
tar -tf /tmp/debian-chroot.tar \
| grep -v '^\./usr/bin/i386$' \
| grep -v '^\./usr/lib/ld-linux\.so\.2$' \
| grep -v '^\./usr/lib/i386-linux-gnu/ld-linux\.so\.2$' \
| grep -v '^\./usr/lib/gcc/i686-linux-gnu/$' \
| grep -v '^\./usr/lib/gcc/i686-linux-gnu/[0-9]\+/$' \
| grep -v '^\./usr/share/man/man8/i386\.8\.gz$' \
| grep -v '^\./usr/share/doc/[^/]\+/changelog\(\.Debian\)\?\.i386\.gz$' \
| sed 's/i386-linux-gnu/x86_64-linux-gnu/' \
| sed 's/i386/amd64/' \
| sed 's/\/stubs-32.ph$/\/stubs-64.ph/'
} | sort >tar2.txt
{
grep <tar1.txt -v '^\./usr/bin/i386$' \
| grep -v '^\./usr/bin/x86_64$' \
| grep -v '^\./usr/lib32/$' \
| grep -v '^\./lib32$' \
| grep -v '^\./lib64$' \
| grep -v '^\./usr/lib64/$' \
| grep -v '^\./usr/lib64/ld-linux-x86-64\.so\.2$' \
| grep -v '^\./usr/lib/gcc/x86_64-linux-gnu/$' \
| grep -v '^\./usr/lib/gcc/x86_64-linux-gnu/[0-9]\+/$' \
| grep -v '^\./usr/lib/x86_64-linux-gnu/ld-linux-x86-64\.so\.2$' \
| grep -v '^\./usr/lib/x86_64-linux-gnu/libmvec\.so\.1$' \
| grep -v '^\./usr/share/doc/[^/]\+/changelog\(\.Debian\)\?\.amd64\.gz$' \
| grep -v '^\./usr/share/man/man8/i386\.8\.gz$' \
| grep -v '^\./usr/share/man/man8/x86_64\.8\.gz$'
} | sort | diff -u - tar2.txt >&2
rm /tmp/debian-chroot.tar

View file

@ -1,11 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -rf /tmp/debian-chroot" EXIT INT TERM
{{ CMD }} --mode=root --variant=apt --include=doc-debian {{ DIST }} /tmp/debian-chroot {{ MIRROR }}
rm /tmp/debian-chroot/usr/share/doc-base/doc-debian.debian-*
rm -r /tmp/debian-chroot/usr/share/doc/debian
rm -r /tmp/debian-chroot/usr/share/doc/doc-debian
rm /tmp/debian-chroot/var/lib/dpkg/info/doc-debian.list
rm /tmp/debian-chroot/var/lib/dpkg/info/doc-debian.md5sums
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort | diff -u tar1.txt -

View file

@ -1,40 +0,0 @@
#!/bin/sh
set -eu
export LC_ALL=C.UTF-8
trap "rm -rf /tmp/dummypkg.deb /tmp/dummypkg" EXIT INT TERM
prefix=
if [ "$(id -u)" -eq 0 ] && [ "{{ MODE }}" != "root" ] && [ "{{ MODE }}" != "auto" ]; then
if ! id "${SUDO_USER:-user}" >/dev/null 2>&1; then
if [ ! -e /mmdebstrap-testenv ]; then
echo "this test modifies the system and should only be run inside a container" >&2
exit 1
fi
useradd --home-dir "/home/${SUDO_USER:-user}" --create-home "${SUDO_USER:-user}"
fi
prefix="runuser -u ${SUDO_USER:-user} --"
fi
# instead of obtaining a .deb from our cache, we create a new package because
# otherwise apt might decide to download the package with the same name and
# version from the cache instead of using the local .deb
mkdir -p /tmp/dummypkg/DEBIAN
cat <<END >"/tmp/dummypkg/DEBIAN/control"
Package: dummypkg
Priority: optional
Section: oldlibs
Maintainer: Johannes Schauer Marin Rodrigues <josch@debian.org>
Architecture: all
Multi-Arch: foreign
Source: dummypkg
Version: 1
Description: dummypkg
END
dpkg-deb --build "/tmp/dummypkg" "/tmp/dummypkg.deb"
$prefix {{ CMD }} --mode={{ MODE }} --variant=apt --include="/tmp/dummypkg.deb" \
--hook-dir=./hooks/file-mirror-automount \
--customize-hook='chroot "$1" dpkg-query -W -f="\${Status}\n" dummypkg | grep "^install ok installed$"' \
{{ DIST }} /dev/null {{ MIRROR }}

View file

@ -1,49 +0,0 @@
#!/bin/sh
#
# to test foreign architecture package installation we choose a package which
# - is not part of the native installation set
# - does not have any dependencies
# - installs only few files
# - doesn't change its name regularly (like gcc-*-base)
case "$(dpkg --print-architecture)" in
arm64)
native_arch=arm64
foreign_arch=amd64
;;
amd64)
native_arch=amd64
foreign_arch=arm64
;;
*)
echo "unsupported native architecture" >&2
exit 1
;;
esac
set -eu
export LC_ALL=C.UTF-8
{{ CMD }} --mode=root --variant=apt \
--architectures="$native_arch,$foreign_arch" \
--include="libmagic-mgc:$foreign_arch" \
{{ DIST }} /tmp/debian-chroot {{ MIRROR }}
{
echo "$native_arch"
echo "$foreign_arch"
} | cmp /tmp/debian-chroot/var/lib/dpkg/arch -
rm /tmp/debian-chroot/usr/lib/file/magic.mgc
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/README.Debian
rm -f /tmp/debian-chroot/usr/share/doc/libmagic-mgc/"changelog.Debian.$foreign_arch.gz"
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/changelog.Debian.gz
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/changelog.gz
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/copyright
rm /tmp/debian-chroot/usr/share/file/magic.mgc
rm /tmp/debian-chroot/usr/share/misc/magic.mgc
rm /tmp/debian-chroot/var/lib/dpkg/info/libmagic-mgc.list
rm /tmp/debian-chroot/var/lib/dpkg/info/libmagic-mgc.md5sums
rmdir /tmp/debian-chroot/usr/share/doc/libmagic-mgc/
rmdir /tmp/debian-chroot/usr/share/file/magic/
rmdir /tmp/debian-chroot/usr/share/file/
rmdir /tmp/debian-chroot/usr/lib/file/
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort | diff -u tar1.txt -
rm -r /tmp/debian-chroot

View file

@ -1,50 +0,0 @@
#!/bin/sh
#
# to test foreign architecture package installation we choose a package which
# - is not part of the native installation set
# - does not have any dependencies
# - installs only few files
# - doesn't change its name regularly (like gcc-*-base)
case "$(dpkg --print-architecture)" in
arm64)
native_arch=arm64
foreign_arch=amd64
;;
amd64)
native_arch=amd64
foreign_arch=arm64
;;
*)
echo "unsupported native architecture" >&2
exit 1
;;
esac
set -eu
export LC_ALL=C.UTF-8
{{ CMD }} --mode=root --variant=apt \
--architectures="$native_arch" \
--architectures="$foreign_arch" \
--include="libmagic-mgc:$foreign_arch" \
{{ DIST }} /tmp/debian-chroot {{ MIRROR }}
{
echo "$native_arch"
echo "$foreign_arch"
} | cmp /tmp/debian-chroot/var/lib/dpkg/arch -
rm /tmp/debian-chroot/usr/lib/file/magic.mgc
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/README.Debian
rm -f /tmp/debian-chroot/usr/share/doc/libmagic-mgc/"changelog.Debian.$foreign_arch.gz"
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/changelog.Debian.gz
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/changelog.gz
rm /tmp/debian-chroot/usr/share/doc/libmagic-mgc/copyright
rm /tmp/debian-chroot/usr/share/file/magic.mgc
rm /tmp/debian-chroot/usr/share/misc/magic.mgc
rm /tmp/debian-chroot/var/lib/dpkg/info/libmagic-mgc.list
rm /tmp/debian-chroot/var/lib/dpkg/info/libmagic-mgc.md5sums
rmdir /tmp/debian-chroot/usr/share/doc/libmagic-mgc/
rmdir /tmp/debian-chroot/usr/share/file/magic/
rmdir /tmp/debian-chroot/usr/share/file/
rmdir /tmp/debian-chroot/usr/lib/file/
tar -C /tmp/debian-chroot --one-file-system -c . | tar -t | sort | diff -u tar1.txt -
rm -r /tmp/debian-chroot

View file

@ -1,10 +0,0 @@
#!/bin/sh
#
# This checks for https://bugs.debian.org/976166
# Since $DEFAULT_DIST varies, we hardcode stable and unstable.
set -eu
export LC_ALL=C.UTF-8
trap "rm -rf /tmp/debian-chroot" EXIT INT TERM
{{ CMD }} --mode=root --variant=minbase --include=doc-debian unstable /tmp/debian-chroot "deb {{ MIRROR }} unstable main" "deb {{ MIRROR }} stable main"
chroot /tmp/debian-chroot dpkg-query --show doc-debian

Some files were not shown because too many files have changed in this diff Show more