overcome dpkg restrictions across distributions #31

Unfortunately, we're unable to setup Ubuntu chroots with releases newer than 20.04 "Focal Fossa" while running mmdebstrap in Debian.

$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux bookworm/sid
Release:        n/a
Codename:       bookworm

$ mmdebstrap --format=directory --variant=apt --architectures=amd64 focal ~/mm-ubuntu-20.04-amd64 http://mirror.yandex.ru/ubuntu/
I: automatically chosen mode: unshare
I: chroot architecture amd64 is equal to the host's architecture
I: running apt-get update...
done
I: downloading packages with apt...
done
I: extracting archives...
done
I: installing essential packages...
done
I: cleaning package lists and apt cache...
done
done
I: success in 15.9259 seconds

$ mmdebstrap --format=directory --variant=apt --architectures=amd64 jammy ~/mm-ubuntu-22.04-amd64 http://mirror.yandex.ru/ubuntu/
I: automatically chosen mode: unshare
I: chroot architecture amd64 is equal to the host's architecture
I: running apt-get update...
done
I: downloading packages with apt...
done
I: extracting archives...
dpkg-deb: error: archive '/home/krd/mm-ubuntu-22.04-amd64//var/cache/apt/archives/gcc-12-base_12-20220319-1ubuntu1_amd64.deb' uses unknown compression for member 'control.tar.zst', giving up
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
E: dpkg-deb --fsys-tarfile failed: 512
I: main() received signal PIPE: waiting for setup...
E: mmdebstrap failed to run

The reason is that packages (in Ubuntu newer that 20.04) use zstd compression which is not supported by Debian.

Unfortunately, we're unable to setup Ubuntu chroots with releases newer than 20.04 "Focal Fossa" while running mmdebstrap in Debian. ``` $ lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux bookworm/sid Release: n/a Codename: bookworm $ mmdebstrap --format=directory --variant=apt --architectures=amd64 focal ~/mm-ubuntu-20.04-amd64 http://mirror.yandex.ru/ubuntu/ I: automatically chosen mode: unshare I: chroot architecture amd64 is equal to the host's architecture I: running apt-get update... done I: downloading packages with apt... done I: extracting archives... done I: installing essential packages... done I: cleaning package lists and apt cache... done done I: success in 15.9259 seconds $ mmdebstrap --format=directory --variant=apt --architectures=amd64 jammy ~/mm-ubuntu-22.04-amd64 http://mirror.yandex.ru/ubuntu/ I: automatically chosen mode: unshare I: chroot architecture amd64 is equal to the host's architecture I: running apt-get update... done I: downloading packages with apt... done I: extracting archives... dpkg-deb: error: archive '/home/krd/mm-ubuntu-22.04-amd64//var/cache/apt/archives/gcc-12-base_12-20220319-1ubuntu1_amd64.deb' uses unknown compression for member 'control.tar.zst', giving up tar: This does not look like a tar archive tar: Exiting with failure status due to previous errors E: dpkg-deb --fsys-tarfile failed: 512 I: main() received signal PIPE: waiting for setup... E: mmdebstrap failed to run ``` The reason is that packages (in Ubuntu newer that 20.04) use zstd compression which is not supported by Debian.

I'd suggest to write simple (so far) wrapper script and replace direct call to dpkg-deb --fsys-tarfile with it. Hovewer, this script would require additional runtime dependencies like ar (from binutils package) and zstd (eponymous package).

I'd suggest to write simple (so far) wrapper script and replace direct call to `dpkg-deb --fsys-tarfile` with it. Hovewer, this script would require additional runtime dependencies like `ar` (from `binutils` package) and `zstd` (eponymous package).

Thank you for your merge request, but no. Two reasons:

Ubuntu knew full well what they were doing when they made *.deb packages in Ubuntu incompatible with dpkg in Debian. They did not do the right thing and knew full well what they were doing but went ahead anyways, using their superior market share to push a feature before it was ready. This is not okay. And now others have to pay the price because of their wrong decisions? No, thank you. I'm not getting bullied into applying a bad hack into my software that I maintain in my free time by some company that makes money from their wrong decisions.
guillem plans to add zstd decompression support into dpkg before the imminent freeze: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=892664#129

Thank you for your merge request, but no. Two reasons: - Ubuntu knew full well what they were doing when they made `*.deb` packages in Ubuntu incompatible with dpkg in Debian. They did not do the right thing and knew full well what they were doing but went ahead anyways, using their superior market share to push a feature before it was ready. This is not okay. And now others have to pay the price because of their wrong decisions? No, thank you. I'm not getting bullied into applying a bad hack into my software that I maintain in my free time by some company that makes money from their wrong decisions. - guillem plans to add zstd decompression support into dpkg before the imminent freeze: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=892664#129

Sorry, as a clarification, I do not think that you, @rockdrilla, are doing any bullying. Thank you really for also putting your free time into mmdebstrap. I meant that the company canononical is doing the bullying.

Im certainly agree with your point of view except one moment: only fresh Debian 12 "Bookworm" (and beyond) installations will receive support for zstd-compressed (foreign) packages and less likely - older installations like Debian 11 "Bullseye" (backporting such functionality is not an issue for maintainers).

Yes, Debian 11 Bullseye will not have this functionality, sadly. But even if we add zstd support to mmdebstrap today, it will change nothing for Debian 11 Bullseye. It's not feasible to backport mmdebstrap to Bullseye, because it requires a more recent version of apt than Bullseye provides and backporting apt is out of question.

Well, I've no more questions left. I'd suggest to leave this issue and linked MR open for year or two until they are obsolete and then close both. Many thanks for clarification! :)

UPD: maybe add some words about Ubuntu and zstd-compressed packages in README.md?

Well, I've no more questions left. I'd suggest to leave this issue and linked MR open for year or two until they are obsolete and then close both. Many thanks for clarification! :) UPD: maybe add some words about Ubuntu and zstd-compressed packages in `README.md`?

backporting apt is out of question

Haven't tried it at all yet, but it should not be too hard to run apt out of a git checkout with a bit of extra config and PATH trickery. I mean, I am doing it on my main system for all my normal apt needs… I just never explicitly tried how much of it works with mmdebstrap thrown into the mix so far (I tend to only need changes for apt inside the chroot, not outside, so I just know that it isn't downright exploding). Would be kinda cool (if a bit pointless) to weaken the current hard-dependency on a somewhat Debian-like system this way (or, you know, turn it up to eleven and bootstrap dpkg and apt from deb packages to then bootstrap a normal chroot with that on a completely foreign distro).

Anyway, if all you want is just wrapping dpkg-deb that should be easy enough to do with PATH or did I miss something fundamentally? Conceptionally that would feel more like one of the optional hook scripts rather than an eternally builtin workaround for a loaded topic.

> backporting apt is out of question Haven't tried it at all yet, but it *should* not be too hard to run apt out of a git checkout with a bit of extra config and `PATH` trickery. I mean, I am doing it on my main system for all my *normal* apt needs… I just never explicitly tried how much of it works with `mmdebstrap` thrown into the mix so far (I tend to only need changes for apt inside the chroot, not outside, so I just know that it isn't downright exploding). Would be kinda cool (if a bit pointless) to weaken the current hard-dependency on a somewhat Debian-like system this way (or, you know, turn it up to eleven and bootstrap dpkg and apt from deb packages to then bootstrap a normal chroot with that on a completely foreign distro). Anyway, if all you want is just wrapping `dpkg-deb` that should be easy enough to do with `PATH` or did I miss something fundamentally? Conceptionally that would feel more like one of the optional hook scripts rather than an eternally builtin workaround for a loaded topic.

@DonKult, you're right - it's better to do partial override via PATH: https://github.com/rockdrilla/debian-container/blob/e86d1f2e/minbase/tarball.sh#L29

Wrapper script looks like this: https://github.com/rockdrilla/debian-container/blob/d11c67c9/minbase/setup/dpkg-deb

@DonKult, you're right - it's better to do partial override via PATH: https://github.com/rockdrilla/debian-container/blob/e86d1f2e/minbase/tarball.sh#L29 Wrapper script looks like this: https://github.com/rockdrilla/debian-container/blob/d11c67c9/minbase/setup/dpkg-deb

Labels Milestones

overcome dpkg restrictions across distributions #31