I have had issues with running native apt-get to install packages from hooks, and I have workarounded it by running apt-get from chroot. However, I just found out that Debian 11 essential rootfs cannot handle https apt repos. This results in a build failure, and the necessary package (ca-certificates) now has to be installed from outside.
The resulting execution makes me feel like it is trying to install to my Ubuntu host environment:
I: running --essential-hook in shell: sh -c 'cat "$MMDEBSTRAP_APT_CONFIG"
APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get install -y apt-transport-https ca-certificates
' exec /tmp/mmdebstrap.jX9H6qqvEp
Apt::Architecture "arm64";
Apt::Architectures "arm64";
Dir "/tmp/mmdebstrap.jX9H6qqvEp";
Dir::State::Status "/tmp/mmdebstrap.jX9H6qqvEp/var/lib/dpkg/status";
Dir::Etc::Trusted "/etc/apt/trusted.gpg";
Dir::Etc::TrustedParts "/tmp/tmp.YjWaoQQLzd/keyrings";
Reading package lists... Done
Building dependency tree... Done
The following additional packages will be installed:
openssl
The following NEW packages will be installed:
apt-transport-https ca-certificates openssl
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 1155 kB of archives.
After this operation, 2039 kB of additional disk space will be used.
Get:1 https://deb.debian.org/debian bullseye/main arm64 openssl arm64 1.1.1w-0+deb11u1 [837 kB]
Get:2 https://deb.debian.org/debian bullseye/main arm64 ca-certificates all 20210119 [158 kB]
Get:3 https://deb.debian.org/debian bullseye/main arm64 apt-transport-https all 2.2.4 [160 kB]
Fetched 1155 kB in 1s (1312 kB/s)
Preconfiguring packages ...
(Reading database ... 22499 files and directories currently installed.)
Preparing to unpack .../openssl_1.1.1w-0+deb11u1_arm64.deb ...
Unpacking openssl:arm64 (1.1.1w-0+deb11u1) over (1.1.1w-0+deb11u1) ...
Preparing to unpack .../ca-certificates_20210119_all.deb ...
Unpacking ca-certificates (20210119) over (20210119) ...
Preparing to unpack .../apt-transport-https_2.2.4_all.deb ...
Unpacking apt-transport-https (2.2.4) over (2.2.4) ...
Setting up apt-transport-https (2.2.4) ...
dpkg: dependency problems prevent configuration of openssl:arm64:
openssl:arm64 depends on libssl1.1 (>= 1.1.1).
dpkg: error processing package openssl:arm64 (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of ca-certificates:
ca-certificates depends on openssl (>= 1.1.1); however:
Package openssl:arm64 is not configured yet.
dpkg: error processing package ca-certificates (--configure):
dependency problems - leaving unconfigured
Processing triggers for man-db (2.10.2-1) ...
Errors were encountered while processing:
openssl:arm64
ca-certificates
E: Sub-process /usr/bin/dpkg returned an error code (1)
E: run_chroot failed: E: command failed: cat "$MMDEBSTRAP_APT_CONFIG"
APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get install -y apt-transport-https ca-certificates
W: listening on child socket failed:
I: removing tempdir /tmp/mmdebstrap.jX9H6qqvEp...
E: mmdebstrap failed to run
My host arch is x86-64, and I'm trying to build for arm64, which might be why dpkg is not providing very helpful error messages. Usually it will tell you the reason for "dependency problems prevent configuration".
I have configured binfmt to run cross platform binaries, and mmdebstrap worked for all other tasks (including cross compiling dkms modules using native arm64 gcc), so I'm really out of idea why this happens.
I have had issues with running native `apt-get` to install packages from hooks, and I have workarounded it by running `apt-get` from `chroot`. However, I just found out that [Debian 11 essential rootfs cannot handle https apt repos](https://github.com/RadxaOS-SDK/rsdk/issues/2). This results in a build failure, and the necessary package (`ca-certificates`) now has to be installed from outside.
Here is what I defined in my jsonnet template:
```
...
"essential-hooks"+:
[
|||
cat "$MMDEBSTRAP_APT_CONFIG"
APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get install -y apt-transport-https ca-certificates
|||,
...
```
which got garbled by bdebstrap as:
```
...
chroot "$1" u-boot-update], essential-hooks: ["cat \"$MMDEBSTRAP_APT_CONFIG\"\n\
APT_CONFIG=\"$MMDEBSTRAP_APT_CONFIG\" apt-get install -y apt-transport-https\
\ ca-certificates\n", "chroot \"$1\" \\\nbash -c \"DEBIAN_FRONTEND=noninteractive\
...
```
The resulting execution makes me feel like it is trying to install to my Ubuntu host environment:
```
I: running --essential-hook in shell: sh -c 'cat "$MMDEBSTRAP_APT_CONFIG"
APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get install -y apt-transport-https ca-certificates
' exec /tmp/mmdebstrap.jX9H6qqvEp
Apt::Architecture "arm64";
Apt::Architectures "arm64";
Dir "/tmp/mmdebstrap.jX9H6qqvEp";
Dir::State::Status "/tmp/mmdebstrap.jX9H6qqvEp/var/lib/dpkg/status";
Dir::Etc::Trusted "/etc/apt/trusted.gpg";
Dir::Etc::TrustedParts "/tmp/tmp.YjWaoQQLzd/keyrings";
Reading package lists... Done
Building dependency tree... Done
The following additional packages will be installed:
openssl
The following NEW packages will be installed:
apt-transport-https ca-certificates openssl
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 1155 kB of archives.
After this operation, 2039 kB of additional disk space will be used.
Get:1 https://deb.debian.org/debian bullseye/main arm64 openssl arm64 1.1.1w-0+deb11u1 [837 kB]
Get:2 https://deb.debian.org/debian bullseye/main arm64 ca-certificates all 20210119 [158 kB]
Get:3 https://deb.debian.org/debian bullseye/main arm64 apt-transport-https all 2.2.4 [160 kB]
Fetched 1155 kB in 1s (1312 kB/s)
Preconfiguring packages ...
(Reading database ... 22499 files and directories currently installed.)
Preparing to unpack .../openssl_1.1.1w-0+deb11u1_arm64.deb ...
Unpacking openssl:arm64 (1.1.1w-0+deb11u1) over (1.1.1w-0+deb11u1) ...
Preparing to unpack .../ca-certificates_20210119_all.deb ...
Unpacking ca-certificates (20210119) over (20210119) ...
Preparing to unpack .../apt-transport-https_2.2.4_all.deb ...
Unpacking apt-transport-https (2.2.4) over (2.2.4) ...
Setting up apt-transport-https (2.2.4) ...
dpkg: dependency problems prevent configuration of openssl:arm64:
openssl:arm64 depends on libssl1.1 (>= 1.1.1).
dpkg: error processing package openssl:arm64 (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of ca-certificates:
ca-certificates depends on openssl (>= 1.1.1); however:
Package openssl:arm64 is not configured yet.
dpkg: error processing package ca-certificates (--configure):
dependency problems - leaving unconfigured
Processing triggers for man-db (2.10.2-1) ...
Errors were encountered while processing:
openssl:arm64
ca-certificates
E: Sub-process /usr/bin/dpkg returned an error code (1)
E: run_chroot failed: E: command failed: cat "$MMDEBSTRAP_APT_CONFIG"
APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get install -y apt-transport-https ca-certificates
W: listening on child socket failed:
I: removing tempdir /tmp/mmdebstrap.jX9H6qqvEp...
E: mmdebstrap failed to run
```
My host arch is x86-64, and I'm trying to build for arm64, which might be why dpkg is not providing very helpful error messages. Usually it will tell you the reason for "dependency problems prevent configuration".
I have configured binfmt to run cross platform binaries, and mmdebstrap worked for all other tasks (including cross compiling dkms modules using native arm64 gcc), so I'm really out of idea why this happens.
Uff... there are multiple things to unpack here...
i have no clue about bdebstrap so i cannot tell you how to use it correctly to do what you want
you do not have to install apt-transport-https as that is provided by apt since apt 1.5, so this is even unnecessary in old-old-stable (buster)
are you running bdebstrap as superuser? Because if you are not, then this cannot install anything on your ubuntu host and will fail much earlier with a permission denied error
i'm not running ubuntu, so in case this is the problem, i'll not be able to reproduce it
when filing a bug, please take some effort to provide a minimal reproducer
to reproduce this, please craft an mmdebstrap invocation that fails for you instead of not even showing your bdebstrap yaml
you do not need to fake-cross compile dkms modules by building them natively under qemu. This just wastes resources and is super slow. It is perfectly possible to cross-build dkms modules from amd64 for arm64 which will be about 20 times faster than taking the fake-cross via emulated native approach
why are you not using the --include option? The mmdebstrap man page says this:
If you specify a https or tor MIRROR and you want the chroot to be able to update itself, don't forget to also install the ca-certificates package, the apt-transport-https package for apt versions less than 1.5 and/or the apt-transport-tor package using the --include option, as necessary.
Uff... there are multiple things to unpack here...
1. i have no clue about bdebstrap so i cannot tell you how to use it correctly to do what you want
2. you do not have to install `apt-transport-https` as that is provided by `apt` since apt 1.5, so this is even unnecessary in old-old-stable (buster)
3. are you running bdebstrap as superuser? Because if you are not, then this cannot install anything on your ubuntu host and will fail much earlier with a permission denied error
4. i'm not running ubuntu, so in case this is the problem, i'll not be able to reproduce it
5. when filing a bug, please take some effort to provide a minimal reproducer
6. to reproduce this, please craft an mmdebstrap invocation that fails for you instead of not even showing your bdebstrap yaml
7. you do not need to fake-cross compile dkms modules by building them natively under qemu. This just wastes resources and is super slow. It is perfectly possible to cross-build dkms modules from amd64 for arm64 which will be about 20 times faster than taking the fake-cross via emulated native approach
8. why are you not using the `--include` option? The mmdebstrap man page says this:
> If you specify a https or tor MIRROR and you want the chroot to be able to update itself, don't forget to also install the ca-certificates package, the apt-transport-https package for apt versions less than 1.5 and/or the apt-transport-tor package using the --include option, as necessary.
Thanks for your prompt reply. It was pretty late last night so I figure I would just describe my immediate symptom first (unable to apt-get from hook) in case I missed something obvious.
Uff... there are multiple things to unpack here...
i have no clue about bdebstrap so i cannot tell you how to use it correctly to do what you want
Fair point. But in this case I think bdebstrap correctly transformed my input to mmdebstrap command invocation, The log output from mmdebstrap also looks like it is trying to do what I specified, but failed unexpectedly.
you do not have to install apt-transport-https as that is provided by apt since apt 1.5, so this is even unnecessary in old-old-stable (buster)
This is a leftover from our previous source code. I double checked the oldest Ubuntu release we might need to support and it is not needed anymore. Thanks.
are you running bdebstrap as superuser? Because if you are not, then this cannot install anything on your ubuntu host and will fail much earlier with a permission denied error
Yes I am. I also plan to package mmdebstrap for NixOS which the last time I tried I can only get root mode working, so this is what I'm sticking with for now. Ultimate plan is to support our system builder running on NixOS natively.
i'm not running ubuntu, so in case this is the problem, i'll not be able to reproduce it
Understand. We just have to use it to install stupid Ubuntu zstd packages. I saw in another issue that a user has a wrapper for this, so we might reconsider this when we encounter other Ubuntu induced issues.
when filing a bug, please take some effort to provide a minimal reproducer
It was late so I only had the relevant code snippets around the apt-get invocation. I'll try prepare something for you today on Debian.
to reproduce this, please craft an mmdebstrap invocation that fails for you instead of not even showing your bdebstrap yaml
bdepstrap generated YAML is in my opinion formatted in the wrong style. The resulting file is hard to read which is why I did not include it. I plan to make a PR with them today.
Here is the generated config.yaml in case you want to know how bad it is. I do not expect you to troubleshoot this file though.
you do not need to fake-cross compile dkms modules by building them natively under qemu. This just wastes resources and is super slow. It is perfectly possible to cross-build dkms modules from amd64 for arm64 which will be about 20 times faster than taking the fake-cross via emulated native approach
We have our dkms source packaged as debs. Our current issue is that they are not installed directly but as Recommends of another metapackages, and installation is controlled by --no-install-recommends|--install-recommends from mmdebstrap hooks. But as we cannot run apt-get the normal way (the reason we opened this issue), we have to chroot to install in slow mode.
It is slow but it is not blocking, since we currently use go-debos and I believe that's the only way for it to build dkms. I'm not touching that package dependency for now since our existing code is still using it. In the future once we fully switch to mmdebstrap (via bdebsrap) we will definitely check if we can just --include them in a separate package to get additional build speed increase.
why are you not using the --include option? The mmdebstrap man page says this:
If you specify a https or tor MIRROR and you want the chroot to be able to update itself, don't forget to also install the ca-certificates package, the apt-transport-https package for apt versions less than 1.5 and/or the apt-transport-tor package using the --include option, as necessary.
bdebstrap does translate that into --include. The issue is that it is not an essential package, and I want to use apt-get in essential-hooks. We use apt variant, so apt-get is available at that stage, but not ca-certificates.
I'll check if I can get some reproducible demo for you today. It is my last work day before Chinese New Year though, so if I can't get it done today I will be back in 2 weeks.
Thanks for your prompt reply. It was pretty late last night so I figure I would just describe my immediate symptom first (unable to apt-get from hook) in case I missed something obvious.
> Uff... there are multiple things to unpack here...
>
> 1. i have no clue about bdebstrap so i cannot tell you how to use it correctly to do what you want
Fair point. But in this case I think bdebstrap correctly transformed my input to mmdebstrap command invocation, The log output from mmdebstrap also looks like it is trying to do what I specified, but failed unexpectedly.
> 2. you do not have to install `apt-transport-https` as that is provided by `apt` since apt 1.5, so this is even unnecessary in old-old-stable (buster)
This is a leftover from our previous source code. I double checked the oldest Ubuntu release we might need to support and it is not needed anymore. Thanks.
> 3. are you running bdebstrap as superuser? Because if you are not, then this cannot install anything on your ubuntu host and will fail much earlier with a permission denied error
Yes I am. I also plan to [package mmdebstrap for NixOS](https://github.com/NixOS/nixpkgs/pull/270095) which the last time I tried I can only get root mode working, so this is what I'm sticking with for now. Ultimate plan is to support our system builder running on NixOS natively.
> 4. i'm not running ubuntu, so in case this is the problem, i'll not be able to reproduce it
Understand. We just have to use it to install stupid Ubuntu zstd packages. I saw in another issue that a user has a wrapper for this, so we might reconsider this when we encounter other Ubuntu induced issues.
> 5. when filing a bug, please take some effort to provide a minimal reproducer
It was late so I only had the relevant code snippets around the apt-get invocation. I'll try prepare something for you today on Debian.
> 6. to reproduce this, please craft an mmdebstrap invocation that fails for you instead of not even showing your bdebstrap yaml
bdepstrap [generated YAML](https://github.com/bdrung/bdebstrap/blob/main/bdebstrap#L737-L740) is in my opinion [formatted in the wrong style](https://github.com/yaml/pyyaml/issues/199#issuecomment-420425200). The resulting file is hard to read which is why I did not include it. I plan to make a PR with them today.
Here is the generated [config.yaml](/attachments/9216bfc7-a6ac-489d-a454-95dcf2a307ee) in case you want to know how bad it is. I do not expect you to troubleshoot this file though.
> 7. you do not need to fake-cross compile dkms modules by building them natively under qemu. This just wastes resources and is super slow. It is perfectly possible to cross-build dkms modules from amd64 for arm64 which will be about 20 times faster than taking the fake-cross via emulated native approach
We have our dkms source packaged as debs. Our current issue is that they are not installed directly but as Recommends of another metapackages, and installation is controlled by `--no-install-recommends`|`--install-recommends` from mmdebstrap hooks. But as we cannot run apt-get the normal way (the reason we opened this issue), we have to chroot to install in slow mode.
It is slow but it is not blocking, since we currently use go-debos and I believe that's the only way for it to build dkms. I'm not touching that package dependency for now since our existing code is still using it. In the future once we fully switch to mmdebstrap (via bdebsrap) we will definitely check if we can just `--include` them in a separate package to get additional build speed increase.
> 8. why are you not using the `--include` option? The mmdebstrap man page says this:
>
> > If you specify a https or tor MIRROR and you want the chroot to be able to update itself, don't forget to also install the ca-certificates package, the apt-transport-https package for apt versions less than 1.5 and/or the apt-transport-tor package using the --include option, as necessary.
bdebstrap does translate [that into `--include`](https://github.com/bdrung/bdebstrap/blob/main/bdebstrap#L108-L109). The issue is that it is not an essential package, and I want to use apt-get in essential-hooks. We use apt variant, so apt-get is available at that stage, but not ca-certificates.
I'll check if I can get some reproducible demo for you today. It is my last work day before Chinese New Year though, so if I can't get it done today I will be back in 2 weeks.
Thanks for your prompt reply. It was pretty late last night so I figure I would just describe my immediate symptom first (unable to apt-get from hook) in case I missed something obvious.
Well... in your initial report you also did not state which version of mmdebstrap you are working with. This is rather essential for a bug report, no? If your version of mmdebstrap is older than 1.4.1, then you also need to include this apt option when trying to run apt from outside the chroot: -oDPkg::Chroot-Directory="$1" so the complete call looks like this:
There is a test-case which runs this, so this is supposed to work and tested to work.
are you running bdebstrap as superuser? Because if you are not, then this cannot install anything on your ubuntu host and will fail much earlier with a permission denied error
Yes I am. I also plan to package mmdebstrap for NixOS which the last time I tried I can only get root mode working, so this is what I'm sticking with for now. Ultimate plan is to support our system builder running on NixOS natively.
I'm not sure whether this can work. mmdebstrap is not designed on distributions that do not use apt as their package manager. But maybe it will work with only minimal changes. Please open an issue if you run into problems.
Reading the diff it strikes me as odd that you tried fakechroot mode? I don't think this can ever work on nixos as the libraries used outside and inside the chroot (first and foremost glibc itself) will be utterly incompatible.
I would also like to discourage you from using root mode. The code that is run as part of the creation of a chroot can be utterly dangerous for your host system. There are debootstrap bugs where users got their whole $HOME wiped when running it. One of the main reasons for using mmdebstrap over debootstrap is that the former does not require superuser privileges. Please have a look at the unshare mode which uses linux user namespaces to avoid ever becoming the superuser. This will make sure that it does not damage your system. Even fakechroot mode can wipe your $HOME. Unshare mode cannot. Please try and see if you can make that work on NixOS.
i'm not running ubuntu, so in case this is the problem, i'll not be able to reproduce it
Understand. We just have to use it to install stupid Ubuntu zstd packages. I saw in another issue that a user has a wrapper for this, so we might reconsider this when we encounter other Ubuntu induced issues.
The latest Ubuntu induced issue is, that they removed the usrmerge package so by default, Ubuntu chroots are now not using merged-/usr again. This is a bug in Ubuntu that mmdebstrap cannot fix and it's not the first such bug. The zstd mess is another example. I'd recommend against using Ubuntu with mmdebstrap.
Here is the generated config.yaml in case you want to know how bad it is. I do not expect you to troubleshoot this file though.
I have a couple of comments for that one. Are you doing this work for your work at a company or as your hobby in your free time?
I'll check if I can get some reproducible demo for you today. It is my last work day before Chinese New Year though, so if I can't get it done today I will be back in 2 weeks.
In that case: happy new year celebration! I'm looking forward to your reply.
> Thanks for your prompt reply. It was pretty late last night so I figure I would just describe my immediate symptom first (unable to apt-get from hook) in case I missed something obvious.
Well... in your initial report you also did not state which version of mmdebstrap you are working with. This is rather essential for a bug report, no? If your version of mmdebstrap is older than 1.4.1, then you also need to include this apt option when trying to run apt from outside the chroot: `-oDPkg::Chroot-Directory="$1"` so the complete call looks like this:
--essential-hook='APT_CONFIG=$MMDEBSTRAP_APT_CONFIG apt-get --yes install -oDPkg::Chroot-Directory="$1" ca-certificates'
There is a test-case which runs this, so this is supposed to work and tested to work.
> > 3. are you running bdebstrap as superuser? Because if you are not, then this cannot install anything on your ubuntu host and will fail much earlier with a permission denied error
>
> Yes I am. I also plan to [package mmdebstrap for NixOS](https://github.com/NixOS/nixpkgs/pull/270095) which the last time I tried I can only get root mode working, so this is what I'm sticking with for now. Ultimate plan is to support our system builder running on NixOS natively.
I'm not sure whether this can work. `mmdebstrap` is not designed on distributions that do not use apt as their package manager. But maybe it will work with only minimal changes. Please open an issue if you run into problems.
Reading the diff it strikes me as odd that you tried fakechroot mode? I don't think this can ever work on nixos as the libraries used outside and inside the chroot (first and foremost glibc itself) will be utterly incompatible.
I would also like to discourage you from using root mode. The code that is run as part of the creation of a chroot can be utterly dangerous for your host system. There are debootstrap bugs where users got their whole $HOME wiped when running it. One of the main reasons for using mmdebstrap over debootstrap is that the former does not require superuser privileges. Please have a look at the unshare mode which uses linux user namespaces to avoid ever becoming the superuser. This will make sure that it does not damage your system. Even fakechroot mode can wipe your $HOME. Unshare mode cannot. Please try and see if you can make that work on NixOS.
> > 4. i'm not running ubuntu, so in case this is the problem, i'll not be able to reproduce it
>
> Understand. We just have to use it to install stupid Ubuntu zstd packages. I saw in another issue that a user has a wrapper for this, so we might reconsider this when we encounter other Ubuntu induced issues.
The latest Ubuntu induced issue is, that they removed the usrmerge package so by default, Ubuntu chroots are now not using merged-/usr again. This is a bug in Ubuntu that mmdebstrap cannot fix and it's not the first such bug. The zstd mess is another example. I'd recommend against using Ubuntu with mmdebstrap.
> Here is the generated [config.yaml](/attachments/9216bfc7-a6ac-489d-a454-95dcf2a307ee) in case you want to know how bad it is. I do not expect you to troubleshoot this file though.
I have a couple of comments for that one. Are you doing this work for your work at a company or as your hobby in your free time?
> I'll check if I can get some reproducible demo for you today. It is my last work day before Chinese New Year though, so if I can't get it done today I will be back in 2 weeks.
In that case: happy new year celebration! I'm looking forward to your reply.
Well... in your initial report you also did not state which version of mmdebstrap you are working with. This is rather essential for a bug report, no? If your version of mmdebstrap is older than 1.4.1, then you also need to include this apt option when trying to run apt from outside the chroot: -oDPkg::Chroot-Directory="$1" so the complete call looks like this:
Thanks. This is exactly the issue.
FYI this is the mmdebstrap version some common distros are shipping:
Distro
Version
Arch (AUR)
1.3.8
Debian 12
1.3.5
Ubuntu 22.04.3 (what I'm using)
0.8.4
So some clarification might be helpful to discover this (common?) usage, since most of users are likely on outdated versions for quite some time.
The hint I was getting is from the man's OPERATION section, where apt-get seemingly works with only APT_CONFIG exported and no magic arguments is needed.
It is my fault to assume a download-only apt-get call works the same for one that needs to also configure the package, but I hope you can see why I was confused when I was expecting the command to just work.
I'm not sure whether this can work. mmdebstrap is not designed on distributions that do not use apt as their package manager. But maybe it will work with only minimal changes. Please open an issue if you run into problems.
There are some standing issues with running apt on NixOS that I fixed on that PR.
Reading the diff it strikes me as odd that you tried fakechroot mode? I don't think this can ever work on nixos as the libraries used outside and inside the chroot (first and foremost glibc itself) will be utterly incompatible.
I would also like to discourage you from using root mode. The code that is run as part of the creation of a chroot can be utterly dangerous for your host system. There are debootstrap bugs where users got their whole $HOME wiped when running it. One of the main reasons for using mmdebstrap over debootstrap is that the former does not require superuser privileges. Please have a look at the unshare mode which uses linux user namespaces to avoid ever becoming the superuser. This will make sure that it does not damage your system. Even fakechroot mode can wipe your $HOME. Unshare mode cannot. Please try and see if you can make that work on NixOS.
I basically tested all supported modes since I really don't want to use the root mode for obvious reasons.
I do intend to get unshare to work eventually, but getting root mode to work at least clears me to package bdebstrap for NixOS, and run our system builder on it.
The latest Ubuntu induced issue is, that they removed the usrmerge package so by default, Ubuntu chroots are now not using merged-/usr again. This is a bug in Ubuntu that mmdebstrap cannot fix and it's not the first such bug. The zstd mess is another example. I'd recommend against using Ubuntu with mmdebstrap.
Do you mean the generated rootfs for Ubuntu is broken due to missing usrmerge right? I don't quite understand how Ubuntu host will have impact in a Debian chroot environment.
I have a couple of comments for that one. Are you doing this work for your work at a company or as your hobby in your free time?
The main reason is that go-debos does not fully support ARM64 due to its fakemachine dependency, while we produce primarily ARM64 products, and customers want to build systems natively.
We also look to support RISC-V systems in the future, and Debian Wiki recommends this project, which is why I did not go straight with debootstrap (which I have considered quite a few times when working with go-debos).
Right now the new project is not announced, and is aiming to be a direct translation of the old project, which is a big hack by itself. If you have any suggestions I'm more than happy to hear it.
You can contact me directly via yt@radxa.com if here is not an appropriate place for that kind of discussion. GitHub issues within our repos are also welcome if you want to keep it open.
As the original issue is now resolved (due to missing argument). I'll also close this issue to reduce the noise from issue list.
> Well... in your initial report you also did not state which version of mmdebstrap you are working with. This is rather essential for a bug report, no? If your version of mmdebstrap is older than 1.4.1, then you also need to include this apt option when trying to run apt from outside the chroot: `-oDPkg::Chroot-Directory="$1"` so the complete call looks like this:
Thanks. This is exactly the issue.
FYI this is the mmdebstrap version some common distros are shipping:
| Distro | Version |
| -------- | -------- |
| Arch (AUR) | 1.3.8 |
| Debian 12 | 1.3.5 |
| Ubuntu 22.04.3 (what I'm using) | 0.8.4 |
So some clarification might be helpful to discover this (common?) usage, since most of users are likely on outdated versions for quite some time.
The hint I was getting is from the man's OPERATION section, where apt-get *seemingly* works with only APT_CONFIG exported and no magic arguments is needed.
It is my fault to assume a download-only apt-get call works the same for one that needs to also configure the package, but I hope you can see why I was confused when I was expecting the command to just work.
> I'm not sure whether this can work. mmdebstrap is not designed on distributions that do not use apt as their package manager. But maybe it will work with only minimal changes. Please open an issue if you run into problems.
There are some standing issues with running apt on NixOS that I fixed on that PR.
> Reading the diff it strikes me as odd that you tried fakechroot mode? I don't think this can ever work on nixos as the libraries used outside and inside the chroot (first and foremost glibc itself) will be utterly incompatible.
>
> I would also like to discourage you from using root mode. The code that is run as part of the creation of a chroot can be utterly dangerous for your host system. There are debootstrap bugs where users got their whole $HOME wiped when running it. One of the main reasons for using mmdebstrap over debootstrap is that the former does not require superuser privileges. Please have a look at the unshare mode which uses linux user namespaces to avoid ever becoming the superuser. This will make sure that it does not damage your system. Even fakechroot mode can wipe your $HOME. Unshare mode cannot. Please try and see if you can make that work on NixOS.
I basically tested all supported modes since I really don't want to use the root mode for obvious reasons.
I do intend to get unshare to work eventually, but getting root mode to work at least clears me to package bdebstrap for NixOS, and run our system builder on it.
> The latest Ubuntu induced issue is, that they removed the usrmerge package so by default, Ubuntu chroots are now not using merged-/usr again. This is a bug in Ubuntu that mmdebstrap cannot fix and it's not the first such bug. The zstd mess is another example. I'd recommend against using Ubuntu with mmdebstrap.
Do you mean the generated rootfs for Ubuntu is broken due to missing usrmerge right? I don't quite understand how Ubuntu host will have impact in a Debian chroot environment.
> I have a couple of comments for that one. Are you doing this work for your work at a company or as your hobby in your free time?
Yes this is for [a work project](https://github.com/RadxaOS-SDK/rsdk). I want to use jsonnet+bdebstrap+mmdebstrap & libguestfs to replace our [current go-debos based solution](https://github.com/radxa-repo/rbuild).
The main reason is that go-debos does not fully support ARM64 due to its fakemachine dependency, while we produce primarily ARM64 products, and customers want to build systems natively.
We also look to support RISC-V systems in the future, and Debian Wiki recommends this project, which is why I did not go straight with debootstrap (which I have considered quite a few times when working with go-debos).
Right now the new project is not announced, and is aiming to be a direct translation of the old project, which is a big hack by itself. If you have any suggestions I'm more than happy to hear it.
You can contact me directly via yt@radxa.com if here is not an appropriate place for that kind of discussion. GitHub issues within our repos are also welcome if you want to keep it open.
---
As the original issue is now resolved (due to missing argument). I'll also close this issue to reduce the noise from issue list.
I have had issues with running native
apt-getto install packages from hooks, and I have workarounded it by runningapt-getfromchroot. However, I just found out that Debian 11 essential rootfs cannot handle https apt repos. This results in a build failure, and the necessary package (ca-certificates) now has to be installed from outside.Here is what I defined in my jsonnet template:
which got garbled by bdebstrap as:
The resulting execution makes me feel like it is trying to install to my Ubuntu host environment:
My host arch is x86-64, and I'm trying to build for arm64, which might be why dpkg is not providing very helpful error messages. Usually it will tell you the reason for "dependency problems prevent configuration".
I have configured binfmt to run cross platform binaries, and mmdebstrap worked for all other tasks (including cross compiling dkms modules using native arm64 gcc), so I'm really out of idea why this happens.
Uff... there are multiple things to unpack here...
apt-transport-httpsas that is provided byaptsince apt 1.5, so this is even unnecessary in old-old-stable (buster)--includeoption? The mmdebstrap man page says this:Thanks for your prompt reply. It was pretty late last night so I figure I would just describe my immediate symptom first (unable to apt-get from hook) in case I missed something obvious.
Fair point. But in this case I think bdebstrap correctly transformed my input to mmdebstrap command invocation, The log output from mmdebstrap also looks like it is trying to do what I specified, but failed unexpectedly.
This is a leftover from our previous source code. I double checked the oldest Ubuntu release we might need to support and it is not needed anymore. Thanks.
Yes I am. I also plan to package mmdebstrap for NixOS which the last time I tried I can only get root mode working, so this is what I'm sticking with for now. Ultimate plan is to support our system builder running on NixOS natively.
Understand. We just have to use it to install stupid Ubuntu zstd packages. I saw in another issue that a user has a wrapper for this, so we might reconsider this when we encounter other Ubuntu induced issues.
It was late so I only had the relevant code snippets around the apt-get invocation. I'll try prepare something for you today on Debian.
bdepstrap generated YAML is in my opinion formatted in the wrong style. The resulting file is hard to read which is why I did not include it. I plan to make a PR with them today.
Here is the generated config.yaml in case you want to know how bad it is. I do not expect you to troubleshoot this file though.
We have our dkms source packaged as debs. Our current issue is that they are not installed directly but as Recommends of another metapackages, and installation is controlled by
--no-install-recommends|--install-recommendsfrom mmdebstrap hooks. But as we cannot run apt-get the normal way (the reason we opened this issue), we have to chroot to install in slow mode.It is slow but it is not blocking, since we currently use go-debos and I believe that's the only way for it to build dkms. I'm not touching that package dependency for now since our existing code is still using it. In the future once we fully switch to mmdebstrap (via bdebsrap) we will definitely check if we can just
--includethem in a separate package to get additional build speed increase.bdebstrap does translate that into
--include. The issue is that it is not an essential package, and I want to use apt-get in essential-hooks. We use apt variant, so apt-get is available at that stage, but not ca-certificates.I'll check if I can get some reproducible demo for you today. It is my last work day before Chinese New Year though, so if I can't get it done today I will be back in 2 weeks.
Well... in your initial report you also did not state which version of mmdebstrap you are working with. This is rather essential for a bug report, no? If your version of mmdebstrap is older than 1.4.1, then you also need to include this apt option when trying to run apt from outside the chroot:
-oDPkg::Chroot-Directory="$1"so the complete call looks like this:There is a test-case which runs this, so this is supposed to work and tested to work.
I'm not sure whether this can work.
mmdebstrapis not designed on distributions that do not use apt as their package manager. But maybe it will work with only minimal changes. Please open an issue if you run into problems.Reading the diff it strikes me as odd that you tried fakechroot mode? I don't think this can ever work on nixos as the libraries used outside and inside the chroot (first and foremost glibc itself) will be utterly incompatible.
I would also like to discourage you from using root mode. The code that is run as part of the creation of a chroot can be utterly dangerous for your host system. There are debootstrap bugs where users got their whole $HOME wiped when running it. One of the main reasons for using mmdebstrap over debootstrap is that the former does not require superuser privileges. Please have a look at the unshare mode which uses linux user namespaces to avoid ever becoming the superuser. This will make sure that it does not damage your system. Even fakechroot mode can wipe your $HOME. Unshare mode cannot. Please try and see if you can make that work on NixOS.
The latest Ubuntu induced issue is, that they removed the usrmerge package so by default, Ubuntu chroots are now not using merged-/usr again. This is a bug in Ubuntu that mmdebstrap cannot fix and it's not the first such bug. The zstd mess is another example. I'd recommend against using Ubuntu with mmdebstrap.
I have a couple of comments for that one. Are you doing this work for your work at a company or as your hobby in your free time?
In that case: happy new year celebration! I'm looking forward to your reply.
Thanks. This is exactly the issue.
FYI this is the mmdebstrap version some common distros are shipping:
So some clarification might be helpful to discover this (common?) usage, since most of users are likely on outdated versions for quite some time.
The hint I was getting is from the man's OPERATION section, where apt-get seemingly works with only APT_CONFIG exported and no magic arguments is needed.
It is my fault to assume a download-only apt-get call works the same for one that needs to also configure the package, but I hope you can see why I was confused when I was expecting the command to just work.
There are some standing issues with running apt on NixOS that I fixed on that PR.
I basically tested all supported modes since I really don't want to use the root mode for obvious reasons.
I do intend to get unshare to work eventually, but getting root mode to work at least clears me to package bdebstrap for NixOS, and run our system builder on it.
Do you mean the generated rootfs for Ubuntu is broken due to missing usrmerge right? I don't quite understand how Ubuntu host will have impact in a Debian chroot environment.
Yes this is for a work project. I want to use jsonnet+bdebstrap+mmdebstrap & libguestfs to replace our current go-debos based solution.
The main reason is that go-debos does not fully support ARM64 due to its fakemachine dependency, while we produce primarily ARM64 products, and customers want to build systems natively.
We also look to support RISC-V systems in the future, and Debian Wiki recommends this project, which is why I did not go straight with debootstrap (which I have considered quite a few times when working with go-debos).
Right now the new project is not announced, and is aiming to be a direct translation of the old project, which is a big hack by itself. If you have any suggestions I'm more than happy to hear it.
You can contact me directly via yt@radxa.com if here is not an appropriate place for that kind of discussion. GitHub issues within our repos are also welcome if you want to keep it open.
As the original issue is now resolved (due to missing argument). I'll also close this issue to reduce the noise from issue list.