variable expansion in special hooks #46

New issue

Open

opened 2024-10-25 10:32:41 +00:00 by mexsus · 2 comments

mexsus commented

2024-10-25 10:32:41 +00:00

Hi. Are variables expanded in special hooks? I know they are in hooks like customize-hooks because I'm using them successfully already.
I'm invoking mmdebstrap via bdebstrap. I pass variable WORKROOT via bbdebstrap like so --env WORKROOT=$MYWORKROOT and I can see WORKROOT set correctly in the merged config.yaml, eg

env:
  WORKROOT: /path/to/dir

If I use WORKROOT in the bdebstrap yaml in a special hook like: tar-out /boot/. $WORKROOT/boot.tar, rather than get /path/to/dir/boot.tar created, boot.tar is created in the directory of the original mmdebstrap invocation and $WORKROOT is a named directory included in that path, ie I get /path/where/mmdebstrap/runs/$WORKROOT/boot.tar. So $WORKROOT is not expanded.

In contrast, I can utilise WORKROOT in customize-hooks and do something like install -m 644 $WORKROOT/my/asset.img $1/boot/ which works fine (and I checked that ```/path/where/mmdebstrap/runs/$WORKROOT/ didn't exist beforehand).

It seems that variables are not expanded in special hooks. I looked in the test code and see there is special-hooks-using-helpers-and-env-vars, but it's not obvious that this is testing the above scenario.

Cheers,
-- Matt

Hi. Are variables expanded in special hooks? I know they are in hooks like ```customize-hooks``` because I'm using them successfully already. I'm invoking mmdebstrap via bdebstrap. I pass variable WORKROOT via bbdebstrap like so ```--env WORKROOT=$MYWORKROOT``` and I can see WORKROOT set correctly in the merged config.yaml, eg ``` env: WORKROOT: /path/to/dir ``` If I use WORKROOT in the bdebstrap yaml in a special hook like: ```tar-out /boot/. $WORKROOT/boot.tar```, rather than get /path/to/dir/boot.tar created, boot.tar is created in the directory of the original mmdebstrap invocation and ```$WORKROOT``` is a named directory included in that path, ie I get ```/path/where/mmdebstrap/runs/$WORKROOT/boot.tar```. So ```$WORKROOT``` is not expanded. In contrast, I can utilise WORKROOT in ```customize-hooks``` and do something like ```install -m 644 $WORKROOT/my/asset.img $1/boot/``` which works fine (and I checked that ```/path/where/mmdebstrap/runs/$WORKROOT/ didn't exist beforehand). It seems that variables are not expanded in special hooks. I looked in the test code and see there is special-hooks-using-helpers-and-env-vars, but it's not obvious that this is testing the above scenario. Cheers, -- Matt

josch commented

2024-10-25 11:24:24 +00:00

Owner

Hi,
I cannot tell you about bdebstrap but I can explain the mmdebstrap situation. This will expand $WORKROOT before passing it to mmdebstrap

--customize-hook="cp foo \"\$1\"/$WORKROOT"

Doing it like that can be problematic if $WORKROOT contains spaces or other special shell characters, so better expand it as part of the hook itself:

--customize-hook='cp foo "$1/$WORKROOT"'

On the other hand, the special hooks are not executed by the shell, so if you write this:

--customize-hook='copy-in foo $WORKROOT'

Then this will create a file inside your chroot that is literally called $WORKROOT. If you really want the special hooks to expand your variable, then you can always rewrite special hooks using the hook-helper like this:

--customize-hook='mmdebstrap --hook-helper "$1" "$MMDEBSTRAP_MODE" "$MMDEBSTRAP_HOOK" env "$MMDEBSTRAP_VERBOSITY" copy-in foo "$WORKROOT" <&"$MMDEBSTRAP_HOOKSOCK" >&"$MMDEBSTRAP_HOOKSOCK"'

This is very ugly and it might be an indicator that instead of adding many individual hooks, you might want to switch to a single hook doing most of the things for you.

Hi, I cannot tell you about `bdebstrap` but I can explain the mmdebstrap situation. This will expand `$WORKROOT` before passing it to mmdebstrap ``` --customize-hook="cp foo \"\$1\"/$WORKROOT" ``` Doing it like that can be problematic if `$WORKROOT` contains spaces or other special shell characters, so better expand it as part of the hook itself: ``` --customize-hook='cp foo "$1/$WORKROOT"' ``` On the other hand, the special hooks are not executed by the shell, so if you write this: ``` --customize-hook='copy-in foo $WORKROOT' ``` Then this will create a file inside your chroot that is literally called `$WORKROOT`. If you really want the special hooks to expand your variable, then you can always rewrite special hooks using the hook-helper like this: ``` --customize-hook='mmdebstrap --hook-helper "$1" "$MMDEBSTRAP_MODE" "$MMDEBSTRAP_HOOK" env "$MMDEBSTRAP_VERBOSITY" copy-in foo "$WORKROOT" <&"$MMDEBSTRAP_HOOKSOCK" >&"$MMDEBSTRAP_HOOKSOCK"' ``` This is very ugly and it might be an indicator that instead of adding many individual hooks, you might want to switch to a single hook doing most of the things for you.

mexsus commented

2024-10-29 10:19:00 +00:00

Author

Thanks @josch. I did wonder if the special hooks weren't execute in the shell. Does that not limit their use somewhat, though?

For example, if a user has to know up-front about the location where to copy-out to, does that not make it a bit more awkward to use special hooks?

Suppose I wanted to copy-out an asset installed during mmdebstrap execution for processing later. I could use /tmp/path/to/directory to copy-out to, but on a multi-use build server this would likely not play nicely with other users. If I wanted to copy-out to a location known to subsequent stages and outside of the build tree, how could I do it?

Whereas if I used a variable in the YAML, I'd be able to define the location to copy-out to meaning that when the special hook runs it expands the variable and performs the copy-out op.

I suppose using a customize-hook is an alternative approach and probably the (only) recommended way of approaching copy-out, tar-out, copy-in from scriptable locations if wanting to incorporate a variable.

Thanks @josch. I did wonder if the special hooks weren't execute in the shell. Does that not limit their use somewhat, though? For example, if a user has to know up-front about the location where to copy-out to, does that not make it a bit more awkward to use special hooks? Suppose I wanted to copy-out an asset installed during mmdebstrap execution for processing later. I could use /tmp/path/to/directory to copy-out to, but on a multi-use build server this would likely not play nicely with other users. If I wanted to copy-out to a location known to subsequent stages and outside of the build tree, how could I do it? Whereas if I used a variable in the YAML, I'd be able to define the location to copy-out to meaning that when the special hook runs it expands the variable and performs the copy-out op. I suppose using a customize-hook is an alternative approach and probably the (only) recommended way of approaching copy-out, tar-out, copy-in from scriptable locations if wanting to incorporate a variable.

No labels

No milestone

No project

No assignees

2 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: josch/mmdebstrap#46

No description provided.