chroot support

[Sean Whitton]

Hello,

I am thinking about implementing support for running commands in a chroot,which would be used like this:

task "foo" => sub {
    chroot "/srv/chroot/foo" => sub {
        file "/etc/foo.conf", source => "confs/foo.conf"; # actually edits /srv/chroot/foo/etc/foo.conf
        run "mycmd", ["--foo", "bar"]; # actually execs `chroot /srv/chroot/foo mycmd --foo bar`
    };
};

Has anyone thought about how this would be implemented in Rex?  My first thoughts are that we probably don't want to use the chroot system call, because that would require forking, which won't work except where the connection type is local.  So it seems we want some sort of connection layer, like the current sudo layer, which transforms paths and prepends chroot(1) to commands.  Does that sound right?

Thanks.

Sean Whitton

[Ferenc Erki]

Hi Sean,

On 2020-01-28 17:50, Sean Whitton wrote:
>I am thinking about implementing support for running commands in a
>chroot,which would be used like this:
>
>task "foo" => sub {
> chroot "/srv/chroot/foo" => sub {
> file "/etc/foo.conf", source => "confs/foo.conf"; # actually edits
>/srv/chroot/foo/etc/foo.conf
> run "mycmd", ["--foo", "bar"]; # actually execs `chroot
>/srv/chroot/foo mycmd --foo bar`
> };
>};
>

sounds great, I'm sure it would a be a useful addition!

>Has anyone thought about how this would be implemented in Rex?

There is a placeholder issue about chroot support with some recent
comment: https://github.com/RexOps/Rex/issues/468

Looks like your proposal matches the second form mentioned there, which
I believe is a well-balanced design choice.

>My first thoughts are that we probably don't want to use the chroot
>system call, because that would require forking, which won't work
>except where the connection type is local.

I guess you mean the perl built-in chroot, right? Technically, if the
managed host has perl, a small wrapper script can be created there, then
executed, and that might work. I would prefer a "cleaner" solution
though, like the one below.

>So it seems we want some sort of connection layer, like the current
>sudo layer, which transforms paths and prepends chroot(1) to commands.
>Does that sound right?

Yes, that could be one approach, definitely. Another could be to still
add a "chroot modifier" for `run`/`i_run` commands and the chroot
feature somehow transforms those calls by e.g. adding the extra
parameters.

Since I expect some experimentation before reaching a stable set of
interface design and implementation details, I would suggest trying to
implement this first outside Rex core as an extension module
(Rex::Commands::Chroot and similar perhaps?).

Finally, some random thoughts to consider, which occurred to me while
replying:

- A chroot feature probably would need to handle not just the
user-defined `run` commands, but also the OS commands executed
internally by rex (`i_run`).
- For some functionality (notably sudo support), rex uses small perl
wrapper scripts on the machine targeted by a task. They probably need
to be double-checked in the context of a chroot feature.
- Related to the point above, chroot would be used by root, so it would
be probably used together with sudo in one way or another. First
iteration might be able to assume "root-only, without sudo" use case,
though, and then perhaps it could be solved later.
- I wonder how this feature would be best to test, and if there's any
relevant differences between various platforms besides "chroot is not
supported on Windows" (fixme)?

In any case, I'd be happy to help your work towards this feature, feel
free to join the discussions on the GitHub issue and/or on IRC too.

Cheers,
FErki

[Sean Whitton]

Hello Ferenc,

Thank you for your reply!

On Wed 29 Jan 2020 at 08:27AM +01, Ferenc Erki wrote:

>>So it seems we want some sort of connection layer, like the current
>>sudo layer, which transforms paths and prepends chroot(1) to commands.
>>Does that sound right?
>
> Yes, that could be one approach, definitely. Another could be to still
> add a "chroot modifier" for `run`/`i_run` commands and the chroot
> feature somehow transforms those calls by e.g. adding the extra
> parameters.

Thinking about this, the sort of usecases I have in mind for this mean
that I'd like it to be possible to arbitrarily nest sudo and chroot
instances, e.g.

task "foo" => sub {
sudo {
chroot "/srv/chroot/foo" => sub {
sudo { user => "spwhitton", command => sub {
run "bar";
}};
};
};
};

Rex would need to run a command like

sudo chroot /srv/chroot/foo sudo -uspwhitton bar

Is Rex's current sudo support capable of doing something like that?

If not, it would not be enough to edit just run and i_run. It seems
like we might need a new abstraction, a stackable "connection layer", of
which both sudo and chroot would be subclasses. A connection layer
could prepend to paths and commands, set environment variables, and
maybe more.

> Since I expect some experimentation before reaching a stable set of
> interface design and implementation details, I would suggest trying to
> implement this first outside Rex core as an extension module
> (Rex::Commands::Chroot and similar perhaps?).

Hmm, I'm not sure this would be possible if we want the arbitrary
nesting described above.

> - For some functionality (notably sudo support), rex uses small perl
> wrapper scripts on the machine targeted by a task. They probably need
> to be double-checked in the context of a chroot feature.

Good point.

As an aside, it might be useful to make the chroot command configurable,
so that it is possible to use schroot(1) as well as chroot(1).

--
Sean Whitton

[Ferenc Erki]

Hi Sean,

thanks for your patience on this thread.

On 2020-01-29 12:24, Sean Whitton wrote:
>Thinking about this, the sort of usecases I have in mind for this mean
>that I'd like it to be possible to arbitrarily nest sudo and chroot
>instances, e.g.
>
> task "foo" => sub {
> sudo {
> chroot "/srv/chroot/foo" => sub {
> sudo { user => "spwhitton", command => sub {
> run "bar";
> }};
> };
> };
> };
>
>Rex would need to run a command like
>
> sudo chroot /srv/chroot/foo sudo -uspwhitton bar
>
>Is Rex's current sudo support capable of doing something like that?

Hmm, I'm not sure about that without testing and reading the
implementation details.

While it's useful to keep these use cases in mind, I'm also not entirely
sure if it's really needed to aim for arbitrary complexity right from
the start.

Could it be that this case can be simplified into something like

sudo chroot $path $command

where:

$path is "/srv/chroot/foo"
$command is "sudo -uspwhitton bar"

That is in Rex code:

task "foo" => sub {
sudo {
chroot "/srv/chroot/foo" => sub {

run "sudo -uspwhitton bar";
};
};
};

Perhaps that amount of nesting is already a good enough approach, at
least for a first iteration?

>If not, it would not be enough to edit just run and i_run. It seems
>like we might need a new abstraction, a stackable "connection layer", of
>which both sudo and chroot would be subclasses. A connection layer
>could prepend to paths and commands, set environment variables, and
>maybe more.

Fully supporting stackable layers in a generic way is an interesting
idea, even if it's a bit of a spinoff (or even maybe prerequisite) of
the chroot topic. Especially if we would find more use cases where it
would be useful as "the simplest approach that could work".

>> Since I expect some experimentation before reaching a stable set of
>> interface design and implementation details, I would suggest trying to
>> implement this first outside Rex core as an extension module
>> (Rex::Commands::Chroot and similar perhaps?).
>
>Hmm, I'm not sure this would be possible if we want the arbitrary
>nesting described above.

Well, if Rex core changes are needed, the whole project can be forked
and modified to accommodate for new use cases.

>As an aside, it might be useful to make the chroot command
>configurable,
>so that it is possible to use schroot(1) as well as chroot(1).

Good point! I feel like that's even more reason to have the chroot
functionality as a plugin (even if core has to provide new APIs for it
first).

cheers,
FErki

[Sean Whitton]

Hello Ferenc,

On Thu 05 Mar 2020 at 02:31PM +01, Ferenc Erki wrote:

> thanks for your patience on this thread.

Thank you for your input!

My thinking here is that it would be possible to write Rex tasks which
can be run under arbitrary users, inside and outside arbitrary chroots,
because the code in the task itself will not need to care about running
'sudo' or 'chroot'.

So you could have a task which does some setup inside the current user's
home directory, and then you could apply it to arbitrary users inside
and outside of chroots. And perhaps the task calls other subroutines to
do its work, and again, the code in those subroutines does not have to
call sudo or chroot but just assumes it is already running as the right
user in the right filesystem.

And then on hosts where you don't have root access, you can use the same
task to do work in the home directory of your account on that host,
without changing the code inside the task.

For example, I have a task which does some basic home directory
setup.[1] I can use that on hosts where I don't have root, and it would
be cool if on hosts where I do have root, I could use it to populate my
home directory inside and outside of chroots, and also /root.

As you can see, arbitrary nesting would be needed for this to work.

[1] https://git.spwhitton.name/dotfiles/tree/Rexfile#n70

--
Sean Whitton