[PATCH] Add support for authentication with gitlab CI

[Alban Bedel]

Gitlab CI predefine many variables in its environment, among them the
CI hostname and a token that can be used to authenticate with this
server. If we find these variables in the environment add the
credentials to .netrc which in turn allow git and other tools to
access resources found on the CI server.

Signed-off-by: Alban Bedel <alban...@aerq.com>
---
kas/libcmds.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/kas/libcmds.py b/kas/libcmds.py
index 0912204eeaa8..c1887a2c164a 100644
--- a/kas/libcmds.py
+++ b/kas/libcmds.py
@@ -162,7 +162,11 @@ class SetupHome(Command):
with open(self.tmpdirname + '/.wgetrc', 'w') as fds:
fds.write('\n')
with open(self.tmpdirname + '/.netrc', 'w') as fds:
- fds.write('\n')
+ if os.environ.get('CI_SERVER_HOST', False) \
+ and os.environ.get('CI_JOB_TOKEN', False):
+ fds.write('machine ' + os.environ['CI_SERVER_HOST'] + '\n'
+ 'login gitlab-ci-token\n'
+ 'password ' + os.environ['CI_JOB_TOKEN'] + '\n')
with open(self.tmpdirname + '/.gitconfig', 'w') as fds:
fds.write('[User]\n'
'\temail = k...@example.com\n'
--
2.30.2

[Jan Kiszka]

On 29.09.21 14:48, Alban Bedel wrote:
> Gitlab CI predefine many variables in its environment, among them the
> CI hostname and a token that can be used to authenticate with this
> server. If we find these variables in the environment add the
> credentials to .netrc which in turn allow git and other tools to
> access resources found on the CI server.
>
> Signed-off-by: Alban Bedel <alban...@aerq.com>
> ---
> kas/libcmds.py | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/kas/libcmds.py b/kas/libcmds.py
> index 0912204eeaa8..c1887a2c164a 100644
> --- a/kas/libcmds.py
> +++ b/kas/libcmds.py
> @@ -162,7 +162,11 @@ class SetupHome(Command):
> with open(self.tmpdirname + '/.wgetrc', 'w') as fds:
> fds.write('\n')
> with open(self.tmpdirname + '/.netrc', 'w') as fds:
> - fds.write('\n')

This is now missing in case CI_SERVER_HOST is not around.

> + if os.environ.get('CI_SERVER_HOST', False) \
> + and os.environ.get('CI_JOB_TOKEN', False):
> + fds.write('machine ' + os.environ['CI_SERVER_HOST'] + '\n'
> + 'login gitlab-ci-token\n'
> + 'password ' + os.environ['CI_JOB_TOKEN'] + '\n')
> with open(self.tmpdirname + '/.gitconfig', 'w') as fds:
> fds.write('[User]\n'
> '\temail = k...@example.com\n'
>

This looks useful, indeed. But shouldn't we also consider CI_SERVER_PORT
or even more?

Please also leave a comment around the code block, pointing out that
this addresses only gitlab - "CI_SERVER_HOST" is not clearly associating
things with that service.

How does this look like for github, do you happen to know this as well?

Jan

[Bedel, Alban]

On Wed, 2021-09-29 at 17:14 +0200, Jan Kiszka wrote:
> On 29.09.21 14:48, Alban Bedel wrote:
> > Gitlab CI predefine many variables in its environment, among them
> > the
> > CI hostname and a token that can be used to authenticate with this
> > server. If we find these variables in the environment add the
> > credentials to .netrc which in turn allow git and other tools to
> > access resources found on the CI server.
> >
> > Signed-off-by: Alban Bedel <alban...@aerq.com>
> > ---
> >  kas/libcmds.py | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/kas/libcmds.py b/kas/libcmds.py
> > index 0912204eeaa8..c1887a2c164a 100644
> > --- a/kas/libcmds.py
> > +++ b/kas/libcmds.py
> > @@ -162,7 +162,11 @@ class SetupHome(Command):
> >          with open(self.tmpdirname + '/.wgetrc', 'w') as fds:
> >              fds.write('\n')
> >          with open(self.tmpdirname + '/.netrc', 'w') as fds:
> > -            fds.write('\n')
>
> This is now missing in case CI_SERVER_HOST is not around.

I was under the impression that these write where used as a kind of
"pass". I'll add it back.

> >
> > +            if os.environ.get('CI_SERVER_HOST', False) \
> > +                    and os.environ.get('CI_JOB_TOKEN', False):
> > +                fds.write('machine ' +
> > os.environ['CI_SERVER_HOST'] + '\n'
> > +                          'login gitlab-ci-token\n'
> > +                          'password ' + os.environ['CI_JOB_TOKEN']
> > + '\n')
> >          with open(self.tmpdirname + '/.gitconfig', 'w') as fds:
> >              fds.write('[User]\n'
> >                        '\temail = k...@example.com\n'
> >
>
> This looks useful, indeed. But shouldn't we also consider
> CI_SERVER_PORT or even more?

The netrc mechanism don't allow for it. Alternatively one can configure
git to match on the URL (so including port, etc) and pass the token
using an Authorize header, but that only work with git, netrc will also
work with wget and a few more.

> Please also leave a comment around the code block, pointing out that
> this addresses only gitlab - "CI_SERVER_HOST" is not clearly
> associating things with that service.

I'll add a comment for this.

> How does this look like for github, do you happen to know this as
> well?

No, I have never used the github CI. But to be honest I would really
much prefer if this patch would not be needed at all. As a new user of
kas I found the undocumented behaviour of creating a pseudo jail a
strange design decision. In my experience reproducibility problems come
foremost because of tools and libraries installed on the host, user
configs are a minor problem in comparison. The pseudo jail only address
the later and can easily be bypassed by using the global config files
in /etc. But the real problem is that it make it next to impossible to
safely pass secrets into the build. As containers already provide a
mature solution for reproducible environments I really don't see the
need for kas to force its own incomplete solution on us.

Alban

[Jan Kiszka]

I'm not sure whether they played a role for the tools, and we still
write one for .wgetrc.

Actually, passing credentials works fine since ages with kas, securely.
We are just normally using ssh access tokens, both locally as well as in CI.

The idea around making those passing explicit, opt-in, is that you think
about what you use and how you need to make that available to other
users of kas builds (including CI). That is inconvenient if you are the
only user but proved to be helpful in shared scenarios.

Jan

[Bedel, Alban]

Only supporting a subset of what ssh allow is not what I meant. What
about host where I need some extra ssh config? What about gnome-
keyring? libsecret? pass? pgp? There are just too many tools to
consider.

> The idea around making those passing explicit, opt-in, is that you
> think about what you use and how you need to make that available to
> other users of kas builds (including CI). That is inconvenient if you
> are the only user but proved to be helpful in shared scenarios.

The current solution force users to modify kas as soon as they are not
using one of the tools blessed by the kas developers. I don't see that
as a viable solution in the long run, creating a good jail is complex
and soon that code will out grow the core function of kas. I think this
should be offloaded to dedicated tools like containers.

But beside our personal opinions on the usefulness of this feature,
would you consider a patch that add a command line option to disable
the pseudo-jail?

Alban

[Jan Kiszka]

Do you run, e.g., gnome-keyring on your CI servers as well?

>
>> The idea around making those passing explicit, opt-in, is that you
>> think about what you use and how you need to make that available to
>> other users of kas builds (including CI). That is inconvenient if you
>> are the only user but proved to be helpful in shared scenarios.
>
> The current solution force users to modify kas as soon as they are not
> using one of the tools blessed by the kas developers. I don't see that
> as a viable solution in the long run, creating a good jail is complex
> and soon that code will out grow the core function of kas. I think this
> should be offloaded to dedicated tools like containers.
>
> But beside our personal opinions on the usefulness of this feature,
> would you consider a patch that add a command line option to disable
> the pseudo-jail?

We can discuss relaxations - provided we find a clean way to propagate
that via kas-container as well. By now, that is another reason to have a
managed pass-in of credential.

Jan

--
Siemens AG, Technology
Competence Center Embedded Linux