kas-container: support git worktrees

[Jörg Sommer]

Hello,

git worktrees contain only a file .git with the reference to the real git
directory that's shared with all worktrees. Hence, this directory needs to
be present in the container, too.

Could a patch like this be applied to kas-container?

diff --git kas-container kas-container
index 46120247..6cb709ce 100755
--- kas-container
+++ kas-container
@@ -455,6 +455,11 @@ if [ "$(id -u)" -eq 0 ] && [ "${KAS_ALLOW_ROOT}" != "yes" ] ; then
"KAS_ALLOW_ROOT=yes to override."
fi

+if [ -f "${KAS_REPO_DIR}/.git" ]; then
+ git_com_dir=$(git -C "${KAS_REPO_DIR}" rev-parse --git-common-dir)
+ set -- "$@" -v "$git_com_dir:$git_com_dir:${KAS_REPO_MOUNT_OPT}"
+fi
+
set -- "$@" -v "${KAS_REPO_DIR}:/repo:${KAS_REPO_MOUNT_OPT}" \
-v "${KAS_WORK_DIR}":/work:rw -e KAS_WORK_DIR=/work \
-v "${KAS_BUILD_DIR}":/build:rw \

Without this patch:

% cat data/fwkos-fgw/.git
gitdir: /home/sommer/data/fwkos/worktrees/fwkos-fgw
% ./kas-container shell -c 'git -C /repo rev-parse @'
fatal: not a git repository: /home/sommer/data/fwkos/worktrees/fwkos-fgw

Without this patch:

% ./kas-container shell -c 'git -C /repo rev-parse @'
6c8858959ede52fc25a9bc3db588c4c0a2214a9c


Kind regards, Jörg

[MOESSBAUER, Felix]

On Thu, 2024-11-21 at 08:43 +0100, 'Jörg Sommer' via kas-devel wrote:
> Hello,
>
> git worktrees contain only a file .git with the reference to the real
> git
> directory that's shared with all worktrees. Hence, this directory
> needs to
> be present in the container, too.

Hi, IIRC we had multiple requests about worktree support in the past,
so it probably is a good idea to add it ;)

>
> Could a patch like this be applied to kas-container?

Does this patch also cover the issues discussed here?
https://github.com/siemens/kas/issues/107

>
> diff --git kas-container kas-container
> index 46120247..6cb709ce 100755
> --- kas-container
> +++ kas-container
> @@ -455,6 +455,11 @@ if [ "$(id -u)" -eq 0 ] && [ "${KAS_ALLOW_ROOT}"
> != "yes" ] ; then
>                     "KAS_ALLOW_ROOT=yes to override."
>  fi
>  
> +if [ -f "${KAS_REPO_DIR}/.git" ]; then
> +       git_com_dir=$(git -C "${KAS_REPO_DIR}" rev-parse --git-
> common-dir)
> +       set -- "$@" -v
> "$git_com_dir:$git_com_dir:${KAS_REPO_MOUNT_OPT}"
> +fi
> +

In general that looks good. I'm just wondering if we should ignore the
case that .git is a file, but not for git (like "echo foo > .git").
Probably that's just a theoretical thing, but if it easily can be
handled better add support for that case as well.

Please re-send this patch as a patch.

Best regards,
Felix

>  set -- "$@" -v "${KAS_REPO_DIR}:/repo:${KAS_REPO_MOUNT_OPT}" \
>         -v "${KAS_WORK_DIR}":/work:rw -e KAS_WORK_DIR=/work \
>         -v "${KAS_BUILD_DIR}":/build:rw \
>
> Without this patch:
>
> % cat data/fwkos-fgw/.git
> gitdir: /home/sommer/data/fwkos/worktrees/fwkos-fgw
> % ./kas-container shell -c 'git -C /repo rev-parse @'
> fatal: not a git repository: /home/sommer/data/fwkos/worktrees/fwkos-
> fgw
>
> Without this patch:
>
> % ./kas-container shell -c 'git -C /repo rev-parse @'
> 6c8858959ede52fc25a9bc3db588c4c0a2214a9c
>
>
> Kind regards, Jörg
>

--
Siemens AG, Technology
Linux Expert Center

[Jörg Sommer]

From: Jörg Sommer <joerg....@navimatix.de>

Git support sharing the .git directory across multiple worktrees each having
its own HEAD, index, rebase-todo and so on. The worktree directory (created
with `git worktree`) contains no directory .git, but a file .git with a
reference to the main git directory. Therefore, this directory must also be
mounted in the container to make git operable.

Closes: #107
Signed-off-by: Jörg Sommer <joerg....@navimatix.de>
---
kas-container | 7 +++++++
1 file changed, 7 insertions(+)

diff --git a/kas-container b/kas-container
index 6b2131c3..2c6505f6 100755
--- a/kas-container
+++ b/kas-container
@@ -452,6 +452,13 @@ if [ "$(id -u)" -eq 0 ] && [ "${KAS_ALLOW_ROOT}" != "yes" ] ; then
"KAS_ALLOW_ROOT=yes to override."
fi

+if git_com_dir=$(git -C "${KAS_REPO_DIR}" rev-parse --git-common-dir 2>/dev/null) \
+ && [ "$git_com_dir" != "$(git -C "${KAS_REPO_DIR}" rev-parse --git-dir 2>/dev/null)" ]; then
+ # if (it's a git repo) and the common dir isn't the git-dir, it is shared worktree and
+ # we have to mount the common dir in the container to make git work

+ set -- "$@" -v "$git_com_dir:$git_com_dir:${KAS_REPO_MOUNT_OPT}"
+fi
+

set -- "$@" -v "${KAS_REPO_DIR}:/repo:${KAS_REPO_MOUNT_OPT}" \
-v "${KAS_WORK_DIR}":/work:rw -e KAS_WORK_DIR=/work \
-v "${KAS_BUILD_DIR}":/build:rw \

--
2.45.2

[joerg....@navimatix.de]

MOESSBAUER, Felix schrieb am Do 21. Nov, 08:13 (+0000):

> On Thu, 2024-11-21 at 08:43 +0100, 'Jörg Sommer' via kas-devel wrote:
> > Could a patch like this be applied to kas-container?
>
> Does this patch also cover the issues discussed here?
> https://github.com/siemens/kas/issues/107

Yes, it fixes this problem, too. I can run `kas-container build` from
git-worktrees.

> > diff --git kas-container kas-container
> > index 46120247..6cb709ce 100755
> > --- kas-container
> > +++ kas-container
> > @@ -455,6 +455,11 @@ if [ "$(id -u)" -eq 0 ] && [ "${KAS_ALLOW_ROOT}"
> > != "yes" ] ; then
> >                     "KAS_ALLOW_ROOT=yes to override."
> >  fi
> >  
> > +if [ -f "${KAS_REPO_DIR}/.git" ]; then
> > +       git_com_dir=$(git -C "${KAS_REPO_DIR}" rev-parse --git-
> > common-dir)
> > +       set -- "$@" -v
> > "$git_com_dir:$git_com_dir:${KAS_REPO_MOUNT_OPT}"
> > +fi
> > +
>
> In general that looks good. I'm just wondering if we should ignore the
> case that .git is a file, but not for git (like "echo foo > .git").
> Probably that's just a theoretical thing, but if it easily can be
> handled better add support for that case as well.

In case it's not a worktree, --git-common-dir reports the same directory as
--git-dir. Then the mount must be skipped.


Thanks, Jörg

[Jan Kiszka]

On 21.11.24 13:30, 'Jörg Sommer' via kas-devel wrote:
> From: Jörg Sommer <joerg....@navimatix.de>
>
> Git support sharing the .git directory across multiple worktrees each having
> its own HEAD, index, rebase-todo and so on. The worktree directory (created
> with `git worktree`) contains no directory .git, but a file .git with a
> reference to the main git directory. Therefore, this directory must also be
> mounted in the container to make git operable.
>
> Closes: #107
> Signed-off-by: Jörg Sommer <joerg....@navimatix.de>
> ---
> kas-container | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/kas-container b/kas-container
> index 6b2131c3..2c6505f6 100755
> --- a/kas-container
> +++ b/kas-container
> @@ -452,6 +452,13 @@ if [ "$(id -u)" -eq 0 ] && [ "${KAS_ALLOW_ROOT}" != "yes" ] ; then
> "KAS_ALLOW_ROOT=yes to override."
> fi
>
> +if git_com_dir=$(git -C "${KAS_REPO_DIR}" rev-parse --git-common-dir 2>/dev/null) \
> + && [ "$git_com_dir" != "$(git -C "${KAS_REPO_DIR}" rev-parse --git-dir 2>/dev/null)" ]; then
> + # if (it's a git repo) and the common dir isn't the git-dir, it is shared worktree and
> + # we have to mount the common dir in the container to make git work

Indention is off.

> + set -- "$@" -v "$git_com_dir:$git_com_dir:${KAS_REPO_MOUNT_OPT}"

Hmm, that may create interesting paths inside the container when you
just mirror in the path from the outside. I don't have a good idea how
to avoid that that, given that ${KAS_REPO_DIR}/.git is set. Or could we
overload that as well with a file that uses a self-controlled path?

> +fi
> +
> set -- "$@" -v "${KAS_REPO_DIR}:/repo:${KAS_REPO_MOUNT_OPT}" \
> -v "${KAS_WORK_DIR}":/work:rw -e KAS_WORK_DIR=/work \
> -v "${KAS_BUILD_DIR}":/build:rw \

Jan

[Jörg Sommer]

Jan Kiszka schrieb am Do 21. Nov, 15:00 (+0100):

The content of the file is set, e.g.

gitdir: /home/joerg_sommer/Projekte/prj1/.git/worktrees/wt1

We mimic what git does (in newer versions). I would not really mess with the
git internals and bind-mount a file in the container over the git file.

Another option might be to set GIT_COMMON_DIR. But this might not get passed
to all processes (e.g. bitbake).

I use these mounts for at least a half year and I had no problems.
Currently, I add them with `--runtime-args '-v …'`, when I detect .git is a
file.


Jörg

[Jan Kiszka]

It's also under kas control which env vars get forwarded to bitbake.

>
> I use these mounts for at least a half year and I had no problems.
> Currently, I add them with `--runtime-args '-v …'`, when I detect .git is a
> file.

I'm sure you put your common dirs at reasonable, non-conflicting
locations. But that need not be the case for everyone.

[Jörg Sommer]

Jan Kiszka schrieb am Fr 22. Nov, 08:21 (+0100):

Also the variables Bitbake includes in its environment? Or does Bitbake
import GIT_* by default? I think about code like meta/lib/oe/buildcfg.py in
poky.

> > I use these mounts for at least a half year and I had no problems.
> > Currently, I add them with `--runtime-args '-v …'`, when I detect .git is a
> > file.
>
> I'm sure you put your common dirs at reasonable, non-conflicting
> locations. But that need not be the case for everyone.

If someone puts its code in /etc or /usr, the GIT_COMMON_DIR would still be
/usr/.git. To get a conflict with a common directory it would have to rename
its .git directory (yes, it's possible). And all this should happen
unintentionally?


Regards, Jörg

[Jan Kiszka]

bb.process.run is just forwarding the env from bitbake unless otherwise
told. And buildcfg.py is not doing that.

>
>>> I use these mounts for at least a half year and I had no problems.
>>> Currently, I add them with `--runtime-args '-v …'`, when I detect .git is a
>>> file.
>>
>> I'm sure you put your common dirs at reasonable, non-conflicting
>> locations. But that need not be the case for everyone.
>
> If someone puts its code in /etc or /usr, the GIT_COMMON_DIR would still be
> /usr/.git. To get a conflict with a common directory it would have to rename
> its .git directory (yes, it's possible). And all this should happen
> unintentionally?

No, the idea is to map the common dir to, say, /repo-common inside the
container and adjust GIT_COMMON_DIR, like we already do for /repo itself
(and many other paths).