[PATCH v2 0/2] add support for shallow cloning

[Felix Moessbauer]

Changes since v1:

- support shallow cloning for branch, tag and commit
- rename variable to KAS_CLONE_DEPTH. While currently only git
is supported, other VCS might be added in the future (mercurial
currently does not support it).
- improve robustness (e.g. disable for legacy refspec)
- improve test coverage

Further, this adresses the feedback from
https://groups.google.com/g/kas-devel/c/W4X7P4Ej7wk/m/uqK9FUeIAAAJ

- stick to env-var, as this is transparent from kas perspective. It
is only a performance improvement.
- With support for branch, commit and tag we cover the whole range
of git refs. No need to exclude combinations.

Best regards,
Felix Moessbauer
Siemens AG

Felix Moessbauer (1):
git: only fetch requested tag

Marek Vasut (1):
repos: Add KAS_CLONE_DEPTH to implement git shallow clone/fetch

docs/command-line/environment-variables.inc | 7 ++++++
kas-container | 2 +-
kas/context.py | 5 +++++
kas/repos.py | 23 ++++++++++++++++++-
tests/conftest.py | 1 +
tests/test_commands.py | 25 ++++++++++++++++++++-
tests/test_commands/test-shallow.yml | 23 +++++++++++++++++++
7 files changed, 83 insertions(+), 3 deletions(-)
create mode 100644 tests/test_commands/test-shallow.yml

--
2.39.2

[Felix Moessbauer]

Changes since v2:

- make shallow cloning fully transparent to kas operations
- fix issues around commit-not-in-branch on shallow clones

- improve robustness (e.g. disable for legacy refspec)
- improve test coverage

Changes since v1:

- support shallow cloning for branch, tag and commit
- rename variable to KAS_CLONE_DEPTH. While currently only git
is supported, other VCS might be added in the future (mercurial
currently does not support it).
- improve robustness (e.g. disable for legacy refspec)
- improve test coverage

Further, this adresses the feedback from
https://groups.google.com/g/kas-devel/c/W4X7P4Ej7wk/m/uqK9FUeIAAAJ

- stick to env-var, as this is transparent from kas perspective. It
is only a performance improvement.
- With support for branch, commit and tag we cover the whole range
of git refs. No need to exclude combinations.

Best regards,
Felix Moessbauer
Siemens AG

Felix Moessbauer (2):

git: only fetch requested tag

expose clone depth as CLI option as well

Marek Vasut (1):
repos: Add KAS_CLONE_DEPTH to implement git shallow clone/fetch

docs/command-line/environment-variables.inc | 16 +++++
kas-container | 2 +-
kas/context.py | 8 +++
kas/libkas.py | 3 +
kas/repos.py | 32 +++++++++-
tests/conftest.py | 1 +
tests/test_commands.py | 67 ++++++++++++++++++++-
tests/test_commands/test-shallow.yml | 23 +++++++
8 files changed, 147 insertions(+), 5 deletions(-)

[Felix Moessbauer]

On git clones, the fetching of tags need to be explicitly requested.
When requesting to fetch a tag, we previously fetched all tags and just
checked out the needed one. However, it is sufficient to only fetch the
requested one.

This is a preparation to support shallow clones.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---
kas/repos.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kas/repos.py b/kas/repos.py
index 62b2bbd75..4b2b0c3e5 100644
--- a/kas/repos.py
+++ b/kas/repos.py
@@ -544,7 +544,7 @@ class GitRepo(RepoImpl):
def fetch_cmd(self):
cmd = ['git', 'fetch', '-q']
if self.tag:
- cmd.append('--tags')
+ cmd.extend(['origin', f'+{self.tag}:refs/tags/{self.tag}'])

branch = self.branch or self.refspec
if branch and branch.startswith('refs/'):
--
2.39.2

[Felix Moessbauer]

From: Marek Vasut <ma...@denx.de>

Add new environment variable KAS_CLONE_DEPTH which adds '--depth=N'
to the 'git clone' and 'git fetch' commands. This forces git to
perform shallow clone, which saves bandwidth and CI runner disk
space. The depth 'N' is derived from KAS_CLONE_DEPTH value.

This is useful in case CI always starts with empty work directory
and this directory is always discarded after the CI run. In that
case, it makes no sense to clone the entire repository, instead
clone just enough to reproduce the desired state of the repository
and assemble the checkout of it.

This is also useful when cloning massive repositories which would
otherwise take long time to clone. Shallow cloning is supported for
specific commits, branches and tags (but disabled on refspec).

[Felix: rebased, sanitize input values, adapted doc entry to new
format, port test over to monkeykas infrastructure, forward from
kas-container]

Signed-off-by: Marek Vasut <ma...@denx.de>

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

docs/command-line/environment-variables.inc | 6 ++

kas-container | 2 +-
kas/context.py | 5 ++

kas/repos.py | 30 +++++++++-
tests/conftest.py | 1 +
tests/test_commands.py | 65 ++++++++++++++++++++-
tests/test_commands/test-shallow.yml | 23 ++++++++
7 files changed, 128 insertions(+), 4 deletions(-)
create mode 100644 tests/test_commands/test-shallow.yml

diff --git a/docs/command-line/environment-variables.inc b/docs/command-line/environment-variables.inc
index db1ac928a..fce53e692 100644
--- a/docs/command-line/environment-variables.inc
+++ b/docs/command-line/environment-variables.inc
@@ -72,6 +72,12 @@ Variables Glossary
| ``DISTRO_APT_PREMIRRORS``| Specifies alternatives for apt URLs. Just like |
| (C) | ``KAS_PREMIRRORS``. |
+--------------------------+--------------------------------------------------+
+| ``KAS_CLONE_DEPTH`` | Perform shallow git clone/fetch using --depth=N |
+| (C, K) | specified by this variable. This is useful in |
+| | case CI always starts with empty work directory |
+| | and this directory is always discarded after the |
+| | CI run. |
++--------------------------+--------------------------------------------------+
| ``SSH_PRIVATE_KEY`` | Variable containing the private key that should |
| (K) | be added to an internal ssh-agent. This key |
| | cannot be password protected. This setting is |
diff --git a/kas-container b/kas-container
index 39c984a29..a94107a88 100755
--- a/kas-container
+++ b/kas-container
@@ -546,7 +546,7 @@ if [ -n "${KAS_REPO_REF_DIR}" ]; then
-e KAS_REPO_REF_DIR=/repo-ref
fi

-for var in TERM KAS_DISTRO KAS_MACHINE KAS_TARGET KAS_TASK \
+for var in TERM KAS_DISTRO KAS_MACHINE KAS_TARGET KAS_TASK KAS_CLONE_DEPTH \
KAS_PREMIRRORS DISTRO_APT_PREMIRRORS BB_NUMBER_THREADS PARALLEL_MAKE \
GIT_CREDENTIAL_USEHTTPPATH; do
if [ -n "$(eval echo \$${var})" ]; then
diff --git a/kas/context.py b/kas/context.py
index 6ca529151..c228a1696 100644
--- a/kas/context.py
+++ b/kas/context.py
@@ -25,6 +25,7 @@

import os
import logging
+from kas.kasusererror import KasUserError

try:
import distro
@@ -78,6 +79,10 @@ class Context:
self.__kas_build_dir = os.path.abspath(build_dir)
ref_dir = os.environ.get('KAS_REPO_REF_DIR', None)
self.__kas_repo_ref_dir = os.path.abspath(ref_dir) if ref_dir else None
+ clone_depth = os.environ.get('KAS_CLONE_DEPTH', '0')
+ if not clone_depth.isdigit():
+ raise KasUserError('KAS_CLONE_DEPTH must be a number')
+ self.repo_clone_depth = max(int(clone_depth), 0)
self.setup_initial_environ()
self.config = None
self.args = args
diff --git a/kas/repos.py b/kas/repos.py
index 4b2b0c3e5..fd0bc3817 100644
--- a/kas/repos.py
+++ b/kas/repos.py
@@ -394,7 +394,9 @@ class RepoImpl(Repo):
raise RepoRefError(
f'Branch "{self.branch}" cannot be found '
f'in repository "{self.name}"')
- if self.commit:
+ # check if branch contains the requested commit.
+ # skip check on shallow clones, as branch information is missing
+ if self.commit and not get_context().repo_clone_depth:
(_, output) = run_cmd(self.branch_contains_ref(),
cwd=self.path,
fail=False)
@@ -523,6 +525,19 @@ class GitRepo(RepoImpl):

def clone_cmd(self, srcdir, createref):
cmd = ['git', 'clone', '-q']
+
+ depth = get_context().repo_clone_depth
+ if depth:
+ if self.refspec:
+ logging.warning('Shallow cloning is not supported for legacy '
+ f'refspec on repository "{self.name}". '
+ 'Performing full clone.')
+ else:
+ cmd.extend(['--depth', str(depth)])
+ if self.branch:
+ cmd.extend(['--branch',
+ self.remove_ref_prefix(self.branch)])
+
if createref:
cmd.extend([self.effective_url, '--bare', srcdir])
elif srcdir:
@@ -543,11 +558,22 @@ class GitRepo(RepoImpl):

def fetch_cmd(self):
cmd = ['git', 'fetch', '-q']

+
+ depth = 0 if self.refspec else get_context().repo_clone_depth
+ if depth:
+ cmd.extend(['--depth', str(depth)])
+
if self.tag:

cmd.extend(['origin', f'+{self.tag}:refs/tags/{self.tag}'])

+ return cmd
+
+ # only fetch this commit (branch information is lost)
+ if depth and self.commit:
+ cmd.extend(['origin', self.commit])
+ return cmd

branch = self.branch or self.refspec

- if branch and branch.startswith('refs/'):
+ if branch and (branch.startswith('refs/') or depth):
branch = self.remove_ref_prefix(branch)
cmd.extend(['origin', f'+{branch}:refs/remotes/origin/{branch}'])

diff --git a/tests/conftest.py b/tests/conftest.py
index ef0dc9d10..d3ad2daa3 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -32,6 +32,7 @@ ENVVARS_KAS = [
'KAS_TARGET',
'KAS_TASK',
'KAS_PREMIRRORS',
+ 'KAS_CLONE_DEPTH',
'SSH_PRIVATE_KEY',
'SSH_PRIVATE_KEY_FILE',
'SSH_AUTH_SOCK',
diff --git a/tests/test_commands.py b/tests/test_commands.py
index 849882e6a..91a4bfb0c 100644
--- a/tests/test_commands.py
+++ b/tests/test_commands.py
@@ -29,7 +29,8 @@ import yaml
import subprocess
import pytest
from kas import kas
-from kas.libkas import TaskExecError
+from kas.libkas import run_cmd
+from kas.libkas import TaskExecError, KasUserError


def test_for_all_repos(monkeykas, tmpdir):
@@ -107,6 +108,68 @@ def test_checkout_create_refs(monkeykas, tmpdir):
assert os.path.exists('kas/.git/objects/info/alternates')


+def test_checkout_shallow(monkeykas, tmpdir):
+ tdir = str(tmpdir / 'test_commands')
+ shutil.copytree('tests/test_commands', tdir)
+ monkeykas.chdir(tdir)
+ with monkeykas.context() as mp:
+ mp.setenv('KAS_CLONE_DEPTH', 'invalid')
+ with pytest.raises(KasUserError):
+ kas.kas(['checkout', 'test-shallow.yml'])
+
+ with monkeykas.context() as mp:
+ mp.setenv('KAS_CLONE_DEPTH', '1')
+ kas.kas(['checkout', 'test-shallow.yml'])
+ for repo in ['kas_1', 'kas_2', 'kas_3', 'kas_4']:
+ (rc, output) = run_cmd(['git', 'rev-list', '--count', 'HEAD'],
+ cwd=repo, fail=False, liveupdate=False)
+ assert rc == 0
+ if repo == 'kas_4':
+ assert output.strip() >= '1'
+ else:
+ assert output.strip() == '1'
+
+
+def test_shallow_updates(monkeykas, tmpdir):
+ def _get_commit(repo):
+ (rc, output) = run_cmd(['git', 'rev-parse', '--verify', 'HEAD'],
+ cwd=repo, fail=False, liveupdate=False)
+ assert rc == 0
+ return output.strip()
+
+ tdir = tmpdir / 'test_commands'
+ tdir.mkdir()
+ shutil.copy('tests/test_commands/oe-init-build-env', tdir)
+ monkeykas.chdir(tdir)
+ monkeykas.setenv('KAS_CLONE_DEPTH', '1')
+ # test non-pinned checkout of master branch
+ base_yml = {'header': {'version': 15}, 'repos': {
+ 'this': {},
+ 'kas': {
+ 'url': 'https://github.com/siemens/kas.git',
+ 'branch': 'master'
+ }}}
+ with open(tdir / 'kas.yml', 'w') as f:
+ yaml.dump(base_yml, f)
+ kas.kas(['checkout', 'kas.yml'])
+ # switch branches, perform checkout again
+ base_yml['repos']['kas']['branch'] = 'next'
+ with open(tdir / 'kas.yml', 'w') as f:
+ yaml.dump(base_yml, f)
+ kas.kas(['checkout', 'kas.yml'])
+ # pin commit on next branch
+ commit = '5d1ab6e8ed3a12c7093c9041f104fb6a2db701a1'
+ base_yml_lock = {'header': {'version': 15},
+ 'overrides': {'repos': {'kas': {'commit': commit}}}}
+ with open(tdir / 'kas.lock.yml', 'w') as f:
+ yaml.dump(base_yml_lock, f)
+ kas.kas(['checkout', 'kas.yml'])
+ assert _get_commit('kas') == commit
+ # update to latest revision of next branch
+ kas.kas(['checkout', '--update', 'kas.yml'])
+ assert _get_commit('kas') != commit
+
+
def test_repo_includes(monkeykas, tmpdir):
tdir = str(tmpdir / 'test_commands')
shutil.copytree('tests/test_repo_includes', tdir)
diff --git a/tests/test_commands/test-shallow.yml b/tests/test_commands/test-shallow.yml
new file mode 100644
index 000000000..dfd711cdd
--- /dev/null
+++ b/tests/test_commands/test-shallow.yml
@@ -0,0 +1,23 @@
+header:
+ version: 14
+
+repos:
+ this:
+
+ kas_1:
+ url: https://github.com/siemens/kas.git
+ branch: master
+
+ kas_2:
+ url: https://github.com/siemens/kas.git
+ tag: '4.3'
+ commit: f650ebe2495a9cbe2fdf4a2c8becc7b3db470d55
+
+ kas_3:
+ url: https://github.com/siemens/kas.git
+ commit: e42a64a666082b77fbc2758b07191b662d17f792
+
+ kas_4:
+ url: https://github.com/siemens/kas.git
+ # keep legacy refspec here for testing purposes
+ refspec: master
--
2.39.2

[Felix Moessbauer]

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

docs/command-line/environment-variables.inc | 10 ++++++++++
kas/context.py | 11 +++++++----
kas/libkas.py | 3 +++
tests/test_commands.py | 2 ++
4 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/docs/command-line/environment-variables.inc b/docs/command-line/environment-variables.inc
index fce53e692..a0d71dfd5 100644
--- a/docs/command-line/environment-variables.inc
+++ b/docs/command-line/environment-variables.inc
@@ -2,6 +2,16 @@ kas uses a number of environment variables to configure its behavior.
The `Variables Glossary`_ provides an overview, wherein the tuple (C,K,E)
denotes the scope of the variable.

+Precedence
+~~~~~~~~~~
+
+In case settings can be specified several ways, the configuration values
+have the following precedence:
+
+- command line arguments
+- environment variables
+- configuration file
+
Variable Scope
~~~~~~~~~~~~~~

diff --git a/kas/context.py b/kas/context.py
index c228a1696..c0287b77f 100644
--- a/kas/context.py
+++ b/kas/context.py
@@ -79,10 +79,13 @@ class Context:

self.__kas_build_dir = os.path.abspath(build_dir)
ref_dir = os.environ.get('KAS_REPO_REF_DIR', None)
self.__kas_repo_ref_dir = os.path.abspath(ref_dir) if ref_dir else None

- clone_depth = os.environ.get('KAS_CLONE_DEPTH', '0')
- if not clone_depth.isdigit():
- raise KasUserError('KAS_CLONE_DEPTH must be a number')
- self.repo_clone_depth = max(int(clone_depth), 0)
+ if hasattr(args, 'depth'):
+ self.repo_clone_depth = max(args.depth, 0)
+ else:

+ clone_depth = os.environ.get('KAS_CLONE_DEPTH', '0')
+ if not clone_depth.isdigit():
+ raise KasUserError('KAS_CLONE_DEPTH must be a number')
+ self.repo_clone_depth = max(int(clone_depth), 0)
self.setup_initial_environ()
self.config = None
self.args = args

diff --git a/kas/libkas.py b/kas/libkas.py
index 06a335f93..fa805a2fa 100644
--- a/kas/libkas.py
+++ b/kas/libkas.py
@@ -426,6 +426,9 @@ def setup_parser_common_args(parser):
parser.add_argument('--update', action='store_true',
help='Pull new upstream changes to the desired '
'branch even if it is already checked out locally')
+ parser.add_argument('--depth',
+ type=int,
+ help='Repository clone depth (if supported)')


def setup_parser_preserve_env_arg(parser):
diff --git a/tests/test_commands.py b/tests/test_commands.py
index 91a4bfb0c..af574cf9d 100644
--- a/tests/test_commands.py
+++ b/tests/test_commands.py
@@ -116,6 +116,8 @@ def test_checkout_shallow(monkeykas, tmpdir):

mp.setenv('KAS_CLONE_DEPTH', 'invalid')

with pytest.raises(KasUserError):

kas.kas(['checkout', 'test-shallow.yml'])

+ # test CLI precedence
+ kas.kas(['checkout', '--depth', '1', 'test-shallow.yml'])

with monkeykas.context() as mp:

mp.setenv('KAS_CLONE_DEPTH', '1')

--
2.39.2

[MOESSBAUER, Felix]

Looks like I lost a "and args.depth" while committing...

Will send a v4.

Felix

--
Siemens AG, Technology
Linux Expert Center

[Felix Moessbauer]

Changes since v3:

- add lost condition when comparing args.depth
- rebased onto next

kas/repos.py | 32 +++++++++-
tests/conftest.py | 1 +

tests/test_commands.py | 67 ++++++++++++++++++++-
tests/test_commands/test-shallow.yml | 23 +++++++

8 files changed, 147 insertions(+), 5 deletions(-)
create mode 100644 tests/test_commands/test-shallow.yml

--
2.39.2

[Felix Moessbauer]

From: Marek Vasut <ma...@denx.de>

Add new environment variable KAS_CLONE_DEPTH which adds '--depth=N'
to the 'git clone' and 'git fetch' commands. This forces git to
perform shallow clone, which saves bandwidth and CI runner disk
space. The depth 'N' is derived from KAS_CLONE_DEPTH value.

This is useful in case CI always starts with empty work directory
and this directory is always discarded after the CI run. In that
case, it makes no sense to clone the entire repository, instead
clone just enough to reproduce the desired state of the repository
and assemble the checkout of it.

This is also useful when cloning massive repositories which would
otherwise take long time to clone. Shallow cloning is supported for
specific commits, branches and tags (but disabled on refspec).

[Felix: rebased, sanitize input values, adapted doc entry to new
format, port test over to monkeykas infrastructure, forward from
kas-container]

Signed-off-by: Marek Vasut <ma...@denx.de>

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

docs/command-line/environment-variables.inc | 6 ++
kas-container | 2 +-
kas/context.py | 5 ++

kas/repos.py | 30 +++++++++-
tests/conftest.py | 1 +

tests/test_commands.py | 65 ++++++++++++++++++++-
tests/test_commands/test-shallow.yml | 23 ++++++++

7 files changed, 128 insertions(+), 4 deletions(-)
create mode 100644 tests/test_commands/test-shallow.yml

diff --git a/docs/command-line/environment-variables.inc b/docs/command-line/environment-variables.inc
index db1ac928a..fce53e692 100644
--- a/docs/command-line/environment-variables.inc
+++ b/docs/command-line/environment-variables.inc

diff --git a/kas/context.py b/kas/context.py
index 6ca529151..c228a1696 100644
--- a/kas/context.py
+++ b/kas/context.py

@@ -25,6 +25,7 @@

import os
import logging
+from kas.kasusererror import KasUserError

try:
import distro

@@ -78,6 +79,10 @@ class Context:

self.__kas_build_dir = os.path.abspath(build_dir)
ref_dir = os.environ.get('KAS_REPO_REF_DIR', None)
self.__kas_repo_ref_dir = os.path.abspath(ref_dir) if ref_dir else None

+ clone_depth = os.environ.get('KAS_CLONE_DEPTH', '0')
+ if not clone_depth.isdigit():
+ raise KasUserError('KAS_CLONE_DEPTH must be a number')
+ self.repo_clone_depth = max(int(clone_depth), 0)
self.setup_initial_environ()
self.config = None
self.args = args

diff --git a/tests/test_commands.py b/tests/test_commands.py
index 849882e6a..91a4bfb0c 100644
--- a/tests/test_commands.py
+++ b/tests/test_commands.py

@@ -29,7 +29,8 @@ import yaml
import subprocess
import pytest
from kas import kas
-from kas.libkas import TaskExecError
+from kas.libkas import run_cmd
+from kas.libkas import TaskExecError, KasUserError


def test_for_all_repos(monkeykas, tmpdir):
@@ -107,6 +108,68 @@ def test_checkout_create_refs(monkeykas, tmpdir):
assert os.path.exists('kas/.git/objects/info/alternates')


+def test_checkout_shallow(monkeykas, tmpdir):
+ tdir = str(tmpdir / 'test_commands')
+ shutil.copytree('tests/test_commands', tdir)
+ monkeykas.chdir(tdir)
+ with monkeykas.context() as mp:

+ mp.setenv('KAS_CLONE_DEPTH', 'invalid')
+ with pytest.raises(KasUserError):
+ kas.kas(['checkout', 'test-shallow.yml'])
+

+ with monkeykas.context() as mp:

[Felix Moessbauer]

On git clones, the fetching of tags need to be explicitly requested.
When requesting to fetch a tag, we previously fetched all tags and just
checked out the needed one. However, it is sufficient to only fetch the
requested one.

This is a preparation to support shallow clones.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

kas/repos.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kas/repos.py b/kas/repos.py
index 62b2bbd75..4b2b0c3e5 100644
--- a/kas/repos.py
+++ b/kas/repos.py
@@ -544,7 +544,7 @@ class GitRepo(RepoImpl):

def fetch_cmd(self):
cmd = ['git', 'fetch', '-q']

if self.tag:
- cmd.append('--tags')

+ cmd.extend(['origin', f'+{self.tag}:refs/tags/{self.tag}'])

branch = self.branch or self.refspec

if branch and branch.startswith('refs/'):

--
2.39.2

[Felix Moessbauer]

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

docs/command-line/environment-variables.inc | 10 ++++++++++
kas/context.py | 11 +++++++----
kas/libkas.py | 3 +++
tests/test_commands.py | 2 ++
4 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/docs/command-line/environment-variables.inc b/docs/command-line/environment-variables.inc
index fce53e692..a0d71dfd5 100644
--- a/docs/command-line/environment-variables.inc
+++ b/docs/command-line/environment-variables.inc

@@ -2,6 +2,16 @@ kas uses a number of environment variables to configure its behavior.
The `Variables Glossary`_ provides an overview, wherein the tuple (C,K,E)
denotes the scope of the variable.

+Precedence
+~~~~~~~~~~
+
+In case settings can be specified several ways, the configuration values
+have the following precedence:
+
+- command line arguments
+- environment variables
+- configuration file
+
Variable Scope
~~~~~~~~~~~~~~

diff --git a/kas/context.py b/kas/context.py
index c228a1696..4911904e2 100644
--- a/kas/context.py
+++ b/kas/context.py
@@ -79,10 +79,13 @@ class Context:

self.__kas_build_dir = os.path.abspath(build_dir)
ref_dir = os.environ.get('KAS_REPO_REF_DIR', None)
self.__kas_repo_ref_dir = os.path.abspath(ref_dir) if ref_dir else None

- clone_depth = os.environ.get('KAS_CLONE_DEPTH', '0')
- if not clone_depth.isdigit():
- raise KasUserError('KAS_CLONE_DEPTH must be a number')

- self.repo_clone_depth = max(int(clone_depth), 0)

+ if hasattr(args, 'depth') and args.depth:

+ self.repo_clone_depth = max(args.depth, 0)
+ else:

+ clone_depth = os.environ.get('KAS_CLONE_DEPTH', '0')
+ if not clone_depth.isdigit():
+ raise KasUserError('KAS_CLONE_DEPTH must be a number')
+ self.repo_clone_depth = max(int(clone_depth), 0)
self.setup_initial_environ()
self.config = None
self.args = args

diff --git a/kas/libkas.py b/kas/libkas.py
index 06a335f93..fa805a2fa 100644
--- a/kas/libkas.py
+++ b/kas/libkas.py
@@ -426,6 +426,9 @@ def setup_parser_common_args(parser):
parser.add_argument('--update', action='store_true',
help='Pull new upstream changes to the desired '
'branch even if it is already checked out locally')
+ parser.add_argument('--depth',
+ type=int,
+ help='Repository clone depth (if supported)')


def setup_parser_preserve_env_arg(parser):

diff --git a/tests/test_commands.py b/tests/test_commands.py
index 91a4bfb0c..af574cf9d 100644
--- a/tests/test_commands.py
+++ b/tests/test_commands.py

@@ -116,6 +116,8 @@ def test_checkout_shallow(monkeykas, tmpdir):

mp.setenv('KAS_CLONE_DEPTH', 'invalid')

with pytest.raises(KasUserError):

kas.kas(['checkout', 'test-shallow.yml'])

+ # test CLI precedence

+ kas.kas(['checkout', '--depth', '1', 'test-shallow.yml'])

with monkeykas.context() as mp:

mp.setenv('KAS_CLONE_DEPTH', '1')

--
2.39.2

[Jan Kiszka]

And... why? Just because we can? I'm starting to like shallow clones,
I'm still not happy about how we manage it.

If we consider the clone depth an aspect of the personal workflow,
picking it up from the environment may make more sense because it could
be preset there by the user more conveniently and permanently. A command
line switch rather motivates to write wrappers because you can't
otherwise make it persistent, and the next operation would just fall
back to whatever default kas will once use.

Jan

[Jan Kiszka]

Thanks, applied 1 and 2 now, leaving out 3 due to an absence of good use
case (may change if someone provides them).

Looking forward to see KAS_CLONE_DEPTH in action for our own test
pipeline... ;)

Jan