[PATCH] dump plugin: add option to create lockfile

[Felix Moessbauer]

This patch adds the --lock option to the dump plugin. When enabled, the
output only contains the resolved refspecs of each repo (as valid kas
format). By that, floating branches can be used in the projects kas files
and these can be pinned to fixed fixed revisions, when required.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

This extension should solve the discussed use-case in "Detached HEAD state
when refspec is a commit-hash".

Example output:

header:
version: 7
repos:
isar:
refspec: 35f9487a849a10e40d4fbb3d9b6f13a25d4a1f96
meta-foo:
refspec: 2857fcbe2d7c4993a1f881986a9eb3f748c85b3d

Best regards,
Felix Moessbauer
Siemens AG

kas/plugins/dump.py | 45 +++++++++++++++++++++++++++++++++++----------
1 file changed, 35 insertions(+), 10 deletions(-)

diff --git a/kas/plugins/dump.py b/kas/plugins/dump.py
index 3773484..0552bf8 100644
--- a/kas/plugins/dump.py
+++ b/kas/plugins/dump.py
@@ -22,13 +22,17 @@
"""
This plugin implements the ``kas dump`` command.

- When this command is executed, kas will parse all referenced config
- files, expand includes and print a flattened yaml version of the
- configuration to stdout. This config is semantically identical to the
- input, but does not include any references to other configuration files.
- The output of this command can be used to further analyse the build
+ When this command is executed in default mode, kas will parse all
+ referenced config files, expand includes and print a flattened yaml version
+ of the configuration to stdout. This config is semantically identical to
+ the input, but does not include any references to other configuration
+ files. The output of this command can be used to further analyse the build
configuration.

+ When running with --lock, a lock-file is created which only contains the
+ exact refspecs of each repository. This file can be used to pin the
+ refspecs of floating branches, while still keeping an easy update path.
+
Please note:

- the dumped config is semantically identical but not bit-by-bit identical
@@ -43,6 +47,15 @@
The generated config can be used as input for kas:

kas build kas-project-expanded.yml
+
+ Example of the locking mechanism (call again to regenerate lockfile):
+
+ kas dump --lock kas-project.yml > kas-project.lock.yml
+
+ The generated file can be used on the kas cmdline to pin the revisions:
+
+ kas build kas-project.yml:kas-project.lock.yml
+
"""

import logging
@@ -94,6 +107,7 @@ class Dump(Checkout):
@classmethod
def setup_parser(cls, parser):
super().setup_parser(parser)
+ lk_or_env = parser.add_mutually_exclusive_group()
parser.add_argument('--format',
choices=['yaml', 'json'],
default='yaml',
@@ -105,9 +119,12 @@ class Dump(Checkout):
parser.add_argument('--resolve-refs',
action='store_true',
help='Replace floating refs with exact SHAs')
- parser.add_argument('--resolve-env',
- action='store_true',
- help='Set env defaults to captured env value')
+ lk_or_env.add_argument('--resolve-env',
+ action='store_true',
+ help='Set env defaults to captured env value')
+ lk_or_env.add_argument('--lock',
+ action='store_true',
+ help='Create lockfile with exact SHAs')

def run(self, args):
args.skip += [
@@ -119,17 +136,25 @@ class Dump(Checkout):

super().run(args)
ctx = get_context()
- config_expanded = ctx.config.get_config()
+ config_expanded = {'header': {'version': 7}} if args.lock \
+ else ctx.config.get_config()
+ repos = ctx.config.get_repos()
+
+ if args.lock:
+ args.resolve_refs = True
+ config_expanded['repos'] = \
+ {r.name: {'refspec': None} for r in repos}

# includes are already expanded, delete the key
if 'includes' in config_expanded['header']:
del config_expanded['header']['includes']

if args.resolve_refs:
- repos = ctx.config.get_repos()
for r in repos:
if r.refspec:
config_expanded['repos'][r.name]['refspec'] = r.revision
+ elif args.lock:
+ del config_expanded['repos'][r.name]

if args.resolve_env and 'env' in config_expanded:
config_expanded['env'] = ctx.config.get_environment()
--
2.30.2

[Jan Kiszka]

Looks good to me, but I like to hear feedback from people previously
asking for such a thing.

Jan

--
Siemens AG, Technology
Competence Center Embedded Linux

[Florian Bezdeka]

On Fri, 2023-01-13 at 08:23 +0100, Jan Kiszka wrote:
> On 13.01.23 07:20, Felix Moessbauer wrote:
> > This patch adds the --lock option to the dump plugin. When enabled, the
> > output only contains the resolved refspecs of each repo (as valid kas
> > format). By that, floating branches can be used in the projects kas files
> > and these can be pinned to fixed fixed revisions, when required.

^^^^^^^^^^^

Duplicate "fixed".

[Ross Burton]

Would this also resolve the SHA for the repository containing the kas files themselves?  If I use kas dump with my meta-arm layer, it resolves all refs but meta-arm:

header:

version: 11

repos:

meta-arm:

layers:

meta-arm:

meta-arm-bsp:

meta-arm-toolchain:

refspec: master

poky:

layers:

meta:

meta-poky:

refspec: 288cc5b879563f7f996733bc9da3b4316aa5ede0

url: https://git.yoctoproject.org/git/poky

meta-openembedded:

layers:

meta-filesystems:

meta-networking:

meta-oe:

meta-python:

refspec: ca7082caa800124b4e0a7b36175f60387d1e3f8c

url: https://git.openembedded.org/meta-openembedded

(thanks google for ignoring the whitespace when I pasted, but you get the gist)

Note how meta-arm:refspec: is master.

Ross

[Jan Kiszka]

On 18.01.23 17:22, Ross Burton wrote:
> Would this also resolve the SHA for the repository containing the kas
> files themselves?

Nope, that information is not tracked by kas. The concept is typically
that the root repo defines the complete dependency chain and the
involved versions. In that model, you would check in the generated dump
as lock-file into the main repo, aside the "unlocked" kas file.

[Felix Moessbauer]

Changes since v1:

- fixed typo in commit message
- added logic to create / update lockfile in-place
- added commit with logic to automatically consider lockfiles on checkout
(logically added before extension of dump plugin, as both share functionality)
- added test for dump --lock and checkout

PS: As requested, this series is also available on github:

https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v2

Best regards,
Felix Moessbauer
Siemens AG

Felix Moessbauer (3):
add support for lockfiles on checkout
dump plugin: add option to create lockfile
add test for lockfile support

kas/config.py | 4 +-
kas/includehandler.py | 19 ++++++-
kas/plugins/dump.py | 109 +++++++++++++++++++++++++++++++++--------
tests/test_commands.py | 38 ++++++++++++++
4 files changed, 148 insertions(+), 22 deletions(-)

--
2.30.2

[Felix Moessbauer]

When checking out repositories, check if a file <filename>.lock.<ext>
exists for the first filename on the kas CLI. In case this file exists
and the --update option is not specified, automatically append this
file to the kas CLI before performing any other kas operations.

When --update is specified, the lockfile is ignored.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

kas/config.py | 4 +++-
kas/includehandler.py | 19 ++++++++++++++++++-
2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/kas/config.py b/kas/config.py
index 091e727..3a50c26 100644
--- a/kas/config.py
+++ b/kas/config.py
@@ -58,7 +58,9 @@ class Config:
'belong to the same repository or all '
'must be outside of versioning control')

- self.handler = IncludeHandler(self.filenames, self.top_repo_path)
+ self.handler = IncludeHandler(self.filenames,
+ self.top_repo_path,
+ ctx.args.update)
self.repo_dict = self._get_repo_dict()

def get_build_system(self):
diff --git a/kas/includehandler.py b/kas/includehandler.py
index 0a85fb4..c699a37 100644
--- a/kas/includehandler.py
+++ b/kas/includehandler.py
@@ -26,6 +26,7 @@
"""

import os
+from pathlib import Path
from collections import OrderedDict
from collections.abc import Mapping
import functools
@@ -121,12 +122,28 @@ class IncludeHandler:
relative to the repository root path.

The includes are read and merged from the deepest level upwards.
+
+ In case ignore_lock is false, kas checks if a file <file>.lock.<ext>
+ exists next to the first entry in top_files. This file is then appended
+ to the list of top_files.
"""

- def __init__(self, top_files, top_repo_path):
+ def __init__(self, top_files, top_repo_path, ignore_lock=False):
self.top_files = top_files
self.top_repo_path = top_repo_path

+ if ignore_lock:
+ return
+
+ lockfile = self.get_lockfile()
+ if Path(lockfile).exists():
+ logging.debug('includehandler: append lockfile %s', lockfile)
+ self.top_files.append(lockfile)
+
+ def get_lockfile(self):
+ file = Path(self.top_files[0])
+ return file.parent / (file.stem + '.lock' + file.suffix)
+
def get_config(self, repos=None):
"""
Parameters:
--
2.30.2

[Felix Moessbauer]

This patch adds the --lock option to the dump plugin. When enabled, the
output only contains the resolved refspecs of each repo (as valid kas
format). By that, floating branches can be used in the projects kas files

and these can be pinned to fixed revisions, when required.

When using --lock in combination with --inplace, a lockfile named
<filename>.lock.<ext> is created next to the <filename>.<ext>. In case
multiple files are added to the kas CLI, the lockfile is only created
for the first file (by considering the merged information from all files).

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

kas/plugins/dump.py | 109 ++++++++++++++++++++++++++++++++++++--------
1 file changed, 89 insertions(+), 20 deletions(-)

diff --git a/kas/plugins/dump.py b/kas/plugins/dump.py
index 3773484..b1c3bc6 100644

--- a/kas/plugins/dump.py
+++ b/kas/plugins/dump.py
@@ -22,13 +22,17 @@
"""
This plugin implements the ``kas dump`` command.

- When this command is executed, kas will parse all referenced config
- files, expand includes and print a flattened yaml version of the
- configuration to stdout. This config is semantically identical to the
- input, but does not include any references to other configuration files.
- The output of this command can be used to further analyse the build
+ When this command is executed in default mode, kas will parse all
+ referenced config files, expand includes and print a flattened yaml version
+ of the configuration to stdout. This config is semantically identical to
+ the input, but does not include any references to other configuration
+ files. The output of this command can be used to further analyse the build
configuration.

+ When running with --lock, a lock-file is created which only contains the
+ exact refspecs of each repository. This file can be used to pin the
+ refspecs of floating branches, while still keeping an easy update path.
+
Please note:

- the dumped config is semantically identical but not bit-by-bit identical

@@ -43,12 +47,23 @@

The generated config can be used as input for kas:

kas build kas-project-expanded.yml
+
+ Example of the locking mechanism (call again to regenerate lockfile):
+

+ kas dump --lock --inplace --update kas-project.yml
+
+ The generated lockfile will automatically be used to pin the revisions:

+
+ kas build kas-project.yml

+
"""

import logging
import sys
import json
import yaml
+from typing import TypeVar, TextIO
+from dataclasses import dataclass
from collections import OrderedDict
from kas.context import get_context
from kas.plugins.checkout import Checkout
@@ -57,6 +72,35 @@ __license__ = 'MIT'
__copyright__ = 'Copyright (c) Siemens AG, 2022'


+@dataclass
+class IoTarget:
+ StrOrTextIO = TypeVar('StrOrTextIO', str, TextIO)
+
+ target: StrOrTextIO
+ managed: bool
+
+
+class IoTargetMonitor:
+ """
+ Simple monitor to unify access to file targets that need
+ to be closed (files) and ambient ones (stdout / stderr)
+ """
+
+ def __init__(self, target: IoTarget):
+ self._target = target
+ self._file = None
+
+ def __enter__(self):
+ if self._target.managed:
+ self._file = open(self._target.target, 'w')
+ return self._file
+ return self._target.target
+
+ def __exit__(self, exc_type, exc_value, traceback):
+ if self._target.managed:
+ self._file.close()
+
+
class Dump(Checkout):
"""
Implements a kas plugin that combines multiple kas configurations
@@ -94,6 +138,7 @@ class Dump(Checkout):

@classmethod
def setup_parser(cls, parser):
super().setup_parser(parser)
+ lk_or_env = parser.add_mutually_exclusive_group()
parser.add_argument('--format',
choices=['yaml', 'json'],
default='yaml',

@@ -105,9 +150,15 @@ class Dump(Checkout):

parser.add_argument('--resolve-refs',
action='store_true',
help='Replace floating refs with exact SHAs')
- parser.add_argument('--resolve-env',

+ lk_or_env.add_argument('--resolve-env',
+ action='store_true',
+ help='Set env defaults to captured env value')
+ lk_or_env.add_argument('--lock',
+ action='store_true',
+ help='Create lockfile with exact SHAs')

+ parser.add_argument('-i', '--inplace',

action='store_true',
- help='Set env defaults to captured env value')

+ help='Update lockfile in-place (reqires --lock)')

def run(self, args):
args.skip += [

@@ -119,32 +170,50 @@ class Dump(Checkout):

super().run(args)
ctx = get_context()
- config_expanded = ctx.config.get_config()
+ config_expanded = {'header': {'version': 7}} if args.lock \
+ else ctx.config.get_config()
+ repos = ctx.config.get_repos()

+ output = IoTarget(target=sys.stdout, managed=False)
+
+ if args.inplace and not args.lock:
+ logging.error('--inplace requires --lock')
+ sys.exit(1)

+
+ if args.lock:
+ args.resolve_refs = True
+ config_expanded['repos'] = \
+ {r.name: {'refspec': None} for r in repos}

+
+ if args.lock and args.inplace:
+ lockfile = ctx.config.handler.get_lockfile()
+ output = IoTarget(target=lockfile, managed=True)

# includes are already expanded, delete the key
if 'includes' in config_expanded['header']:
del config_expanded['header']['includes']

if args.resolve_refs:
- repos = ctx.config.get_repos()
for r in repos:
if r.refspec:
config_expanded['repos'][r.name]['refspec'] = r.revision
+ elif args.lock:
+ del config_expanded['repos'][r.name]

if args.resolve_env and 'env' in config_expanded:
config_expanded['env'] = ctx.config.get_environment()

- if args.format == 'json':
- json.dump(config_expanded, sys.stdout, indent=args.indent)
- sys.stdout.write('\n')
- elif args.format == 'yaml':
- yaml.dump(
- config_expanded, sys.stdout,
- indent=args.indent,
- Dumper=self.KasYamlDumper)
- else:
- logging.error('invalid format %s', args.format)
- sys.exit(1)
+ with IoTargetMonitor(output) as f:
+ if args.format == 'json':
+ json.dump(config_expanded, f, indent=args.indent)
+ sys.stdout.write('\n')
+ elif args.format == 'yaml':
+ yaml.dump(
+ config_expanded, f,
+ indent=args.indent,
+ Dumper=self.KasYamlDumper)
+ else:
+ logging.error('invalid format %s', args.format)
+ sys.exit(1)


__KAS_PLUGINS__ = [Dump]
--
2.30.2

[Felix Moessbauer]

This commit adds a test that checks the creation, effectiveness and
update of a lockfile. Testing this functionality via the dump plugin is
sufficient, as the plugin directly uses the checkout workflow.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

tests/test_commands.py | 38 ++++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)

diff --git a/tests/test_commands.py b/tests/test_commands.py
index 712a764..e8b3417 100644
--- a/tests/test_commands.py
+++ b/tests/test_commands.py
@@ -125,3 +125,41 @@ def test_dump(changedir, tmpdir, capsys):
shutil.rmtree('%s/build' % tdir, ignore_errors=True)
kas.kas(('checkout %s' % outfile).split())
assert os.path.exists('build/conf/local.conf')
+
+
+def test_lockfile(changedir, tmpdir, capsys):
+ tdir = str(tmpdir.mkdir('test_commands'))
+ shutil.rmtree(tdir, ignore_errors=True)
+ shutil.copytree('tests/test_repo_includes', tdir)
+ os.chdir(tdir)
+
+ # no lockfile yet, branches are floating
+ kas.kas('dump test.yml'.split())
+ rawspec = yaml.safe_load(capsys.readouterr().out)
+ assert(rawspec['repos']['externalrepo']['refspec'] == 'master')
+
+ # create lockfile
+ kas.kas('dump --lock --inplace test.yml'.split())
+ assert(os.path.exists('test.lock.yml'))
+
+ # lockfile is considered during import, expect pinned branches
+ kas.kas('dump test.yml'.split())
+ lockspec = yaml.safe_load(capsys.readouterr().out)
+ assert(lockspec['repos']['externalrepo']['refspec'] != 'master')
+
+ # insert older refspec into lockfile (kas 3.2 tag)
+ test_refspec = 'dc44638cd87c4d0045ea2ca441e682f3525d8b91'
+ lockspec['repos']['externalrepo']['refspec'] = test_refspec
+ with open('test.lock.yml', 'w') as f:
+ yaml.safe_dump(lockspec, f)
+
+ # check if repo is moved to specified commit
+ kas.kas('dump test.yml'.split())
+ lockspec = yaml.safe_load(capsys.readouterr().out)
+ assert(lockspec['repos']['externalrepo']['refspec'] == test_refspec)
+
+ # update lockfile, check if repo is pinned to other commit
+ kas.kas('dump --lock --inplace --update test.yml'.split())
+ with open('test.lock.yml', 'r') as f:
+ lockspec = yaml.safe_load(f)
+ assert(lockspec['repos']['externalrepo']['refspec'] != test_refspec)
--
2.30.2

[Moessbauer, Felix]

On Tue, 2023-01-24 at 11:56 +0000, Ross Burton wrote:
> So I'm seeing different behaviour on two systems that I can't easily
> explain.

Let's bring that back to the ML.
Unfortunately the initial mail somehow lost all CCs.

>
> On my local machine, kas dump --lock foo.yml does this:
>
> header:
>     version: 7
> repos:
>     meta-arm:
>         refspec: 0099eee0d01e0284aa2da5a82643cc3aeb6c6ff1
>     poky:
>         refspec: 3c3fd6a65e8103f74ae382d196d486b31a168b39
>
> But in CI:
>
> header:
>     version: 7
> repos:
>     meta-arm:
>         refspec: master
>     poky:
>         refspec: 3c3fd6a65e8103f74ae382d196d486b31a168b39
>
> Any idea why one would be saying master and the other a real SHA?

Did you run the kas dump --lock foo.yml command in both cases?
This would be indeed strange. Could it be, that the kas-version in the
CI is a different one? But that would be strange anyways, as invalid
args are reported as errors.

What would really help is a minimal reproducer. Could you just send the
foo.yml file, so I can have a look?

Best regards,
Felix

>
> Cheers,
> Ross
>
> On Tue, 24 Jan 2023 at 11:20, Ross Burton <ro...@burtonini.com> wrote:
> >
> > Thanks Felix, much appreciated.
> >
> > Ross

[Ross Burton]

On Tue, 24 Jan 2023 at 12:07, Moessbauer, Felix
<felix.mo...@siemens.com> wrote:
> > Any idea why one would be saying master and the other a real SHA?
>
> Did you run the kas dump --lock foo.yml command in both cases?
> This would be indeed strange. Could it be, that the kas-version in the
> CI is a different one? But that would be strange anyways, as invalid
> args are reported as errors.

I did indeed run the same command, the file is just the revspecs and
not the rest of the configuration.

Definitely both using your branch.

> What would really help is a minimal reproducer. Could you just send the
> foo.yml file, so I can have a look?

It's not minimal but a real reproducer is
https://gitlab.com/rossburton/meta-arm/-/tree/lock.

The update-repos step runs:

$ kas dump --lock ci/qemuarm64.yml:ci/meta-openembedded.yml

Ross

[Moessbauer, Felix]

On Tue, 2023-01-24 at 12:15 +0000, Ross Burton wrote:
> On Tue, 24 Jan 2023 at 12:07, Moessbauer, Felix
> <felix.mo...@siemens.com> wrote:
> > > Any idea why one would be saying master and the other a real SHA?
> >
> > Did you run the kas dump --lock foo.yml command in both cases?
> > This would be indeed strange. Could it be, that the kas-version in
> > the
> > CI is a different one? But that would be strange anyways, as
> > invalid
> > args are reported as errors.
>
> I did indeed run the same command, the file is just the revspecs and
> not the rest of the configuration.
>
> Definitely both using your branch.
>
> > What would really help is a minimal reproducer. Could you just send
> > the
> > foo.yml file, so I can have a look?
>
> It's not minimal but a real reproducer is
> https://gitlab.com/rossburton/meta-arm/-/tree/lock.

Thanks for the reproducer. This is a rather big scenario which is quite
valuable for our tests. One difference I see, that you use defaults for
the refspec.

>
> The update-repos step runs:
>
> $ kas dump --lock ci/qemuarm64.yml:ci/meta-openembedded.yml

I just tried that a couple of times with wiping the repos in-between,
but always get the same, fixed SHAs. Maybe you could explain a bit more
detailed how you ran the commands.

Just to make sure we are on the same page:

kas dump --lock <files...> does NOT lock anything. It just writes what
is needed to lock it to the current checkout. Nontheless, the generated
file should never contain any non-resolved refspecs.

Also, running kas dump --lock multiple times in a row does NOT fetch
new upstream changes. To do so, you need to add the --update flag in
addition (which tells any checkout-based plugin to update first).

To create a lock, you basically have two choices:

1. automatic:
kas dump --lock --inplace <file1:...>
kas [build|...] <file1:...>
this automatically looks for file1.lock.yml and appends it to the
cmdline.

2. manual:
kas dump --lock --inplace <file1:...> > my-lockfile.yml
kas [build|...] <file1:...>:my-lockfile.yml

To update the lockfile, run the same "dump" command and add --update.

Best regards,
Felix

>
> Ross
>

[Ross Burton]

I think I'm trying to hit kas+lockfiles in a way that it doesn't like.

A summary of our CI: We have a prep stage which simply updates the
reference git repos, and a build stage which has many jobs, each one
running kas with a slightly different set of yaml files. The yml
files all use branch names as refspecs so we track the branches. The
problem of this is that sometimes a repo we fetch will update during
the build, so the sources may differ.

Lockfiles solve this, surely. My plan was to just do a quick kas dump
--lock on the set of all yml files that pull in layers in the prep
stage and then pass that via artifacts to the build steps. However
because of how kas repo arrays are handled, this doesn't work.

Say I have a lock.yml with poky: and meta-openembedded: fixed SHAs in,
and then overlay that with a kas build which just uses poky. That
turns into the following parsed datastructure:

repos:
meta-arm:
layers:
meta-arm:
poky:
url: https://git.yoctoproject.org/git/poky
refspec: 3c3fd6a65e8103f74ae382d196d486b31a168b39
layers:
meta:
refspec: 3c3fd6a65e8103f74ae382d196d486b31a168b39
meta-openembedded:
refspec: ceceffcb1e3ef4f9ba7708f77c27e30a7aea61e5

Kas now gets confused and thinks that meta-arm and meta-openembedded
both refer to the current repository, so it builds an incorrect
bblayers.conf.

You could argue that I'm Doing It Wrong, but it seems like if I have
about thirty top-level build jobs, yml fragments, and I want to
snapshot the SHAs to use at once there is no other way. Do you have
any thoughts?

Cheers,
Ross

[Moessbauer, Felix]

Yes, that is indeed a problem with the current implementation.

>
> You could argue that I'm Doing It Wrong, but it seems like if I have
> about thirty top-level build jobs, yml fragments, and I want to
> snapshot the SHAs to use at once there is no other way.  Do you have
> any thoughts?

I would not say that you are using it wrong, but that at least was not
what I had in mind when implementing it. We basically have two options
to solve this:

1. Keep all repo definitions (url + branch) in one file and include
that either directly or transitively using the first file on the
cmdline (in other words: no repos in files appended to the cmdline).

2. Include the URL in the lockfile.

The probably safest way is to include the url in the lockfile.
While this might lead to more clones on checkout, it should not hurt as
these are not added to the bblayers.conf (except if you also have
default layers).

Happy to hear your opinion.

Felix

[Ross Burton]

On Tue, 24 Jan 2023 at 16:15, Moessbauer, Felix
<felix.mo...@siemens.com> wrote:
> I would not say that you are using it wrong, but that at least was not
> what I had in mind when implementing it. We basically have two options
> to solve this:
>
> 1. Keep all repo definitions (url + branch) in one file and include
> that either directly or transitively using the first file on the
> cmdline (in other words: no repos in files appended to the cmdline).
>
> 2. Include the URL in the lockfile.
>
> The probably safest way is to include the url in the lockfile.
> While this might lead to more clones on checkout, it should not hurt as
> these are not added to the bblayers.conf (except if you also have
> default layers).

If we're writing SHAs then adding the repo they came from is probably
the right thing to do.

Ross

[Ross Burton]

Wrote the one-liner patch to dump repo.url into the lockfile and I
suspect this might not be what we want:

repos:
meta-arm:
refspec: master
url: /builds/engineering/yocto/meta-arm

(when meta-arm is the repo which contains the yml files)

Ross

[Felix Moessbauer]

Changes since v2:

- add original urls to each repo in lockfile
- only add repos with operations enabled
(others cannot be changed by kas anyways)
- rebased onto next

Github: https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v3

Changes since v1:

- fixed typo in commit message
- added logic to create / update lockfile in-place
- added commit with logic to automatically consider lockfiles on checkout
(logically added before extension of dump plugin, as both share functionality)
- added test for dump --lock and checkout

PS: As requested, this series is also available on github:

https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v2

Best regards,
Felix Moessbauer
Siemens AG

Felix Moessbauer (3):
add support for lockfiles on checkout
dump plugin: add option to create lockfile
add test for lockfile support

kas/config.py | 4 +-
kas/includehandler.py | 19 ++++++-

kas/plugins/dump.py | 112 +++++++++++++++++++++++++++++++++--------
tests/test_commands.py | 38 ++++++++++++++
4 files changed, 151 insertions(+), 22 deletions(-)

--
2.30.2

[Felix Moessbauer]

When checking out repositories, check if a file <filename>.lock.<ext>
exists for the first filename on the kas CLI. In case this file exists
and the --update option is not specified, automatically append this
file to the kas CLI before performing any other kas operations.

When --update is specified, the lockfile is ignored.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

kas/config.py | 4 +++-
kas/includehandler.py | 19 ++++++++++++++++++-
2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/kas/config.py b/kas/config.py
index 091e727..3a50c26 100644
--- a/kas/config.py
+++ b/kas/config.py
@@ -58,7 +58,9 @@ class Config:
'belong to the same repository or all '
'must be outside of versioning control')

- self.handler = IncludeHandler(self.filenames, self.top_repo_path)
+ self.handler = IncludeHandler(self.filenames,
+ self.top_repo_path,
+ ctx.args.update)
self.repo_dict = self._get_repo_dict()

def get_build_system(self):
diff --git a/kas/includehandler.py b/kas/includehandler.py
index 0a85fb4..c699a37 100644
--- a/kas/includehandler.py
+++ b/kas/includehandler.py
@@ -26,6 +26,7 @@
"""

import os
+from pathlib import Path

from collections import OrderedDict

[Felix Moessbauer]

This patch adds the --lock option to the dump plugin. When enabled, the
output only contains the resolved refspecs of each repo (as valid kas
format). By that, floating branches can be used in the projects kas files
and these can be pinned to fixed revisions, when required.

When using --lock in combination with --inplace, a lockfile named
<filename>.lock.<ext> is created next to the <filename>.<ext>. In case
multiple files are added to the kas CLI, the lockfile is only created
for the first file (by considering the merged information from all files).

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

kas/plugins/dump.py | 112 ++++++++++++++++++++++++++++++++++++--------
1 file changed, 92 insertions(+), 20 deletions(-)

diff --git a/kas/plugins/dump.py b/kas/plugins/dump.py
index 3773484..7031f01 100644

from collections import OrderedDict

@@ -119,14 +170,34 @@ class Dump(Checkout):

super().run(args)
ctx = get_context()
- config_expanded = ctx.config.get_config()
+ config_expanded = {'header': {'version': 7}} if args.lock \
+ else ctx.config.get_config()
+ repos = ctx.config.get_repos()
+ output = IoTarget(target=sys.stdout, managed=False)
+
+ if args.inplace and not args.lock:
+ logging.error('--inplace requires --lock')
+ sys.exit(1)
+
+ if args.lock:
+ args.resolve_refs = True

+ # when locking, only consider repos managed by kas
+ repos = [r for r in repos if not r.operations_disabled]

+ config_expanded['repos'] = \
+ {r.name: {

+ 'refspec': None,
+ 'url': r.url
+ } for r in repos}

+
+ if args.lock and args.inplace:
+ lockfile = ctx.config.handler.get_lockfile()
+ output = IoTarget(target=lockfile, managed=True)

# includes are already expanded, delete the key
if 'includes' in config_expanded['header']:
del config_expanded['header']['includes']

if args.resolve_refs:
- repos = ctx.config.get_repos()
for r in repos:
if r.refspec:
config_expanded['repos'][r.name]['refspec'] = r.revision

@@ -134,17 +205,18 @@ class Dump(Checkout):

[Felix Moessbauer]

This commit adds a test that check the creation, effectiveness and

update of a lockfile. Testing this functionality via the dump plugin is
sufficient, as the plugin directly uses the checkout workflow.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

[Ross Burton]

On Wed, 25 Jan 2023 at 10:50, Felix Moessbauer
<felix.mo...@siemens.com> wrote:
> Changes since v2:
>
> - add original urls to each repo in lockfile
> - only add repos with operations enabled
> (others cannot be changed by kas anyways)
> - rebased onto next
>
> Github: https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v3

Actually embedded the URLs doesn't work. Unused references will still
be added to bblayers, and may even be incorrect.

If a lockfile contains:

repos:
meta-openembedded:
refspec: ceceffcb1e3ef4f9ba7708f77c27e30a7aea61e5
url: https://git.openembedded.org/meta-openembedded

But the build doesn't use meta-oe, then meta-oe is cloned (that's
fine) and then used as a layer:

BBLAYERS ?= " \
${TOPDIR}/../../meta-arm \
${TOPDIR}/../../meta-arm-bsp \
${TOPDIR}/../../meta-arm-toolchain \
${TOPDIR}/../meta-openembedded \
${TOPDIR}/../poky/meta \
${TOPDIR}/../poky/meta-poky"

That's bad, and in this case we're lucky that the meta-openembedded
repo isn't a valid layer, so the parse fails quickly.

Ross

[Moessbauer, Felix]

On Wed, 2023-01-25 at 11:56 +0000, Ross Burton wrote:
> On Wed, 25 Jan 2023 at 10:50, Felix Moessbauer
> <felix.mo...@siemens.com> wrote:
> > Changes since v2:
> >
> > - add original urls to each repo in lockfile
> > - only add repos with operations enabled
> >   (others cannot be changed by kas anyways)
> > - rebased onto next
> >
> > Github: https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v3
>
> Actually embedded the URLs doesn't work.  Unused references will
> still
> be added to bblayers, and may even be incorrect.

Just had a look at the docs and saw that this is actually the intended
behavior. Don't know why it had been implemented like that in the first
place. Anyways, that leaves us out of options for in-config approaches
where the lockfile is just a regular kas file that overwrites some
things. We could either add some magic keyword like
"repos / <name> / aslock: True" by what a repo is only locked if
already specified. But this logic again depends on the include order
and also is hard to implement (the config merger currently simply
merges the config dicts depth-first, top-bottom).

Another option would be to use a dedicated lockfile with a new schema.
However that's also not easy to get right.
The flexibility of kas configs bites us here.

I vote for keeping the current behavior (either with URL or without)
and error out when combining repos via multiple files on the kas
cmdline and locking. As stated before, the current approach works in
case all repos are provided either directly in the first kas.yml file
or in any transitively included one (because this config set is
static). This also fits well to the idea to create lockfiles based on
the first file on the kas cmdline. It further is also aligned with the
pattern that all kas-cmdline files have to come from the same repo.

All other use-cases are simply too complex to specify properly.

@Jan: What is your opinion?

Felix

[Jan Kiszka]

I cannot follow your discussion yet and are specifically unsure how
"unused reference" could make it into a build with lock files. My
expectations would have been that you generate a lock file for the same
set of configs that you later want to use it on, thus nothing would be
"unused" in that case. IOW: Why not using multiple lock files then, one
per each build that you want to lock?

Key for this new feature is that it gets a sufficiently larger number of
users. We may not be able to make everyone happy if things become too
complex and special, but at least a vast majority.

[Ross Burton]

On Wed, 1 Feb 2023 at 06:16, Jan Kiszka <jan.k...@siemens.com> wrote:
> I cannot follow your discussion yet and are specifically unsure how
> "unused reference" could make it into a build with lock files. My
> expectations would have been that you generate a lock file for the same
> set of configs that you later want to use it on, thus nothing would be
> "unused" in that case. IOW: Why not using multiple lock files then, one
> per each build that you want to lock?

I'd like our CI to run all of the jobs with the same SHAs, so we want
to generate the lockfile once at the beginning of the build. The exact
layers which get used in each job vary so a single mega-lockfile
currently doesn't work (as repo:sha entries without a corresponding
url are transformed into "this repository").

Ross

[Ross Burton]

This came up again at work so I'd like to try and bring this conversation to a conclusion.

My usecase is that I have a CI pipeline which has ~50 jobs, each constructed via overlaying a number of Kas yml files.  We have a base.yml (pulls in the current repository and poky), machine-specific yml fragments, and overlays for features like clang or xen (which pull in the layers they need).  The repos all use branch names as the refspec.  We have the minimal number of layers enabled in each build so that we don't accidentally depend on a recipe in eg meta-python depending on the layer explicitly.

We'd like to be able to resolve the SHAs we are using _once_ at the beginning of the pipeline and lock them down for all of the jobs, so we don't have jobs that are using different SHAs if a repository gets pushed to during the pipeline.  The kas dump lockfile support looks ideal for this: dump the SHAs in a pre-build job, and use that lockfile in every kas invocation.

However, as discussed this doesn't work due to how kas parses repositories with no url.  The uber-lockfile will contain SHAs for meta-clang and meta-virtualisation but a build using GCC and no Xen won't pull those layers in.  Our main repository isn't a layer itself but contains layers, but with the lockfile Kas reads the clang: revspec: abcd lines and interprets that as being an entry for the current directory as a layer, writes a malformed bblayers.conf, and the build breaks.

I believe this is a very valid usecase of Kas lockfiles so hopefully there's a solution to be found.

Ross

[Jan Kiszka]

If the "only" blocker is that a repo entry without a layer and a URL is
interpreted as "self" while we need a placeholder in case this repo pops
up, we have to make this placeholder case explicit. We can't change the
convenient "self" form as it would break all existing recipes, and that
only for a subset of use cases.

How about adding "refspec-locked" as node, functionally equivalent to
"refspec", maybe just with high prio and without the implicit definition
of the self-repo?

[Ross Burton]

On Tue, 21 Feb 2023 at 17:21, Jan Kiszka <jan.k...@siemens.com> wrote:
> If the "only" blocker is that a repo entry without a layer and a URL is
> interpreted as "self" while we need a placeholder in case this repo pops
> up, we have to make this placeholder case explicit. We can't change the
> convenient "self" form as it would break all existing recipes, and that
> only for a subset of use cases.
>
> How about adding "refspec-locked" as node, functionally equivalent to
> "refspec", maybe just with high prio and without the implicit definition
> of the self-repo?

refspec-locked is certainly a less-worse option that others I've considered...

Ross

[Ross Burton]

Thinking about this more, a less invasive option would be a separate locked-refspecs block at the same level as repos.  This could just map repo names to refspecs and override the value in the repos dictionaries.  The advantage of this is that the semantics on how repos works wouldn't need to be changed.

Ross 

[Jan Kiszka]

OTOH, a repo entry (after folding all includes in) like

repo-name:
refspec: XXX

would be a combination that has no meaning in current kas. We could
filter that out, I think.

[Ross Burton]

On Fri, 24 Feb 2023 at 08:32, Jan Kiszka <jan.k...@siemens.com> wrote:
> > Thinking about this more, a less invasive option would be a separate
> > locked-refspecs block at the same level as repos. This could just map
> > repo names to refspecs and override the value in the repos dictionaries.
> > The advantage of this is that the semantics on how repos works wouldn't
> > need to be changed.
>
> OTOH, a repo entry (after folding all includes in) like
>
> repo-name:
> refspec: XXX
>
> would be a combination that has no meaning in current kas. We could
> filter that out, I think.

That's perfectly possible, if you have a refspec set in defaults and
are referring to the current directory.

Ross

[Jan Kiszka]

I see - too bad. But clearer node names aren't a bad idea per se. We
have issues around refspec being a commit ID or a branch.

[Ross Burton]

On Thursday, 23 February 2023 at 12:35:13 UTC Ross Burton wrote:

Thinking about this more, a less invasive option would be a separate locked-refspecs block at the same level as repos.  This could just map repo names to refspecs and override the value in the repos dictionaries.  The advantage of this is that the semantics on how repos works wouldn't need to be changed.

So I went and implemented this and it does the right thing for my use-case.  It's not pretty but it works and should help move the discussion on.

Reiterating, I have two needs:

1) A CI pipeline consists of multiple jobs. We do many builds with the minimal layers required (to verify that we don't accidentally have undocumented dependencies) but want to pin all of the SHAs _once_ at the beginning of the build.

2) Our CI uses floating branch references, but given a known good pipeline, we should be able to replicate that easily.

The lock plugin I wrote in https://github.com/rossburton/kas/commits/lock solves these needs.

(1) is done by having a job that runs first that does 'kas lock' with the union of all layers that we use. This generates a lock.yml with the refspecs in an 'overrides' section, which takes priority over repos and defaults objects when parsing:

header:
    version: 13
overrides:
    repos:
        meta-openembedded:
            refspec: ceceffcb1e3ef4f9ba7708f77c27e30a7aea61e5
        poky:
            refspec: ea4c56e2e883b904380bc34e9202436322f41bf1

This lock.yml is then used in the build jobs, so the SHAs are pinned during the pipeline execution.

(2) is to make feature branches easier to work with.  If a branch we're tracking temporarily breaks (say, oe-core introduces a breaking change which means other layers need to catch up), we can trivially take the SHAs from the last good pipeline (as we'll persist lock.yml as an artefact) and use that in local development/feature branches to override the floating references until the breakage is resolved.

Any thoughts?

Ross

[Felix Moessbauer]

Changes since v3:

- complete redesign of locking mechanism (as worked out by Ross and Jan)
- implementation of locking using dedicated key (based on
pre-work by Ross, many thanks!)
- improvements of documentation
- rebased onto next

Changes since v2:

- add original urls to each repo in lockfile
- only add repos with operations enabled
(others cannot be changed by kas anyways)
- rebased onto next

Github: https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v3

Changes since v1:

- fixed typo in commit message
- added logic to create / update lockfile in-place
- added commit with logic to automatically consider lockfiles on checkout
(logically added before extension of dump plugin, as both share functionality)
- added test for dump --lock and checkout

PS: As requested, this series is also available on github:

https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v2

Best regards,
Felix Moessbauer
Siemens AG

Felix Moessbauer (4):
add support to override refspec of repos

add support for lockfiles on checkout
dump plugin: add option to create lockfile
add test for lockfile support

docs/format-changelog.rst | 1 +
docs/userguide.rst | 16 ++++++
kas/config.py | 9 ++-
kas/includehandler.py | 19 ++++++-
kas/plugins/dump.py | 112 +++++++++++++++++++++++++++++++-------
kas/repos.py | 7 ++-
kas/schema-kas.json | 18 ++++++
tests/test_commands.py | 38 +++++++++++++
8 files changed, 193 insertions(+), 27 deletions(-)

--
2.30.2

[Felix Moessbauer]

When checking out repositories, check if a file <filename>.lock.<ext>
exists for the first filename on the kas CLI. In case this file exists
and the --update option is not specified, automatically append this
file to the kas CLI before performing any other kas operations.

When --update is specified, the lockfile is ignored.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

kas/config.py | 4 +++-
kas/includehandler.py | 19 ++++++++++++++++++-
2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/kas/config.py b/kas/config.py

index b3eb311..f5c0bc4 100644

--- a/kas/config.py
+++ b/kas/config.py
@@ -58,7 +58,9 @@ class Config:
'belong to the same repository or all '
'must be outside of versioning control')

- self.handler = IncludeHandler(self.filenames, self.top_repo_path)
+ self.handler = IncludeHandler(self.filenames,
+ self.top_repo_path,
+ ctx.args.update)
self.repo_dict = self._get_repo_dict()

def get_build_system(self):
diff --git a/kas/includehandler.py b/kas/includehandler.py
index 0a85fb4..c699a37 100644
--- a/kas/includehandler.py
+++ b/kas/includehandler.py
@@ -26,6 +26,7 @@
"""

import os
+from pathlib import Path

from collections import OrderedDict

[Felix Moessbauer]

This patch adds the top-level `overrides` entry, which is used to
override (or pin) the refspec of repositories. The main difference to a
direct override is that this logic only applies to repos that are
already defined. By that, a superset of all repos can be added to this
entry (similar to a global lockfile), but only the currently active ones
are affected.

Proposed-by: Ross Burton <ro...@burtonini.com>

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

docs/format-changelog.rst | 1 +
docs/userguide.rst | 16 ++++++++++++++++
kas/config.py | 5 ++++-
kas/repos.py | 7 ++++---
kas/schema-kas.json | 18 ++++++++++++++++++
5 files changed, 43 insertions(+), 4 deletions(-)

diff --git a/docs/format-changelog.rst b/docs/format-changelog.rst
index 073ea0f..b97906e 100644
--- a/docs/format-changelog.rst
+++ b/docs/format-changelog.rst
@@ -139,3 +139,4 @@ Added

- The integrity of repositories can now be securely validated via providing a
``sha256sum`` value.
+- The ``overrides`` top-level entry can be used to pin floating repo refspecs.
diff --git a/docs/userguide.rst b/docs/userguide.rst
index b72dbb1..d77358f 100644
--- a/docs/userguide.rst
+++ b/docs/userguide.rst
@@ -455,6 +455,22 @@ Configuration reference
* ``path``: string [required]
The path to one patch file or a quilt formatted patchset directory.

+* ``overrides``: dict [optional]
+ This object provides a mechanism to override kas configuration items without
+ defining them. By that, only items that already exist are overridden. Note,
+ that all entries below this key are reserved for auto-generation using kas
+ plugins. Do not manually add entries.
+
+ * ``repos``: dict [optional]
+ Mapps to the top-level ``repos`` entry.
+
+ * ``<repo-id>``: dict [optional]
+ Mapps to the ``<repo-id>`` entry.
+
+ * ``refspec``: string [optional]
+ Pinned refspec which overrides the ``refspec`` of the corresponding
+ repo. This refspec must be resolved (i.e. no branch or tag name).
+
* ``bblayers_conf_header``: dict [optional]
This contains strings that should be added to the ``bblayers.conf`` before
any layers are included.
diff --git a/kas/config.py b/kas/config.py
index 091e727..b3eb311 100644
--- a/kas/config.py
+++ b/kas/config.py
@@ -104,11 +104,14 @@ class Config:
`name`.
"""
repo_defaults = self._config.get('defaults', {}).get('repos', {})
+ overrides = self._config.get('overrides', {}) \
+ .get('repos', {}).get(name, {})
config = self.get_repos_config()[name] or {}
return Repo.factory(name,
config,
repo_defaults,
- self.top_repo_path)
+ self.top_repo_path,
+ overrides)

def _get_repo_dict(self):
"""
diff --git a/kas/repos.py b/kas/repos.py
index 40f3c24..b470bff 100644
--- a/kas/repos.py
+++ b/kas/repos.py
@@ -93,7 +93,8 @@ class Repo:
self.path, self._layers)

@staticmethod
- def factory(name, repo_config, repo_defaults, repo_fallback_path):
+ def factory(name, repo_config, repo_defaults, repo_fallback_path,
+ repo_overrides={}):
"""
Returns a Repo instance depending on params.
"""
@@ -122,8 +123,8 @@ class Repo:
url = repo_config.get('url', None)
name = repo_config.get('name', name)
typ = repo_config.get('type', 'git')
- refspec = repo_config.get('refspec',
- repo_defaults.get('refspec', None))
+ refspec = repo_overrides.get('refspec', repo_config.get('refspec',
+ repo_defaults.get('refspec', None)))
if refspec is None and url is not None:
logging.error('No refspec specified for repository "%s". This is '
'only allowed for local repositories.', name)
diff --git a/kas/schema-kas.json b/kas/schema-kas.json
index da8a9c1..39af600 100644
--- a/kas/schema-kas.json
+++ b/kas/schema-kas.json
@@ -85,6 +85,24 @@
}
}
},
+ "overrides": {
+ "type": "object",
+ "additionalProperties": false,
+ "properties": {
+ "repos": {
+ "type": "object",
+ "additionalProperties": {
+ "type": "object",
+ "additionalProperties": false,
+ "properties": {
+ "refspec" : {
+ "type": "string"
+ }
+ }
+ }
+ }
+ }
+ },
"machine": {
"type": "string"
},
--
2.30.2

[Felix Moessbauer]

This patch adds the --lock option to the dump plugin. When enabled, the
output only contains the resolved refspecs of each repo (as valid kas
format). By that, floating branches can be used in the projects kas files
and these can be pinned to fixed revisions, when required.

When using --lock in combination with --inplace, a lockfile named
<filename>.lock.<ext> is created next to the <filename>.<ext>. In case
multiple files are added to the kas CLI, the lockfile is only created
for the first file (by considering the merged information from all files).

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

kas/plugins/dump.py | 112 +++++++++++++++++++++++++++++++++++---------
1 file changed, 91 insertions(+), 21 deletions(-)

diff --git a/kas/plugins/dump.py b/kas/plugins/dump.py
index 3773484..1d11f37 100644

from collections import OrderedDict

@@ -119,14 +170,32 @@ class Dump(Checkout):

super().run(args)
ctx = get_context()
- config_expanded = ctx.config.get_config()

+ schema_v = 14 if args.lock else 7
+ config_expanded = {'header': {'version': schema_v}} if args.lock \

+ else ctx.config.get_config()
+ repos = ctx.config.get_repos()
+ output = IoTarget(target=sys.stdout, managed=False)
+
+ if args.inplace and not args.lock:
+ logging.error('--inplace requires --lock')
+ sys.exit(1)
+
+ if args.lock:
+ args.resolve_refs = True
+ # when locking, only consider repos managed by kas
+ repos = [r for r in repos if not r.operations_disabled]

+ config_expanded['overrides'] = \
+ {'repos': {r.name: {'refspec': r.revision} for r in repos}}

+
+ if args.lock and args.inplace:
+ lockfile = ctx.config.handler.get_lockfile()
+ output = IoTarget(target=lockfile, managed=True)

# includes are already expanded, delete the key
if 'includes' in config_expanded['header']:
del config_expanded['header']['includes']

- if args.resolve_refs:
- repos = ctx.config.get_repos()
+ if args.resolve_refs and not args.lock:

for r in repos:
if r.refspec:
config_expanded['repos'][r.name]['refspec'] = r.revision

@@ -134,17 +203,18 @@ class Dump(Checkout):

[Felix Moessbauer]

This commit adds a test that check the creation, effectiveness and
update of a lockfile. Testing this functionality via the dump plugin is
sufficient, as the plugin directly uses the checkout workflow.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

tests/test_commands.py | 38 ++++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)

diff --git a/tests/test_commands.py b/tests/test_commands.py

index d08988b..5020d17 100644
--- a/tests/test_commands.py
+++ b/tests/test_commands.py
@@ -120,3 +120,41 @@ def test_dump(changedir, tmpdir, capsys):

shutil.rmtree('%s/build' % tdir, ignore_errors=True)
kas.kas(('checkout %s' % outfile).split())
assert os.path.exists('build/conf/local.conf')
+
+
+def test_lockfile(changedir, tmpdir, capsys):
+ tdir = str(tmpdir.mkdir('test_commands'))
+ shutil.rmtree(tdir, ignore_errors=True)
+ shutil.copytree('tests/test_repo_includes', tdir)
+ os.chdir(tdir)
+
+ # no lockfile yet, branches are floating
+ kas.kas('dump test.yml'.split())
+ rawspec = yaml.safe_load(capsys.readouterr().out)
+ assert(rawspec['repos']['externalrepo']['refspec'] == 'master')
+
+ # create lockfile
+ kas.kas('dump --lock --inplace test.yml'.split())
+ assert(os.path.exists('test.lock.yml'))
+
+ # lockfile is considered during import, expect pinned branches
+ kas.kas('dump test.yml'.split())
+ lockspec = yaml.safe_load(capsys.readouterr().out)

+ assert(lockspec['overrides']['repos']['externalrepo']['refspec'] != 'master')

+
+ # insert older refspec into lockfile (kas 3.2 tag)
+ test_refspec = 'dc44638cd87c4d0045ea2ca441e682f3525d8b91'

+ lockspec['overrides']['repos']['externalrepo']['refspec'] = test_refspec

+ with open('test.lock.yml', 'w') as f:
+ yaml.safe_dump(lockspec, f)
+
+ # check if repo is moved to specified commit
+ kas.kas('dump test.yml'.split())
+ lockspec = yaml.safe_load(capsys.readouterr().out)

+ assert(lockspec['overrides']['repos']['externalrepo']['refspec'] == test_refspec)

+
+ # update lockfile, check if repo is pinned to other commit
+ kas.kas('dump --lock --inplace --update test.yml'.split())
+ with open('test.lock.yml', 'r') as f:
+ lockspec = yaml.safe_load(f)

+ assert(lockspec['overrides']['repos']['externalrepo']['refspec'] != test_refspec)
--
2.30.2

[Moessbauer, Felix]

On Fri, 2023-03-03 at 15:29 +0000, Ross Burton wrote:
> Thanks for running with my proof-of-concept!  Can you push that to
> github to make it easier to test in CI?

Sure, here you go:
https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v4

I also asked Jan offline if he could build a container to test that
more easily in downstream projects.

Felix

>
> Cheers,
> Ross

[Jan Kiszka]

On 03.03.23 16:37, Moessbauer, Felix (T CED INW-CN) wrote:
> On Fri, 2023-03-03 at 15:29 +0000, Ross Burton wrote:
>> Thanks for running with my proof-of-concept! Can you push that to
>> github to make it easier to test in CI?
>
> Sure, here you go:
> https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v4
>
> I also asked Jan offline if he could build a container to test that
> more easily in downstream projects.
>

OK, exceptionally taking to next without in-depth review so that you can
play if the weekend is grey. ;)

[Felix Moessbauer]

dump plugin: add option to create lockfile

Changes since v4:

- style fixes (now compliant with scripts/checkcode.sh)
- python3.6 compatible (as dedicated commit to be reverted later)

Changes since v3:

- complete redesign of locking mechanism (as worked out by Ross and Jan)
- implementation of locking using dedicated key (based on pre-work by
Ross, many thanks!)
- improvements of documentation
- rebased onto next

Changes since v2:

- add original urls to each repo in lockfile
- only add repos with operations enabled
(others cannot be changed by kas anyways)
- rebased onto next

Github: https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v3

Changes since v1:

- fixed typo in commit message
- added logic to create / update lockfile in-place
- added commit with logic to automatically consider lockfiles on checkout
(logically added before extension of dump plugin, as both share functionality)
- added test for dump --lock and checkout

PS: As requested, this series is also available on github:

https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v2

Best regards,
Felix Moessbauer
Siemens AG

Felix Moessbauer (5):

add support to override refspec of repos
add support for lockfiles on checkout
dump plugin: add option to create lockfile

dump plugin: make python3.6 compatible

add test for lockfile support

docs/format-changelog.rst | 1 +
docs/userguide.rst | 16 ++++++
kas/config.py | 9 ++-
kas/includehandler.py | 19 ++++++-

kas/plugins/dump.py | 114 +++++++++++++++++++++++++++++++-------

kas/repos.py | 7 ++-
kas/schema-kas.json | 18 ++++++

tests/test_commands.py | 41 ++++++++++++++
8 files changed, 198 insertions(+), 27 deletions(-)

--
2.30.2

[Felix Moessbauer]

When checking out repositories, check if a file <filename>.lock.<ext>
exists for the first filename on the kas CLI. In case this file exists
and the --update option is not specified, automatically append this
file to the kas CLI before performing any other kas operations.

When --update is specified, the lockfile is ignored.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

kas/config.py | 4 +++-
kas/includehandler.py | 19 ++++++++++++++++++-
2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/kas/config.py b/kas/config.py
index b3eb311..f5c0bc4 100644
--- a/kas/config.py
+++ b/kas/config.py

@@ -58,7 +58,9 @@ class Config:
'belong to the same repository or all '
'must be outside of versioning control')

- self.handler = IncludeHandler(self.filenames, self.top_repo_path)
+ self.handler = IncludeHandler(self.filenames,
+ self.top_repo_path,
+ ctx.args.update)
self.repo_dict = self._get_repo_dict()

def get_build_system(self):
diff --git a/kas/includehandler.py b/kas/includehandler.py
index 0a85fb4..c699a37 100644
--- a/kas/includehandler.py
+++ b/kas/includehandler.py
@@ -26,6 +26,7 @@
"""

import os
+from pathlib import Path

from collections import OrderedDict

[Felix Moessbauer]

This patch adds the top-level `overrides` entry, which is used to
override (or pin) the refspec of repositories. The main difference to a
direct override is that this logic only applies to repos that are
already defined. By that, a superset of all repos can be added to this
entry (similar to a global lockfile), but only the currently active ones
are affected.

Proposed-by: Ross Burton <ro...@burtonini.com>

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

docs/format-changelog.rst | 1 +

docs/userguide.rst | 16 ++++++++++++++++
kas/config.py | 5 ++++-
kas/repos.py | 7 ++++---
kas/schema-kas.json | 18 ++++++++++++++++++
5 files changed, 43 insertions(+), 4 deletions(-)

diff --git a/docs/format-changelog.rst b/docs/format-changelog.rst
index 073ea0f..b97906e 100644
--- a/docs/format-changelog.rst
+++ b/docs/format-changelog.rst
@@ -139,3 +139,4 @@ Added

- The integrity of repositories can now be securely validated via providing a
``sha256sum`` value.
+- The ``overrides`` top-level entry can be used to pin floating repo refspecs.
diff --git a/docs/userguide.rst b/docs/userguide.rst

index b72dbb1..a6b1c5b 100644

--- a/docs/userguide.rst
+++ b/docs/userguide.rst
@@ -455,6 +455,22 @@ Configuration reference
* ``path``: string [required]
The path to one patch file or a quilt formatted patchset directory.

+* ``overrides``: dict [optional]
+ This object provides a mechanism to override kas configuration items

+ without defining them. By that, only items that already exist are
+ overridden. Note, that all entries below this key are reserved for
+ auto-generation using kas plugins. Do not manually add entries.

+
+ * ``repos``: dict [optional]
+ Mapps to the top-level ``repos`` entry.
+
+ * ``<repo-id>``: dict [optional]
+ Mapps to the ``<repo-id>`` entry.
+
+ * ``refspec``: string [optional]
+ Pinned refspec which overrides the ``refspec`` of the corresponding
+ repo. This refspec must be resolved (i.e. no branch or tag name).
+
* ``bblayers_conf_header``: dict [optional]
This contains strings that should be added to the ``bblayers.conf`` before
any layers are included.

diff --git a/kas/config.py b/kas/config.py
index 091e727..b3eb311 100644
--- a/kas/config.py
+++ b/kas/config.py

[Felix Moessbauer]

This patch avoids the use of dataclasses in the dump plugin to make it
compatible with python3.6. Once the support for that version is dropped,
this commit should be reverted.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

kas/plugins/dump.py | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/kas/plugins/dump.py b/kas/plugins/dump.py
index 1d11f37..7d51194 100644
--- a/kas/plugins/dump.py
+++ b/kas/plugins/dump.py
@@ -63,7 +63,6 @@ import sys
import json
import yaml
from typing import TypeVar, TextIO
-from dataclasses import dataclass
from collections import OrderedDict

from kas.context import get_context
from kas.plugins.checkout import Checkout

@@ -72,13 +71,16 @@ __license__ = 'MIT'

__copyright__ = 'Copyright (c) Siemens AG, 2022'

-@dataclass
class IoTarget:

StrOrTextIO = TypeVar('StrOrTextIO', str, TextIO)

target: StrOrTextIO
managed: bool

+ def __init__(self, target, managed):
+ self.target = target
+ self.managed = managed
+

class IoTargetMonitor:
"""
--
2.30.2

[Felix Moessbauer]

This patch adds the --lock option to the dump plugin. When enabled, the
output only contains the resolved refspecs of each repo (as valid kas
format). By that, floating branches can be used in the projects kas files
and these can be pinned to fixed revisions, when required.

When using --lock in combination with --inplace, a lockfile named
<filename>.lock.<ext> is created next to the <filename>.<ext>. In case
multiple files are added to the kas CLI, the lockfile is only created
for the first file (by considering the merged information from all files).

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

kas/plugins/dump.py | 112 +++++++++++++++++++++++++++++++++++---------
1 file changed, 91 insertions(+), 21 deletions(-)

diff --git a/kas/plugins/dump.py b/kas/plugins/dump.py
index 3773484..1d11f37 100644
--- a/kas/plugins/dump.py
+++ b/kas/plugins/dump.py

import sys
import json
import yaml

+from typing import TypeVar, TextIO

+from dataclasses import dataclass

from collections import OrderedDict
from kas.context import get_context
from kas.plugins.checkout import Checkout

@@ -57,6 +72,35 @@ __license__ = 'MIT'

__copyright__ = 'Copyright (c) Siemens AG, 2022'

[Felix Moessbauer]

This commit adds a test that check the creation, effectiveness and
update of a lockfile. Testing this functionality via the dump plugin is
sufficient, as the plugin directly uses the checkout workflow.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

tests/test_commands.py | 41 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)

diff --git a/tests/test_commands.py b/tests/test_commands.py
index d08988b..0d32fd9 100644
--- a/tests/test_commands.py
+++ b/tests/test_commands.py
@@ -120,3 +120,44 @@ def test_dump(changedir, tmpdir, capsys):

shutil.rmtree('%s/build' % tdir, ignore_errors=True)
kas.kas(('checkout %s' % outfile).split())
assert os.path.exists('build/conf/local.conf')
+
+
+def test_lockfile(changedir, tmpdir, capsys):
+ tdir = str(tmpdir.mkdir('test_commands'))
+ shutil.rmtree(tdir, ignore_errors=True)
+ shutil.copytree('tests/test_repo_includes', tdir)
+ os.chdir(tdir)
+
+ # no lockfile yet, branches are floating
+ kas.kas('dump test.yml'.split())
+ rawspec = yaml.safe_load(capsys.readouterr().out)
+ assert(rawspec['repos']['externalrepo']['refspec'] == 'master')
+
+ # create lockfile
+ kas.kas('dump --lock --inplace test.yml'.split())
+ assert(os.path.exists('test.lock.yml'))
+
+ # lockfile is considered during import, expect pinned branches
+ kas.kas('dump test.yml'.split())
+ lockspec = yaml.safe_load(capsys.readouterr().out)
+ assert(lockspec['overrides']['repos']['externalrepo']['refspec']

+ != 'master')

+
+ # insert older refspec into lockfile (kas 3.2 tag)
+ test_refspec = 'dc44638cd87c4d0045ea2ca441e682f3525d8b91'
+ lockspec['overrides']['repos']['externalrepo']['refspec'] = test_refspec
+ with open('test.lock.yml', 'w') as f:
+ yaml.safe_dump(lockspec, f)
+
+ # check if repo is moved to specified commit
+ kas.kas('dump test.yml'.split())
+ lockspec = yaml.safe_load(capsys.readouterr().out)
+ assert(lockspec['overrides']['repos']['externalrepo']['refspec']

+ == test_refspec)

+
+ # update lockfile, check if repo is pinned to other commit
+ kas.kas('dump --lock --inplace --update test.yml'.split())
+ with open('test.lock.yml', 'r') as f:
+ lockspec = yaml.safe_load(f)
+ assert(lockspec['overrides']['repos']['externalrepo']['refspec']

+ != test_refspec)
--
2.30.2

[Jan Kiszka]

That's not worth the bisection breakage, specifically as we have lots of
that cases in the code base already. I will merge it into patch 3.

[Jan Kiszka]

On 03.03.23 18:25, Felix Moessbauer wrote:

assert <condition>, not assert(<condition>). I had fixed this already in
the previous version which I'm using now again.

[Jan Kiszka]

On 03.03.23 18:25, Felix Moessbauer wrote:

Still not python3.5 compatible. If it's as easy to fix as the dataclass
topic, please do. I'm not sure which LTS distros still ship 3.5 (3.6 is
in SUSE at least), but I'm also not categorically against dropping
support for it in the next major release. That would be step one before
taking this pattern, though.

Jan

[Ross Burton]

On Sat, 4 Mar 2023 at 06:58, Jan Kiszka <jan.k...@siemens.com> wrote:

Still not python3.5 compatible. If it's as easy to fix as the dataclass
topic, please do. I'm not sure which LTS distros still ship 3.5 (3.6 is
in SUSE at least), but I'm also not categorically against dropping
support for it in the next major release. That would be step one before
taking this pattern, though.

FWIW, current bitbake needs 3.8. Some distros still ship

older I believe but those are almost EOL and need the buildtools to work, so have a new Python from that.

Ross

[Moessbauer, Felix]

Everything is possible, but is it really worth the additional
maintenance effort? Biggest issue for me is that it is hard to test if
something is compatible with older python versions.

Anyways, both Python 3.5 and 3.6 are EOL [1]. Hence, I vote for
dropping at least support for 3.5, better for both.

[1] https://devguide.python.org/versions/

Felix

[Jan Kiszka]

Right, but oldest Yocto LTS, living until 2024 apparently, is still
3.5-compatible.

Jan

[Ross Burton]

On Friday, 3 March 2023 at 16:36:52 UTC Jan Kiszka wrote:

OK, exceptionally taking to next without in-depth review so that you can
play if the weekend is grey. ;)

Beyond the minor comments you've already made, do you have any more serious feedback?

Thanks,

Ross

[Moessbauer, Felix]

Unfortunately this series fails in python 3.5. While trying to make my
patches 3.5 compatible, I stumbled upon python internal bugs in 3.5
which cannot be easily circumvented. Unfortunately this bug [1] affects
the libkas and not just the plugin, so either we rewrite bigger parts
of the code, or we simply drop 3.5.

For major distros, the situation looks as following, whereby the
version denotes the lowest still supported python version in any
supported distro version.

- Debian: 3.7 (buster)
- Ubuntu: 3.7 (18.04)
- Fedora: 3.11 (Fedora 36)
- RHEL: 2.x (RHEL 7), 3.6 (RHEL 8)
- OpenSUSE: 3.6 (Leap / 15.4)

By that, I conclude that we can safely drop support for python 3.5.

Best regards,
Felix

[1]
File "/work/kas/includehandler.py", line 56, in load_config
(_, ext) = os.path.splitext(filename)
File "/usr/local/lib/python3.5/posixpath.py", line 122, in splitext
return genericpath._splitext(p, sep, None, extsep)
File "/usr/local/lib/python3.5/genericpath.py", line 118, in
_splitext
sepIndex = p.rfind(sep)
AttributeError: 'PosixPath' object has no attribute 'rfind'


>
> Thanks,
> Ross
> --
> You received this message because you are subscribed to the Google
> Groups "kas-devel" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to kas-devel+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kas-devel/ff234fe3-832b-4024-a721-cbede76e65b6n%40googlegroups.com
> .

[Felix Moessbauer]

As all currently supported distros provide at least python 3.6, we
drop the support for 3.5. At time of this commit, the python versions
of the supported distros are as following:

- Debian: 3.7 (buster)
- Ubuntu: 3.7 (18.04)
- Fedora: 3.11 (Fedora 36)
- RHEL: 2.x (RHEL 7), 3.6 (RHEL 8)
- OpenSUSE: 3.6 (Leap / 15.4)

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

.github/workflows/next.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/next.yml b/.github/workflows/next.yml
index 57a52d3..14c3c64 100644
--- a/.github/workflows/next.yml
+++ b/.github/workflows/next.yml
@@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-20.04
strategy:
matrix:
- python-version: ["3.5", "3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]
+ python-version: ["3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]
steps:
- name: Check out repo
uses: actions/checkout@v3
--
2.30.2

[Felix Moessbauer]

dump plugin: add option to create lockfile

kas: add auto-locking support

Changes since v5:

- rebased onto next
- add patch to drop python 3.5
- avoid dedicated python 3.6 commit
- fix issue when args object does not have update attribute

Changes since v4:

- style fixes (now compliant with scripts/checkcode.sh)
- python3.6 compatible (as dedicated commit to be reverted later)

Changes since v3:

- complete redesign of locking mechanism (as worked out by Ross and Jan)
- implementation of locking using dedicated key (based on pre-work by
Ross, many thanks!)
- improvements of documentation
- rebased onto next

Changes since v2:

- add original urls to each repo in lockfile
- only add repos with operations enabled
(others cannot be changed by kas anyways)
- rebased onto next

Github: https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v3

Changes since v1:

- fixed typo in commit message
- added logic to create / update lockfile in-place
- added commit with logic to automatically consider lockfiles on checkout
(logically added before extension of dump plugin, as both share functionality)
- added test for dump --lock and checkout

PS: As requested, this series is also available on github:

https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v6

Best regards,
Felix Moessbauer
Siemens AG

Felix Moessbauer (5):

drop support for Python 3.5

add support to override refspec of repos
add support for lockfiles on checkout
dump plugin: add option to create lockfile

add test for lockfile support

.github/workflows/next.yml | 2 +-

docs/format-changelog.rst | 1 +
docs/userguide.rst | 16 ++++++

kas/config.py | 11 +++-
kas/includehandler.py | 19 ++++++-
kas/plugins/dump.py | 114 ++++++++++++++++++++++++++++++-------

kas/repos.py | 7 ++-
kas/schema-kas.json | 18 ++++++
tests/test_commands.py | 41 +++++++++++++

9 files changed, 201 insertions(+), 28 deletions(-)

--
2.30.2

[Felix Moessbauer]

This patch adds the top-level `overrides` entry, which is used to
override (or pin) the refspec of repositories. The main difference to a
direct override is that this logic only applies to repos that are
already defined. By that, a superset of all repos can be added to this
entry (similar to a global lockfile), but only the currently active ones
are affected.

Proposed-by: Ross Burton <ro...@burtonini.com>
Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
docs/format-changelog.rst | 1 +

[Felix Moessbauer]

This patch adds the --lock option to the dump plugin. When enabled, the
output only contains the resolved refspecs of each repo (as valid kas
format). By that, floating branches can be used in the projects kas files
and these can be pinned to fixed revisions, when required.

When using --lock in combination with --inplace, a lockfile named
<filename>.lock.<ext> is created next to the <filename>.<ext>. In case
multiple files are added to the kas CLI, the lockfile is only created
for the first file (by considering the merged information from all files).

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>

[Jan: fold in Python 3.6 support]

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

kas/plugins/dump.py | 114 ++++++++++++++++++++++++++++++++++++--------
1 file changed, 93 insertions(+), 21 deletions(-)

diff --git a/kas/plugins/dump.py b/kas/plugins/dump.py
index 3773484..7d51194 100644

--- a/kas/plugins/dump.py
+++ b/kas/plugins/dump.py
@@ -22,13 +22,17 @@
"""
This plugin implements the ``kas dump`` command.

- When this command is executed, kas will parse all referenced config
- files, expand includes and print a flattened yaml version of the
- configuration to stdout. This config is semantically identical to the
- input, but does not include any references to other configuration files.
- The output of this command can be used to further analyse the build
+ When this command is executed in default mode, kas will parse all
+ referenced config files, expand includes and print a flattened yaml version
+ of the configuration to stdout. This config is semantically identical to
+ the input, but does not include any references to other configuration
+ files. The output of this command can be used to further analyse the build
configuration.

+ When running with --lock, a lock-file is created which only contains the

+ exact refspecs of each repository. This file can be used to pin the

+ refspecs of floating branches, while still keeping an easy update path.
+
Please note:

- the dumped config is semantically identical but not bit-by-bit identical

@@ -43,12 +47,22 @@

The generated config can be used as input for kas:

kas build kas-project-expanded.yml
+
+ Example of the locking mechanism (call again to regenerate lockfile):
+
+ kas dump --lock --inplace --update kas-project.yml
+
+ The generated lockfile will automatically be used to pin the revisions:
+
+ kas build kas-project.yml
+
"""

import logging
import sys
import json
import yaml
+from typing import TypeVar, TextIO

from collections import OrderedDict
from kas.context import get_context
from kas.plugins.checkout import Checkout

@@ -57,6 +71,38 @@ __license__ = 'MIT'

__copyright__ = 'Copyright (c) Siemens AG, 2022'

+class IoTarget:
+ StrOrTextIO = TypeVar('StrOrTextIO', str, TextIO)
+
+ target: StrOrTextIO
+ managed: bool

+

+ def __init__(self, target, managed):
+ self.target = target
+ self.managed = managed

@@ -94,6 +140,7 @@ class Dump(Checkout):

@classmethod
def setup_parser(cls, parser):
super().setup_parser(parser)
+ lk_or_env = parser.add_mutually_exclusive_group()
parser.add_argument('--format',
choices=['yaml', 'json'],
default='yaml',

@@ -105,9 +152,15 @@ class Dump(Checkout):

parser.add_argument('--resolve-refs',
action='store_true',
help='Replace floating refs with exact SHAs')
- parser.add_argument('--resolve-env',
+ lk_or_env.add_argument('--resolve-env',
+ action='store_true',
+ help='Set env defaults to captured env value')
+ lk_or_env.add_argument('--lock',
+ action='store_true',
+ help='Create lockfile with exact SHAs')
+ parser.add_argument('-i', '--inplace',
action='store_true',
- help='Set env defaults to captured env value')
+ help='Update lockfile in-place (reqires --lock)')

def run(self, args):
args.skip += [

@@ -119,14 +172,32 @@ class Dump(Checkout):

@@ -134,17 +205,18 @@ class Dump(Checkout):

--
2.30.2

[Felix Moessbauer]

This commit adds a test that check the creation, effectiveness and
update of a lockfile. Testing this functionality via the dump plugin is
sufficient, as the plugin directly uses the checkout workflow.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
[Jan: fix over-long lines and removed assert brackets]

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

tests/test_commands.py | 41 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)

diff --git a/tests/test_commands.py b/tests/test_commands.py

index d08988b..57edf54 100644

--- a/tests/test_commands.py
+++ b/tests/test_commands.py
@@ -120,3 +120,44 @@ def test_dump(changedir, tmpdir, capsys):
shutil.rmtree('%s/build' % tdir, ignore_errors=True)
kas.kas(('checkout %s' % outfile).split())
assert os.path.exists('build/conf/local.conf')
+
+
+def test_lockfile(changedir, tmpdir, capsys):
+ tdir = str(tmpdir.mkdir('test_commands'))
+ shutil.rmtree(tdir, ignore_errors=True)
+ shutil.copytree('tests/test_repo_includes', tdir)
+ os.chdir(tdir)
+
+ # no lockfile yet, branches are floating
+ kas.kas('dump test.yml'.split())
+ rawspec = yaml.safe_load(capsys.readouterr().out)

+ assert rawspec['repos']['externalrepo']['refspec'] == 'master'

+
+ # create lockfile
+ kas.kas('dump --lock --inplace test.yml'.split())

+ assert os.path.exists('test.lock.yml')

+
+ # lockfile is considered during import, expect pinned branches
+ kas.kas('dump test.yml'.split())
+ lockspec = yaml.safe_load(capsys.readouterr().out)

+ assert lockspec['overrides']['repos']['externalrepo']['refspec'] \
+ != 'master'

+
+ # insert older refspec into lockfile (kas 3.2 tag)
+ test_refspec = 'dc44638cd87c4d0045ea2ca441e682f3525d8b91'
+ lockspec['overrides']['repos']['externalrepo']['refspec'] = test_refspec
+ with open('test.lock.yml', 'w') as f:
+ yaml.safe_dump(lockspec, f)
+
+ # check if repo is moved to specified commit
+ kas.kas('dump test.yml'.split())
+ lockspec = yaml.safe_load(capsys.readouterr().out)

+ assert lockspec['overrides']['repos']['externalrepo']['refspec'] \
+ == test_refspec

+
+ # update lockfile, check if repo is pinned to other commit
+ kas.kas('dump --lock --inplace --update test.yml'.split())
+ with open('test.lock.yml', 'r') as f:
+ lockspec = yaml.safe_load(f)

+ assert lockspec['overrides']['repos']['externalrepo']['refspec'] \
+ != test_refspec
--
2.30.2

[Felix Moessbauer]

When checking out repositories, check if a file <filename>.lock.<ext>
exists for the first filename on the kas CLI. In case this file exists
and the --update option is not specified, automatically append this
file to the kas CLI before performing any other kas operations.

When --update is specified, the lockfile is ignored.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

kas/config.py | 6 +++++-
kas/includehandler.py | 19 ++++++++++++++++++-
2 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/kas/config.py b/kas/config.py
index b3eb311..6b2f477 100644
--- a/kas/config.py
+++ b/kas/config.py
@@ -58,7 +58,11 @@ class Config:

'belong to the same repository or all '
'must be outside of versioning control')

- self.handler = IncludeHandler(self.filenames, self.top_repo_path)

+ update = ctx.args.update if hasattr(ctx.args, 'update') else False
+

+ self.handler = IncludeHandler(self.filenames,
+ self.top_repo_path,

+ update)

self.repo_dict = self._get_repo_dict()

def get_build_system(self):
diff --git a/kas/includehandler.py b/kas/includehandler.py
index 0a85fb4..c699a37 100644
--- a/kas/includehandler.py
+++ b/kas/includehandler.py
@@ -26,6 +26,7 @@
"""

import os
+from pathlib import Path

from collections import OrderedDict

[Jan Kiszka]

You should also update setup.py. In fact, that needs updating to cover
our upper limits as well.

[Jan Kiszka]

On 09.03.23 22:25, Felix Moessbauer wrote:

> This patch adds the top-level `overrides` entry, which is used to
> override (or pin) the refspec of repositories. The main difference to a
> direct override is that this logic only applies to repos that are
> already defined. By that, a superset of all repos can be added to this
> entry (similar to a global lockfile), but only the currently active ones
> are affected.

The reasoning should go here as well why a special handling of

some-repo:
refspec: <locked>

ie. the absence of path and url during the presence of refspec is not
enough. I already forgot that again.

On the one side, I like the clear semantic for the specific use case. On
the other, kas includes are all about overrides. I'm concerned that
users may get confused to use this for things it does not even support.

Is that correctly indented? At least it is hard to read this way.

This would also need a format revision bump.

[Jan Kiszka]

On 09.03.23 22:25, Felix Moessbauer wrote:

> When checking out repositories, check if a file <filename>.lock.<ext>
> exists for the first filename on the kas CLI. In case this file exists
> and the --update option is not specified, automatically append this
> file to the kas CLI before performing any other kas operations.

Where should this file be located? Only in the folder of that first
config file? Or also the kas workdir? I assume its the former so that
the file could also be checked in, but that should be clarified and
explained.

This could probably also go under a "if not ignore_lock:" block. Then
you may consider renaming the argument to "use_lock".

> +
> + def get_lockfile(self):
> + file = Path(self.top_files[0])
> + return file.parent / (file.stem + '.lock' + file.suffix)
> +
> def get_config(self, repos=None):
> """
> Parameters:

[Felix Moessbauer]

This patch adds the top-level `overrides` entry, which is used to
override (or pin) the refspec of repositories. The main difference to a
direct override is that this logic only applies to repos that are
already defined. By that, a superset of all repos can be added to this
entry (similar to a global lockfile), but only the currently active ones

are affected. A new top-level keyword is required because everything
below the "repos" keyword is potentially defined by "default" values.
For the locking mechanism, a clear separation between overrides (only
override if existing) and definitions is required to be able to use a
global lockfile with all possible repos, while just defining some repos.

Proposed-by: Ross Burton <ro...@burtonini.com>

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

+
+ * ``repos``: dict [optional]
+ Mapps to the top-level ``repos`` entry.
+
+ * ``<repo-id>``: dict [optional]
+ Mapps to the ``<repo-id>`` entry.
+
+ * ``refspec``: string [optional]
+ Pinned refspec which overrides the ``refspec`` of the corresponding
+ repo. This refspec must be resolved (i.e. no branch or tag name).
+
* ``bblayers_conf_header``: dict [optional]
This contains strings that should be added to the ``bblayers.conf`` before
any layers are included.

diff --git a/kas/config.py b/kas/config.py
index 091e727..b3eb311 100644
--- a/kas/config.py
+++ b/kas/config.py

--
2.30.2

[Felix Moessbauer]

dump plugin: add option to create lockfile
kas: add auto-locking support

Changes since v6:

- p1: sync with python versions in setup.py
- p2: add reson for overrides keyword in commit message
- p3: be more precise about where the lockfile needs to be located,
invert implementation of locking (`use_lock` instead of `ignore_lock`),
no user-facing change.

Changes since v5:

- rebased onto next
- add patch to drop python 3.5
- avoid dedicated python 3.6 commit
- fix issue when args object does not have update attribute

Changes since v4:

- style fixes (now compliant with scripts/checkcode.sh)
- python3.6 compatible (as dedicated commit to be reverted later)

Changes since v3:

- complete redesign of locking mechanism (as worked out by Ross and Jan)
- implementation of locking using dedicated key (based on pre-work by
Ross, many thanks!)
- improvements of documentation
- rebased onto next

Changes since v2:

- add original urls to each repo in lockfile
- only add repos with operations enabled
(others cannot be changed by kas anyways)
- rebased onto next

Changes since v1:

- fixed typo in commit message
- added logic to create / update lockfile in-place
- added commit with logic to automatically consider lockfiles on checkout
(logically added before extension of dump plugin, as both share functionality)
- added test for dump --lock and checkout

PS: As requested, this series is also available on github:

https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v7

Best regards,
Felix Moessbauer
Siemens AG

Felix Moessbauer (5):
drop support for Python 3.5
add support to override refspec of repos
add support for lockfiles on checkout
dump plugin: add option to create lockfile
add test for lockfile support

.github/workflows/next.yml | 2 +-

docs/format-changelog.rst | 1 +

docs/userguide.rst | 16 ++++++
kas/config.py | 11 +++-

kas/includehandler.py | 17 +++++-

kas/plugins/dump.py | 114 ++++++++++++++++++++++++++++++-------
kas/repos.py | 7 ++-
kas/schema-kas.json | 18 ++++++

setup.py | 3 +-
tests/test_commands.py | 41 +++++++++++++
10 files changed, 201 insertions(+), 29 deletions(-)

--
2.30.2

[Felix Moessbauer]

As all currently supported distros provide at least python 3.6, we
drop the support for 3.5. At time of this commit, the python versions
of the supported distros are as following:

- Debian: 3.7 (buster)
- Ubuntu: 3.7 (18.04)
- Fedora: 3.11 (Fedora 36)
- RHEL: 2.x (RHEL 7), 3.6 (RHEL 8)
- OpenSUSE: 3.6 (Leap / 15.4)

While updating the lower bound version, we also unify the upper bound in
the setup.py script with the versions tested in the CI.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---
.github/workflows/next.yml | 2 +-

setup.py | 3 ++-
2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/next.yml b/.github/workflows/next.yml
index 57a52d3..14c3c64 100644
--- a/.github/workflows/next.yml
+++ b/.github/workflows/next.yml
@@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-20.04
strategy:
matrix:
- python-version: ["3.5", "3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]
+ python-version: ["3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]
steps:
- name: Check out repo
uses: actions/checkout@v3

diff --git a/setup.py b/setup.py
index fccff1a..2536cba 100644
--- a/setup.py
+++ b/setup.py
@@ -70,11 +70,12 @@ setup(
'License :: OSI Approved :: MIT License',

'Programming Language :: Python :: 3',
- 'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
+ 'Programming Language :: Python :: 3.10',
+ 'Programming Language :: Python :: 3.11',
],
keywords='OpenEmbedded bitbake development',

--
2.30.2

[Felix Moessbauer]

When checking out repositories, check if a file <filename>.lock.<ext>

exists next to the file specified first on the kas CLI. In case this

file exists and the --update option is not specified, automatically
append this file to the kas CLI before performing any other kas
operations.

When --update is specified, the lockfile is ignored.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

kas/config.py | 6 +++++-
kas/includehandler.py | 17 ++++++++++++++++-
2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/kas/config.py b/kas/config.py
index b3eb311..ede8544 100644
--- a/kas/config.py
+++ b/kas/config.py

@@ -58,7 +58,11 @@ class Config:
'belong to the same repository or all '
'must be outside of versioning control')

- self.handler = IncludeHandler(self.filenames, self.top_repo_path)
+ update = ctx.args.update if hasattr(ctx.args, 'update') else False
+
+ self.handler = IncludeHandler(self.filenames,
+ self.top_repo_path,

+ not update)

self.repo_dict = self._get_repo_dict()

def get_build_system(self):
diff --git a/kas/includehandler.py b/kas/includehandler.py

index 0a85fb4..c42b227 100644

--- a/kas/includehandler.py
+++ b/kas/includehandler.py
@@ -26,6 +26,7 @@
"""

import os
+from pathlib import Path
from collections import OrderedDict
from collections.abc import Mapping
import functools

@@ -121,12 +122,26 @@ class IncludeHandler:

relative to the repository root path.

The includes are read and merged from the deepest level upwards.
+
+ In case ignore_lock is false, kas checks if a file <file>.lock.<ext>
+ exists next to the first entry in top_files. This file is then appended
+ to the list of top_files.
"""

- def __init__(self, top_files, top_repo_path):

+ def __init__(self, top_files, top_repo_path, use_lock=True):

self.top_files = top_files
self.top_repo_path = top_repo_path

+ if use_lock:

+ lockfile = self.get_lockfile()
+ if Path(lockfile).exists():
+ logging.debug('includehandler: append lockfile %s', lockfile)
+ self.top_files.append(lockfile)

+
+ def get_lockfile(self):
+ file = Path(self.top_files[0])
+ return file.parent / (file.stem + '.lock' + file.suffix)
+
def get_config(self, repos=None):
"""
Parameters:

--
2.30.2

[Felix Moessbauer]

This patch adds the --lock option to the dump plugin. When enabled, the
output only contains the resolved refspecs of each repo (as valid kas
format). By that, floating branches can be used in the projects kas files
and these can be pinned to fixed revisions, when required.

When using --lock in combination with --inplace, a lockfile named
<filename>.lock.<ext> is created next to the <filename>.<ext>. In case
multiple files are added to the kas CLI, the lockfile is only created
for the first file (by considering the merged information from all files).

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>

[Jan: fold in Python 3.6 support]

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

from collections import OrderedDict

[Felix Moessbauer]

This commit adds a test that check the creation, effectiveness and
update of a lockfile. Testing this functionality via the dump plugin is
sufficient, as the plugin directly uses the checkout workflow.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>

[Jan: fix over-long lines and removed assert brackets]

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

[Ross Burton]

Hi,

Any progress on reviewing this? We'd like to use this feature in our CI.

Ross

[Jan Kiszka]

On 09.03.23 22:25, Felix Moessbauer wrote:

As I'm starting to merge things now, feel free to rebase and address the
open issues in some v7.

[Moessbauer, Felix]

Hi Jan,

these points already have been implemented in the v7, sent out on
03/13/23. Never mind, I'll rebase it and send it as v8.

Felix

>
> Jan
>

[Felix Moessbauer]

dump plugin: add option to create lockfile
kas: add auto-locking support

Changes since v7:

- rebased onto next
- sync python version in "python_requires" in setup.py

Changes since v6:

- p1: sync with python versions in setup.py
- p2: add reson for overrides keyword in commit message
- p3: be more precise about where the lockfile needs to be located,
invert implementation of locking (`use_lock` instead of `ignore_lock`),
no user-facing change.

Changes since v5:

- rebased onto next
- add patch to drop python 3.5
- avoid dedicated python 3.6 commit
- fix issue when args object does not have update attribute

Changes since v4:

- style fixes (now compliant with scripts/checkcode.sh)
- python3.6 compatible (as dedicated commit to be reverted later)

Changes since v3:

- complete redesign of locking mechanism (as worked out by Ross and Jan)
- implementation of locking using dedicated key (based on pre-work by
Ross, many thanks!)
- improvements of documentation
- rebased onto next

Changes since v2:

- add original urls to each repo in lockfile
- only add repos with operations enabled
(others cannot be changed by kas anyways)
- rebased onto next

Changes since v1:

- fixed typo in commit message
- added logic to create / update lockfile in-place
- added commit with logic to automatically consider lockfiles on checkout
(logically added before extension of dump plugin, as both share functionality)
- added test for dump --lock and checkout

PS: As requested, this series is also available on github:

https://github.com/fmoessbauer/kas/tree/fm/dump-lock-v8

Best regards,
Felix Moessbauer
Siemens AG

Felix Moessbauer (5):
drop support for Python 3.5
add support to override refspec of repos
add support for lockfiles on checkout
dump plugin: add option to create lockfile
add test for lockfile support

.github/workflows/next.yml | 2 +-

docs/format-changelog.rst | 8 +++

docs/userguide.rst | 16 ++++++
kas/config.py | 11 +++-

kas/includehandler.py | 17 +++++-
kas/plugins/dump.py | 114 ++++++++++++++++++++++++++++++-------

kas/repos.py | 7 ++-
kas/schema-kas.json | 18 ++++++

setup.py | 7 ++-
tests/test_commands.py | 41 +++++++++++++
10 files changed, 210 insertions(+), 31 deletions(-)

--
2.39.2

[Felix Moessbauer]

As all currently supported distros provide at least python 3.6, we
drop the support for 3.5. At time of this commit, the python versions
of the supported distros are as following:

- Debian: 3.7 (buster)
- Ubuntu: 3.7 (18.04)
- Fedora: 3.11 (Fedora 36)
- RHEL: 2.x (RHEL 7), 3.6 (RHEL 8)
- OpenSUSE: 3.6 (Leap / 15.4)

While updating the lower bound version, we also unify the upper bound in
the setup.py script with the versions tested in the CI.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---
.github/workflows/next.yml | 2 +-

setup.py | 7 ++++---
2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/.github/workflows/next.yml b/.github/workflows/next.yml
index 57a52d3..14c3c64 100644
--- a/.github/workflows/next.yml
+++ b/.github/workflows/next.yml
@@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-20.04
strategy:
matrix:
- python-version: ["3.5", "3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]
+ python-version: ["3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]
steps:
- name: Check out repo
uses: actions/checkout@v3
diff --git a/setup.py b/setup.py

index fccff1a..cec38c7 100644

--- a/setup.py
+++ b/setup.py
@@ -70,11 +70,12 @@ setup(
'License :: OSI Approved :: MIT License',

'Programming Language :: Python :: 3',
- 'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
+ 'Programming Language :: Python :: 3.10',
+ 'Programming Language :: Python :: 3.11',
],
keywords='OpenEmbedded bitbake development',

@@ -95,6 +96,6 @@ setup(
'kconfiglib>=14.1.0,<15',
],

- # At least python 3.5 is needed by now for PyYAML:
- python_requires='>=3.5',
+ # At least python 3.6 is needed by now:
+ python_requires='>=3.6',
)
--
2.39.2

[Felix Moessbauer]

This patch adds the top-level `overrides` entry, which is used to
override (or pin) the refspec of repositories. The main difference to a
direct override is that this logic only applies to repos that are
already defined. By that, a superset of all repos can be added to this
entry (similar to a global lockfile), but only the currently active ones
are affected. A new top-level keyword is required because everything
below the "repos" keyword is potentially defined by "default" values.
For the locking mechanism, a clear separation between overrides (only

override if existing) and definitions is required to be able to define a

global lockfile with all possible repos, while just defining some repos.

Proposed-by: Ross Burton <ro...@burtonini.com>
Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

docs/format-changelog.rst | 8 ++++++++

docs/userguide.rst | 16 ++++++++++++++++
kas/config.py | 5 ++++-
kas/repos.py | 7 ++++---
kas/schema-kas.json | 18 ++++++++++++++++++

5 files changed, 50 insertions(+), 4 deletions(-)

diff --git a/docs/format-changelog.rst b/docs/format-changelog.rst
index 23c5380..bc7d24d 100644
--- a/docs/format-changelog.rst
+++ b/docs/format-changelog.rst
@@ -130,3 +130,11 @@ Added

- Variables used in the ``env`` section can now be assigned 'None' as value. In
this case the variables are only exported to the bb env whitelist.
+
+Version 14
+----------
+
+Added
+~~~~~
+

diff --git a/kas/config.py b/kas/config.py
index 091e727..b3eb311 100644
--- a/kas/config.py
+++ b/kas/config.py

@@ -104,11 +104,14 @@ class Config:
`name`.
"""
repo_defaults = self._config.get('defaults', {}).get('repos', {})
+ overrides = self._config.get('overrides', {}) \
+ .get('repos', {}).get(name, {})
config = self.get_repos_config()[name] or {}
return Repo.factory(name,
config,
repo_defaults,
- self.top_repo_path)
+ self.top_repo_path,
+ overrides)

def _get_repo_dict(self):
"""
diff --git a/kas/repos.py b/kas/repos.py

index cfec816..410e290 100644
--- a/kas/repos.py
+++ b/kas/repos.py
@@ -90,7 +90,8 @@ class Repo:

self.path, self._layers)

@staticmethod
- def factory(name, repo_config, repo_defaults, repo_fallback_path):
+ def factory(name, repo_config, repo_defaults, repo_fallback_path,
+ repo_overrides={}):
"""
Returns a Repo instance depending on params.
"""

@@ -119,8 +120,8 @@ class Repo:

url = repo_config.get('url', None)
name = repo_config.get('name', name)
typ = repo_config.get('type', 'git')
- refspec = repo_config.get('refspec',
- repo_defaults.get('refspec', None))
+ refspec = repo_overrides.get('refspec', repo_config.get('refspec',
+ repo_defaults.get('refspec', None)))
if refspec is None and url is not None:
logging.error('No refspec specified for repository "%s". This is '
'only allowed for local repositories.', name)
diff --git a/kas/schema-kas.json b/kas/schema-kas.json

index 6775463..77072e7 100644
--- a/kas/schema-kas.json
+++ b/kas/schema-kas.json
@@ -89,6 +89,24 @@

}
}
},
+ "overrides": {
+ "type": "object",
+ "additionalProperties": false,
+ "properties": {
+ "repos": {
+ "type": "object",
+ "additionalProperties": {
+ "type": "object",
+ "additionalProperties": false,
+ "properties": {
+ "refspec" : {
+ "type": "string"
+ }
+ }
+ }
+ }
+ }
+ },
"machine": {
"type": "string"
},
--

2.39.2

[Felix Moessbauer]

When checking out repositories, check if a file <filename>.lock.<ext>
exists next to the file specified first on the kas CLI. In case this
file exists and the --update option is not specified, automatically
append this file to the kas CLI before performing any other kas
operations.

When --update is specified, the lockfile is ignored.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

kas/config.py | 6 +++++-
kas/includehandler.py | 17 ++++++++++++++++-
2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/kas/config.py b/kas/config.py
index b3eb311..ede8544 100644
--- a/kas/config.py
+++ b/kas/config.py

@@ -58,7 +58,11 @@ class Config:
'belong to the same repository or all '
'must be outside of versioning control')

- self.handler = IncludeHandler(self.filenames, self.top_repo_path)
+ update = ctx.args.update if hasattr(ctx.args, 'update') else False
+
+ self.handler = IncludeHandler(self.filenames,
+ self.top_repo_path,
+ not update)
self.repo_dict = self._get_repo_dict()

def get_build_system(self):
diff --git a/kas/includehandler.py b/kas/includehandler.py
index 0a85fb4..c42b227 100644
--- a/kas/includehandler.py
+++ b/kas/includehandler.py
@@ -26,6 +26,7 @@
"""

import os
+from pathlib import Path

from collections import OrderedDict

2.39.2

[Felix Moessbauer]

This patch adds the --lock option to the dump plugin. When enabled, the
output only contains the resolved refspecs of each repo (as valid kas
format). By that, floating branches can be used in the projects kas files
and these can be pinned to fixed revisions, when required.

When using --lock in combination with --inplace, a lockfile named
<filename>.lock.<ext> is created next to the <filename>.<ext>. In case
multiple files are added to the kas CLI, the lockfile is only created
for the first file (by considering the merged information from all files).

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>

[Jan: fold in Python 3.6 support]

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

kas/plugins/dump.py | 114 ++++++++++++++++++++++++++++++++++++--------
1 file changed, 93 insertions(+), 21 deletions(-)

diff --git a/kas/plugins/dump.py b/kas/plugins/dump.py

index 8b765e7..1832b2b 100644

from collections import OrderedDict

2.39.2

[Felix Moessbauer]

This commit adds a test that check the creation, effectiveness and
update of a lockfile. Testing this functionality via the dump plugin is
sufficient, as the plugin directly uses the checkout workflow.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>

[Jan: fix over-long lines and removed assert brackets]

Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---

2.39.2

[Jan Kiszka]

Thanks, applied.

If we introduce 'commit'/'branch' properties and deprecate 'refspec'
before the release, override would be switched over to 'commit' along
that. But I still need to write that code.

[Jan Kiszka]

On 23.04.23 11:42, Felix Moessbauer wrote:

You forgot to bump __file_version__ - fixing up...

Jan