[PATCH 3/5] only warn about artifact timestamp on attestation

[Felix Moessbauer]

Previously, a warning was issued whenever a kas build was executed and
the artifacts specified in the config file have not been created
in-between the start and the end of the build. However, these
artifacts can also be there from a previous build (or partial build),
hence the timestamps are expected to differ on local executions.

This patch moves this check to the attestation part, which is expected
to always start from a clean state (sstate is ok, but no lingering and
untracked files). There it is an important debugging mechanism.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---
kas/attestation.py | 17 ++++++++++++++++-
kas/plugins/build.py | 18 ------------------
2 files changed, 16 insertions(+), 19 deletions(-)

diff --git a/kas/attestation.py b/kas/attestation.py
index b290eb804..d42416b41 100644
--- a/kas/attestation.py
+++ b/kas/attestation.py
@@ -33,7 +33,7 @@ import sys
from enum import Enum
from urllib.parse import urlparse
from pathlib import Path
-from datetime import timezone
+from datetime import datetime, timezone
from kas import __version__ as KASVERSION


@@ -202,11 +202,26 @@ class Statement:
self._t_started = t_started
self._t_finished = t_finished

+ def _check_artifact_timestamp(self, name, path):
+ """
+ Warn if artifact timestamp is not within the build range.
+ """
+ logging.debug(f'Found artifact {name}:{path} in build dir')
+ fullpath = Path(self._ctx.build_dir) / path
+ mtime = datetime.fromtimestamp(fullpath.stat().st_mtime)
+ if mtime < self._t_started or mtime > self._t_finished:
+ logging.warning(
+ f'Artifact {name}:{path.name} mtime {mtime.strftime("%c")}'
+ f' not in build range '
+ f'[{self._t_started.strftime("%c")} - '
+ f'{self._t_finished.strftime("%c")}]')
+
def as_dict(self):
pt = self._predicate.type_()
pp = self._predicate.as_dict()
subjects = []
for n, s in self._ctx.config.get_artifacts():
+ self._check_artifact_timestamp(n, s)
fullpath = Path(self._ctx.build_dir) / s
with open(fullpath, "rb") as f:
if sys.version_info < (3, 11):
diff --git a/kas/plugins/build.py b/kas/plugins/build.py
index 8028aa98a..b3b9f08cd 100644
--- a/kas/plugins/build.py
+++ b/kas/plugins/build.py
@@ -128,22 +128,6 @@ class BuildCommand(Command):
f.write(json.dumps(stmt, indent=4))
f.write('\n')

- def _warn_artifact_timestamp(self, ctx, artifacts,
- t_started, t_finished):
- """
- Warn if artifact timestamp is not within the build range.
- """
- for n, s in artifacts:
- logging.debug(f'Found artifact {n}:{s} in build dir')
- fullpath = Path(ctx.build_dir) / s
- mtime = datetime.fromtimestamp(fullpath.stat().st_mtime)
- if mtime < t_started or mtime > t_finished:
- logging.warning(
- f'Artifact {n}:{s.name} mtime {mtime.strftime("%c")}'
- f' not in build range '
- f'[{t_started.strftime("%c")} - '
- f'{t_finished.strftime("%c")}]')
-
def execute(self, ctx):
"""
Executes the bitbake build command.
@@ -164,8 +148,6 @@ class BuildCommand(Command):
time_finished = datetime.now()

artifacts = ctx.config.get_artifacts()
- self._warn_artifact_timestamp(ctx, artifacts,
- time_started, time_finished)

if ctx.args.provenance:
mode = Provenance.Mode.MAX if ctx.args.provenance == 'mode=max' \
--
2.39.2

[Felix Moessbauer]

While the in-toto spec uses the term "subjects", the corresponding key
in the kas project config is called "artifacts". By that, we also name
it "artifacts" in the error message.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

kas/attestation.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kas/attestation.py b/kas/attestation.py
index 3e5f72795..b290eb804 100644
--- a/kas/attestation.py
+++ b/kas/attestation.py
@@ -219,7 +219,7 @@ class Statement:
}
subjects.append(rd)
if len(subjects) == 0:
- logging.warning('Attestation does not contain any subjects.')
+ logging.warning('Attestation does not contain any artifacts.')
st = {
'_type': INTOTO_STATEMENT_TYPE,
'subject': subjects,
--
2.39.2

[Felix Moessbauer]

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

docs/userguide/build-attestation.rst | 2 ++
1 file changed, 2 insertions(+)

diff --git a/docs/userguide/build-attestation.rst b/docs/userguide/build-attestation.rst
index d426fb40f..9164c612b 100644
--- a/docs/userguide/build-attestation.rst
+++ b/docs/userguide/build-attestation.rst
@@ -19,6 +19,8 @@ the mode controls the amount of information that is included in the
attestation. The CLI options hereby loosely follow the ``docker buildx``
provenance options, which are described in detail in
`Provenance attestations <https://docs.docker.com/build/attestations/slsa-provenance/>`_.
+For compatibility with the ``docker buildx`` CLI options, we also support
+``--provenance true``, which is equivalent to ``--provenance mode=min``.

In min mode, the provenance attestation will contain information about:

--
2.39.2

[Felix Moessbauer]

Previously, the build was marked failed in case at least one artifact
was not found after the bitbake returned. However, this is too strict,
as some build commands which are used during development (e.g. clean),
don't generate artifacts by design.

Instead, we now only require the artifacts to be present when also
generating build attestations. This patch updates the specification
(docs), the implementation as well as the test for missing artifacts.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

docs/userguide/project-configuration.rst | 5 +++--
kas/attestation.py | 2 +-
kas/config.py | 6 ++++--
kas/plugins/build.py | 6 ++++--
tests/test_build.py | 8 +++++---
5 files changed, 17 insertions(+), 10 deletions(-)

diff --git a/docs/userguide/project-configuration.rst b/docs/userguide/project-configuration.rst
index e4d9ff36c..878f9a17a 100644
--- a/docs/userguide/project-configuration.rst
+++ b/docs/userguide/project-configuration.rst
@@ -446,8 +446,9 @@ Configuration reference
This entry describes artifacts which are expected to be present after
executing the build. Each key-value pair describes an identifier and a
path relative to the kas build dir, whereby the path can contain wildcards
- like '*'. Unix-style globbing is applied to all paths and at least one file
- per entry must match. Otherwise the build is considered to be failed.
+ like '*'. Unix-style globbing is applied to all paths. In case no artifact
+ is found, the build is considered successful, if not stated otherwise by
+ the used plugin and mode of operation.

.. note:: There are no further semantics attached to the identifiers (yet).
Both the author and the consumer of the artifacts node need to
diff --git a/kas/attestation.py b/kas/attestation.py
index d42416b41..dfa5d22a4 100644
--- a/kas/attestation.py
+++ b/kas/attestation.py
@@ -220,7 +220,7 @@ class Statement:

pt = self._predicate.type_()
pp = self._predicate.as_dict()
subjects = []

- for n, s in self._ctx.config.get_artifacts():
+ for n, s in self._ctx.config.get_artifacts(missing_ok=False):

self._check_artifact_timestamp(n, s)
fullpath = Path(self._ctx.build_dir) / s
with open(fullpath, "rb") as f:

diff --git a/kas/config.py b/kas/config.py
index 8281178d7..de50ca9b3 100644
--- a/kas/config.py
+++ b/kas/config.py
@@ -223,16 +223,18 @@ class Config:
multiconfigs.add(target.split(':')[1])
return ' '.join(multiconfigs)

- def get_artifacts(self):
+ def get_artifacts(self, missing_ok=True):
"""
Returns the found artifacts after glob expansion, relative
to the build_dir as a list of tuples (name, path).
+ If missing_ok=False, raises an ArtifactNotFoundError if no
+ artifact for a given name is found.
"""
arts = self._config.get('artifacts', {})
foundfiles = []
for name, art in arts.items():
files = list(Path(self._build_dir).glob(art))
- if len(files) == 0:
+ if not missing_ok and len(files) == 0:
raise ArtifactNotFoundError(name, art)
foundfiles.extend([(name, f) for f in files])
return [(n, f.relative_to(self._build_dir))
diff --git a/kas/plugins/build.py b/kas/plugins/build.py
index b3b9f08cd..f626b5c22 100644
--- a/kas/plugins/build.py
+++ b/kas/plugins/build.py
@@ -30,6 +30,10 @@
provenance attestation for the build. The attestation will be stored in
``attestation/kas-build.provenance.json`` in the build directory.
For details about provenance, see the build attestation chapter.
+
+ .. note::
+ In provenance mode, the command returns with a non-zero exit
+ code in case no artifact is found for at least one entry.
"""

import logging
@@ -147,8 +151,6 @@ class BuildCommand(Command):
run_cmd(cmd, cwd=ctx.build_dir, liveupdate=True)
time_finished = datetime.now()

- artifacts = ctx.config.get_artifacts()
-

if ctx.args.provenance:
mode = Provenance.Mode.MAX if ctx.args.provenance == 'mode=max' \

else Provenance.Mode.MIN
diff --git a/tests/test_build.py b/tests/test_build.py
index 1903103bb..1771a5a6d 100644
--- a/tests/test_build.py
+++ b/tests/test_build.py
@@ -36,9 +36,7 @@ def test_artifact_node(monkeykas, tmpdir):
monkeykas.chdir(tdir)
kas.kas(['build', 'artifact-named.yml'])
kas.kas(['build', 'artifact-glob.yml'])
-
- with pytest.raises(ArtifactNotFoundError):
- kas.kas(['build', 'artifact-invalid.yml'])
+ kas.kas(['build', 'artifact-invalid.yml'])


def test_provenance(monkeykas, tmpdir):
@@ -46,6 +44,10 @@ def test_provenance(monkeykas, tmpdir):
shutil.copytree('tests/test_build', tdir)
monkeykas.chdir(tdir)

+ with pytest.raises(ArtifactNotFoundError):
+ kas.kas(['build', '--provenance', 'mode=min',
+ 'artifact-invalid.yml'])
+
kas.kas(['build', '--provenance', 'mode=min', 'provenance.yml'])
with open('build/attestation/kas-build.provenance.json', 'r') as f:
prov = json.load(f)
--
2.39.2

[Felix Moessbauer]

This patch improves the attestation signing part of the documentation by
adding links to the external tooling, as well as where to find the
external documentation.

Proposed-by: Joerg Sommer <joerg....@navimatix.de>

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---

docs/userguide/build-attestation.rst | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/docs/userguide/build-attestation.rst b/docs/userguide/build-attestation.rst
index 9164c612b..3747a9518 100644
--- a/docs/userguide/build-attestation.rst
+++ b/docs/userguide/build-attestation.rst
@@ -45,10 +45,12 @@ For example, to build the configuration described in the file
Working with sigstore cosign
----------------------------

-The sigstore cosign tool has native support for in-toto build predicates.
-However, it currently can only operate directly on the predicate but not
-on the enclosing attestation (cosign 2.2.4). By that, the predicate first
-needs to be extracted (provenance in this example)::
+The `cosign tool <https://github.com/sigstore/cosign>`_ from the `sigstore
+project <https://www.sigstore.dev/>`_ (`documentation <https://docs.sigstore.dev/>`_)
+has native support for in-toto build predicates. However, it currently can only
+operate directly on the predicate but not on the enclosing attestation
+(cosign 2.2.4). By that, the predicate first needs to be extracted (provenance
+in this example)::

cat build/attestation/kas-build.provenance.json | jq '.predicate' > provenance.json

--
2.39.2

[Jan Kiszka]

Thanks, all 5 applied to next.

Jan

--
Siemens AG, Technology
Linux Expert Center