[PATCH v3 1/1] clean: avoid creation of tmp data to recover from full-disk scenario

[Felix Moessbauer]

When running kas clean with a provided config file, kas needs to setup
the environment to process that config. This requires some temporary
space, which is not available on a full disk. By that, the clean fails
and makes it impossible to recover from that situation (at least with
kas mechanisms).

We fix this by only parsing the top-level config file to get the build
system on clean. This is done without the need for temporary storage.
On purge, we still parse the full config, as we need the repository
information. As the clean task is run beforehand (due to the
inheritance), we should have enough space to setup the tmpdirs and
perform the repo cloning.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---
This is a replacement for patch "clean: add option to recover from
disk-full situation".

Felix

kas/plugins/clean.py | 26 ++++++++++++++++++--------
1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/kas/plugins/clean.py b/kas/plugins/clean.py
index 0b4a79533..dfc7cab9f 100644
--- a/kas/plugins/clean.py
+++ b/kas/plugins/clean.py
@@ -32,6 +32,7 @@ import subprocess
from pathlib import Path
from kas.context import create_global_context, get_context
from kas.config import Config, CONFIG_YAML_FILE
+from kas.includehandler import ConfigFile
from kas.libcmds import Macro
from kas.kasusererror import KasUserError
from kas import plugins
@@ -51,6 +52,7 @@ class Clean():
helpmsg = (
'Clean build artifacts, keep sstate cache and downloads.'
)
+ config_files = None

@classmethod
def setup_parser(cls, parser):
@@ -71,14 +73,15 @@ class Clean():

def run(self, args):
ctx = create_global_context(args)
+ default_conf_file = Path(ctx.kas_work_dir) / CONFIG_YAML_FILE
build_system = None
- if args.config or (Path(ctx.kas_work_dir) / CONFIG_YAML_FILE).exists():
- ctx.config = Config(ctx, args.config)
- # to read the config, we need all repos (but no build env),
- macro = Macro()
- macro.run(ctx, skip=['repos_apply_patches', 'write_bb_config',
- 'setup_environ'])
- build_system = ctx.config.get_build_system()
+ if args.config:
+ self.config_files = args.config
+ elif default_conf_file.exists():
+ self.config_files = str(default_conf_file)
+ if self.config_files:
+ cf = ConfigFile.load(self.config_files.split(':')[0], False, False)
+ build_system = cf.config.get('build_system')
if args.isar:
build_system = 'isar'

@@ -182,10 +185,17 @@ class Purge(CleanAll):
def run(self, args):
super().run(args)
ctx = get_context()
- if not ctx.config:
+
+ if not self.config_files:
raise KasUserError('Purge requires a config file to locate '
'managed repos.')

+ ctx.config = Config(ctx, self.config_files)
+ # to read the config, we need all repos (but no build env),
+ macro = Macro()
+ macro.run(ctx, skip=['repos_apply_patches', 'write_bb_config',
+ 'setup_environ'])
+
for r in ctx.config.get_repos():
if r.operations_disabled:
logging.debug(f'Skipping {r.name} as not managed by kas')
--
2.50.1

[Jan Kiszka]

This deserves a comment as reminder:

# By definition, build_system key must be present in the first
# config file to take effect.

I'm adding that on merge.

> if args.isar:
> build_system = 'isar'
>
> @@ -182,10 +185,17 @@ class Purge(CleanAll):
> def run(self, args):
> super().run(args)
> ctx = get_context()
> - if not ctx.config:
> +
> + if not self.config_files:
> raise KasUserError('Purge requires a config file to locate '
> 'managed repos.')
>
> + ctx.config = Config(ctx, self.config_files)
> + # to read the config, we need all repos (but no build env),
> + macro = Macro()
> + macro.run(ctx, skip=['repos_apply_patches', 'write_bb_config',
> + 'setup_environ'])
> +
> for r in ctx.config.get_repos():
> if r.operations_disabled:
> logging.debug(f'Skipping {r.name} as not managed by kas')

Thanks, applies with that extra comment.

Jan

--
Siemens AG, Foundational Technologies
Linux Expert Center