[swugenerator] Variable expansion in included files

[Ernestas Kulik]

While playing around with includes and variable expansion in my sw-
description files, I discovered a rather serious shortcoming of the
implementation: variable expansion (and function evaluation) is
performed before parsing with libconf, meaning that any included files
in sw-description are left as-is.

Example:

swugenerator.cfg:
variables =
{
VERSION = "1.0.0";
IMG_VERSION = "2.0.0";
};

sw-description:
software =
{
version = "@@VERSION@@";

@include "images.cfg"
}

images.cfg:
images: (
{
...
version = "@@IMG_VERSION@@";
...
}
);

In the above example, only VERSION will be expanded after running
swugenerator.

I have a fix I can use, but it includes monkey-patching libconf to call
_expand_variables() and _exec_functions() on each call to
libconf.TokenStream.from_file(). Not something I would prefer to submit
as a patch, but conceptually it is probably the best solution.

Any thoughts?

[Stefano Babic]

Hi Ernestas,

On 1/7/26 17:05, 'Ernestas Kulik' via swupdate wrote:
> While playing around with includes and variable expansion in my sw-
> description files, I discovered a rather serious shortcoming of the
> implementation: variable expansion (and function evaluation) is
> performed before parsing with libconf, meaning that any included files
> in sw-description are left as-is.
>

Understood - yes, I can confirm this behavior.

> Example:
>
> swugenerator.cfg:
> variables =
> {
> VERSION = "1.0.0";
> IMG_VERSION = "2.0.0";
> };
>
> sw-description:
> software =
> {
> version = "@@VERSION@@";
>
> @include "images.cfg"
> }
>
> images.cfg:
> images: (
> {
> ...
> version = "@@IMG_VERSION@@";
> ...
> }
> );
>
> In the above example, only VERSION will be expanded after running
> swugenerator.
>
> I have a fix I can use, but it includes monkey-patching libconf to call
> _expand_variables() and _exec_functions() on each call to
> libconf.TokenStream.from_file(). Not something I would prefer to submit
> as a patch, but conceptually it is probably the best solution.

I do not like to patch another package due to restriction in our
software. I haven't tested, but it makes sense to rearrange the order in
swugenerator anhd use libconf.dumps to get the full s-description with
include files.

So it should be quite:

1. parse all files ==> libconf.load
2. get back the full file from libconf ==> libconf.dumps
3. apply expand_variables and exec_function to the result
4. reload the configuration via libconf.load

Does it make sense for you ?

Best regards,
Stefano Babic

[Ernestas Kulik]

On Thu, 2026-01-08 at 11:31 +0100, Stefano Babic wrote:
> Hi Ernestas,

Hi, Stefano,

> I do not like to patch another package due to restriction in our
> software. I haven't tested, but it makes sense to rearrange the order
> in
> swugenerator anhd use libconf.dumps to get the full s-description
> with
> include files.
>
> So it should be quite:
>
> 1. parse all files ==> libconf.load
> 2. get back the full file from libconf ==> libconf.dumps
> 3. apply expand_variables and exec_function to the result
> 4. reload the configuration via libconf.load
>
> Does it make sense for you ?

It is something I considered as well, but it gets trickier when include
paths contain variable references, since libconf will immediately throw
an exception if a file cannot be opened. Unfortunately, I rely on such
a feature myself. :|

Hooking into file reads allows for fixing up such include statements
before libconf sees them, but we both agree that it is a terrible
solution.

I’ll try creating an issue in the libconf project to see if more
control could be given to library users.