config.h in redo

[Страхиња Радић]

Hi, I am trying to add support for config.def.h and config.h files to my
project, which uses apenwarr/redo. The functionality needed is:

- By default, only config.def.h is included in the project's repository.

- config.h is initially generated from config.def.h, for example:

#------------config.h.do------------
cat config.def.h >$3
#------------config.h.do------------

- User should be able to modify config.h.

The issue is that once the user modifies config.h, its dependants are always
regenerated, and redo issues the message (for example, if util.c depends on
config.h by including it - see default.o.do):

redo (util.o)
redo: config.h - you modified it; skipping
redo util.o
redo: config.h - you modified it; skipping

Is there a way to record the state of the user-modified config.h file, so that
the next time redo is called (without any further source code modifications),
no .o files are regenerated? I have:

#------------default.o.do------------
sed -n '/^#include "/s/^#include "\([^"]*\)"/\1/p' ${2%.o}.c | \
xargs redo-ifchange
redo-ifchange $2.c
gcc -g -Wall -std=c99 -c $2.c -o $3
#------------default.o.do------------

#------------default.do------------
if [ -r $1.in ]; then
redo-ifchange $1.in version date
read VERSION <version
read DATE <date
sed -e "s/%VERSION%/$VERSION/g" \
-e "s/%DATE%/$DATE/g" <$1.in >$3
else
printf "%s: don't know how to build '%s'\n" "$0" "$1" >&2
exit 99
fi
#------------default.do------------

#------------date.do------------
FALLBACKDATE=${FALLBACKDATE:-unknown}
user=$(stat -c %U .)
e_user=$(id -un)
if command -v git >/dev/null 2>&1; then
if [ "$e_user" = "$user" ]; then
env LC_ALL=C git log --format=format:%cd \
--date=format:"%d %b %Y" -1 @ >$3
else
su "${user}" -c 'env LC_ALL=C git log --format=format:%cd '\
'--date=format:"%d %b %Y" -1 @' >$3
fi
echo >>$3
else
echo $FALLBACKDATE >$3
fi
redo-always
redo-stamp <$3
#------------date.do------------

#------------version.do------------
FALLBACKVER=${FALLBACKVER:-unknown}
user=$(stat -c %U .)
e_user=$(id -un)
if command -v git >/dev/null 2>&1; then
if [ "$e_user" = "$user" ]; then
git describe 2>/dev/null | sed 's/^v//' >$3
else
su "${user}" -c 'git describe' 2>/dev/null | sed 's/^v//' >$3
fi
else
echo $FALLBACKVER >$3
fi
redo-always
redo-stamp <$3
#------------all.do------------

#------------all.do------------
redo-ifchange my_prog
#------------all.do------------

#------------my_prog.do------------
redo-ifchange my_prog.o util.o
gcc -g -Wall -std=c99 -o $3 my_prog.o util.o >/dev/null
#------------my_prog.do------------

Thanks in advance,
Strahinya

[spacefro...@meterriblecrew.net]

The solution is to let the user use a config.usr.h and rewrite config.h.do to use config.def.h as long as config.usr.h does not exist and config.usr.h otherwise. Remember to use redo-ifcreate config.usr.h.

Kind regards,
–Michael

[Страхиња Радић]

On 23/03/13 01:46PM, spacefro...@meterriblecrew.net wrote:
> The solution is to let the user use a config.usr.h and rewrite config.h.do to
> use config.def.h as long as config.usr.h does not exist and config.usr.h
> otherwise. Remember to use redo-ifcreate config.usr.h.

Since I posted the question, I figured out[1] how to get around the issue
myself. My solution is similar, but it doesn't use redo-ifcreate. Instead, I
modified the *.c files to #include _config.h instead of config.h, and created
_config.h.do:

#------------_config.h.do------------
hfile=${1#_}
if [ ! -r $hfile ]; then
cat config.def.h >$hfile
fi
cat $hfile >$3
redo-always
redo-stamp <$3
#------------_config.h.do------------

Now, config.h is generated from config.def.h, the user can still modify
config.h, but config.h itself is just the basis to fill the generated file
_config.h, which is actually #include-d in the source.

[1]: https://git.sr.ht/~strahinja/sled/commit/1cf23f4eb4165b9d3a1d029e378331bcbe4358df

[spacefro...@meterriblecrew.net]

This solution has the drawback that it does not track changes to the user's config.h and config.def.h. The correctness of your approach hinges on the usage of redo-always, which might not be obvious to the readers of the .do file.

–Michael

[Prakhar Goel]

You might want to use redo-ifcreate and redo-ifchange instead of redo-always as spacefrogg mentioned.

-- PG

[Страхиња Радић]

On 23/03/13 12:18PM, Prakhar Goel wrote:
> You might want to use redo-ifcreate and redo-ifchange instead of
> redo-always as spacefrogg mentioned.

From what I understood about Michael's solution (which isn't fully fleshed out
in the form of .do snippets), it has the following pros and cons:

Pros:
- Doesn't require replacing config.h -> _config.h in *.c files

Cons:
- Relies on user manually copying config.def.h to config.usr.h
initially (if I understood correctly?)
- Uses nonstandard (from suckless' "user's" point of view) filename
config.usr.h

My solution:

Pros:
- Uses standard names config.def.h and user-editable config.h
- Automatically generates config.h from config.def.h on first run

Cons:
- Requires replacing config.h -> _config.h in *.c files

I haven't yet used redo-ifcreate, so I don't know if it would have any
advantages over the redo-always + redo-stamp idiom for the issue. In any case,
the file config.h needs to be automatically generated from config.def.h
initially. IMHO, my current solution works sufficiently well enough for now.

[spacefro...@meterriblecrew.net]

What I meant was this setup:

config.h.do:

if [ -f config.usr.h ]; do
  redo-ifchange config.usr.h
  cp config.usr.h "$3"
else
  redo-ifcreate config.usr.h
  redo-ifchange config.def.h
  cp config.def.h "$3"
fi

The file names do not matter, just their dependency relation does. If file names are important to you, you can rename config.h.do to _config.h.do an include _config.h in your C files and rename config.usr.h to config.h.

Another common name would be config.h.in for the file under user control, ie. your config.h.

–Michael

[spacefro...@meterriblecrew.net]

Regarding your thoughts on using redo-always + redo-stamp:

This pattern only yields correct results for input files that are themselves not created by redo. By not using redo-ifchange, you broke the dependency chain. Example:
- config.def.h is now generated by config.def.h.do
- now, while config.h.do still always runs, this won't trigger rebuilding config.def.h, because you didn't tell redo about this dependency. So config.h might be outdated even though the target was executed.

Additionally, it is a waste of compute resources to always run this target, because if you had marked your input dependencies, redo would just have known that nothing has changed. Now it must run this target to find out, every time you build something that depends on config.h.

This is not how redo is supposed to be used. redo-always + redo-stamp is a blunt instrument. Goredo even considers redo-always to be dysfunctional.

–Michael

[Страхиња Радић]

On 23/03/13 09:13PM, spacefro...@meterriblecrew.net wrote:
> This pattern only yields correct results for input files that are themselves
> not created by redo.

Which is what is needed in this case.

> - config.def.h is now generated by config.def.h.do

config.def.h in suckless software should not be generated. It is part
of the upstream repository (unlike config.h, which is supposed to be kept in
the user's local branch or a fork), a sort of a "factory default". For example,
dwm's Makefile has:

config.h:
cp config.def.h $@

> Additionally, it is a waste of compute resources to always run this target,

I agree, this is a downside. However, from the code:

> if [ -f config.usr.h ]; do
>   redo-ifchange config.usr.h
>   cp config.usr.h "$3"
> else
>   redo-ifcreate config.usr.h
>   redo-ifchange config.def.h
>   cp config.def.h "$3"
> fi

with the proposed name substitutions:

> #---------------_config.h.do---------------
> if [ -f config.h ]; do
>   redo-ifchange config.h
>   cp config.h "$3"
> else
>   redo-ifcreate config.h

>   redo-ifchange config.def.h
>   cp config.def.h "$3"
> fi

config.h would still not be generated, it seems here it is supposed to be
manually copied by the user, so that the else-branch with the redo-ifcreate
comes into play. Both in the original Makefiles and my proposal config.h is
autogenerated from config.def.h on build.

Names are important because of the established convention. Users are used to
configure suckless programs by editing config.h specifically.

[spacefro...@meterriblecrew.net]

>     Which is what is needed in this case.

Yes, in this case maybe. The argument was supposed to be: It is better when the correctness of the build process is independent of such assumptions. The correctness is no longer a local property of _config.h.do. In your solution, I have to know the content of _config.h.do when I want to modify the build process of config.def.h. Even if in this case, it may never happen, it is easy to see why this approach is not suitable for generalisation. You do not allow your build system to easily grow in complexity by adding these intricate and undocumented dependencies relations. Again, I fully grasp that this does not apply to this particular case. See it as general advice.

> config.h would still not be generated, it seems here it is supposed to be
> manually copied by the user, so that the else-branch with the redo-ifcreate
> comes into play. Both in the original Makefiles and my proposal config.h is
> autogenerated from config.def.h on build.

You can do this:

if [ -f config.h ]; do
  redo-ifchange config.h
  cp config.h "$3"
else
  redo-ifcreate config.h
  redo-ifchange config.def.h
  cp config.def.h "$3"

  cp config.def.h config.h
fi

On the first run it uses config.def.h and immediately invalidates itself by creating config.h on the side. This is does not break or violate the dependency chain.
Then, on the second run, it will build _config.h again from config.h, because it is new to redo. On subsequent runs, it will only trigger if config.h changed.

There you have your names and your full automation and a working dependency chain that makes no assumptions about how config.def.h is maintained.

[Страхиња Радић]

On 23/03/14 09:15AM, spacefro...@meterriblecrew.net wrote:
> if [ -f config.h ]; do
>   redo-ifchange config.h
>   cp config.h "$3"
> else
>   redo-ifcreate config.h
>   redo-ifchange config.def.h
>   cp config.def.h "$3"
>   cp config.def.h config.h
> fi

This iteration seems to work as intended and avoids generating _config.h when
that is not necessary, thank you!

It unfortunately shares with my solution the reliance on modifying *.c files
to include the generated _config.h instead of config.h. I know the alternative
would be to change the naming of those files, so the users customize the
program by editing _config.h instead of config.h, but that would break away
with the established practice and create confusion for the user. For now, the
first option, of modifying #include statements, seems better.

[spacefro...@meterriblecrew.net]

One addendum,

To make the solution completely match your initial intentions, you would add:

redo-stamp <$3

as the last line in the _config.do file.

Your observation regarding the naming conflict in the suckless naming convention goes beyond redo. Redo's cleaner dependency tracking just makes the problem obvious. Make, at this point, blurres the problem and lets you pass with your half-built half-changed config.h file. Strange things can happen to your config.h if for some reason its rule is triggered. This may lead to overwriting the customised file with the default content.