"Expected to write to file" - any possible breakwaters to use when output will not be modified?

[Ivan Morén]

Hi!

I get that the right answer to the following "problem" statement probably  is that it is in tup's nature to just continue chewing on every output, but then I thought; a lot has happened with tup in recent years. Maybe there is a solution lurking in the knowledge of you tup gurus. So here we go!

I have a C project that uses monkey patching right before linking, in order to be able to reuse the compiled code in a modular way. This poses a problem, or rather an ugly solution, when building it with tup.

The build process is basically to (1); compile the source files independently to object files, then (2); read all the object files that are to be linked and generate per-file redefines in separate files, (3); monkey patch, working pairwise on each object file and its respective file containing the redefines and finally (4) link the resulting object files to an executable.

The "problem" is in step 2 above: Since all the primary object files are inputs to the redefines-generation all the independent output files need to be rewritten - even if only some or even none of them actually changed. Most of these redefines files are just empty, and seldom change, but trying to insert a little sha1sum check or something makes tup error with 'Expected to write to file'.

for each [4 .c files] -> [4 .o files]

[4 .o files] -> [1 clump.redefines file]

[1 clump.redefines file] -> [4 .redefines files]

/* here somewhere I've tried to add in some kind of file will not change check, but tup complains when I don't write to the output... */

for each pair of [4 .o files] and [4 .redefines files] -> [4 final .o files]

[4 final .o files] -> [1 executable]

I'm using the lua version of tup 0.7.11.

Any ideas? I mean everything works out and compiles fine, it's just doing a whole lot of unnecessary monkey patching every time I change a file, and it would just be super satisfying if there was some way to add in an option to mark an output as buffered, and have tup diff that output against the previous version to deduce whether to continue to propagate the rules affected. But even manually saving and comparing sha1sums would be more than enough. 

Thank you for a wonderful build system and sorry for my perhaps not super pedagogical explanation;

Ivan

[Mike Shal]

Hi Ivan,

Have you looked into the ^o flag? Or does that not suit your needs in this case? If the .redefines files aren't changing often, that sounds like a good use case for ^o. That will cause tup to compare the new output to the old one, and skip any downstream commands if they are the same (unless some other input for the downstream commands has also changed, of course). The syntax is a little weird since the lua parser inherits most of the syntax from the Tupfile parser, so it would look something like this:

tup.rule('*.o', '^o^ [command to generate clump.redefines]', 'clump.redefines')

tup.rule('clump.redefines', '^o^ [command to generate separate redefines files]', {'redefine1', 'redefine2', ...})

In theory you could add ^o everywhere, but it does add extra work to do the comparison, so it is up to you to add it where it will have the most impact. For example, your compilation and linking commands probably would not benefit since you would expect the .o file & final executable to change most of the times you change an input .c/.h file.

Let me know if that isn't working for you.

-Mike

[Ivan Morén]

Wow, almost a year later.

It worked beautifully Mike! I got so excited back then that I just carried on with the project and forgot to reply! Thank you so much.

/Ivan