Build correctness

[reckbo]

Hi,

I'm impressed with the elegance of redo, it's helped me avoid a thorny Makefile in a particular case.

But I'm also impressed with the correctness of tup, and in my dreams I envision a world of simple and
flexible redo in conjunction with correct builds. 

Tracking file reads/writes I think is beyond redo, but has anyone given thought to the deletion of outdated
intermediate files produced by an old DAG?  i.e. how might the following be adapted for redo:

pre_update_file(newDAG, oldDAG)
    foreach non-leaf target in oldDAG that is not in newDAG {
    remove target from filesystem
    }

post_update_file(newDAG)
    save newDAG as oldDAG

(taken from gittup.org/tup/build_system_rules_and_algorithms.pdf)

Thanks for all the hard work,

Ryan

[Jason Catena]

Speaking from the perspective of my credo variation, rather than strictly redo.

reckbo wrote:

Tracking file reads/writes I think is beyond redo,

It'd be useful if the /proc filesystem had, for each process, "reads" and "writes" files, which lists which files a process opens.  The build tool, after it runs a command process, would update the command's target's persistent (in credo) DAG: adding read files to the target's dependency list; the target to dependency lists of written files; and each written file to the target's *.redep file (which lists other targets which depend the target).  The tool crelay (which traverses *.redep files from a given file to its leaf targets) could determine a minimal set of leaf targets which include all written files, and possibly execute those targets to bring the system up to date.

 

but has anyone given thought to the deletion of outdated intermediate files produced by an old DAG?

In essence, this is a reference-counting, unreachability, garbage-collection problem.  One approach would be to maintain reference counts.  Another would be to write a tool to compare what looks like targets, against what is currently discoverable in the DAG.  Credo maintains files in the filesystem (such as *.do) which identify files as a credo target.  We could delete each such target for which its *.redep file (which lists which files built it) either does not exist after a complete clean build, contains no targets, or refers only to targets which do not (or no longer) depend back on the checked target.  I wouldn't clean such files automatically, in case someone wants to generate and deliver or compare variants.