redo -j8 clean all

[jgottula]

Hi,

I'm very impressed with redo. It fixes most of the limitations I was running into with makefiles, but keeps the overall simplicity of design that make had. For a long time, I'd wanted to switch to another build system, but the alternatives seemed mostly to be makefile generators, or else complicated in a way that annoyed me. redo, on the other hand, is just right.

I have one problem with redo right now, and it might just be that I haven't adjusted fully to thinking in terms of redo's way of doing things. Although I use git for all my projects, I'm not too keen on 'git clean' because I don't feel like the contents of my .gitignore map to what I want cleaned up all of the time. (I have, for example, workspace files from my code editor that don't really belong in the repo, but I which don't want cleaned up if I can avoid it, either.) So, I have a 'clean' target defined in my single, monolithic default.do file so that I can simply run 'redo clean all' when I want to rebuild everything. It's how I did it with make, and in large part, it still does what I want with redo.

Here's the thing: I want to use -j8 so that compilation happens in parallel; I have all these CPU threads lying around, after all. I don't, of course, want parallelism between 'clean' and 'all'; clearly, these two targets need to be run in series. But I don't know how to make this happen.

I first attempted to put the '-j8' directly in the 'redo <list of all the targets>' command that gets run on 'redo all', but that didn't work. From what I can tell, the number of threads gets set on the first recursive invocation of redo (in jwack.setup) and doesn't change thereafter (correct me if that's wrong). If that's the case, I can't really complain too much; having a fixed number of worker threads seems like a prudent design decision in general. Plus, specifying parallelism within the 'all' target as I tried to do here effectively bakes it into the script and takes it out of the hands of the user, which isn't ideal for some other user who doesn't happen to want to run 8 threads.

I could invent an additional target, 'cleanall', that does 'redo clean' and then 'redo all' serially within the shell script. This seems to resolve the parallelism conflict between 'clean' and 'all' and puts control over parallelism back into the original invocation of redo, which I think is what we want. But inventing a third target just for this reason is a little kludgy. And this wouldn't fix 'redo clean <individual target>' either; I'd need yet another 'clean<...>' target just for that case.

I read through the documentation fairly thoroughly, but I haven't determined a way to make 'clean' a dependency of 'all' for parallelism purposes (so it gets run first, serially) without having 'all' always run 'clean'. The 'unset MAKEFLAGS' approach seems to affect all children, so that's out. A way to specify 'always run clean before anything else' would be a clean (pardon the pun) solution. Doing this with dependencies, though, seems most in line with the spirit of redo, but I don't know how.


Thanks for all the work,

Justin

[Aaron Denney]

What's wrong with "redo clean && redo <target>", when necessary?

--
Aaron Denney
-><-

> --
> You received this message because you are subscribed to the Google Groups
> "redo" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to redo-list+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--
Aaron Denney
-><-

[jus...@jgottula.com]

Aaaand, mysteriously, Google Groups has deleted my reply. No idea why. Here's what it said:


Nothing, really. I think I'm just spoiled by the fact that 'make clean <target>' (or the equivalent with redo) has always just happened to work as a consequence of not having more than one build thread, so it seemed like the most expedient way to do it. Thinking about it more, it doesn't really make sense in the context of multiple threads.

I guess the closest thing to the 'right' solution, then, is to explicitly chain the two commands as you suggested. Or, better yet, make a shell alias for that.


Justin