Marking child files as built

[Dan Fithian]

Hi,

I figured someone must have figured this out, but I can't find an example in the group.

I want to notify Shake that multiple files have been built for the specific case that stack build does less work than stack build --test --no-run-tests.

The way I have been trying to do this is to say that for .build/compile, which runs stack build, and .build/compile-tests, which runs stack build --test --no-run-tests, the latter produces [".build/compile", ".build/compile-tests"]. However, even when I write both files as part of shake .build/compile-tests, shake .build/compile runs.

Is it possible to make this work as I described above?

Thanks,

Dan Fithian

[Neil Mitchell]

Hi Dan,

Generally you can't just write produces to say that a file is produced
by something - it just isn't that powerful. I will make the the docs
clearer on that, and what it can do.

It sounds like you have the situation where you have one rule that
builds A and B (let's call it pAB), and another rule that builds only
A (pA). That is quite difficult for a build system to represent, since
there's no reason for it not to try running pAB and pA in parallel,
which would go badly. It also doesn't know if you demand A to be up to
date whether to run pA or pAB.

How do you generally expect to decide between stack build and test
build --test in your example?

Thanks, Neil

[Dan Fithian]

Thanks for your reply! I came up with essentially the same conclusion. I haven't implemented it yet, but I was wondering about feedback on a possible solution: would it be possible to use the |%> or &%> operator so that if the stack build and stack build --test --no-run-tests rules were needed at the same time I could optimize for whichever one needed to be run? When do those operators receive multiple rules?

Thanks,

Dan Fithian

--

Daniel Fithian

TVision | Senior Software Engineer

[Neil Mitchell]

The |%> operator is just sugar over map (|%>) so it only gets single
things at a time.

The &%> operator always builds multiple files - so it does all of them
at once never a subset.

Perhaps you want batch?
https://hackage.haskell.org/package/shake-0.17.6/docs/Development-Shake.html#v:batch

Thanks, Neil

[Dan Fithian]

Interesting, that could work, but follow up questions: What are the conditions under which batch executes? Only when the maximum is reached?

[Neil Mitchell]

If Shake has nothing else to do, it will do a batch. That means if it
only sees one rule, and no other things are available or it has spare
capacity, it will split up a batch. The maximum is only to split
further - once it reaches the max it will execute then and there.

Thanks, Neil

On Tue, Mar 12, 2019 at 3:04 PM Dan Fithian

[Dan Fithian]

Excellent, thanks for the input! Appreciate it.