I am new to shake and haskell.
I have a bunch of source files, around 2000 and each is used, together with some other dependencies, to produce a result each.
To process a single file, I can run one command, for example
process -O outdir a1.in
and it will produce a1.out in the directory outdir.
The command `process` is somewhat expensive, but can be batched.
By this I mean, I can run
process -O outdir <filelist>
to process several files at once and it will be considerably faster,
then starting `process` several times.
I now would like to batch all files together, which might need rebuilding, as I know that most of the time, all results will be needed in a build.
I thought about using `&?>` using a function that maps each aN.out to [ "a1.out" .. "a2000.out" ] and an action that calls need [ "a1.in" .. "a2000.in" ] but then I can only run all input files through `process` which is unnecessary long.
Am I trying something stupid?
Is there a version of `need` which returns a list of those dependencies, which had been considered "out-of-date" (rule for dependency run or source file has a different time stamp since last run), so I can implement my approach above? (Maybe using some own Ruletype, which uses something faster than a list.)
Yours sincerely,
Eric
after some additional thought, it appears to me, this is more
complicated, then I first thought.
The needHasChanged (to borrow the term from the issue) is only
one part of it. I did a prototypical implementation of it for
source files with a custom builtin rule type. But it should
be not that complicated, to add it shake for File Rules in
general. (My route of implementation would be:
1. Add a new Ruletype FileHasChangedQ -> FileHasChangedA
2. In its builtin rule, delegate most of the work to
ruleLint and ruleRun from Development.Shake.Internal.Rules.File
and analyse the result of ruleRun.
3. Construct FileHasChangedA from the resulting FileA and the
RunChanged value.
4. Provide the API-Call needHasChanged for it and export that.
If you want to consider it, I could try to implement that and send
you a pull request. But I am a haskell beginner, so you would
certainly need to go over it.
For the specific use case, of some batch process a second part is needed.
The user rule implementing the batch process needs to know not only
which dependencies are considered out of date, but also, which targets
are considered inconsistent and need to be rebuild too. In my opinion
shakes property of checking for messed with targets is a pretty nice
one and I would not like to give up on it. (And in my use case it is not
practical to check for this myself, as that would amount to rebuilding
each target and diff the results.)
I would suggest some variant of the Files Rule here, where the user
rule gets as a parameter the list of inconsistent targets too,
if that is possible. (I will try to prototype that, too, when I have
the time for it. Probably tomorrow, as it is a german holiday, or the
upcoming weekend.)
Yours sincerely,
Eric
PS.: Is the google group the right place for such a discussion or should
I open a ticket and bring the discussion to the appropriate ticket comments?
PSS.: Thanks for your answer to my stackoverflow question. (I am Krom there).