shake lock limits shake's usefulness as a make replacement ?

[Simon Michael]

Hi Neil, all,

in the hledger project I have a Makefile which I use as front end for all scripts and common tasks. For more portability and programmability, I added a Shake.hs, which currently handles the more complex tasks (eg generating docs).

Every so often I think about going all in on shake, using it as a general scripting tool as I do with make. It would be nice to be able to focus on one portable and powerful system, with (compared to make, bash, and ordinary haskell scripts) the combination of portability, haskell power, nice Command utilities, and a powerful build system available for tasks which need that.

Then I remember one big limitation: I can't run two shake tasks in a project at the same time, because of "failed to acquire a file lock on .shake/.shake.lock". Some situations where this has come up:

- when I think about automating a complex release process, with many steps, some of them not yet automated, requiring human review etc. It seems desirable to be able to add a new script to my shake file that, at least at first, just runs the same commands that I would, some which involve running existing shake scripts.

- in a different project: a long-running script that shows an updating time dashboard. I realised this couldn't be a shake script, or it would prevent using shake for anything else in that project.

Any thoughts on this ? I wondered if there'd be a flag to use a different lockfile/state directory, at least, but I think not (--share doesn't seem to do it).

Thanks a lot for shake. Best,
-Simon

[Adam Sandberg Eriksson]

I think you might want to looka at ShakeOptions.shakeFiles: https://hackage.haskell.org/package/shake-0.18.5/docs/Development-Shake.html#v:shakeFiles

It looks like you can set it with --metadata / -m from the command line too.

Cheers,
Adam

> --
> You received this message because you are subscribed to the Google
> Groups "Shake build system" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to shake-build-sys...@googlegroups.com.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/shake-build-system/8D62B096-89FF-4712-B40C-E73E977580EE%40joyful.com.
>

[Neil Mitchell]

Hi Simon,

--share doesn't do anything like you might hope, and does nothing interesting for you. Unfortunately the Shake lock is pretty important, as there is a single Shake database, and only one thing should be mutating it at once. It would be possible to have a memory-lock and then marshal different users from different processes into one execution of the database, and that's been started with the Database module in the most recent Shake, but it's probably more complex than you want.

It sounds like you want to have some things that do builds, and some that are just wrappers around a few scripts. I suggest having a Main.hs with a dispatch on command line argument, and then only on the "make-like" tasks do you go into Shake. Hopefully that means that shared lock is a lot less of a problem. Now you are using Haskell for a general purpose scripting language and command spawning thing, and using Shake for those bits that use build system pieces.

Thanks, Neil