Keys become "Missing"

[ryan....@gmail.com]

I have been using custom keys for a while now but have started noticing an issue. In my Main.hs, changing what keys I want will sometimes trigger a rebuild of keys that already exist
and have not changed. Here's example output of when that happens:

% Waiting -> Ready, OracleQ (Config "ANTs-hash")
% result (OracleQ (Config "ANTs-hash")) = (OracleA (Just "f3ff16c")) (unchanged)
% Missing -> Waiting, ANTs "f3ff16c"

'ANTs ff3ff16c' is a key that happily existed through many builds until I asked for a different
set of keys to be built, but ones that relied on the exact same key.

The shake source code suggests it can't find the key in the database any longer, but why would that happen? Any idea of what I could be doing wrong?

Thanks,
Ryan

[ryan....@gmail.com]

I think I know what happened -- shakeFiles wasn't set as it was in previous builds, so shake made a fresh database in the default .shake directory, where obviously all previous keys were missing.

I'm relieved it's a silly mistake of mine that's easily fixed.

[ryan....@gmail.com]

Actually it's still happening, this is what's causing it:

Error when reading Shake database _data/.shake.database
Data.Binary.Get.runGet at position 65: Not a valid Unicode code point!
CallStack (from HasCallStack):
error, called at libraries/binary/src/Data/Binary/Get.hs:342:5 in binary-0.8.3.0:Data.Binary.Get
All files will be rebuilt

That's without changing the code.

[ryan....@gmail.com]

Found the bug -- separate keys in their own modules had the same name for their types,
so shake could not differentiate between them. Fixed it by renaming one of the types.

[Neil Mitchell]

Sorry for the delay in replying - festive season and whatnot has my
access to a computer somewhat limited.

The bug here is that Shake is relying on equality of serialised types
by comparing their Show instance, which is identical for identically
named types in different modules. The solution is to distinguish them
in some way when serialising. If there was a "showTypeRepQualified"
function that would be ideal, but it's not built in, and TypeRep
changes significantly between GHC versions. I'm left with two options:

* Add my own showTypeRepQualified based on splitTyConApp, which at
least looks plausible.

* Serialise the FingerPrint along with the Show of the TypeRep, but
FingerPrint is only available back to GHC 7.10.

I've raised https://github.com/ndmitchell/shake/issues/506 to track it.

Thanks, Neil

[Neil Mitchell]

This is now fixed in Shake HEAD by appending the hash. The next release will have the fix in it, although that might not be until mid-January.

Thanks, Neil