Are cached-attributes thread-safe

[Fred Teunissen]

Hi,

We are trying to improve the performance of our 'compiler'. We have a lot of rule[T]'s which are used to validate consistency and generate error-messages. These rules aren't changing the Tree but are making use of the cached-attributes.

When we run all those rules sequential (on one thread), all works as expected. When we run them in parallel (e.g. on 8 threads), we have unstable results, most of the times it works, but sometimes it fails. 

I'm suspecting that the cached-attributes aren't thread-safe. Can you confirm this? If so, can we make the cached-attributes thread-safe? 

Regards,

Fred Teunissen

[Tony Sloane]

Hi Fred,

Thanks for using Kiama. I hope it’s working well for you apart from this issue.

My intention is that Kiama cached attributes are thread safe, but I confess that we haven’t done a lot of testing of this. The memo tables for cached attributes are made using 

    new Memoiser(IdentityKeys)

(see Memoiser.scala). The actual map is made using code equivalent to

    import com.google.common.collect.MapMaker

    new MapMaker().weakKeys.concurrencyLevel(1).makeMap()

Now I’m far from an expert in this area, but my understanding of concurrencyLevel from the doc is that "A value of one permits only one thread to modify the map at a time, but since read operations can proceed concurrently, this still yields higher concurrency than full synchronization.”

So I *think* everything should be ok… 

I don’t suppose there’s any chance of getting a reproducible crash?

cheers,

Tony

--
You received this message because you are subscribed to the Google Groups "kiama" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kiama+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kiama/1e6301f4-9528-4b29-bd62-bd01c5c8167c%40googlegroups.com.

[Tony Sloane]

I should have also said that this code is relevant for the current SNAPSHOT version: 2.3.0-SNAPSHOT. 

Which version are you using? If it’s not the SNAPSHOT, can you move over to it and see what happens there. Let me know if I can help with the transition.

regards,

Tony

To view this discussion on the web visit https://groups.google.com/d/msgid/kiama/913331FF-D01F-43AD-88C8-E80B0555F8A7%40gmail.com.

[Fred Teunissen]

Hi Tony,

Kiama is working very well for us, however we would like it to be a bit faster :-), That is why we are trying to stuff in parallel and use more of the available cores.

We are using version 2.2.1 with scala 2.12.10. 

I've created a small test project (on github). Unfortunately, I don't get exactly the same result as with our big project, but I do get some errors when using multiple threads which don't appear with only 1 thread.

I've created one cached attribute which is directly used by the filterByThreshold method. This method is called in two unit tests. The unit test using only 1 thread succeeds, the unit test using 4 threads fails

I added a wrapper function for the cached attribute which synchronises on the node and then uses the cached attribute. This is used by the synchronizedFilterByThreshold method. This method is called in two unit tests (1 thread and 4 threads). Both unit test succeed.

Btw. I got the same results when I updated the test project to Kiama version 2.3.0-SNAPSHOT

Regards,

Fred

.

Tony

To unsubscribe from this group and stop receiving emails from it, send an email to ki...@googlegroups.com.

[Tony Sloane]

Hi Fred,

Thanks for the test project. I will take a proper look as soon as I can, but maybe not till next week due to other commitments.

Is it possible for you to create an issue for this on https://bitbucket.org/inkytonik/kiama so we can track things there rather than on the mailing list?

regards,

Tony

On 2 Oct 2019, at 9:19 pm, 'Fred Teunissen' via kiama <ki...@googlegroups.com> wrote:

Hi Tony,

Kiama is working very well for us, however we would like it to be a bit faster :-), That is why we are trying to stuff in parallel and use more of the available cores.

We are using version 2.2.1 with scala 2.12.10. 

I've created a small test project (on github). Unfortunately, I don't get exactly the same result as with our big project, but I do get some errors when using multiple threads which don't appear with only 1 thread.

I've created one cached attribute which is directly used by the filterByThreshold method. This method is called in two unit tests. The unit test using only 1 thread succeeds, the unit test using 4 threads fails

I added a wrapper function for the cached attribute which synchronises on the node and then uses the cached attribute. This is used by the synchronizedFilterByThresholdmethod. This method is called in two unit tests (1 thread and 4 threads). Both unit test succeed.

To unsubscribe from this group and stop receiving emails from it, send an email to kiama+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kiama/b9bcdc6c-d069-4392-b719-47b745f8af8e%40googlegroups.com.