Kotin, mutability, reactor and thread-safety (JMM actually)

[Jacob P]

Hi,

let's consider the following piece of code in Kotlin:

    data class A(val params: Map<String, String>)

    private fun f(a: A, str: String, map: Map<String, Any>): Mono<String> {
        Mono.just("")
             .doOnNext {
                   // do some action with a, str, map. These actions
                   // are executed in other thread because of `subscribeOn`.
                   // Let's call that thread T1
              }
            .subscribeOn(Schedulers.elastic())
            .subscribe()
    }


    fun main(args: Array<String>){
       f() // called with required parameters
    }

1. Is is thread-safe? Why?
2. How can be ensured that T1 sees completely initialized objects?
3. What if one of arguments, for example map will be mutable?

[yawkat]

Hi,

With many high-level APIs there are no solid specified happens-before relations. However, there are two reasons that point towards the code being thread-safe:

- The parameters are all captured variables. This means they are final in the anonymous class: https://javap.yawk.at/#YSqg0D/procyon - this gives you the guarantees of final fields, which should solve any thread safety issues in your example. I don't believe kotlin specifies this behavior, though.

- Concurrency APIs *should* have a happens-before edge between task submission and execution, or you could easily shoot yourself in the foot. Stdlib Executor is the standard example for this: "Actions in a thread prior to submitting a Runnable object to an Executor happen-before its execution begins, perhaps in another thread." (https://docs.oracle.com/en/java/javase/13/docs/api/java.base/java/util/concurrent/Executor.html). I'm not familiar with reactor, and I doubt this behavior is specified anywhere in reactor, but it is reasonable to assume the same holds there.

The immutability of the three parameters is not relevant to these two points. It is certainly helpful because you do not need to worry about safely publishing truly immutable objects, but it's less helpful than it may seem: Map in particular is often backed by a mutable map even though it is unmodifiable API, so the rules for immutable objects do not technically apply to it.

(HashMap in particular has one final field: https://java-browser.yawk.at/java/14/java.base/java/util/HashMap.java#java.util.HashMap%23loadFactor - with the implementation of final concurrency guarantees this *should* be enough to make safe publishing of HashMap objects unnecessary on hotspot, but this is again beyond the actual JMM rules.)

Finally, if there are parallel executions of doOnNext and you are mutating one of the parameters, you may of course need synchronization then. But that is not what you asked.

- Jonas

[Jacob P]

Thanks a lot for your response. It seems that I understand everything, but some doubts appear so if you can look at my thoughts, please.

 

Finally, if there are parallel executions of doOnNext and you are mutating one of the parameters, you may of course need synchronization then. But that is not what you asked.

This is why I highlight the fact that params are immutable - we don't need to consider wheter doOnNext mutatates parameters or not. Otherwise, we need to take care of synchronization. But, it is not relevant to safe publication, isn't it?

- Concurrency APIs *should* have a happens-before edge between task submission and execution, or you could easily shoot yourself in the foot. Stdlib Executor is the standard example for this: "Actions in a thread prior to submitting a Runnable object to an Executor happen-before its execution begins, perhaps in another thread." (https://docs.oracle.com/en/java/javase/13/docs/api/java.base/java/util/concurrent/Executor.html). I'm not familiar with reactor, and I doubt this behavior is specified anywhere in reactor, but it is reasonable to assume the same holds there.

You right and I assume that there is a happens-before edge here. Without that assumption it is very hard (if possible) to get threadsafe solution generally.

- The parameters are all captured variables. This means they are final in the anonymous class: https://javap.yawk.at/#YSqg0D/procyon - this gives you the guarantees of final fields, which should solve any thread safety issues in your example. I don't believe kotlin specifies this behavior, though.

So, the lambda (object) is safely published becasue of final field and **then** is passed to the thread to be exectued (but is completely intialized because of final guarantee).  Do I understand correctly?

The immutability of the three parameters is not relevant to these two points. It is certainly helpful because you do not need to worry about safely publishing truly immutable objects, but it's less helpful than it may seem: Map in particular is often backed by a mutable map even though it is unmodifiable API, so the rules for immutable objects do not technically apply to it.

Yes, to these two points (safe publication / full initialization) it is not relevant, but - however- it is essential for `doOnNext` - params are readonly so there is no need for synchronization, yes? However we need to take care about safe publication.

It is certainly helpful because you do not need to worry about safely publishing truly immutable objects

What  do you mean by truly immutable objects? If you mean final guarantees - ok I get it. Then we actually don't need to take care about safe publication, yes? But here we need to and this is why I write that post.

[yawkat]

Hey,

Am Sonntag, 19. April 2020 14:00:06 UTC+2 schrieb Jacob P:

Thanks a lot for your response. It seems that I understand everything, but some doubts appear so if you can look at my thoughts, please.

 

Finally, if there are parallel executions of doOnNext and you are mutating one of the parameters, you may of course need synchronization then. But that is not what you asked.

 

This is why I highlight the fact that params are immutable - we don't need to consider wheter doOnNext mutatates parameters or not. Otherwise, we need to take care of synchronization. But, it is not relevant to safe publication, isn't it?

Immutability certainly is relevant to safe publication. But it's a bit difficult, more on this below.

 

- Concurrency APIs *should* have a happens-before edge between task submission and execution, or you could easily shoot yourself in the foot. Stdlib Executor is the standard example for this: "Actions in a thread prior to submitting a Runnable object to an Executor happen-before its execution begins, perhaps in another thread." (https://docs.oracle.com/en/java/javase/13/docs/api/java.base/java/util/concurrent/Executor.html). I'm not familiar with reactor, and I doubt this behavior is specified anywhere in reactor, but it is reasonable to assume the same holds there.

You right and I assume that there is a happens-before edge here. Without that assumption it is very hard (if possible) to get threadsafe solution generally.

- The parameters are all captured variables. This means they are final in the anonymous class: https://javap.yawk.at/#YSqg0D/procyon - this gives you the guarantees of final fields, which should solve any thread safety issues in your example. I don't believe kotlin specifies this behavior, though.

So, the lambda (object) is safely published becasue of final field and **then** is passed to the thread to be exectued (but is completely intialized because of final guarantee).  Do I understand correctly?

In the JMM, there is a clause that if you can see an object, you can also see all its final fields and potential objects that you access through those final fields. Because the captured variables are final in the lambda object, they are safe to access. You understand correctly :)

 

The immutability of the three parameters is not relevant to these two points. It is certainly helpful because you do not need to worry about safely publishing truly immutable objects, but it's less helpful than it may seem: Map in particular is often backed by a mutable map even though it is unmodifiable API, so the rules for immutable objects do not technically apply to it.

Yes, to these two points (safe publication / full initialization) it is not relevant, but - however- it is essential for `doOnNext` - params are readonly so there is no need for synchronization, yes? However we need to take care about safe publication.

Yes, we need no synchronization, only safe publication if the parameters are never modified. This works for both truly immutable objects and "unmodifiable" objects like Map.

 

It is certainly helpful because you do not need to worry about safely publishing truly immutable objects

What  do you mean by truly immutable objects? If you mean final guarantees - ok I get it. Then we actually don't need to take care about safe publication, yes? But here we need to and this is why I write that post.

Yes, I mean objects with only final fields (I wrote on this definition here: https://javachannel.org/posts/immutability-in-java/ ). For those objects you don't need safe publication. In your example, we have safe publication so it doesn't matter, but even if we didn't have safe publication, both A and String are truly immutable and do not need safe publishing. The Map *might* need safe publishing, depending on implementation.

- Jonas

[Jacob P]

Everything is clear, thanks! :)

On Saturday, April 18, 2020 at 1:16:08 PM UTC+2, Jacob P wrote: