Waiting (barrier) for SingleAsyncWork

[Valentin Churavy]

I was wondering if someone with a bit more detailed knowledge of libuv could help me out.

In https://github.com/dmlc/MXNet.jl/pull/16 I have a function that is called from a c/c++ library like this

function _wrapper(data :: Ptr{Float64}, jf :: Ptr{Void})
  julia_function = unsafe_pointer_to_objref(jf) :: Function
  julia_function(data)
  return nothing
end

Now I am trying to make this call thread-safe (http://docs.julialang.org/en/latest/manual/calling-c-and-fortran-code/#thread-safety) by rewriting this function as

function _wrapper(data :: Ptr{Float64}, jf :: Ptr{Void})
  julia_function = unsafe_pointer_to_objref(jf) :: Function
  cb_packaged = Base.SingleAsyncWork(_ -> julia_function(data))
  ccall(:uv_async_send, Void, (Ptr{Void},), cb_packaged.handle)
  wait(cb_packaged)
  return nothing
end

The problem is that this is not a callback and the library calling the julia function expects output in data. Thus just packing the function in a SingleAsyncWork is not going to work out and I need to wait on the function to finish, before I can return to the caller. Libuv 1.X has a barrier implementation for that. Does anybody have a idea how I could do this with Julia v0.4?

Best,

Valentin

[Yichao Yu]

On Thu, Nov 5, 2015 at 12:55 AM, Valentin Churavy <v.ch...@gmail.com> wrote:
>
> I was wondering if someone with a bit more detailed knowledge of libuv could
> help me out.
>
> In https://github.com/dmlc/MXNet.jl/pull/16 I have a function that is called
> from a c/c++ library like this
>
> function _wrapper(data :: Ptr{Float64}, jf :: Ptr{Void})
> julia_function = unsafe_pointer_to_objref(jf) :: Function
> julia_function(data)
> return nothing
> end
>
> Now I am trying to make this call thread-safe
> (http://docs.julialang.org/en/latest/manual/calling-c-and-fortran-code/#thread-safety)
> by rewriting this function as
>
> function _wrapper(data :: Ptr{Float64}, jf :: Ptr{Void})
> julia_function = unsafe_pointer_to_objref(jf) :: Function
> cb_packaged = Base.SingleAsyncWork(_ -> julia_function(data))
> ccall(:uv_async_send, Void, (Ptr{Void},), cb_packaged.handle)
> wait(cb_packaged)
> return nothing
> end
>

IIUC, you want to execute `_wrapper` in a different thread. This is
not going to work. Note that in the document[1], it explicitly
mentioned,

> The callback you pass to C should only execute a ccall to :uv_async_send, passing cb.handle as the argument.

The runtime for 0.4 is not thread safe so you cannot call any runtime
in your callback function. (typeassert, allocate memory, setup GC
frame, call other non-threadsafe functions)

You can probably construct the callback as a struct including the
actual function, the barrier and the slot for the result then only do
the scheduling, waiting and returning in the callback wrapper.

[1] http://docs.julialang.org/en/latest/manual/calling-c-and-fortran-code/#thread-safety

[Valentin Churavy]

Yeah that is exactly the problem and I know that the first version is not going to work. The question is how to best do the waiting for the callback to end.

[Valentin Churavy]

So I extended the implementation of SingleAsyncWork to carry a Condition that I can wait upon, see https://github.com/dmlc/MXNet.jl/commit/00e24c6dedcf970a2420f2dcc2186031a94e3273

Yichao could you take a look and tell me if anything I am doing is either stupid or dangerous. With my limited testing it seems to work. 

[Jameson Nash]

No, that's very wrong. The example in the docs with SingleAsyncWork represents roughly the most you are allowed to do from another thread. Things that your code does that are invalid include:
° calling unsafe_pointer_to_objref, wait, and other functions. these create GC frames on the wrong thread

° calling wait. this requires a context switch which requires changing the state of the correct thread

° blocking after calling uv_async_send. this function is a signal, not a queue

° modify state (e.g. create _Async)

It's not possible to make a comprehensive list. A much shorter list is the things that code is allowed to do. The complete list is pretty much limited to:

° Simple math (scalars, not arrays)

° ccall (on isbits values, not Types)

° basic pointer operations (unsafe_load / unsafe_store) on isbits values

° it may be possible to use the new (0.5) TatasLock and Mutex objects, but you need to be careful to ensure that codegen isn't going to decide to create a GC root for them in your async function

[Valentin Churavy]

Thanks for the detailed response. I was wondering about a few things.

° modify state (e.g. create _Async)

The example in the documentation creates a Base.SingleAsyncWork and an anonymous function (closure), so what are the limitations on that?

cb = Base.SingleAsyncWork(data -> my_real_callback(args))

I can rewrite the _wrapper functions to not use unsafe_pointer_to_objref, but I was trying to avoid creating a closure.

° calling wait. this requires a context switch which requires changing the state of the correct thread
° blocking after calling uv_async_send. this function is a signal, not a queue

 

I understand that I can't use wait, but why am I not allowed to block? (I was thinking of doing a spinlock, instead of wait/notify).

Is there a way to check if I am creating a GC frame?

Best,

Valentin

[Jameson Nash]

On Fri, Nov 6, 2015 at 12:36 AM Valentin Churavy <v.ch...@gmail.com> wrote:

Thanks for the detailed response. I was wondering about a few things.

° modify state (e.g. create _Async)

The example in the documentation creates a Base.SingleAsyncWork and an anonymous function (closure), so what are the limitations on that?

cb = Base.SingleAsyncWork(data -> my_real_callback(args))

None. This happens on the Julia side, not in the async callback. The point is that the async worker side is prohibited from creating any objects of any sort or causing any other side-effects in the Julia library.
 

I can rewrite the _wrapper functions to not use unsafe_pointer_to_objref, but I was trying to avoid creating a closure.

° calling wait. this requires a context switch which requires changing the state of the correct thread
° blocking after calling uv_async_send. this function is a signal, not a queue

 

I understand that I can't use wait, but why am I not allowed to block? (I was thinking of doing a spinlock, instead of wait/notify).

I don't know how the library calls this function, but since you indicated it is multithreaded, you'll need a queuing mechanism to dispatch the results back to the correct thread. Julia doesn't really care how you do this, since it isn't Julia's thread anyhow. But uv_async_send is a notification (wake-up) signal, not a data queue -- regardless of how many times uv_async_send is called before it is serviced, it will only wake-up the julia callback once.

Is there a way to check if I am creating a GC frame?

Look at code_llvm for references to jl_pgcstack (julia-pointer-to-the-gc-shadown-stack)

[Valentin Churavy]

Thanks again, then I seriously misunderstood the documentation when I implemented Callbacks in OpenCL.jl (https://github.com/JuliaGPU/OpenCL.jl/blob/master/src/event.jl#L97-L106) and I am even more amazed that it didn't blow up in my face yet. I will go fix that code and think about an alternative to doing that.

[Valentin Churavy]

So it took a while, but in OpenCL.jl I settled for this https://github.com/JuliaGPU/OpenCL.jl/blob/88da740fa1e0894022fbaa17e35a2168119b414c/src/event.jl#L103-L141