[erlang-questions] driver_create_port overhead and driver_caller

[QDev]

I have linked in driver which deals with embedded library, can make calls some in outputv, some using driver_async, but now have to support some other long running operations requiring thread affinity and (more importantly) need to receive multiple commands/actions from Erlang client code associated with long running transaction in library API. Currently considering to spawn (or pool) some new thread for these (long running tasks) and using ErlDrvMutex and simple queue/array data structure to communicate with it from outputv, but there is also affinity between Erlang callers process id and long running task thread (since caller originates with erlang:send/2 direct to port, which is accessed via ets table), because replies are sent via driver_send_term so need to also store client's process id in ErlDrvTermData and use driver_monitor_process to ensure that if client dies, work is rolled back. IIRC driver_caller not thread safe, so need to only use this in outputv is that still
correct?

Now, just getting used to driver API changes in R16 (haven't written driver since R13B) and have seen driver_create_port. This seems to create new port running driver code, so wonder if this is better approach for these long running tasks, and what are benefits/advantage of doing this?

Thanks
QD
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[QDev]

Is there any advice from OTP team on best way to handle long running task in driver that needs to do ongoing communication with peer Erlang process? Can Erlang process calls outputv and use mutex to talk to async thread (or manual spawned thread) be best? Is locking in outpuv not risk to schedulers if outputv take too long, like with NIF calls? Is there other way - would it be better in outputv to put binary/ErlIOVec directly in driver queue with driver_enqv or driver_enq_bin but then port data lock has to be use to synchronise reads/writes to driver queue anyway. That means more contention if many concurrent threads are working no? So I was thinking better to use mutex per long running thread instead.

Also, is it considered necessary to pool worker thread manually? In previous releases, using async thread pool could have detrimental effect on I/O performance - is this still the case? Because if using async thread pool is fine now, maybe easier to do that instead of pooling threads manually, though problem of signalling is still there.

Thanks
QD

[Tim Watson]

On 14 Jun 2013, at 11:19, QDev wrote:

> Is there any advice from OTP team on best way to handle long running task in driver that needs to do ongoing communication with peer Erlang process? Can Erlang process calls outputv and use mutex to talk to async thread (or manual spawned thread) be best?

I've seen driver code that worked that way which was pretty efficient.

> Is locking in outpuv not risk to schedulers if outputv take too long, like with NIF calls?

If I understand it correctly, yes, this is a risk. Use the async thread pool or pool/spawn your own threads to compensate.

> Is there other way - would it be better in outputv to put binary/ErlIOVec directly in driver queue with driver_enqv or driver_enq_bin but then port data lock has to be use to synchronise reads/writes to driver queue anyway. That means more contention if many concurrent threads are working no? So I was thinking better to use mutex per long running thread instead.

I've not used the driver queue much in the past, so not sure about that.

> Also, is it considered necessary to pool worker thread manually? In previous releases, using async thread pool could have detrimental effect on I/O performance - is this still the case?

ISTR reading that the async threads used by drivers are now separate from those used by the I/O drv internally.

Cheers,
Tim

[Lukas Larsson]

On Tue, Jun 18, 2013 at 1:39 PM, Tim Watson <watson....@gmail.com> wrote:

On 14 Jun 2013, at 11:19, QDev wrote:

> Is there other way - would it be better in outputv to put binary/ErlIOVec directly in driver queue with driver_enqv or driver_enq_bin but then port data lock has to be use to synchronise reads/writes to driver queue anyway. That means more contention if many concurrent threads are working no? So I was thinking better to use mutex per long running thread instead.

I've not used the driver queue much in the past, so not sure about that.

 

The good thing about using the driver queue is that if there is data and a flush callback is implemented you can make sure that data is handled before closing the port. So that you don't loose things which you might need. If you do notice that you get a lot of contentions on the pdl you can of course roll your own more fine grained locking and just stick a byte of data in the driver queue in order to trigger flushes.

> Also, is it considered necessary to pool worker thread manually? In previous releases, using async thread pool could have detrimental effect on I/O performance - is this still the case?

ISTR reading that the async threads used by drivers are now separate from those used by the I/O drv internally.

File I/O uses the same async threads as drivers. In fact the I/O drivers do not use any functionality which is not available to any other driver. So if you do have long running jobs in the async pool it could interfere with the latency of file I/O.

 Lukas

[Tim Watson]

On 18 Jun 2013, at 14:35, Lukas Larsson wrote:

On Tue, Jun 18, 2013 at 1:39 PM, Tim Watson <watson....@gmail.com> wrote:

On 14 Jun 2013, at 11:19, QDev wrote:

> Is there other way - would it be better in outputv to put binary/ErlIOVec directly in driver queue with driver_enqv or driver_enq_bin but then port data lock has to be use to synchronise reads/writes to driver queue anyway. That means more contention if many concurrent threads are working no? So I was thinking better to use mutex per long running thread instead.

I've not used the driver queue much in the past, so not sure about that.

 

The good thing about using the driver queue is that if there is data and a flush callback is implemented you can make sure that data is handled before closing the port. So that you don't loose things which you might need. If you do notice that you get a lot of contentions on the pdl you can of course roll your own more fine grained locking and just stick a byte of data in the driver queue in order to trigger flushes.

That sounds like workable approach, will look into it.

> Also, is it considered necessary to pool worker thread manually? In previous releases, using async thread pool could have detrimental effect on I/O performance - is this still the case?

ISTR reading that the async threads used by drivers are now separate from those used by the I/O drv internally.

File I/O uses the same async threads as drivers. In fact the I/O drivers do not use any functionality which is not available to any other driver. So if you do have long running jobs in the async pool it could interfere with the latency of file I/O.

Long running jobs will use own thread pool then.

Thanks!

QD

[Tim Watson]

On 18 Jun 2013, at 14:35, Lukas Larsson wrote:
>

> The good thing about using the driver queue is that if there is data and a flush callback is implemented you can make sure that data is handled before closing the port. So that you don't loose things which you might need. If you do notice that you get a lot of contentions on the pdl you can of course roll your own more fine grained locking and just stick a byte of data in the driver queue in order to trigger flushes.
>

Oh, that's nifty, I like it. Is there any benefit to doing that versus creating a pipe and using driver_select to signal/check?

> File I/O uses the same async threads as drivers. In fact the I/O drivers do not use any functionality which is not available to any other driver. So if you do have long running jobs in the async pool it could interfere with the latency of file I/O.
>

Hmn, ok that's for clearing that up - it's important to be aware of the risks there.

[Lukas Larsson]

On Tue, Jun 18, 2013 at 4:02 PM, Tim Watson <watson....@gmail.com> wrote:

On 18 Jun 2013, at 14:35, Lukas Larsson wrote:
>

> The good thing about using the driver queue is that if there is data and a flush callback is implemented you can make sure that data is handled before closing the port. So that you don't loose things which you might need. If you do notice that you get a lot of contentions on the pdl you can of course roll your own more fine grained locking and just stick a byte of data in the driver queue in order to trigger flushes.
>

Oh, that's nifty, I like it. Is there any benefit to doing that versus creating a pipe and using driver_select to signal/check?

Not sure. What will happen is the port will remain in the system (but all links will be triggered and you cannot interact with it from erlang) and you can continue doing ready_output/ready_input until you clear the driver queue. When the queue is cleared the normal stop behaviour happens.

IMO it sometimes makes for nicer code as you don't have to worry about the port being tugged away from under your feet when you have data in the queue which has to be delivered.

 

Lukas