[erlang-questions] Does port driver block scheduler threads?

[Zhemzhitsky Sergey]

Hi erlangers,

 

Recently we have developed port (linked-in) driver and faced with some performance issues.

 

The port uses ERL_DRV_FLAG_USE_PORT_LOCKING locking mode, so as we understand calls to different port instances should have not blocked each other.

SMP count is greater than 1.

 

So the first question is:

Is it possible to have multiple port instances executing time-consuming operation in parallel, i.e. at the same time?

 

We expected port instances do not influence on each other. However, all the calls are blocked but the first one. So the scheduler thread seems to be blocked during the port instance invocation.

Is driver_async function required for the issues like above?

Is it possible to unblock the scheduler thread before erlang:port_call completes?

 

Best Regards,

Sergey

 

_______________________________________________________

 

The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia.

If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp

 

[Serge Aleynikov]

Driver callbacks are executed by a scheduler thread in synchronous
manner, so what you are describing is a normal behavior of a linked-in
driver. Use driver_async to off-load a long computation to another
thread from a pool different from the scheduling threads.

> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://erlang.org/mailman/listinfo/erlang-questions
>

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Zhemzhitsky Sergey]

Hi Serge,

Thanks a lot.

Whether it is correct that the same behavior is applicable to the ordinary ports?

Best Regards,
Sergey

[Zabrane Mickael]

Hi Serge,

Where one can find a simple driver_async example to learn from?

thx
Zab

Regards,
Zabrane

[Ilyushonak Barys]

Hi,

http://www.erlang.org/doc/apps/erts/driver.html#id82769
and source code from OTP erts/example/next_perm.cc

Please, note, there is no implementation of do_free method in this file:
Line 106, driver_async(port, NULL, do_perm, async_data, NULL);

Hope this helps.

To be honest it would be great to find more complex example.
It would great contribution If someone can share it.

Regards,
Boris

-----Original Message-----
From: erlang-quest...@erlang.org [mailto:erlang-quest...@erlang.org] On Behalf Of Zabrane Mickael
Sent: Tuesday, July 03, 2012 6:26 PM
To: Serge Aleynikov
Cc: erlang-q...@erlang.org
Subject: Re: [erlang-questions] Does port driver block scheduler threads?

[Zabrane Mickael]

Thanks for sharing Joseph.

Zabrane

On Jul 3, 2012, at 4:35 PM, Joseph Wayne Norton wrote:

>
> Zabrane -
>
> Here is a nif and driver implementation for interfacing with leveldb. The contrast/comparison between the two implementations might be helpful.
>
> https://github.com/norton/lets/tree/master/c_src
>
> and a pointer to one of the commands with optional async support:
>
> https://github.com/norton/lets/blob/master/c_src/lets_drv.cc#L577
>
> Not sure if it is simple or not ... but it might help you with your development.
>
> thanks,

[Lukas Larsson]

If you want a much more complex example, go looking through
https://github.com/erlang/otp/blob/maint/erts/emulator/drivers/common/efile_drv.c
:D

4000 lines of asynch driver goodness :)

Lukas

[Michael Truog]

Hi Zabrane,

If you want to make it a bit easier, you can use GEPD and just set a flag (1 instead of 0) to make the function async (so the code generation uses driver_async). If you wanted to, you could look at the generated code, but it may be simpler to look at the GEPD code... it depends if the preprocessor scares you. The link is https://github.com/okeuday/generic-erlang-port--driver-

Best Regards,
Michael

[Serge Aleynikov]

The interface with ordinary ports is different. Port programs are
started as external executables and pipes are used to marshal data back
and forth between Erlang processes (serviced by the emulator's
scheduling threads) and these external programs. This involves no
blocking of scheduling threads as every communication with a port
program merely involves serializing/deserializing data written to or
read from a file descriptor in non-blocking manner.