Incomplete/inconsistent threading/blocking API?

[ngollan]

Hi,

I'm trying to make the pcap gem work on Rubinius. That gem uses the
trapping mechanism (TRAP_BEG/TRAP_END) from the 1.8.7 API, which
Rubinius does not provide; instead, there's the
rb_thread_blocking_region call from the 1.9 API. Now things get
complicated, since Rubinius announces RUBY_VERSION 1.8.7, so I cannot
use the version string for disambiguation.

For now, it seems to be possible to use the (rbx-specific) RUBY_ENGINE
constant, but that's not defined in MRI 1.8.x. What I'm doing in
extconf.rb right now is:

begin
if RUBY_VERSION =~ /^1\.9\./ or RUBY_ENGINE == "rbx"
cflags << '-DHAVE_1_9_THREAD_API'
end
rescue NameError
# ignore errors trying to accept RUBY_ENGINE
end

It would be nice if rbx provided the TRAP_* macros like MRI does, at
least as long as it announces version 1.8.

[Dirkjan Bussink]

Hello,

The best way to handle this, is to detect whether rb_thread_blocking_region is available or not and switch based on that instead of relying on version numbers. In general it's a better approach to test for the availability of certain functionality instead of hard coding dependencies on specific versions.

I don't know if it's at all possible to provide TRAP_BEG / TRAP_END, maybe someone else can weigh in on that.

--
Regards,

Dirkjan

> --
> --- !ruby/object:MailingList
> name: rubinius-dev
> view: http://groups.google.com/group/rubinius-dev?hl=en
> post: rubini...@googlegroups.com
> unsubscribe: rubinius-dev...@googlegroups.com

[Tim Felgentreff]

The availability of rb_thread_blocking_region is announced by a #define HAVE_RB_THREAD_BLOCKING_REGION. At least on JRuby, supporting the TRAP_X macros would be a lot of work an it would be nice if pcap could run there, too, sometime, without forcing ruby version 1.9

-Tim

-- Gesendet von meinem Palm Pre

---

[ngollan]

On Sep 24, 9:42 pm, "Tim Felgentreff" <timfelgentr...@gmail.com>
wrote:

> The availability of rb_thread_blocking_region is announced by a #define HAVE_RB_THREAD_BLOCKING_REGION. At least on JRuby, supporting the TRAP_X macros would be a lot of work an it would be nice if pcap could run there, too, sometime, without forcing ruby version 1.9

MRI 1.9.1 does not seem to define that flag, so that's not a one-stop
solution either. My local version of the pcap gem now works on rbx, as
well as 1.8.7 and 1.9.1 with a rather simple preprocessor trick and
some more version magic in extconf.rbto work around an issue with
rb_thread_fd_close() which does not exist in rbx, but in 1.9.x and
1.8.7. For now I'm just omitting that call where it doesn't exist, as
that's compliant with the current "/* TODO: fix me */" implementation
in MRI.

Another nicety is that pcap uses SIZEOF_VOIDP which is not provided in
rbx, but that looks like a rather icky design decision in pcap to
ensure IPv4 addresses can be encoded in VALUE.

If you are interested, you can look at the progress so far at
http://github.com/ngollan/pcap-ruby-apis

I am a total beginner at Ruby C API stuff, so I'm pretty much going
from error to error until it compiles, and then doesn't break
horribly. If anything I did is total bogus, please correct me ;-)

[Dirkjan Bussink]

Hello,

You can use regular extconf.rb detection mechanisms to check whether
rb_thread_blocking_region is available, so that's definitely the
preferred way to go. You should check how various other projects to
method detection. You can use something very simple like the
following:

have_func('rb_thread_blocking_region')

If you use this in your extconf.rb, you get a Makefile that defines
the appropriate HAVE_RB_THREAD_BLOCKING_REGION define so it will also
work with MRI. Rubinius actually defines
HAVE_RB_THREAD_BLOCKING_REGION manually because it can't be linked as
a shared library, unlike MRI. The shared library linking is used by
extconf for checking whether methods are available.

If you find other methods to be missing in Rubinius, please report an
issue, we want the C-api to be as complete as possible. If you are
able to add specs and the missing method, that would of course be even
more awesome :).

--
Regards,

Dirkjan Bussink

[ngollan]

Thanks for the help.

Right now, I'm starting to doubt that the pcap gem is done properly
(and seeing how it was apparently written for Ruby 1.4, there's likely
some pretty antique proper stuff in there as well). The sections that
were going outside the GIL are yielding values back to the VM, so that
results in a failure as most of the internal API is no-go inside a
blocking region, and effectively the callback handler would need to re-
acquire the lock for the yield call.

I'll see if I can find the time to play with that more, but right now
I'm running into quite massive performance issues with scripting
languages, as I need to do a non-trivial amount of number-crunching,
so I guess I'll have to go back to C++ for the time being. (That's
"<1min vs. hours to days" performance issues, basically I'm beating
the integer pipeline silly which is still the area of optimising
compilers.)

On Sep 27, 3:09 pm, Dirkjan Bussink <d.buss...@gmail.com> wrote:
> You can use regular extconf.rb detection mechanisms to check whether
> rb_thread_blocking_region is available, so that's definitely the
> preferred way to go. You should check how various other projects to
> method detection. You can use something very simple like the
> following:
>
> have_func('rb_thread_blocking_region')

On the upside, things seem to Just Work with have_func(), which is a
nice improvement over my hack.