Is v0.5 considered thread-safe?

[Tasos Laskos]

Hi,

I've encountered intermittent, but rare, segfaults with v0.3.3 (I plat
to port to v0.5) when Hydra.run is called from different threads (but
not necessarily concurrently) so I wanted to ask, is v0.5 more stable
under these conditions?

I do that because I need an RPC call to return immediately so
Hydra.run is being put inside a Thread.

Not sure if this is caused by thread-safety, or the GC, or a bug in
the old C extension and the following is probably useless but you
never know, it might be informative:
-------------------------------------
/home/zapotek/builds/arachni/gems/gems/typhoeus-0.3.3/lib/typhoeus/
multi.rb:21: [BUG] Segmentation fault
ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux]

-- Control frame information
-----------------------------------------------
c:0014 p:---- s:0045 b:0045 l:000044 d:000044 CFUNC :multi_perform
c:0013 p:0019 s:0042 b:0042 l:000041 d:000041 METHOD /home/zapotek/
builds/arachni/gems/gems/typhoeus-0.3.3/lib/typhoeus/multi.rb:21
c:0012 p:0131 s:0039 b:0039 l:000038 d:000038 METHOD /home/zapotek/
builds/arachni/gems/gems/typhoeus-0.3.3/lib/typhoeus/hydra.rb:95
c:0011 p:0061 s:0033 b:0033 l:000032 d:000032 METHOD /home/zapotek/
workspace/arachni/lib/arachni/http.rb:597
c:0010 p:0015 s:0030 b:0030 l:000b18 d:000029 BLOCK /home/zapotek/
workspace/arachni/lib/arachni/http.rb:187
c:0009 p:---- s:0028 b:0028 l:000027 d:000027 FINISH
c:0008 p:---- s:0026 b:0026 l:000025 d:000025 CFUNC :call
c:0007 p:0016 s:0023 b:0023 l:000022 d:000022 METHOD /home/zapotek/
workspace/arachni/lib/arachni/utilities.rb:320
c:0006 p:0011 s:0017 b:0017 l:000b18 d:000b18 METHOD /home/zapotek/
workspace/arachni/lib/arachni/http.rb:185
c:0005 p:0023 s:0014 b:0014 l:000013 d:000013 METHOD /home/zapotek/
workspace/arachni/lib/arachni/http.rb:588
c:0004 p:---- s:0008 b:0008 l:000007 d:000007 FINISH
c:0003 p:0015 s:0006 b:0006 l:0012d0 d:000005 BLOCK tmp/http.rb:10
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:---- s:0002 b:0002 l:000001 d:000001 TOP

-- Ruby level backtrace information
----------------------------------------
tmp/http.rb:10:in `block (2 levels) in <top (required)>'
/home/zapotek/workspace/arachni/lib/arachni/http.rb:588:in
`method_missing'
/home/zapotek/workspace/arachni/lib/arachni/http.rb:185:in `run'
/home/zapotek/workspace/arachni/lib/arachni/utilities.rb:320:in
`exception_jail'
/home/zapotek/workspace/arachni/lib/arachni/utilities.rb:320:in `call'
/home/zapotek/workspace/arachni/lib/arachni/http.rb:187:in `block in
run'
/home/zapotek/workspace/arachni/lib/arachni/http.rb:597:in `hydra_run'
/home/zapotek/builds/arachni/gems/gems/typhoeus-0.3.3/lib/typhoeus/
hydra.rb:95:in `run'
/home/zapotek/builds/arachni/gems/gems/typhoeus-0.3.3/lib/typhoeus/
multi.rb:21:in `perform'
/home/zapotek/builds/arachni/gems/gems/typhoeus-0.3.3/lib/typhoeus/
multi.rb:21:in `multi_perform'
-------------------------------------

[Hans Hasselberg]

Hey Tasos,

I'm not sure wether Typhoeus is thread-safe - I think it is, but you still must be careful (http://curl.haxx.se/libcurl/c/libcurl-tutorial.html under Multi-threading Issues):

The first basic rule is that you must never simultaneously share a libcurl handle (be it easy or multi or whatever) between multiple threads. Only use one handle in one thread at any time. You can pass the handles around among threads, but you must never use a single handle from more than one thread at any given time.

Did that help? I need to do more research in that direction…


--
Hans

> --
> You received this message because you are subscribed to the Google Groups "Typhoeus" group.
> To post to this group, send email to typh...@googlegroups.com (mailto:typh...@googlegroups.com).
> To unsubscribe from this group, send email to typhoeus+u...@googlegroups.com (mailto:typhoeus+u...@googlegroups.com).
> For more options, visit this group at http://groups.google.com/group/typhoeus?hl=en.

[Tasos Laskos]

I'm not sure what that means to be honest...

You can't share handles but you can pass them between threads but never
use the same handle from more than one thread.

However, I think that not having to worry about that in Ruby would be
very nice and is worth looking into.

Unfortunately, I won't have time to look into this myself until it comes
time to port my system to v0.5.

On 11/08/2012 05:05 PM, Hans Hasselberg wrote:
> Hey Tasos,
>
> I'm not sure wether Typhoeus is thread-safe - I think it is, but you still must be careful (http://curl.haxx.se/libcurl/c/libcurl-tutorial.html under Multi-threading Issues):
>
> The first basic rule is that you must never simultaneously share a libcurl handle (be it easy or multi or whatever) between multiple threads. Only use one handle in one thread at any time. You can pass the handles around among threads, but you must never use a single handle from more than one thread at any given time.
>

> Did that help? I need to do more research in that direction�

[Hans Hasselberg]

I'll look into it and come up with some documentation about my findings.
I created an issue as a reminder: https://github.com/typhoeus/typhoeus/issues/234.

--
Hans

On Thursday, November 8, 2012 at 4:36 PM, Tasos Laskos wrote:

> I'm not sure what that means to be honest...
>
> You can't share handles but you can pass them between threads but never
> use the same handle from more than one thread.
>
> However, I think that not having to worry about that in Ruby would be
> very nice and is worth looking into.
>
> Unfortunately, I won't have time to look into this myself until it comes
> time to port my system to v0.5.
>
> On 11/08/2012 05:05 PM, Hans Hasselberg wrote:
> > Hey Tasos,
> >
> > I'm not sure wether Typhoeus is thread-safe - I think it is, but you still must be careful (http://curl.haxx.se/libcurl/c/libcurl-tutorial.html under Multi-threading Issues):
> >
> > The first basic rule is that you must never simultaneously share a libcurl handle (be it easy or multi or whatever) between multiple threads. Only use one handle in one thread at any time. You can pass the handles around among threads, but you must never use a single handle from more than one thread at any given time.
> >

> > Did that help? I need to do more research in that direction…