Ruby Threads 101
Ben
learn Ruby. We are reading chapters 11-13 for our next session, and
those chapters include Ruby Threads.
I have promised to find some exercise that uses multiple threads. The
group is very experienced with multi-threaded programming (we're
mostly embedded C++ developers) so the thread coordination part
doesn't need to be overly simple. The Rubyish-ness shouldn't be to
complicated - if that makes any sense.
I looked through rubyquiz.com (where I've been pulling a few of our
exercises), but nothing jumped out at me. Did I miss something?
Any suggestions?
Thanks
-Ben
James Edward Gray II
> I looked through rubyquiz.com (where I've been pulling a few of our
> exercises), but nothing jumped out at me. Did I miss something?
Glad to hear it's proving useful.
We haven't had a good threading problem yet, but I'm very open to the
idea if someone can think one up...
James Edward Gray II
Joe Van Dyk
I have one about a bunch of hungry commie philosophers...
Robert Klemme
LOL
Ben, you could start with a simple problem (like farmer worker e.g. a small
app that crawls web pages in parallel with controllable degree of
parallelism) to get the basic feeling and then work your way up to more
complicated things.
I once cooked something up on the Wiki (including a superfluous queue
implementation - there is one that comes with Ruby) although I guess it
won't be much use for you as you seem to be rather senior when it comes to
threading.
http://www.rubygarden.org/ruby?MultiThreading
One thing to keep in mind is that Ruby threads are not (yet) native threads.
Despite of that they work quite nicely IMHO.
Kind regards
robert
Ben
> I once cooked something up on the Wiki (including a superfluous queue
> implementation - there is one that comes with Ruby) although I guess it
> won't be much use for you as you seem to be rather senior when it comes to
> threading.
> http://www.rubygarden.org/ruby?MultiThreading
>
> One thing to keep in mind is that Ruby threads are not (yet) native threads.
> Despite of that they work quite nicely IMHO.
>
Thanks for the wiki pointer. It's always nice to see how something
familiar is done in another language. Gives me a feeling for what the
differences are.
Why is it that Ruby threads aren't native? Lack of time to make
them so, or is it the long term plan to never use native threads?
Also, the book was a little vague about when all threads block
waiting for one thread. It said a "call to the operating system"
would cause all threads to block. What did that mean exactly? Shell
commands? Opening files? Sockets? A lot of common actions results
in system calls.
As for my project, I am considering having the group write an app
that downloads fantasy football stats for several different positions
from a web page. (Each position would be grabbed and parsed by a
different thread). Those stats would be scored and each positions
players would be sorted by score. A shared table sorted by overall
score would be updated on a player by player basis.
DId that make sense? Thoughts?
-Ben
Zed A. Shaw
I had a lot of fun implementing the following patterns as a way of
avoiding almost all Thread primitives in a distributed video
processing system and boosting performance quite a bit:
http://alumnus.caltech.edu/~croft/research/agent/tuplespaces/
In fact, Ruby comes with a nice tuple space implementation, so you
could take the above patterns, implement single instance versions, and
then try to distribute them.
Those patterns are also good examples of why the logical proof of
single machine thread primitives does not instantly extend to
distributed processing. They are much simpler to use and do
exactly the same thing as distributed locking with nothing more than
simple tuple-space semantics rather than the complicated mutex and
semaphore semantics.
I would also take a look at the work done on lock-free data structures
and algorithms. Lots of really great stuff that basically shows you
don't need locking on many data structures. Even more proof
that 1970's locking technology needs to be rethunk.
My favorite fun project is doing a Replicated-Worker pattern with just
message queues. Sure the Queue needs locking, but why do you need to
know that?
Enjoy!
Zed A. Shaw
http://www.zedshaw.com/
Zed A. Shaw
up.
On Tue, 27 Sep 2005 03:54:37 +0900
Joe Van Dyk <joev...@gmail.com> wrote:
Lyndon Samson
thread each ) writing to a queue flushed to a db from another thread.
Logan Capaldo
Speaking of lock-free data sructures (sort of) has anyone ever
implemented Erlang or E -style promises in Ruby?
Pit Capitain
>
> Speaking of lock-free data sructures (sort of) has anyone ever
> implemented Erlang or E -style promises in Ruby?
What features are you looking for?
Regards,
Pit
Robert Klemme
> Ben,
>
> I had a lot of fun implementing the following patterns as a way of
> avoiding almost all Thread primitives in a distributed video
> processing system and boosting performance quite a bit:
>
> http://alumnus.caltech.edu/~croft/research/agent/tuplespaces/
>
> In fact, Ruby comes with a nice tuple space implementation, so you
> could take the above patterns, implement single instance versions, and
> then try to distribute them.
As far as I can see the idea is basically to have a synchronized
repository (typically a queue) where several workers can fetch their tasks
from and during working on the task there is no synchronization needed (in
Doug Lea terms "lightweight processing framework" with "thread
confinement"). Not really fancy or did I miss something?
> Those patterns are also good examples of why the logical proof of
> single machine thread primitives does not instantly extend to
> distributed processing. They are much simpler to use and do
> exactly the same thing as distributed locking with nothing more than
> simple tuple-space semantics rather than the complicated mutex and
> semaphore semantics.
Honestly, I don't think mutexes and semaphores are out of business simply
because you don't see them directly. Concurrency - however done - adds
significantly to the complexity of an application and people creating such
applications shoule be well aware of the basic problems they have to solve
to make them reliably do what they want.
> I would also take a look at the work done on lock-free data structures
> and algorithms. Lots of really great stuff that basically shows you
> don't need locking on many data structures. Even more proof
> that 1970's locking technology needs to be rethunk.
Hmm... Still you need *some* form of synchronization / locking. The
point is to make it as efficient as possible, i.e., try to reduce
synchronization points. I don't know what exactly you mean by "1970's
locking technology" but I think the basics didn't change that much from
the 70's on.
> My favorite fun project is doing a Replicated-Worker pattern with just
> message queues. Sure the Queue needs locking, but why do you need to
> know that?
In oder to understand what you are doing. People that apply this pattern
(see above) need to know why it's working IMHO.
Kind regards
robert
Robert Klemme
> On 9/26/05, Robert Klemme <bob....@gmx.net> wrote:
>> I once cooked something up on the Wiki (including a superfluous queue
>> implementation - there is one that comes with Ruby) although I guess
>> it won't be much use for you as you seem to be rather senior when it
>> comes to threading.
>> http://www.rubygarden.org/ruby?MultiThreading
>>
>> One thing to keep in mind is that Ruby threads are not (yet) native
>> threads. Despite of that they work quite nicely IMHO.
>
> Thanks for the wiki pointer. It's always nice to see how something
> familiar is done in another language. Gives me a feeling for what the
> differences are.
>
> Why is it that Ruby threads aren't native? Lack of time to make
> them so, or is it the long term plan to never use native threads?
No, AFAIK it's planned to use native threads in Ruby 2. I believe the
reason behind the current design is portability (at the time Ruby was
created). Matz?
> Also, the book was a little vague about when all threads block
> waiting for one thread. It said a "call to the operating system"
> would cause all threads to block. What did that mean exactly? Shell
> commands? Opening files? Sockets? A lot of common actions results
> in system calls.
Yes. IO is done nonblocking under the hood so you don't have to fear that
while reading from a file no other thread can be active. Other than that
since Ruby threads are completely user space the interpreter has no way to
make a context switch between system calls. But usually this is not a
problem apart from DNS lookups that sometimes seem to make scripts slow.
> As for my project, I am considering having the group write an app
> that downloads fantasy football stats for several different positions
> from a web page. (Each position would be grabbed and parsed by a
> different thread). Those stats would be scored and each positions
> players would be sorted by score. A shared table sorted by overall
> score would be updated on a player by player basis.
>
> DId that make sense? Thoughts?
Sounds ok for a first bit. You have at least one shared resource (the
table) and can extend that with different approaches to thread creation
(create a thread per position, create n threads and let them do the work
farmer-worker like etc.).
Kind regards
robert