How to do non-blocking Dir.glob ?

[Arthur]

Hello,

I am using eventmachine to handle multiple jobs on directories and
their files. When a job is created I have to Dir.glob through the new
directory. This directory may be on a samba share and it might be
slow. When multiple jobs now occur the reactor is blocked until all
the jobs are loaded (which means until all the directories have been
scanned).

How may I do a non-blocking Dir.glob ?

Greetings, Arthur

[Iñaki Baz Castillo]

2012/5/14 Arthur <leoc...@googlemail.com>:

Maybe this can help you:

https://github.com/martinkozak/em-files

--
Iñaki Baz Castillo
<i...@aliax.net>

[Arthur]

Thank you, but I already know this gem.

Unfortunately I don´t want to read from or write to files.
I want to do a non-blocking `Dir['./some/dir/**/*']` or similar.

For now I would go with a `find -printf ...` command. And read the
lines into a queue, which processes the files.

Isn´t there a better way?

On May 14, 2:05 pm, Iñaki Baz Castillo <i...@aliax.net> wrote:
> 2012/5/14 Arthur <leoc....@googlemail.com>:

[Bill Kelly]

Arthur wrote:
> Unfortunately I don�t want to read from or write to files.
> I want to do a non-blocking `Dir['./some/dir/**/*']` or similar.
>
> For now I would go with a `find -printf ...` command. And read the
> lines into a queue, which processes the files.
>
> Isn�t there a better way?

Perhaps a job for `defer` ?

http://eventmachine.rubyforge.org/EventMachine.html#M000486


This assumes, though, that Dir.glob doesn't itself block all
Ruby threads from running. In ruby 1.8 it might block every
ruby thread anyway. In ruby 1.9 it might run asynchronously
in its own thread (assuming Dir.glob's C implementation is
wrapped with the appropriate thread_nonblocking_region calls.)

So, assuming Dir.glob does not block all threads while it
executes, I'd expect EventMachine.defer to work.


Regards,

Bill

[Christoph Jasinski]

Hi

EM.defer do…end

exactly what I wanted to say. You can drive this through some tests, to check performance and then you know whether it blocks or not.

Cheers,
Chris

> --
> You received this message because you are subscribed to the Google Groups "EventMachine" group.
> To post to this group, send email to eventm...@googlegroups.com.
> To unsubscribe from this group, send email to eventmachine...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/eventmachine?hl=en.
>

[Iñaki Baz Castillo]

2012/5/14 Bill Kelly <bi...@cts.com>:

> This assumes, though, that Dir.glob doesn't itself block all
> Ruby threads from running.  In ruby 1.8 it might block every
> ruby thread anyway.  In ruby 1.9 it might run asynchronously
> in its own thread (assuming Dir.glob's C implementation is
> wrapped with the appropriate thread_nonblocking_region calls.)
>
> So, assuming Dir.glob does not block all threads while it
> executes, I'd expect EventMachine.defer to work.

Searching by "gvl" in dir.c (Ruby 1.9.3) I see no matches.

[James Tucker]

Use Find in EM.defer

>
> Greetings, Arthur