How to turn an array of IO descriptors into an fd_set pointer

[Tiago Cardoso]

I'm having a hard time figuring out this one. I'm porting net-snmp into a ruby API while also exposing an API which could potentially be embedded into any event loop (eventmachine, celluloid-io...). For that, I have to circumvent the netsnmp lib way of doing things:

so, the libsnmp way:

# after establishing an snmp "session"

snmp_sess_async_read(&session, &pdu, callback, null)

# next call will write the ready-to-read descriptors into this fdset

snmp_sess_select_info(&session, &numfds, &fdset, &timeval,  &block)

# fdset now contains a pointer to the fdset of selectable readers

select(numfds, &fdset,  null null, tout)

# now fdsets are all readable, and we can read snmp

snmp_sess_read(&session, &fdset)

# this call will read from all the readable fdset. the callback will therefore be activated

So, I found a way to get the file descriptor from the session and exposing it to ruby, which means, I can directly wait on it to be readable using ruby API, my problem being, I still need an fdset pointer to pass to the snmp_sess_read method. So my question would be, how can I do this? I'm assuming I'll have to do some FFI magic, as ruby itself doesn't itself expose an IO collection as an fdset (I've looked into the source code, that's something that happens at the VM level):

# after establishing an snmp "session"

RubyLib.snmp_sess_async_read(session, pdu, callback, nil)

# I find the file descriptors and expose them as io objects in ruby land, and select on them

readers = IO.select([snmp_descriptors])

# now readers are all readable, but how to create the fdset to pass to snmp_sess_read????

RubyLib.snmp_sess_read(session, readers_to_fdset)

Hope someone can help me on this. 

[Matijs van Zuijlen]

Looking at http://linux.die.net/include/sys/select.h it seems fd_set is
manipulated using macro's. This makes it hard to access from FFI.

> RubyLib.snmp_sess_read(session, readers_to_fdset)
>
> Hope someone can help me on this.

Perhaps you can explain a bit more why you cannot use `snmp_sess_select_info` to
help create a set of file descriptors? Perhaps you'll need to hook into
celluloid etc. some other way?

Regards,

--
Matijs

[Tiago Cardoso]

I can use the function (in fact, my first working solution is using it), but the macro problem still stands. And since I wait for IO in ruby land (i still have to instantiate the IO for that), it just seems a bit too FFI-overhead to create all the structures necessary to call the other function. 

The IO object (currently implemented with IO.select) will be the object I'll be passing to celluloid to wait on. I'll try to do something like this:

https://gist.github.com/tpitale/3915671

plus the fdset overhead.

[Matijs van Zuijlen]

On 21/10/15 18:02, Tiago Cardoso wrote:
> I can use the function (in fact, my first working solution is using it), but the
> macro problem still stands. And since I wait for IO in ruby land (i still have
> to instantiate the IO for that), it just seems a bit too FFI-overhead to create
> all the structures necessary to call the other function.

So you cannot pass an uninitialized fd_set pointer to snmp_sess_select_info to
offload setting it up to that function?

Perhaps you can post that first working solution somewhere?

I don't think you need to worry about FFI overhead, especially not at first. I
find it is pretty snappy.

> The IO object (currently implemented with IO.select) will be the object I'll be
> passing to celluloid to wait on. I'll try to do something like this:
>
> https://gist.github.com/tpitale/3915671
>
> plus the fdset overhead.

Yes, that will be the tricky part if libsnmp does its own waiting.

> --
> You received this message because you are subscribed to the Google Groups
> "ruby-ffi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to ruby-ffi+u...@googlegroups.com
> <mailto:ruby-ffi+u...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/ruby-ffi.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ruby-ffi/b142d821-7646-47b8-a3d3-f3b3194ef99d%40googlegroups.com
> <https://groups.google.com/d/msgid/ruby-ffi/b142d821-7646-47b8-a3d3-f3b3194ef99d%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.


--
Matijs

[Tiago Cardoso]

I finally have some functional (partially) ongoing at the moment. Situation is as follows:

After sending the pdu, I use the weird netsnmp api to fetch the fdset. (Situation described above). I previously get the fd from the session structure ( I read the code, and I found out that if I cast it to the session list, I get access to the transport struct, which has the int fd:

list = LibSNMP::Structures::SessionList.new signature

return if not list or list.pointer.null?

transport = LibSNMP::Structures::Transport.new list[:transport]

IO.new(transport[:sock])

With this IO instance I can finally use the ruby IO.select method after I get the fdset. This fdset is therefore used after to call snmp_sess_read. And it works... most of the time.

The main issues I have are with Errno::EBADF, which apparently can happen when I select on that socket instance. I don't know how to handle these kind of errors (do they happen often? can I ignore them?). Sometimes, after I wait for the socket and the use snmp_sess_read, I don't get the callback to be called, which might mean that the async pdu send might have been ignored(???). Among other stuff. 

My assumption is that the fdset might have other descriptors there. I would preferably get the descriptors from there. the dset is a structure which (in linux, at least) has access to a fds_bits structure, which is in FFI an InlineArray. What I'm investigating is whether one can "read" the fds from the fdset using some FFI API, but haven't been successful until now.