Loop implementation for a daemon

[Yann Lambret]

Hi,

I'd like to build a daemon that would be able to catch netlink broadcast messages, and also to manipulate local neihgbor tables (among other things). It looks like previous versions of pyroute2 featured some kind of loop primitives, but it's not the case anymore. Are there recommended ways to build the main loop ? I intended to use asyncio but I'm not sure if it fits well with the callback registration mechanism of pyroute2. Also, is it possible to create an IPDB object in the main loop and additional IPRoute objects in the callbacks ?

Thanks for your help.

Yann

 

[Peter Saveliev]

There are three main scenarios:

1. IPRoute without caching — just a plain socket object, suitable to be used in poll/select loops. It provides also high-level netlink API, so it is possible to register the socket in the main poll() loop and then call IPRoute.get() to get the next broadcast message.

2. IPRoute with caching — can not be used with poll/select. May be used in exclusive endless loops, since IPRoute.get() is a blocking call.

3. IPDB, which has it's own main loop, but provides callback registration via IPDB.register_callback(), see also [1] and [2]. There are pre- and post-callbacks, former are blocking and run in the loop thread before own IPDB callbacks and latter are async and run in separate threads.

> Are there recommended ways to build the main loop ?

Probably, the simplest is to use IPRoute. E.g., IPDB with it's own main loop is built on top of this scheme.

> Also, is it possible to create an IPDB object in the main loop and additional IPRoute objects in the callbacks ?

Not sure, what do you mean here, but you can create up to almost thousand netlink sockets within one process; each IPDB instance usually creates/occupies two netlink sockets. Pls keep in mind that IPDB.mnl  — a monitoring netlink socket, used in IPDB — is a caching socket and can not be used in poll/select cycles.

Pls write if there will be any question.

[1] https://github.com/svinota/pyroute2/blob/master/tests/general/test_ipdb.py#L2200
[2] https://github.com/svinota/pyroute2/blob/master/tests/general/test_ipdb.py#L2220

--
You received this message because you are subscribed to the Google Groups "pyroute2-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyroute2-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Yann Lambret]

Thanks for your answer.

Actually, it looks like I get the same behavior with or without caching (blocking call). If I got it right, caching is activated by this call:

ipr.bind(async=True)

If I do the following:

ipr = IPRoute()

ipr.bind(async=True)

ipr.get()

The last call is blocking if used inside a loop, but it's also true for 'ipr.bind()' statement.

Any thoughts ?

To unsubscribe from this group and stop receiving emails from it, send an email to pyroute2-dev...@googlegroups.com.

[Peter Saveliev]

> The last call is blocking if used inside a loop

Yes, surely. It is a blocking call, it always blocks until the next message, like recv() usually does. It is only the netlink cache that may be «async» here. And that's why IPRoute sockets should be used with poll/select (remember not to set async, using IPRoute with poll/select):

import select

from pyroute2 import IPRoute

with IPRoute() as ipr:

  poll = select.poll()

  poll.register(ipr, …)

  …  # register other file descriptors to be polled

  ipr.bind()  # receive broadcasts on IPRoute

  while (some condition):

    events = poll.poll()

    for fd, flags in events:

        if fd == ipr.fileno():

          do_something_useful_with(ipr.get())
        …

Like that.

To unsubscribe from this group and stop receiving emails from it, send an email to pyroute2-dev+unsubscribe@googlegroups.com.

[Yann Lambret]

Ok, I get it. I've got an additional question regarding error handling. Does IPRoute raise regular socket exceptions (i.e. socket.error or OSError starting from Python 3) and should I manage myself the "edge cases" like select.EPOLLHUP or select.EPOLLERR ?

Something like:

try:
    with IPRoute() as ipr:
        # callback registration...

        epoll = select.epoll()
        epoll.register(ipr, select.EPOLLIN | select.EPOLLPRI)

        ipr.bind()

        while True:
            if self.shutdown.is_set():
                logger.info('stopping, cleaning resources')
                break
            else:
                events = epoll.poll(1e-3)
                for fd, flag in events:
                    if fd == ipr.fileno() and flag & select.EPOLLIN | select.EPOLLPRI:
                        ipr.get()
                    elif fd == ipr.fileno() and flag & select.EPOLLHUP:
                        # handle issue
                    elif fd == ipr.fileno() and flag & select.EPOLLERR:
                        # handle issue
except OSError as e:
    logger.critical('netlink socket error: %s', e.strerror)

Thanks again for your help.