Problem shutting down UDP broker

[Eric Pederson]

Hello, we are having issues with CAF brokers which use UDP servants, particularly during cleanup. When a broker with a UDP servant is running and the actor_system reaches its destructor, it hangs because it is trying to join the default_multiplexer, which is stuck at the WSAPoll.

It also hangs on Linux and Mac.

 

However, we have also noticed that when the broker receives a datagram during execution, it does not encounter this error. The following example illustrates.

#include <iostream>
#include <chrono>
#include <thread>
#include "caf/all.hpp"
#include "caf/io/all.hpp"
#include "caf/config.hpp"

using namespace caf;
using namespace caf::io;

namespace {
    behavior server(broker* self, uint16_t port) {
        auto epair = self->add_udp_datagram_servant(port);
        self->set_exit_handler([=](exit_msg& msg) {
            aout(self) << "Received exit" << std::endl;
            aout(self) << "Closed: " << self->close(epair.value().first) << std::endl;
            self->quit(msg.reason);
        });
        return {
            [=](const new_datagram_msg& msg) {
                aout(self) << "Received msg: " << std::string(msg.buf.data(), msg.buf.size()) << std::endl;
            }
        };
    }
}

int main() {
    actor_system_config cfg;
    cfg.load<middleman>();
    actor_system system(cfg);
    auto serverActor = system.middleman().spawn_broker(server, 9952);
    std::this_thread::sleep_for(std::chrono::seconds(10));
    anon_send_exit(serverActor, exit_reason::user_shutdown);
    return 0;
}

If the broker receives a datagram before exit is sent, then the program ends normally. However, if it does not, then it hangs in the destruction of the actor_system.   Interestingly, the self->close() returns false either way.  When the program hangs netstat still shows the UDP port active.  

$ netstat -an | grep 9952                

udp46      0      0  *.9952                 *.*

Here is a small Python script that we are using to send datagrams to the program:

import socket

ADDR = "127.0.0.1"
PORT = 9952
MESSAGE = b'TEST'

if __name__ == '__main__':
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    sock.sendto(MESSAGE, (ADDR, PORT))

Are we shutting the broker and the udp_servant properly?

Thanks,

[Joseph Noir]

Hi,

it looks like you are doing everything correctly.

As far as I can tell the datagram servant wasn't added to the broker until a new endpoint was discovered. I addressed the problem in this commit [1], currently in a separate branch.

Does that fix the bug for you?

Kind regards


[1] https://github.com/actor-framework/actor-framework/commit/0238d11bb97d542be976dfb3e3f15e2744040035

> --
> You received this message because you are subscribed to the Google Groups "actor-framework" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to actor-framewo...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/actor-framework/bc203eca-7bbf-4c4e-aecc-4377621a4cbdn%40googlegroups.com.

[Eric Pederson]

I tried it on Mac and it looks like it works.  I am going to try it on Windows and Linux.

[Eric Pederson]

Works on Windows and Linux too.   Thanks!

[Joseph Noir]

Nice, I opened a PR against the 0.17 branch. [1]


[1] https://github.com/actor-framework/actor-framework/pull/1133

> To view this discussion on the web visit https://groups.google.com/d/msgid/actor-framework/61b26c09-2b71-452e-b6be-07c210ac8246n%40googlegroups.com.