correct shut down of a websocket server

[Michael Grotz]

Hello everyone,
I just started doing some experiments with websocket++ a few days ago and I really like it.
If I terminate a server and restart it, it will fail, because the port is still taken . If I close the browser, I can restart the server. But the websocket connection closes if the server terminates, maybe it is a problem with the webserver (nginx)?
I modified the associative_storage.cpp example and added a server_exit function.
And I have another question: how Do I close a connection by server? I tried something like  m_server.close(hdl), but it didn't work.
thank you..

#include <iostream>
#include <map>
#include<thread>
#include <exception>
#include <typeinfo>
#include<boost/property_tree/ptree.hpp>
#include<boost/property_tree/json_parser.hpp>
#include<boost/property_tree/exceptions.hpp>
#include <websocketpp/config/asio_no_tls.hpp>
#include <websocketpp/server.hpp>
#include "libs/connection_data.hpp"
#include "libs/json_handler.hpp"

/********************************************************/
/********************************************************/
typedef websocketpp::server<websocketpp::config::asio> server;

using websocketpp::connection_hdl;
using websocketpp::lib::placeholders::_1;
using websocketpp::lib::placeholders::_2;
using websocketpp::lib::bind;

/********************************************************/

/********************************************************/
class print_server {
public:
    print_server() : m_next_sessionid(1) {
        m_server.init_asio();

        m_server.set_open_handler(bind(&print_server::on_open,this,::_1));
        m_server.set_close_handler(bind(&print_server::on_close,this,::_1));
        m_server.set_message_handler(bind(&print_server::on_message,this,::_1,::_2));
    }
/********************************************************/
/********************************************************/
    void on_open(connection_hdl hdl) {
        connection_data data;

        data.sessionid = m_next_sessionid++;
        data.name = "";

        m_connections[hdl] = data;
    }
/********************************************************/
/********************************************************/
    void on_close(connection_hdl hdl) {
        connection_data& data = get_data_from_hdl(hdl);

        std::cout << "Closing connection " << data.name
                  << " with sessionid " << data.sessionid << std::endl;

        m_connections.erase(hdl);
    }
/********************************************************/
/********************************************************/

    void on_message(connection_hdl hdl, server::message_ptr msg) {
        connection_data& data = get_data_from_hdl(hdl);


    try {
        m_server.send(hdl, msg->get_payload(), msg->get_opcode());
    } catch (const websocketpp::lib::error_code& e) {
        std::cout << "Echo failed because: " << e
                  << "(" << e.message() << ")" << std::endl;
    }


    data.json=jsonh::json2ptree(msg->get_payload());
    jsonh::print_session_ID(data);


        std::cout<<"message: "<<msg->get_payload()<<std::endl;
        if (data.name == "") {
            data.name = msg->get_payload();
            std::cout << "Setting name of connection with sessionid "
                      << data.sessionid << " to " << data.name << std::endl;
        } else {
            std::cout << "Got a message from connection " << data.name
                      << " with sessionid " << data.sessionid << std::endl;
        }


    }
/********************************************************/
/********************************************************/
    connection_data& get_data_from_hdl(connection_hdl hdl) {
        auto it = m_connections.find(hdl);

        if (it == m_connections.end()) {
            // this connection is not in the list. This really shouldn't happen
            // and probably means something else is wrong.
            throw std::invalid_argument("No data available for session");
        }

        return it->second;
    }
/********************************************************/
/********************************************************/
    void run(uint16_t &port) {

        m_server.listen(port);
        m_server.start_accept();
        m_server.run();

    }
/********************************************************/
/********************************************************/
    void exit_server() {
        m_server.stop();
    }
/********************************************************/
/********************************************************/
private:


    typedef std::map<connection_hdl,connection_data,std::owner_less<connection_hdl>> con_list;

    unsigned int m_next_sessionid;
    server m_server;
    con_list m_connections;
};

void control(print_server &server){
    while(1){
        std::string input;
        std::cout<<"please enter a command"<<std::endl;
        std::cout<<"EXIT to enter"<<std::endl;
        std::cin>>input;
        if(input=="EXIT"){
            std::cout<<"exiting!"<<std::endl;
            server.exit_server();
            break;
        }
    }

}

/********************************************************/
/********************************************************/
int main() {

    uint16_t port=9002;
    print_server server;
    std::thread t1(control,std::ref(server));
    server.run(port);
    t1.join();

}
/********************************************************/
/********************************************************/

[jimbo...@gmail.com]

This is very easy, and the functions are found here https://github.com/zaphoyd/websocketpp/blob/edb26d7721bd8c8c95f5f29807f274fa5afd7c4e/websocketpp/transport/asio/endpoint.hpp

I recommend you call stop_listening() first to allow your threads to process the remaining messages then finally call stop() to properly stop asio.  Both should be called on m_server in your case.

This lib is filled with tons of convenient functions!

[Michael Grotz]

Thank you Jimbo!

I changed my exit_function to:

    void exit_server() {
        m_server.stop_listening();
        std::cout<<"server is listening?: "<<m_server.is_listening()<<std::endl;
        m_server.stop();

    }

but now I get following error:

server is listening?: 0
[2014-05-15 17:44:14] [info] asio async_shutdown error: system:9 (Bad file descriptor)
[2014-05-15 17:44:14] [disconnect] Failed: Operation canceled
[2014-05-15 17:44:14] [info] handle_accept error: Operation canceled
[2014-05-15 17:44:14] [info] Stopping acceptance of new connections because the underlying transport is no longer listening.
$ ./associative_storage.wspp
please enter a command
EXIT to enter
[2014-05-15 17:44:45] [info] asio listen error: system:98 (Address already in use)
terminate called after throwing an instance of 'std::error_code'

Have you any idea?
thank you

[Peter Thorson]

Stop() should only be used to forcibly cancel all operations. Using it will leave sockets uncleanly closed, which may contribute to your address in use error when attempting to restart.

Closing in general for the asio transport:

The run() method will keep running until all "work" is done. By default there is a job that listens for new connections. As long as this job is running the server will not exit. Stop_listening() stops this listen job. After calling stop_listening() the server will exit once all active connections have been closed. Stop_listening() doesn't close connections, you (or the remote endpoints) are responsible for doing this.

Websocketpp also has a perpetual mode that allows it to continue running while not listening and with no connections active. This is mostly used by clients, but if you are using it as a server you obviously will want to disable perpetual mode also if you want a clean exit.

--
You received this message because you are subscribed to the Google Groups "WebSocket++" group.
To unsubscribe from this group and stop receiving emails from it, send an email to websocketpp...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Grotz]

Thanks for your response. is there a method to close an active connection by server?
thank you.

[Peter Thorson]

endpoint::close(hdl,…) will initiate a clean close on the connection identified by hdl. There are a few different overloads that handle errors differently and include different information to send for the reason the close is happening. This can be done either by the client or server.

A common pattern for a clean/soft server close would be to stop listening, then loop through all active connections and close each, then wait. Remember closing is an asynchronous/non-blocking operation. You can initiate all the closes at once and they will happen in parallel. When the last close is complete the original endpoint::run method will return.

[Michael Grotz]

Thank you a lot, that is the method I was looking for. But I guess I didn't get it.

I modified my exit-method:

    void exit_server() {

        m_server.stop_listening();
        for (auto it : m_connections) {
                try{
                m_server.close(it.first,404,"shutdown");
                }catch(websocketpp::lib::error_code ec){
                    std::cout<<"lib::error_code "<<ec<<std::endl;
                }
            }

    }

output:

[2014-05-16 01:34:33] [info] asio async_shutdown error: system:9 (Bad file descriptor)
[2014-05-16 01:34:33] [disconnect] Failed: Operation canceled
[2014-05-16 01:34:33] [info] handle_accept error: Operation canceled
[2014-05-16 01:34:33] [info] Stopping acceptance of new connections because the underlying transport is no longer listening.
lib::error_code websocketpp.processor:25

thank you for your help.

[Michael Grotz]

sorry, my bad

    void exit_server() {
        m_server.stop_listening();
        for (auto& it : m_connections) {
                try{
                m_server.close(it.first, websocketpp::close::status::normal, "");

                }catch(websocketpp::lib::error_code ec){
                    std::cout<<"lib::error_code "<<ec<<std::endl;
                }
            }

    }

Now the errorcode is gone. Bur restart still messed up.
But it is not an big issue to me, I can work with this library. Thank you a lot for your engagement.

[Peter Thorson]

After you call exit_server are you waiting on run() to return? Everything done in exit_server is asynchronous, so if you call that all then try and drop out of main or call exit() it will still uncleanly disconnect. If you are running endpoint::run() in another thread you'll want to call join on that thread in main.

One other thing to consider if faster restarts from unclean server shutdowns are needed is to turn on the SO_REUSEADDR socket option. There is an endpoint method to do this. It will eliminate the address in use errors. There are other downsides to do thing this though that may or may not be applicable. Please read up on the reuse addr socket option for your platform for details.

[jimbo...@gmail.com]

If you're on linux running Chrome, Chrome seems to hold on to ports even if the connection's been closed by the server.  

Try to also disconnect there before restarting your server.

---------------------------------

Tangential rant: I love being able to call send from anywhere inside the class, print_server in this case.  I don't think I can think of anything else more convenient.  My amped up process_messages thread sends messages to worker threads, who can work and then send out new information independent of the original process_messages loop.  Rarely do I have to send anything back to it, and that is simply amazing!

[Michael Grotz]

thank you for your response. I don't start run as a seperate thread, so I expect the main program to end if run returns. Just the control-loop which calls exit_server is started as a thread.

[David Chappelle]

What about when the process terminates abnormally? I get this error for a period of time when trying to restart my server after a crash:

    [2015-01-21 14:43:01] [info] asio listen error: system:98 (Address already in use)

After a while it goes away and I am able to successfully start the server. I am a bit concerned about this in a production environment. If the software crashes it will fail for a period of time trying to auto restart. Any advice?

[David Chappelle]

Maybe the underlying connection handling needs to specify SO_REUSEADDR when creating the listening socket?

[Peter Thorson]

Take a look at the SO_REUSEADDR and/or SO_REUSEPORT socket options. Be aware that they behave a little differently on different OSes so please consult your system documentation before deciding exactly what to use and the security implications (particularly on windows machines). For the former there is a wrapper, endpoint::set_reuse_addr(bool). Reuse port is less standard and there is no asio wrapper for it. You can use websocket++’s socket init handler to hook in and manually set platform specific socket options if needed.

[David Chappelle]

Awesome. I missed that part of the API. Thanks for pointing that out.

[David Chappelle]

The problem is here in websocketpp/roles/server_endpoint.hpp:

    void start_accept(lib::error_code & ec) {

        if (!transport_type::is_listening()) {

            ec = error::make_error_code(error::async_accept_not_listening);

            return;

        }

        ec = lib::error_code();

        connection_ptr con = get_connection();

        transport_type::async_accept(

            lib::static_pointer_cast<transport_con_type>(con),

            lib::bind(&type::handle_accept,this,con,lib::placeholders::_1),

            ec

        );

        if (ec && con) {

            // If the connection was constructed but the accept failed,

            // terminate the connection to prevent memory leaks

            con->terminate(lib::error_code());

        }

    }

A smart pointer to a connection object is created and bound into the accept handler. The socket from this connection is passed into the async_accept call as well. What happens here on shutdown is the acceptor defers its cleanup to the io_service. So if you stop the io_service prematurely or are trying to shutdown in the context of an io_service completion handler then the socket does not get cleaned up properly. An easy fix for this is to separate the socket from the connection object. In doing so you can maintain a concrete instance of the socket defer creation of the connection until the handle_accept is executed. At which time you can pass the socket of to the new connection. What this allows you to do then is explicitly close the socket when stop_listening() is called along with telling the acceptor to close. In this way, even if closing the acceptor is deferred and you screw that up then the socket at least is closed. The most common use case here

[Matthew von Arx]

Can you please give us a code sample of separating the socket and connection handlers?  I am having the same unclean shutdown problem referenced above.