Multi threading websocket server on 0.4.0

[Takatoshi Kondo]

Hello,

I'm writing a message broker using webscoket++ 0.4.0. The message broker uses websocketpp::server<websocketpp::config::asio>. 

It is similar to the following documentation:

http://www.za.com/wphoydbesocketpp/manual/building-program-websocket

The message broker is a single thread server application, so far. I'd like to run the message broker on  Amazon EC2. The message broker receives and sends tons of message, so high network performance instance as c3.8xlarge is required. It has 32 vCPUs. I'd like to use them efficiently.So I started updating the message broker as multi-threading.

According to the following documentation, transport::asio_mt is not implemented yet: 

http://www.zaphoyd.com/websocketpp/manual/reference/thread-safety

So I can't call io_service::run() from multiple threads.

Is there any good way to implement the multi-threading message broker?

I'd like to use boost::asio based approach because I'm using io_service not only for websocket++ but also for other purposes. 

e.g.) console input and a http server to report health status for Amazon Elastic Load Ballancer.

However, boost::asio based is not mandatory requirement, if there is a better multihreading approach, I'd like to know that.

Regards,

Takatoshi Kondo

[Peter Thorson]

Hi Takatoshi,

That documentation is out of date. The currently shipping transport::asio supports multithreaded use via multiple threads calling io_service.run(). Do keep in mind that the regular limitations of asio’s multi-threaded io_service apply. You should do your own profiling, but I have seen diminishing returns for multi-threaded io_services around 4 workers. Anything you can pull out of the io_service loop will help (see the broadcast_server example for an example).

--
You received this message because you are subscribed to the Google Groups "WebSocket++" group.
To unsubscribe from this group and stop receiving emails from it, send an email to websocketpp...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Takatoshi Kondo]

Hi Peter,

Thank you for your quick reply.

2014年12月25日木曜日 2時19分44秒 UTC+9 Peter Thorson:

Hi Takatoshi,

That documentation is out of date. The currently shipping transport::asio supports multithreaded use via multiple threads calling io_service.run().

Sounds nice!

Do keep in mind that the regular limitations of asio’s multi-threaded io_service apply. You should do your own profiling, but I have seen diminishing returns for multi-threaded io_services around 4 workers. Anything you can pull out of the io_service loop will help (see the broadcast_server example for an example).

I've checked it out. I came up with one question. It seems that the example doesn't call io_service::run() multiple times. io_service::run() is called once from server_instance.run(9002) as follows:

https://github.com/zaphoyd/websocketpp/blob/master/examples/broadcast_server/broadcast_server.cpp#L152

https://github.com/zaphoyd/websocketpp/blob/master/examples/broadcast_server/broadcast_server.cpp#L66

process_messages() is a threads entry point. And threads are controlled by condition_variable and mutex manually.

https://github.com/zaphoyd/websocketpp/blob/master/examples/broadcast_server/broadcast_server.cpp#L101

I think that it is an multi-threaded example but it uses the different approach from calling multiple io_service::run(). Instead of do that, I can call websocketpp::server<websocketpp::config::asio>::run() from multiple threads. Am I understand correctly? Are there any limitations or pitfalls?

Here is an example that calls io_service::run() from threads:

http://www.boost.org/doc/libs/1_57_0/doc/html/boost_asio/examples/cpp03_examples.html#boost_asio.examples.cpp03_examples.http_server_3

http://www.boost.org/doc/libs/1_57_0/doc/html/boost_asio/example/cpp03/http/server3/server.cpp

void server::run()
{
  // Create a pool of threads to run all of the io_services.
  std::vector<boost::shared_ptr<boost::thread> > threads;
  for (std::size_t i = 0; i < thread_pool_size_; ++i)
  {
    boost::shared_ptr<boost::thread> thread(new boost::thread(
          boost::bind(&boost::asio::io_service::run, &io_service_)));
    threads.push_back(thread);
  }

  // Wait for all threads in the pool to exit.
  for (std::size_t i = 0; i < threads.size(); ++i)
    threads[i]->join();
}

Thanks,

Takatoshi

[Takatoshi Kondo]

I realized that I can't just call io_service::run() from multiple threads in my usecase. Because receive packets order would become unpredictable.

Takatoshi

2014年12月25日木曜日 13時17分23秒 UTC+9 Takatoshi Kondo:

[Peter Thorson]

When running multiple io_service::run threads, messages coming to a single connection will be in order and deterministic. Messages coming to two different connections will interleaved non-deterministicly.

[Takatoshi Kondo]

Hi Peter,

Good to know.  Thanks! I will consider my design again.

Takatoshi

2015年1月5日月曜日 19時36分08秒 UTC+9 Peter Thorson: