[Boost-users] Asio and ZeroMQ
Matthias Vallentin
socket [1]. Based on his code, I put together a thin library that wraps the
implementation details. Since other have requested a working example, here it
is now at github:
Any feedback is highly appreciated. A basic usage example looks like this:
#include <boost/asio/io_service.hpp>
#include <bmq.h>
int main()
{
boost::asio::io_service io;
bmq::context ctx(1);
bmq::component c(ctx, io);
auto sink = c.add_sink(ZMQ_PAIR, "inproc://x");
c.subscribe(sink, [](bmq::message&& msg)
{
std::string str(static_cast<char*>(msg.data()), msg.size());
std::cout << str << std::endl;
});
auto source = c.add_source(ZMQ_PAIR, "inproc://x");
bmq::message msg(const_cast<char*>("foo"), 3, nullptr);
source->send(msg);
// Execute message handler.
io.poll();
return 0;
}
At this point, the library is really thin and only provides very rudimentary
wrapping of ZeroMQ. In the future, I plan to extend it in order to facilitate
applications that build on message-passing. The dataflow abstraction I envision
is essentially a directed graph, where nodes represent components and edges
communication links. 0mq provides the edges and the Asio handles asynchronous
event processing at the component level. The motiviation is that this model
lends itself well for scaling out distributed systems over large clusters.
Matthias
[1] http://permalink.gmane.org/gmane.network.zeromq.devel/9538
_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
asif saeed
I was actually looking for ASIO support in ZeroMQ. Thank you. But does your library provide what you say? I mean does it really use ZeroMQ on edges and use ASIO on a binary component level for distributed systems? I am looking for something like that and doing this myself would be the last option due to time constraints.
Matthias Vallentin
> ZeroMQ on edges and use ASIO on a binary component level for
> distributed systems?
I'm sorry if I was not clear enough. At this point, the library (bmq) is
really thin and basic. It is a proof-of-principle that you can use
ZeroMQ and Asio together; bmq does not provide a lot abstractions beyond
what ZeroMQ already offers. That is, it associates sockets with what I
call components, i.e., logical units of structuring your applications.
(If it helps, think of components as SEDA stages [1].) It also takes
these ZeroMQ sockets and hands them to the asynchronous io_service
proactor.
I will use library in my own projects, so it will probably change quite
a bit once I figure out what the right level of abstraction are.
In the long run, I would like to have message-passing that allows me to
create component-based software which I can easily distribute over
multiple machines, where bottlneck components can be executed on
separate nodes, or replicated to reduce the load on the edges. Hopefully
this helps to better understand what I am after.
Another aspect concern parallelism, both multi-core and across several
machines. Regarding the former, Both Asio and ZeroMQ use threads
(io_service/context). If messages cannot be sent fast enough, maybe it
makes sense to add more threads to the ZeroMQ context. If messages
cannot be consumed fast enough, it may make sense to launch more threads
on io_service::run(). Perhaps one can create a monitoring mechanism that
finds out these things automatically. Again, all this is just a vision.
Matthias
[1] http://matt-welsh.blogspot.com/2010/07/retrospective-on-seda.html
Raymond Burkholder
> I will use library in my own projects, so it will probably change quite
> a bit once I figure out what the right level of abstraction are.
> In the long run, I would like to have message-passing that allows me to
> create component-based software which I can easily distribute over
> multiple machines, where bottlneck components can be executed on
> separate nodes, or replicated to reduce the load on the edges. Hopefully
> this helps to better understand what I am after.
You might want to look at Zero for similar patterns. It isn't Boost
related, but is something worth looking at.
http://www.zeroc.com/icecpp.html
No affiliation, just a user.
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
Akira Takahashi
2012/2/8 Matthias Vallentin <vall...@icsi.berkeley.edu>:
> socket [1]. Based on his code, I put together a thin library that wraps the
> implementation details. Since other have requested a working example, here it
> is now at github:
>
> https://github.com/mavam/bmq
>
> Any feedback is highly appreciated. A basic usage example looks like this:
>
> #include <boost/asio/io_service.hpp>
> #include <bmq.h>
>
> int main()
> {
> boost::asio::io_service io;
> bmq::context ctx(1);
> bmq::component c(ctx, io);
>
> auto sink = c.add_sink(ZMQ_PAIR, "inproc://x");
> c.subscribe(sink, [](bmq::message&& msg)
> {
> std::string str(static_cast<char*>(msg.data()), msg.size());
> std::cout << str << std::endl;
> });
>
> auto source = c.add_source(ZMQ_PAIR, "inproc://x");
>
> bmq::message msg(const_cast<char*>("foo"), 3, nullptr);
> source->send(msg);
>
> // Execute message handler.
> io.poll();
>
> return 0;
> }
>
I cannot find this library. Did move repository?
>>========================
Akira Takahashi
mailto:faitha...@gmail.com
site: https://sites.google.com/site/faithandbrave/about/en