[rabbitmq-discuss] C client and thread safety

[Jim Irrer]

Greetings -

Firstly, I remain impressed with Rabbit as a group that can make solid, useful
software.

I'm working with the RabbitMQ C client, and had a few questions
about thread safety.  In particular:

Must a socket returned by amqp_open_socket be exclusively
associated with one connection, or can it be used for multiple
connections?

Must a socket returned by amqp_open_socket be exclusively
used by one thread, or can it be used by multiple threads?

Must a connection returned by amqp_new_connection be exclusively
used to read from or write to one channel/queue, or is using multiple ok,
even in a multi-threaded environment.

If any of these can be used by multiple threads, is there a performance
advantage in reusing the same socket or connection, or is using multiple
ones better?

It looks like channels (created with amqp_channel_open) are identified by (small?)
integers chosen by the application and are associated with the connection.  This
means that it is the responsibility of the application to choose unique numbers,
properly close them, etc..   It would seem cleaner to pass a connection as a
single parameter to amqp_channel_open and have it return a pointer to a channel,
and the list of channels would be maintained in the connection.

Also, it looks like amqp_new_connection uses 'calloc' to allocate
memory for a connection, but amqp_connection_close does not do
a 'free' to avoid the memory leak.  Maybe that was intentional, but if
not it should be changed.

Thanks,

- Jim

Jim Irrer     ir...@umich.edu       (734) 647-4409
University of Michigan Hospital Radiation Oncology
519 W. William St.             Ann Arbor, MI 48103

[Tony Garnock-Jones]

Hi Jim,

Jim Irrer wrote:
> I'm working with the RabbitMQ C client, and had a few questions
> about thread safety.

In general, there is no thread awareness built in to librabbitmq at all. It is
currently the responsibility of applications using the library to ensure that
multiple threads don't screw up the library's state.

That said, there is no longer any global state at all in the library, so if you
stick to one connection (and thus one socket) per thread, you should be fine.

> Must a socket returned by amqp_open_socket be exclusively
> associated with one connection, or can it be used for multiple
> connections?

One connection. Each AMQP connection needs exactly one socket to communicate over.

> Must a socket returned by amqp_open_socket be exclusively
> used by one thread, or can it be used by multiple threads?

The connection associated with the socket must be used by one thread *at a
time*. The locking can get quite complicated when you take into account the way
librabbitmq currently deals with incoming events, so sticking with one
connection for each thread makes life much easier.

> Must a connection returned by amqp_new_connection be exclusively
> used to read from or write to one channel/queue, or is using multiple ok,
> even in a multi-threaded environment.

Multiple are OK, but again, each connection is best kept to a single thread.
Think of multiple channels as separate *transaction* and *error reporting*
contexts; they don't buy much else in current versions of the AMQP protocol :-)

> If any of these can be used by multiple threads, is there a performance
> advantage in reusing the same socket or connection, or is using multiple
> ones better?

Using a single connection with multiple channels means that TCPs
congestion-avoidance counts all AMQP traffic as a single stream, whereas using
multiple connections with a single channel each means that TCP will count each
stream separately. Which is more appropriate to your application depends on the
structure of your network: the former might be more appropriate for a WAN link,
the latter for a LAN or a datacentre, etc.

> It looks like channels (created with amqp_channel_open) are identified
> by (small?)
> integers chosen by the application and are associated with the
> connection. This
> means that it is the responsibility of the application to choose unique
> numbers,
> properly close them, etc.. It would seem cleaner to pass a connection as a
> single parameter to amqp_channel_open and have it return a pointer to a
> channel,
> and the list of channels would be maintained in the connection.

Yes, there is a lot of per-channel state management the library currently
doesn't do. A set of "active" channel numbers might not be too hard to manage.

> Also, it looks like amqp_new_connection uses 'calloc' to allocate
> memory for a connection, but amqp_connection_close does not do
> a 'free' to avoid the memory leak. Maybe that was intentional, but if
> not it should be changed.

The amqp_connection_close() function causes an AMQP Connection.Close RPC to
happen on the open socket, but does not then tear down any of the application
state: that's what amqp_destroy_connection() is for. The relevant pairings are
roughly

For local connection state management (calloc/free):
amqp_new_connection
amqp_destroy_connection

For remote connection state management (Connection.Open/Tune/Close, once local
connection state exists):
amqp_login
amqp_connection_close
...plus interactions required on receipt of a Connection.Close method frame
from the peer.

Cheers,
Tony

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq...@lists.rabbitmq.com
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

[Jim Irrer]

Tony -

Thanks, that really helps.

- Jim

Jim Irrer     ir...@umich.edu       (734) 647-4409
University of Michigan Hospital Radiation Oncology
519 W. William St.             Ann Arbor, MI 48103

[Ralf Edmund Stranzenbach]

Hi,

for a long time, i simply ignored RabbitMQ-C client due to the fact, that it is marked as experimental code.

But the last week i spent some time checking this AMQP implementation and found it usable - at least on the Mac-OSX development machine I use. The target environment will be the iPad / iPhone and the time i write this, I had no time to check this library in that environment.

Currently my first simple tests derived from the example code seem to work fine. So i'll declare this attempt a success.

Currently I'll try to implement a wrapper on top of the C-API that manages a connection from the iPhone to the server that (correctly) reacts on the Networking events send by the iOS. My target application requires a (more or less) stable connection to the RabbitMQ infrastructure, but this requires frequent dis- and reconnects as the networking environment for the devices changes.

The exchange of message should happen in the background - but I expect this to be a single thread handling all that AMPQ communications...

For the next steps it would be useful to have some API specification or better a tutorial on the amqp_ procedures provided by the library. Is there any easy understandable documentation on this topic?

Regards,
- Ralf

[Scott Brooks]

I wrote a wrapper around the rabbitmq-c client quite a while ago, and
I used the XML docs, and PDF's from www.amqp.org as my main reference.

The language used internally is quite similar to the specs so I used
those specs to track down the order I had to do things in.

Scott Brooks