Is the Erlang PID present anywhere in the Management HTTP API?

[David Diehl]

I am using RabbitMQ version 3.7.3

Is the Erlang PID present anywhere in the Management HTTP API?

So, when I run this command:   rabbitmqctl list_connections pid state user

The output has the Erlang PID as the first column.

However, when I run this command:  curl -i -u guest:guest 'http://localhost:15672/api/connections'

and examine the output, I don't see anything that corresponds to the Erlang PID.  Is there any plan to add it?

Note:  I'd like to know the Erlang PID in order to match connections up with the output of the connection_created and connection_closed events that are emitted by the Event Exchange Plugin.  On application startup I wanted to capture all of the current connections, and then use the connection_created and connection_closed events as an efficient way to keep the current state of what connections are currently active.

[Michael Klishin]

Why would you need that information over HTTP API?

With HTTP API connections are closed using their name, which is returned by the API.

Connection process IDs are an implementation detail that was exposed to `list_connections` in the

very early days.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[David Diehl]

Why would I want the Erlang PID in the HTTP API?  I tried to explain that in my original email in the last paragraph that started with "Notes:".  I probably didn't explain it enough.  I was wanting to use the Event Exchange Plugin and it has the Erlang PID in the data it provides.  Using the Event Exchange Plugin I can register for connection open and closed events, and since it includes the Erlang PID, I know which connection is the one that is being opened/closed.  If I start with an original set of connections (upon my application startup) that I could obtain using the HTTP API, then I can use the Erlang PID from the Event Exchange Plugin to match up which connection is being closed.

So, let's say I start by application and call the HTTP API upon startup.  Suppose that gives me a set of 3 connections.  I could store that set of 3 connections, perhaps "keyed" by the Erlang PID.  Now, at some later point in time, I get connection closed event from the Event Exchange Plugin.  Since it includes the Erlang PID, I can perform a lookup in my set of 3 connections and remove the one with the matching Erlang PID.  This way I can maintain an accurate running state of the connections.

The alternative is to repeatedly invoke the HTTP API to get the full set of connections.  That is what I am doing now, and the only problem with that is how long it takes that call to respond.  I am making this call every 15 seconds, and there are times where it takes 7 or 8 seconds for the (RabbitMQ) server to respond to that call.  So I was looking at the Event Exchange Plugin as a means to make this (hopefully) much more efficient.  Things could be slower for me because I have about 2300 or so connections to my RabbitMQ server.  Please advise if you think that is an unusually high number  of connections to have to one server (1 server, so not a cluster).

On Friday, May 4, 2018 at 10:46:07 PM UTC-4, Michael Klishin wrote:

Why would you need that information over HTTP API?

With HTTP API connections are closed using their name, which is returned by the API.

Connection process IDs are an implementation detail that was exposed to `list_connections` in the

very early days.

On Fri, May 4, 2018 at 11:50 AM, David Diehl <vengi...@gmail.com> wrote:

I am using RabbitMQ version 3.7.3

Is the Erlang PID present anywhere in the Management HTTP API?

So, when I run this command:   rabbitmqctl list_connections pid state user

The output has the Erlang PID as the first column.

However, when I run this command:  curl -i -u guest:guest 'http://localhost:15672/api/connections'

and examine the output, I don't see anything that corresponds to the Erlang PID.  Is there any plan to add it?

Note:  I'd like to know the Erlang PID in order to match connections up with the output of the connection_created and connection_closed events that are emitted by the Event Exchange Plugin.  On application startup I wanted to capture all of the current connections, and then use the connection_created and connection_closed events as an efficient way to keep the current state of what connections are currently active.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

Why isn't the set of connections reported by the HTTP API good enough?

I think the right thing to do here is to emit the same connection name as used by the HTTP API into

internal events. Erlang PIDs make no sense to non-Erlang systems but string identifiers usually do.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[David Diehl]

Well, I see a "client_properties"  <<"connection_name"">> present on the "connection.created" events.  However, that information is not present on the "connection.closed" events.  Are you suggesting that this same connection name be added to the "connection.closed" events that are produced by the Event Exchange Plugin?  If that is what you mean, how should I go about requesting that change?  If that is not what you meant, then I suppose I don't understand what you meant by "internal events".

[Michael Klishin]

The events are not produced by the events-exchange plugin. The plugin merely exposes them to RabbitMQ clients.

We can add connection name to those events. However, adding it in too many places is a real concern I have.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[David Diehl]

Have you made a decision after giving it some time and more thought?  I can only add that there isn't very much data at all in the "connection.closed" event, so it certainly doesn't seem that adding this one field would be bloating that.  Further, without the connection name there, it's impossible to correlate that event to the set of connections that can be obtained from the management API.  And that is all I am trying to do.

Oh, and thank you very much for fixing #40 so quickly.  It may have been low-effort, but still, that's awesome.  I already tested this in 3.7.5 and it works great.

[Michael Klishin]

We would be up for adding more fields but exposing Erlang pids sounds like a bad idea because it is an implementation detail that

makes no sense to programming languages not hosted on the BEAM runtime.

Other than that, what fields specifically would you like to see added to what connection events?

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[David Diehl]

Specifically <<"connection.name">> on the connection.closed event.  That's it.

[Michael Klishin]

https://github.com/rabbitmq/rabbitmq-server/issues/1596.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Daniil Fedotov]

Hi.

It looks like you can use a 'name' field to identify connections. The field looks like this (for localhost connection): `127.0.0.1:59329 -> 127.0.0.1:5672` This field is available in the connection.closed event, in the connection.created event, in the CLI (`rabbitmqctl list_connections name`) and the management API. In the management API it is used as an API pint to manipulate connections.

It may still make sense to include client properties into connection.closed event, but in your case using the `name` field looks like a better solution.

[David Diehl]

Thanks for pointing this out, but alas, this won't work for me.  My RabbitMQ clients are behind a proxy (a load balancer actually) so I see the IP address in the name as the proxy and not my actual client.  Therefore, all of my name values are the same value (the IP address of the proxy).

--

[Michael Klishin]

There is a solution for that: https://www.rabbitmq.com/networking.html#proxy-protocol.

To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Diehl]

That is interesting, however, I have the situation where there are multiple connections from the same server, and I need to distinguish the connections from each other; so even the IP address of the client isn't enough.  I hoping the issue that Michael created for me recently will do the trick:  https://github.com/rabbitmq/rabbitmq-server/issues/1596

On Tuesday, May 15, 2018 at 6:19:17 PM UTC-4, Michael Klishin wrote:

There is a solution for that: https://www.rabbitmq.com/networking.html#proxy-protocol.

On Tue, May 15, 2018 at 3:59 PM, David Diehl <vengi...@gmail.com> wrote:

Thanks for pointing this out, but alas, this won't work for me.  My RabbitMQ clients are behind a proxy (a load balancer actually) so I see the IP address in the name as the proxy and not my actual client.  Therefore, all of my name values are the same value (the IP address of the proxy).

On Tue, May 15, 2018 at 7:11 AM, Daniil Fedotov <fedoto...@gmail.com> wrote:

Hi.

It looks like you can use a 'name' field to identify connections. The field looks like this (for localhost connection): `127.0.0.1:59329 -> 127.0.0.1:5672` This field is available in the connection.closed event, in the connection.created event, in the CLI (`rabbitmqctl list_connections name`) and the management API. In the management API it is used as an API pint to manipulate connections.

It may still make sense to include client properties into connection.closed event, but in your case using the `name` field looks like a better solution.

On Friday, 4 May 2018 17:50:26 UTC+1, David Diehl wrote:

I am using RabbitMQ version 3.7.3

Is the Erlang PID present anywhere in the Management HTTP API?

So, when I run this command:   rabbitmqctl list_connections pid state user

The output has the Erlang PID as the first column.

However, when I run this command:  curl -i -u guest:guest 'http://localhost:15672/api/connections'

and examine the output, I don't see anything that corresponds to the Erlang PID.  Is there any plan to add it?

Note:  I'd like to know the Erlang PID in order to match connections up with the output of the connection_created and connection_closed events that are emitted by the Event Exchange Plugin.  On application startup I wanted to capture all of the current connections, and then use the connection_created and connection_closed events as an efficient way to keep the current state of what connections are currently active.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--