When will RabbitMQ Management HTTP API be production ready?

[Ye Wang]

Hi,

I noticed that on every instance of RabbitMQ that has Management plugin enabled, there is this HTTP API reference page: rabbithost:15672/api

In the middle of the reference page, for "/api/exchanges/vhost/name/publish" end point, it has this statement:

"Please note that the publish / get paths in the HTTP API are intended for injecting test messages, diagnostics etc - they do not implement reliable delivery and so should be treated as a sysadmin's tool rather than a general API for messaging."

My question is, does this mean that the HTTP REST API is not production ready? What does it mean by the "reliable delivery"? Our back office system currently only talks in JSON via REST APIs, so can we use this management plugin's HTTP REST API for production use?Or somehow we have build a middle ware that translate between our backoffice platform and the RabbitMQ? Anyone knows the differences between talking to RabbitMQ via the native RabbitMQ client libraries vs. using this REST API provided by Management plugin. 

Thanks!!

[Michael Klishin]

On 27 June 2014 at 00:26:38, Ye Wang (ye.wang.a...@gmail.com) wrote:
> > My question is, does this mean that the HTTP REST API is not production
> ready? What does it mean by the "reliable delivery"? Our back
> office system currently only talks in JSON via REST APIs, so can
> we use this management plugin's HTTP REST API for production
> use

HTTP API is production ready but it was create for monitoring and ops issues,
not messaging. HTTP API does not support publisher confirms, for example.

The management plugin, which has been around for quite a while, is built on the
HTTP API. 
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Klishin]

 On 27 June 2014 at 00:26:33, Ye Wang (ye.wang.a...@gmail.com) wrote:
> > Anyone knows the differences between talking to RabbitMQ via
> the native RabbitMQ client libraries vs. using this REST API
> provided by Management plugin.

RabbitMQ client libraries provide a very different set of features. So do
STOMP and MQTT clients. HTTP API is for monitoring, not publishing
and consumption of messages, and does not try to replicate AMQP, STOMP or MQTT
features (although there is some overlap).

[Ye Wang]

Thanks Daniel & Michael for chiming in. 

I got what the intended usages of Management HTTP API are for. However, there are two constraints in our case:

1. Our backoffice/ERP software is a SaaS platform - it's NOT possible to use any of the client libraries to program AMQP implementation on that SaaS platform because all lower level APIs are not available to us (cannot install 3rd party library, no socket programming API etc.).

2. The backoffice/ERP software would only communicate in JSON via REST APIs, so STOMP or MQTT protocols don't apply here. This is a web application in a web-centric enterprise environment - not client/server or mainframe environment like the C/C++/Java/.Net are suited for.

Having said that, does this mean that Rabbit is inflexible to be used as a Message Queue for JSON / REST API based web-centric applications? 

Obviously there is this last resort where we can write a middleware to bridge the JSON/REST API with AMQP, but it's duplicating the efforts on the Management HTTP API, IMMO.

[Michael Klishin]

On 27 June 2014 at 18:42:07, Ye Wang (ye.wang.a...@gmail.com) wrote:
> > Having said that, does this mean that Rabbit is inflexible to
> be used as a Message Queue for JSON / REST API based web-centric
> applications?

You *can* use HTTP API to publish and consume messages. It just is not why it was created,
and it lacks some important features other protocols provide.

Countless "JSON / REST API based web-centric applications" use RabbitMQ,
even in very restrictive environments such as Heroku. 

[Ye Wang]

In fact, the question we had can be rephrased as this:

Doesnt RabbitMQ support a solid (100% or almost feature complete) REST API + JSON interface for other systems that doesn't speak AMQP to communicate with? 

RabbitMQ has been excellent in providing a few dozen different client libraries in almost any programming languages, however, in the current web-centric world and SaaS / PaaS flourishing days, installing those client libraries are really not possible.

Management HTTP API provides a reasonably set of features and REST end points, I really hope it can be grown into a fully fledge REST API interface for messaging needs.

[Michael Klishin]

 On 27 June 2014 at 18:49:55, Ye Wang (ye.wang.a...@gmail.com) wrote:
> > Doesnt RabbitMQ support a solid (100% or almost feature complete)
> REST API + JSON interface for other systems that doesn't speak
> AMQP to communicate with?

You can publish and consume messages over HTTP API. They will be routed
internally using all the regular AMQP 0-9-1 features, so you can take advantage
of different exchanges types, for example.

> RabbitMQ has been excellent in providing a few dozen different
> client libraries in almost any programming languages, however,
> in the current web-centric world and SaaS / PaaS flourishing
> days, installing those client libraries are really not possible.

Like I said earlier, plenty of environments do not have the restrictions your environment has
(that clients can only use HTTP). Even Heroku and similar ones support all
kinds of services (PostgreSQL, RabbitMQ, Redis, MongoDB, etc) that do not use HTTP
or HTTP is not their primary protocol.

> Management HTTP API provides a reasonably set of features and
> REST end points, I really hope it can be grown into a fully fledge
> REST API interface for messaging needs.

Sorry, you keep using "production ready" and "fully fledged" without specifying
what exactly it is you need. You can publish and consume messages over RabbitMQ HTTP API.
These features have been around for basically as long as the management plugin has been
(which is about 5 years).

What else *specifically* do you need? I don't know if HTTP API is "production ready",
"fully fledged" or "Web scale" but it has been by far the most heavily used plugin
for several years.

[Ye Wang]

Thank Michael. It's great to hear some validation above. 

My intention is not to say RabbitMQ is not capable or not production ready - but is to find out why and how to use it in our unique use case, more specifically via REST API/JSON, described above.

To follow up on what I was curious to know - is it on GoPivotal's roadmap to adopt full AMQP spec support on the Management HTTP API, perhaps a spin off from the Management plugin and grow it into a fully fledge REST API stand alone plugin even though it wasn't originally intended to be that way?

For example, you said HTTP API message publish doesn't implement message acknowledgement. Will you support that fully in the future? In other words, is it possible to build a REST API plugin for Rabbit with 100% AMQP spec feature implemented? To me, that's the big differentiator to assure other usages in hosted web applications - Heroku and alikes.

[Michael Klishin]

On 27 June 2014 at 19:00:51, Ye Wang (ye.wang.a...@gmail.com) wrote:
> > To follow up on what I was curious to know - is it on GoPivotal's
> roadmap to adopt full AMQP spec support on the Management HTTP
> API, perhaps a spin off from the Management plugin and grow it
> into a fully fledge REST API stand alone plugin even though it
> wasn't originally intended to be that way?

You already have support for most AMQP 0-9-1 methods:

* queue.declare
* queue.bind
* queue.delete
* exchange.declare
* exchange.bind
* exchange.delete
* basic.publish
* basic.get

over HTTP API. You don't have support for basic.consume because HTTP is stateless.

What else do you need?

> For example, you said HTTP API message publish doesn't implement
> message acknowledgement. Will you support that fully in the
> future? In other words, is it possible to build a REST API plugin
> for Rabbit with 100% AMQP spec feature implemented? To me, that's
> the big differentiator to assure other usages in hosted web applications
> - Heroku and alikes.

HTTP is lacking features for stateful protocols and messaging. That's why
WebSockets, SPDY/HTTP2, etc. keep being invented.

Publisher confirms — which is what I was referring to — assumes the server
can send data to the client as certain events happen. This is not possible with
plain HTTP and with techniques such as long polling or SSE, you need to have
a more advanced (and custom) client. And if you are willing to use custom clients,
you might as well consider a different protocol, be it Web STOMP, STOMP, or binary
ones like AMQP 0-9-1 and MQTT.

So if you are restricted to just plain HTTP, you already have virtually
all the features RabbitMQ HTTP API can offer. 

[Ye Wang]

Perfect and got it. That's the exact answer what I was looking for - HTTP API is only restricted by the protocol itself, not by the Management HTTP API implementation.

Thank you again Michael for your help.