RabbitMQ COAP plugin (experimental)
151 views
Skip to first unread message
Laing, Michael
Dec 18, 2015, 8:02:39 PM12/18/15
to rabbitm...@googlegroups.com
The gotthardp COAP implementation would be more useful if it used bindings on a common topic exchange rather than distinct exchanges for topics. The current COAP implementation makes it difficult to interoperate with the RabbitMQ MQTT implementation, a desirable goal IMHO.
ml
Gotthard, Petr
Dec 19, 2015, 5:41:01 AM12/19/15
to rabbitm...@googlegroups.com
Hello.
This deserves thorough discussion. The protocols are very different. MQTT is designed for messaging, whereas CoAP is for accessing resources. CoAP operations are required to be idempotent: a resource can be read multiple times, storing a resource that has been stored before has no effect. MQTT and messaging in general is dealing with events rather than resources. Publishing a message that has been published will publish it again, message consumption is a destructive operation. Is is not idempotent.
To implement a CoAP access to a messaging broker one has to defime what will be the resource being accessed. There are multiple options depending on what you want to achieve. In my implementation CoAP is used to access the LVC cache. The cache entries are the resources. This nicely blends messaging with the REST paradigm, but as you pointed out it requires the lvc exchange. Since the coap-pubsub plugin uses a generic CoAP implementation it is very easy to do anothet plugin that will use CoAP to access some other resources. Do you have something specific in mind you want to access via CoAP?
One could think of accessing message queues via CoAP. This is similar to the recurring discusions on this list on how to use HTTP to consume messages. It is a tricky subject because CoAP operations are assumed to be idempotent. One could use CoAP GET to non-destructively read a message, but what shall CoAP PUT do? Remember that PUT-ing a resource that is already there shall have no effect.
I dont think there is a single obvious solution for CoAP support in Rabbit. I just implemented one possible option and I was hoping to learn about other pepople needs and use-cases.
Best Regards,
Petr
________________________________________
From: rabbitm...@googlegroups.com [rabbitm...@googlegroups.com] on behalf of Laing, Michael [michae...@nytimes.com]
Sent: Saturday, December 19, 2015 02:02
To: rabbitm...@googlegroups.com
Subject: [rabbitmq-users] RabbitMQ COAP plugin (experimental)
--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com<mailto:rabbitmq-user...@googlegroups.com>.
To post to this group, send email to rabbitm...@googlegroups.com<mailto:rabbitm...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.
This deserves thorough discussion. The protocols are very different. MQTT is designed for messaging, whereas CoAP is for accessing resources. CoAP operations are required to be idempotent: a resource can be read multiple times, storing a resource that has been stored before has no effect. MQTT and messaging in general is dealing with events rather than resources. Publishing a message that has been published will publish it again, message consumption is a destructive operation. Is is not idempotent.
To implement a CoAP access to a messaging broker one has to defime what will be the resource being accessed. There are multiple options depending on what you want to achieve. In my implementation CoAP is used to access the LVC cache. The cache entries are the resources. This nicely blends messaging with the REST paradigm, but as you pointed out it requires the lvc exchange. Since the coap-pubsub plugin uses a generic CoAP implementation it is very easy to do anothet plugin that will use CoAP to access some other resources. Do you have something specific in mind you want to access via CoAP?
One could think of accessing message queues via CoAP. This is similar to the recurring discusions on this list on how to use HTTP to consume messages. It is a tricky subject because CoAP operations are assumed to be idempotent. One could use CoAP GET to non-destructively read a message, but what shall CoAP PUT do? Remember that PUT-ing a resource that is already there shall have no effect.
I dont think there is a single obvious solution for CoAP support in Rabbit. I just implemented one possible option and I was hoping to learn about other pepople needs and use-cases.
Best Regards,
Petr
________________________________________
From: rabbitm...@googlegroups.com [rabbitm...@googlegroups.com] on behalf of Laing, Michael [michae...@nytimes.com]
Sent: Saturday, December 19, 2015 02:02
To: rabbitm...@googlegroups.com
Subject: [rabbitmq-users] RabbitMQ COAP plugin (experimental)
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com<mailto:rabbitmq-user...@googlegroups.com>.
To post to this group, send email to rabbitm...@googlegroups.com<mailto:rabbitm...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.
Laing, Michael
Dec 19, 2015, 7:49:31 AM12/19/15
to rabbitm...@googlegroups.com
I agree it needs discussion - and thanks for work you have done!
In general, we would be placing services - some complex - in our servers to respond to requests and facilitate side effects such as persistence and analysis. The LVC is suitable for a subset of requirements, but inconvenient for complex services as well as hindering interoperability IMHO.
We would be utilizing the 'observe' functionality of COAP (RFC 7641) for 'push' to clients, and in that sense we are concerned about messaging.
Overall, we would like to experiment with a suite of protocols (AMQP, MQTT, COAP, STOMP, ...) and establish interoperability amongst them as needed with a set of internal services and standards.
Hence a COAP client observing an endpoint might receive a sequence of representations as a result of actions by clients using other protocols.
To enable this, it's easier if the protocol implementations in RabbitMQ use interoperable approaches.
For example: each client has a queue, there is a single topic exchange for the service, bindings are dynamic, etc.
Using this example it would be quite easy to build simple internal services, implementing internal standards, to transform messages among the protocol implementations and facilitate common services using multiple protocols - we already do this with MQTT and AMQP.
But perhaps there are other approaches as well :)
Cheers,
ml
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send an email to rabbitm...@googlegroups.com.
Gotthard, Petr
Dec 19, 2015, 8:54:41 AM12/19/15
to rabbitm...@googlegroups.com
I see your struggle. Is the thing you want to observe something else than the last published value? If not, then it may be the lvc plugin that needs improvements, not the coap-pubsub plugin. If there was a last value cache accessible via CoAP for every exchange type (both standard and all custom ones) would this solve your concern?
Petr
________________________________________
From: rabbitm...@googlegroups.com [rabbitm...@googlegroups.com] on behalf of Laing, Michael [michae...@nytimes.com]
Sent: Saturday, December 19, 2015 13:49
To: rabbitm...@googlegroups.com
Subject: Re: [rabbitmq-users] RabbitMQ COAP plugin (experimental)
I agree it needs discussion - and thanks for work you have done!
In general, we would be placing services - some complex - in our servers to respond to requests and facilitate side effects such as persistence and analysis. The LVC is suitable for a subset of requirements, but inconvenient for complex services as well as hindering interoperability IMHO.
We would be utilizing the 'observe' functionality of COAP (RFC 7641) for 'push' to clients, and in that sense we are concerned about messaging.
Overall, we would like to experiment with a suite of protocols (AMQP, MQTT, COAP, STOMP, ...) and establish interoperability amongst them as needed with a set of internal services and standards.
Hence a COAP client observing an endpoint might receive a sequence of representations as a result of actions by clients using other protocols.
To enable this, it's easier if the protocol implementations in RabbitMQ use interoperable approaches.
For example: each client has a queue, there is a single topic exchange for the service, bindings are dynamic, etc.
Using this example it would be quite easy to build simple internal services, implementing internal standards, to transform messages among the protocol implementations and facilitate common services using multiple protocols - we already do this with MQTT and AMQP.
But perhaps there are other approaches as well :)
Cheers,
ml
To post to this group, send email to rabbitm...@googlegroups.com<mailto:rabbitm...@googlegroups.com><mailto:rabbitm...@googlegroups.com<mailto:rabbitm...@googlegroups.com>>.
To post to this group, send an email to rabbitm...@googlegroups.com<mailto:rabbitm...@googlegroups.com>.
Petr
________________________________________
From: rabbitm...@googlegroups.com [rabbitm...@googlegroups.com] on behalf of Laing, Michael [michae...@nytimes.com]
To: rabbitm...@googlegroups.com
Subject: Re: [rabbitmq-users] RabbitMQ COAP plugin (experimental)
I agree it needs discussion - and thanks for work you have done!
In general, we would be placing services - some complex - in our servers to respond to requests and facilitate side effects such as persistence and analysis. The LVC is suitable for a subset of requirements, but inconvenient for complex services as well as hindering interoperability IMHO.
We would be utilizing the 'observe' functionality of COAP (RFC 7641) for 'push' to clients, and in that sense we are concerned about messaging.
Overall, we would like to experiment with a suite of protocols (AMQP, MQTT, COAP, STOMP, ...) and establish interoperability amongst them as needed with a set of internal services and standards.
Hence a COAP client observing an endpoint might receive a sequence of representations as a result of actions by clients using other protocols.
To enable this, it's easier if the protocol implementations in RabbitMQ use interoperable approaches.
For example: each client has a queue, there is a single topic exchange for the service, bindings are dynamic, etc.
Using this example it would be quite easy to build simple internal services, implementing internal standards, to transform messages among the protocol implementations and facilitate common services using multiple protocols - we already do this with MQTT and AMQP.
But perhaps there are other approaches as well :)
Cheers,
ml
On Sat, Dec 19, 2015 at 5:40 AM, Gotthard, Petr <Petr.G...@konicaminolta.cz<mailto:Petr.G...@konicaminolta.cz>> wrote:
Hello.
This deserves thorough discussion. The protocols are very different. MQTT is designed for messaging, whereas CoAP is for accessing resources. CoAP operations are required to be idempotent: a resource can be read multiple times, storing a resource that has been stored before has no effect. MQTT and messaging in general is dealing with events rather than resources. Publishing a message that has been published will publish it again, message consumption is a destructive operation. Is is not idempotent.
To implement a CoAP access to a messaging broker one has to defime what will be the resource being accessed. There are multiple options depending on what you want to achieve. In my implementation CoAP is used to access the LVC cache. The cache entries are the resources. This nicely blends messaging with the REST paradigm, but as you pointed out it requires the lvc exchange. Since the coap-pubsub plugin uses a generic CoAP implementation it is very easy to do anothet plugin that will use CoAP to access some other resources. Do you have something specific in mind you want to access via CoAP?
One could think of accessing message queues via CoAP. This is similar to the recurring discusions on this list on how to use HTTP to consume messages. It is a tricky subject because CoAP operations are assumed to be idempotent. One could use CoAP GET to non-destructively read a message, but what shall CoAP PUT do? Remember that PUT-ing a resource that is already there shall have no effect.
I dont think there is a single obvious solution for CoAP support in Rabbit. I just implemented one possible option and I was hoping to learn about other pepople needs and use-cases.
Best Regards,
Petr
________________________________________
From: rabbitm...@googlegroups.com<mailto:rabbitm...@googlegroups.com> [rabbitm...@googlegroups.com<mailto:rabbitm...@googlegroups.com>] on behalf of Laing, Michael [michae...@nytimes.com<mailto:michae...@nytimes.com>]
Hello.
This deserves thorough discussion. The protocols are very different. MQTT is designed for messaging, whereas CoAP is for accessing resources. CoAP operations are required to be idempotent: a resource can be read multiple times, storing a resource that has been stored before has no effect. MQTT and messaging in general is dealing with events rather than resources. Publishing a message that has been published will publish it again, message consumption is a destructive operation. Is is not idempotent.
To implement a CoAP access to a messaging broker one has to defime what will be the resource being accessed. There are multiple options depending on what you want to achieve. In my implementation CoAP is used to access the LVC cache. The cache entries are the resources. This nicely blends messaging with the REST paradigm, but as you pointed out it requires the lvc exchange. Since the coap-pubsub plugin uses a generic CoAP implementation it is very easy to do anothet plugin that will use CoAP to access some other resources. Do you have something specific in mind you want to access via CoAP?
One could think of accessing message queues via CoAP. This is similar to the recurring discusions on this list on how to use HTTP to consume messages. It is a tricky subject because CoAP operations are assumed to be idempotent. One could use CoAP GET to non-destructively read a message, but what shall CoAP PUT do? Remember that PUT-ing a resource that is already there shall have no effect.
I dont think there is a single obvious solution for CoAP support in Rabbit. I just implemented one possible option and I was hoping to learn about other pepople needs and use-cases.
Best Regards,
Petr
________________________________________
Sent: Saturday, December 19, 2015 02:02
To: rabbitm...@googlegroups.com<mailto:rabbitm...@googlegroups.com>
Subject: [rabbitmq-users] RabbitMQ COAP plugin (experimental)
The gotthardp COAP implementation would be more useful if it used bindings on a common topic exchange rather than distinct exchanges for topics. The current COAP implementation makes it difficult to interoperate with the RabbitMQ MQTT implementation, a desirable goal IMHO.
ml
https://github.com/rabbitmq/rabbitmq-server/issues/89
https://github.com/gotthardp/rabbitmq-coap-pubsub
--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com<mailto:rabbitmq-users%2Bunsu...@googlegroups.com><mailto:rabbitmq-user...@googlegroups.com<mailto:rabbitmq-users%2Bunsu...@googlegroups.com>>.
The gotthardp COAP implementation would be more useful if it used bindings on a common topic exchange rather than distinct exchanges for topics. The current COAP implementation makes it difficult to interoperate with the RabbitMQ MQTT implementation, a desirable goal IMHO.
ml
https://github.com/rabbitmq/rabbitmq-server/issues/89
https://github.com/gotthardp/rabbitmq-coap-pubsub
--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To post to this group, send email to rabbitm...@googlegroups.com<mailto:rabbitm...@googlegroups.com><mailto:rabbitm...@googlegroups.com<mailto:rabbitm...@googlegroups.com>>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com<mailto:rabbitmq-users%2Bunsu...@googlegroups.com>.
--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To post to this group, send an email to rabbitm...@googlegroups.com<mailto:rabbitm...@googlegroups.com>.
Laing, Michael
Dec 21, 2015, 8:14:55 AM12/21/15
to rabbitm...@googlegroups.com
You are right that I am basically thinking of CoAP as a messaging alternative rather than request/response.
This is still just a thought experiment, paralleling actual experiments/development using AMQP and MQTT...
While initializing a typical client would do a GET/OBSERVE on an endpoint unique to that client/session.
Then, typically, the client would POST a request to a service with a REPLY_TO private option or query parameter containing the endpoint.
The client continues to GET/POST/PUT/DELETE asynchronously to/from services wrapping state and mutations.
We have millions of endpoints.
This can be handled by bindings on topic exchanges because the patricia trie implementation is very efficient - and we manipulate our topics to create dense trie structures. When a queue expires, its bindings go too, so cleanup is easy. The bindings are propagated by federation.
But if exchanges are used, we cannot benefit from a trie for routing, cleanup becomes a problem, and federation doesn't work.
We would also like the services to be general.
With binding-based implementations like MQTT or STOMP, we can write a single AMQP transformation service to enable them to interoperate with each other and AMQP-based services.
With the exchange-based approach, a simple transformer is more difficult and I think we would have to fall back to differentiating services by protocol, giving up interoperability, and bringing things together at our persistence layer.
But all this is still a thought experiment. My internal clients have not heard of CoAP (yet), I am just thinking ahead, particularly as we push into the gaming and chat application domains. When they (re)discover UDP, I want to be prepared with a structured, standard protocol alternative.
ml
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send an email to rabbitm...@googlegroups.com.
Gotthard, Petr
Dec 21, 2015, 9:14:12 AM12/21/15
to rabbitm...@googlegroups.com
What do you mean by “if exchanges are used, we cannot benefit from …”. Topic exchanges are also exchanges, aren’t they?
Petr
From: rabbitm...@googlegroups.com [mailto:rabbitm...@googlegroups.com]
On Behalf Of Laing, Michael
Sent: pondělí 21. prosince 2015 14:15
To: rabbitm...@googlegroups.com
Subject: Re: [rabbitmq-users] RabbitMQ COAP plugin (experimental)
You are right that I am basically thinking of CoAP as a messaging alternative rather than request/response.
Laing, Michael
Dec 21, 2015, 10:21:50 AM12/21/15
to rabbitm...@googlegroups.com
I meant if exchanges are used for endpoints, which in our applications are numerous (millions) and transient.
--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages