RabbitMQ - Smart Meters Design (Enterprise)

[Matthew Clarke]

Hello, 

We are currently building out a green field service to manage a large volume of smart meters within the utility industry. 

We are looking at RabbitMQ to handle the message brokering from the Smart Meters out towards an external service aswell as to an internal web service and test framework. 

However, one core aspect of the service is the requirement to send and receive responses to a service we have no control over. So we can create a producer on the external service. 

Does anyone have any idea about how to handle the inbound response from an external service into rabbitmq? 

I've attached a draft soultions architecture to give a better idea. 

Ta, 

Matt Clarke 

[Karl Nilsson]

The external service must have some means of calling back once processing has been done so it all depends on what that external service supports in terms of protocol and discoverability. if it is a protocol RabbitMQ doesn't support directly (.e.g HTTP) you'd have to bridge it else you can go directly to RabbitMQ (e.g. AMQP or MQTT).

Cheers

Karl

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/d3d3f89c-d7b6-44c3-956e-f98d53cee516n%40googlegroups.com.

--

Karl Nilsson

[Artur Wroblewski]

On Fri, Mar 31, 2023 at 05:50:28AM -0700, Matthew Clarke wrote:
[...]

> However, one core aspect of the service is the requirement to send and
> receive responses to a service we have no control over.

Probably your best bet is RabbitMQ MQTT plugin for external services. I am
using it on much smaller scale (an inverter for solar, and a few of Shelly
EM devices), but it works for months without a hitch now.

Shelly EM devices send data over MQTT, which I route to RabbitMQ Streams,
and my custom application uses power of RabbitMQ (routing, libraries).

I find the solution quite powerful, but YMMV at different scale, of course.

[...]

Best regards,

Artur

--
https://mortgage.diy-labs.eu/

[Matthew Clarke]

Hey, 

This is interesting thank you. 

I think we will look at the MQTT plugin and carry out some POC to see if we can get it working to a reasonable level. If we do we will share out the implementation with the community. 

I've seen projects like yours or research papers but no large-scale implementations. 

Ta, 

Matt Clarke 

--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/l5wgmKu03sE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/ZCbgacC1YnEyfjRX%40greymatter.

[Michal Kuratczyk]

Have a look at https://blog.rabbitmq.com/posts/2023/03/native-mqtt/

Massive improvements coming to MQTT in 3.12 (very soon, beta versions available already).

Best,

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/CAA9hvT5U1YTOhrCsriSQ8AErM_4u3O3DTL9TDdFzM2bqVb3enQ%40mail.gmail.com.

--

Michał

RabbitMQ team

[Matthew Clarke]

Hey, 

Thanks for the response. 

I think we might need some sort of bridge service. Unsure at this stage. The challenge will be that the external service is expecting a specific XML payload via HTTPs then will asynchronously send a response back once it's processed the requests. 

But I don't know how RabbitMQ would handle sending or consuming a message from the service if the external service isn't aware of RabbitMQ and that external services has neither a consumer nor publisher. A bridge could exist that would send and receive the messages from the external services but that would require HTTP pooling and defeat the purpose of having a message broker, to begin with.

Ideally: 

1. Internal web service sends a request to RMQ
2. The message broker then sends the request to the external service 
3. External service acknowledges the request 
4. External services sends back the response to RMQ when completed 
5. RMQ updates the internal web service and any other services that need to be aware of the response

It might be down to a lack of experience with RMQ  and how we can implement it. Or it may not be practical given the external service constraint. 

Matt 

You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/l5wgmKu03sE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/CAHC35TBD-4O1eWCht2Zs9TxxFmA%3DAVHHr7QCnPXYe67gFeK9kw%40mail.gmail.com.