PSR Idea: Messaging API

[Márk Sági-Kazár]

Hi all,

For quite a few years now, the world goes towards distributed systems. It became very important for these systems to communicate with each other in an efficient way.

Also, with domain driven design, CQS, CQRS, etc it became important to exchange messages within the application as well.

Whether the message's destination is another software component or another external service, it doesn't matter.

Therefore, I am suggesting to create a PSR for such messaging API.

Java's JMS is quite impressing from my point of view: https://en.wikipedia.org/wiki/Java_Message_Service
(although I am not sure whether it can easily be adopted to PHP)

Existing PHP examples for some kind of messaging:
Bernard also does some sort of messaging: https://github.com/bernardphp/bernard
Simple Bus: https://github.com/SimpleBus/
Prooph: https://github.com/prooph

What do you think? I have a fear that this topic is too big to be standardized, so I would really love to hear any ideas on this topic.

Kind regards,
Mark

[Larry Garfield]

My initial thought is that "message passing" is indeed too big of a
topic for a single standard to address. It's too general, much like
"DI" or "ORM". A specific, focused subset of messages, maybe. Eg,
would queue services be standardizable? (Common interface for
Beanstalk, Amazon SQS, *MQ, etc.) Not being an expert in that area I've
no idea, but that's more the level that a good PSR would work, IMO.

Focus on "as a consumer of some other service, can I abstract how I talk
to that service?"


--
--Larry Garfield

[Maksim Kotlyar]

We at FormaPro open sourced our messaging solution. It is based on JMS like interfaces https://github.com/formapro/fms. We tried to keep it as close as possible to JMS. Could be used as foundation for the standard. 

there are two implementations: https://github.com/formapro/amqp-ext-transport, https://github.com/formapro/stomp-transport

Here's the doc: https://github.com/formapro/message-queue-dev/blob/master/docs/index.md

[Maksim Kotlyar]

I’d like to draw attention to this topic again. 

Current status:

We all know that PHP’s performance leaves much to be desired because of its sync nature. PHP does not have a convenient and easy to use tool to do async\parallel programming. 

The messaging API could help us here and opens new horizons for all of us PHP developers. 

There is another problem: Nowadays everybody is building its own messaging library which is not compatible with others. The community is divided, loses the pace. 

No way to build truly interoperable solutions which rely on messaging. 

Solution:

I suggest starting with Java Message Service (JMS) specification. It was designed well and what is more important time-tested.

Here’s the implementation of the spec https://github.com/php-enqueue/psr-queue, we tried to be as close to it as it's possible. 

Here's a bunch of transports implemented (such as  AMQP, STOMP, Redis, Amazon SQS, Filesystem, Doctrine DBAL, Null)

I am looking forward to discussing it.

If you are a voting member and you think it worth it please consider supporting me (become a PSR sponsor).

Examples:

We’ll be able to build solutions, libraries based on it which is much more efficient. 

Here are a few examples of what we could do (mostly from Symfony world): 

1. LiipImagineBundle now able to process images in the background
2. The extension to FOSElasticaBundle which improves the populate command performance. 
3. Async events 
4. RPC over MQ
5. All sorts of message buses and application communications.
6. Doctrine extensions that work asynchronously. Like data audit. 
   
7. Distributed job scheduling library. 

Thanks.

[kon...@codeliner.ws]

Hi,

I'm one of the core maintainers of the prooph project. 

We've defined a common Message interface https://github.com/prooph/common/blob/master/src/Messaging/Message.php

that acts as the basic building block for all our packages. We tried to keep the interface as simple as possible.

As you can see prooph messages support the architectural patterns CQRS and Event Sourcing (our main focus) by defining a message type

and a message name.

Type is important because commands and queries should only have one handler/consumer whereby events can have many listeners (fanout).

Message name is mainly used for routing and to describe the intent of the message (DDD / Event Sourcing).

Our messages are immutable similar to what PSR-7 defines for Req / Res.

Furthermore messages have:

- a uuid (could be generalized to be only a string or we need a PSR for UUIDs ;) )

- metadata (custom key => value pairs)

- payload (application data)

- createdAt (a \DateTimeImmutable because it includes micro seconds)

IMHO a messaging API PSR is too much but a Message PSR (like PSR-7 is) would be nice.

Http req / res are very similar to other types of messages and in fact Zend promotes this for years:

stdlib MessageInterface: https://github.com/zendframework/zend-stdlib/blob/master/src/MessageInterface.php

http messages implement stdlib: https://github.com/zendframework/zend-http/blob/master/src/AbstractMessage.php 

We have PSR-7 (which is a good thing) but we have no general PSR for messages. If we would have such a PSR then

libraries dealing with messaging could build their logic on top of it. 

For example prooph messaging system integrations rely on prooph's message interface. We have integrations for Bernard, amqp, ZeroMQ, AWS SQS (not open source)

and HTTP for inter process communication and the same message interface is used to route messages within an application, let event sourced aggregates record domain events

which are then stored in an event store and pulled by projections.

All these use cases are supported by this "simple" interface and it did not change much over the last 4 years. 

I'm not saying it is a perfect foundation for a PSR (f.e. we rely on Ramsey\Uuid) but I think a message interface defined by a PSR should not be more complex than the prooph message interface. 

Everything else can be designed around it (like done with middleware PSR-15)

Best regads,

Alex

[Maksim Kotlyar]

The message interface is not enough. We have to standardize a way we send and receive messages as well. 

The idea is to provide libraries\frameworks with a set of interfaces to send and to process messages so they can build logic on top of them without relying on any particular MQ library\transport. I post some working example on what it could look like (see links in the previous message). 

The standard way should not be too abstract or to specific. The message may have an id but we should not force it to be UUID. The message type does not look good too. It is a broker job to deliver a message. What you need is a topic\queue name to send a message too. 

Also, there must be a way to use transport specific features (AMQP has a lot of extra features in its spec).

JMS address all those things. It is matured and widely adopted standard. Why shouldn't we stick to it instead of reinventing everything from scratch? 

[Maksim Kotlyar]

Here's one more example: A MQ based spool for Swiftmailer.

example: https://github.com/php-enqueue/enqueue-sandbox/blob/master/swiftmailer/send.php

spool implementation https://github.com/php-enqueue/enqueue-sandbox/blob/master/swiftmailer/QueueSpool.php

On Tuesday, May 23, 2017 at 1:54:17 PM UTC+3, kon...@codeliner.ws wrote:

[Matthew Kaufman]

Finally..

--
You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+unsubscribe@googlegroups.com.
To post to this group, send email to php...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/1b5e09e4-546f-4a83-9617-9cf35641033a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Maksim Kotlyar]

One more example. The formapro/pvm (it could be used to build bpmn or workflow processes) library uses MQ interfaces to execute tasks in parallel.

https://github.com/formapro/pvm/blob/master/docs/parallel-execution-with-enqueue.md

[Maksim Kotlyar]

I moved MQ interfaces out of Enqueue project to a separate repository https://github.com/queue-interop/queue-interop

That's Java JMS based interface.

Everybody is welcome to join the movement. 

On Monday, March 28, 2016 at 12:53:16 PM UTC+3, Márk Sági-Kazár wrote: