Akka HTTP for microservice orchestration?

[Marc Schlegel]

Hello everyone

I would like to know if Akka Http is suitable to implement a small orchestration layer.
The idea is to send events from the microservice to the orchestration layer (Akka with Akka Http). I think Akka is a nice fit for orchestration but I am not sure if Akka Http is the best solution to work with external systems.

Note: the events are implemented by Google Protocolbuffers because my demo-projects implements the services in several languages (Go, Java, Rust) and by using protobuf I can generate consistent DTOs. I've first looked into Alpakka but it seems this has a different focus and currently no official support for protobuf messages.

regards
Marc

[Alan Johnson]

I'm not exactly sure what you mean by an orchestration layer (I've heard a lot of different schemes called this), especially one that receives events, because that makes it sound stateful. But Akka HTTP makes a totally reasonable external API layer, especially for services, as opposed to rendering content. 

--
*****************************************************************************************************
** New discussion forum: https://discuss.akka.io/ replacing akka-user google-group soon.
** This group will soon be put into read-only mode, and replaced by discuss.akka.io
** More details: https://akka.io/blog/news/2018/03/13/discuss.akka.io-announced
*****************************************************************************************************
>>>>>>>>>>
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[Marc Schlegel]

Yeah microservices and orchestration are bloody buzzwords :-)

As said, my sample is rather simple, but it should be enough to experience the pain-points of microservices (distributed systems). And instead of having my services communicate directly to each other they just raise events which get send to the orchestration-service which will also be implemented in several styles (Akka and the other planned solution is BPMN saga with Camunda).
Once those are done, I am planning to implement everything using plain Akka (each service its own actor-system) to get a comparison.

Thanks for your confirmation. I thought that I read somewhere that Akka Http was only intended for internal communication within actor-systems and it should not be used directly...but obviously I was mistaken.

[Alan Burlison]

I've prototyped something that sounds a little similar to this and it
works well. I have a controller actor running on one node that maintains
a list of pending work. Worker actors are started on other nodes and
connect to the controller actor to request work which they then process.
The worker actors are just thin wrappers around instances of a large,
legacy C++ application which is in effect a batch process. The worker
nodes create a worker actor for each instance of the legacy app, plus
there is a shared localhost-only Akka HTTP server actor to provide a
protobuf endpoint for the C++ app instances on the worker node to talk
to. The C++ app has been modified to use libcurl & protobuf to connect
to the localhost HTTP server to request work details and provide
completion information which is 'proxied' back by the worker actors to
the controller actor on the controller node.

All the external communication is done with protobuf, both between the
akka actors on the master and worker nodes and between the worker actors
and the legacy C++ app, via the localhost HTTP actor. I've used akka
remoting for the cross-node control rather than akka cluster as if the
controller node dies there is nothing further that can be done anyway
and the worker actors just kill any running C++ app instances and exit.
Workers can be added dynamically and will be issued pending work and if
they die the controller will reschedule the work they were doing to
another worker.

This all works nicely and protobuf integrates well into Akka HTTP, with
the addition of a tiny amount of glue code to to the protobuf
marshalling for akka HTTP - see
https://github.com/scalapb/ScalaPB/issues/247 for the details. With that
in place, the routing logic for processing looks similar to this
outline, where JobFinishedMsg is a protobuf message:

path("job_finished") {
post {
entity(as[JobFinishedMsg])(msg => {
log.debug("received message {}", msg.id)
???
}
}
}

--
Alan Burlison
--

[Alan Burlison]

On 13/04/18 12:28, Marc Schlegel wrote:

> Thanks for your confirmation. I thought that I read somewhere that Akka
> Http was only intended for internal communication within actor-systems and
> it should not be used directly...but obviously I was mistaken.

I don't believe so, I've used it to implement a fully-blown REST
endpoint for an application.

--
Alan Burlison
--

[Konrad “ktoso” Malawski]

Just to confirm

> Thanks for your confirmation. I thought that I read somewhere that Akka Http was only intended for internal communication within actor-systems and it should not be used directly...but obviously I was mistaken.

Perhaps it was talking about Akka *actor* messaging — which yes indeed, is meant for “within a service” communication.

Akka HTTP absolutely is built for talking to other services or even publicly exposing endpoints.

-- 

Cheers,

Konrad 'ktoso' Malawski

Akka @ Lightbend

[Marc Schlegel]

Great. Thanks everyone.