[akka-streams] ActorSubscriber to ActorPublisher flow -- how to?

[Alexey Shuksto]

Hello there,

I could not find answer to this particular question neither in docs nor in this group:

Is it possible to wire Sink.actorSubscriber(...) into Source.actorPublisher(...) in one (Partial-) Flow? And by 'wire' I mean to somehow send message from ActorSubscriber, instantiated from given props, to the 'same flow' instance of ActorPublisher?

I was thinking about giving some unique Id to both of them and then publish/subscribe events through `context.system.eventStream`, but then I realized that all materialized flows would have same Id and messages would mess up... :(

In fact, I'm interested in even more simple task -- create an Actor, that could act as PartialFlow[In, Out]: when you push a value into flow, actor receives it, processes and sends further down the line.

Of course, I always could do something like Flow[In] mapAsync (data => (actor ? data).mapTo[classOf[Out]]) -- but I would really, really prefer to do this message way.

[Konrad Malawski]

Hello there Alexey,

I'm not sure I understand what you want to achieve, is it simply to connect such Sink to the Source?

That's simply: Source.actorPublisher(...).to(Sink.actorSubscriber(...)).run()

Or do you mean to put another processing stage between them?

If so then read about custom processing stages: doc.akka.io/docs/akka-stream-and-http-experimental/1.0/scala/stream-customize.html

-- 

Cheers,
Konrad Malawski

Akka @ Typesafe

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[Alex Shuksto]

Well, I want to simply connect them, but I want to connect Sink.actorSubscriber(...) ~> Source.actorPublisher(...), not other way around, like you said.

I've got Actor, that enriches HTTP Requests that our server receives. After some enrichment, I need to be able to send request further down for processing:

```

val source = Source.single(/*HttpRequest*/)

val sink = Sink.head[ProcessedRequest]

val enrichment: Flow[HttpRequest, EnrichedRequest] = ???

val processing: Flow[EnrichedRequest, ProcessedRequest] = ???

source.via(enrichment).via(processing).to(sink)

```

I want to be able to use our enrichment Actor in Flow[HttpRequest, EnrichedRequest] stage. Actor is able to receive HttpRequest and reply (or forward) ProcessedRequest, but I don't know how to create Flow from it.

 

чт, 20 авг. 2015 г. в 18:20, Konrad Malawski <konrad....@typesafe.com>:

[Alexey Shuksto]

Well, I was able to achieve needed behavior with following flow scheme:

Flow() { implicit builder ⇒
  val promises = builder add Source(() ⇒ Iterator continually Promise[O]())

  val fan = builder add Broadcast[Promise[O]](2)
  val zip = builder add Zip[I, Promise[O]]()

  val flow = builder add Flow[Promise[O]].mapAsync(1)(_.future)
  val sink = builder add Sink.actorSubscriber[I, Promise[O]](props)

  //+Flow: @formatter:off
  promises ~> fan ~> zip.in1
  flow     <~ fan
  sink            <~ zip.out
  //-Flow: @formatter:on

  zip.in0 → flow.outlet
}

Where `I` and `O` are input and output types, respectfully, and `props` is `Props(..)` of enrichment Actor, who receives Tuple(i: I, p: Promise[O]), enriches i to instance of `O` and fulfills promise.

But there is still some questions I would very much like to know answers to:

1. Is there a way to create lazy repeated Source of (: => Promise[O])?

Logs show that Source(() => Iterator continually Promise[O]()) eagerly produces several promised values, and if actual inlet completes after very first instance of `I` they are discarded.

2. Is there a simpler way to produce Flow[O] from Flow[Promise[O]] than via `.mapAsync(1)(_.future)`?

3. Will Sink.actorSubscriber(..) shutdown underlying actor after stream is completed or it needs to be done from actor itself after OnComplete event? And what about errors?

четверг, 20 августа 2015 г., 18:33:43 UTC+3 пользователь Alexey Shuksto написал:

To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscribe@googlegroups.com.

[Patrik Nordwall]

On Fri, Aug 21, 2015 at 5:46 PM, Alexey Shuksto <sei...@gmail.com> wrote:

Well, I was able to achieve needed behavior with following flow scheme:

Flow() { implicit builder ⇒
  val promises = builder add Source(() ⇒ Iterator continually Promise[O]())

  val fan = builder add Broadcast[Promise[O]](2)
  val zip = builder add Zip[I, Promise[O]]()

  val flow = builder add Flow[Promise[O]].mapAsync(1)(_.future)
  val sink = builder add Sink.actorSubscriber[I, Promise[O]](props)

  //+Flow: @formatter:off
  promises ~> fan ~> zip.in1
  flow     <~ fan
  sink            <~ zip.out
  //-Flow: @formatter:on

  zip.in0 → flow.outlet
}

Where `I` and `O` are input and output types, respectfully, and `props` is `Props(..)` of enrichment Actor, who receives Tuple(i: I, p: Promise[O]), enriches i to instance of `O` and fulfills promise.

But there is still some questions I would very much like to know answers to:

1. Is there a way to create lazy repeated Source of (: => Promise[O])?

Logs show that Source(() => Iterator continually Promise[O]()) eagerly produces several promised values, and if actual inlet completes after very first instance of `I` they are discarded.

I think that is because elements are prefetched from the source, but that should be bounded to the buffer size.

 

2. Is there a simpler way to produce Flow[O] from Flow[Promise[O]] than via `.mapAsync(1)(_.future)`?

Isn't that simple enough?

 

3. Will Sink.actorSubscriber(..) shutdown underlying actor after stream is completed or it needs to be done from actor itself after OnComplete event? And what about errors?

ActorSubscriber doesn't (currently) stop automatically when it receives OnComplete/OnError. There are some use cases when it is needed to not stop, but I think we should change the default behaviour to stop.

There is several tickets around this: https://github.com/akka/akka/issues/17286

 

To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.

To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--

>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.

To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--

Patrik Nordwall
Typesafe -  Reactive apps on the JVM
Twitter: @patriknw

[Алексей Шуксто]

чт, 3 сент. 2015 г. в 11:38, Patrik Nordwall <patrik....@gmail.com>:

On Fri, Aug 21, 2015 at 5:46 PM, Alexey Shuksto <sei...@gmail.com> wrote:

Well, I was able to achieve needed behavior with following flow scheme:

Flow() { implicit builder ⇒
  val promises = builder add Source(() ⇒ Iterator continually Promise[O]())

  val fan = builder add Broadcast[Promise[O]](2)
  val zip = builder add Zip[I, Promise[O]]()

  val flow = builder add Flow[Promise[O]].mapAsync(1)(_.future)
  val sink = builder add Sink.actorSubscriber[I, Promise[O]](props)

  //+Flow: @formatter:off
  promises ~> fan ~> zip.in1
  flow     <~ fan
  sink            <~ zip.out
  //-Flow: @formatter:on

  zip.in0 → flow.outlet
}

Where `I` and `O` are input and output types, respectfully, and `props` is `Props(..)` of enrichment Actor, who receives Tuple(i: I, p: Promise[O]), enriches i to instance of `O` and fulfills promise.

But there is still some questions I would very much like to know answers to:

1. Is there a way to create lazy repeated Source of (: => Promise[O])?

Logs show that Source(() => Iterator continually Promise[O]()) eagerly produces several promised values, and if actual inlet completes after very first instance of `I` they are discarded.

I think that is because elements are prefetched from the source, but that should be bounded to the buffer size.

Is there a way to bound downstream buffer of Source?

Or should I add .buffer(1, OverflowStrategy.dropBuffer) to

```

val fan = builder add Broadcast[Promise[O]](2) buffer (1, OverflowStrategy.dropBuffer)

```

I've tried that, but `fan` type becomes `FlowGraph.Implicits.PortOps[Promise[O], Unit]` and graph conversions and operations no longer works.

 

 

2. Is there a simpler way to produce Flow[O] from Flow[Promise[O]] than via `.mapAsync(1)(_.future)`?

Isn't that simple enough?

Yes it is, but I was wondering if I overlooked some operation like `Future.flatMap` for flow -- as we already hold a promise and actual work potentially happens in different execution context maybe parallelism factor and work dispatching could be unnecessary.

 

 

3. Will Sink.actorSubscriber(..) shutdown underlying actor after stream is completed or it needs to be done from actor itself after OnComplete event? And what about errors?

ActorSubscriber doesn't (currently) stop automatically when it receives OnComplete/OnError. There are some use cases when it is needed to not stop, but I think we should change the default behaviour to stop.

There is several tickets around this: https://github.com/akka/akka/issues/17286

Thanks a lot!

 

You received this message because you are subscribed to a topic in the Google Groups "Akka User List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/akka-user/JyWMymHFuXs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to akka-user+...@googlegroups.com.