Akka stream is idling with merged ActorPublishers

[Jakub Liska]

Hey,

I have a Source that merges ActorPublishers, this is a simplification : 

Source[Foo]() { implicit b =>
        val actorSources = myActorPublisherArray
        b.add(Merge[Foo](actorSources.length))
        for (i <- 0 until actorSources.length) {
          b.addEdge(b.add(actorSources(i)), merge.in(i))
        }
        merge.out
      }
    }

class MyActorPublisher extends ActorPublisher[Foo] {
  var batchIterator = someBlockingIterator


  
  def pushNext: Boolean = {
        if (batchIterator.hasNext) {

      val nextMnfResource = batchIterator.next()
      onNext(nextMnfResource)
      true
    } else {
      onCompleteThenStop()
      false
    }
  }
  def receive: Receive = {
    case Request(n) if totalDemand > 0 && isActive =>
      (1L to Math.min(n, totalDemand)).foldLeft(true) {
        case (acc,i) => if (acc) pushNext else false
      }
    case Cancel =>
      context.stop(self)
  }
}



But the stream is idling most of the time with like 1% CPU usage like there was no `pull demand` for some reason... I'm trying to reproduce it and find the catch for 2 days already. 

Any idea why that could be? I think it started with RC1 or RC2 when I set Flow#mapAsync(parallelism) as required in following Flows, I removed all custom "Stages" and I don't really do anything except Map and MapAsync in flows...

Also it doesn't seem to be happening with 

      .withInputBuffer (
        initialSize = 1,
        maxSize = 1
      )

But then the throughtput is insufficient... 

[Endre Varga]

Hi Jakub,

On Fri, May 29, 2015 at 11:36 AM, Jakub Liska <liska...@gmail.com> wrote:

Hey,

I have a Source that merges ActorPublishers, this is a simplification : 

Source[Foo]() { implicit b =>
        val actorSources = myActorPublisherArray
        b.add(Merge[Foo](actorSources.length))
        for (i <- 0 until actorSources.length) {
          b.addEdge(b.add(actorSources(i)), merge.in(i))
        }
        merge.out
      }
    }

class MyActorPublisher extends ActorPublisher[Foo] {
  var batchIterator = someBlockingIterator


  
  def pushNext: Boolean = {
        if (batchIterator.hasNext) {

      val nextMnfResource = batchIterator.next()
      onNext(nextMnfResource)
      true
    } else {
      onCompleteThenStop()
      false
    }
  }
  def receive: Receive = {
    case Request(n) if totalDemand > 0 && isActive =>
      (1L to Math.min(n, totalDemand)).foldLeft(true) {

What is the above line intending to do? Why are you taking a minimum of n and totalDemand? Why are you not using totalDemand directly?

 


        case (acc,i) => if (acc) pushNext else false
      }
    case Cancel =>
      context.stop(self)
  }
}



But the stream is idling most of the time with like 1% CPU usage like there was no `pull demand` for some reason... I'm trying to reproduce it and find the catch for 2 days already. 

Any idea why that could be? I think it started with RC1 or RC2 when I set Flow#mapAsync(parallelism) as required in following Flows, I removed all custom "Stages" and I don't really do anything except Map and MapAsync in flows...

Also it doesn't seem to be happening with 

      .withInputBuffer (
        initialSize = 1,
        maxSize = 1
      )

But then the throughtput is insufficient... 

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[Jakub Liska]

  def receive: Receive = {
    case Request(n) if totalDemand > 0 && isActive =>
      (1L to Math.min(n, totalDemand)).foldLeft(true) {




 

What is the above line intending to do? Why are you taking a minimum of n and totalDemand? Why are you not using totalDemand directly?

 

Well I couldn't really say how many elements is actually demanded, whether "n" or "totalDemand" ... "Request(n)" comes saying push me "n" elements, right? That is the actual demand.

[Jakub Liska]

The documentation says : 

totalNumber : 

Total number of requested elements from the stream subscriber.
This actor automatically keeps tracks of this amount based on
incoming request messages and outgoing `onNext`.

n : 

n number of requested elements

 

As I see it,  "n" represents "totalNumber"  in this context, but totalNumber is also a stateful representation for internal purposes...

[Yann Simon]

you can receive r requests with n elements each.
You do not know when the subscriber sends them

The total number of request is tracked in totalNumber.

This is definitely what you should use.

--

[Jakub Liska]

The total number of request is tracked in totalNumber.

You mean "total number of requests"? Like "total number of requested elements that may come in x requests" ?

Anyway I changed it to : 

    case Request(n) if totalDemand > 0 && isActive =>

      (1L to totalDemand).foldLeft(true) {

        case (acc,i) => if (acc) pushNext else false
      }

And it is still idling ...

[Jakub Liska]

I got it, it wasn't related to akka-stream but to one of the services on a particular host the stream is communicating with, it's throughtput decreased from day to day about 95% and the entire stream depended on it so it decreased about 95% too... 

It's pretty hard to detect such state, it'd really need some built-in profiling mechanism https://groups.google.com/forum/#!topic/akka-user/emBAWO2rxsU

[Endre Varga]

HI Jakub,

Please file a ticket for providing a Flow in the library that kills off the stream if observed throughput drops below a threshold for a prolonged time, or alternatively, logs it. It should properly distinguish between idle periods from reduced throughput.

-Endre

On Fri, May 29, 2015 at 3:27 PM, Jakub Liska <liska...@gmail.com> wrote:

I got it, it wasn't related to akka-stream but to one of the services on a particular host the stream is communicating with, it's throughtput decreased from day to day about 95% and the entire stream depended on it so it decreased about 95% too... 

It's pretty hard to detect such state, it'd really need some built-in profiling mechanism https://groups.google.com/forum/#!topic/akka-user/emBAWO2rxsU

--

[Jakub Liska]

Done, thank you  https://github.com/akka/akka/issues/17614