Help with Akka persistence and dead letters

[mat...@wondr.it]

Hi, I'm trying to write a resilient photo uploading system using Akka persistence, but I'm having some issues.

I have a supervisor called PhotoUploadSupervisor that spawns a child actor when I need to upload a photo.

class PhotoUploadSupervisor extends Actor with ActorLogging {

  import context._

  def newChild = actorOf(Props[PhotoUploadActor])

  override val supervisorStrategy =

    OneForOneStrategy(maxNrOfRetries = 10, withinTimeRange = 1 minute) {

      case _: IOException => Restart

      case _ => Escalate

    }

  def receive = {

    case postPhoto: POSTPhoto => newChild ! UploadPhoto(postPhoto)

  }

}

POSTPhoto is a simple case class:

case class POSTPhoto(

    blob: String,

) {

  require(blob.nonEmpty)

}

This is the actor:

sealed trait Command

case class UploadPhoto(data: POSTPhoto) extends Command

sealed trait Event

case class StateChanged(data: POSTPhoto) extends Event

class PhotoUploadActor extends PersistentActor with ActorLogging {

  override def persistenceId = "photoUploadActor"

  var state = POSTPhoto("fakeBlob")

  def updateState(evt: StateChanged) = {

    state = POSTPhoto(evt.data.blob)

  }

  val receiveRecover: Receive = {

    case evt: StateChanged =>

      updateState(evt)

      uploadToS3()

  }

  val receiveCommand: Receive = {

    case UploadPhoto(data) =>

      persist(StateChanged(data)) { changed =>

        updateState(changed)

        uploadToS3()

      }

  }

  private def uploadToS3() = {

    println(f"Uploading photo in actor ${self.path}")

    /*Photo upload code goes here*/

    /*It might throw an IOException*/

    context.stop(self)

  }

}

I want the actor to upload the photo and then stop.

The supervisor should spawn one child per photo upload request.

It looks like it's working, but I get a bunch of dead letters:

[INFO] [09/23/2014 13:33:21.587] [on-spray-can-akka.actor.default-dispatcher-2] [akka://on-spray-can/user/$a/$a] Message [akka.persistence.JournalProtocol$ReplayedMessage] from Actor[akka://on-spray-can/user/$a#-494389522] to Actor[akka://on-spray-can/user/$a/$a#1751213071] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.

[INFO] [09/23/2014 13:33:21.587] [on-spray-can-akka.actor.default-dispatcher-2] [akka://on-spray-can/user/$a/$a] Message [akka.persistence.JournalProtocol$ReplayedMessage] from Actor[akka://on-spray-can/user/$a#-494389522] to Actor[akka://on-spray-can/user/$a/$a#1751213071] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.

[INFO] [09/23/2014 13:33:21.587] [on-spray-can-akka.actor.default-dispatcher-2] [akka://on-spray-can/user/$a/$a] Message [akka.persistence.JournalProtocol$ReplayedMessage] from Actor[akka://on-spray-can/user/$a#644188776] to Actor[akka://on-spray-can/user/$a/$a#1751213071] was not delivered. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.

[INFO] [09/23/2014 13:33:21.587] [on-spray-can-akka.actor.default-dispatcher-2] [akka://on-spray-can/user/$a/$a] Message [akka.persistence.JournalProtocol$ReplayedMessage] from Actor[akka://on-spray-can/user/$a#1549609203] to Actor[akka://on-spray-can/user/$a/$a#1751213071] was not delivered. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.

[INFO] [09/23/2014 13:33:21.587] [on-spray-can-akka.actor.default-dispatcher-2] [akka://on-spray-can/user/$a/$a] Message [akka.persistence.JournalProtocol$ReplayedMessage] from Actor[akka://on-spray-can/user/$a#1549609203] to Actor[akka://on-spray-can/user/$a/$a#1751213071] was not delivered. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.

[INFO] [09/23/2014 13:33:21.587] [on-spray-can-akka.actor.default-dispatcher-2] [akka://on-spray-can/user/$a/$a] Message [akka.persistence.JournalProtocol$ReplayMessagesSuccess$] from Actor[akka://on-spray-can/system/journal#-343657892] to Actor[akka://on-spray-can/user/$a/$a#1751213071] was not delivered. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.

Messages "ReplayedMessage" and "ReplayedMessagesSuccess" can't be delivered apparently!

Is that an issue or not?

Thanks!

M.

[Jeroen Gordijn]

Hi M.,

Your persistenceId should be unique per request or per photo. There should only be one instance of an actor holding the same persistenceId. Otherwise there are multiple threads writing and reading from the same store. Every time you instantiate a new actor it will replay the journal and resend the first photo. After that, the actor stops and all other messages go to dead letters.

What are your requirements? What do you want to make resilient. Your current direction you're heading won't make it resilient, but it will store every request you receive.

BTW: receiveRecover should have no side effects. You are sending to S3 on every recovery.

Cheers,

Jeroen

Op dinsdag 23 september 2014 14:37:04 UTC+2 schreef mat...@wondr.it: