Deleting Specific Message in Akka-Persistence

[Harit Himanshu]

Hello there,

I am working on akka-persistence and my use case looks like following

• Jobs are submitted to PersistentActor, which are then journaled to leveldb. I also maintain a state variable (as shown in the akka docs)
• There are worker actors who work on these jobs.
• Once worker completes the job successfully, I would like to remove the job from leveldb and update the state variable.

I looked at the message deletion section, which has deleteMessages(toSequenceNr) API, but my confusion is that I do not know the toSequenceNr. Infact, there may be many workers working on different jobs at any time, and I want to remove a specific job which is completed successfully.

Once the message is deleted, I am ideally making this journal bounded (otherwise it will increases forever). In this case, snapshots will be smaller as well.

How can I achieve this use case? Or am I not doing things right?

My current code looks like

import akka.actor.{Props, ActorSystem, ActorLogging}

import akka.event.LoggingReceive

import akka.persistence.{SaveSnapshotFailure, SaveSnapshotSuccess, SnapshotOffer, PersistentActor}

case class Command(data: String)

case class Event(data: String)

case object InternalState

case object TakeSnapshot

case object ShutDown

case object Fail

case class State(queue: List[String] = Nil) {

  def updated(event: Event): State = copy(event.data :: queue)

  def size: Int = queue.length

  override def toString: String = queue.reverse.toString

}

class PersistentSnapshotActor extends PersistentActor with ActorLogging {

  override def persistenceId = "snapshot-persistence-id"

  var state = State()

  def updateState(event: Event) = state = state.updated(event)

  def numberOfEvents = state.size

  def receiveRecover = LoggingReceive {

    case event: Event => updateState(event)

    case SnapshotOffer(_, snapshot: State) =>

      log.debug(s"offered state: $snapshot")

      state = snapshot

  }

  def receiveCommand = LoggingReceive {

    case Command(data) => persist(Event(data))(updateState)

    case Fail => throw new Exception("killing persistent actor.")

    case ShutDown => context.stop(self)

    case InternalState => println(state)

    case TakeSnapshot => saveSnapshot(state)

      delete

    case SaveSnapshotSuccess => log.debug("snapshot saved successfully.")

    case SaveSnapshotFailure(_, reason) => log.error(s"failed to save snapshot: $reason")

  }

}

object PersistentSnapshotActorApp extends App {

  val system = ActorSystem("snapshotSystem")

  val persistentActor = system.actorOf(Props[PersistentSnapshotActor], "persistentSnapshotActor")

  persistentActor ! Command("1")

  persistentActor ! Command("2")

  persistentActor ! Command("3")

  persistentActor ! Command("4")

  persistentActor ! TakeSnapshot

  persistentActor ! InternalState

  persistentActor ! Command("5")

  persistentActor ! TakeSnapshot

  persistentActor ! Fail

  persistentActor ! InternalState

}

Thanks a lot

[Martynas Mickevičius]

Hi,

On Sat, Nov 28, 2015 at 1:42 AM, Harit Himanshu <harit.sub...@gmail.com> wrote:

Hello there,

I am working on akka-persistence and my use case looks like following

• Jobs are submitted to PersistentActor, which are then journaled to leveldb. I also maintain a state variable (as shown in the akka docs)
• There are worker actors who work on these jobs.
• Once worker completes the job successfully, I would like to remove the job from leveldb and update the state variable.

You will not be able to recover the state if your persistent actor crashes just after the state update. What is more, it seems that kafka storage backend is better suited for your usecase, which has a configurable retention time.

 

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--

Martynas Mickevičius

Typesafe – Reactive Apps on the JVM

[Tim Pigden]

Hi

Not sure if this is still a problem for you. 

Faced with a similar type of problem I create persistent actors per "job" and have a master which keeps the state of all.

This means that on recover, I quickly can see which of my jobs were completed and which not. So my "master" would have a structure something like

case clas Job(jobId, status)

var jobs = Set[Job]

Events on master would be job started, job completed, job deleted.

When recovering jobs you then restart the actors for the jobs that were not completed.

The persistent job actor would just maintain the individual job state and there would be another actor to actually run the job. When you want to get rid of it you can delete the entire journal for that job and have the master know that it's been deleted.

So master is technically unbounded but it's also small, so you probably wouldn't need to do much to keep size under control. You might also find it's worth keeping permanently if your status included information about the success or failure of jobs (rather depends on what the jobs are). However it's also easy to control its load time snapshot etc.