How do you browse contents of your akka-persistence journals?

[Magnus Andersson]

Hi

I'm looking for advice/experiences for browsing and maintaining the journals akka-persistence writes. The events are serialized as binary blobs of whatever serialization mechanism akka is configured to use. The journals are not human readable (it is possible to with some detective work, but still not easy). 

I gave a custom JSON serializer with Pickling a try. It became a clunky implementation so I abandoned it, but kind of worked so that I could browse data. Of course I could build an application specific tool myself that can show and query for our events, but I'm wondering how other projects have tackled this? 

I'm even starting to consider hacking at a journal plugin that store events serialized as JSON in a clob+type column instead of a blob.

/Magnus

[Konrad Malawski]

Hi Magnus,

the answer to this question is very journal dependent.

It's up to the Journal to pick how to serialize your messages.

If a journal picks to serialize in human-readable formats you'd get that.

For example in a database like mongo you could just pick a JSON serializer to put it into the document, in databases with strict schema it's not as easy but certainly doable - perhaps as additional human readable format stored (as discussed below). 

So in one sentence: it's up to persistence plugins to support this use case, we don't directly control how and where-to the serialization is performed by the plugins. The plugin knows it's datastore and could do human-readable things.

On the other hand, you can use PersistentViews to use the same infra to deserializa and emit a stream of human readable projectsions if you need to.

There was an interesting discussion on this recently on the cassandra plugin repo:

https://github.com/krasserm/akka-persistence-cassandra/issues/50

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--

Cheers,

Konrad 'ktoso' Malawski

Akka @ Typesafe

[Magnus Andersson]

Hi Konrad

I was the one who asked that question. :)

As you say there is a possibility to hook up persistent views to a tool you build. Akka analytics and spark sounds intetesting as suggested in the github thread.

Anyhow, it is an important piece of the tooling puzzle to be able to view journaled data "out of band". If anyone else have experience to share I would love to hear about it.

How did you solve it and how did it tun out.

You received this message because you are subscribed to a topic in the Google Groups "Akka User List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/akka-user/enEQ9tde70Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to akka-user+...@googlegroups.com.

[delasoul]

There was discusion about this also for the mongo plugin:

https://github.com/ironfish/akka-persistence-mongo/issues/88

michael

[Magnus Andersson]

Michael: Thanks. Interesting.

To me it looks like the most general solution would be a tool that could leverage the existing akka persistent views and use the configured (de)serialization and get case classes and display those in an UI. Have not yet done a deep dive into spark so that could also be an alternative.

/Magnus