out-of-the-box support for json serialization for akka persistence?

[Tim Pigden]

Hi

Reading 

http://doc.akka.io/docs/akka/2.4.10/scala/persistence-schema-evolution.html

and thinking about what we currently do, I've concluded that I'd like to serialize my events as json.

The article suggests I should check the list of plugins to see which specifically support json. But I can't find clear indications that any of them do from the respective github pages (apologies if I've missed something). Most seem to want the target database tables to be a BLOB with no option of a native json type.

My current database target is Postgres (because we use postgis for mapping and because I need a proper sql database for data warehouse). However, for the event serialization I could certainly contemplate something else if it made the current development path quicker and easier. MongoDB or Cassandra are obvious possible targets.

So my question is: is anyone actually using json as a serialization format, and if so

a) what database and driver?

b) is the database-native json support being used (if any?)

c) how much effort was it? I can already serialize everthing to json that needs to go in the persistence journal or snapshots.

it's got to be something with active support/interest.

Thanks

tim

[Justin du coeur]

A few general reactions (personally, I'm using Kryo over Cassandra, so I can't address the direct questions):

Keep in mind that json is a relatively bulky serialization format -- it's going to take rather more disk space than more typical options like protobuf.

In *general*, the serialization format and database are pretty orthogonal concerns.  It's possible that json might have specific advantages when used with a json-native DB like Mongo, but AFAICT the Cassandra driver is the most mature in general.  Are you planning on using the json-ness in some particular way?

Finally, remember that json doesn't save you from all the complications of schema evolution: you're still going to need to think about how you deal with missing fields, name changes, and stuff like that.  It's probably a bit easier than Kryo, but only a bit.  Schema evolution is a real bear -- making sure I can deal with it took most of my effort in getting Persistence up and running...

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscribe@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[Tim Pigden]

Hi Justin

Thanks for the comments. However, as I said I have decided that I do want to use json asa format, so I'm really looking to see who has direct experience of that. 

The serialization format is less orthogonal that you might expect, given that the database table layouts are all part of the plugin and as far as I can see, all use blob rather than json format with no obvious way to change that.

And yes the json-ness does make a difference to me. Without going into elaborate CQRS "Q" coding, it will enable the developers to find out about how the system is being used, what sort of messages are being created with what payloads, through simple sql. My increased data volumes are extremely unlikely to burden a single server db in the short to medium term.

Tim

To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.

[Patrik Nordwall]

For Lagom I wrote json serializer that is using Jackson. You can look at it for inspiration. It's stored in Cassandra as bytes, but how it's stored is less important as far as I can see.

/Patrik

[Jan-Pieter van den Heuvel]

Hi Tim,

You might also have a look at Stamina. Keep in mind that this only has support for spray-json.

Kind regards,
Jan-Pieter

Op woensdag 21 september 2016 22:02:47 UTC+2 schreef Patrik Nordwall:

[Morten Kjetland]

Hi,

We have been using Akka Persistence with Json-serialization (Jackson) in production for almost two years now, and it has been a great success.

Since we currently have more than 10 different applications using this technique, we build some common utilities and addons that we are using.

We opensources this code a year ago: https://github.com/NextGenTel/akka-tools

It contains akka-tools-json-serializing which is a akka-serializer using Jackson. In addition to the normal easy json-schema-evolving when adding new properties, it also has build in support for breaking schema-changes (we have used this in also production).

At the time we started, there was no other jdbc-journal-plugin, so we wrote our own: akka-tools-jdbc-journal - We have only used it with Oracle and H2, but I guess it would also work with Postgres - If not, I would be happy to help you make it work.

When using our akka-tools-jdbc-journal in combination with akka-tools-json-serializing you get a "killer feature" :) .. We write the json as plain-text in an extra write-only-column.

This way you can do a manual select from the journal-table and see for your own what the events looks like.

-Morten