Protocol Buffers for events?

1,188 views
Skip to first unread message

Preston Marshall

unread,
Nov 4, 2016, 4:38:19 PM11/4/16
to Event Store
I'm considering using Protocol Buffers for the actual events themselves, for a number of reasons. Is anyone doing this currently? Any reason not to use them for events?

Thanks

Greg Young

unread,
Nov 4, 2016, 4:53:33 PM11/4/16
to event...@googlegroups.com
eventstore supports it.

A few things will not work.

1) Stream browsing UI (well it kind of will but is essentially useless)
2) Projections (they will not know how to read your data

In general though storage etc works without any issue.
> --
> You received this message because you are subscribed to the Google Groups
> "Event Store" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to event-store...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Studying for the Turing test

Preston Marshall

unread,
Nov 4, 2016, 6:51:11 PM11/4/16
to Event Store
Thanks, I figured it would support it since I saw the application/octet-stream content-type stuff. I am not planning on using projections at this time, but I assume I could probably use protobuf.js if I really wanted to down the road. Have you heard of anyone else using protocol buffers for their events? I think it has some advantages but I am wary of swimming against the stream.

Thanks,
Preston

Greg Young

unread,
Nov 4, 2016, 6:52:54 PM11/4/16
to event...@googlegroups.com
You could feasibly do it, there is a raw version of data that can be provided.

ahjohannessen

unread,
Nov 5, 2016, 2:50:55 PM11/5/16
to Event Store
We use protobuf for our events, a bit irritating sometimes that we cannot browse data, but overall we would still pick protobuf over json. 
There is always the possibility of changing to different serialization by migrating streams in a cluster to a new cluster that changes the serialization.

Greg Young

unread,
Nov 5, 2016, 3:01:19 PM11/5/16
to event...@googlegroups.com
There isn't really a good way to support being able to view your
events as we don't have schemas for them

ahjohannessen

unread,
Nov 5, 2016, 3:15:34 PM11/5/16
to Event Store
Yeah, I know. Anyway, we only use the awesome by_category projection which is a dealbreaker! :)
With a little of work we probably could create an endpoint that did unmarshalling wrt. protobuf->json, 
however most of our data ends in PostgreSQL or similar anyway so our need is not critical.

ahjohannessen

unread,
Nov 5, 2016, 3:19:31 PM11/5/16
to Event Store
Greg, would it be feasible to introduce some kind of plugin system wrt. the UI of Eventstore a la
ES -> serialization plugin -> ES UI wrt. being able to show data?


On Saturday, November 5, 2016 at 7:01:19 PM UTC, Greg Young wrote:

Greg Young

unread,
Nov 5, 2016, 5:47:33 PM11/5/16
to event...@googlegroups.com
It is possible... PR? :) If you want I can show you where to look.
Basically its a plugin that would convert protobufs etc to json which
would work in place. The one difficult thing would be supporting the
varying mime types as it would need a hook as well

Fredrik Skeel Løkke

unread,
Nov 18, 2016, 5:18:48 PM11/18/16
to Event Store
We are doing it and are quite happy with it. In our context it gives more flexibility e.g. renaming of event properties with no consequences but improved clarity. It's about 4 times faster consuming protobuffed streams than json so it enables some scenarios that would be out of reach if json was used. On a different note why would you spend cpu cycles parsing text when its completely unnecesary..
Reply all
Reply to author
Forward
0 new messages