[ANN] Event Horizon - first realease of CQRS/ES library

[Max Persson]

Hi Gophers,

I have just released a CQRS/ES library called Event Horizon (https://github.com/looplab/eventhorizon). Be sure to check it out if you are interested in ORM libraries and other data modeling technique! CQRS/ES has helped me a lot in defining a better domain by forcing me to think in a non-data centric way. I think that is where the technique really shines.

From the readme:

 Event Horizon is a CQRS/ES toolkit for Go.

CQRS stands for Command Query Responsibility Segregation and is a technique where object access (the Query part) and modification (the Command part) are separated from each other. This helps in designing complex data models where the actions can be totally independent from the data output.

ES stands for Event Sourcing and is a technique where all events that have happened in a system are recorded, and all future actions are based on the events instead of a single data model. The main benefit of adding Event Sourcing is tracability of changes, for use in a audit log for example. Additionally "incorrect" events that happened in the past (for example due to a bug) can be changed which will make the current data "correct", as it is based on all past events.

Read more about CQRS/ES from one of the major authors/contributors on the subject: http://codebetter.com/gregyoung/2010/02/16/cqrs-task-based-uis-event-sourcing-agh/

The library is still in it's early days and I will add support for storage backings and more over the next months as I'm using it in a large scale web app on my current assignment!

Cheers,

Max

[Dave Cheney]

Hi Max,

I think you could improve the layout of your code to be more idiomatic. Simply put there are too many files and too many packages.

Consider merging each package into a single pair of files (.go and test) and merging them all into a single package.

I think you'll find the number of types exported shrinks dramatically and the visible surface are of your package will become smaller.

Dave

[Max Persson]

Hi Dave,

Thanks for the feedback! Much appreciated.

I did start out with a single package but moved components out to their own packages to promote the toolkit idea of the package. At some point I needed to create the domain package, which are shared components, and that seems most awkward to me but I had to to avoid cyclic dependencies.

Many of the packages (for example the event store) can come in different variations that all implement the same basic interface, ranging from a simple in-memory version to a distributed DB backed version. 

How would you recommend structuring a "plugin" architecture like that using a single package?

/Max

-- 

You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/Ro4Ba-8MVOg/unsubscribe.

To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Max Persson

Software Engineer

+46 708 710504

m...@looplab.se

＞ looplab.se

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/Ro4Ba-8MVOg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dave Cheney]

On Tue, Nov 18, 2014 at 8:07 AM, Max Persson <m...@looplab.se> wrote:
> Hi Dave,
>
> Thanks for the feedback! Much appreciated.
>
> I did start out with a single package but moved components out to their own packages to promote the toolkit idea of the package. At some point I needed to create the domain package, which are shared components, and that seems most awkward to me but I had to to avoid cyclic dependencies.

The number one leading cause of cyclic dependencies is having too many
packages. It cannot be solved by adding more packages.

> Many of the packages (for example the event store) can come in different variations that all implement the same basic interface, ranging from a simple in-memory version to a distributed DB backed version.
>
> How would you recommend structuring a "plugin" architecture like that using a single package?

Just put all your public types in the same package, and all the
implementations as well. Remember the yagni rule, don't solve the
problem til you have the requirement.

[Max Persson]

So obvious when you put it like that. Will have a go at it tomorrow (no pun intended).

Thanks for taking the time to give feedback.

/Max

[Max Persson]

The package is now flattened and feels a lot nicer! Thanks again for the early feedback.

Cheers,

Max

On Monday, November 17, 2014 10:23:19 PM UTC+1, Dave Cheney wrote:

[Ingo Oeser]

You might want to fix those, too: http://go-lint.appspot.com/github.com/looplab/eventhorizon

[Max Persson]

Thanks for pointing that out, will fix those issues.

/Max

[egon]

In the example create a package called "invite" then you can name the events/lists as

invite.Info

invite.Projector

invite.Created

invite.Accepted

invite.Declined

Regarding the different packages... I think you went too far with the flattening :)

As a general tip, if you have 3 or more parts for your interface/struct name then you should structure them better.

I didn't yet intend to upload my approach, but... https://github.com/egonelbre/event

I'm not convinced that the aggregate approach as in most CQRS examples is good enough for Go.

With my approach when implementing some aggregate you would do it:

package thing

type Aggregate struct {

event.Aggregate

}

func (thing *Aggregate) Apply(e event.Event) {

thing.Record(e)

switch e := e.(type) {

case Created: ...

// and methods

func (thing *Aggregate) Append(value string){

thing.Apply(Appended{value})

}

The repository would look like:

package thing

type Repository struct {

event.Store

}

func (repo *Repository) Create() *Aggregate {

ep := &Aggregate{}

ep.Id = event.GenerateId()

return ep

}

func (repo *Repository) ById(id event.AggregateId) (thing *Aggregate, ok bool) {

events, ok := repo.Store.List(id)

if !ok {

return nil, false

}

ep = &Aggregate{}

ep.Id = id

for _, event := range events {

ep.Version = event.Version

ep.Apply(event.Data)

}

ep.Changes = nil

return ep, true

}

In the main entry points...

type ThingService struct {

store     event.Store

repo      *thing.Repository

}

func (srv *ThingService) Apply(id event.AggregateId, text string) error {

thing, ok := srv.repo.ById(id)

if !ok { return fmt.Errorf("thing doesn't exist") }

thing.Apply(text)

return srv.store.SaveChanges(thing)

}

Anyways... I'm not comfortable that the Aggregate composes the current state and recording of changes, but I haven't found a way to resolve that nicely.

Hopefully it gives some ideas how to improve things, but I'm not convinced that either one is the best solution for Go.

+ Egon

[Max Persson]

Hi Egon,

Interesting points on the design. Read my comments below!

On Tuesday, November 18, 2014 11:32:32 AM UTC+1, egon wrote:

In the example create a package called "invite" then you can name the events/lists as

invite.Info

invite.Projector

invite.Created

invite.Accepted

invite.Declined

Regarding the different packages... I think you went too far with the flattening :)

As a general tip, if you have 3 or more parts for your interface/struct name then you should structure them better.

Maybe I went too far, and it was mostly on purpose after feedback from Dave. The event store and read repository are definitely candidates for separate packaging. But as suggested above I will let that happen when the need for it comes.

 

I didn't yet intend to upload my approach, but... https://github.com/egonelbre/event

I'm not convinced that the aggregate approach as in most CQRS examples is good enough for Go.

Sweet, another library! I'll definitely have a closer look at it. Always interesting to see different solutions.

 

With my approach when implementing some aggregate you would do it:

package thing

type Aggregate struct {

event.Aggregate

}

func (thing *Aggregate) Apply(e event.Event) {

thing.Record(e)

switch e := e.(type) {

case Created: ...

// and methods

func (thing *Aggregate) Append(value string){

thing.Apply(Appended{value})

}

The Aggregate in Event Horizon can also be implemented with a switch on the type, for example named AssertAggregate. As a matter of fact I used that approach in another app where the event handler is Nanomsg-based.

 

The repository would look like:

package thing

type Repository struct {

event.Store

}

func (repo *Repository) Create() *Aggregate {

ep := &Aggregate{}

ep.Id = event.GenerateId()

return ep

}

I suggest generating the ID in the UI/client, recommended by Greg Young. This way the UI can track the item right from the start by observing the ID it used when creating it.

 

func (repo *Repository) ById(id event.AggregateId) (thing *Aggregate, ok bool) {

events, ok := repo.Store.List(id)

if !ok {

return nil, false

}

ep = &Aggregate{}

ep.Id = id

for _, event := range events {

ep.Version = event.Version

ep.Apply(event.Data)

}

ep.Changes = nil

return ep, true

}

In the main entry points...

type ThingService struct {

store     event.Store

repo      *thing.Repository

}

func (srv *ThingService) Apply(id event.AggregateId, text string) error {

thing, ok := srv.repo.ById(id)

if !ok { return fmt.Errorf("thing doesn't exist") }

thing.Apply(text)

return srv.store.SaveChanges(thing)

}

Anyways... I'm not comfortable that the Aggregate composes the current state and recording of changes, but I haven't found a way to resolve that nicely.

Hopefully it gives some ideas how to improve things, but I'm not convinced that either one is the best solution for Go.

Thanks for the example, again very useful to see another implementation!

With Event Horizon I'm trying to build something that is flexible enough to be configured in all sizes, from in memory testing to full scale distributed DBs in production with different DB-backings for the event store and the read model. That is why I have tried to use interfaces in many places, which also helps a lot in testing by being able to create mocks without a library! I will make the necessary changes needed to support all use cases along the way, as I'm currently using it in a large scale system implementation.

 

+ Egon

I would also love to explore using Go channels in some way, I have a feeling they can be of some use!

Thanks again for sharing your thoughts! Much appreciated.

/Max

[egon]

On Tuesday, 18 November 2014 14:56:21 UTC+2, Max Persson wrote:

Hi Egon,

Interesting points on the design. Read my comments below!

On Tuesday, November 18, 2014 11:32:32 AM UTC+1, egon wrote:

In the example create a package called "invite" then you can name the events/lists as

invite.Info

invite.Projector

invite.Created

invite.Accepted

invite.Declined

Regarding the different packages... I think you went too far with the flattening :)

As a general tip, if you have 3 or more parts for your interface/struct name then you should structure them better.

Maybe I went too far, and it was mostly on purpose after feedback from Dave. The event store and read repository are definitely candidates for separate packaging. But as suggested above I will let that happen when the need for it comes.

The separate packing isn't actually for separation concerns... it's because the names will get shorter and clearer. (Similarly to the "invite" package).

 

 

I didn't yet intend to upload my approach, but... https://github.com/egonelbre/event

I'm not convinced that the aggregate approach as in most CQRS examples is good enough for Go.

Sweet, another library! I'll definitely have a closer look at it. Always interesting to see different solutions.

 

With my approach when implementing some aggregate you would do it:

package thing

type Aggregate struct {

event.Aggregate

}

func (thing *Aggregate) Apply(e event.Event) {

thing.Record(e)

switch e := e.(type) {

case Created: ...

// and methods

func (thing *Aggregate) Append(value string){

thing.Apply(Appended{value})

}

The Aggregate in Event Horizon can also be implemented with a switch on the type, for example named AssertAggregate. As a matter of fact I used that approach in another app where the event handler is Nanomsg-based.

I initially tried several ways with reflection and all of those had too much "magic" and I didn't like where those implementations were going. Eventually decided to simply use a switch.

 

 

The repository would look like:

package thing

type Repository struct {

event.Store

}

func (repo *Repository) Create() *Aggregate {

ep := &Aggregate{}

ep.Id = event.GenerateId()

return ep

}

I suggest generating the ID in the UI/client, recommended by Greg Young. This way the UI can track the item right from the start by observing the ID it used when creating it.

I'm not sure whether UI can be trusted to generate that ID, but I can see the usefulness of creating that in UI/client.

 

 

func (repo *Repository) ById(id event.AggregateId) (thing *Aggregate, ok bool) {

events, ok := repo.Store.List(id)

if !ok {

return nil, false

}

ep = &Aggregate{}

ep.Id = id

for _, event := range events {

ep.Version = event.Version

ep.Apply(event.Data)

}

ep.Changes = nil

return ep, true

}

In the main entry points...

type ThingService struct {

store     event.Store

repo      *thing.Repository

}

func (srv *ThingService) Apply(id event.AggregateId, text string) error {

thing, ok := srv.repo.ById(id)

if !ok { return fmt.Errorf("thing doesn't exist") }

thing.Apply(text)

return srv.store.SaveChanges(thing)

}

Anyways... I'm not comfortable that the Aggregate composes the current state and recording of changes, but I haven't found a way to resolve that nicely.

Hopefully it gives some ideas how to improve things, but I'm not convinced that either one is the best solution for Go.

Thanks for the example, again very useful to see another implementation!

With Event Horizon I'm trying to build something that is flexible enough to be configured in all sizes, from in memory testing to full scale distributed DBs in production with different DB-backings for the event store and the read model. That is why I have tried to use interfaces in many places, which also helps a lot in testing by being able to create mocks without a library! I will make the necessary changes needed to support all use cases along the way, as I'm currently using it in a large scale system implementation.

Don't try to make it flexible in configuration... a clean implementation that can be easily modified is probably a better approach. Making everything configurable adds another layer of complexity. Greg Young has mentioned that before... don't write a framework for CQRS.

The implementation I showed, solved a particular problem in my code... depending on the situation my implementation would be different.

(That was one of the reasons I hadn't published my implementation, it's too fine-grained... making it into a proper CQRS/ES example would've been probably a better idea... making a package for eventstore would probably make more sense.)

 

 

+ Egon

I would also love to explore using Go channels in some way, I have a feeling they can be of some use!

I tried, but didn't come up with any good uses... the code only got more confusing. Although there could be some uses in communicating with the Client part. (e.g. multiple async requests)

[Max Persson]

I really enjoy the discussion! Here are my thoughts:

On Tuesday, November 18, 2014 2:22:13 PM UTC+1, egon wrote:

On Tuesday, 18 November 2014 14:56:21 UTC+2, Max Persson wrote:

Hi Egon,

Interesting points on the design. Read my comments below!

On Tuesday, November 18, 2014 11:32:32 AM UTC+1, egon wrote:

In the example create a package called "invite" then you can name the events/lists as

invite.Info

invite.Projector

invite.Created

invite.Accepted

invite.Declined

Regarding the different packages... I think you went too far with the flattening :)

As a general tip, if you have 3 or more parts for your interface/struct name then you should structure them better.

Maybe I went too far, and it was mostly on purpose after feedback from Dave. The event store and read repository are definitely candidates for separate packaging. But as suggested above I will let that happen when the need for it comes.

The separate packing isn't actually for separation concerns... it's because the names will get shorter and clearer. (Similarly to the "invite" package).

I don't like using packages for the convenience of shorter names, autocompletion in my editor solves that. I do like types to have a name that is useful in logging with %#v, ie CreateUser instead of Create.

 

 

 

I didn't yet intend to upload my approach, but... https://github.com/egonelbre/event

I'm not convinced that the aggregate approach as in most CQRS examples is good enough for Go.

Sweet, another library! I'll definitely have a closer look at it. Always interesting to see different solutions.

 

With my approach when implementing some aggregate you would do it:

package thing

type Aggregate struct {

event.Aggregate

}

func (thing *Aggregate) Apply(e event.Event) {

thing.Record(e)

switch e := e.(type) {

case Created: ...

// and methods

func (thing *Aggregate) Append(value string){

thing.Apply(Appended{value})

}

The Aggregate in Event Horizon can also be implemented with a switch on the type, for example named AssertAggregate. As a matter of fact I used that approach in another app where the event handler is Nanomsg-based.

I initially tried several ways with reflection and all of those had too much "magic" and I didn't like where those implementations were going. Eventually decided to simply use a switch.

Sure, the aggregate I have is as I said just one implementation of many of the Aggregate interface. It's quiet possible that I switch to the assert implementation to optimize for speed if the reflection package proves to be too slow.

That is why I call it a toolkit, and not a framework, where components can be cherrypicked for the purpose, or reimplemented by using the interfaces provided.

I have now experienced the complexity of creating a toolkit for CQRS/ES, in the end I just want to save time in implementation by not having to write for example a command handler or an event store in every project where I want to use CQRS/ES.

[egon]

On Tuesday, 18 November 2014 16:56:05 UTC+2, Max Persson wrote:

I really enjoy the discussion! Here are my thoughts:

On Tuesday, November 18, 2014 2:22:13 PM UTC+1, egon wrote:

On Tuesday, 18 November 2014 14:56:21 UTC+2, Max Persson wrote:

Hi Egon,

Interesting points on the design. Read my comments below!

On Tuesday, November 18, 2014 11:32:32 AM UTC+1, egon wrote:

In the example create a package called "invite" then you can name the events/lists as

invite.Info

invite.Projector

invite.Created

invite.Accepted

invite.Declined

Regarding the different packages... I think you went too far with the flattening :)

As a general tip, if you have 3 or more parts for your interface/struct name then you should structure them better.

Maybe I went too far, and it was mostly on purpose after feedback from Dave. The event store and read repository are definitely candidates for separate packaging. But as suggested above I will let that happen when the need for it comes.

The separate packing isn't actually for separation concerns... it's because the names will get shorter and clearer. (Similarly to the "invite" package).

I don't like using packages for the convenience of shorter names, autocompletion in my editor solves that.

I don't do it for convenience... it's more for clarity. And, yes, it's a subjective point.

Initially I the same approach as you, but when including something from a third place the code didn't look so good any more... e.g. let's say the place where you declared "CreateUser" was package "domain"... then in package "admin" where you create the user list projection you would always need to write:

switch event.(type){

  case domain.UserCreated: ...

  case domain.UserRemoved: ...

}

instead of:

switch event.(type){

  case user.Created: ...

  case user.Removed: ...

}

Of course, when there are some things related to "user" domain e.g. user.AddressUpdated / user.NameUpdated, then it makes sense to have them in "user".

I do like types to have a name that is useful in logging with %#v, ie CreateUser instead of Create.

%#v also prints the package name: http://play.golang.org/p/r07511mMCO

So "type Created" in package "user" would print "user.Created{}".

[Max Persson]

The library now passes golint fully. Thanks for the motivation.

On Tuesday, November 18, 2014 8:38:07 AM UTC+1, Ingo Oeser wrote:

[Max Persson]

Fair enough, I will give it a try and see how it feels in the example!

[Max Persson]

Hi Egon,

I have now implemented a new dispatcher and aggregate that uses delegation to let domain aggregates handle type assertion of commands and events themselves instead of the slightly "magical" reflection version that I first implemented. It would be interesting to get your thoughts on the implementation!

See the example here: https://github.com/looplab/eventhorizon/tree/master/examples/delegation

Cheers,

Max

On Tuesday, November 18, 2014 11:32:32 AM UTC+1, egon wrote:

[egon]

On Friday, 21 November 2014 15:45:55 UTC+2, Max Persson wrote:

Hi Egon,

I have now implemented a new dispatcher and aggregate that uses delegation

It's it's forwarding, not delegation... http://en.wikipedia.org/wiki/Delegation_(programming) 

I do like it more than the reflection based version... I also have experimented with having CommandDispatcher and EventBus separate... that would get rid of 

disp.AddHandler, disp.Dispatch -> disp.Register, disp.Handle

disp.AddSubscriber, disp.Dispatch -> bus.Listen, bus.Publish

This also means, I am able to minimize the dependencies as well. If something needs only the bus, I'll only use the bus...

Also if you add the type as the last parameter in AddSubscriber / AddHandler then you can use varargs... e.g.

disp.Listen(guestListProjector, InviteCreated{}, InviteAccepted{}, InviteDeclined{})

I like the returning of EventStream in CommandHandling, but maybe there is a better way of doing it. The reason I haven't used that approach is because if you emit multiple events and one of the later events depends on the effects of some previous event inside the aggregate then the returning approach will be more confusing.

Also I would probably simply use []Event instead of EventStream.

PS: I also implemented the guestlist example:

https://github.com/egonelbre/event/tree/master/example/guestlist

Not a verbatim translation, but demonstrates how I would currently implement it. (I'm still not happy with the Aggregate approach...)

+ Egon

[Max Persson]

Thanks for the reply!

Good that you like the new version, I implemented it partly to show and test the flexibility of the toolkit. I don't want to debate design patterns, but I think it's perfectly ok to define it as delegation. From the Wikipedia article on the pattern:

"a helper object, known as a delegate, is given the responsibility to execute a task for the delegator."

http://en.wikipedia.org/wiki/Delegation_pattern

Anyway, regarding the command handler and event bus, I did have them as separate objects before but decided to merge them for now to simplify. I think I will split them apart soon though, I like the other approach better.

Regarding the EventStream I agree with you that []Event could be a bit more idiomatic, I may change that!

I'm also thinking of changing the returned events to be a parameter instead of a return, much like the wonderful http.Handler interface. Lots of inspiration to get from that. I have started using the library in a fairly large project and I will probably change and add a lot when I get real (and different) use cases.

Cheers,

Max

--

Max Persson

Software Engineer

+46 708 710504

m...@looplab.se

＞ looplab.se

--

[Bimal Kaluarachchi]

Hi Max

I am trying to learn to implement eventhorizen  is there possibility  to give me little example  of implementation.

existing sample bit complected for as i am still learning this ddd concept.

thanks so much

cheers

Bimal