Early availability of an embeddable event sourcing database for Java

[Yurii Rashkovskii]

Hi,

I've recently started extracting a Java event sourcing implementation I've used in a few projects, into a separate project.

Some of the core objectives were to produce an embeddable, lazy event sourcing database with primary focus on event search for the purpose of building the state of the world.

While still young, it is already suitable for some projects. If you are interested, you can find more information on http://eventsourcing.com.

Feedback will be appreciated!

[Danil Suits]

Reactions on reading the docs

1) "Eventsourcing is an event sourcing framework for Java."

I feel like the name is just asking for confusion.  Too damn easy to insert or elide the white space in writing, too difficult to distinguish the white space.  You've already put in a claim on es4j, why not use that consistently throughout your documentation?

2) Examples should be written with care

return resultSet.iterator().next().get().get().name();

Beyond the immediate concern that it appears as though little thought has been given to the example (or for that matter, the usability of the framework), it also has me wondering whether these constructions mirror those in the source, and if so, will the maintainer be able to keep improving the framework, or will it collapse under the weight of the technical debt.

3) Non trivial examples

CRUD domains are not particularly compelling; I'd rather see examples that have clear (if trivial) "business invariants" that are being enforced.

[Yurii Rashkovskii]

Danil,

Thanks for the feedback! Really appreciate it. To address your points:

1) Very valid point. I went ahead and updated the documentation.

2) I get your concern here (or so I hope). I too find that construct (specifically, the get().get()) part suboptimal. The original motivation for it has to do with how ES4J does lazy/on-demand results and trying to avoid loading data from the storage when it is not necessary. After some more thinking, as a potential solution, I just extended EntityHandle API to include both getOptional() and get() for usability reasons, so one doesn't need to write get().get() anymore, but just get() [in order to retrieve the entity in question]. Is this the part that was most annoying/unpleasant to you or did you have a different concern there? (As a side note, I do find the indexing/querying API to be a little verbose and clunky at times, but I am yet to find a good way to simplify it [I am primarily relying on the API provided by CQEngine)

3) I agree. It's difficult to find examples that are both very succinct (for readability/comprehension reasons) and cover a wider range of problems expected by different readers. I'd love to hear about any examples you might have had in mind, it can help me substantially to write better documentation _and_ software.

Thanks,

Yurii.

[Greg Young]

Including a link to github would be useful. I cant seem to find it

> --
> You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Studying for the Turing test

[Yurii Rashkovskii]

Good point. [Download] button links to GitHub, but that wasn't obvious indeed. I've added an explicit link.

Thanks!

--
You received this message because you are subscribed to a topic in the Google Groups "DDD/CQRS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dddcqrs/Bmx7bypW2uw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dddcqrs+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Y.

[Greg Young]

Thanks! The best place to put it would be the github icon/text

[Greg Young]

I am looking for the code where you implement the index/transaction file.

I might be a bit dumb but I can't seem to find them

[Yurii Rashkovskii]

Greg,

The hierarchy of the Journal class is what records the commands and events (I primarily use MVStoreJournal these days, MemoryJournal is more of an silly example). Indexing is done using IndexEngine hierarchy (I use a combination of MVStoreIndexEngine and MemoryIndexEngine by combining them into a CascadingIndexEngine instance) and the underlying implementation (largely exposed through the API) is CQEngine (https://github.com/npgall/cqengine). Hope this helps, let me know if there's anything else I can clarify. The project is young (as I mentioned) so not all of the documentation has been written yet.

Thanks,

Yurii.

[Danil Suits]

2) Your revised documentation is an improvement, but I'm still leery; the example still doesn't make it look easy to write correct code.  (The idiom for initializing the user object, in particular, violates my own understanding of best practices -- why aren't the properties being passed into the constructor.

3) I don't have a good answer here yet, but I can offer this hint: I'm pretty sure the example I'm looking for would have the domain model as the book of record (as opposed to entities that are just reflections of other domains), and be showing a command handler where the command might be reasonably rejected by the domain.

[Yurii Rashkovskii]

Hi Danil,

Thanks again for your feedback. I am happy to let you know that event and command properties are being passed into a constructor in the upcoming release of es4j (http://es4j-doc.eventsourcing.com/core_concepts/event.html and http://es4j-doc.eventsourcing.com/core_concepts/command.html). The documentation still needs a lot of work, of course — but we're getting there. You can see other upcoming updates at https://github.com/eventsourcing/es4j/blob/master/CHANGELOG.md

Yurii.

[Ruslan Rusu]

Hey Yurii,

Your documentation does not use the Ubiquitous Language of this community. 

Would you care to explain how is following snippet related to DDD/CQRS/ES ?

===============here is a snippet for docs http://es4j-doc.eventsourcing.com/core_concepts/repository.html ======================

Now we're ready to publish the command:

User result = repository.publish(new CreateUser("f...@bar.com")).get();

===============here is a snippet for docs http://es4j-doc.eventsourcing.com/core_concepts/repository.html ======================

<ruslan/>  

[Yurii Rashkovskii]

Ruslan,

The current version of the documentation has been put together very hastily and is really not that great — an the current example uses quite a poor language, I wholeheartedly agree with that. I recently began improving it, starting from a new demo application as a guide. You can see some of this work in progress at https://github.com/yrashk/es4j/tree/doc-update/examples/foodsourcing/src/main/java/foodsourcing. I am also interested in collaborating with others, documentation is best done that way.

Ubiquitous language is often a a result of iterative work. It helps dealing with uncertainty by avoiding ambiguity. Eventsourcing is designed to be aligned with its goals and methods. I am hoping to release the updated documentation some time this month, most likely alongside the upcoming 0.4 release.

Yurii.

[Ruslan Rusu]

Yurii,

I'll try to be explicit, ***repository.publish*** does not belong UL and you have it in the code too

-- Repositories

           An abstraction deals with persistence concerns. 

           The api surface area is very small repository.GetById(aggid) and respository.Save(aggObj).

-- Domain events

          Are propagated using Publish(evt)

-- Commands

          Submitted to Component A which might reject it

Is this hostility or contribution ?

<ruslan/>

[Yurii Rashkovskii]

Hi Ruslan,

Would you be open to expressing your concerns in a form of a patch to contribute a positive resolution? The project is governed by the C4 process to facilitate diverse contributions (optimistic PR merging). I also hope that will help me understanding your point much more clearly.

Yurii

[Ruslan Rusu]

Yurii,

I stated with another occasion that yet another framework will not make life easier to deliver business value. 

We had plenty of them around and they do not live more than 2 years. Engaging in a constructive conversation 

is the max I can do. 

Back to ***repository.publish*** here is an example of what I mean. 

https://github.com/gregoryyoung/m-r/blob/49ed659d674b0914aad3749530310596a6fc8b7d/SimpleCQRS/Domain.cs#L98

<ruslan/>

[Yurii Rashkovskii]

Hi Ruslan,

I think I get your points better now. Speaking of `repository.publish`. Just because the language and/or API surface's shape of Eventsourcing's Repository is different from some other definitions, it doesn't make it any less of a repository. Its shape is dictated by certain design choices, like late domain binding and using indices as the ultimate read model.

Now, the point of frameworks (or libraries). There are different opinions, and I tend to believe that neither extreme is right. Sometimes its easier to achieve the end goal without a framework, sometimes it isn't. I've started this project because my "I don't need a framework" embedded solution outgrew the host project. 

What is more important, death of these projects is not a zero-sum game — there's always a gain with just about every project, subtle or not — for many or just a few. (Side note: humans also die after some time, but we don't stop making babies!)

Most projects die if nobody contributes. I can understand the "I don't need this" reason for not contributing. Not contributing "because they all die anyway" is more likely to shorten project's lifetime (humans die, so what's the point of feeding them?)

Anyway, thanks a lot for your contribution. Hopefully somebody will find this conversation useful down the road!

Yurii.

[Ben Kloosterman]

In DDD you still have repository.publish - but you dont have it in your business logic. ie Pure logic no infrastructure concerns /  code hence the usual style of a handler calling the BL and doing the persistence..

Ben

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.

[Yurii Rashkovskii]

Hi Ben,

Just to note, Eventsourcing doesn't litter the business logic with `repository.publish`. It is only used to "enact" commands. Commands themselves should not publish other commands, that's what "subscribers" are for.

That said, when commands produce events they can, of course, use the querying and locking capabilities of Eventsourcing to either decide the actual set of events to be produced or to generate an exception. This particular interfacing with querying and locking should be (but is not required to be) implemented elsewhere to keep commands as clean and readable as possible. So the last bit of "infrastructure" concern is that Eventsourcing passes a repository to `Command#events()` and thus concerns it with the infrastructure a little bit. Theoretically this could be wrapped into a higher level concept to hide this detail but I personally saw little value in this. However, as I mentioned above, the project is intended to be driven by contributors, so if somebody feels strongly about this, they are welcome to send a pull request (https://github.com/eventsourcing/es4j/blob/master/CONTRIBUTING.md).

Hope this helps!

Yurii.

[Greg Young]

"That said, when commands produce events they can, of course, use the
querying and locking capabilities of Eventsourcing to either decide
the actual set of events to be produced or to generate an exception.
This particular interfacing with querying and locking should be (but
is not required to be) implemented elsewhere to keep commands as clean
and readable as possible."

I read the words and understand them but have no idea what you are
saying. interfacing with querying and locking facilities should not be
implemented in commands?! Commands are messages....

[Yurii Rashkovskii]

Hi Greg,

I am sorry about that. Let me try to explain this again.

In Eventsourcing, classes implementing a `Command` interface have a method called `events` which is used to determine which events to be produced as a result of this command. The `events` method is effectively a "command message handler".

Often enough, the implementation of this method is quite simple, it just returns a stream of events. Pretty straightforward. Sometimes, this method would need to check against the state of the world (in a broad sense: the Repository, an external API call, whatever) before deciding what events to produce, or whether an exception needs to be thrown. Now, you can put repository querying right into the `events` method, but that will make it harder to read or refactor. So it makes sense to put this logic in some other place, give it an easy to understand name (`boolean starsAlignedCorrectly(Repository)` or whatever) and call that from `events` so it reads as close to plain English as possible, while the logic interfacing with the repository, external resources, locking, etc. is compartmentalized elsewhere.

Does this make more sense to you?

[Greg Young]

Yes you are using the command pattern from GoF. There is a reason why
command handlers exist instead of doing this. For starters you end up
with two interfaces instead of a single, the command handlers share an
interface with event handlers (they are all just message handlers).
There is also the fact that very often dependencies are often needed
"resolving a command from a container" is certainly an odd phrase. But
the most core reason is that commands and events are schema and should
be implemented as such.

[Yurii Rashkovskii]

Greg,

In Eventsourcing, the emphasis is put on late domain binding, so often times events are just saved to the journal and indexed — custom event handlers are not required to incrementally build the index (which is done automatically and is used for late domain binding). There is a "subscribers" mechanism that allows to handle events, of course — as there are plenty cases when they are needed, such as notifications, aggressive caching, etc.

Commands and events in Eventsourcing are very much *the schema*. The only one that's actually persistent (as opposed to fluid and dynamic code-driven "microschemas" of models, see https://blog.eventsourcing.com/why-use-eventsourcing-database-6b5e2ac61848). The layout of every command and every event (since both are journaled to preserve causal information) is strictly defined as a composition of its name and properties (which have types and names themselves) and any change in that layout will change the fingerprint of the schema so that events with even slightly different layouts can not be confused.

Hope this helps.

[Greg Young]

" custom event handlers are not required to incrementally build the
index (which is done automatically and is used for late domain
binding). There is a "subscribers" mechanism that allows to handle
events, of course — as there are plenty cases when they are needed,
such as notifications, aggressive caching, etc."

Who needs things like projections. I have seen very few event sourced
systems that did not need them.

[Yurii Rashkovskii]

Greg,

I did *not* say that some form of "projection" is not necessary. The design choice I made was to give a (default) option to defer binding to the domain for as long as possible (you can see more on that in the blog post I mentioned earlier in this thread). Hence:

1. If you need to make a projection eagerly (during event journalling), you can do so using the subscribers mechanism, which I mentioned in my previous message and you quoted. 

2. If you use the "default" late domain binding approach, you do dynamic "projections" based on indices and journal records that were recorded earlier. If these projection/queries become too expensive, you can optimize/cache them either eagerly ("subscribers") or on-demand (when used first).