Security and Serialization in Axon

[António Mota]

Hi Allard and all.

We are close to start a quite big project and we (almost) decided to use Axon in a "progressive" way. That means we are planning to use it to model a Human Task processor and integrating it with our existent structure, and later expand gradually the use of Axon to implement a bunch of micro-services.

For now we identified 3 areas  of concern:

Security: our infrastructure is heavily based on Spring, including Spring Security. As far as I know this won't play nicely once we start to use distributed services, since the Spring Security authentication is based om thread-bounded variables. What we are trying to implement is to create a UAA service for all the distributed services to use, which will probably be based in the Cloud Foundry UAA project and the JWT standard. But I have no idea where to start this integration. Is there any "how to" or "best practices" for security in Axon?

Events and Sagas storage: the storage of events and sagas is completely separated and uses different structures. Is there a way to "unify" this storage?

And related to this, Events and Sagas serialization. At the moment Sagas are serialized using binary serialization. Is there a way to do all the serialization in XML or JSON?

And of course, if someone implemented Human Tasks in Axon I would love to hear about it...


Thanks all.

[António Mota]

Hmmm, it seems I forgot a very important thing, to read the documentation... I just saw there is a JdbcSagaRepository that serves our purpose of "unification" and serilaization, so the only question here is about the Security.

Thanks and sorry bout this...

[António Mota]

Wadda ya know, it seems Sagas can only be stores as byte[] after all... I had to extend PostgresSagaSqlSchema and JdbcSagaRepository.storeSaga to get rid off the SagaEntry.class that seriializes as byte[] and it's not extensible...

Should be a big deal to make it extensible thougth.

Still looking at Security now...

Cheers/

[António Mota]

Well, after successfully putting both Events and Sagas being serialized with XStreamSerializer and JacksonSerializer, the powers that be decided that in fact they have to be serialized to XML using JAXB, which is causing me to much pain...

Any alternative here behind writing my own?

Cheers.

Ho, I forgot to talk about the Security ...

[Nedo Sab]

Hi,

For the security and authorization related concerns I would implement a custom CommandBusInterceptor which would collaborate with the UAA service to verify the JWT signature, and then decide if the command should be dispatched to the command handler or denied. Hope that helps.

--
You received this message because you are subscribed to the Google Groups "Axon Framework Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to axonframewor...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[António Mota]

Yes, that would definitely help, I still have to finish my UAA side but looks like a good idea indeed.

Cheers.

[Allard Buijze]

Hi Antonio,

personally, I don't think serializing using JAXB is a sensible choice to make. I'd like "power that be" to focus on the outcome of serialization formats, instead of the technologies used to generate them. Creating a serializer shouldn't be too hard, there is a number of examples in Axon already. Let's hope JAXB can do the job. Otherwise, why don't you ask the powers that be to develop it? They made the dcision, after all.....

That with Nedo's answer about security (which I completely agree with) should address your concerns, right?

Cheers,

Allard

[António Mota]

Hi Allard.

Well, the "powers that be" also sign my check so it's better to leave it that way... :)  No, really, the issue is our entire model is schema based and we use JAXB to create the model classes from it. And we plan to use that also for our commands and events classes, so it makes sense to use JAXB only. I did some work implementing a JAXB serializer, is far from complete but I'm happy with the proof-of-concept.

Regarding the security I think Nedo's solution does the trick, yes. However I couldn't test it yet because I'm having a heck of a time just to put the Authentication to work using the cloudfoundry uaa service, let alone the Authorization part... I've been in Spring configuration hell for 2 days now... Do you have anything (examples, pointers) of using a SSO solution with Axon?

What I'm concern now is with using Event Sourcing to implement Human Tasks and corresponding control (like a workflow but less rigid). This was my idea so it's my *** on the line and I'm starting to be uncomfortable wit it because it's a new territory for me. We don't need a implementation of the WS-HT specification, it's more like a glorified ToDo, so you have any poiters to a thing like that? I think I saw a Axon example of a to-do list but written in old versions of Axon...

I'll try to get rid of this authentication problem today to try and start "real" tests with Axon.


Thanks for your support.

[Allard Buijze]

Hi,

I'm afraid I don't have any examples in full fledged uaa.
You should use dispatch interceptors to trigger authentication. Attach the user id and perhaps a bit of profile information (e.g. roles) to the outgoing command's meta data.
For the authorization part, you can use dispatch interceptors if your command classes have authorization configuration annotated on them. If the command handlers need a full profile, you can use command handler interceptors to fetch the full user profile based on the id of the meta data abd attach it to the command. You can also attach the profile to a threadlocal and use a ParameterResolverFactory to allow for the profile to be injected into an @CommandHandler method parameter.

Cheers,

Allard

[António Mota]

Hi again.

I finally have the UAA working with my Axon prototype, so I'll begin the tests today. I also modeled a simple Task that is basically a id, a name and a owner, and half-dozen commands and events to handle the task lifecycle (create, assign, start, complete). So I have a Task Aggregate for this.

But now (and this is the question of the day) I concluded that one task is in fact a compositions of several tasks. For instance, a Audit task can be comprised of HandlePaper task plus a PhoneCompany task. The HandlePaper task can be comprised of MakePhotocopies and ArchivePapers. And actually, since this are Human Tasks, the human can create "had hoc" tasks as "children" of any tasks. All of this tasks have the same lifecycle, being that the Audit task can only be completed when all children tasks are completed, and so on.

So now my problem is, how can I model this as a aggregate? My design until now is only for a "independent" task.

Any suggestion?

Thanks.

PS. Did you saw my comment above about the Sagas?

Sagas can only be stores as byte[] after all... I had to extend PostgresSagaSqlSchema and JdbcSagaRepository.storeSaga to get rid off the SagaEntry.class that seriializes as byte[] and it's not extensible

Don't you think it will be good to allow extensibility to serialize sagas in other formats like the Events? (Json, XML, ...). If you want I can send you what I did, but it's extended only to my particular case...

[Allard Buijze]

If you have aggregates whis behavior depends on things happening in other aggregates, two things can be going on. First, it's possible that your aggregate boundary is wrong. Maybe, subtasks should be part of the parent task's aggregate, so that it can manage the workflow properly. Note that this decision is completely independent of the query model structure, where each task is likely to be an 'independent' entity.
Secondly, if they are separate aggregates, you'll need to have a Saga to orchestrate behavior between related tasks.

My gut feeling here is that you should investigate option 1.

Cheers,

Allard

[Allard Buijze]

About the Sagas, you can store them any way you like. However, the database table used by the generic JPA SagaRepository (and JDBC) uses a column type of byte[] to store the data. That still alows you to store json, xml, etc in there.

Cheers,

Allard

[António Mota]

Hmmm, I was thinking in using something like you describe in 4.2.3. Complex Aggregate structures of the Ref Guide, but since you didn't mention it I assume my understating of Non-Aggregate Root Entities is wrong... Are there any "real" examples of those "complex aggregates"?

Regarding your pont 1, what I was thinking was having a kind of "nested" aggregates, with parent/child relations. I was thinking about it to maintain the lifecycle of each child task within it's aggregate, but now that I'm saying it out loud maybe it's just bad design. Also, my boss says that "task" per se doesn't have a "business meaning", so in terms of DDD should not be a Aggregate. So maybe I should put all this "lifecycle" thing inside the Task class itself and create the Aggregates around one or more tasks that makes business sense, like Audit, BookFligth and things like that.

That will also avoid the use of Sagas to coordinate behaviour, but in fact some of those "business tasks" may be long lived, and that would indicate the use of a Saga, correct?

Regarding the Saga store, our plans in store everything (events, sagas, metadata) in XML is to make use of the XML capabilities of postgres, for instance, updating columns with a the result of a XSLT applied to their values.

Nevertheless my main concern is to test the security part for now.

Thanks again for your support.

[António Mota]

Well, what I was talking about the Complex Aggregates seems to work like this:

RootTaskAggregate

 @EventSourcedMember
 protected ist<ChildTaskAggregate> childs = new ArrayList<ChildTaskAggregate>();

and

ChildTaskAggregate extends RootTaskAggregate implements EventSourcedEntity

but I have the feeling that I turned the complicometer on... I guess is going to be difficult to decide to which Childs a event should be applied or not.

What do you think of this?

Cheers.

[Allard Buijze]

This is the intended way. When working with entities in collections, make sure the event handler methods of these entities check whether the event 'belongs' to that specific entity.
Nothing complicated....

Cheers,

Allard

[António Mota]

Yes, but actually after my poc I decided that it was too much complexity in one place. I reimplemented the Aggregate with only one root Task that include other tasks that include... It's running very close to what I envisaged in the first place. And yet...

My boss is not very convinced with the model (a "task" is hardly a domain model) and he argues that a "task" should not have any knowledge of the business process, it should be a agnostic service to be used by other services (services in the of independent micro-services). That means a Task aggregate shouldn't know nothing about the more business-oriented tasks like  AuditDocument, BookFligth, ByTheBook and thus can't control this processes. He suggests, with reason, to control those high-level processes with Sagas, that would be responsible for sending commands to the Task Aggregate and coordinate all the tasks necessary. 

That seems OK to me, however I was under the impression that the "normal" flow of ES should be Command->Aggregate->Event->Saga, and with this approach in fact we'll have Event->Saga->Command->Aggregate. Now other than a question of "style" I can't see nothing wrong with this from a technical point of view. But it's a question of "it works in practice, but it will work in theory?"

Do you fond any drawbacks of using this Saga-first approach?

Cheers.

[Allard Buijze]

Hi Antonio,

using Sagas to control these processes will work fine. Modelling is a complex proces. I can only assume that you have more in-depth knowledge of all the intricacies of your domain. It's the little things that can make all the difference.

My advice would be to create the model and see if you can do so without resorting to "strange" commands and events. Once that happens, it's an indication your model is wrong.

In the end, theory will have to accept the fact that things have worked out in practice ;-)

Cheers,

Allard

[António Mota]

Hi again.


Finally it's official, we do are going to use Axon in our project :) and my PoC is now being "migrated" to a full production-level service. For that I did some work in:

Serialization:
We have to support Postgres with XML data types. That means serialize Metadata, Events and Sagas to XML (or a string representation of) and store then as XML. For that I had to implement a Axon Serializer, extend GenericEventSqlSchema, PostgresSagaSqlSchema and JdbcSagaRepository. Not a big problem here but the code could be more clean with some small changes in Axon itself, if you want to discuss that I'll be happy to do it.


Security:
I'm going to implement a CommandDispatchInterceptor for Identity and a CommandHandlerInterceptor for Access Control. For the first I don't see any problems, however for the second I did got into trouble. The intention here was to access the Aggregate itself from the CommandHandlerInterceptor to check some constraints in the aggregate entity itself. I do have access to the unitOfWork there and I was hoping to access the aggregate from there, but I couldn't find a way. Did I forgot something?

A few more doubts I'm having. I tend to have a command raising a event only. However in some cases it may make sense to raise more than one event, or raising different events depending on some conditions. Is this OK or is considered a bad practice?

Some times it make sense to me to do the "work" in the commands and pass the results to the event, while other times I tend to do that "work" in the events themselves. But I think this second option can be dangerous because of the replays. Should the "work" always be done in the command handler? A example:

1) "work" done on the event

@CommandHandler
public TaskAggregate(CreateTaskCommand command) throws Exception {
 apply(new TaskCreatedEvent(command.getId()) );
}

@EventSourcingHandler
public void on(TaskCreatedEvent event) throws Exception {
 this.id = event.getId();
 Task task = new Task(); 
 task.creationDate = DateTime.now();
 task.setState(TaskState.CREATED);
}


2) "work" done on the command

@CommandHandler
public TaskAggregate(CreateTaskCommand command) throws Exception {
 Task task = new Task(); 
 task.creationDate = DateTime.now();
 apply(new TaskCreatedEvent(task) );
}

@EventSourcingHandler
public void on(TaskCreatedEvent event) throws Exception {
 this.id = event.getId();
 Task task = event.getTask();
 task.setState(TaskState.CREATED);
}


3) hybrid "work"

@CommandHandler
public TaskAggregate(CreateTaskCommand command) throws Exception {
 apply(new TaskCreatedEvent(command.getId(),  DateTime.now());
}

@EventSourcingHandler
public void on(TaskCreatedEvent event) throws Exception {
 this.id = event.getId();
 Task task = new Task(); 
 task.creationDate = event.getDate()
 task.setState(TaskState.CREATED);
}

This is a simple example, but taking into account that for the creation of a entity it will take more than just creating a object (call injected services, access repositories, etc.) what is the "correct" way of do it? Assuming there is such a thing...

Thanks again for your precious support.

Cheers.

[Allard Buijze]

Hi Antonio,

events should always provide enough information about the context, so that a handler can find all relevant information in that event. Doing DateTime.now() based on receiving a TaskCreatedEvent is not likely to produce the result that you expect. When loading an aggregate, and streaming events on it, DateTime.now() is not the creation time of the Task anymore. Therefore, include it in the event.

The taskState created is a logical result of a Created Event, so you could omit if for that reason. On the other hand, explicitly including it can be a good thing, too.

Be careful not to expose entities (such as Task) in your event. Entities are stateful and are very bad candidates to include in events (as they effectively make the event stateful as well).

Cheers,

Allard

[António Mota]

So, a s a general rule of thumb, you'll you say that "work" should be done in the command and the strictly-relevant information set in the Event so the Handler will use it to set the aggregate internal state only?

I quite don;t understand what you mean by "Be careful not to expose entities (such as Task) in your event.". In my example, you mean the Task should be created in the event, as in 3) instead of in the command as in 2) ? Note that a Task it's not the "entity" per se, it's the "main" entity, because there can be several child tasks on this task, that also can have childs and so on (I kept then in a flat structure though). The Aggregate however is of type Task. I forgot to add the line below in the 3 examples:

task.setState(TaskState.CREATED);
tasks.put(task.getId(), task);

I do agree that in this case the CREATED is not needed but I kept it there to be more explicit.

And yes, 1) I knew it was wrong because of the date, it was the first error I fixed :)

[Allard Buijze]

Hi,

with that remark, I was referring to the following lines:

 Task task = event.getTask();
 task.setState(TaskState.CREATED);

Apparently, the Task class contains mutable state. Objects with mutable state should never be part of your events, as they effectively make your event mutable. Just imagine multiple handlers concurrently handling an event whose state is changing. The outcome is then completely non-deterministic.

Cheers,

Allard

[António Mota]

So if I have something like

@CommandHandler
public TaskAggregate(CreateTaskCommand command) throws Exception {
 Task task = new Task();
 task.creationDate = DateTime.now();
 apply(new TaskCreatedEvent(task) );

 apply(new TaskStartedEvent(task) );

}

@EventSourcingHandler
public void on(TaskCreatedEvent event) throws Exception {
 this.id = event.getId();
 Task task = event.getTask();
 task.setState(TaskState.CREATED);
}

@EventSourcingHandler

public void on(TaskStartedEvent event) throws Exception {
 Task task = event.getTask();
 task.setState(TaskState.STARTED);
}

I may not have the Task set as STARTED after UOW commits? And BTW, is it OK to have more than one Event raised by one Command?

Also a important question is that one of the CommandHandlerInterceptor accessing the aggregate itself. Is there a way to get the Aggregate using the UnitOfWork in the CommandHandlerInterceptor?

Thanks.

...

[Allard Buijze]

HI,

it's not so much about the unit of work. When you have an entity as part of your events, you don't have control over what component will change it's state. Event Handler (outside of the aggregate) may be handling an event asynchronously, causing annoying race conditions.

So instead of including the Task in your event, describe the task. The Task class is an implementation detail only relevant to the command model. Including the Task object in your event will cause that model to be "suitable" for both mutation and queries, basically removing the whole idea of CQRS. Besides the danger of shared mutable objects....

Raising multiple events from a single command is a very common thing. In the end, events must have functional meaning. A single command can have multiple effects.

Cheers,

Allard

--

[António Mota]

OK, I understand now, I'm assuming the Task will be in a state that can actually be modified by a external listener, which rends the "state keeping" function of the Aggregate moot.

What about the  question about the CommandHandlerInterceptor accessing the aggregate?. Is there a way to get the Aggregate using the UnitOfWork in the CommandHandlerInterceptor?

Sorry to insist but I'm working on it right now and a lot of code (actually including some design decisions) are depending on being able or not to do it.

Cheers.

...

[Allard Buijze]

Hi Antonio,

at this moment, the command handler interceptor cannot access the aggregate. If you need to validate anything that requires aggregate state, you must do so within the aggregate itself.

Cheers,

Allard

--

[António Mota]

Hi again...

In relation to what I said earlier

Serialization:
We have to support Postgres with XML data types. That means serialize Metadata, Events and Sagas to XML (or a string representation of) and store then as XML. For that I had to implement a Axon Serializer, extend GenericEventSqlSchema, PostgresSagaSqlSchema and JdbcSagaRepository. Not a big problem here but the code could be more clean with some small changes in Axon itself, if you want to discuss that I'll be happy to do it.

we are having a big concern. At the moment my AxonPostgresqlSerializer is using:

public <T> SerializedObject<T> serialize(Object object, Class<T> expectedRepresentation) {

if (object instanceof MetaData || object instanceof Saga) {
 // use XStream
} else {
 // use JAXB
}

That is because we have control over our own commands/events (the payloads of Message) that are currently annotated with JAXB that guarantees the object will always be dessiralizable . However, we have no control over Metadata and Saga and thus we use the XStream introspection mechanism. But that means if something changes in the Saga implementation, introspection will fail in the future.

Is there any mechanism in place to take care of this? Like a Saga Upcasting or something?

Cheers.

...

[Allard Buijze]

Hi,

the SagaRepository doesn't use upcasters, by default. However, in a similar situation, I have once implemented a wrapper for a Serializer that invokes some upcasters before invoking the wrapped serializer. It's fairly simple to implement and does the job.

Cheers,

Allard