Clean Architecture: too many layers of indirection?

[Reed Law]

As the saying goes, "All problems in computer science can be solved by another level of indirection...except for the problem of too many layers of indirection."

Clean Architecture solves many problems such as difficulty of testing business logic, too many responsibilities in one class, and the tendency to write spaghetti code in MVC applications. But if you understand those problems and can avoid them by disciplined coding, does it really offer concrete benefits? Having written a large application with Clean Architecture I've learned to appreciate the TDD approach to business logic and enjoy the speed and simplicity of OOD especially within Entity classes. But when it comes to transforming data into Entities and Entities into plain data objects, it's not so much fun. It almost feels like I'm rewriting parts of ActiveRecord. For instance, to instantiate an Entity from a Repository class I need a Builder that knows all the attributes of the Entity and knows how to transform the query results into one or more entities with associated child objects. Once we have a collection of entities it becomes quite easy to get the results needed by the Use Case. But then the entities have to be transformed into plain data objects. That means a Serializer class, json templates, or some such mechanism to ensure the views have only plain data. Serializers need to know which attributes and child objects to serialize (which seems to duplicate a lot of Builder code, or relies on reflection of Entity classes). On top of that, we need presenters to transform dates and do other formatting (or is it the presenter's job to serialize entities?).

So now we have Repository -> Builder -> Entity -> Interactor -> Serializer -> Presenter -> Controller -> View. The main benefit I see is a lot more control over every stage. Sometimes this is useful but many times you simply want what ActiveRecord provides out of the box. Sometimes I wonder if we moved all our interactors into controller actions and all the business logic into models what would we lose? We wouldn't need builders or serializers. We'd have to be diligent not to call business logic methods in the views.

Rails does have a lot of frustrations due to the amount of "magic" it performs. I'd like to have only one db query per action, but if you're not careful it's easy to get N+1. But it is very convenient when you stay within its conventions. What I'd really like is an alternate set of conventions--"Clean Rails"--that makes it easier to isolate business logic and separate responsibilities. Maybe by the time we're done with this application this new framework will have emerged, but there are still a lot of questions and uncertainty about what constitutes an improvement and what might actually be "test-induced design damage".

My question is: am I doing this wrong? I can see the benefits but I still feel a way off from declaring Clean Architecture a total victory. I know there's no silver bullet. There are a lot details to consider for each application. I keep hoping for a large open source project to come out as an exemplar of Clean Architecture so I can study it in detail.

[Norbert Nemes]

Hi

I know exactly how you feel. :)

It's strange though that this post comes from a Ruby guy. I was starting to think that people involved with clean architecture chose Ruby on purpose as their weapon of choice, because it has no interfaces (= less code to write). Consider another language, like Java or Objective-C and add the overhead of writing interfaces as well for everything. ;)

Usually when I show a clean(ish) project to somebody else the first thing they say is "OMG that's a lot of files!".

But lately, I'm coming to the conclusion that a "true" object oriented / clean project contains a lot more classes/parts that the common coder is used to and/or comfortable with. And depending on how crufty the language it's written in is, things can get worse.

However, I think complaining about the number of classes/files in the project might be akin to complaining about a library having thousands of books. 

Of course, if they're all on the floor, that's a nightmare, but if you can quickly find what you're looking for, who cares?

Hope this helps.

[Christian B]

Uncle bob made a project in java in his CleanCodeStudy on Github. Well, he did not have any presenters, in fact, his interactors were returning the response, but in clean architecture, the interactor would give the response to the presenter. I think you can leave some things out, and later, if u need, you can refactor easily. You have to consider the tradeoffs when u are doing things. I personally dont use any presenters in web applications in php/ruby, because i know for sure, there will be no desktop client or something like this. In Java it could be different, you could have an android app, or a spring web application. It depends on the requirements a bit aswell IN MY OPINION.

[Julian Seeger]

About ActiveRecord:

I think you are using ActiveRecord as a synonym for "something that creates my entities from the DB response". There is no Reason to not use something like an ORM that does all that for you. So you don't have to reinvent the wheel. It's just not ActiveRecord because that makes your Entities extend ActiveRecord ... and so makes your BusinessRules dependent on a third party lib (as long as you pass your Entities through the GatewayInterface).

About the Serializer:

I don't take the argument "the view should not be able to call businesslogik" as an argument for not using the real entities because if the view would be acting evil, it could instantiate an Entitiy, open a database connection and be evil anyways ^^. Passing the Entitiy to the Presenter has nothing to do with passing the Entitiy to the View with a separation of ResponseModel and ViewModel, imho.

But basically that was what I wanted to clear with https://groups.google.com/forum/#!topic/clean-code-discussion/driBfMeixzM and I take the silence in that thread as a silent aggreement. The GatewayInterfaces should be a boundary just like the PresenterInterfaces. And as you can see here https://github.com/cleancoders/CleanCodeCaseStudy/blob/master/src/cleancoderscom/CodecastGateway.java is seems to be ok for UncleBob to pass Entities through those boundaries. So I see no reason why you should not pass Entities to the Presenter (as long as the presenter doesn't pass them to the view) because in most applications I know, the Presenter works like an Adapter between your application and some third party templating engine (what makes the presenter part of your code and totally ok to depend on the entities).

So I've come to no longer argue with anyone about Serializers and Builders (to take your wording) and use something like this:
Repository returns Entities. Interactor passes a ResponseModel (which might contain Entities) to the Presenter. Presenter passes a ViewModel (which does not contain Entities) to the View.

The Repository, the Interactor and the Presenter depends on the Entities, but the view does not.

Perhaps you have to see this just like the SOLID principles. Everytime you violate the complete indirection and separation by passing an Entitiy directly and not only some representation of it's state, there should be a little voice telling you that what you are doing is dangerous and probably wrong. And if you can avoid it with a justifiable amount of work, you should to better. But if you have more work writing all those Builders and Serializers, then with the two lines of business logic itself, it may be worth the tradeoff. ... anyways, don't use ActiveRecord ;)

[Norbert Nemes]

Hi Julian

I just read your post (linked above) and had an Eureka moment.

I think you are totally right!

Gateways and presenters are simply adapters between the application and as such should be treated equally. 

It is ok for entities to be passed on to that layer, with the condition that NOBODY calls any business logic on them there.

This would prevent the overhead of writing serializers and such. However, if you are afraid that one day some noob might do it anyway, then it would be advisable to create factories that would return state objects based on the entities.

[Julian Seeger]

Nice to hear that ;)

I'm still not sure, how to create these DTOs from entities if needed. Like I said, writing all those getters only to make the object serializable can't be the solution. Maybe you could ask the Entitiy to provide the DTO itself like that: user.fillState(UserState dto).

Then, the PresentableUser (or whatever your Boundary-DTO is called) could be an Adapter between the PresentableUserInterface and the UserState. That way the "DTO" doesn't need a serializer and works more like a proxy ... but I'm probably missing something ^^

[jk]

In Rails applications, I tend to take similar shortcuts that Julian mentioned.

I think the formal definition of clean architecture has so much indirection because of the focus on independent deploy ability. (See the component case study clean code episode). Splitting the application up into components takes it to the next level. (I have not tried this in rails). With that in mind....

By using entities directly inside a presenter, a change in the entity component would impact the presenter component. In a compilable language like C++ this would require the presenter component to recompile... Even if the change in entity had nothing to do with how it's presented/rendered. This seems weird and unnecessary.

Soo when you have components/dlls/individual gems -- the data structure transport makes more sense. If all the classes of your application are together--- it seems less worth-while.

[Norbert Nemes]

I gave it some more thought after a night of coding (and drinking) and it seems to me that since you do not want to pass an entity straight to the UI level, you would have to do the conversion anyway. Either in the presenter or the interactor. So why not do it in the interactor and just pass the DTO to the presenter?  

As for generation, your idea about having the Entity do it sounds interesting.

But I would do it in a slightly different way.

In objective-c, classes usually have a method called "description" that returns a string description/representation of the class.

So following this example, why not create a method like user.getState() that would return a UserState object with all the fields of the user but none of its methods?

UserState is a base type of your system that describes the User right, so having User know about user state seems ok. Also, the only time UserState would change is when User changes so...

[Norbert Nemes]

Or even better! 

Compose user with a UserState class and return that :) All the User class' methods would act on the UserState class!
user.getState would return the UserState!

[Caio Fernando Bertoldi Paes de Andrade]

Norbert, I think you’re on the right track.

I believe entities don’t need to have direct state, meaning they can hold a reference to the DTOs and operate their rules on them. This way when you need the DTOs, you can just ask the entity for them, and when you need an entity instance, you can just hand the DTOs to the constructor/builder.  No serialisers and no complex builders. ;-)

Caio

--
The only way to go fast is to go well.
---
You received this message because you are subscribed to the Google Groups "Clean Code Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clean-code-discu...@googlegroups.com.
To post to this group, send email to clean-code...@googlegroups.com.
Visit this group at http://groups.google.com/group/clean-code-discussion.

[Julian Seeger]

> you would have to do the conversion anyway. Either in the presenter or the interactor. So why not do it in the interactor and just pass the DTO to the presenter?  

To that point, I must disagree. While the InteractorResponse must contain all information of the entity (because it should not know what information is shown in the view), the ViewModel (the thing that is passed to the view by the presenter) should only contain as few information as possible (becuase the presenter knows what the view needs).

While an Entitiy fits to the needs of a ResponseModel (contain all information), neither does it fit for a ViewModel (as few information as possible) nor does a ResponseModel fit for a ViewModel.

> In a compilable language like C++ this would require the presenter component to recompile..

It's also about independent developability, I think. If one team develops on the Entity-Side and the other deam develops some themes (on the presenter-side) they don't interfere. But it is a tradeoff you probably only need in a team of over 15 people, isn't it?

[Norbert Nemes]

@Caio

After thinking a bit more about it, it might not be a good idea to pass out the DTO as is, because if someone changes anything on it, those changes are propagated straight back to the Entity, essentially circumventing its (application independent) business rules. So perhaps passing out a deep copy of the DTO would be in order.

@Julian

The DTO would contain all the information of the entity. And that would get filtered out by the presenter, as you said.

It would just mean you would have to create yet another DTO for that. Which could be a pain.

So here is my compromise proposal:

Entity should not return DTOs. The Interactor would create (or have a factory create) a DTO appropriate for the use case in question. This DTO will be populated with the minimum amount of information required by the use case. The presenter would create a structure for the view using this minimal DTO that contains any additional UI info needed by the view. So there will be 1 DTO / use case.

For example (shopping app):

I have an Item entity with all sorts of fields. 

The lisItems interactor gets the list of items and creates an array of DisplayItems. Each DisplayItem data structure contains only the information that is necessary to identify and/or display an item (itemId, name, price). The interactor passes this array of DisplayItems to the presenter.

The presenter creates an array of Sections. Each section object contains information like header title, color, and a list of DisplayItems that need to be displayed in each tableView section. It passes this array to the view.

The view just updates itself based on this information. 

What say ye?

[Caio Fernando Paes de Andrade]

After thinking a bit more about it, it might not be a good idea to pass out the DTO as is, because if someone changes anything on it, those changes are propagated straight back to the Entity, essentially circumventing its (application independent) business rules. So perhaps passing out a deep copy of the DTO would be in order.

But if the Entity holds only immutable DTOs, you don't have that problem anymore. ;-)

Of course the purest approach would be to have different (and independent) DTOs for persistence, representation and presentation, but since we're trying to reuse them without compromising the architecture, making the representation DTOs immutable would allow us to reuse them both in presentation and persistence without too much trouble.

Caio

Sent from my iPhone

[Israel Fonseca]

@caio
I think that by 'changes anything' he mean 'change the name of a DTO attribute', and that would affect everybody attached directly to it.

@norbert

I like your proposal, just a question about your presenter, and it could be a noob question. Why the presenter creates another object (Section), and pass it thought the view? Is there any gain on that? Shouldn't the DisplayItems be already enough?

// webapplcation.code

@route('listItems/')

   displayItems = listItemInteractor.execute()

   render('listItemsTemplate.html', displayItens)

Any problems with the above pseudocode? The template engine would be the presenter itself (in this web example).

[Norbert Nemes]

@caio
Some languages would allow you to create such immutable DTOs, unfortunately , Java and objective-c are not such languages.
Plus, if you have an immutable DTO, how would the entity methods operate on it? Would you create it anew every time? Again, it depends if the language allows you to create such a thing.

@Israel
It's something Uncle Bob said that keeps ringing in my ears. At one of his many conferences, he described the clean architecture and he said that interactors return data structures to the presenters, but the presenters do not hand the result straight to the view. Instead they organize the data into a form that makes it completely brain-dead for the view to present.

In my case, the items are presented in a tabular form in a UITableView. Such a table view consists of sections and each section has cells. Sections can have special views that can present information (something like table headers).
So an array of Section objects containing DisplayItems (that will be rendered in cells) is the most brain-dead way to pass the data to the view.  Sections might also contain the background color of each section header, title, and so on.

The presenter creates this structure, because only the presenter knows how the information received from the interactor(s) should be presented.
Of course, if your view only consists of an array of DisplayItems, then passing them on from the presenter without any modifications would be the right way, I think.

[Israel Fonseca]

Got it Norbert! I still need more field experience witch clean architecture to see the edge cases, and reason about. Glad to have this list to talk about. :) 

[Norbert Nemes]

Me too :).

I'm a noob to the whole thing myself and there still are quite a few spots that I'm not quite sure about.

Hopefully this forum and Uncle Bob's Java case study will illuminate them :)