CQRS simplified with WP7, Android, WCF Dataservice, EF
Kallex
We are about to finish publishing the CQRS simplification with a
technology that genuinely raises the level of abstraction.
I would ask for those with little time in hands to wait for few more
days so that we can upload the CQRS specific videos, but for those
pondering especially usage of CQRS with proper OData service and
mobile clients, I encourage to try it out.
For any problems or enquiries I'm glad to help out and answer either
in this thread or through the sites forum.
http://abstractiondev.wordpress.com
I would like to point out that the technology is moreover a
methodology. It's completely open by nature, simple structured use of
T4 code generators combined with XML schema and documentation.
The videos found within the material demonstrate the technology use.
Although they're not yet properly CQRS specified, they do apply alike
to the CQRS abstraction.
Enjoy!
Kalle Launiala
Rinat Abdullin
Kallex
Kallex
Tom Janssens
Positive:
- looks great
- forces you to respect Cqrs principles
- low barrier for people who want to get started with CQRS
Doubts:
- IMHO (and after numerous attempts myself) cqrs does not really need a framework, except for storage and message passing; anything added to that might overcomplicate things (i.e. edge cases/exceptions/modifications/brownfield/...) I have to admit I did not use your app, BTW...
- Similar to the previous doubt: it looks a lot like a 4GL, and I assume we all know what that means...
In conclusion: it looks like you put a lot of effort in a good looking and well polished cqrs framework. I think this would probably be great to introduce people to cqrs, but IMO I think would only advise it for learning projects (for now)
Kallex
Angel Java Lopez
Tom, can you elaborate your assertion "it looks a lot like a 4GL, and I assume we all know what that means..."?
Angel "Java" Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez
Greg Young
Have you watched my talk on occasionally connected clients a while ago at skillsmatter? There isnt really anything to abstract away
Tom Janssens
Tom, can you elaborate your assertion "it looks a lot like a 4GL, and I assume we all know what that means..."?
WARNING: This might be a bit off-topic here, but I do want to clarify my **PERSONAL** opinion on 4GLs.
[AD MODE]As an independent consultant one of my key specializations is brownfield migrations (together with non-communicating teams).[/AD MODE]
Having seen numerous examples, I think most of these homegrown x-year old+ MSAccess or similar applications are usually the perfect example of why one should respect the SOLID principles.
Here a typical evolution of a 4GL app:
There is a business need for a small piece of software (think CRUD client list), which does not need to do a lot. So someone drags and drops a CRUD app together in a few hours/days/weeks depending on the scope... Everybody is thrilled and excited about the custom piece of software, which was developed in a fraction of the time, it is custom built,cheaper,[Add the obvious here],....
Unfortunately, an app is bound to grow more complicated as the business is evolving, f.e. "Could you add products/quotes/","except for this and that this should do that", ...
After some time, such an app is also bound to have "user hacks" (i.e. changing a field's meaning to work around a limitation for example, because it would cost to much to implement the change), so when this specific coworker is on holiday nobody knows how to work with the app...
Educating new users/software devs might also become quite a steep hill to climb.
I usually get consulted by the time the app has been growing for about 3 to 10 years into an unmaintainable behemoth with a massive amount of implicit business knowledge in it, where a single change might break everything or take numerous days to implement,
I do not intend to describe the whole process we need to go through, but I can assure you it takes a lot of time/effort/resources, and requires quite a lot of engagement from both the management and the software team.
A business in this kind of situation needs to handle this problem, better sooner then later, because it is bound to constrain the business growth/evolution sooner or later.
[sarcastic mode]
If you do decide to implement an app in a 4GL, please send me your coordinates, and I will give you a call in about a year or 3
[/sarcastic mode]
Kalle Launiala
Yes now I recognize the meaning. I have had my fair share of trying to
find the "sweet spot" with 4GL, but the compromises and "cutting
corners" always bite back.
These abstractions do not cause compromises. If something is missing
in the logical level, it should be added, as well as the
implementation can be added/modified by chaning the T4 generators that
are available.
The modifications to the schema or generators is faster to do than
revisit/update the example/reference HOW-TO documentation to guide the
developers.
I'll be updating the CQRS specific modifications as well, but the
"Modifying Add-In Abstraction Part 2 - Adding CheckBox Features" shows
the generic "alter architecture and update generators" modification.
Applies as-is to any abstraction modification.
Kalle
Kalle Launiala
Greg, tried to search for your talk, but could you provide a link.
Logically there isn't anything complex, but I guess what I mean with
"abstracting" does not match with OOP "abstracting". "Abstracting" (as
to raise the abstraction level that is) of "occasionally connected
clients" will show to end-developer as simply an additional boolean
"createOfflineSupportForProxy", where T4 generators then create the
local storage for storing commands, as well as handling the
"business-logic-simulation" on client side.
The client is not trusted though, so when connection is
re-established, simply the stored commands are sent back with some
options for collision detection/handling.
Kalle
Greg Young
dependency on a framework it can be handled in 2-3 lines of code. I am
not sure what I am gaining by an additional level of abstraction. This
is sounding much to me like MDA/CASE tools that I have found actually
increase complexity, how is it different?
btw: video is here
http://skillsmatter.com/podcast/open-source-dot-net/cqrs-not-just-for-server-systems
--
Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
de votre attention
Rinat Abdullin
Kalle Launiala
This technology has similiarities with DSLs, DDD + generators,
MDA/CASE tools as well. I have underlined some of the key differences
here. Two largest ones are the practical effectiveness, that allows
multiple "mini-DSL"s that do not require compromises and the fact,
that everything is just source code within the project source code
system. No binary constraints to worry in distribution.
I think you are still a bit stranded of what "abstraction" means in
object oriented programming. This is not a framework, this is totally
platform-agnostic technology to genuinely raise the level of
abstraction. Nothing bound to .NET, Java or any specific programming
language. All you end up is a (well) structured source code, with the
generators right there available in the source form as well.
One moves to higher level of abstraction simply for the reason, that
the amount of information required to specify something greatly
reduces, when you speak in precise terms. The key question indeed is,
what you gain and what you lose by doing it.
Let me compare the abstracted vs. traditional by using the
offline-mobile-client example here:
Traditional:
- You have to provide the example code and guidance documentation for
any chosen implementation.
- Any library such as ncqrs is a 3rd party dependency that is more
difficult to alter than your own code (this applies also to
open-source libraries; although you can include it as in source code,
you end up building even stronger dependency that way)
- When you make the end-developer to write the code as required by the
guidance (if he gets it wrong, his program does not work properly),
you actually also create a logical dependency.
- What if he cannot use some part of your example, although logically
he still needs to get the offline functionality there?
- Perhaps he's writing on another platform where there are some binary
restrictions to use some library?
Also you have little control over keeping the end-developer in the
his/her chosen path. If they miss ceremonial code somewhere, their
program does not work properly.
The way to minimize the technical dependencies (such as libraries)
would be to provide source code in a way that it can be plugged in
as-is to existing project.
Abstracted:
- There is single bit of information (literally)
"provideOfflineSupportForClient" true/false
- The people currently giving the guidance (anyone, including people
such as Greg Young or Udi Dahan) provide the examples in a form of
actual code generators, that implement good reference code in stead of
just giving static example of "this is how you should do it"
- The template simply outputs as a source code and is right there.
- You can alter the template without worrying how much you break
something else, because the immediate 1:1 from template to code is
practically there.
- The change is local by all means, but in case it applies to the
logical abstraction elsewhere, you can branch/merge it to the other
places in your project structure as well.
Abstractions enforce the end-developer to stick to the path. The
architect provides the implementation spots through generating code
hooks in the abstraction; the end-developer is simply complying with
the chosen architecture by implementing them. Everybody benefits from
the fact, that the implementation is strongly enforced by the design.
What about "Tomorrow":
- In abstracted world, the offline "true" is not technology or
platform bound. Simply adding the generators for Windows Phone (C#),
Android (Java/Dalvik), iPhone (Objective-C) and MeeGo (C++), the
end-developer experience remains the same and as-optimal-as-possible.
- Abstraction can be updated by distributing the artifacts; XML schema
and all the generators separate). All source control merging
technologies are in place to support the merge between self-modified
parts and updated parts from the provider.
- In traditional world the guidance needs to be written separately to
every supported platform. Updating the guidance so that the
end-developers get the result requires action from the end-developer
to specifically change his code. In all the places where he applied
the guidance, regardless of how well he understood the code that he
wrote there.
Kalle
Kalle Launiala
story. We have lot of real-world experience backing this up as well;
as soon as one accepts the intellisense-aware XML as a form of
defining your design level, you're good to go.
Kalle
Kallex
Kallex
Kallex
full-scale CQRS stack.
Closing in to the consumer level development, this technical prototype
demonstrates the level of simplicity. There is no compromise, as the
higher level abstraction simply targets the full-scale CQRS
abstraction, that can be used beneath simultaneously.
http://www.youtube.com/watch?v=wn9qt7LMRXU&p=B3366B17004D5DB9
Kalle
Kallex
As you chose to respond and ask for "how this is different", care to
comment on the answers that I wrote out?
I've tried to explain how this is different; we have relatively decent
video stack accompanied with full-open-source downloads.
As you like to refer to your existing videos on how you do things, are
you not humble enough to check ours? :-)
Tom's commenting on this thread was exactly what we have achieved on
this. This is end-developer simplification without compromises - the
high-profile architect loses no freedom or power; rather gains it.
He's the one who alters the existing abstractions as preferred or
makes totally new ones.
To help motivate this to you, would you like this response of yours be
last on this technology, that I claim will change the whole industry:
http://abstractiondev.wordpress.com/2011/06/22/all-the-world-of-difference-linux-developer-ecosystem-equalizing-to-microsofts/
Now I suppose this has been claimed before, but who can you tell the
difference when for once it's for real? I've wrote out pretty well why
this changes the whole deal and why we can put CQRS accompanied with
Event Sourcing everywhere.
I already have bunch of non-software-developers on relatively high
positions for some reason positioned on the opposite side of the table
(because they happened to ignore this methodology when their superiors
asked them to check it out). I really hate to see when technically
highly valued people position themselves as-ignorant and claim
(without a basis) that there is nothing new, when we've more than open
to explain this methodology thoroughly.
Kalle
On 30 touko, 22:35, Greg Young <gregoryyou...@gmail.com> wrote:
> This is all handled through pipes and filters. Without a large
> dependency on a framework it can be handled in 2-3 lines of code. I am
> not sure what I am gaining by an additional level of abstraction. This
> is sounding much to me like MDA/CASE tools that I have found actually
> increase complexity, how is it different?
>
Rinat Abdullin
Kallex
Greg Young
I tried looking at what you had up. The ramp up time to understanding your generation techniques and how to get what i wanted out of them seemed to be low roi for my personal time.
You discuss 'ceremonial' code, can you give concrete examples? Also how does the time saved factor into how much time it takes to learn a tool (as well as risks that said tool may be wrong or later not be able to do something that i want todo). In the same line of thinking what is the overhead for bringing on a new dev?
This is not to say that such a tool is impossible to build. I have seen mda style tools used successfully on projects. I have also seen them fail miserably.
I will also agree with you that such tools do not necesarily increase complexity, they do however have additional knowledge requirements/education requirements. This of course would not be such an issue if your tool were widespread in use and i could put it on a job offer and get back many applicants, but now we have a chicken and egg problem.
The most expensive part of building an application with a'cqrs style architecture' is the figuring out and capturing intent and the modelling of aggregates/getting.language right. The other 20-30% is coding. Of that coding process on the high end you have lets say 20% ceremony (its actually much lower imho). Getting rid of this enormous amount of ceremony will speed up my team by 4-6% and any gains can be easily lost by a single use case running into say boundary problems.
As such i would consider such a tool a.mivro optimization. I would not consider bringing it in until the team was very well versed in building aggregates and determining intent.
Unfortunately no tool that i have seen has been built can help with getting boundaries or use cases right. This is where i worry about getting productivity goals from.
Greg
Please accept apologies in advance for typos as this was written on my phone.
Kallex
Thanks for putting the effort, this is much valued response.
This methodology's strenght is that it is not a tool. Not compromises,
nor constraints. You can use it, then drop it (although there's no
reason to) and your project is unaffected. It's mainstream tool usage
only; XML Schema, XML editor intellisense and Visual Studio T4
generators.
All the outcome is just identical to "manually written code". Where
generators fall short, you simply edit the T4 generator as you would
do with manual code. Or in case there is no competency for that (we
have experience that indicates that mid-skilled C# coders can alter T4
just fine, when they're simply put into position that they either make
the code changes by-hand or learn to use the generator).
In the last "safety-net" for abstractions is, that they just produce
source code. Nothing more, nothing less. Properly made generator does
not even produce shitty code. It's just the code that traditionally
matches the guidance "HOW-TO Code in this project".
The better existing support class structure you already have in your
existing project, the less the code generators have to generate; yet
they still completely replace the HOW-TO guidance for bringing the new
devs and syncing the job for the existing ones.
> You discuss 'ceremonial' code, can you give concrete examples? Also how does
> the time saved factor into how much time it takes to learn a tool (as well
> as risks that said tool may be wrong or later not be able to do something
> that i want todo). In the same line of thinking what is the overhead for
> bringing on a new dev?
Abstraction", as ceremonial code I mean all the code that is
autogenerated to call the method that requires to be hand-written.
That's actually what realizes the abstraction level raise. The
implementor actually focuses on the real-strongly-named implementation-
method (and of course proper submethods in his code) that the higher
level design generators require to be there by the compiler.
This methodology was born to be able to bring in new devs, that have
low-mid technical skills and have to cope with high-end technically
advanced architectures. We were in a situation where all of our times
with new devs would have gone to teach them the way we were doing. At
certain level of skill, you cannot even use the object-oriented
technology (inheritance, interfaces, generics) as new resources simply
might not have the technical skills to cope with them.
In addition traditionally you need to have guidance and understanding
for everyone to produce relatively similar code to each other.
Generators are much less burden to change. For properly created
abstraction, the generators and usually are altered without any affect
on custom code. When distributing the generators through branch/merge,
you can actually update all your applications in a way, you could
never do with manual coding or using binary built libraries.
To summarize
- To use the abstraction, one only needs to be able to enter the
design level information with either intellisense-aware XML or form
that procudes the same XML.
- To alter the generators, one needs to understand T4, that does have
a learning curve, but it's relatively easy for C# developer. If
they're presented the alternative to manually maintain the code for
that area or change the generator, they surprisingly often choose to
learn the T4.
- To alter the architectural abstraction, one needs to understand XML
Schema design. This is also relatively easy, when using proper tool
such as XML Spy. I would argue, that an architect that cannot do this
or doesn't find learning XML Schema design useful, migth not be best
fit for being an architect in the first place.
And this applies to all the abstractions alike. The first time will
take some time, but then its all the same from there.
There is really active dialogue in Software Craftmanship group about
"Communicating design", that abstractions were built to solve for
their main focus.
https://groups.google.com/group/software_craftsmanship/t/98d6500ce3d2386d
Unfortunately I haven't been able to get my replies on that
discussions through as the group's moderators seem to be idle for the
time being.
> The most expensive part of building an application with a'cqrs style
> architecture' is the figuring out and capturing intent and the modelling of
> aggregates/getting.language right. The other 20-30% is coding. Of that
> coding process on the high end you have lets say 20% ceremony (its actually
> much lower imho). Getting rid of this enormous amount of ceremony will speed
> up my team by 4-6% and any gains can be easily lost by a single use case
> running into say boundary problems.
>
> As such i would consider such a tool a.mivro optimization. I would not
> consider bringing it in until the team was very well versed in building
> aggregates and determining intent.
problem domain is and hence the focus should be.
This is where "abstractions" can enforce the focus to be. The
generators doing all the "dirty work", in particular on enforcing the
implementation to be identical regardless of the implementor (as the
lines of code drop below to one-glance-understandable) it makes the
refactoring (when it is needed due to understanding that some of the
design decision indeed needed to be changed).
Abstractions allow design changes with minimal overhead on the coder's
side. When design change is recognized, the total cost on software
development is cut the sooner the design flaw is fixed. In cases of
circumventing it with "fixup code" will practically always cause
higher costs during the relatively short time snap compared to the
life-cycle for the software.
Or say it by other means, design decisions that are going wrong way
have a risk of driving other alike "minor" changes the same way.
Allowing design to drift away will eventually lead into much more
complex oddities. Including in particular guidance for new devs for
logical "oddities".
> Unfortunately no tool that i have seen has been built can help with getting
> boundaries or use cases right. This is where i worry about getting
> productivity goals from.
professional architect, but to ease the job for the architect. When
architect is greatly reduced from hand-controlling the newcomers code
and can spread the technical job properly to technically capable
people (for example making the decision of how the abstraction should
be altered/improved, the senior devs can do the dirty job and the
outcome is exactly what was wanted).
I am currently in process of introducing proper example of how we can
include requirements (customer requirements, performance requirements,
requirement analysis) on the abstraction design phase. It gets
relatively interesting, when we can generate the code with proper code-
contracts (pre- and post-conditions, alongside with profiler or time
constraints for diagnostic builds), that enforce the performance and
other technically designed/measurable requirements.
We have had great experience on communicating with "Microsoft Word"
documentation with customer side that are familiar with documents but
not necessarily with Use-Case tools or Use-Case diagrams. This is when
we can draw full "This is how you do the software development without
personalization and the work absolutely minimized, roles and role
requirements underlined".
You're right on the ball on "Chicken and egg" case. This is what I've
faced with the theory; our experience comes from restricted area
software that we cannot publish as an example. It took a while to
build general-purpose examples for the proper demonstrations. I expect
us to break through with the methodology relatively soon when we're
about to push mobile development + could for any writing capable
logically thinking phone user. Download an app from mobile-marketplace
and start making mobile + clould back-end apps with no compromises,
continue with browser and finally with full-IDE; on the very same
application.
Really hoping to catch interest on few evangelists (that would be
currently the first ones to enter the field in addition to our team
btw) to start getting attention from the developer community as well.
Kalle