FullOO (a.k.a. DCI) Summit in Oslo

[James O. Coplien]

Cope went to visit Trygve for about three days in the beginning of August and, as usual, there was a lot of energetic discussion. In addition to the usual fun smoke and heat, some light also came out. Here are the major upshots of the week.

Abstraction Layers and the difference between Restricted and Full OO

Alan Kay coined the term "object orientation" and likened an object system to a system having thousands and thousands of computers all hooked together by a very fast network. That perspective grounds some of what follows here. Think of computer hardware as a programming API. It provides an abstraction layer. Trygve and Cope define an abstraction layer as a shared understanding of the essential behaviors of an object as seen from a client, while the details of the implementation are hidden. We want to follow the #1 rule of software design: no surprises. That implies that the hardware can't reach up into the software and do unexpected things. If the garbage collector leaks, then the hardware doesn't implement a good abstraction layer. It also implies that the software can't do things to the hardware. You can generalize this concept to layers of software, so the operating system also provides an abstraction layer. 

Trygve is now talking about "Restricted OO" — which is OO-as-we-knew-it-before-DCI with restrictions that makes the code readable. This in contrast to "Full OO" which DCI implements. Maybe there will be other ways to implement it, too. There is a simple but stunningly powerful difference between the two and we spent much collective effort on deepening our understanding of those perspectives. The difference is in the programmer's observation point.

Restricted OO

With Restricted OO, the programmer observes the system from the inside of an object. The class defines what the programmer can and cannot see. A programmer can see:

• The state of an instance as defined by its class attributes.
• What messages an instance can receive.
• The methods that will be triggered by those messages.
• The attributes that are changed by those methods
• The messages that are and the types of the objects that will receive those messages.

There are many things the programmer cannot see by reading the class code. Examples:

• The sender of a received message and the purpose of this message seen within a wider context.
• The class  of the receiver of a message sent from an observed object. 

The reason for these limitations is that this receiver lives within an abstraction boundary. The programmer cannot know, at compile time, which behavior will be invoked at runtime – by design.  A programmer can understand the program behavior only in general terms.

This is an essential part of the "power" of object orientation.

Unrestricted programming with classes — which is OO-as-we-knew-it-before-DCI – can lead to unreadable spaghetti code. We call it  “Unrestricted OO”.

We spent quite some time discussing the restrictions needed to get readable code. It appeared to have something to do with the abstraction boundary between a client and the inside of an object. Perhaps it would be illegal for an execution to cross back up across the boundary to do unpredictable things to the client. We did not conclude and the work continues. The result shall be a rule that is the “restricted” in Restricted OO. We hope that Unrestricted OO some time in the future will be relegated to history.

Full OO

With Full OO, the programmer observes the system from the space between the objects. Programmers can see several objects at once and observe the interaction messages that flow between them. They can’t see inside the objects because each lives behind its own abstraction layer boundary. They can't even know, at compile time, which behavior would be invoked at run time — by design.

DCI implements system behavior by pulling together the behavior of a system operation into several co-located roles, each representing an object. The programmer can see the roles together and reason about their end-to-end behavior. The system behavior is explicitly given by the RoleMethods that are outside any abstraction boundary. There is still an abstraction barrier: between the roles and the objects that play those roles. This barrier has the usual restriction: The object implementation can't reach up into the Interaction and do unexpected things. However, it is a very narrow abstraction layer whose operations should be viewed as primitive, and which, by design, always return. To be able to reason across a sequence of mutually invoking roles means to be able to reason about several objects at once.

Define a compression as a way to represent that information in fewer bits of data than the amount of information it contains, using contextual knowledge that makes it possible to losslessly expand that representation into the original information. In Restricted OO, a programmer cannot compress the understanding of anything: everything is an abstraction, except the current method. There is a large lack of contextualization. In Full OO, RoleMethods are compressions.

In DCI terms: Data classes are coded with Restricted OO. Interactions are coded with Full OO.

Restricted OO is very much like the Atomic Event Architecture of the Lean Architecture book (e.g., a traditional "shapes" OO program, with very simple operations); Full OO is like DCI.

We spent a lot of energy and time on these concepts and feel they will prove useful when worked out in detail.
 

Handoffs

We had a few discussions about handoffs. In Lean architecture, you want to avoid handoffs, so there should be no major (human) communication boundary between Context code and Data code. This means that the writer of the code for a Context with its roles and role methods should cooperate closely with the writer of the associated Data classes to adapt the abstractions to the needs of the system as a whole.

On the other hand, there are abstractions within abstractions and the transparency of the abstractions are reduced down the abstraction hierarchy. (There is no hard-and-fast rule; the developers of the first Smalltalk runtime system created private microcode for their Alto computer to optimize their system). Theoreticians will claim that any interface to an object must be supported by precise definitions of its operations with preconditions, postconditions etc.

In some sense, the whole purpose behind the DCI encapsulation of roles is that they are all in one place where they can be considered together without going across an abstraction boundary. But with a role method saying self–>something, we cross an abstraction boundary and must rely on it being implemented correctly. (Otherwise, the code of the RoleMethod would be unreadable).

Abstraction boundaries, therefore, are an agreement based on mutual trust. They are maybe a way of encapsulating change at some level. But that isn't a reason to really "hide" the changes. Everybody, all together, from early on — on everything.

So abstraction boundaries, rather than being viewed as "don't look here" fences, draw attention as key points of agreement around crucial architectural issues that the abstraction boundaries bring into focus.

 
Context / Role Integration

We agreed it would be a good idea if every Context could play some kind of Context role. That would make it possible to view the Context as a role from inside itself, and as an ordinary object from outside itself, thereby providing a smoother impedance match across the Context abstraction layer boundary. The Context could be a repository for state common to the whole interaction and visible to  RoleMethods through its role name, standardized at e.g., CurrentContext. This idea needs more work. For example an implementation — the true killer of good ideas.

 
DCI and Systems Complexity

We talked a little about simple, complicated, complex and chaotic systems (Snowden's taxonomy). DCI is interesting because it can implement an adaptive system. It responds to changes in its environment, implementing an open system that has a degree of self-organization. While Restricted OO systems are also open systems, they aren't adaptive.

DCI does this by effectively implementing a weak form of reflection.

 
Articles

Trygve gave Cope comments on his Elephant article, first suggesting that it be re-written from scratch with more precisely defined concepts. He has later withdrawn the suggestion since he realized that the entertainment value of the current form would be lost with very little gain. Cope continues to refine the article, and won't do a major upheaval.

Web Site

We have bought FullOO.info on a Norwegian web hosting site. Cope is supposed to set that up but is a bit lost right now, partly because their setup is a bit mysterious. (They already sold us oo.info and after a week of not being able to make it work they admitted a bug in their system that allowed them to sell us a URL that had previously been allocated to someone else.) The main issue seems to be that though he purchased a URL and 8 gigabytes of storage, we still don't appear to have anything with an IP address. There is something wrong with the hosting site. Trygve will  phone them (in Norwegian) and expect they'll fix the problem quickly. Then we can get down to the much wanted Q&A section and collect other material that should go to the site.
 
 

[Ant Kutschera]

Sounds good. Nice to have this so far mysterious term "Restricted OO"
defined clearly.

One comment: The section "DCI and Systems Complexity" left me going
"huh?". Maybe because I don't know Snowden's work. I didn't
understand what you meant by "implementing a weak form of reflection".

Cheers,
Ant

[James O. Coplien]

On Aug 20, 2011, at 8:33 , Ant Kutschera wrote:

> Sounds good. Nice to have this so far mysterious term "Restricted OO"
> defined clearly.
>
> One comment: The section "DCI and Systems Complexity" left me going
> "huh?". Maybe because I don't know Snowden's work.

Maybe. Snowden is a good starting point for a widely cited theory of complexity. This is kind of a tangential gloss on DCI.

> I didn't understand what you meant by "implementing a weak form of reflection".

That has nothing to do with Snowden, and the relationship to complexity is, well, too complex to dive into here. It is not just a tangential gloss, however, but is central to what DCI is about. Reflection is the ability of a system (as expressed in the definition of a programming language) to reason about itself, usually at run time. Most symbolic languages are said to be reflective. One of the main uses of reflection is to be able to play with type associations at run time: to be able to change the behaviors of an object by adding and removing behaviors. DCI depends on a constrained form of reflection as implemented by the contexts.

Reflection allows the programmer to do metaprogramming. The Context is a form of metaprogram that is used to re-wire objects and roles to create a new program on-the-fly in response to an event in some environment of objects.

Metaprogramming has a long and storied history, particularly in the Lisp community. Metaprogramming alone is insufficient for DCI, which is why so many metaprogramming approaches figure prominently in the Elephant article.

[Ant Kutschera]

OK, that's clearer, thanks. (I wasn't sure you were talking about the
kind of reflection I know, use and which is well supported in Java, or
whether you were talking about something more deep rooted or
philosophical). Sadly, Java's reflection doesn't allow you to add
methods on the fly to existing objects - how frustrating!

Cheers,
Ant

[James O. Coplien]

On Aug 21, 2011, at 9:20 , Ant Kutschera wrote:

> Sadly, Java's reflection doesn't allow you to add
> methods on the fly to existing objects - how frustrating!

For this (and other deeply related reasons) I still claim that you can't do DCI in Java. You can do something that kind of looks like it with wrappers, but fundamentally you get stuck. You need to add a layer of language above it, as Qi4j does.

[Ant Kutschera]

Yep - Java fails DCI certification, it's true.

[Ant Kutschera]

On Aug 20, 6:21 pm, "James O. Coplien" <jcopl...@gmail.com> wrote:
> We want to follow the #1 rule of software design: no surprises. That implies that the hardware can't reach up into the software and do unexpected things.

> ...

> Perhaps it would be illegal for an execution to cross back up across the boundary to do unpredictable things to the client.

I have a question: what does this mean to observer/observable, and MVC
models which notify attached observers? I have some code at work
which surprises me everytime I debug it, and it's due to poorly
implemented MVC and crossing that boundary in the wrong direction.

And while Restricted OO might have a rule to forbid such crossing back
up across a boundary, will the same rule apply to full OO?

[Trygve Reenskaug]

Restricted OO is for dumb, dumb Data classes. The restrictions in restrictedOO shall guard against surprises. Anything involving the collaboration of two or more objects needs full OO - the system observed from the inter-object space where you can see all participants together. MVC is an obvious solution if you think in terms of collaborating objects. That's why I refused to publish it at the time. MVC seems to be unbelievably complicated to people who are restricted to think in terms of classes.
--Trygve

--

Trygve Reenskaug       mailto: try...@ifi.uio.no

Morgedalsvn. 5A         http://folk.uio.no/trygver/

N-0378 Oslo               Tel: (+47) 22 49 57 27

Norway

[Ant Kutschera]

Hi Trygve,

I'm not sure I fully understand how your reply relates to my question,
but I think I understand you.

I'll put my question slightly differently.

If I have a system which has several views, and each is attached as an
observer on the model, and that model changes, the views are notified,
but in a decoupled way. The observable does not need to know anything
about the observer, just that it should be called (using a standard
method call), when a model change is made.

Am I interpreting it wrongly, if I say that a notification from the
observable back to the observer, is in the wrong direction, as it
crosses the boundary in the wrong direction?

Thanks for clarification,
Ant

> Trygve Reenskaug       mailto: tryg...@ifi.uio.no

[Trygve Reenskaug]

Hi Ant,
The essence of the Observer pattern is that more than one object has to be taken into account to understand what's going on. A callback across an abstraction boundary is clearly an example because it involves the collaboration of two peer objects. FullOO is required to fully describe what happens at runtime.

The Observer pattern is a very interesting example because the interaction algorithm is in two parts separated in time: "register dependency" and, later, "update observer".  A single sequential program is not the solution. Perhaps the Observer pattern could be used get us started on something that describes what happens in the inter-object space over time.

This is one of many issues that still need to be investigated with DCI. Fortunately, there are lots of fun projects waiting in the future.

--Trygve

--

Trygve Reenskaug       mailto: try...@ifi.uio.no

Morgedalsvn. 5A         http://folk.uio.no/trygver/