Injection-less DCI works (was DCI Language for .NET)

[Trygve Reenskaug]

Hi Rune,
Injection-less DCI has proven a tough nut to crack, yet the solution is simple and obvious. You have suggested two possible roads of attack: One is to do more at runtime. Another is to define the language so that the programmer can't do dangerous things.

I have tried the runtime solution for quite some time now. Several crafty solutions have been tried. There was a problem with every solution. This led to a patch in the compiler and/or the runtime system. More and more patches made the solution less and less trustworthy. It all boiled down to lack of referential integrity. (AKA self schizophrenia). I had to give in. This path only lead to more complicated solutions. It was clearly a case where all I could hope for was what Tony Hoare called "making it so complicated that there are no obvious deficiencies".

Your second road was to fix the language. This works and it's simple. The story is as follows:

• A DCI programmer understands that roles are synonyms for the objects they represent.
• The programmer also understands that RoleMethods are attached to the roles and needs to be given special treatment.
  
• Instance method invocation is specified in the usual manner
    rolename.instancemethod()
  
• RoleMethods are very special in that they are tied to the role rather than the object. They can only be invoked explicitly in the code. The rule is here that a RoleMethod has precedence of an instance method.
         rolename.rolemethod()
• The rolename is a synonym for the roleplayer object in all other cases, including the use of self in various contexts.

 In this simple syntax, a roleMethod overrides an instance method in the roleplayer object, making the latter method inaccessible. It may be a good idea to distinguish between role methods and instance methods by giving role method names a distinguishing sigil (see Wikipedia). This will make he code more readable since it is clear what is intended.

To quote Tony Hoare again, the above syntax solution appears  to be "so simple that there are obviously no deficiencies".

The solution works for my six Squeak examples. The blunders I did when adapting my examples to the new syntax led to obvious error messages and were easy to fix.

I hope nobody finds an obvious deficiency in this solution.
--Trygve

[ant.ku...@gmail.com]

Hi Trygve,

What does injection-less mean?  From the programmers perspective, can role methods be called on role playing objects in the same way that instance methods can be?  Eg:

anObject.roleMethod();
anObject.instanceMethod();

Thanks,
Ant

[rune funch]

Den 24/11/2011 kl. 13.02 skrev "ant.ku...@gmail.com"
<ant.ku...@gmail.com>:

> anObject.roleMethod();
> anObject.instanceMethod();

That's the goal

[rune funch]

That's kind of where I stand and so far my opinion is that it's actually a good thing from a readability point of vies too. If you treat it as a role it's a role otherwise it's any other object. It's not sometimes a role an sometimes any other object which is a potential scenario if you could alias the alias that's the role

Mvh

Rune

[Risto Välimäki]

Hi Trygve,

I like this idea of injection-less DCI. In fact, it seems to have all the good sides of my Role-as-a-wrapper -technique I use with PHP, but with no downsides (in case of PHP-DCIV: a bit odd /misleading syntax when attaching Roles to Objects).

I just want to these two statements:

• RoleMethods are very special in that they are tied to the role rather than the object. They can only be invoked explicitly in the code. The rule is here that a RoleMethod has precedence of an instance method.      

          rolename.rolemethod()

DCI with injection does (unless somehow taken care of) have problem of accidentally overriding Role methods with instance methods or vice versa. Problem is not completely solved if only "RoleMethod has precedense of an instance method".

Problem in depth is this:

Role SomeRole {

   method foo() {

     this.bar(); // no SomeRole::bar() available, so call "object".bar() this time SomeObject::bar()

   }

   method foobar() {

     foo(); // this should always call SomeRole::foo()!!

   }

}

Class SomeObject {

   method foo() {}

   method bar() {

     foo(); // this should always call SomeObject::foo(), and never ever SomeRole::foo()!!

   }

}

When role is used:

SomeRole.foo(); //should call SomeRole::foo() which in turn should call SomeObject::bar()

SomeRole.foobar(); //should call SomeRole::foobar() which should call SomeRole::foo().. etc.

SomeRole.bar(); //should call SomeObject::bar() which should call SomeObject::foo()

I could sum it up that:

1. In "class/data/object" -code "instance methods" are not only preferred over "role methods", but of course "role methods" should not even be available. (real problem with DCI with injection, if not taken care of someway)

2. In Role code Role methods are preferred over instance methods, but also instance methods are available.

3. In Context code Role methods are preferred over instance methods, but also instance methods are available.

Do we have an agreement with above? 

• The rolename is a synonym for the roleplayer object in all other cases, including the use of self in various contexts.

Just want to clarify, that this "self" here probably refers to that language-specific 'pointer' to "this object", so that in Java (or some DCI flavor of Java) it would be "this" and not "self". It's only that I have seen "self" used for either Role or Object and then "this" used for the another.

-Risto

Ps. I have done last weeks software development on soft-PLC (programmable logic controller) with CoDeSys v3 environment (extended IEC 61131-3 system)  with somewhat Pascal style language called "Structured Text" or "ST". They have just added object oriented (or rather "class-oriented") paradigm support to the language. Funny environment. Every "object" (Function Block) is a bit like singleton. You can never add or delete objects or other variables runtime. And with PLC you just pray that your program works forever, and you never divide by zero or something like that. Otherwise that's probably your hundreds of tons of harbour cranes gone wild somewhere.

2011/11/24 Trygve Reenskaug <try...@ifi.uio.no>

[Risto Välimäki]

"I just want to give my 2 cents to these two statements", of course. Sorry for rapid firing without proofreading...

-Risto

2011/11/24 Risto Välimäki <risto.v...@gmail.com>

[ant.ku...@gmail.com]

I'm not sure I understand how this injectionless stuff works.  Can someone explain it for my simple mind?

----- Reply message -----
From: "Risto Välimäki" <risto.v...@gmail.com>
To: <dci-ev...@googlegroups.com>
Subject: Injection-less DCI works (was DCI Language for .NET)

[Rune Funch Søltoft]

2011/11/24 ant.ku...@gmail.com <ant.ku...@gmail.com>

I'm not sure I understand how this injectionless stuff works.  Can someone explain it for my simple mind?

I can tell you have it works in my compiler.

It's essentially a lot of rewritting by the compiler

a definition of a role is 

role RoleName{

    string RoleMethod(){

             this.GetType().Name;

     }

}

that is a role with the name RoleName and a role method called RoleMethod

when the RoleMethod is called it returns the name of the run time type of the object (this incidentally is another scenario that fails with schizophrenia)

if we assign the role to an object we can then write code like

       RoleName.RoleMethod() ;

The syntax is exactly the same as for  invoking an instance method however in my particular case it's rewritten by the compiler to a call to a private static method taking one argument

     <>_RoleNameRoleMethod(RoleName);

This is essentially the same rewrite used for extension methods in C#/VB.NET the rules guiding the overload resolution and scoping is different and the way you defined them is in direct contrast to each other. E.g. extension methods can't be private role methods in Marvin are always private.

The effect of the rewrite is that to the programmer it feels like and looks like an instance method. Essentially there's no difference for non virtual methods. a non virtual method call is rewritten in exactly the same manner it just happens in a latter compiler step. The binary code emitted for a no virtual instance method equals the binary code emitted for a similar static method. The only difference is that in the static case the code reflects that "this" is passed as an argument whereas in the instance case the compiler "secretely" rewrites the method so that it takes an additional first argument, namely "this".
Of course this varies with the compiler but it true for all .NET languages (that's a requirement in the CIL) it's also true for Java and all the C++ compilers I know.

So in the end the role methods actually behaves entirely like non virtual instance methods. Since my compiler works on the .NET platform the IL is different because the verification might otherwise fail.
I've actually tried fooling the IL passing an object of a different type for this. You can get that to work and the IL would then be exactly as for an instance method but very fragile since the verification/runtime might blow up and kill your pet dog at any time

Hope that explains the workings

Rune

[James Coplien]

Time for another paper, Trygve … :-)

Simply said:

The brute-force we make objects smart by 1. going to their class, 2. injecting methods from the role construct into that class and then 3. executing.

Most implementations (e.g., Ruby, Smalltalk) have done something slightly smarter. We make objects smart by 1. creating them from their class; 2. injecting roles into them by adding the role methods to the object method table and then 3. executing.

The problem with these two approaches is that the objects are stuck with all that smartness for now and forever. That means that, in theory, an object could respond to a role method outside of the context scoping the role. Nasty, nasty, nasty.

So instead of putting the smarts inside the object, we put it inside of its call. The object is created as an instance of the dumb data class. The caller invokes a role method on such an object. For the duration of that role method call, that role method becomes, in effect, a method of that object. At the end of the call it becomes dissociated from the object.

This has lots of advantages, such as getting rid of any naming conflict problems between roles (e.g., of the object is simultaneously playing roles in several different open contexts, and some of the role method names clash). And it solves the problem without all the horrors of wrappers.

However, all the role methods from one role still live together in one administrative unit that we call a role. And the object still becomes associated with a role at Context instantiation time (or during the Context's rebind method in the case of recursion). This creates a distinction between association (role to object) and binding (role method to object). 

This is a closer move to something called allomorphism, rather than polymorphism. it is a long-known approach in programming languages.

[Risto Välimäki]

To everyone,

Have a look on a new, incomplete and simplified sheet of known possible drawbacks on different DCI implementation techniques (not languages):

https://docs.google.com/spreadsheet/ccc?key=0Aq2Jhn8JeG3kdDNKYzBGS1lzZ0k2UjAxR0J1SXRuMkE

Feel free to modify the document and add missing possible drawbacks. 

So far it seems that I can't think of too much drawbacks using "Injectionless DCI".

-Risto

2011/11/24 James Coplien <jcop...@gmail.com>

[Trygve Reenskaug]

On 2011.11.24 16:01, ant.ku...@gmail.com wrote:

I'm not sure I understand how this injectionless stuff works.  Can someone explain it for my simple mind?

The following is a long explanation, but I would like to give the whole story. I have actually only hacked the compiler in two places with 12 lines of code in the one and 3 lines of code in the second. These few lines call on my two classes, both being small extensions of existing Squeak compiler classes.

The extensions may be simple, but it took a lot of 'guess and learn' to find them.  This post is not written for simple minds. That's OK, since there are no simple minds on this list. The hacks are meant to be proof of concept. They may hopefully give a  knowledgeable compiler writer some ideas.

In the following, I build on the good old money transfer example. I have trimmed it down to three roles: TransferMoneySource, TransferMoneySink, and Amount. The last is  a method-less role (just a number, in fact). The other two have RoleMethods. This post explains how the compiler translates the RoleMethod source code to the runtime.

The complete example listing is at
    http://folk.uio.no/trygver/2011/SqueakExamples/BB5aBank/BB5aBank.html

------ First, the TransferMoneySink role method: ----

deposit   
    self increase: Amount.


Squeak has a decompiler that reverse engineers a binary method to create a synthetic source code. The decompile tells us what the compiler delivered:

deposit
    self increase: (BB1ContextStack playerForRole: #Amount)

I keep the context stack in what effectively is a global: A class with static variables, static methods and noting on the instance side. (Forget the BB1 part of my class names. It is an artifact I use in lieu of a package facility)

Quite simple, so far. The problem was, of course, to make the compiler generate the above code. There is an Encoder that transforms source code to nodes for the parse tree. Many different node classes; the interesting one is class VariableNode. A VariableNode has a method for generating code for the variable that needs to be fixed for my purpose.

Given a class, an Encoder builds VariableNodes for all variables visible in the class. Now for my hack: I wrote a subclass of VariableNode called (BB1)RoleNode that knows what to generate for me:

(BB1)RoleNode>>asVariable: roleName contextName: ctxNam
    | arg1 |
    comment := nil.
    receiver := VariableNode new
                name: 'self'
                key: #BB1ContextStack -> BB1ContextStack
                code: -4.
    selector := SelectorNode new comment: nil;
                 key: #playerForRole: code: -5.
    arg1 := LiteralVariableNode new.
    arg1 key: roleName asSymbol code: LdLitType negated.
    arg1 name: roleName.
    arguments := OrderedCollection with: arg1.

This magic makes the compiler generate the required code. Don't ask me how that happens, I'm not a compiler guy. I can only say that I made an analog of the superclass VariableNode and it seems to work. That's all.

Now for the Parser. The Parser initializes the Encoder with the message

   Encoder>> init: aClass context: aContext notifying: req
     --some code--

RoleMethods are compiled in the context of Traits. So in my special case, the attribute  aClass in the above message signature is a Trait. I added my own stuff to the method as follows:

Encoder>> init: aClass context: aContext notifying: req
    --some old code--
    " Begin DCI special. "
    ((class isKindOf: BB1RoleTrait)
            and: [class roleContextClass notNil])
        ifTrue:
            [((class roleContextClass collaboratorsFor:
                        (class roleContextClass roleNameFromTraitName: class name))
                        , #(cc))
            do:
                [:roleName |
                    self
                        scopeTableAt: roleName
                        put: (BB1RoleNode new
                                asVariable: roleName
                                contextName: class roleContextClassName)]].
    " End DCI special. "
    --more old code--

Again, it seems to work.

This concludes the compiler hacks that support the illusion of a role name being synonymous with a RolePlayer object.


 If you are still with me, we can continue with

------ one of the TransferMoneySource role methods: ----

run 
  TransferMoneySource transfer.

Note that the transfer-message is sent to the role rather than self because self is ALWAYS the roleplaying object. Here, the programmer explicitly send the message to the Role and thus invokes the RoleMethod if it exists.

The decompiler tells us that that the compiler generates this code:

run
  (BB1ContextStack playerForRole: #cc)
        to: #TransferMoneySource
        send: #transfer
        withArgs: {}

cc is a hidden role that is invisible to the programmer and is used by the compiler . It is always bound to the CurrentContext.

A message to the role is diverged to the CurrentContext where it picks up the compiled RoleMethod: (This is in the context superclass, the programmer does not see it. (This prevents  any funny tricks)

(BB1)Context>>to: roleName send: selector withArgs: argArray 
    | receiver trait compiledMethod |
    receiver := roleMap at: roleName.
    trait := self class roleTraitForRoleName: roleName.
        " A name convention for Traits makes this possible. "
    trait
        ifNotNil: [compiledMethod := trait methodDict at: selector ifAbsent: [nil]].
    compiledMethod
        ifNil: [^receiver perform: selector withArguments: argArray]
        ifNotNil: [^receiver withArgs: argArray executeMethod: compiledMethod]

If the compiledMethod is nil, there is no RoleMethod and we have a normal message send to the rolePlayer object.
If the compiledMethod exists, we have a Trait method and it is executed in the context of the RolePlayer object.This is the key to the execution of RoleMethods without touching the RolePlayer objects or their classes.

What's left is to make the compiler generate the required code. I catch the Parser where it sends a message and add a small hack:

Parser>>messagePart: level repeat: repeat 
    --some existing code--
    " Begin DCI special. "     
    messNodeClass := (encoder classIsTrait and: [receiver isKindOf: BB1RoleNode])
                            ifTrue: [ BB1MessageNode] ifFalse:[MessageNode].
    " End DCI special. "

    --the rest of the existing code--

and now this Parser method does what it always does without knowing anything about my (BB1)MessageNode (A subclass of existing MessageNode).
   

Finally, the following (BB1)MessageNode.method is called from I do not know where:

   (BB1)MessageNode>>receiver: rcvr selector: selName arguments: args precedence: p from: encoder sourceRange: range
       encoder noteSourceRange: range forNode: self.
        ^self
            receiver: (encoder  scopeTableAt:  #cc)
            selector: #to:send:withArgs:
            arguments: (OrderedCollection
                with: (encoder encodeLiteral: rcvr roleName) 
                with: (encoder encodeLiteral: selName)
                with: (BraceNode new elements: args))
            precedence: p
            from: encoder.

Almost identical to the corresponding superclass method, only it generates what I want rather than its usual stuff.

That's all, folks. I hope you enjoyed it.

[James Coplien]

This concludes the compiler hacks that support the illusion of a role name being synonymous with a RolePlayer object.

Many of the discussions leading up to this one indicated that a small language change was in order: to change the semantics (and / or maybe the syntax, too) of a Role method invocation from that of a Data object invocation:

On Nov 22, 2011, at 5:39 , Trygve Reenskaug wrote:

 I cannot see any way around it without making the distinction visible to the programmer.

The quoted text at the start of this mail suggests to me that you did not do that — a single syntax suffices.

Am I right? If so, I am happy. If not, I have more questions.

[rune funch]

Den 04/12/2011 kl. 15.44 skrev James Coplien <jcop...@gmail.com>:

This concludes the compiler hacks that support the illusion of a role name being synonymous with a RolePlayer object.

Many of the discussions leading up to this one indicated that a small language change was in order: to change the semantics (and / or maybe the syntax, too) of a Role method invocation from that of a Data object invocation:

On Nov 22, 2011, at 5:39 , Trygve Reenskaug wrote:

 I cannot see any way around it without making the distinction visible to the programmer.

The quoted text at the start of this mail suggests to me that you did not do that — a single syntax suffices.

Am I right? If so, I am happy. If not, I have more questions.

In my case it's syntactic sugar. I'm using the same syntax for all method calls RoleMethods and instance methods. The compiler rewrites the former into a call to a static method private to the context

[James Coplien]

On Dec 4, 2011, at 5:49 , rune funch wrote:

In my case it's syntactic sugar. I'm using the same syntax for all method calls RoleMethods and instance methods. The compiler rewrites the former into a call to a static method private to the context

I don't understand. "Syntactic sugar" usually implies that there is a difference (the sugar) that distinguishes one case from another, but you say your cases are the same. (I am assuming that this is also true in the case of Trygve's new Squeak implementation.)

What, specifically, is the "sugar"? Or is it "semantic sugar" while being syntactically identical?

Maybe an example would help.

[Rune Funch Søltoft]

It is a rewrite that changes what looks like an instance method invocation into an invokation of a static method. The mechanism is the same as the one used for extension methods in C# which is usually referred to as syntactic sugar. Which could be said to be misguided since it does actually change the semantics. so in my case it's syntactically identical.

2011/12/4 James Coplien <jcop...@gmail.com>

[James Coplien]

On Dec 4, 2011, at 10:31 , Rune Funch Søltoft wrote:

so in my case it's syntactically identical.

+1.