[groovy-user] field preferred to getter? Shouldn't it be the other way?

[OC]

Hello there,

hunting for an ugly "why does not an overridden getter work properly" bug I've just bumped into this:

334 /tmp> <q.groovy
class qq {
String foo='foo'
String getFoo() { "${this.@foo} through the getter" }
def test() {
println "Internal, direct all right: ${this.@foo}"
println "Internal, should use getter, does not: ${this.foo}" //*
println "Internal, should use getter, does not: $foo" //*
println "Internal, seems the only way to get getter: ${getFoo()}"
}
}
def f=new qq()
println "External, direct all right: ${f.@foo}"
println "External, getter all right: ${f.foo}"
f.test()
335 /tmp> groovy q
External, direct all right: foo
External, getter all right: foo through the getter
Internal, direct all right: foo
Internal, should use getter, does not: foo
Internal, should use getter, does not: foo
Internal, seems the only way to get getter: foo through the getter
336 /tmp> groovyc q.groovy&& java -cp .:/Users/ocs//Downloads/groovy-sdk-2.1.1/embeddable/groovy-all-2.1.1.jar q

Is this really the presumed and proper behaviour? It would seem to me both the OO approach and plain consistence would rather dictate to use getters in the //* cases, but as always, I can be overlooking something of importance.

Thanks,
OC


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

[OC]

Just a follow-up, the same problem...

On Mar 13, 2013, at 1:42 PM, OC <o...@ocs.cz> wrote:

> Is this really the presumed and proper behaviour? It would seem to me both the OO approach and plain consistence would rather dictate to use getters in the //* cases, but as always, I can be overlooking something of importance.

... here it's even more pronounced:

339 /tmp> <q.groovy

class qq {
String foo='foo'
String getFoo() { "${this.@foo} through the getter" }
def test() {

def what=this.class==qq?'Internal':'Inherited'
println "$what this.foo, should use getter, does not: ${this.foo}"
println "$what foo, should use getter, does not: $foo"
println "$what getFoo, seems the only way to get getter: ${getFoo()}"
println "$what @foo, direct all right: ${this.@foo}"
}
}
class qqq extends qq {
String getFoo() { "overridden in qqq" }
def test2() {
println "Subclass this.foo, gravely inconsistent: ${this.foo}"
println "Subclass foo, gravely inconsistent: $foo"
println "Subclass getFoo, seems the only way to get consistence, but ugly as hell: ${getFoo()}"
println "Subclass, direct crashes, triple weird considering //* works!: ${this.@foo}"
}
}
def f=new qq()
println "External, direct all right: ${f.@foo}" //*

println "External, getter all right: ${f.foo}"
f.test()

(f=new qqq()).test()
f.test2()
340 /tmp> groovy q

External, direct all right: foo
External, getter all right: foo through the getter

Internal this.foo, should use getter, does not: foo
Internal foo, should use getter, does not: foo
Internal getFoo, seems the only way to get getter: foo through the getter
Internal @foo, direct all right: foo
Inherited this.foo, should use getter, does not: foo
Inherited foo, should use getter, does not: foo
Inherited getFoo, seems the only way to get getter: overridden in qqq
Inherited @foo, direct all right: foo
Subclass this.foo, gravely inconsistent: overridden in qqq
Subclass foo, gravely inconsistent: overridden in qqq
Subclass getFoo, seems the only way to get consistence, but ugly as hell: overridden in qqq
Caught: groovy.lang.MissingFieldException: No such field: foo for class: qqq
groovy.lang.MissingFieldException: No such field: foo for class: qqq
at qqq.test2(q.groovy:18)
at qqq$test2.call(Unknown Source)
at q.run(q.groovy:26)
341 /tmp>

[Cédric Champeau]

This is by design. Internal references to field go direct, not using the
getter. Imagine you have the following code:

class Foo {
String field
String getField() { field.toUpperCase() }
}

Then you would create an infinite loop. Of course, you can say that you
can use the attribute notation, but this would make a lot of code less
readable, so the choice was made to use fields instead of getters from
inside the class.

Le 13/03/2013 13:42, OC a �crit :

--
C�dric Champeau
SpringSource - A Division Of VMware
http://www.springsource.com/
http://twitter.com/CedricChampeau

[Dinko Srkoc]

On 13 March 2013 13:53, OC <o...@ocs.cz> wrote:
> Just a follow-up, the same problem...
>
> On Mar 13, 2013, at 1:42 PM, OC <o...@ocs.cz> wrote:
>
>> Is this really the presumed and proper behaviour? It would seem to me both the OO approach and plain consistence would rather dictate to use getters in the //* cases, but as always, I can be overlooking something of importance.

I believe that, for better or for worse, this is really the presumed
and proper behaviour. As far as I remember, one of the reasons for
introducing this (in Groovy 1.5 perhaps?) was the following case:

class A {
def foo

// let's override the setter
void setFoo(bar) { foo = bar + 1 }
}

Without this feature, the above code would lead to StackOverflowError,
and that was enough common case that people took notice. So, the rule
is, more or less: inside a class, `foo` is a field (the same as
`@foo`), outside a property.

As far as inheritance goes, if the above rule is taken strictly, i.e.
it doesn't apply to subclasses (rationale - if `foo` is a private
field, than the subclass should not have access to it anyway, so `foo`
is then a property), all is fine. Well, almost all, since if one can
access object's private parts from the outside, why not from the
subclass (the case of `@foo`)?

Of course, as you say, I can be overlooking something of importance too.

Cheers,
Dinko

[Ondřej Čada]

Cédric,

On Mar 13, 2013, at 2:40 PM, Cédric Champeau wrote:

> This is by design. Internal references to field go direct, not using the getter. Imagine you have the following code:
>
> class Foo {
> String field
> String getField() { field.toUpperCase() }
> }
>
> Then you would create an infinite loop.

Well, in my eyes, this is wrong precisely the same way as "def foo() { foo() }", and the infinite loop _should_ happen.

> Of course, you can say that you can use the attribute notation, but this would make a lot of code less readable

I would say quite the contrary. I'd say it's conceptually wrong to access ivars from wherever other than their accessors; thus, the only code affected would be the accessors themselves -- and you pretty rarely write them yourself in Groovy, most time the generated ones are all what you need. And I would also argue that seeing at the first look the distinction betwixt direct ivar access and accessor usage actually makes the code considerably _better_ readable. Nevertheless...

> so the choice was made to use fields instead of getters from inside the class.

... even if I managed to convince you, there's no way to change it now due to the backward compatibility with the existing codebase anyway, so I won't even try :)

Well, I presume it should not be that difficult to write an AST transform to fix the problem locally for me and my sources :)

Thanks a lot and all the best,
---
Ondra Čada
OCSoftware: o...@ocs.cz http://www.ocs.cz
private on...@ocs.cz http://www.ocs.cz/oc

[Ondřej Čada]

Hello there,

On Mar 13, 2013, at 4:16 PM, Ondřej Čada wrote:
> Well, I presume it should not be that difficult to write an AST transform to fix the problem locally for me and my sources :)

OK, this one by my testing seems to do exactly what it should -- incl. making the abomination of "instanceOfAnotherClass.@itsPrivatePart" to crash, which I consider a big benefit, too :)

If anyone sees a potential problem in there, I'd be grateful if you let me know:

import org.codehaus.groovy.ast.*
import org.codehaus.groovy.ast.expr.*
import org.codehaus.groovy.control.*
import org.codehaus.groovy.transform.*
@GroovyASTTransformation(phase = CompilePhase.SEMANTIC_ANALYSIS)
class astt extends ClassCodeExpressionTransformer implements ASTTransformation {
private def fields
public Expression transform(final Expression exp) { // btw what's the rationale behind exp being final? Does not make sense to me
if (!exp) return exp // is it normal to get null expressions sometimes? I do get them for some sources!
def e=super.transform(exp)
if ((e in AttributeExpression) && (e.objectExpression in VariableExpression) && (e.objectExpression.name=='this') && (e.property in ConstantExpression)) {
def nn=fields[e.propertyAsString]
if (nn) e=new AttributeExpression(e.objectExpression,new ConstantExpression(nn))
}
e
}
SourceUnit sourceUnit
void visit(ASTNode[] nodes, SourceUnit su) {
(sourceUnit=su)?.ast?.classes?.each {
fields=[:]
it.fields.each { fn ->
def nn=fields[fn.name]=fn.name+'$cz_ocs_no_direct_access_please'
fn.rename(nn)
}
visitClass(it)
}
}
}

Thanks a big lot,

[Ondřej Čada]

Dinko,

On Mar 13, 2013, at 3:37 PM, Dinko Srkoc wrote:
> class A {
> def foo
>
> // let's override the setter
> void setFoo(bar) { foo = bar + 1 }
> }
>
> Without this feature, the above code would lead to StackOverflowError

Which can be easily cured by writing the proper code, like this

void setFoo(bar) { this.@foo=bar+1 }

without breaking consistence and encapsulation both :(

> As far as inheritance goes, if the above rule is taken strictly, i.e.
> it doesn't apply to subclasses (rationale - if `foo` is a private
> field, than the subclass should not have access to it anyway, so `foo`
> is then a property),

Agreed, subclass should not have access to the field. Nor should the outside world, of course, to which I'll get back later.

> all is fine.

Alas not all is fine, not by far.

First, this brings pretty ugly inconsistencies -- do you _really_ consider this right?

class q {
def foo='direct'
def getFoo() { 'getter' }
def wow() {
def me=this
println "How comes ${me.foo} and ${this.foo} differ now?"
}
}

What's worse, it breaks encapsulation when inheriting methods and overriding accessors -- consider

class q {
def foo='the value for q, which is completely wrong!'
def weDoSomethingWithFoo() {
println "Doin' sumthin' with $foo"
}
}
class w extends q {
def getFoo() { 'the value for w' }
def weDoSomethingMoreWithFoo() {
println "Doin' sumthin' moar with $foo, now to the orig process..."
weDoSomethingWithFoo()

}
}

> Well, almost all, since if one can
> access object's private parts from the outside, why not from the
> subclass (the case of `@foo`)?

Actually quite the contrary -- accessing private fields directly from a subclass is bad, accessing them directly from outside is worse, and definitely should not work. Again, for the same reasons: encapsulation. If a subclass (or a proxy or whatever) overrides the accessor methods, they should make the interface of the object to the outside world, and the private fields shoud be hidden under the encapsulation horisont.

On the other hand, this is a completely unimportant trifle, for from outside you would use @ explicitly only in cases where there's no getter overriding possible (and if you do otherwise, serves you right! :)), and in this case, you never get the direct access implicitly without wanting -- therefore, there's in practice no danger at all.

The real problem is getting the direct access implicitly without using @ in the class which defines the field. Well again, thanks to the groovy flexibility of the groovy Groovy (sorry for that), it's not a real problem either, for whomever wants to can easily get rid of the problem using a pretty simple ASTT :)

Thanks and all the best,

---
Ondra Čada
OCSoftware: o...@ocs.cz http://www.ocs.cz
private on...@ocs.cz http://www.ocs.cz/oc

[Dinko Srkoc]

Ondra,

On 13 March 2013 17:53, Ondřej Čada <o...@ocs.cz> wrote:
> Dinko,
>
> On Mar 13, 2013, at 3:37 PM, Dinko Srkoc wrote:
>> class A {
>> def foo
>>
>> // let's override the setter
>> void setFoo(bar) { foo = bar + 1 }
>> }
>>
>> Without this feature, the above code would lead to StackOverflowError
>
> Which can be easily cured by writing the proper code, like this
>
> void setFoo(bar) { this.@foo=bar+1 }
>
> without breaking consistence and encapsulation both :(

I agree, but apparently it was very easy for people to make that mistake.

>
>> As far as inheritance goes, if the above rule is taken strictly, i.e.
>> it doesn't apply to subclasses (rationale - if `foo` is a private
>> field, than the subclass should not have access to it anyway, so `foo`
>> is then a property),
>
> Agreed, subclass should not have access to the field. Nor should the outside world, of course, to which I'll get back later.
>
>> all is fine.
>
> Alas not all is fine, not by far.

Yes, "all" may not be the most suitable word here. ;-)

>
> First, this brings pretty ugly inconsistencies -- do you _really_ consider this right?
>
> class q {
> def foo='direct'
> def getFoo() { 'getter' }
> def wow() {
> def me=this
> println "How comes ${me.foo} and ${this.foo} differ now?"
> }
> }

Wow indeed! That is nasty.

>
> What's worse, it breaks encapsulation when inheriting methods and overriding accessors -- consider
>
> class q {
> def foo='the value for q, which is completely wrong!'
> def weDoSomethingWithFoo() {
> println "Doin' sumthin' with $foo"
> }
> }
> class w extends q {
> def getFoo() { 'the value for w' }
> def weDoSomethingMoreWithFoo() {
> println "Doin' sumthin' moar with $foo, now to the orig process..."
> weDoSomethingWithFoo()
> }
> }
>

Well, this is at least consistent, and if you know the semantics well,
it's not surprising. I admit that the rules here are complicated and
unintuitive which is never a good thing in a language.

>> Well, almost all, since if one can
>> access object's private parts from the outside, why not from the
>> subclass (the case of `@foo`)?
>
> Actually quite the contrary -- accessing private fields directly from a subclass is bad, accessing them directly from outside is worse, and definitely should not work. Again, for the same reasons: encapsulation. If a subclass (or a proxy or whatever) overrides the accessor methods, they should make the interface of the object to the outside world, and the private fields shoud be hidden under the encapsulation horisont.
>

Groovy was never big on encapsulation, but it's hardly lonely on that
front. E.g. Python, I believe, allows access to its private fields
from the outside (someone may want to correct me here, I'm not much of
an expert on Python), in Java you can do it via reflection, and Ruby
allows accessing its instance variables (equivalent to private fields)
across the hierarchy.

Incidentally, I'm quite fond of the syntax `@foo`, as it reminds me of
Ruby's instance variables.

> On the other hand, this is a completely unimportant trifle, for from outside you would use @ explicitly only in cases where there's no getter overriding possible (and if you do otherwise, serves you right! :)), and in this case, you never get the direct access implicitly without wanting -- therefore, there's in practice no danger at all.
>

Consider this than:

class A { private foo = 42 }

def a = new A()
assert a.@foo == 42
// and now ... the punchline
assert a.foo == 42

// lucky we didn't say anything about the dirty knife

On the other hand, this can have its uses. I know tests are grateful for it.

Cheers,
Dinko

[Jochen Theodorou]

Am 14.03.2013 02:14, schrieb Dinko Srkoc:
[...]

>> First, this brings pretty ugly inconsistencies -- do you _really_ consider this right?
>>
>> class q {
>> def foo='direct'
>> def getFoo() { 'getter' }
>> def wow() {
>> def me=this
>> println "How comes ${me.foo} and ${this.foo} differ now?"
>> }
>> }
>
> Wow indeed! That is nasty.

The rules is that:
if you have "this.x" or simply "x" and x is a field in the current
class and not in an open block, then it is a direct field access.

What we normally would have needed to keep compatibility to Java and not
have a conceptual problem is having a different notation for property
access. But that is not what we want. So there is no fine way out. The
problems with inner classes and open blocks accessing the fields have
not been mentioned yet.

If you have a better idea to spread the gap, then I am eager to hear it.
In the last 5 years I did not here any.

[...]

>>> Well, almost all, since if one can
>>> access object's private parts from the outside, why not from the
>>> subclass (the case of `@foo`)?
>>
>> Actually quite the contrary -- accessing private fields directly
>> from a subclass is bad, accessing them directly from outside is worse, and
>> definitely should not work. Again, for the same reasons: encapsulation.
>> If a subclass (or a proxy or whatever) overrides the accessor methods,
>> they should make the interface of the object to the outside world, and
>> the private fields shoud be hidden under the encapsulation horisont.
>
> Groovy was never big on encapsulation, but it's hardly lonely on that
> front. E.g. Python, I believe, allows access to its private fields
> from the outside (someone may want to correct me here, I'm not much of
> an expert on Python), in Java you can do it via reflection, and Ruby
> allows accessing its instance variables (equivalent to private fields)
> across the hierarchy.
>
> Incidentally, I'm quite fond of the syntax `@foo`, as it reminds me of
> Ruby's instance variables.

in the new mop we are going to make some changes here. The standard meta
class will then have more or less the same rules as java has for private
access. But there will be an easy way to enable access to private.

The current situation is btw:

class X { private foo = 1 }
class Y extends X {}
assert new X().foo == 1
assert new Y().foo == 1 // throws MissingPropertyException

bye blackdrag

--
Jochen "blackdrag" Theodorou - Groovy Project Tech Lead
blog: http://blackdragsview.blogspot.com/
german groovy discussion newsgroup: de.comp.lang.misc
For Groovy programming sources visit http://groovy-lang.org

[Dinko Srkoc]

Jochen,

On 14 March 2013 08:37, Jochen Theodorou <blac...@gmx.org> wrote:
> Am 14.03.2013 02:14, schrieb Dinko Srkoc:
> [...]
>
>>> First, this brings pretty ugly inconsistencies -- do you _really_
>>> consider this right?
>>>
>>> class q {
>>> def foo='direct'
>>> def getFoo() { 'getter' }
>>> def wow() {
>>> def me=this
>>> println "How comes ${me.foo} and ${this.foo} differ now?"
>>> }
>>> }
>>
>>
>> Wow indeed! That is nasty.
>
>
> The rules is that:
> if you have "this.x" or simply "x" and x is a field in the current class
> and not in an open block, then it is a direct field access.

But I think you will agree that the above example is highly
counterintuitive from the user's standpoint.

>
> What we normally would have needed to keep compatibility to Java and not
> have a conceptual problem is having a different notation for property
> access. But that is not what we want. So there is no fine way out. The
> problems with inner classes and open blocks accessing the fields have not
> been mentioned yet.
>
> If you have a better idea to spread the gap, then I am eager to hear it. In
> the last 5 years I did not here any.

I'm afraid not. This seems a tough nut to crack without breaking
backwards compatibility.

>
> [...]
>
>>>> Well, almost all, since if one can
>>>> access object's private parts from the outside, why not from the
>>>> subclass (the case of `@foo`)?
>>>
>>>
>>> Actually quite the contrary -- accessing private fields directly
>>> from a subclass is bad, accessing them directly from outside is worse,
>>> and
>>> definitely should not work. Again, for the same reasons: encapsulation.
>>> If a subclass (or a proxy or whatever) overrides the accessor methods,
>>> they should make the interface of the object to the outside world, and
>>> the private fields shoud be hidden under the encapsulation horisont.
>>
>>
>> Groovy was never big on encapsulation, but it's hardly lonely on that
>> front. E.g. Python, I believe, allows access to its private fields
>> from the outside (someone may want to correct me here, I'm not much of
>> an expert on Python), in Java you can do it via reflection, and Ruby
>> allows accessing its instance variables (equivalent to private fields)
>> across the hierarchy.

>> [...]

>
> in the new mop we are going to make some changes here. The standard meta
> class will then have more or less the same rules as java has for private
> access. But there will be an easy way to enable access to private.
>

That is good to hear - on both accounts. I would hate losing access to
private methods in my tests.

While at it, are you perhaps considering fixing final local variables
too? Pretty please?

def foo(final a) {
final b = a
b = b + 1
// a = a + 1 // compilation error, ok
++a // is this a bug under current implementation?
b + a
}

assert foo(1) == 4

Cheers,
Dinko

[Jochen Theodorou]

Am 14.03.2013 10:46, schrieb Dinko Srkoc:
> Jochen,
>
> On 14 March 2013 08:37, Jochen Theodorou <blac...@gmx.org> wrote:
>> Am 14.03.2013 02:14, schrieb Dinko Srkoc:
>> [...]
>>
>>>> First, this brings pretty ugly inconsistencies -- do you _really_
>>>> consider this right?
>>>>
>>>> class q {
>>>> def foo='direct'
>>>> def getFoo() { 'getter' }
>>>> def wow() {
>>>> def me=this
>>>> println "How comes ${me.foo} and ${this.foo} differ now?"
>>>> }
>>>> }
>>>
>>>
>>> Wow indeed! That is nasty.
>>
>> The rules is that:
>> if you have "this.x" or simply "x" and x is a field in the current class
>> and not in an open block, then it is a direct field access.
>
> But I think you will agree that the above example is highly
> counterintuitive from the user's standpoint.

you mean them behaving different? sure, yes.

>> What we normally would have needed to keep compatibility to Java and not
>> have a conceptual problem is having a different notation for property
>> access. But that is not what we want. So there is no fine way out. The
>> problems with inner classes and open blocks accessing the fields have not
>> been mentioned yet.
>>
>> If you have a better idea to spread the gap, then I am eager to hear it. In
>> the last 5 years I did not here any.
>
> I'm afraid not. This seems a tough nut to crack without breaking
> backwards compatibility.

well, don't worry about compatibility then for now.

[...]

> While at it, are you perhaps considering fixing final local variables
> too? Pretty please?
>
> def foo(final a) {
> final b = a
> b = b + 1
> // a = a + 1 // compilation error, ok
> ++a // is this a bug under current implementation?
> b + a
> }
>
> assert foo(1) == 4

I think that is a bug in the implementation. Can you please make a
report? Thanks

[Ondřej Čada]

Dinko,

On Mar 14, 2013, at 2:14 AM, Dinko Srkoc wrote:

>> What's worse, it breaks encapsulation when inheriting methods and overriding accessors -- consider
>>
>> class q {
>> def foo='the value for q, which is completely wrong!'
>> def weDoSomethingWithFoo() {
>> println "Doin' sumthin' with $foo"
>> }
>> }
>> class w extends q {
>> def getFoo() { 'the value for w' }
>> def weDoSomethingMoreWithFoo() {
>> println "Doin' sumthin' moar with $foo, now to the orig process..."
>> weDoSomethingWithFoo()
>> }
>> }
>
> Well, this is at least consistent

Does not seem consistent to me. You use foo in two methods, one in base class, one in subclass; both work differently. Given foo is essentially(*) presumed to be a nice groovy shortcut for getFoo(), it can come as a pretty nasty surprise and rather inconsistent behaviour.

(*) Of course, as soon as one learns the smallprint "... unless it is in a class which defines the foo field and unless yadda yadda yadda", it becomes sorta consistent and completely understandable, but still, weird :) And frankly, I can't recall this addendum was in the documentation, though it would rather be caused by my weak memory than by the doc incompleteness :)

> ... I admit that the rules here are complicated and unintuitive which is never a good thing in a language.

Myself, I've already added the ASTT which renames private fields so that they are not anymore accessible but by this.@foo to my toolchain, and unless I find a serious and irrepairable bug in it, I have absolutely no intention to part with the thing :)

When/if I have time to sit down and write GEPs, I consider to recommend that this (or similar) approach becomes the default in 3.0 or later, with a backward-compatibility switch (or annotation or something like that). Nevertheless, it's the future, and I might change my opinion meantime :)

> Groovy was never big on encapsulation, but it's hardly lonely on that front....

Absolutely. Just consider the disaster of Java "class methods" which in fact are hardly methods at all and only superficially they belong to classes (Groovy somewhat improves that, but not that much). With them, there's essentially no encapsulation at all -- e.g., this

class A { static def sayhello() { println "Hi, I'm $this!" } }
class B extends A {}
B.sayhello()

does not work the way it should in an OO environment! Alas can't be helped easily; I've made a proof-of-concept AST transform which is able to solve the problem, provided a call of a class method is always recognized compile-time. Which of course is rather at the impossible side :(

> ... and Ruby allows accessing its instance variables (equivalent to private fields) across the hierarchy.

Even ObjC was born with a direct access to ivars, and one extremely fragile and ugly at that: essentially, you could typecast an object to a plain-C struct pointer and access its fields. Quadruple ick! I believe it actually does not work anymore in ObjC 2. Never had the guts to try, and besides, long ago I happily forgot the keyword which did that abomination :)

In fact I don't mind if a language allows breaking the encapsulation (or does other powerful magic), provided the appropriate tools are easily recognizable and hardly to be usen by a mistake. What I am queasy with are normal operations which look completely innocent, but in some conditions _might_ do things.

> Incidentally, I'm quite fond of the syntax `@foo`, as it reminds me of Ruby's instance variables.

Same here. Also it would be nice to be able to write just "@foo" to be interpreted as "this.@foo", but the parser can't stomach that -- presumably, it would be considered an annotation or something alike. Since I've got a preprocessor anyway, I've tried to fix it for myself, but alas, I can't still reliably recognize strings (with all those /'s and $/'s and GStrings it's far from easy), and it's not that important anyway :)

> Consider this than:
> class A { private foo = 42 }

> ...
> assert a.foo == 42

What the heck?!? :-O I didn't know of this. Even passes the typecheck, to add insult to ijury! :)

> // lucky we didn't say anything about the dirty knife

Indeed :)

[Jochen Theodorou]

Am 14.03.2013 03:10, schrieb Ondřej Čada:
[...]

> Myself, I've already added the ASTT which renames private fields so
> that they are not anymore accessible but by this.@foo to my toolchain,
> and unless I find a serious and irrepairable bug in it, I have
> absolutely no intention to part with the thing :)

and in this.@foo the foo is the new name?

[...]

>> Consider this than:
>> class A { private foo = 42 }
>> ...
>> assert a.foo == 42
>
> What the heck?!? :-O I didn't know of this. Even passes the typecheck, to add insult to ijury! :)

lol. It is loved feature by many doing unit testing

bye blackdrag

--
Jochen "blackdrag" Theodorou - Groovy Project Tech Lead
blog: http://blackdragsview.blogspot.com/
german groovy discussion newsgroup: de.comp.lang.misc
For Groovy programming sources visit http://groovy-lang.org

[Cédric Champeau]

Not only unit testing :)

>
> lol. It is loved feature by many doing unit testing
>
> bye blackdrag
>


--

Cédric Champeau

SpringSource - A Division Of VMware
http://www.springsource.com/
http://twitter.com/CedricChampeau

[Dinko Srkoc]

On 14 March 2013 11:16, Jochen Theodorou <blac...@gmx.org> wrote:
> Am 14.03.2013 10:46, schrieb Dinko Srkoc:
> [...]
>
>> While at it, are you perhaps considering fixing final local variables
>> too? Pretty please?
>>
>> def foo(final a) {
>> final b = a
>> b = b + 1
>> // a = a + 1 // compilation error, ok
>> ++a // is this a bug under current implementation?
>> b + a
>> }
>>
>> assert foo(1) == 4
>
>
> I think that is a bug in the implementation. Can you please make a report?
> Thanks

Here it is:
http://jira.codehaus.org/browse/GROOVY-6052

Cheers,
Dinko

[Ondřej Čada]

Jochen,

On Mar 14, 2013, at 12:37 PM, Jochen Theodorou wrote:

> Am 14.03.2013 03:10, schrieb Ondřej Čada:
> [...]
>> Myself, I've already added the ASTT which renames private fields so that they are not anymore accessible but by this.@foo to my toolchain, and unless I find a serious and irrepairable bug in it, I have absolutely no intention to part with the thing :)
>
> and in this.@foo the foo is the new name?

Yup. The ASTT is pretty simple (and would probably break things with protected fields, which I don't care for myself :)) -- see below for reference.

> [...]
>>> Consider this than:
>>> class A { private foo = 42 }
>>> ...
>>> assert a.foo == 42
>>
>> What the heck?!? :-O I didn't know of this. Even passes the typecheck, to add insult to ijury! :)
>
> lol. It is loved feature by many doing unit testing

I must be missing something very important here. I can see sometimes the direct access to the field gets extremely handy -- nothing against that at all!

But what advantage there is in allowing this instead of using the conceptually much more proper a.@foo?

Thanks and all the best,
---
Ondra Čada
OCSoftware: o...@ocs.cz http://www.ocs.cz
private on...@ocs.cz http://www.ocs.cz/oc

=== the don't-touch-my-privates ASTT ===

@GroovyASTTransformation(phase = CompilePhase.SEMANTIC_ANALYSIS)
class astt extends ClassCodeExpressionTransformer implements ASTTransformation {
private def fields
public Expression transform(final Expression exp) { // btw what's the rationale behind exp being final? Does not make sense to me
if (!exp) return exp // is it normal to get null expressions sometimes? I do get them for some sources!
def e=super.transform(exp)
if ((e in AttributeExpression) && (e.objectExpression in VariableExpression) && (e.objectExpression.name=='this') && (e.property in ConstantExpression)) {
def nn=fields[e.propertyAsString]
if (nn) e=new AttributeExpression(e.objectExpression,new ConstantExpression(nn))
}
e
}
SourceUnit sourceUnit
void visit(ASTNode[] nodes, SourceUnit su) {
(sourceUnit=su)?.ast?.classes?.each {
fields=[:]
it.fields.each { fn ->
def nn=fields[fn.name]=fn.name+'$cz_ocs_no_direct_access_please'
fn.rename(nn)
}
visitClass(it)
}
}
}

[Ondřej Čada]

Jochen,

On Mar 14, 2013, at 11:16 AM, Jochen Theodorou wrote:

>>> What we normally would have needed to keep compatibility to Java and not
>>> have a conceptual problem is having a different notation for property
>>> access. But that is not what we want. So there is no fine way out. The
>>> problems with inner classes and open blocks accessing the fields have not
>>> been mentioned yet.
>>>
>>> If you have a better idea to spread the gap, then I am eager to hear it. In
>>> the last 5 years I did not here any.
>>
>> I'm afraid not. This seems a tough nut to crack without breaking
>> backwards compatibility.
>
> well, don't worry about compatibility then for now.

Well if we don't worry about backward compatibility to either Java or the existing Groovy codebase (which (*) we in fact can't ever afford to, but let's pretend for a moment we can :)), I'd say the proper solution is pretty easy, and would go _roughly_ along these lines:

1. a field can be marked "public" (the default), "readonly", "private", and also "protected" or "world_access"
- "public", "readonly" and "private" make a private field, direct access only from its creator class (plus @@ below)
- "protected" makes a field to which direct access with @ is possible from subclasses, too
- "world_access" makes a field to which direct access with @ is possible from anywhere.
With an explicit warning in the documentation that "protected" and "world" break encapsulation and should be used rarely and with caution.

2. when/if at all there is time to think this through thoroughly, a couple of other modifiers could be added, e.g., "cloned" or "lazy" or "synchronized". It's rather arguable which of them belong to the language and which better to stay annotations

3. accessors are generated
- both for "public"
- getter for "readonly"
- none for the other three, unless they are marked "readonly" (which generated getter) or "readwrite" (which generates both)
As always, if an accessor is already implemented, it won't be generated. If some extra modifiers as mentioned in (2) are added, they of course change the accessor contents, e.g., "cloned" would turn "@field=value" to "@field=value.clone()" etc.

4. there's only one way for direct field access, '@'. Nothing else ever works!
- the class which declared the field can use this.@foo or @foo with exactly same semantic
- with "protected" fields the same can do its subclasses
- with "world" whomever anywhere can use foo.@bar
- and, as a special tool for unit checks and other dirty work, always anybody anywhere can use foo.@@bar to access directly any field, regardless its type and owner. With an explicit warning, yadda yadda.

5. the property access foo.bar consistently and always uses only accessors, never ever accesses fields directly. If appropriate accessor does not exist, it always fails.

This of course would break both existing codebase and Java compatibility, and therefore it's rather impossible in general case :( On the other hand, might be a good option for "groovy-only" projects.

All the best,

---
Ondra Čada
OCSoftware: o...@ocs.cz http://www.ocs.cz
private on...@ocs.cz http://www.ocs.cz/oc

[Dinko Srkoc]

On 14 March 2013 14:26, Ondřej Čada <o...@ocs.cz> wrote:
>> [...]
>>>> Consider this than:
>>>> class A { private foo = 42 }
>>>> ...
>>>> assert a.foo == 42
>>>
>>> What the heck?!? :-O I didn't know of this. Even passes the typecheck, to add insult to ijury! :)
>>
>> lol. It is loved feature by many doing unit testing
>
> I must be missing something very important here. I can see sometimes the direct access to the field gets extremely handy -- nothing against that at all!
>
> But what advantage there is in allowing this instead of using the conceptually much more proper a.@foo?

I believe it is not so much accessing private fields as it is
accessing private methods that one wants to test.

Cheers,
Dinko

[Ondřej Čada]

P.S. a couple of comments I forgot originally, sorry:

On Mar 14, 2013, at 3:04 PM, Ondřej Čada wrote:

> Well if we don't worry about backward compatibility to either Java or the existing Groovy codebase (which (*) we in fact can't ever afford to, but let's pretend for a moment we can :)), I'd say the proper solution is pretty easy, and would go _roughly_ along these lines:
>
> 1. a field can be marked "public" (the default), "readonly", "private", and also "protected" or "world_access"
> - "public", "readonly" and "private" make a private field, direct access only from its creator class (plus @@ below)
> - "protected" makes a field to which direct access with @ is possible from subclasses, too
> - "world_access" makes a field to which direct access with @ is possible from anywhere.
> With an explicit warning in the documentation that "protected" and "world" break encapsulation and should be used rarely and with caution.

Note: myself, I would skip the "protected" and "world", and for those who do need them, stick with @@ only.

> 2. when/if at all there is time to think this through thoroughly, a couple of other modifiers could be added, e.g., "cloned" or "lazy" or "synchronized". It's rather arguable which of them belong to the language and which better to stay annotations
>
> 3. accessors are generated
> - both for "public"
> - getter for "readonly"
> - none for the other three, unless they are marked "readonly" (which generated getter) or "readwrite" (which generates both)
> As always, if an accessor is already implemented, it won't be generated. If some extra modifiers as mentioned in (2) are added, they of course change the accessor contents, e.g., "cloned" would turn "@field=value" to "@field=value.clone()" etc.

This applies for generated accessors, of course. If you write the accessor yourself, the compiler of course won't change it (unless other ASTTs, completely unrelated to this, do). If anyone wants to declare a field cloneable and then shoot himself in his leg by implementing non-cloning setter, be our guest by all means :)

> 4. there's only one way for direct field access, '@'. Nothing else ever works!
> - the class which declared the field can use this.@foo or @foo with exactly same semantic
> - with "protected" fields the same can do its subclasses
> - with "world" whomever anywhere can use foo.@bar
> - and, as a special tool for unit checks and other dirty work, always anybody anywhere can use foo.@@bar to access directly any field, regardless its type and owner. With an explicit warning, yadda yadda.

Note unlike the current case, this should probably work even for subclasses, so that

class Foo { private foo,bar }
class Bar extends Foo { private bar }

def f=new Foo(),b=new Bar()
f.@@foo -- self evident
f.@@bar -- self evident
b.@@foo -- this is a new, access to Foo.foo inside b
b.@@bar -- to keep consistence, must be access to Bar.bar inside b
b.@@Foo.bar -- we might add this ugly contraption to access Foo.bar inside b, too :)

Myself, I don't really think this is needed, but well, someone other might find it pretty handy perhaps :)

[Jochen Theodorou]

Am 14.03.2013 15:15, schrieb Dinko Srkoc:
[...]

> I believe it is not so much accessing private fields as it is
> accessing private methods that one wants to test.

well, it is also about setting an object into a certain state.

bye blackdrag

--
Jochen "blackdrag" Theodorou - Groovy Project Tech Lead
blog: http://blackdragsview.blogspot.com/
german groovy discussion newsgroup: de.comp.lang.misc
For Groovy programming sources visit http://groovy-lang.org

[Jochen Theodorou]

Am 14.03.2013 15:04, schrieb Ondřej Čada:
[...]

>> well, don't worry about compatibility then for now.
>
> Well if we don't worry about backward compatibility to either Java
> or the existing Groovy codebase (which (*) we in fact can't ever afford to,
> but let's pretend for a moment we can :)),

actually I didn't mean Java, only Groovy, but let's see

so basically you suggest to use .@@ for field access ignoring
visibility, .@ for field access with visibility rules and . only for
property access.

bye blackdrag

--
Jochen "blackdrag" Theodorou - Groovy Project Tech Lead
blog: http://blackdragsview.blogspot.com/
german groovy discussion newsgroup: de.comp.lang.misc
For Groovy programming sources visit http://groovy-lang.org

[Ondřej Čada]

Dinko,

(you might get the message twice -- I took the liberty to bcc you to work around the lately terribly long list latency. Hope you don't mind, if you do, apologies and let me know not to anymore)

On Mar 14, 2013, at 3:15 PM, Dinko Srkoc wrote:
>>> [...]
>>>>> Consider this than:
>>>>> class A { private foo = 42 }
>>>>> ...
>>>>> assert a.foo == 42
>>>>
>>>> What the heck?!? :-O I didn't know of this. Even passes the typecheck, to add insult to ijury! :)
>>>
>>> lol. It is loved feature by many doing unit testing
>>
>> I must be missing something very important here. I can see sometimes the direct access to the field gets extremely handy -- nothing against that at all!
>>
>> But what advantage there is in allowing this instead of using the conceptually much more proper a.@foo?
>
> I believe it is not so much accessing private fields as it is accessing private methods that one wants to test.

You lost me here. Did you mean that for a private field a private accessor method is created, e.g., that the above class gets implicitly

class A {
private foo // if you write this in the source...
private getFoo() { this.@foo } // ...you get this for free from the compiler
}

? Does not seem (a) to be documented so, (b) to behave so in fact, (c) to make any sense to do that -- but I might be missing something, as always :)

Anyway. As for private (and protected) _methods_, that's quite another can of worm, not really related to fields, or, at best, very very loosely. Thus, I am changing the subject, to keep some orientation in the threads.

To compare -- ObjC does not have those things at all (all methods are always fully public in there, the only possible kind of "privateness" is that the world is not informed of their names, and it can be always solved by introspection), and I've actually never bumped into any problem due to that -- not in decades and not in millions of source lines, some of them maintained for those decades. Thus I tend to forget private/protected methods at all, am not using them myself, and if I bump into them in others' code, I tend to be surprised something like that exists at all :)

Anyway, there are three points of view there I guess:

(i) myself, I would be completely contented if Groovy simply ignored all the private/protected modifiers for methods, just as it, far as I understand, more or less does now :)

(ii) as for a correct general future solution regardless backward compatibility with the current codebase... I guess something like granting the access limitations _and_ adding an extra modifier which would say "bang the rules!" would be the best way. Something roughly similar to

class A { static private foo() { ... } }
A.foo() // error, exactly as would be in Java
A!.foo() // calls it all right, we ignore the Java rules here

Would probably need some more elaboration with cases like super!.privatesupermethod() etc., but at the first look seems feasible to me.

If something like this really gets ever implemented in Groovy, of course it should be consistent with the field-access rules (see my suggestion of the super-access @@ modifier in a previous message -- it would be much better to use same construction for both, of course, i.e., instead of foo.@@privatefield we should have foo!.@privatefield).

(iii) as for a correct solution which would _not_ break backward compatibility with the current codebase... no idea, to be frank. Nevertheless it seems to me here the prospects are better than with fields :)

Thanks and all the best,
---
Ondra Čada
OCSoftware: o...@ocs.cz http://www.ocs.cz
private on...@ocs.cz http://www.ocs.cz/oc

[Jochen Theodorou]

Am 14.03.2013 15:22, schrieb Ondřej Čada:
[...]

> class Foo { private foo,bar }
> class Bar extends Foo { private bar }
>
> def f=new Foo(),b=new Bar()
> f.@@foo -- self evident
> f.@@bar -- self evident
> b.@@foo -- this is a new, access to Foo.foo inside b
> b.@@bar -- to keep consistence, must be access to Bar.bar inside b
> b.@@Foo.bar -- we might add this ugly contraption to access Foo.bar inside b, too :)
>
> Myself, I don't really think this is needed, but well, someone other might find it pretty handy perhaps :)

b.@@Foo.bar is problematic, since it is not clear if Foo is a field

bye blackdrag

--
Jochen "blackdrag" Theodorou - Groovy Project Tech Lead
blog: http://blackdragsview.blogspot.com/
german groovy discussion newsgroup: de.comp.lang.misc
For Groovy programming sources visit http://groovy-lang.org

[Ondřej Čada]

Jochen,

On Mar 14, 2013, at 4:48 PM, Jochen Theodorou wrote:

> so basically you suggest to use ... .@ for field access with visibility rules and . only for property access.

Precisely. If we disregard the backward compatibility issues, I think it should be so, clearly and unequivocally distinguished.

> .@@ for field access ignoring visibility

More or less.

I suggest to use _some_ specific construct for field access ignoring visibility -- a construct simple enough not to be annoying, but, at the same time, distinct enough to differ from the above at the first look.

Whether it is .@@ or !.@ (*) or something other and much better than those two, that's a question :)

Note: myself, I would be completely contented if there was no language-level construct for this, and it was possible only though low-level runtime methods, something like

foo.getFieldValueIgnoringVisibility('bar') // same as foo.@@bar or foo!.@bar

but I understand there are people out there who would consider this a _serious_ annoyance :)

And I've got your next message meantime, and since it's very closely related, am answering here, too:

>> class Foo { private foo,bar }
>> class Bar extends Foo { private bar }
>> def f=new Foo(),b=new Bar()

> ...

>> b.@@Foo.bar -- we might add this ugly contraption to access Foo.bar inside b, too :)
>> Myself, I don't really think this is needed, but well, someone other might find it pretty handy perhaps :)
>
> b.@@Foo.bar is problematic, since it is not clear if Foo is a field

Right you are, of course -- would need some other syntax, if should be allowed at all. Again, like with the note above, myself, I would be quite satisfied with just low-level access through something like

foo.getFieldValueIgnoringVisibility('bar',Foo)

where the second argument would be either null for the top-level field of that name, or a class (class name perhaps? Or any of them?) to specify the inheritance level to be accessed in case private fields are "overridden" in subclasses.

Frankly I have no idea whether this level of introspection really needs a language construct. I think not. But as always, I can be wrong :)

(*) the bang-construction would have the advantage that it could be used exactly the same way to ignore visibility of methods, like, "foo!.privateMethod()". I did not think about method visibility rules at all when I wrote the original message :)

(Oh, and again, I haven't checked the language grammar: no idea whether the bang construction might clash with some current thing; if so, another syntax would have to be chosen.)

Thanks and all the best,

---
Ondra Čada
OCSoftware: o...@ocs.cz http://www.ocs.cz
private on...@ocs.cz http://www.ocs.cz/oc

[Wujek Srujek]

Just my 2c about the second solution (private access results in exception, but can be forced with special syntax): what purpose will it serve? I mean, I see two points of view:

a) library users: they call a private method, get an exception, so they think 'Ok, let's just add the exclamation mark (whatever) and get it over with' - so the private access exception is just a minor nuisance and has no real value

b) library developers - they must simply forget private because it will always be possible to work around it; the fact that private access throws some runtime exception will just confuse some and give a false sense of security

In general I like the fact that groovy doesn't follow Java's access restrictions, just as I like Python's lack thereof (name mangling with the __ prefix doesn't really count). These, and many other languages, are doing just great without this.

I could live with either real access restrictions, or none at all, but the fact that there is some syntax to get around it (and hence making things more complex) would just break the language a tiny bit in my opinion.

wujek

[Ondřej Čada]

Wujek,

On Mar 14, 2013, at 9:32 PM, Wujek Srujek wrote:

> Just my 2c about the second solution (private access results in exception, but can be forced with special syntax): what purpose will it serve?

First let me please emphasize (i) below -- myself, I'm not a fan of private/protected methods.

Nevertheless if I try to put on a hat of one, I'd say

- it's nice not only to use some methods privately or for-subclassing-only (this self-evidently we all do all the time), but _also_ to explicitly mark them so, not just in comments, but in a formal way
- it's nice the compiler and runtime make sure nobody could call these methods by mistake, which means we can afford simple names.

In ObjC, I would either name my private helper sum method "__ocs_sum", or perhaps I would even use a static function with a self argument (ObjC is a C superset, so whatever C allows, ObjC does too). I would never dare to define private helper method "sum", since the name is ubiquitous, and the danger a subclass inadvertently overrides it with disastrous results (etc) is great.

In Java I could name the thing just "sum" and make it private, and that's that.

In Groovy the fact someone _can_ call it does not change anything -- just like in ObjC anyone _can_ call my __ocs_sum method does not mean it is not private. If someone does that and it blows into his face, it's _his_ funeral, not mine: the method was not part of the API contract, and _he_ took the chances to use it.

> a) library users: they call a private method, get an exception, so they think 'Ok, let's just add the exclamation mark (whatever) and get it over with' - so the private access exception is just a minor nuisance and has no real value

Quite the contrary.

Library users -- at least, those of them who are sane -- don't _want to_ call private methods, for it would potentially break their code when new library release comes.

Private methods help them to achieve the goal, for it lessens the danger of inadvertent using of private API.

Thus, they won't use the bang unless they _have to_, rebus desperatis, to work-around a library bug, or to do something highly non-standard. And they would be extra careful to check the library version before such a code gets run.

Of course, there might be also blokes who would simply use the bang without thinking all the time. Their code would be unstable, crashing, and dangerous. They would hardly have any clients. Serves them jolly well right.

> b) library developers - they must simply forget private because it will always be possible to work around it; the fact that private access throws some runtime exception will just confuse some and give a false sense of security

Again, quite the contrary.

See above: the privateness would allow them to rely on simple-named private helper methods without ugly prefixes (and thus generally better readable code).

Also to them, the bang would allow easy writing of test code. No danger there, for they manage the test code along with the library, and if one gets changed, the other does, too.

The fact that someone out there might use the bang has absolutely no consequence for the writers of libraries. It's a dangerous practice not covered by the API contract, and whomever does it, he explicitly takes the chance his code stops working at the first upgrade, and he knows that and behaves accordingly. Or does not behave accordingly and shoots his own leg, and serves him right.

That's also why it's important the bang is a specifict, distinct syntax, which can't be mistaken for normal code.

Note: as I wrote in similar situations previously, myself, I would be completely contented if there was no special language feature for this, and if instead of the bang only low-level runtime API was available, i.e.,

foo!.privateMethod(args) // nope, instead you have to do ...
foo.groovyRuntime.callPrivateMethod('foo',args) // ... something essentially like this

But I can see it would be pretty annoying for test writers, that's why I suggest the bang (or something similar).

> In general I like the fact that groovy doesn't follow Java's access restrictions, just as I like Python's lack thereof (name mangling with the __ prefix doesn't really count). These, and many other languages, are doing just great without this.

Sure, see please again (i) below :)

> I could live with either real access restrictions, or none at all, but the fact that there is some syntax to get around it (and hence making things more complex) would just break the language a tiny bit in my opinion.

I don't like _unbreakable_ rules, for always sooner or later an _exceptional_ need arises to break them. In Java, it leads to hundreds of ugly boilerplate work-arounds, where otherwise one well-marked "dangerous" line would suffice and actually be much less dangerous, for it's easier to guard for known big danger at one small place, than to find small bugs in hundreds and hundreds of "innocent" lines.

Therefore I think a language _should_ offer APIs to do all kinds of dirty stuff, to be available if needed.

On the other hand, these dangerous APIs -- like calling private methods, direct access to ivars, etc. -- should be very distinct and should never happen implicitly.

Thanks and all the best,

OC

=== original ===

> (i) myself, I would be completely contented if Groovy simply ignored all the private/protected modifiers for methods, just as it, far as I understand, more or less does now :)
>

> To compare -- ObjC does not have those things at all (all methods are always fully public in there, the only possible kind of "privateness" is that the world is not informed of their names, and it can be always solved by introspection), and I've actually never bumped into any problem due to that -- not in decades and not in millions of source lines, some of them maintained for those decades. Thus I tend to forget private/protected methods at all, am not using them myself, and if I bump into them in others' code, I tend to be surprised something like that exists at all :)
>

> (ii) as for a correct general future solution regardless backward compatibility with the current codebase... I guess something like granting the access limitations _and_ adding an extra modifier which would say "bang the rules!" would be the best way. Something roughly similar to
>
> class A { static private foo() { ... } }
> A.foo() // error, exactly as would be in Java
> A!.foo() // calls it all right, we ignore the Java rules here
>
> Would probably need some more elaboration with cases like super!.privatesupermethod() etc., but at the first look seems feasible to me.
>
> If something like this really gets ever implemented in Groovy, of course it should be consistent with the field-access rules (see my suggestion of the super-access @@ modifier in a previous message -- it would be much better to use same construction for both, of course, i.e., instead of foo.@@privatefield we should have foo!.@privatefield).
>
> (iii) as for a correct solution which would _not_ break backward compatibility with the current codebase... no idea, to be frank. Nevertheless it seems to me here the prospects are better than with fields :)