At the OSCON 2006 Hackathon, it was decided that we should separate
vtables from methods and add a new :vtable label for PIR subs to mark
them as vtable functions. This gets rid of the current namespace
pollution caused by vtables and allows us to abandon the leading
underscores for PIR vtable functions.
.sub get_string :vtable
.return("end namespace pollution!")
.end
It appears that this never made it into RT.
--
Matt Diephouse
Thanks,
Jonathan
> Just to check, that this is still meant to happen?
Unless Allison objects, it seems like the cleanest way to support overloading
semantics from various languages (Perl 5: FETCH, Python:
__some_really_ugly_method_name__) without actually making namespace
collisions.
> Anyone feel it should be put off until the objects/namespaces stuff is
> sorted out, or shall I just dive right in?
As long as the interface works (and it seems sensible to me), we can fix the
implementation in plenty of ways.
-- c
This is the main thing Chip and I talked about in our last face-to-face
meeting. We came up with 3 basic parameters: whether a method is a
vtable method, whether it has a vtable name distinct from the method
name, and whether it has a method name at all (or is anonymous, i.e.
only a vtable method). The interface I scrawled out over coffee is:
# method name is the same as vtable name
.sub get_string :method :vtable
# accessible as either $obj.stringify() or vtable
.sub stringify :method :vtable('get_string')
# accessible only as vtable
.sub get_string :method :anon :vtable
.sub stringify :method :anon :vtable('get_string')
Which reuses the existing concept of:
# method has an entry in the namespace
.sub stringify :method
# method has no entry in the namespace
.sub stringify :method :anon
(There's not much point in using the 4th option where you specify a name
for the method, make it anonymous, and then specify a separate name for
the vtable entry, but we'll support it for the sake of orthogonality.)
The reasoning for making :vtable additive rather than exclusive with
:method is that it's common to want to add behavior that's both a method
on the object and a vtable entry.
As a side effect of this solution, the leading double underscore goes
away for vtable names. So it's now just 'init' instead of '__init',
'get_bool' instead of '__get_bool', etc. The underscores were just a
hack to alleviate namespace pollution anyway.
Even though "pure" vtable methods aren't stored in the addressable part
of the namespace, we came to the conclusion that they should be attached
to the namespace, because an anonymous class can be associated with a
namespace at runtime, and when it is, it should get not only the methods
of that namespace, but also the overloaded vtable entries.
(I can see an argument of brevity for an alternate solution where the
3rd and 4th options have only :vtable and not :method and :anon:
# accessible only as vtable
.sub get_string :vtable
.sub stringify :vtable('get_string')
but I don't like the fact that this would make :vtable mean different
things in different contexts. Together with :method, :vtable would mean
"also make an entry in the vtable overload list for this namespace", but
without :method it would mean "don't make an entry for this subroutine
in the namespace, and treat this subroutine as a method, and make an
entry in the vtable list". I prefer to keep each of those separated into
their own adverbs, especially since we already have the adverbs.)
Allison
+1
Pm
Thanks Allison for clarifications and @other for agreement; I've taken
this ticket and will get cracking on implementing this.
Jonathan
As of r15039, :vtable and :vtable("...") are now both implemented. See
example code at the end of this email.
As of now, please use this new syntax. I have left in support for the
old "__"-prefix lookup as plenty of old code is using it; please note
that if you have a :vtable("xxx") and a "__xxx" then the first will be
called, not the second.
I'll leave it to Chip and/or Allison to decide how long the deprecation
cycle is for this.
Implementation wise, at the moment I'm just storing these entries in
"\0VTABLE\0" sub-namespace within the class namespace, however I'm not
sure I like this (feels like we got rid of one bit of namespace
pollution by adding another, albeit a less noticeable one). I don't see
another way to attach the VTABLE namespace to the main one, however. But
that's an implementation detail - even if we change that any
:vtable-using code will go on working fine.
Tests and some extra error checking code to come.
Jonathan
.sub main :main
$P0 = newclass [ "Test" ]
$P1 = new [ "Test" ]
$I1 = $P1[11]
print $I1
print "\n"
$S1 = $P1[11]
print $S1
print "\n"
.end
.namespace [ "Test" ]
.sub get_integer_keyed_int :method :vtable
.param int key
.return(42)
.end
.sub monkey :method :vtable("get_string_keyed_int")
.param int key
.return("monkey")
.end
.sub not_a_vtable_method :method :vtable
.sub badger :method :vtable("not_a_vtable_method")
It's a compiler error.
I'll leave this ticket open a few more days for comments, then if there
are none close it up.
Thanks,
Jonathan
Awesome!
> As of now, please use this new syntax. I have left in support for the
> old "__"-prefix lookup as plenty of old code is using it; please
> note that if you have a :vtable("xxx") and a "__xxx" then the first
> will be called, not the second.
Updating Punie went smoothly, but when I updated TGE::Rule:
-.sub __init :method
+.sub 'init' :method :anon :vtable
I got this error:
../../parrot -o TGE/Rule.pbc --output-pbc TGE/Rule.pir
error:imcc:'init' is not a v-table method, but was used with :vtable.
in file 'TGE/Rule.pir' line 49
> I'll leave it to Chip and/or Allison to decide how long the
> deprecation cycle is for this.
Since it is so widely used, let's deprecate it two releases from now, so
we can have one release with both options, and then deprecate it in the
following release.
> Implementation wise, at the moment I'm just storing these entries in
> "\0VTABLE\0" sub-namespace within the class namespace, however I'm
> not sure I like this (feels like we got rid of one bit of namespace
> pollution by adding another, albeit a less noticeable one). I don't
> see another way to attach the VTABLE namespace to the main one,
> however. But that's an implementation detail - even if we change that
> any :vtable-using code will go on working fine.
This is a good place to start, but I agree it's not what we want
ultimately. Chip and I talked about attaching a data structure of vtable
entries as a property on the namespace object.
Allison
> Since it is so widely used, let's deprecate it two releases from now,
> so we can have one release with both options, and then deprecate it in
> the following release.
Sounds good.
> This is a good place to start, but I agree it's not what we want
> ultimately. Chip and I talked about attaching a data structure of
> vtable entries as a property on the namespace object.
I implemented this to test it out and it kinda worked. However,
properties don't get serialized so if you compile the program to a PBC
and run that it doesn't work. Additionally, properties don't get inherited.
It'd be nice to actually attach these methods to a v-table itself
somehow. However, unless we make classes a compile-time construct rather
than "turning a namespace into a class" using newclass as we do now, I
don't see how this can happen (at least, not cleanly). Maybe something
to be addressed in the objects PDD.
Thanks,
Jonathan
> It'd be nice to actually attach these methods to a v-table itself
> somehow. However, unless we make classes a compile-time construct rather
> than "turning a namespace into a class" using newclass as we do now, I
> don't see how this can happen (at least, not cleanly). Maybe something
> to be addressed in the objects PDD.
I think that was part of the motivation for :init.
-- c
Jonathan
Why don't they get serialized?
> Additionally, properties don't get inherited.
In the sense that they're only attached to individual object instances
of the class, not to the class itself. And even if they were attached to
the class, it would be to the class PMC, and wouldn't be picked up by
subclass PMCs.
Although, in this case what we're talking about is a property attached
to an object instance of the namespace class. Inheritance is determined
by the classes associated with the namespace object, not the namespace
object itself. The storage of vtable methods in the namespace is really
only temporary until they can be loaded as vtable overloads on the
class. (It's also there for the sake of later use by other classes if
they become associated with that namespace.) The vtable overloads only
enter into inheritance once they are loaded into the vtable.
This is why properties seemed like a good fit: the vtable entries in
namespaces are optional, out-of-band information, only used in loading.
But, they could just as well be implemented as a separate attribute in
the namespace class.
> It'd be nice to actually attach these methods to a v-table itself
> somehow. However, unless we make classes a compile-time construct rather
> than "turning a namespace into a class" using newclass as we do now, I
> don't see how this can happen (at least, not cleanly).
It also wouldn't allow for runtime association of an anonymous class
with a namespace, or runtime switching of the namespace a class is
associated with.
> Maybe something to be addressed in the objects PDD.
Yes, good entry for the questions section.
Allison
>> Additionally, properties don't get inherited.
> In the sense that they're only attached to individual object instances
> of the class, not to the class itself. And even if they were attached
> to the class, it would be to the class PMC, and wouldn't be picked up
> by subclass PMCs.
>
> Although, in this case what we're talking about is a property attached
> to an object instance of the namespace class. Inheritance is
> determined by the classes associated with the namespace object, not
> the namespace object itself. The storage of vtable methods in the
> namespace is really only temporary until they can be loaded as vtable
> overloads on the class. (It's also there for the sake of later use by
> other classes if they become associated with that namespace.) The
> vtable overloads only enter into inheritance once they are loaded into
> the vtable.
>
> This is why properties seemed like a good fit: the vtable entries in
> namespaces are optional, out-of-band information, only used in
> loading. But, they could just as well be implemented as a separate
> attribute in the namespace class.
Agree; looking now at what I had to do (walk the MRO) to get inheritance
of :vtable methods to work already, doing it with the data in properties
is no harder. That leaves the serialization issue as the main one. A
ruling on that, and I can implement :vtable using a property. (leo
pointed out the double indirection, as you're looking up the vtable hash
in the property hash, is a performance issue - we can perhaps optimize
this later...I care more about getting the correct behavior right now).
>> It'd be nice to actually attach these methods to a v-table itself
>> somehow. However, unless we make classes a compile-time construct
>> rather than "turning a namespace into a class" using newclass as we
>> do now, I don't see how this can happen (at least, not cleanly).
> It also wouldn't allow for runtime association of an anonymous class
> with a namespace, or runtime switching of the namespace a class is
> associated with.
Must...get...head...out...of....static-language-world. :-)
Thanks,
Jonathan
> Agree; looking now at what I had to do (walk the MRO) to get inheritance
> of :vtable methods to work already, doing it with the data in properties
> is no harder.
This is perhaps a sign that it's time for a super or next opcode.
-- c
This is fixed now (many thanks!)
I still have failing tests. It looks like the 'init' methods aren't
being called. (In TGE::Compiler the tree transformation rules are never
added to tree grammar. In TGE::Tree the AddrRegistry PMC in the 'agid'
attribute is never created, and in TGE::Grammar the ResizablePMCArray in
the 'rules' attribute is never created, so they cause 'Null PMC
access...' errors.)
Is there a lingering piece of Parrot that depends on finding the init
vtable methods prepended with "__"?
Allison
Can anyone think of a reason we shouldn't serialize properties? If not,
let's try it. If we don't like the results, we can back it out.
I suspect that originally it was expected that properties would only be
attached at runtime. But, since we have the option of flagging subs to
execute at various stages of compilation, it seems safe to assume that
properties can be attached at various stages of compilation too.
> (leo
> pointed out the double indirection, as you're looking up the vtable hash
> in the property hash, is a performance issue - we can perhaps optimize
> this later...I care more about getting the correct behavior right now).
He has a good point, but agreed, let's get it working and optimize later.
Allison
It's at least partially implemented. But I think going via props in that case
isn't the best way to implement it.
We have basically the problem that classes are only constructed at runtime,
which has 2 negative impacts: associating the vtable info with the class
needs some helper storage like the props and namespaces can only be vaguely
attached to the correct class at runtime too.
To get around that the following came to my mind:
- we create a class when needed during compile time
- it get's a flag set (one of private0..7) not_yet_finished_FLAG or such
- newclass, when checking if a class exists, examines that flag and continues
adding the runtime information
- the classes constructed in that way are frozen into the PBC
leo
Indeed, that's part of why Perl 6 eventually threw out properties
entirely, on the supposition that properties are simply attributes
that are mixed in at run-time rather than compiled in.
Larry
From an HLL perspective, I agree. But there's always more than one way
to satisfy an HLL abstraction.
From the Parrot perspective, attributes can only be added to classes,
while properties can be added to instantiated objects without modifying
the class (essentially creating a very lightweight singleton).
Properties can also be added to classes without appearing in the
instantiated objects of that class. It's a useful distinction.
I can see an argument for having an interface that can retrieve and
store both attributes and properties, but that's orthogonal to the
question of where the two kinds of data members are stored in the
object/class structure.
Allison
As a virtual machine for dynamic languages, one of the core requirements
is that it has to be able to construct a class completely at runtime. We
may add compile-time construction of classes at some point as an
optimization, but we'll never eliminate runtime construction. That
"vague attachment" is an intentional distinction between namespaces and
the classes associated with those namespaces, allowing for runtime
association of a class with a namespace (even an anonymous class).
> To get around that the following came to my mind:
>
> - we create a class when needed during compile time
> - it get's a flag set (one of private0..7) not_yet_finished_FLAG or such
> - newclass, when checking if a class exists, examines that flag and continues
> adding the runtime information
> - the classes constructed in that way are frozen into the PBC
I can see the appeal, but unfortunately it won't give us all the
features we need. This is certainly an option to consider if/when we do
add compile-time construction of classes, with the understanding that
only a subset of cases would be able to use it (as is true of most
optimizations).
Allison
When serialising data structures, is parrot going to need to serialise any
anonymous or otherwise run time generated classes?
If so, isn't this mechanism going to be capable of achieving "compile time"
class generation?
Nicholas Clark
Of course not. I never said that.
> > To get around that the following came to my mind:
> >
> > - we create a class when needed during compile time
> I can see the appeal, but unfortunately it won't give us all the
> features we need. This is certainly an option to consider if/when we do
> add compile-time construction of classes, with the understanding that
> only a subset of cases would be able to use it (as is true of most
> optimizations).
It's actually not an optimization IMHO. It could be one, if we add support for
adding attributes at compile-time too [1]. But for now, I'm just speaking of
consolidating class construction code.
We now have:
compile-time run-time
-----------------------------------------------
.namespace ['Foo']
.sub bar :method
.sub baz :method :vtable(..)
cl = newclass 'Foo'
addattribute cl, 'x'
addmethod cl, 'blurb', meth
The namespace directive in combination with :method is defining parts of a
class namely all the statically compiled methods and the namespace. The
problem is how to forward the compile-time part of a class properly to the
run-time.
Actually the current implementation is partly wrong to deal with too much
usage of:
.namespace ['Foo']
.sub _init :load
cl = newclass ['Foo'] # would be ['Foo';'Foo']
and similar. Above should be
.sub _init :load
cl = newclass ['Foo'] # ['Foo']
...
.namespace ['Foo']
Having a class available at compile-time would elminate this ambuigity.
> Allison
leo
[1] in fact, we can do that already - see my answer to Nicholas.
Yep, and it does so already.
> If so, isn't this mechanism going to be capable of achieving "compile time"
> class generation?
Indeed. There's a hackish implementation[1] for achieving compile-time class
(and object) creation:
$ cat t/pmc/sub_42.pir
.sub make_obj :immediate, :anon
.local pmc cl, o
cl = newclass "Foo"
addattribute cl, 'x'
o = new 'Foo'
$P0 = new String
$P0 = "ok 1\n"
setattribute o, 'x', $P0
.return (o)
.end
.sub main :main
.const .Sub o = "make_obj"
$P0 = getattribute o, 'x'
print $P0
.end
$ ./parrot -o s42.pbc t/pmc/sub_42.pir
$ ./parrot -t s42.pbc
23 set P1, PC8 ...
26 getattribute P0, P1, "x" ... P1=Object(Foo)= ...
30 print P0 P0=String=PMC(0xb61440 Str:"ok 1\n")
ok 1
32 end
(trace edited for clarity)
But attaching the namespace to the class still suffers from the mentioned
problem, only earlier so.
> Nicholas Clark
leo
[1] the return result of the :immediate (if any) replaces the subroutine
object.