What the heck is active data?
Dan Sugalski
know what I'm talking about, but that's not true anymore, so an
explanation of this is in order.
"Active Data" is data that takes some active role in its
use--reading, writing, modifying, deleting, or using in some activity.
As a point of reference, data in the C sense, ints, floats, pointers,
and so forth, are passive. It doesn't matter what's in them, your
program has complete control and it doesn't matter what data is in
those variables, things always behave the same. What's nice here is
that the compiler can predict, soley from the code, what will happen
when an operation occurs. It may lose track of the referents to data
(if, for example, a pointer escapes the code visible to the compiler)
but everything's nicely static, and the only thing actually *doing*
anything is the code.
Active data is data that in some way affects program behavior
independent (or semi-independent) of the code being executed.
The common case, one most folks are familiar with (if they're
familiar with this stuff at all) is operator overloading. Addition
may not really be addition, if one or the other of the data in the
operation overloads the addition operator. (Whether both sides, or
just the left, is checked, and whether the types of both sides
matters depends on the language) The compiler can't necessarily tell
whether addition is really addition any more--it might really be
subtraction, or multiplication, or some bizarre way to encrypt the
contents of your hard drive.
In some languages the compiler can at least make some predictions of
what operations will do, but we're not really interested in those
languages. Or, rather, it doesn't matter if we are or not, since
perl, python, and ruby are all untyped so there's not much to be
done, and full type inferencing's a halting-problem sort of thing.
(Though a useful subset of many programs can be analyzed enough to do
some optimization)
Perl, at least, and because of it Parrot, goes one step further. Not
only are operations overloaded, but assignment is overloaded. If
you're coming from a pure OO sort of language where assignment is
just the association of a name and pointer to an object, this may
seem kind of strange, but it is really useful.
The canonical example is associating a hash with a database. Reading
or writing from the hash does a lookup in the backing database and
reads or writes from it. A handy thing, with the variable getting in
the way of reads or writes to itself. However, you can do this with
plain scalar variables.
For example, if you had code:
int Foo;
Dog Bar = new();
Foo = Bar;
you wouldn't want there to be a Dog in Foo--you've declared it an
integer, so it can only hold an integer. The assignment can't do a
plain pointer copy the way it would in the case where both sides can
hold objects.
As an alternate example, try this:
TempMeter kitchentemp = new('kitchenroom');
TempMeter bedroomtemp = new('bedroom');
bedroomtemp = kitchentemp;
Where in this case kitchentemp *is* an object, but one of type
TempMeter, and bound to an object that represents the temperature in
your kitchen, while bedroomtemp is a TempMeter object that represents
the temperature in your bedroom. (Code I'd not recommend using in CGI
programs....) The assignment does *not* bind the bedroomtemp name to
the object for the kitchen temp, rather it sets the bedroom
temperature to be the same as that in the kitchen. Even being an
object, the object's methods still intercept read and write requests
and instead of assignment being rebinding, it's instead a get or set
call on the object.
The part that affects us is that we can't tell at compiletime whether
assignment is rebinding or is a get/set, because we could have code
like:
kitchentemp = new('kitchenroom');
bedroomtemp = new('bedroom');
bedroomtemp = kitchentemp;
which is typeless, and yet should still respect the fact that the
objects bound to the name intercept assignment (basically overloading
it) rather than having the compiler or runtime doing the rebinding
for you.
Since, of course, we're dealing with basically typeless languages we
have to do all the checking at runtime (lucky us) which is why the
PMCs have generic get and set methods so they can decide whether
we're doing rebinding or something more funky. (Whether this affects
languages where = is an explicit binding rather than assignment is up
in the air, but neither python nor ruby is truly that way, though
they're close. But we can fake it so that things do what they ought
when they ought)
--
Dan
--------------------------------------"it's like this"-------------------
Dan Sugalski even samurai
d...@sidhe.org have teddy bears and even
teddy bears get drunk
Matt Fowles
I still don't see why this necessitates keyed variants of all the ops.
Couldn't we have a "prepkeyed" op which would return a specialized pmc
to use in the next op? Thus instead of
set P0, P1[P2]
we would have
prepkey P3, P1[P2]
set P0, p#
similarly
add P0[IO], P1
would become
prepkey P2, P0[I0]
set P2, P1
Sorry if I am being a little slow here. But thanks for the full
explanation of active data.
Matt
Luke Palmer
> Dan~
>
> I still don't see why this necessitates keyed variants of all the ops.
> Couldn't we have a "prepkeyed" op which would return a specialized pmc
> to use in the next op?
Yes, we could. But it would be in exchange for both speed (because
we're allocating a new pmc each keyed acces) and code complexity on the
Parrot side (because we have to add a level of indirection for the proxy
pmc). And complex code often leads to even slower code.
Op count isn't that important. In those few places where it is
(embedded systems, eg.), it would theoretically possible to configure
which ops had keyed variants, and convert the bytecode accordingly (at
the possible expense of an extra, dedicated register as the
intermediate). But that probably won't happen, because then we have to
write both keyed ops and proxy ops, defeating the complexity argument.
There's something to be said for this way of doing things, however. In
particular, when you want eg. to tie to a database, and want things to
work just like normal except for fetches and stores, you only have to
overload the prepkey vtable to return a proxy pointing to a different
place, instead of all the vtable.
I like this proposal in general; I'm mostly worried about speed. If
someone who understands what makes things fast better than I do can
figure out a way to make this comparable in speed, I'd support it
(probably; I'd have to think some more first :-).
Luke
Togos
> compiletime whether
> assignment is rebinding or is a get/set, because we
> could have code
> like:
>
> kitchentemp = new('kitchenroom');
> bedroomtemp = new('bedroom');
> bedroomtemp = kitchentemp;
>
>
> which is typeless, and yet should still respect the
> fact that the
> objects bound to the name intercept assignment
> (basically overloading
> it) rather than having the compiler or runtime doing
> the rebinding
> for you.
>
> Since, of course, we're dealing with basically
> typeless languages we
> have to do all the checking at runtime (lucky us)
> which is why the
> PMCs have generic get and set methods so they can
> decide whether
> we're doing rebinding or something more funky.
Humm. Are you saying that whether 'set' does binding
or 'morphing' on a PMC will not be known at compile
time at the PASM level? So we don't know exactly what
set Px, Py
will do?
> (Whether this affects languages where = is an
> explicit binding rather than assignment is up
> in the air, but neither python nor ruby is truly
> that way, though they're close. But we can fake
> it so that things do what they ought when they
> ought)
Maybe there should be 3 'set' ops: 'assign'
which changes the object, 'bind' which
replaces it, and 'set' which does whatever
the heck the object wants it to do... It'd be
good for compilers that like to know what's
going on.
__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
Leopold Toetsch
> ...is operator overloading. Addition
> may not really be addition, if one or the other of the data in the
> operation overloads the addition operator.
pdd02 has 7 vtable slots for C<add>. How will
use overload '+' => \&my_add;
look like in Parrot?
leo
Dan Sugalski
> Dan~
>
> I still don't see why this necessitates keyed variants of all the ops.
> Couldn't we have a "prepkeyed" op which would return a specialized pmc
> to use in the next op?
We could--perl 5 uses a variant of this where active aggregates return a
placeholder variable with magic on it that then does the appropriate thing
when operated on. I can, though, learn from other people's mistakes if
they hurt enough. :)
Besides requiring active aggregates to create magic PMCs, it makes
accessing compact aggregates very inefficient. Consider the case where we
have an integer array we're working with--it means code like:
foreach loopvar (1..10000) {
total[loopvar] = somedata[loopvar] + otherdata[loopvar];
}
(That addition, BTW, could be a single op with keyed addition in the
vtables, though I wouldn't expect the majority of the usage to be that
simple)
If either (or both) of somedata or otherdata isn't an array of PMCs,
there's going to be an awful lot of temporary PMC creation, which is a
waste. We're trying to encourage the use of more compact aggregate
representations with Parrot, as currently Perl/Python/Ruby all have really
portly aggregates and part of the design goals is to allow for compact
versions in places we can be compact.
Dan
Dan Sugalski
The perl compiler will have to fabricate a bunch of phantom functions. Not
great, but this is already the case in perl 5 when using overloads, so
it's not a big deal.
Dan
Dan Sugalski
No. Set copies the pointer from source to destination. That's all it's
ever done, and what it's supposed to do.
> Maybe there should be 3 'set' ops: 'assign'
> which changes the object, 'bind' which
> replaces it, and 'set' which does whatever
> the heck the object wants it to do...
We already have this. Assign will copy the value in the source into the
destination PMC, and the destination is free to do what's needed to make
things work. The various lexical and global namespace stores are the
binding changes.
Most objects in Parrot will be dealt with by reference, which is to say
that the PMC that a name is bound to is not an object, but rather a
reference PMC that points to the object PMC. An assign, in that case, just
copies the value in the source reference (which is the pointer to the
referred to object) into the destination reference, so it all works out
OK.
I think I need to put together something more detailed on how this works,
though. I expect this explanation will just make things more, rather than
less, confusing.
Dan
Togos
> reference, which is to say
> that the PMC that a name is bound to is not an
> object, but rather a
> reference PMC that points to the object PMC. An
> assign, in that case, just
> copies the value in the source reference (which is
> the pointer to the
> referred to object) into the destination reference,
> so it all works out
> OK.
Ah. I see. I was confused because my language uses
the registers themselves as variables, instead of
storing variables in the registers. This would
explain a lot of the 'set-vs-assign' confusion,
too. Where I do a 'set', everyone else does
'assign'.
Not that I don't think that my way of thinking
about these things makes more sense in the general
case ;-), but... that explains a lot. I expect
IMCCs "=" to do a 'set', but everyone else expects
it to 'assign'. But last I checked it sometimes did
one and sometimes another :-P From what you wrote
earlier, it seems like IMCC's "=" operator should
always do 'assign'.
(see my rants here:
http://groups.google.com/groups?selm=20030822181316.4849.qmail%40web41413.mail.yahoo.com
, and here:
http://groups.google.com/groups?selm=20030814083616.27613.qmail%40web41415.mail.yahoo.com
, if you're interested in my idea of how registers
and values should be thought about)
> I think I need to put together something
> more detailed on how this works, though.
That would be nice. Maybe while writing it
you'll see the light and introduce a ":="
operator for IMCC ;-)
Leopold Toetsch
> foreach loopvar (1..10000) {
> total[loopvar] = somedata[loopvar] + otherdata[loopvar];
> }
> If either (or both) of somedata or otherdata isn't an array of PMCs,
> there's going to be an awful lot of temporary PMC creation, which is a
> waste.
You would need one temporary PMC per operand, which can be assigned a
value and it can be created outside of the loop. To achieve the value
semantics that above addition has, you have to create new destination
PMCs anyway (presuming that C<total> holds PMCs).
> ... We're trying to encourage the use of more compact aggregate
> representations with Parrot, as currently Perl/Python/Ruby all have really
> portly aggregates and part of the design goals is to allow for compact
> versions in places we can be compact.
I don't see the point of "compact" with ops-files 60 times of the
current size (with all ki?c? permutations) and huge classes files
implementing all the keyed variants.
When only add_p_k_p_k_p_k is implemented, we have additional checks for
NULL keys, additional calls to key_integer() to extract the array
index, and additonal code to set the value of the key(s).
Please read (again) my proposal in:
Subject: Re: keyed vtables
Date: Mon, 18 Aug 2003 09:36:40 +0200
> Dan
leo
Benjamin Goldberg
Be a bit verbose in saying why this is so. Something like:
The compiler can always tell that "+" between variables known to be
numeric types will always be addition; further, it can always tell that
"+" on variables known to be objects with an overloaded "+" will always
be a subroutine call. What it *can't* always tell, is whether "+"
between two variables whose types it doesn't know will be addition or a
subroutine call.
A language like C++ never has to deal with the uncertainty of the latter
situation, since it always knows the types of the variables.
In a language like perl5, the types of the variables aren't ever known,
so it generally can't *know* whether "+" will be addition or an
overloaded operator. (In few cases where it *can* know that it's
numeric addition, then it can usually constant fold. After constant
folding, we're left back in the ambigous case).
> In some languages the compiler can at least make some predictions of
> what operations will do, but we're not really interested in those
> languages.
Why not? You mean we won't be implementing them?
> Or, rather, it doesn't matter if we are or not, since
> perl, python, and ruby are all untyped so there's not much to be
> done, and full type inferencing's a halting-problem sort of thing.
> (Though a useful subset of many programs can be analyzed enough to do
> some optimization)
If it's a halting-problem sort of thing, then how does smalltalk work?
> Perl, at least, and because of it Parrot, goes one step further. Not
> only are operations overloaded, but assignment is overloaded.
In C++, overloading "=" overloads assignment.
In perl, overloading "=" doesn't overload assignment (it's still just a
pointer copy). Instead, it overloads how perl reacts to doing an
overloaded mutating operation (+=, -=, *=, /=, .=, etc.) on an object
whose refcount is > 1. (Obviously, that refcount only *got* to a number
> 1 due to an assignment. But it's the later overloaded operation, not the assignment itself, which triggers the subroutine which we passed to 'use overload "=" => ..."). The overloaded "=" is used to make a clone of the original object, and that clone is put into the variable... thus, your += (or whatever) changes the clone, not any of the other variables which formerly shared that pointer with it.
I suspect that in perl6, overloading "=" will overload assignment.
> If you're coming from a pure OO sort of language where assignment is
> just the association of a name and pointer to an object, this may
> seem kind of strange, but it is really useful.
>
> The canonical example is associating a hash with a database.
*Blink*. Now we're switching discussion from overloading to tieing.
Or were we really talking about tieing, and I was confused and thinking
of overloading?
> Reading or writing from the hash does a lookup in the backing database
> and reads or writes from it. A handy thing, with the variable getting in
> the way of reads or writes to itself. However, you can do this with
> plain scalar variables.
>
> For example, if you had code:
>
> int Foo;
> Dog Bar = new();
> Foo = Bar;
>
> you wouldn't want there to be a Dog in Foo--you've declared it an
> integer, so it can only hold an integer. The assignment can't do a
> plain pointer copy the way it would in the case where both sides can
> hold objects.
Indeed ... the compiler needs to detect that a conversion is necessary.
Fortunatly, since we've told the compiler the types of the variables, it
can do that.
> As an alternate example, try this:
>
> TempMeter kitchentemp = new('kitchenroom');
> TempMeter bedroomtemp = new('bedroom');
> bedroomtemp = kitchentemp;
>
> Where in this case kitchentemp *is* an object, but one of type
> TempMeter, and bound to an object that represents the temperature in
> your kitchen, while bedroomtemp is a TempMeter object that represents
> the temperature in your bedroom. (Code I'd not recommend using in CGI
> programs....) The assignment does *not* bind the bedroomtemp name to
> the object for the kitchen temp, rather it sets the bedroom
> temperature to be the same as that in the kitchen. Even being an
> object, the object's methods still intercept read and write requests
> and instead of assignment being rebinding, it's instead a get or set
> call on the object.
Again, we've told the compiler the types, so it can do that.
> The part that affects us is that we can't tell at compiletime whether
> assignment is rebinding or is a get/set, because we could have code
> like:
>
> kitchentemp = new('kitchenroom');
> bedroomtemp = new('bedroom');
> bedroomtemp = kitchentemp;
>
> which is typeless,
Actually, we *could* try to infer the types of the variables, based on
our knowledge of the types of the values assigned to them. I'll ignore
that,
and pretend that we don't know the return type of new() at compile time.
>and yet should still respect the fact that the
> objects bound to the name intercept assignment (basically overloading
> it) rather than having the compiler or runtime doing the rebinding
> for you.
>
> Since, of course, we're dealing with basically typeless languages we
> have to do all the checking at runtime (lucky us) which is why the
> PMCs have generic get and set methods so they can decide whether
> we're doing rebinding or something more funky. (Whether this affects
> languages where = is an explicit binding rather than assignment is up
> in the air, but neither python nor ruby is truly that way, though
> they're close. But we can fake it so that things do what they ought
> when they ought)
I forget, should "bedroomtemp = kitchentemp" be using set_pmc, or clone?
--
$a=24;split//,240513;s/\B/ => /for@@=qw(ac ab bc ba cb ca
);{push(@b,$a),($a-=6)^=1 for 2..$a/6x--$|;print "$@[$a%6
]\n";((6<=($a-=6))?$a+=$_[$a%6]-$a%6:($a=pop @b))&&redo;}
Leopold Toetsch
> Most objects in Parrot will be dealt with by reference,
Dou you have anything about references? I'm thinking about putting in a
default C<Ref> PMC class, which delegates almost all its methods to
C<SELF->cache.pmc_val>, autogenerated inside pmc2c.pl.
> Dan
leo
Piers Cawley
>> Or, rather, it doesn't matter if we are or not, since
>> perl, python, and ruby are all untyped so there's not much to be
>> done, and full type inferencing's a halting-problem sort of thing.
>> (Though a useful subset of many programs can be analyzed enough to do
>> some optimization)
>
> If it's a halting-problem sort of thing, then how does smalltalk
> work?
It doesn't do type inferencing. All messages are resolved at runtime
by the class of the object receiving the message. Unless I've
completely misunderstood what's going on.
>> Reading or writing from the hash does a lookup in the backing database
>> and reads or writes from it. A handy thing, with the variable getting in
>> the way of reads or writes to itself. However, you can do this with
>> plain scalar variables.
>>
>> For example, if you had code:
>>
>> int Foo;
>> Dog Bar = new();
>> Foo = Bar;
>>
>> you wouldn't want there to be a Dog in Foo--you've declared it an
>> integer, so it can only hold an integer. The assignment can't do a
>> plain pointer copy the way it would in the case where both sides can
>> hold objects.
>
> Indeed ... the compiler needs to detect that a conversion is necessary.
>
> Fortunatly, since we've told the compiler the types of the variables, it
> can do that.
How about this then:
sub wibble($arg is rw) {
$arg = Dog.new;
}
my int $foo;
my Cat $bar;
wibble(rand < 0.5 ?? $foo :: $bar);
The compiler does not and cannot know the type of thing passed
into wibble until runtime, so assignment must be handled by the
target thing. Yes, this is a pathological case, but it is a case that
must be handled.
There's no pretending required. In the vast majority of cases you
don't *know* the types returned by new. After all, something could
have altered the behaviour of Class::new after the code in question
is compiled but before it gets run.
>> and yet should still respect the fact that the
>> objects bound to the name intercept assignment (basically overloading
>> it) rather than having the compiler or runtime doing the rebinding
>> for you.
>>
>> Since, of course, we're dealing with basically typeless languages we
>> have to do all the checking at runtime (lucky us) which is why the
>> PMCs have generic get and set methods so they can decide whether
>> we're doing rebinding or something more funky. (Whether this affects
>> languages where = is an explicit binding rather than assignment is up
>> in the air, but neither python nor ruby is truly that way, though
>> they're close. But we can fake it so that things do what they ought
>> when they ought)
>
> I forget, should "bedroomtemp = kitchentemp" be using set_pmc, or
> clone?
set_pmc
Benjamin Goldberg
>
> Benjamin Goldberg <ben.go...@hotpop.com> writes:
[snip]
>>> Reading or writing from the hash does a lookup in the backing database
>>> and reads or writes from it. A handy thing, with the variable getting
>>> in the way of reads or writes to itself. However, you can do this with
>>> plain scalar variables.
>>>
>>> For example, if you had code:
>>>
>>> int Foo;
>>> Dog Bar = new();
>>> Foo = Bar;
>>>
>>> you wouldn't want there to be a Dog in Foo--you've declared it an
>>> integer, so it can only hold an integer. The assignment can't do a
>>> plain pointer copy the way it would in the case where both sides can
>>> hold objects.
>>
>> Indeed ... the compiler needs to detect that a conversion is necessary.
>>
>> Fortunatly, since we've told the compiler the types of the variables,
>> it can do that.
>
> How about this then:
>
> sub wibble($arg is rw) {
> $arg = Dog.new;
> }
>
> my int $foo;
> my Cat $bar;
>
> wibble(rand < 0.5 ?? $foo :: $bar);
>
> The compiler does not and cannot know the type of thing passed
> into wibble until runtime,
It knows that it's one(Cat, int). (OT for this thread, but I think
perl6 ought to have a "oneof" alias for the "one" superposition).
> so assignment must be handled by the target thing. Yes, this is a
> pathological case, but it is a case that must be handled.
But the target pmc doesn't know what kind of variable it's in!
Consider: At what point should the following code die?
sub wibble($arg is rw) {
$arg = Dog.new;
}
sub wobble(Mammal $arg is rw) {
my Mammal $m = $arg;
wibble($m);
wibble($arg);
}
my Mammal $foo;
my Cat $bar;
wobble( int(rand(2)) ?? $foo :: $bar );
Should wibble($m) always work without dieing? What about wibble($arg)?
Does wibble($m) cause $m to contain a new pmc (does the assignment
inside of wibble have set semantics to the containing variable), or does
it cause the existing pmc to be changed (does the assignment inside
wibble have assign semantics to the passed in pmc)?
It's a good thing we (will) have a Notification system, then, eh?
We can "know" the return type of new() at compile time, and if it
changes, and be notified and respond accordingly.
If the new return type of new() is sufficiently compatible with the old
type, no changes need to be made to the sub which is calling new(). If
the new type is sufficiently different, then the calling sub needs to be
recompiled (not from scratch, but from the point just before the type
inferencing engine got at it). This might be expensive, but how often
do the return types of functions change? Also, recompilation doesn't
need to happen right away -- we could set a flag, and defer until
necessary.
(Actually, it would be pretty cool if by default, perl only compiled
subs into ASTs, and didn't produce parrot bytecode until the sub first
gets called. This way, we can get the metainfo of a sub right away, as
soon as it's been seen, which is useful for code analysis and
inter-subroutine optomization, but we don't have to do the extra work if
it doesn't ever get called... which is good for big modules/libraries).
> >> and yet should still respect the fact that the
> >> objects bound to the name intercept assignment (basically overloading
> >> it) rather than having the compiler or runtime doing the rebinding
> >> for you.
> >>
> >> Since, of course, we're dealing with basically typeless languages we
> >> have to do all the checking at runtime (lucky us) which is why the
> >> PMCs have generic get and set methods so they can decide whether
> >> we're doing rebinding or something more funky. (Whether this affects
> >> languages where = is an explicit binding rather than assignment is up
> >> in the air, but neither python nor ruby is truly that way, though
> >> they're close. But we can fake it so that things do what they ought
> >> when they ought)
> >
> > I forget, should "bedroomtemp = kitchentemp" be using set_pmc, or
> > clone?
>
> set_pmc
--
Dan Sugalski
Yeah, that was the way I was planning on going. (I'm not sure if I
sent this already, but a resend never hurts :)
Leopold Toetsch
> At 10:57 AM +0200 8/29/03, Leopold Toetsch wrote:
>>Dou you have anything about references? I'm thinking about putting in a
>>default C<Ref> PMC class, which delegates almost all its methods to
>>C<SELF->cache.pmc_val>, autogenerated inside pmc2c.pl.
> Yeah, that was the way I was planning on going. (I'm not sure if I
> sent this already, but a resend never hurts :)
Done. As a reference delegates assign too, there is currently no way to
rebind a Ref PMC to another variable. (We could implement a "bind"
method for this or not delegate set_pmc).
Please re^Wsend design docs ;-)
leo