Using Parrot Magic Cookies in perl 5.10
Arthur Bergman
About a week late here is my report on moving to PMCs for perl 5.
Goal, removing SVt_* and replacing them with PMC instead.
Why,
1) SV_t* is limited today
2) tie is slower than slower
3) tie semantics are wrong
4) overloading is a bad hack
5) wrapping data for thread shared data is more or less impossible
6) we spend alot of time branching around checking what kind of Sv we
are dealing with
However, trying to compare sv.c against PMC makes me think that this is
might not be worthwhile why? just because all things are bad hacks that
rely on poking inside SVs even if they shouldn't.
In theory it would be simple, change SvIV(a) to PMC_get_integer(a);
(which would be a->vtable->get_integer(a)), however checking in the
source code there are many MANY places were we abuse the SV_ANY thread
so much that one would have to visist every line of code using svc.
If we decide to proceed it is just a simple matter of going over every
line and making sure it does the right thing
so essentially the code for printing
case SVt_NULL:
if (ckWARN(WARN_UNINITIALIZED))
report_uninit();
return TRUE;
case SVt_IV:
if (SvIOK(sv)) {
if (SvGMAGICAL(sv))
mg_get(sv);
if (SvIsUV(sv))
PerlIO_printf(fp, "%"UVuf, (UV)SvUVX(sv));
else
PerlIO_printf(fp, "%"IVdf, (IV)SvIVX(sv));
return !PerlIO_error(fp);
}
/* FALL THROUGH */
default:
if (PerlIO_isutf8(fp)) {
if (!SvUTF8(sv))
sv_utf8_upgrade(sv = sv_mortalcopy(sv));
}
else if (DO_UTF8(sv)) {
if (!sv_utf8_downgrade((sv = sv_mortalcopy(sv)), TRUE)
&& ckWARN_d(WARN_UTF8))
{
Perl_warner(aTHX_ packWARN(WARN_UTF8), "Wide character
in print"
);
}
}
tmps = SvPV(sv, len);
break;
}
if (len && (PerlIO_write(fp,tmps,len) == 0))
return FALSE;
return !PerlIO_error(fp);
with basicly
PerlIO_write(sv->vtable->get_string(sv)); #hows that for
readability :-),
with eventually some way to check unicodeness on the sv and the perlio
or somesuch.
The code for helem
if (SvTYPE(hv) == SVt_PVHV) {
if (PL_op->op_private & OPpLVAL_INTRO) {
MAGIC *mg;
HV *stash;
/* does the element we're localizing already exist? */
preeminent =
/* can we determine whether it exists? */
( !SvRMAGICAL(hv)
|| mg_find((SV*)hv, PERL_MAGIC_env)
|| ( (mg = mg_find((SV*)hv, PERL_MAGIC_tied))
/* Try to preserve the existenceness of a tied
hash
* element by using EXISTS and DELETE if
possible.
* Fallback to FETCH and STORE otherwise */
&& (stash = SvSTASH(SvRV(SvTIED_obj((SV*)hv,
mg))))
&& gv_fetchmethod_autoload(stash, "EXISTS", TRUE)
&& gv_fetchmethod_autoload(stash, "DELETE", TRUE)
)
) ? hv_exists_ent(hv, keysv, 0) : 1;
}
he = hv_fetch_ent(hv, keysv, lval && !defer, hash);
svp = he ? &HeVAL(he) : 0;
}
pp_push would be
PP(pp_push)
{
dSP; dMARK; dORIGMARK; dTARGET;
register AV *ary = (AV*)*++MARK;
register SV *sv = &PL_sv_undef;
MAGIC *mg;
if ((mg = SvTIED_mg((SV*)ary, PERL_MAGIC_tied))) {
*MARK-- = SvTIED_obj((SV*)ary, mg);
PUSHMARK(MARK);
PUTBACK;
ENTER;
call_method("PUSH",G_SCALAR|G_DISCARD);
LEAVE;
SPAGAIN;
}
else {
/* Why no pre-extend of ary here ? */
for (++MARK; MARK <= SP; MARK++) {
sv = NEWSV(51, 0);
if (*MARK)
sv_setsv(sv, *MARK);
av_push(ary, sv);
}
}
SP = ORIGMARK;
PUSHi( AvFILL(ary) + 1 );
RETURN;
}
would essentially be
PP(pp_push)
{
register PMC* av = *MARK++;
dSP; dMARK; dORIGMARK; dTARGET;
for (++MARK; MARK <= SP; MARK++) {
av->vtable->push_pmc(av,*MARK->vtable->clone(*MARK));
}
}
sv_setsv is a good example of a method that would be much much more
simple to implmenet
( new_sv->vtable->set_pmc(new_sv, sv) ) making it actually fit in the
cache ;)
To make this happen I would need one or two dedicated volunteers willing
to work the next couple of months with a good round of tuits available,
I alone can't make this happen, one bonus of the work would of course be
extensive testing of the PMCs (so maybe even some perl6 people would be
interested ;-)
Arthur
ps, I know this is a controversial issue, if you intend to press reply
and say
"it can't be done" please don't, everything can be done :-)
"it will break XS" to make this happen obviously we cannot break any XS
that uses perl documented APIs, so old magic and old code should do the
right thing, this is given and if this can't happen then there is no
point in going ahead, so if you are going to say "it will break XS",
please don't press send
Graham Barr
> ps, I know this is a controversial issue, if you intend to press reply
> and say
>
> "it can't be done" please don't, everything can be done :-)
>
> "it will break XS" to make this happen obviously we cannot break any XS
> that uses perl documented APIs, so old magic and old code should do the
> right thing, this is given and if this can't happen then there is no
> point in going ahead, so if you are going to say "it will break XS",
> please don't press send
Well I am not going to say it cannot be done, but it will take a lot of time
to do it. When are we expecting 5.10 to be released ?
As this is such a huge project, I would suggest it be done in a separate branch
in perforce, just like Nick did with perlio, and then merged into the main
lne when there is something close to stable.
That way if it does take a lot longer than you expect, we can release 5.10 and
it could be merged into 5.12. Or after you have done a lot of work, and I hope
this does not happen, you find out that it is not going to work, we wont
have to try to revert code.
Graham.
Arthur Bergman
On fredag, augusti 30, 2002, at 09:19 , Graham Barr wrote:
>
> Well I am not going to say it cannot be done, but it will take a lot of
> time
> to do it. When are we expecting 5.10 to be released ?
>
> As this is such a huge project, I would suggest it be done in a
> separate branch
> in perforce, just like Nick did with perlio, and then merged into the
> main
> lne when there is something close to stable.
>
Agreed, it would have to be developed in a separate branch.
> That way if it does take a lot longer than you expect, we can release
> 5.10 and
> it could be merged into 5.12. Or after you have done a lot of work, and
> I hope
> this does not happen, you find out that it is not going to work, we wont
> have to try to revert code.
>
> Graham.
Arthur
Mark-Jason Dominus
Is seems the me that this comparison is deceptive, because you have
omitted all the code for push_pmc. I wonder what the code savings
would really look like.
Arthur Bergman
On fredag, augusti 30, 2002, at 01:16 , Mark-Jason Dominus wrote:
>
> Is seems the me that this comparison is deceptive, because you have
> omitted all the code for push_pmc. I wonder what the code savings
> would really look like.
>
>
Indeed I did, the code for push_pmc would essentially be the same code
that already today exists in av_push, no big change there, the point is
that the opcode doesn't need to distinguish between tied and non tied
(or other magic arrays), instead tied arrays would use an different
vtable that would call the PUSH function.
Arthur
Dave Mitchell
> On Thu, Aug 29, 2002 at 11:30:13PM +0200, Arthur Bergman wrote:
> > ps, I know this is a controversial issue, if you intend to press reply
> > and say
> >
> > "it can't be done" please don't, everything can be done :-)
> >
> > "it will break XS" to make this happen obviously we cannot break any XS
> > that uses perl documented APIs, so old magic and old code should do the
> > right thing, this is given and if this can't happen then there is no
> > point in going ahead, so if you are going to say "it will break XS",
> > please don't press send
>
> Well I am not going to say it cannot be done, but it will take a lot of time
> to do it. When are we expecting 5.10 to be released ?
My personal feeling on the matter is that it's such a big project and at
will be so hard to (a) to get everything working right without breaking
subtle semantics all over the place, and (b) avoid breaking lots of XS
code, that I can't help feeling that any such programming effort would be
better off applied to Perl 6 instead.
Just MHO of course.
Dave.
--
But Pity stayed his hand. "It's a pity I've run out of bullets", he
thought. - "Bored of the Rings"
h...@crypt.org
:Goal, removing SVt_* and replacing them with PMC instead.
:Why,
:1) SV_t* is limited today
:2) tie is slower than slower
:3) tie semantics are wrong
:4) overloading is a bad hack
:5) wrapping data for thread shared data is more or less impossible
:6) we spend alot of time branching around checking what kind of Sv we
:are dealing with
Thanks for the synopsis.
There are two primary issues at stake here: code cleanliness and speed.
I am confident that this approach would allow us to make the source
code much cleaner, since we wouldn't need to spread knowledge of things
like RMAGIC all around the source code. What about the speed issue?
As I see it, this approach saves us somewhere between one and several
tests (usually bit tests on flags or masked integer tests on SvTYPE),
and costs us a goto or a function call, with all the cache penalties
that implies. So is the approach that "it's worth the cost for the
code cleanliness benefits"? Or do you think that speed cost can be
clawed back either simply by the number of tests saved or elsewhere?
My other issue - one I also worry will impact perl6, though it may
already have been resolved on some mailing list I don't read - is
what happens with contention for PMC slots? By which I mean, what
happens if two types of magic attached to the same SV both want to
overload the same PMC slot? Currently, the behaviour will depend
on the order the magics appear in the chain in what I believe is
a relatively predictable manner (though it causes us some problems
already, as evidenced by pos/taint contention). In particular,
I don't think we can rely on cooperative chaining: it would be
a problem if a badly written magic override could break tainting,
for example.
On the gripping hand, I'm guessing that XS/C code will run through
some relatively slow compatibility layer unless rewritten to use
the PMCs directly. If so, can we make any predictions about the
degree of slowdown likely to be involved?
Don't get me wrong - I'd love to see this happen. I want only to
help think of the potential problems before you waste a lot of time
heading down an unviable path.
Hugo