RFC: extending the interface for tied scalars
Tassilo Von Parseval
when looking at the things that can be done with tied scalars, hashes,
arrays and filehandles, I find it a little limiting that one can only
define a STORE and FETCH method for scalars.
The problem is that when you do a
substr($tied_scalar, ...);
this becomes
substr(tied($tied_scalar)->FETCH, ...);
when it could be much more powerful if that would become
tied($tied_scalar)->SUBSTR(...);
I was once bitten by this when I was toying with the idea of writing
zlib-compressed scalars. My idea was to store very huge strings in a
scalar and save on memory. But every operation on the scalar other than
assigning or reading it blew my memory because the FETCH operation
returned the whole string uncompressed.
I looked at a possible way of implementing it. What needs to be done for
this is (all examples just use SUBSTR as the additional method that can
be defined for the moment):
1) Change Perl_magic_getpack:
int
Perl_magic_getpack(pTHX_ SV *sv, MAGIC *mg)
{
dSP;
if (mg->mg_ptr)
mg->mg_flags |= MGf_GSKIP;
switch(PL_op->op_type) {
case OP_SUBSTR:
break;
default:
magic_methpack(sv,mg,"FETCH");
}
return 0;
}
This will prevent triggering "FETCH" when the current op is OP_SUBSTR.
Every op that can be re-implemented in the tied class needs to be listed
here:
case OP_SUBSTR:
case OP_CHOP:
case OP_etc:
break;
2) Change PP(pp_OPERATION) accordingly. For substr():
PP(pp_substr)
{
...
MAGIC *mg;
...
SvTAINTED_off(TARG); /* decontaminate */
SvUTF8_off(TARG); /* decontaminate */
if (num_args > 2) {
if (num_args > 3) {
repl_sv = POPs;
repl = SvPV(repl_sv, repl_len);
repl_is_utf8 = DO_UTF8(repl_sv) && SvCUR(repl_sv);
}
len = POPi;
}
pos = POPi;
sv = POPs;
PUTBACK;
if (SvMAGICAL(sv) && (mg = mg_find(sv, PERL_MAGIC_tiedscalar)))
/* trigger "SUBSTR" method here and return */
...
}
Does the sound like something that could be tackled for 5.10.0 or is it
too scary and gross? The necessary changes are confined to
Perl_magic_getpack and pp_OPERATION, I think. It's just that they could
become pretty numerous if one decides to allow redefinitions of many Perl
built-ins.
There are also some interface considerations. For instance, I wouldn't
want to force a programmer to define every single scalar operation.
Instead, one could make these additional methods optional so that still
only TIESCALAR, FETCH and STORE would need to be implemented. If these
optional methods aren't defines, the old behaviour (with a preliminary
FETCH) would still be in effect. As far as I see it, this would ensure
backwards-compatibility.
Tassilo
--
$_=q#",}])!JAPH!qq(tsuJ[{@"tnirp}3..0}_$;//::niam/s~=)]3[))_$-3(rellac(=_$({
pam{rekcahbus})(rekcah{lrePbus})(lreP{rehtonabus})!JAPH!qq(rehtona{tsuJbus#;
$_=reverse,s+(?<=sub).+q#q!'"qq.\t$&."'!#+sexisexiixesixeseg;y~\n~~dddd;eval
Elizabeth Mattijsen
>I looked at a possible way of implementing it. What needs to be done for
>this is (all examples just use SUBSTR as the additional method that can
>be defined for the moment):
Sounds to me like your efforts would be better directed at using
Parro't vtable's in Ponie. That should allow all of the things you
want to do, at least the way I understand vtables.
Liz
Rafael Garcia-Suarez
> The problem is that when you do a
>
> substr($tied_scalar, ...);
>
> this becomes
>
> substr(tied($tied_scalar)->FETCH, ...);
>
> when it could be much more powerful if that would become
>
> tied($tied_scalar)->SUBSTR(...);
So SUBSTR could be a lvalue method ? (I suspect hidden difficulties.)
Tassilo Von Parseval
I am not familiar with the Parrot internals, so I can't say. But if I
understand Ponie correctly, it would have to be in blead anyway in order
to get into Ponie as well, no?
Elizabeth Mattijsen
>On Thu, Dec 04, 2003 at 11:15:38AM +0100 Elizabeth Mattijsen wrote:
>> At 11:01 +0100 12/4/03, Tassilo von Parseval wrote:
>> >I looked at a possible way of implementing it. What needs to be done for
>> >this is (all examples just use SUBSTR as the additional method that can
> > >be defined for the moment):
> > Sounds to me like your efforts would be better directed at using
>> Parro't vtable's in Ponie. That should allow all of the things you
> > want to do, at least the way I understand vtables.
>I am not familiar with the Parrot internals, so I can't say. But if I
>understand Ponie correctly, it would have to be in blead anyway in order
>to get into Ponie as well, no?
That's my understanding also. Maybe Arthur might be able to shed
some more light on this.
Liz
Dave Mitchell
[suggestion for optional SUBSTR etc methods which trump FETCH ]
I've always disliked proposals which add an optional tie method that
defaults to FETCH when not found, because it does nasty things in the face
of inheritance. See for example the discussion of FETCHSLICE on p5p a
couple of years back.
A typical example of the problem:
Person A adds a useful module to CPAN that does something with tying, and
has a FETCH method. That module becomes popular and widely used.
Person B writes a bit of code which uses that module but subclasses
it, where say they implement their own wrapper to FETCH:
package Derived;
use base 'PopularTieModule';
sub FETCH { ..do something...; SUPER->FETCH(@_); }
Person A learns of the new optional SUBSTR facility and adds it to a new
release of PopularTieModule.
Person B (or someone using B's code) downloads the newer version of
PopularTieModule. Suddenly code using Derived stops calling
Derived::FETCH, and calls PopularTieModule::SUBSTR instead, since Perl
Searches through the inheritance tree for SUBSTR before searching again
for FETCH. Whoops!
I did suggest at the time that all such methods could be searched for in
parallel (ie look for SUBSTR then FETCH in the current class before
searching the parents), but at the time this was discussed it was felt
that this might be too big a change with implications of its own.
--
In England there is a special word which means the last sunshine
of the summer. That word is "spring".
Tassilo Von Parseval
Alright, I didn't quite anticipate this case. :-/
But playing around a little bit with it, it looks as though this is
doable as well. And it's not even very hard. This
substr($tied, 1, 1) = "bla";
must translate into
# call STORE with either 0 or 1 as fifth argument
# 1 means it is lvalue-substr
SUBSTR(tied($tied), 1, 1, undef, 1);
# afterwards call store
tied($tied)->STORE("bla");
So the only thing that pp_substr() has to ensure is that afterwards STORE
is called on tied($tied). Since the user implementing the tie can check the fifth
argument of SUBSTR(), he knows whether there will be a STORE-operation
afterwards or not and so he could set a flag somewhere that he could
later check in STORE().
The pleasant side-effect of this is that there is no longer any
PERL_MAGIC_substr involved for tied scalars:
if (SvMAGICAL(sv) && (mg = mg_find(sv, PERL_MAGIC_tiedscalar))) {
/* call SUBSTR */
...
/* ensure that the STORE is triggered properly on sv */
TARG = sv;
RETURN;
}
Here's a fictious example how a lvalue-substr would look like in the
tie-class:
package Tie::Substr;
sub TIESCALAR {
bless { _string => undef,
_lval => [],
} => $_[0];
}
sub STORE {
my ($self, $val) = @_;
my $lv = $self->{_lval};
if ($lv) {
undef $self->{_lval};
return substr $self->{_string}, $lv->[0], $lv->[1], $val;
}
$self->{_string} = $val;
}
sub SUBSTR {
my ($self, $start, $length, $repl, $lval) = @_;
if ($lval) {
$self->{ _lval } = [ $start, $length ];
return;
}
if ($repl) {
return substr $self->{_string}, $start, $length, $repl;
}
return substr $self->{_string}, $start, $length;
}
This should make the tied scalar behave identically with respect to
substr() compared to an untied counterpart.
Arthur Bergman
On Thursday, December 4, 2003, at 10:23 am, Tassilo von Parseval wrote:
>
> I am not familiar with the Parrot internals, so I can't say. But if I
> understand Ponie correctly, it would have to be in blead anyway in
> order
> to get into Ponie as well, no?
>
>
No
Arthur
Tassilo Von Parseval
On Thu, Dec 04, 2003 at 01:15:39PM +0000 Dave Mitchell wrote:
> On Thu, Dec 04, 2003 at 01:19:20PM +0100, Tassilo von Parseval wrote:
> > On Thu, Dec 04, 2003 at 10:54:31AM +0000 Dave Mitchell wrote:
> >
> > > On Thu, Dec 04, 2003 at 11:01:56AM +0100, Tassilo von Parseval wrote:
> > >
> > > [suggestion for optional SUBSTR etc methods which trump FETCH ]
> > >
> > > I've always disliked proposals which add an optional tie method that
> > > defaults to FETCH when not found, because it does nasty things in the face
> > > of inheritance. See for example the discussion of FETCHSLICE on p5p a
> > > couple of years back.
> > >
> > > A typical example of the problem:
> > >
> > > Person A adds a useful module to CPAN that does something with tying, and
> > > has a FETCH method. That module becomes popular and widely used.
> > >
> > > Person B writes a bit of code which uses that module but subclasses
> > > it, where say they implement their own wrapper to FETCH:
> > >
> > > package Derived;
> > > use base 'PopularTieModule';
> > > sub FETCH { ..do something...; SUPER->FETCH(@_); }
> > >
> > > Person A learns of the new optional SUBSTR facility and adds it to a new
> > > release of PopularTieModule.
> > >
> > > Person B (or someone using B's code) downloads the newer version of
> > > PopularTieModule. Suddenly code using Derived stops calling
> > > Derived::FETCH, and calls PopularTieModule::SUBSTR instead, since Perl
> > > Searches through the inheritance tree for SUBSTR before searching again
> > > for FETCH. Whoops!
> >
> > Hmmh, no, why should that happen? With 'optional' I meant something
> > else.
> >
> > SUBSTR() is _only_ ever called when 'substr($tied,...)' happens. What I
> > meant with optional was that the SUBSTR() method does not have to exist.
> > If none has been defined, it should simply revert to the current
> > behaviour of
> >
> > substr( tied($tied)->FETCH, ...);
> >
> > I don't think that this is dangerous with respect to inheritance or so.
>
> [ was your post meant to to be CC'ed to P5p? ]
Sorry, yes, it was. For some reason group-reply of my MUA missed the porters.
> No, it *is* dangerous.
>
> A more detailed example.
>
> Popular CPAN module looks like this:
>
> package PopularTieModule;
> sub FETCH { print "PopularTieModule FETCH called\n"; .... }
>
> a local module:
>
> package Derived;
> use base 'PopularTieModule';
> sub FETCH { print "Derived::FETCH called\n"; SUPER->FETCH(@_); }
>
> local code:
>
> use Derived;
> tie $s, 'Derived';
> substr($s,....);
>
> running the local code outputs:
>
> Derived::FETCH called
> PopularTieModule FETCH called
>
> Now a new version of PopularTieModule is released to CPAN:
>
> package PopularTieModule;
> sub FETCH { print "PopularTieModule FETCH called\n"; .... }
> sub SUBSTR { print "PopularTieModule SUBSTR called\n"; .... }
>
> the local site installs this newer version, and runs their local code
> again; this code now outputs:
>
> PopularTieModule SUBSTR called
>
> Note how the wrapper in Derived is suddenly no longer called.
This entirely depends on how the existance of "SUBSTR" is checked. As I
planned it, the above wouldn't happen because I would not scan the classes
@ISA for the existance of the suitable method. In Perl-code, all I would
do is
if (defined &Derived::SUBSTR) {
tied($tied)->SUBSTR(...);
} else {
# don't even bother whether @{ ref(tied($tied)) . "::ISA" }
# contains a SUBSTR method somewhere
substr(tied($tied), ...);
}
But now I see another problem. If we did it this way, we suddenly had two
different kinds of methods: namely those that get inherited (TIESCALAR,
FETCH, STORE) and those that don't. I don't find this particularly
convincing either.
This makes subclassing awkward since you'd have to provide these methods
even if your superclass defines them. Or you import them, but
explicitely importing methods is, ahem, strange.
Right now I can't think of a clean way to get out of this dilema. :-(
I'd go for no inheritance for those methods, but I am quite aware that
many people will have some valid objections against that.
Mark Jason Dominus
> even if your superclass defines them. Or you import them, but
> explicitely importing methods is, ahem, strange.
I once got the idea that @ISA might have been a mistake, and that
inheritance should have been accomplished through importation
instead. A derived class would say something like
use base Base_Class;
and that would (a) load Base_Class if it hadn't been already, and (b)
import all the methods from Base_Class to Subclass, something like
this:
while (my ($name, $glob) = each %Base_Class::) {
if (is_method(\&$glob) && ! defined &{"Subclass::$name"}) {
*{Subclass::$name} = \&$glob;
}
}
If you implement is_method like this:
sub is_method { 1 }
you get behavior something like the current behavior. I think a lot
of odd problems go away, although I of course don't have a clear view
of the odd problems this regime would introduce.
As an orthogonal issue, I wanted to define is_method() like this:
sub is_method {
my $coderef = shift;
prototype($coderef) eq "Method";
}
then, to define a method, you would say
sub new (Method) { ... }
sub get_color (Method) { ... }
. This used to work until someone decided to make it a syntax error
for no particular reason.
Rafael Garcia-Suarez
> I once got the idea that @ISA might have been a mistake, and that
> inheritance should have been accomplished through importation
> instead. A derived class would say something like
>
> use base Base_Class;
>
> and that would (a) load Base_Class if it hadn't been already, and (b)
> import all the methods from Base_Class to Subclass, something like
> this:
>
> while (my ($name, $glob) = each %Base_Class::) {
> if (is_method(\&$glob) && ! defined &{"Subclass::$name"}) {
> *{Subclass::$name} = \&$glob;
> }
> }
>
> If you implement is_method like this:
>
> sub is_method { 1 }
>
> you get behavior something like the current behavior. I think a lot
> of odd problems go away, although I of course don't have a clear view
> of the odd problems this regime would introduce.
Doesn't this modify the search order for methods when there is
diamond multiple inheritance ?
> As an orthogonal issue, I wanted to define is_method() like this:
>
> sub is_method {
> my $coderef = shift;
> prototype($coderef) eq "Method";
> }
>
> then, to define a method, you would say
>
> sub new (Method) { ... }
> sub get_color (Method) { ... }
>
> . This used to work until someone decided to make it a syntax error
> for no particular reason.
The :method built-in attribute exists (and AFAIK isn't documented and
used, apart from an obscure sub-case of the lexer.)
Steve Hay
>Mark Jason Dominus wrote:
>
>
>>As an orthogonal issue, I wanted to define is_method() like this:
>>
>> sub is_method {
>> my $coderef = shift;
>> prototype($coderef) eq "Method";
>> }
>>
>>then, to define a method, you would say
>>
>> sub new (Method) { ... }
>> sub get_color (Method) { ... }
>>
>>. This used to work until someone decided to make it a syntax error
>>for no particular reason.
>>
>>
>
>The :method built-in attribute exists (and AFAIK isn't documented and
>used, apart from an obscure sub-case of the lexer.)
>
>
The :method attribute is documented in the attributes.pm manpage ;)
I sometimes use it when declaring a "Perl Method Handler" in mod_perl.
- Steve
------------------------------------------------
Radan Computational Ltd.
The information contained in this message and any files transmitted with it are confidential and intended for the addressee(s) only. If you have received this message in error or there are any problems, please notify the sender immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Radan Computational Ltd. The recipient(s) of this message should check it and any attached files for viruses: Radan Computational will accept no liability for any damage caused by any virus transmitted by this email.
Mark Jason Dominus
> Mark Jason Dominus wrote:
> > I once got the idea that @ISA might have been a mistake, and that
> > inheritance should have been accomplished through importation
> > instead.
>
> diamond multiple inheritance ?
Even better, it rules out multiple inheritance entirely.
(I didn't mean to suggest that my idea was correct, or that it was a
good idea.)
Michael G Schwern
> I once got the idea that @ISA might have been a mistake, and that
> inheritance should have been accomplished through importation
> instead. A derived class would say something like
>
> use base Base_Class;
>
> and that would (a) load Base_Class if it hadn't been already, and (b)
> import all the methods from Base_Class to Subclass, something like
> this:
>
> while (my ($name, $glob) = each %Base_Class::) {
> if (is_method(\&$glob) && ! defined &{"Subclass::$name"}) {
> *{Subclass::$name} = \&$glob;
> }
> }
There is a name for this in the Ruby world: mixin. Ruby has only single
inheritence. To emulate multiple inheritence you use a mixin. This
neatly solves the diamond inheritence problem and the problem of accidental
inheritence.
I started an implementation of this in Perl a while ago. Its immature.
http://search.cpan.org/dist/mixin/
--
Michael G Schwern sch...@pobox.com http://www.pobox.com/~schwern/
Cottleston, Cottleston, Cottleston Pie.
A fly can't bird, but a bird can fly.
Ask me a riddle and I reply:
"Cottleston, Cottleston, Cottleston Pie."
Michael G Schwern
> too scary and gross?
I'd say...
A) If we're going to start extending the tied scalar interface, why stop
with SUBSTR? Why not chomp and chop and length and s/// and all that?
What's special about SUBSTR that it needs its own tie method?
B) I agree with what has already been said about the problems of having
a fallback method if a specialized tied method exists. This is a problem
I'd rather just leave for Perl 6.
And my emotional response: Ties are frightening. Don't add to the problem.
Wait for PONIE to clean up the mess.
--
Michael G Schwern sch...@pobox.com http://www.pobox.com/~schwern/
<GuRuThuG> make a channel called #Perl, and infest it with joking and
fun.... it doesnt make alot of sense.
Tassilo Von Parseval
> On Thu, Dec 04, 2003 at 11:01:56AM +0100, Tassilo von Parseval wrote:
> > Does the sound like something that could be tackled for 5.10.0 or is it
> > too scary and gross?
>
> I'd say...
>
> A) If we're going to start extending the tied scalar interface, why stop
> with SUBSTR? Why not chomp and chop and length and s/// and all that?
> What's special about SUBSTR that it needs its own tie method?
There is nothing special at all about substr(). I merely used this as an
example since it was the function for which I test-implemented the
tying. This turned out to be a good test-case since it involved
lvalueness as Rafael reminded me of.
As for what functions should be (if ever) included in it, I thought about
those listed under "Functions for SCALARs or strings" in perlfunc.pod.
> B) I agree with what has already been said about the problems of having
> a fallback method if a specialized tied method exists. This is a problem
> I'd rather just leave for Perl 6.
Yes, this is the tough part here.
> And my emotional response: Ties are frightening. Don't add to the problem.
> Wait for PONIE to clean up the mess.
That's probably a matter of personal taste. I find other parts of the
interpreter much harder to grasp. I was rather surprised to see that it
could be done for one function with relative ease.
Nick Ing-Simmons
>> This makes subclassing awkward since you'd have to provide these methods
>> even if your superclass defines them. Or you import them, but
>> explicitely importing methods is, ahem, strange.
>
>
>I once got the idea that @ISA might have been a mistake, and that
>inheritance should have been accomplished through importation
>instead. A derived class would say something like
>
> use base Base_Class;
>
>and that would (a) load Base_Class if it hadn't been already, and (b)
>import all the methods from Base_Class to Subclass, something like
>this:
>
> while (my ($name, $glob) = each %Base_Class::) {
> if (is_method(\&$glob) && ! defined &{"Subclass::$name"}) {
> *{Subclass::$name} = \&$glob;
> }
> }
Which won't help the case described for SUBSTR method added to base class
but not added to derived. It will still import the SUBSTR which won't
have the derived class's FETCH addditions.
(The above effectively happens in a lazy manner with current code.)
Also with that scheme how do you do ->SUPER::Method
Nick Ing-Simmons
>On Thu, Dec 04, 2003 at 02:23:21PM -0800 Michael G Schwern wrote:
>> On Thu, Dec 04, 2003 at 11:01:56AM +0100, Tassilo von Parseval wrote:
>> > Does the sound like something that could be tackled for 5.10.0 or is it
>> > too scary and gross?
>>
>> I'd say...
>>
>> A) If we're going to start extending the tied scalar interface, why stop
>> with SUBSTR? Why not chomp and chop and length and s/// and all that?
>> What's special about SUBSTR that it needs its own tie method?
>
>There is nothing special at all about substr().
Snag is there are a whole lot of ops that can modify a scalar.
I hope perl6 will clear up the duality between tie and overload.
$tied .= $what;
can't be done cleanly right now
Abigail
What about adding 2 arguments to FETCH (and perhaps STORE),
which indicate what action caused the FETCH? For instance, a
substr $tied, 1, 2, "foo";
causes
FETCH ($tied, "substr", [1, 2, "foo"])
to be called?
It would not solve all the inheritance problems, but it would solve the
problem mentioned by Dave.
It also makes is easier to add similar tie magic to chomp, chop, and
other argument modifying functions (s///!)
Abigail
Tassilo Von Parseval
The advantage is that the the class implementing the tie does not need
any new methods. All could be done from FETCH and STORE. And since
STORE (2) and FETCH (1) always receive the same amount of arguments,
existing code would probably not have to be changed unless it was
written in a very strange way.
> It also makes is easier to add similar tie magic to chomp, chop, and
> other argument modifying functions (s///!)
I didn't yet check, but if I understand the above correctly, the only
parts of the core that would need to be changed are
Perl_magic_(g|s)etpack. The current operation can be figured out by
looking at PL_op. Then the right amount of arguments are to be stuffed
into an array and the operation plus the arrayref pushed onto the stack.
Implementation-wise this is quite nice.
Tassilo Von Parseval
> What about adding 2 arguments to FETCH (and perhaps STORE),
> which indicate what action caused the FETCH? For instance, a
>
> substr $tied, 1, 2, "foo";
>
> causes
>
> FETCH ($tied, "substr", [1, 2, "foo"])
>
> to be called?
>
>
> It would not solve all the inheritance problems, but it would solve the
> problem mentioned by Dave.
Having thought this over a little, I see a much more serious problem
with this:
substr($tied, ...);
becomes:
$val = tied($tied)->FETCH("substr", [...]);
substr($val, ...);
What's with the substr() statement? Should it be called or not? Perl has
no chance of figuring out whether the user's FETCH method does handle
the substr() or not. If it handles it, the current OP (which is
OP_SUBSTR) would have to be skipped. Otherwise, CORE::substr() would
have to be called. Core-wise, the problem is whether to
PL_op = PL_op->op_next;
or not in Perl_magic_getpack(). If there was a way to figure this out
(although there is, for instance by having the user set a special
variable to true when the next op should be skipped), the idea with the
additional arguments would be quite tempting.
Anyway, waiting for ponie is probably the best thing to do. There are
more urgent issues.
David Nicol
>
> A) If we're going to start extending the tied scalar interface, why stop
> with SUBSTR? Why not chomp and chop and length and s/// and all that?
> What's special about SUBSTR that it needs its own tie method?
SUBSTR is how we look at parts of a string without looking at the
whole thing. All the functions that work with parts of strings can
be described in terms of SUBSTR. Cho(m)p, s///, y///, etc.
Even fetch and store can be described in terms of substr. since substr
can be used as a lvalue.
If scalar tying had a way other than STORE to deal with lvalues -- a
declarative rather than procedural approach -- it might make sense to
have all scalar tying in terms of SUBSTR instead of FETCH/STORE.
However, at some point we need to declare what procedure is to be
followed at assignment time. As long as the perl scalar is a union
of string/number/perl-reference, we could split the tie interface into
multiple pieces depending on what is wanted -- but then we have broken
the perl abstraction.
What kinds of applications really require into-the-string access?
Tieing a scalar to a file comes to mind. We can't set up a tie::slurp
or a tie::mmap module using STORE/FETCH, because there is no way to
trap the perl-internal partial scalar accesses using the tie interface.
That is the perceived need that adding SUBSTR would fill.
What we can do, however, when we want to refer to too-large data using
some kind of numbered offset through a perltie interface, is set up a
tied array interface instead.
Lets say you want to set up a tied perl variable that represents an
entire new 180G hard drive.
We can't tie it to a scalar because it is too large for the memory.
What we might be able to do with it is tie it to an array of
manageable-size pieces, perhaps 64K super-pages available like so:
tie @WholeDisk, 'Tie::WholeDisk', '/dev/hde', chunksize=>65536;
which would give us read/write access to /dev/hde in $WholeDisk[0]
through $WholeDisk[2949119]
which lets us hold two consecutive blocks in memory at once and run
regexen on the overlaps:
sub metaregex(\@$$){
my @results;
my ($aref, $regex, $overlapsize) = @_;
my $index;
while( $index <= $#$aref ){
@results =
$aref[$index++].substr($aref[$index],0,$overlapsize)
=~ $regex
and return @results
};
};
C<substr> is for mincing around with data we've already got inside
a string-type native perl scalar.
If you want to have perl-level access into something that you can
look up by number, you want an array.
(imagining more pathological tie modules that let you tie scalars
to arrays and arrays to scalars in such a way that splice and
substr become interchangeable)
Basta!
David Nicol
Abigail
> On Thu, 2003-12-04 at 16:23, Michael G Schwern wrote:
> >
> > A) If we're going to start extending the tied scalar interface, why stop
> > with SUBSTR? Why not chomp and chop and length and s/// and all that?
> > What's special about SUBSTR that it needs its own tie method?
>
> SUBSTR is how we look at parts of a string without looking at the
> whole thing. All the functions that work with parts of strings can
> be described in terms of SUBSTR. Cho(m)p, s///, y///, etc.
Perhaps a simpler argument: SUBSTR would be scalar equivalent of
SPLICE, which is an event trappable with the tie mechanism.
Abigail
Tassilo Von Parseval
I am quite impressed how ingenious I was (in retrospective) in
mentioning SUBSTR in all my messages. However, I assure everyone that
it was pure coincidence. I used SUBSTR as my most prominent example
because I once was in the situation where I wanted to change the current
behaviour of substr() through tie. It could have been CHOMP or INDEX
just as easily. And so everything I wrote was under the premise that a
whole bunch of new methods should be defineable for tied
scalars...SUBSTR, but also INDEX, CHOP and all those other scalar
manipulating functions.
Probably a good thing that many porters misunderstood that, otherwise my
request for comments might have been shred to pieces right from the
start. ;-)
Nick Ing-Simmons
>On Thu, Dec 11, 2003 at 09:35:08PM +0100 Abigail wrote:
>> On Thu, Dec 11, 2003 at 05:28:53AM -0600, david nicol wrote:
>> > On Thu, 2003-12-04 at 16:23, Michael G Schwern wrote:
>> > >
>> > > A) If we're going to start extending the tied scalar interface, why stop
>> > > with SUBSTR? Why not chomp and chop and length and s/// and all that?
>> > > What's special about SUBSTR that it needs its own tie method?
>> >
>> > SUBSTR is how we look at parts of a string without looking at the
>> > whole thing. All the functions that work with parts of strings can
>> > be described in terms of SUBSTR. Cho(m)p, s///, y///, etc.
>>
>>
>> Perhaps a simpler argument: SUBSTR would be scalar equivalent of
>> SPLICE, which is an event trappable with the tie mechanism.
An argument which makes some sense to me.
>
>I am quite impressed how ingenious I was (in retrospective) in
>mentioning SUBSTR in all my messages. However, I assure everyone that
>it was pure coincidence. I used SUBSTR as my most prominent example
>because I once was in the situation where I wanted to change the current
>behaviour of substr() through tie. It could have been CHOMP or INDEX
>just as easily. And so everything I wrote was under the premise that a
>whole bunch of new methods should be defineable for tied
>scalars...SUBSTR, but also INDEX, CHOP and all those other scalar
>manipulating functions.
Adding an method for every op that operates on a scalar in anyway
at all however is getting out of hand for perl5 - perl6 could perhaps
build such concepts in from the start.
It still think those in favour of this need to explain how
it inter-operates with 'overload' - or why it doesn't.
Tassilo Von Parseval
> >I am quite impressed how ingenious I was (in retrospective) in
> >mentioning SUBSTR in all my messages. However, I assure everyone that
> >it was pure coincidence. I used SUBSTR as my most prominent example
> >because I once was in the situation where I wanted to change the current
> >behaviour of substr() through tie. It could have been CHOMP or INDEX
> >just as easily. And so everything I wrote was under the premise that a
> >whole bunch of new methods should be defineable for tied
> >scalars...SUBSTR, but also INDEX, CHOP and all those other scalar
> >manipulating functions.
>
> Adding an method for every op that operates on a scalar in anyway
> at all however is getting out of hand for perl5 - perl6 could perhaps
> build such concepts in from the start.
This is a valid concern. The necessary changes to the core would remain
local to the affected OPs, but there would be quite a number of those
changes. Maintaing backwards-compatibility is another unsolved problem.
It could be done when sacrificing inheritance: looking up these methods
only happens in the tying package and not in its superclasses. When they
aren't found, the current behaviour is restored which means calling
CORE::OPERATION after doing a FETCH.
> It still think those in favour of this need to explain how
> it inter-operates with 'overload' - or why it doesn't.
Both overloading and tying involve blessed references, but in the case
of tie(), the object doesn't leak into the interface. On the surface,
you still have an ordinary scalar. Since tieing and overloading somewhat
overlap conceptually, it's not easy to say where such an addition would
belong. In any way, I just had a look at how overloading is implemented
and it looks as though the amount of changes to the core would be very
similar: You have to change each affected OP.
Slightly rhetorical question: would going the overload-path make such an
addition to perl more acceptable and likely? :-)
Yves Orton
> make such an addition to perl more acceptable and likely? :-)
I got the impression from the overload docs that Tie and overload are
orthagonal due to "camel hair".
By this I mean that
$x=$y;
in overload is a copy operation. With a tie its a STORE operation.
So for substr to work it has to be at the tie level and not the overload
level I would have thought.
I may totally misunderstand this tho. I look forward to being corrected. :-)
Yves
Nick Ing-Simmons
>> Slightly rhetorical question: would going the overload-path
>> make such an addition to perl more acceptable and likely? :-)
>
>I got the impression from the overload docs that Tie and overload are
>orthagonal due to "camel hair".
And this is the "camel grooming club" - we have to make the hair
as unhairy as possible while making things work, or explain why they don't.
Both these things are MAGICal and hairs are likely to get tangled
(or get the mange) unless we take care.
>
>By this I mean that
>
> $x=$y;
>
>in overload is a copy operation. With a tie its a STORE operation.
and/or a FETCH operation.
If $y is tied and FETCH returns an overloaded thing then
overload didn't work last time I tried.
>
>So for substr to work it has to be at the tie level and not the overload
>level I would have thought.
Yes.
But what about
$x .= "Something"; # Is that an overloaded '.' or another tie method.
How migh this differ:
$x = $x . "Something";
And will
$substr($x,...) . 'Something' # work when new SUBSTR method returns
and overloaded value?
All the tie discussion to date is about strings - what about tied
scalars that are numeric?
John Peacock
> I got the impression from the overload docs that Tie and overload are
> orthagonal due to "camel hair".
At the last OSCon, I got into a discussion with Dan Sugalski, who suggested that
this might be curable, but I never got back to look at the code. He suggested
that overload magic doesn't know/respect tie magic, but that it might be
possible to fix that (in gv.c:Perl_amagic_call() if I recall correctly).
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748
David Nicol
On Fri, 2003-12-12 at 03:38, Nick Ing-Simmons wrote:
> Tassilo Parseval <tassilo....@post.rwth-aachen.de> writes:
> >On Thu, Dec 11, 2003 at 09:35:08PM +0100 Abigail wrote:
> >> On Thu, Dec 11, 2003 at 05:28:53AM -0600, david nicol wrote:
> >> > On Thu, 2003-12-04 at 16:23, Michael G Schwern wrote:
> >> > >
> >> > > A) If we're going to start extending the tied scalar interface, why stop
> >> > > with SUBSTR? Why not chomp and chop and length and s/// and all that?
> >> > > What's special about SUBSTR that it needs its own tie method?
> >> >
> >> > SUBSTR is how we look at parts of a string without looking at the
> >> > whole thing. All the functions that work with parts of strings can
> >> > be described in terms of SUBSTR. Cho(m)p, s///, y///, etc.
> >>
> >>
> >> Perhaps a simpler argument: SUBSTR would be scalar equivalent of
> >> SPLICE, which is an event trappable with the tie mechanism.
>
> An argument which makes some sense to me.
Best trappable by going to fully replaceable vtables instead of
late binding.
If all the stringmanip ops were written in terms of a substr primitive,
which is equivalent to, say, if perl scalars used STL strings and used
the STL string interface...
> >
> >I am quite impressed how ingenious I was (in retrospective) in
> >mentioning SUBSTR in all my messages. However, I assure everyone that
> >it was pure coincidence. I used SUBSTR as my most prominent example
> >because I once was in the situation where I wanted to change the current
> >behaviour of substr() through tie. It could have been CHOMP or INDEX
And I thought that it had to do with the comments in the source of
Net::FTP::blat but on inspection I find that I was merely dealing with
issues of tieing SUBSTR on my side of the keyboard: I did not include
the whole debate in the comments.
> >just as easily. And so everything I wrote was under the premise that a
> >whole bunch of new methods should be defineable for tied
> >scalars...SUBSTR, but also INDEX, CHOP and all those other scalar
> >manipulating functions.
>
> Adding an method for every op that operates on a scalar in anyway
> at all however is getting out of hand for perl5 - perl6 could perhaps
> build such concepts in from the start.
> It still think those in favour of this need to explain how
> it inter-operates with 'overload' - or why it doesn't.
A stopgap -- at a higher level of abstraction -- might be to modify the
regex engine to somehow allow it to find matches that overlap multiple
scalars without simply invoking substr and concat to put the overlap
zone into a memory block that can be the object of a regex.
But this too would be more trouble than gain.
The TIE interface as we currently have it is sufficient to abstract
into perl data as we understand it. It breaks down when we want
access "within" or "behind" the scalar value. I do not know if XS
and Explicit Magic are powerful enough to do everything that can
be imagined.
Example: a way to bind a scalar to a file using memory mapping, so that
all internal perl access to the scalar is related to the file contents
via the OS's mmap system. This can't be done with TIE because we have
no way of working with anything smaller than the whole string at a time.
It may be trivial to trap replacement and realloc operations with magic
and handle them in relation to mmap calls. I don't know.
Maybe a perl-level interface to the magic system would be a good thing,
maybe it too would be more trouble than worth.
--
david nicol
Where the hell did I put my coffee?
David Nicol
> $x=$y;
>
> in overload is a copy operation. With a tie its a STORE operation.
>
> So for substr to work it has to be at the tie level and not the
> overload level I would have thought.
both STORE and FETCH can be mimicked with substr:
substr($x,0,length($x),$y); # $x = $y
a way to replace the entire internal representation of a scalar
(or at least the string side) is needed before language-level
access to it makes sense.