Perl equivalent of a Python tuple?
sk...@calendar.com
which serializes most basic Python types. The intent is that it be easily
portable to Perl, since I have needs for some basic (and fast) Perl<->Python
data exchange.
Most things map well. I'm ignoring long ints, complex numbers, higher-level
constructs like classes, and very Python-specific stuff like code objects,
partly because I don't need them and partly because they don't map well.
(I'll let you know when I get byte code interchange working <wink>.) I
expect to handle None by adopting some convention on the Perl side (create a
unique object called $None, for example). My biggest unknown at this point
is tuples. I'm not aware of an immutable array-like basic type in Perl, so
conversion from Python tuples will probably be to Perl arrays. Conversion
back will be to lists, which will, of course, cause occasional problems.
Any ideas how to tackle this?
Thx,
--
Skip Montanaro
Mojam: http://www.mojam.com/
Musi-Cal: http://concerts.calendar.com/
-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/ Search, Read, Discuss, or Start Your Own
Lars Marius Garshol
* sk...@calendar.com
|
| Any ideas how to tackle this?
I think you should take a look at the current discussions on the
XML-SIG mailing lists. Currently several different XML solutions for
Python<->Perl (and many other languages) data interchange are being
discussed.
--Lars M.
sk...@calendar.com
> I think you should take a look at the current discussions on the
> XML-SIG mailing lists. Currently several different XML solutions for
> Python<->Perl (and many other languages) data interchange are being
> discussed.
Thanks for the suggestion. I will take a look at the XML sig. I have seen
examples of a marshal DTD. I'm skeptical XML will provide a solution,
however. Rather, I think it will simply provide another intermediate form.
Here's my dilemma:
1. Encode a Python tuple using whatever scheme you choose (XML, marshal,
pickle, etc). 2. Transmit that encoding to a Perl program using any means
available. 3. Decode it into some basic Perl data type (e.g., array). 4.
Encode that data again. 5. Transmit it back to the Python program. 6. Decode
it.
What do you get? If the mapping is
Python data type Perl data type
---------------- --------------
int int
float float
list array
dict hash
tuple array
going
Python tuple -> encoded form -> Perl array -> encoded form -> Python list
isn't going to be very satisfactory. Either on the Python side I have to
give up on using tuples or anticipate and handle cases where I'm expecting
tuples but get lists. Note that I've said nothing about the encoding used.
XML is just one of a number of different choices.
I could probably dream up a Perl class that pretends it's a tuple-ish sort of
thing (wrapping an array but preventing modifications after initialization),
but I'm not sure that for interchanging basic data I should map basic objects
from one language onto higher-level objects in another language, partly
because it may be difficult to make those higher-level objects behave as
expected in all situations, and partly because the class may not provide
clean indexing syntax.
Tim Peters
> ...
> Python tuple -> encoded form -> Perl array -> encoded form ->
> Python list
>
> isn't going to be very satisfactory.
Are you sure? Python often accepts a list where a tuple might be more
proper. In any case, you named what appear to be even less satisfactory
alternatives <wink>:
> Either on the Python side I have to give up on using tuples or
> anticipate and handle cases where I'm expecting tuples but get lists.
> ...
> I could probably dream up a Perl class that pretends it's a tuple-ish
> sort of thing (wrapping an array but preventing modifications after
> initialization),
I'll attach one below to save you the pain. Note the funky use of "tie",
and try to mimic it rather than understand it <0.5 wink>.
> but I'm not sure that for interchanging basic data I should map
> basic objects from one language onto higher-level objects in another
> language, partly because it may be difficult to make those higher-
> level objects behave as expected in all situations,
That's for sure.
> and partly because the class may not provide clean indexing syntax.
The attached gives scalar indexing of Tuples as clean as Perl's array
indexing. You're on your own for slices, though.
i-feel-like-we-should-both-be-arrested-ly y'rs - tim
package Tuple;
sub TIEARRAY {
my ($class, @data) = @_;
return bless \@data, $class;
}
sub FETCH {
my ($self, $i) = @_;
die "tuple index $i out of bounds\n" if $i >= @$self or
$i < -@$self;
return $$self[$i];
}
sub STORE {
die "Tuples are immutable, bub!\n";
}
# add other methods as desired
sub len {
my $self = shift;
return scalar @$self;
}
package main;
# note that $t and @t are different vrbls in Perl; $t gets at the
# blessed array ref returned by Tuple::TIEARRAY, so is used to access
# non-magic methods. @t is the actual tuple mimic, and is used like
# any ordinary Perl array vrbl except that (1) it magically redirects
# indexing notation to $t->FETCH and $t->STORE; and (2) outside of
# those it's surprising (e.g. @t in scalar context returns 0 and in
# array context returns an empty list; Programming Perl 2nd ed said
# someday they'd try to do a more complete emulation, but I lost
# interest & don't know whether Perl ever did).
$t = tie @t, Tuple, 6, 7, 42;
$len = $t->len();
print "len $len\n"; # prints len 3
# block prints
# t[-3] == 6
# t[-2] == 7
# t[-1] == 42
# t[0] == 6
# t[1] == 7
# t[2] == 42
for ($i = -$len; $i < $len; ++$i) {
print "t[$i] == $t[$i]\n";
}
# tuple index 3 out of bounds
eval { $n = $t[$len]; print "$n\n"; }; warn $@ if $@;
# Tuples are immutable, bub!
eval { $t[0] = 5; }; warn $@ if $@;
Greg Ward
> i-feel-like-we-should-both-be-arrested-ly y'rs - tim
I feel that way all the time. You should have seen the looks I got
around here (CNRI) when I wore my Perl Conference t-shirt.
> # note that $t and @t are different vrbls in Perl; $t gets at the
> # blessed array ref returned by Tuple::TIEARRAY, so is used to access
> # non-magic methods. @t is the actual tuple mimic, and is used like
> # any ordinary Perl array vrbl except that (1) it magically redirects
> # indexing notation to $t->FETCH and $t->STORE; and (2) outside of
> # those it's surprising (e.g. @t in scalar context returns 0 and in
> # array context returns an empty list; Programming Perl 2nd ed said
> # someday they'd try to do a more complete emulation, but I lost
> # interest & don't know whether Perl ever did).
FWIW: the implementation of tied arrays in Perl 5.005 apparently is
"fixed", so I assume this is the hole they filled (or at least one of
them).
Hmmm, if you implement Python tuples as a Perl class, you should
probably also do the same for lists and dictionaries: after all, the
semantics are quite different in that the Python beasts raise an
exception when you go out of bounds, but Perl happily accepts this --
and in certain circumstances will create the missing element. (This is
what "auto-vivification" means, in case anyone has ever been mystified
by that bit of Perl-jargon.)
Just another /^P(erl|ython)$/ hacker...
Greg
--
Greg Ward - software developer gw...@cnri.reston.va.us
Corporation for National Research Initiatives
1895 Preston White Drive voice: +1-703-620-8990 x287
Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913
Andrew M. Kuchling
>but I'm not sure that for interchanging basic data I should map basic objects
>from one language onto higher-level objects in another language, partly
>expected in all situations, and partly because the class may not provide
>clean indexing syntax.
The attitude that I'm taking for the XML marshalling code is
that it won't try to provide complete transparency when you go from
language X -> language Y -> language X. For example, in XML-RPC an
array element always becomes a Python list, but both tuples and lists
are marshalled into the array XML element, because the distinction
between them isn't present in XML-RPC. The caller who unmarshals an
objects will have to be aware
And IMHO it's not required to simulate language X's semantics
in language Y. The intention of these DTDs is to allow interchanging
objects between languages, so I can instantiate something in Python,
send it to Greg who modifies it using some Perl code, and then read it
back into Python again. This means that objects unmarshalled in
Python should behave in a Pythonish way, and in Perl they'll be
Perlish equivalents; auto-vivification works in Perl and doesn't in
Python, nothing at all works in JavaScript, and so forth. This isn't
completely attainable -- Python has no built-in (or even standard)
date/time type and most of these DTDs do, so dates will have to be
unmarshalled as some object -- but it's not difficult to handle the
simple cases of strings, numbers, and array-like things.
--
A.M. Kuchling http://starship.skyport.net/crew/amk/
The point is that newbies almost always read more into the semantics of
release than are specified, so it's worthwile to be explicit about how little
is being said <wink>.
-- Tim Peters, 12 Feb 1998
sk...@calendar.com
> Hmmm, if you implement Python tuples as a Perl class, you should
> probably also do the same for lists and dictionaries: after all, the
> semantics are quite different in that the Python beasts raise an
> exception when you go out of bounds, but Perl happily accepts this --
> and in certain circumstances will create the missing element. (This is
> what "auto-vivification" means, in case anyone has ever been mystified
> by that bit of Perl-jargon.)
I'm not so sure that's necessary. If I pass [1, 2, 3] across the moat to a
Perl program that then reads its fifth element and passes it back, I'll see
[1, 2, 3, 0, 0], right? It's still a valid list. It just has a different
value. I'm not really worried about differences between Perl's and Python's
operational semantics, just that types are preserved as objects fly back and
forth across the moat.
Did I misunderstand what you said?
holds.
sk...@calendar.com
> Hmmm, if you implement Python tuples as a Perl class, you should
> probably also do the same for lists and dictionaries: after all, the
> semantics are quite different in that the Python beasts raise an
> exception when you go out of bounds, but Perl happily accepts this --
> and in certain circumstances will create the missing element. (This is
> what "auto-vivification" means, in case anyone has ever been mystified
> by that bit of Perl-jargon.)
I'm not so sure that's necessary. If I pass [1, 2, 3] across the moat to a
Perl program that then reads its fifth element and passes it back, I'll see
[1, 2, 3, 0, 0], right? It's still a valid list. It just has a different
value. I'm not really worried about differences between Perl's and Python's
operational semantics, just that types are preserved as objects fly back and
forth across the moat.
... or did I misunderstand what you said?
Phil Harris
There is already a Python module for it.
Web Distributed Date eXchange.
Works for me.
Phil
<sk...@calendar.com> wrote in message news:75k3no$tpq$1...@nnrp1.dejanews.com...
>I've written a small extension module that is a simplification of marshal
>which serializes most basic Python types. The intent is that it be easily
>portable to Perl, since I have needs for some basic (and fast)
Perl<->Python
>data exchange.
>
>Most things map well. I'm ignoring long ints, complex numbers,
higher-level
>constructs like classes, and very Python-specific stuff like code objects,
>partly because I don't need them and partly because they don't map well.
>(I'll let you know when I get byte code interchange working <wink>.) I
>expect to handle None by adopting some convention on the Perl side (create
a
>unique object called $None, for example). My biggest unknown at this point
>is tuples. I'm not aware of an immutable array-like basic type in Perl, so
>conversion from Python tuples will probably be to Perl arrays. Conversion
>back will be to lists, which will, of course, cause occasional problems.
>
>Any ideas how to tackle this?
>
>Thx,
Tim Peters
> Hmmm, if you implement Python tuples as a Perl class, you should
> probably also do the same for lists and dictionaries: after all, the
> semantics are quite different in that the Python beasts raise an
> exception when you go out of bounds, but Perl happily accepts this --
> and in certain circumstances will create the missing element. (This is
> what "auto-vivification" means, in case anyone has ever been mystified
> by that bit of Perl-jargon.)
[The Skipster]
> I'm not so sure that's necessary. If I pass [1, 2, 3] across the
> moat to a Perl program that then reads its fifth element and passes it
> back, I'll see [1, 2, 3, 0, 0], right?
Sometimes I think you weren't born 100 years ago, Skip! Like anything could
be that easy <choke>. Merely accessing an out-of-bounds array element in
Perl doesn't auto-vivify; it simply returns the well-defined undefined value
<0.9 wink>; the size of the array is unchanged. The Perl Tuple class I sent
avoids that by raising an error for an out-of-bounds reference.
*Storing* to an out-of-bounds Perl array index will extend the size of the
array by magic, but "the gap" (if any) isn't filled with zeroes but with
that same well-defined undefined value; e.g.,
@a = (1, 2, 3);
$a4 = $a[4];
print "a4 undefined\n" unless defined $a4; # triggers
print "len ", scalar @a, "\n"; # prints "len 3"
$a[4] = 666;
print "a: @a\n"; # prints "a: 1 2 3 666" and under -w prints ...
# ... warning about use of uninitialized value
print "len ", scalar @a, "\n"; # prints "len 5"
print "a3 undefined\n" unless defined $a[3]; # triggers
> It's still a valid list.
It's a valid *Perl* list, yes <wink>. I'd probably map Perl's undefined
value into Python's None.
> It just has a different value. I'm not really worried about
> differences between Perl's and Python's operational semantics, just
> that types are preserved as objects fly back and forth across the moat.
That does clarify your intent. I'm still not clear on what you're doing, so
won't suggest the obvious <wink>: incorporate type tags into your API so
you can resolve the type issues directly & transparently in the interface
glue; no gonzo Perl programmer worth their chutzpah is going to use some
Python-inspired panty-waist Tuple class anyway.
at-least-not-before-larry-reinvents-it-ly y'rs - tim
sk...@calendar.com
Tim> I'm still not clear on what you're doing ...
I'm solving the same problem as marshal or Pickle, just with a restricted
set of types and between different languages. I could use either marshal's
or Pickle's formats I suppose (I was using marshal when it was just
Python-to-Python communication), but since I'm not handling everything they
handle (no long, complex, code or instance types) I didn't want to deal with
the potential confusion that would ensue if someday someone (e.g.) used
marshal to encode some data and shipped it to a Perl module which had no
idea what to do with a complex number. Which is not to say Perl doesn't or
can't understand those types, just that I have no need for them and don't
know Perl well enough to add the smarts needed. Someone else can handle
them...
Oh yes, merry/happy everything everyone...
Skip Montanaro | Mojam: "Uniting the World of Music" http://www.mojam.com/
sk...@calendar.com | Musi-Cal: http://concerts.calendar.com/
518-372-5583