Thoughts From TPC: New Modules In 5.9
Michael G Schwern
remember.
Jos and I were talking about what new modules we'd like to see in 5.9 and
came up with a simple criterion for deciding what should and should not come
in the perl core.
Will this module help users install more modules?
The problem with placing modules in the core is everyone has their
Technology Du Jour. XML, CGI, HTML, SOAP, LWP, encryption, etc... and
there's really no end to the possibilities. And there's new ones all the
time, so we'll constantly be growing. Worse yet, these technologies can
become obsolescent over time and seem silly to have in the core, yet we're
stuck with them for backwards compatibility. Text::Soundex is a prime
example. Even CGI.pm is starting to look a little silly.
Instead we focus on making the module search, download, unpack, build and
installation process easier. There are a fairly fixed set of technologies
necessary to do this, and if we do it right it will no longer be necessary
to assimilate new CPAN modules with every release.
What modules, then, do we need?
Search:
CPANPLUS *need it*
CPAN.pm is showing it's age. CPANPLUS is getting better and better
with every release. Better yet, it includes a nice programatic
API to allow others to interact with CPAN. There's even a plan
for CPANPLUS to completely emulate CPAN.pm's shell and programatic
interface thus eliminating the need for the old CPAN.pm code at all.
Download:
Net::FTP *got it*
To avoid alot of the search around for various system specific
tools and the cruft around them (ftp, ncftp, lynx, etc...) it's
nice to have our own FTP client library to download modules.
LWP *would be nice, too big, need a slimmed down version*
Many CPAN mirrors use http instead of ftp. http usually causes
less problems than ftp: firewalls usually have port 80 open; don't
have to worry about passive vs active ftp. But LWP is just too
large. It would be nice if a simple, stripped down HTTP client
library could be created and assimilated.
Digest::MD5 *got it*
Necessarly for module checksums.
Unpack:
Archive::Tar *need them*
Archive::Zip
Compress::Zlib
Adding these to the core means we not longer have to hunt around
for zip, gzip and tar binaries and guess what flavor they are.
They will need some work.
Build:
Module::Build *need it*
The dependency of perl's module installation on make is it's great
flaw. It adds heaps and heaps of complexity dealing with all the
different flavors of make, plus not everyone has make, plus it
makes customization of the build process very difficult since
you have to directly alter fragments of the Makefile which can
change from version to version and OS to OS.
Module::Build does not depend on make and can replace most of
MakeMaker's role. It still needs some work, but it's usable.
YAML *needed by Module::Build*
MakeMaker relies on running a perl program (Makefile.PL) in order
to perform the installation. It has been pointed out in the past
that this running of untrusted and potentially malicious code
is just waiting for someone to come in and do Evil Things with it.
Module::Build has an option to spit out a YAML description file
for your module which contains all the information necessary to
build and install the module. Additionally, it contains meta-
information normally only available by running the Makefile.PL
and doing nasty hacks parsing the Makefile and module files.
Adding YAML to the core and the subsequent description files to
replace Makefile.PL makes installing modules more secure and
statistical scans of CPAN easier.
YAML has many other benefits, but Ingy would be better to talk
about them.
Inline *need it*
XS is a weak point in perl. Inline automates the vast majority
of the magic necessary to hook together C and Perl. Having it in
core encourages more CPAN authors to optimize their modules using
C and it also allows us to simplify much of the modules in ext/.
Finally, using Inline we can test the internal Perl C API functions
using the existing Perl testing libraries.
Installation:
File::Copy *got it*
Installation basically boils down to copying some files.
OS::Spec *need them, vaporware*
File::System::Spec
There's a lot of information we have to guess about the system
during the build and installation process: Is it a case-insensitive
filesystem? Does the OS support fork? Is there a C compiler?
This lore is scattered amongst Configure, MakeMaker, Module::Build
and the core tests. I've been tossing around the idea for a while
of pulling all this info together into a set of modules to allow
you to query your environment. This will allow much of the cruft
in MakeMaker, Module::Build, and the core tests to be removed
as well as make sure the lessons learned in one subsystem of Perl
are applied to all of them.
These modules are all fairly small and fairly simple. Most importantly,
they should be the *last* modules we need to add to Perl.
--
Michael G. Schwern <sch...@pobox.com> http://www.pobox.com/~schwern/
Perl Quality Assurance <per...@perl.org> Kwalitee Is Job One
Pancakes is the better part of valor.
http://www.goats.com/archive/971202.html
Michael G Schwern
> Jos and I were talking about what new modules we'd like to see in 5.9 and
> came up with a simple criterion for deciding what should and should not come
> in the perl core.
>
> Will this module help users install more modules?
Oh yeah, there was one other criterion.
Will this module help in implementing the core?
--
Michael G. Schwern <sch...@pobox.com> http://www.pobox.com/~schwern/
Perl Quality Assurance <per...@perl.org> Kwalitee Is Job One
Cottleston, Cottleston, Cottleston Pie.
Why does a chicken, I don't know why.
Ask me a riddle and I reply:
"Cottleston, Cottleston, Cottleston Pie."
Slaven Rezic
[...]
>
> These modules are all fairly small and fairly simple. Most importantly,
> they should be the *last* modules we need to add to Perl.
How about: the *only* modules which should come with perl? There could
be a minimal source distribution which would contain only the modules
needed for fetching and building other modules.
Regards,
Slaven
--
Slaven Rezic - slaven...@berlin.de
tksm - Perl/Tk program for searching and replacing in multiple files
http://ptktools.sourceforge.net/#tksm
h...@crypt.org
: CPANPLUS *need it*
Agreed. I'm not happy about the name, but I'm not sure precisely why
and I don't have a better one.
: Archive::Tar *need them*
: Archive::Zip
: Compress::Zlib
I haven't thought about these. I'll take a look.
: Module::Build *need it*
Agreed.
: YAML *needed by Module::Build*
I didn't know that. I'll take a look.
: Inline *need it*
Agreed. I'd like confirmation from Ingy that he's happy that Inline
and Inline::C are both stable and complete enough for the core.
Hugo
Andreas J. Koenig
> Michael G Schwern <sch...@pobox.com> wrote:
> : CPANPLUS *need it*
> Agreed. I'm not happy about the name, but I'm not sure precisely why
> and I don't have a better one.
CPANPLUS can be renamed to CPAN if that's what people want. If the
community is happy with the replacement, I'm not intending to continue
maintaining CPAN.pm just for myself.
--
andreas
Matt Sergeant
> : YAML *needed by Module::Build*
>
> I didn't know that. I'll take a look.
>
One thing that's been worrying me recently about YAML is that the spec
is over twice as long as the XML spec. This doesn't necessarily mean its
more complex than XML, as Ingy reassured me its mostly examples, but it
does cause conflict in me - do we include a complex new (and mostly
unproven) data serialisation format in Perl just for the sake of one
module?
Matt.
Nicholas Clark
> Michael G Schwern <sch...@pobox.com> wrote:
> : CPANPLUS *need it*
>
> Agreed. I'm not happy about the name, but I'm not sure precisely why
> and I don't have a better one.
>
> : Archive::Tar *need them*
> : Archive::Zip
> : Compress::Zlib
>
> I haven't thought about these. I'll take a look.
Jarkko's concern was that we'd need to distribute zlib with the core.
Size not a problem, the licensing is not a problem, but we aren't sure it's
portable to all platforms that perl is, and Jarkko didn't want to impose the
burden of asking the obscure porters to port it.
(from memory so the archive URL isn't much use as it's not fully qualified and
more importantly unroutable (brain://nick/... , and I believe it was Jarkko)
I suspect it's intrinsicly at least as portable as perl, and suspect it may well
have been actively built on more platforms than perl. Question is, do the
obscure platforms overlap
Nicholas Clark
--
Even better than the real thing: http://nms-cgi.sourceforge.net/
Slaven Rezic
But what about backward compatibility? There may be some code using
the API of CPAN.pm and I don't know whether CPANPLUS will emulate all
of it.
Regards,
Slaven
--
Slaven Rezic - slaven...@berlin.de
tknotes - A knotes clone, written in Perl/Tk.
http://ptktools.sourceforge.net/#tknotes
Johan Vromans
> Build:
> Module::Build *need it*
Good.
> YAML *needed by Module::Build*
> MakeMaker relies on running a perl program (Makefile.PL) in order
> to perform the installation. It has been pointed out in the past
> that this running of untrusted and potentially malicious code
> is just waiting for someone to come in and do Evil Things with it.
Module::Build runs a perl program Build.PL.
> Module::Build has an option to spit out a YAML description file
> for your module which contains all the information necessary to
> build and install the module.
And how would this help? If a malicent user can tamper with
Makefile.PL she can also modify the YAML file. After all, as Ingy once
told me, it's yet another markup language. If we can protect the YAML
file from being changed, we can also protect the Makefile.PL and
Build.PL.
> Additionally, it contains meta-
> information normally only available by running the Makefile.PL
> and doing nasty hacks parsing the Makefile and module files.
From looking at the META.yaml examples from the Module::Build
distribution, my first impression is that 10 lines of perl can write
and parse these files as well. Why not use Data::Dumper?
Besides, it uses whitespace to denote structure so it must be python :-).
I think YAML is a great tool, but plain overkill for this purpose.
-- Johan
Colin Meyer
>
> > Module::Build has an option to spit out a YAML description file
> > for your module which contains all the information necessary to
> > build and install the module.
>
> And how would this help? If a malicent user can tamper with
> Makefile.PL she can also modify the YAML file. After all, as Ingy once
> told me, it's yet another markup language. If we can protect the YAML
> file from being changed, we can also protect the Makefile.PL and
> Build.PL.
If the malicious user put system(qw/rm -rf */) into a YAML doc, it
wouldn't have the same effect as in a Makefile.PL or Build.PL.
-Colin.
Nick Ing-Simmons
> Archive::Tar *need them*
> Archive::Zip
> Compress::Zlib
> Adding these to the core means we not longer have to hunt around
> for zip, gzip and tar binaries and guess what flavor they are.
> They will need some work.
>
And then:
>bleadperl snapshots were distributed as both bzip2 and gzip. 5.8.0 was not.
>perl compresses down better with bzip than gzip:
>
>$ ls -l /tmp/perl-current.tar
>-rw-r--r-- 1 schwern schwern 47872000 Aug 1 09:31 /tmp/perl-current.tar
>$ ls -l /tmp/perl-current.tar.gz
>-rw-r--r-- 1 schwern schwern 10902563 Aug 1 09:31 /tmp/perl-current.tar.gz
>$ ls -l /tmp/perl-current.tar.bz2
>-rw-r--r-- 1 schwern schwern 8624920 Aug 1 09:31 /tmp/perl-current.tar.bz
>
>For 5.9, provide both bzip and gzip versions and that will save us 20%+ on
>distribution size right there.
Is there a module which does bzip yet?
--
Nick Ing-Simmons
http://www.ni-s.u-net.com/
Robin Berjon
> >For 5.9, provide both bzip and gzip versions and that will save us 20%+ on
> >distribution size right there.
>
> Is there a module which does bzip yet?
Yup, Compress::Bzip2 by Brian Horakh.
--
Robin Berjon <ro...@knowscape.com>
Don't panic.
Brian Ingerson
I don't get excited about putting modules in the core. In fact, I would
be very pleased if the core module set was minimalized or eliminated
(pipe dream). But I digress...
I think YAML would be a killer module to be in the core, and the least
of the reasons is because Module::Build depends on it. Here are some
good reasons:
- Since YAML is nice for config files, caches, and logs, it will
probably become a prereq for a lot of other Perl modules.
- Data::Dumper is our only other robust human readable serializer, but
you can't safely restore it's output in an untrusted environment.
- YAML is uniquely a *cross language* serialization format.
Implementations have already been started in Python, Ruby, and Java.
- YAML is powerful debugging tool. It dumps more data structures (like
globs and file handles) than D::D.
Matt is right that this is a new, complex (read complete), and unproven
technology. It is still evolving. But given that we are not on a tight
timeline right now, I'd suggest putting it in and seeing how it goes. I'm
pretty sure it'll be a good thing for Perl.
Cheers, Brian
BTW, The version of YAML.pm that would go into the core should be an XS
wrapper of Neil Watkiss libyaml. libyaml will become the core engine for
Python and Ruby YAMLs as well. This work will probably become stable
towards the end of the year. The pure Perl version could be used as a
placeholder for now, to test Module::Build.
Nicholas Clark
Well, I've been thinking about writing PerlIO::bzip2, (based rather strongly
on the code in PerlIO::gzip, if not the same with #ifdefs) but I've not yet
got to working on the thread stuff in PerlIO::gzip, because I'm currently busy
with some parrot things (possibly nearly finished)
Johan Vromans
Neither would it when the Makefile.PL was fed to the C compiler, or to
the SNOBOL runtime. So this is a void argument.
-- Johan
Benjamin Goldberg
>
> On 03/08/02 10:22 +0100, Matt Sergeant wrote:
> > On Saturday, August 3, 2002, at 03:43 AM, h...@crypt.org wrote:
> >
> > > : YAML *needed by Module::Build*
> > >
> > > I didn't know that. I'll take a look.
> > >
> >
> - Since YAML is nice for config files, caches, and logs, it will
> probably become a prereq for a lot of other Perl modules.
> - Data::Dumper is our only other robust human readable serializer,
> but you can't safely restore it's output in an untrusted
> environment.
What about Data::Denter ?
[Yes, I realize that this doesn't address the other points made; it's
just a suggestion]
--
tr/`4/ /d, print "@{[map --$| ? ucfirst lc : lc, split]},\n" for
pack 'u', pack 'H*', 'ab5cf4021bafd28972030972b00a218eb9720000';
Benjamin Goldberg
>
> On Saturday 03 August 2002 19:05, Nick Ing-Simmons wrote:
> > >For 5.9, provide both bzip and gzip versions and that will save us
> > >20%+ on distribution size right there.
> >
> > Is there a module which does bzip yet?
>
> Yup, Compress::Bzip2 by Brian Horakh.
But that only compresses strings, not streams.
There's no way to create any kind of "context" object, as you can with
Compress::Zlib.
Andreas J. Koenig
>> CPANPLUS can be renamed to CPAN if that's what people want. If the
>> community is happy with the replacement, I'm not intending to continue
>> maintaining CPAN.pm just for myself.
>>
> But what about backward compatibility? There may be some code using
> the API of CPAN.pm and I don't know whether CPANPLUS will emulate all
> of it.
Without compatibility no rename. (IMHO)
--
andreas
Michael G Schwern
> What about Data::Denter ?
Data::Denter is the deprecated prototype for YAML.
--
Michael G. Schwern <sch...@pobox.com> http://www.pobox.com/~schwern/
Perl Quality Assurance <per...@perl.org> Kwalitee Is Job One
They just don't make any good porn music anymore, do they?
- WXDX DJ refering to "More, More, More"
Vadim Konovalov
while trying to migrate, I found another incompatibility ussue between 5.6
and 5.8.
(First one was when sockets are now by default do \r\n=>\n translation on
Win32).
Attached script p58-dif.pl and data to it udata.txt give different output
using 5.6 and 5.8 perls.
The point of a problem is that, when file is in text mode, "tell" and "seek"
now sometimes misbehave, because in 5.8 perl mis-interprets file position
because of "\r\n"=>"\n" translation.
Earlier 5.6 behaves correctly: seek returned exactly into same position
which was reported by "tell" function.
Setting default layer to binmode will do something bad in my other programs.
Also, I've attached a script p58-dif-d.pl which contains same problem with
__DATA__.
Could you please someone suggest a solution to my incompatibility problem?
Thanks,
Vadim.
Brian Ingerson
> On 03/08/02 10:22 +0100, Matt Sergeant wrote:
> I think YAML would be a killer module to be in the core, and the least
> of the reasons is because Module::Build depends on it. Here are some
> good reasons:
>
> - Since YAML is nice for config files, caches, and logs, it will
> probably become a prereq for a lot of other Perl modules.
> - Data::Dumper is our only other robust human readable serializer, but
> you can't safely restore it's output in an untrusted environment.
> - YAML is uniquely a *cross language* serialization format.
> Implementations have already been started in Python, Ruby, and Java.
> - YAML is powerful debugging tool. It dumps more data structures (like
> globs and file handles) than D::D.
One other nice thing about YAML.pm that I forgot to mention is that it
has an interface for serializing "opaque" objects. (ie objects that
aren't implemented using hashes or arrays) It let's a class define its
own to_yaml() and from_yaml() methods. This could be useful for many
things, including smaller-grained regression tests for core classes.
Cheers, Brian
Nick Ing-Simmons
>Dear all!
>
>while trying to migrate, I found another incompatibility ussue between 5.6
>and 5.8.
>(First one was when sockets are now by default do \r\n=>\n translation on
>Win32).
>
>Attached script p58-dif.pl and data to it udata.txt give different output
>using 5.6 and 5.8 perls.
>The point of a problem is that, when file is in text mode, "tell" and "seek"
>now sometimes misbehave, because in 5.8 perl mis-interprets file position
>because of "\r\n"=>"\n" translation.
That is not supposed to happen.
>Earlier 5.6 behaves correctly: seek returned exactly into same position
>which was reported by "tell" function.
That is supposed to happen with 5.8 as well.
>
>Setting default layer to binmode will do something bad in my other programs.
>
>Also, I've attached a script p58-dif-d.pl which contains same problem with
>__DATA__.
>
>Could you please someone suggest a solution to my incompatibility problem?
>
>Thanks,
>Vadim.
Vadim Konovalov
5.6
> >and 5.8.
> >(First one was when sockets are now by default do \r\n=>\n translation on
> >Win32).
> >
> >Attached script p58-dif.pl and data to it udata.txt give different output
> >using 5.6 and 5.8 perls.
> >The point of a problem is that, when file is in text mode, "tell" and
"seek"
> >now sometimes misbehave, because in 5.8 perl mis-interprets file position
> >because of "\r\n"=>"\n" translation.
>
> That is not supposed to happen.
In an ideal world this will not happen.
In a real world, this broke my existing program, which makes me stop
migration (I hope for a short time).
>
> >Earlier 5.6 behaves correctly: seek returned exactly into same position
> >which was reported by "tell" function.
>
> That is supposed to happen with 5.8 as well.
Thank you for agreeing with me at this point.
I think tomorrow I'll try narrowing down a problem into a single
5.8.0-script which will use \r\n=>\n translation layer, and with in-memory
filehandle, which will show that after some "tell" and "seek" there could be
a mess.
BTW I built perl with MSVC++.
Best wishes,
Vadim.
Graham Barr
> > for your module which contains all the information necessary to
> > build and install the module.
>
> And how would this help?
It would make more information avaliable on the search site. Right
now Makefile.PL holds a lot of information, but to get it you have
to run it. That is not a realistic option for the search site.
Saving this info into a file and including in the distribution will
be the first step to makein that information avaliable.
Graham.
If a malicent user can tamper with
> Makefile.PL she can also modify the YAML file. After all, as Ingy once
> told me, it's yet another markup language. If we can protect the YAML
> file from being changed, we can also protect the Makefile.PL and
> Build.PL.
>
> > Additionally, it contains meta-
> > information normally only available by running the Makefile.PL
> > and doing nasty hacks parsing the Makefile and module files.
>
> >From looking at the META.yaml examples from the Module::Build
> distribution, my first impression is that 10 lines of perl can write
> and parse these files as well. Why not use Data::Dumper?
> Besides, it uses whitespace to denote structure so it must be python :-).
>
> I think YAML is a great tool, but plain overkill for this purpose.
I agree. We should also look towards standards. Someone should look into
if the open software description (OSD) is relevant here
http://www.w3.org/TR/NOTE-OSD.html
Graham.
Abigail
> Michael G Schwern <sch...@pobox.com> writes:
>
> >
> > These modules are all fairly small and fairly simple. Most importantly,
> > they should be the *last* modules we need to add to Perl.
>
> How about: the *only* modules which should come with perl? There could
> be a minimal source distribution which would contain only the modules
> needed for fetching and building other modules.
But fetching isn't as easy as it sounds. I'm a consultant, so I work at
different places. More often than not I'm behind a firewall neither CPAN
or LWP is able to work its way through. And from what I've understood,
CPANPLUS will not be able to either.
Common problems: proxies with username/password.
Filtering of URLs that end with '.gz'.
It's all doable manually, but it takes some time. But I'd rather stay
with 5.004 than installing all of 5.8.0's modules by hand. Not to
mention that it can easily take a week before the responsible person
with appropriate access has time to type a 'make install'. It'll take
a year if all modules need to be done that way. (If I were the sysadmin
I wouldn't blindly type 'make install' for all my users either).
While I don't disagree that it's time to take a look at the current set
of modules coming with Perl, I do think Perl should come with a healthy
set of useful modules, and shouldn't rely on good and constant net
connectivity.
Abigail
Michael G Schwern
> > How about: the *only* modules which should come with perl? There could
> > be a minimal source distribution which would contain only the modules
> > needed for fetching and building other modules.
>
>
> But fetching isn't as easy as it sounds. I'm a consultant, so I work at
> different places. More often than not I'm behind a firewall neither CPAN
> or LWP is able to work its way through. And from what I've understood,
> CPANPLUS will not be able to either.
I believe Slaven is suggesting multiple distributions. One would be a
minimal Perl with just enough modules to allow you to get more modules.
This would save heaps of space (for example, Encode and unicore alone are
14+ megs).
In your case you'd just grab the normal, full distribution.
Anyhow, don't you have a CPAN mirror on CD yet, Mr. Consultant? ;)
> Common problems: proxies with username/password.
> Filtering of URLs that end with '.gz'.
Wow, how do they justify that last one?
--
Michael G. Schwern <sch...@pobox.com> http://www.pobox.com/~schwern/
Perl Quality Assurance <per...@perl.org> Kwalitee Is Job One
You'll do.
Johan Vromans
> > I think YAML is a great tool, but plain overkill for this purpose.
>
> I agree. We should also look towards standards. Someone should look into
> if the open software description (OSD) is relevant here
> http://www.w3.org/TR/NOTE-OSD.html
This would require to add substantial XML support to the core (which
may be good).
-- Johan
Rafael Garcia-Suarez
>
> In your case you'd just grab the normal, full distribution.
>
> Anyhow, don't you have a CPAN mirror on CD yet, Mr. Consultant? ;)
That doesn't give him to rights to make install. Or even to mount
a CD.
Does CPAN(PLUS)? handle fetching tarballs from a local filesystem ?
>>Common problems: proxies with username/password.
A trimmed-down LWP-like (as you were suggesting) can use the
env vars http_proxy and http_user.
>> Filtering of URLs that end with '.gz'.
>
>
> Wow, how do they justify that last one?
Do they filter URLs that end with ".%67%7a" ?
Michael G Schwern
> Does CPAN(PLUS)? handle fetching tarballs from a local filesystem ?
CPAN Terminal> install Acme::Time::Baby
Checking if source files are up to date
Retrieving /home/schwern/.cpanplus/mailrc.2.04.stored
Retrieving /home/schwern/.cpanplus/dslip.2.04.stored
Retrieving /home/schwern/.cpanplus/packages.2.04.stored
Installing: Acme::Time::Baby
Trying to get /home/archive/CPAN/authors/id/A/AB/ABIGAIL/Acme-Time-Baby-1.7.tar.gz from localhost
Trying to get /home/archive/CPAN/authors/id/A/AB/ABIGAIL/CHECKSUMS from localhost
Checksum for /home/schwern/.cpanplus/authors/id/A/AB/ABIGAIL/Acme-Time-Baby-1.7.tar.gz OK
Untarring /home/schwern/.cpanplus/authors/id/A/AB/ABIGAIL/Acme-Time-Baby-1.7.tar.gz
Extracting Acme-Time-Baby-1.7/
....etc...
--
Michael G. Schwern <sch...@pobox.com> http://www.pobox.com/~schwern/
Perl Quality Assurance <per...@perl.org> Kwalitee Is Job One
Worship the sig file
Michael G Schwern
YAML is overkill for module description files, but XML is not?
--
Michael G. Schwern <sch...@pobox.com> http://www.pobox.com/~schwern/
Perl Quality Assurance <per...@perl.org> Kwalitee Is Job One
It's Ecstacy time!
John P. Linderman
I agree with Abigail. And network access isn't the only problem.
In a corporate environment where I don't wear the sysadmin hat,
I can convince the admins to install the "standard" distribution
everywhere, so my scripts run everywhere, too. As soon as I
have to start fetching modules, I'm stuck installing them in a
private library (replicated on all the systems I visit), and
sharing software becomes much more difficult.
I think its a great idea to conceive of a super-minimal, Perl lite
distribution for the disk-poor, self administered community.
But that shouldn't come at the expense of making sharing difficult
for the disk-rich. It's more work for the porters to offer a
range of distributions, but you're always looking for more work,
aren't you :-) -- jpl
Arthur Bergman
On måndag, augusti 5, 2002, at 11:01 , Graham Barr wrote:
>
> I agree. We should also look towards standards. Someone should look into
> if the open software description (OSD) is relevant here
> http://www.w3.org/TR/NOTE-OSD.html
>
> Graham.
Also Dublin Core could be looked at.
Arthur
Michael G Schwern
> > > I agree. We should also look towards standards. Someone should look into
> > > if the open software description (OSD) is relevant here
> > > http://www.w3.org/TR/NOTE-OSD.html
> >
> > may be good).
>
> YAML is overkill for module description files, but XML is not?
Let me put it a better way. Which would you rather read and write:
This (example taken from the OSD proposal)
<SOFTPKG NAME="com.foobar.www.Solitaire" VERSION="1,0,0,0">
<TITLE>Solitaire</TITLE>
<ABSTRACT>Solitaire by FooBar Corporation</ABSTRACT>
<LICENSE HREF="http://www.foobar.com/solitaire/license.html" />
<!--FooBar Solitaire is implemented in native code for Win32, Java code for other platforms -->
<IMPLEMENTATION>
<OS VALUE="WinNT"><OSVERSION VALUE="4,0,0,0"/></OS>
<OS VALUE="Win95"/>
<PROCESSOR VALUE="x86" />
<LANGUAGE VALUE="en" />
<CODEBASE HREF="http://www.foobar.org/solitaire.cab" />
</IMPLEMENTATION>
<IMPLEMENTATION>
<IMPLTYPE VALUE="Java" />
<CODEBASE HREF="http://www.foobar.org/solitaire.jar" />
<!-- The Java implementation needs the DeckOfCards object -->
<DEPENDENCY>
<CODEBASE HREF="http://www.foobar.org/cards.osd" />
</DEPENDENCY>
</IMPLEMENTATION>
</SOFTPKG>
Or this (same example, in YAML)
Softpkg:
Name: com.foobar.www.Solitare
Version: 1,0,0,0
Title: Solitare
Abstract: Solitare by FooBar Corporation
License: http://www.foobar.com/solitaire/license.html
# FooBar Solitaire is implemented in native code for Win32, Java code
# for other platforms.
Implementation:
WinNT:
Version: 4,0,0,0
Win95
Processor: x86
Language: en
Codebase: http://www.foobar.org/solitaire.cab
Implementation:
Impltype: Java
Codebase: http://www.foobar.org/solitaire.jar
# The Java implementation needs the DeckOfCards object
Dependency:
Codebase: http://www.foobar.org/cards.osd
--
Michael G. Schwern <sch...@pobox.com> http://www.pobox.com/~schwern/
Perl Quality Assurance <per...@perl.org> Kwalitee Is Job One
Hold on while I slip into something a little more naked.
Jonathan Stowe
>
> I agree. We should also look towards standards. Someone should look into
> if the open software description (OSD) is relevant here
> http://www.w3.org/TR/NOTE-OSD.html
>
OSD is what the Activestate PPD files are isn't it ?
/J\
--
Jonathan Stowe |
<http://www.gellyfish.com> | This space for rent
|
Horsley Tom
> Archive::Tar *need them*
Having just attempted to build Archive::Tar on my Windows XP
system, I would just mention that it appears to be highly
Unix dependent, dragging in all sorts of weird and
wondrous fcntl() operations on files for no reason I could
decrypt in the 30 seconds I spent attempting to understand them.
Possibly it is only the tar file creation code that works this way,
the unpacking may work on win32 (but I didn't try it).
Graham Barr
> On Mon, Aug 05, 2002 at 11:35:18AM +0200, Johan Vromans wrote:
> > [Quoting Graham Barr, on August 5 2002, 10:01, in "Re: Thoughts From TP"]
> > > > I think YAML is a great tool, but plain overkill for this purpose.
> > >
> > > I agree. We should also look towards standards. Someone should look into
> > > if the open software description (OSD) is relevant here
> > > http://www.w3.org/TR/NOTE-OSD.html
> >
> > This would require to add substantial XML support to the core (which
> > may be good).
>
> YAML is overkill for module description files, but XML is not?
I dont disagree that they may both be overkill for for this purpose.
XML is overkill for most of the places it is used. But if the description
file is going to force inclusion of modules, we need to assess how much use
those modules will be for other purposes. IMO, XML modules are likely
to be used much more than YAML.
Graham.
Graham Barr
> On Mon, 5 Aug 2002, Graham Barr wrote:
> >
> > I agree. We should also look towards standards. Someone should look into
> > if the open software description (OSD) is relevant here
> > http://www.w3.org/TR/NOTE-OSD.html
> >
>
> OSD is what the Activestate PPD files are isn't it ?
Yes, PPM files are based on OSD.
Graham.
Graham Barr
> At 03:17 AM 8/5/02 -0700, Michael G Schwern wrote:
> > > YAML is overkill for module description files, but XML is not?
> >Let me put it a better way. Which would you rather read and write:
> >This (example taken from the OSD proposal)
> > <SOFTPKG NAME="com.foobar.www.Solitaire" VERSION="1,0,0,0">
> > Softpkg:
> > Name: com.foobar.www.Solitare
> > Version: 1,0,0,0
>
>
> And it looks like YAML is much more natural to write.
>
> So I would vote for YAML in this respect and have the OSD be generated from
> that if needed...
I would buy that. As I said in another mail, the file that is uploaded
should be generated. And I agree that YAML is probably easier to write
than OSD. So let YAML be one possible way to provide information
and generate the OSD which is uploaded.
Graham.
Graham Barr
> On Mon, 5 Aug 2002 Michael G Schwern wrote:
> > > > I agree. We should also look towards standards. Someone should look into
> > > > if the open software description (OSD) is relevant here
> > > > http://www.w3.org/TR/NOTE-OSD.html
> > >
> > > This would require to add substantial XML support to the core (which
> > > may be good).
> >
> > YAML is overkill for module description files, but XML is not?
>
> Let me put it a better way. Which would you rather read and write:
IMO, the file should not be human written. It should be generated from
data that is provided.
Graham.
Elizabeth Mattijsen
> > YAML is overkill for module description files, but XML is not?
>Let me put it a better way. Which would you rather read and write:
>This (example taken from the OSD proposal)
> <SOFTPKG NAME="com.foobar.www.Solitaire" VERSION="1,0,0,0">
> Softpkg:
> Name: com.foobar.www.Solitare
> Version: 1,0,0,0
Seems to me that a YAML -> OSD translator would be trivial to write.
And it looks like YAML is much more natural to write.
So I would vote for YAML in this respect and have the OSD be generated from
that if needed...
Liz
Nick Ing-Simmons
>Michael G Schwern wrote:
>>
>> In your case you'd just grab the normal, full distribution.
>>
>> Anyhow, don't you have a CPAN mirror on CD yet, Mr. Consultant? ;)
CPAN does not fit on a CD anymore. And I don't yet have a DVD writer.
My solution is a CPAN mirror on my laptop - but if I was a general
consultant I doubt if sysadim would let me drop laptop on their
network.
>
>That doesn't give him to rights to make install. Or even to mount
>a CD.
>
>Does CPAN(PLUS)? handle fetching tarballs from a local filesystem ?
CPAN does provided you are not on Win32 (may even work there if you try
hard enough). But you have to manualy go an edit the config file
as it insists on prompting for region and URL from its list as
1st entry.
Nick Ing-Simmons
>> > > I agree. We should also look towards standards. Someone should look into
>> > > if the open software description (OSD) is relevant here
>> > > http://www.w3.org/TR/NOTE-OSD.html
>> >
>> > This would require to add substantial XML support to the core (which
>> > may be good).
>>
>> YAML is overkill for module description files, but XML is not?
>
>Let me put it a better way. Which would you rather read and write:
I really don't mind - but then I have to read XML for day-job.
With a suitable style sheet you can probably make the XML
_look_ like the YAML in a browser. If one was really messing with
them on a regular basis one would write a perl/Tk GUI for either.
Is YAML realy pythonesque in that indentation has semantics?
Or does one just know which tags can be inside which tags?
Nick Ing-Simmons
http://www.ni-s.u-net.com/
Nick Ing-Simmons
>> [Quoting Graham Barr, on August 5 2002, 10:01, in "Re: Thoughts From TP"]
>> > > I think YAML is a great tool, but plain overkill for this purpose.
>> >
>> > I agree. We should also look towards standards. Someone should look into
>> > if the open software description (OSD) is relevant here
>> > http://www.w3.org/TR/NOTE-OSD.html
>>
>> This would require to add substantial XML support to the core (which
>> may be good).
>
>YAML is overkill for module description files, but XML is not?
XML support is useful for other things (YAML may be as well I have
not looked into it yet).
pdca...@bofh.org.uk
By whom? In what format?
--
Piers
"It is a truth universally acknowledged that a language in
possession of a rich syntax must be in need of a rewrite."
-- Jane Austen?
Johan Vromans
> Let me put it a better way. Which would you rather read and write:
Aren't we mixing two things?
To get back to the original discussion, Module::Build uses a
programmer-friendly Build.PL file and has an option to generate a
concise description file that can be used for subsequent
build/installs (and other things). The actual format and layout of the
description file is not important, since it's both written and read by
computer programs. For the purpose of Module::Build it could be YAML,
XML, Data::Dumper, Lisp, whatever.
If, however, it is mandatory that users can maintain the data in a
friendly and intuitive matter, most of these formats are a bad
choice. I'd prefer lisp, your choice will be different. In any case, I
think it is a bad choice to have a data language that uses whitespace
to denote structure (Yes, I've had this discussion with Ingy
already). Just imagine two users that have a different setting for tab
width in their editors.
-- Johan
Nick Ing-Simmons
>> > > YAML is overkill for module description files, but XML is not?
>> >
>> data that is provided.
>
>By whom? In what format?
It depends what the data _is_ and if the dog is wagging the tail or vice
versa. e.g. a lot of info is implied in existing Makefiles, or you could
specify it as a ML and generate Makefiles. A lot of data is collected
by Configure, ...
Norton Allen
>
> On Sat, Aug 03, 2002 at 03:43:17AM +0100, h...@crypt.org wrote:
> > Michael G Schwern <sch...@pobox.com> wrote:
> > : CPANPLUS *need it*
> >
> > Agreed. I'm not happy about the name, but I'm not sure precisely why
> > and I don't have a better one.
> >
> > : Archive::Tar *need them*
> > : Archive::Zip
> > : Compress::Zlib
> >
> > I haven't thought about these. I'll take a look.
>
>
> Jarkko's concern was that we'd need to distribute zlib with the core.
> Size not a problem, the licensing is not a problem, but we aren't sure it's
> portable to all platforms that perl is, and Jarkko didn't want to impose the
> burden of asking the obscure porters to port it.
Jarkko has a mailing list of obscure porters. It might be worth
polling them to see if they know where they stand vis-a-vis zlib.
-Norton
Andy Lester
of Module::Build and YAML is that it does away with the need for a
Makefile.PL, which would be a source of outside potential badness.
Instead, your local version of Makefile.PL (or Build.PL), that you've used
and trusted ever since you installed Module::Build, would read the YAML
file and parse and install the stuff that's found therein. There would
be much more security in installing using a module's descriptive text,
where Build.PL would check for any potential naughtiness, than having
to run a new, different Makefile.PL for each module.
Based on this premise, the .yaml description file for a module becomes
more important, and easier to deal with. The writability of YAML is
not as important as the readability in this case. Right now, it takes
knowledge of MakeMaker to read a Makefile.PL, but the .yaml would be
easier to handle.
So this improved readability of the .yaml file means that we might
actually have MORE people looking at their module internals before
installing. I suspect that most people don't bother even glancing at
their Makefile.PL files.
xoxo,
Andy
--
'Andy Lester an...@petdance.com
Programmer/author petdance.com
Daddy parsley.org/quinn Jk'=~/.+/s;print((split//,$&)
[unpack'C*',"n2]3%+>\"34.'%&.'^%4+!o.'"])
Graham Barr
> On Sat, Aug 03, 2002 at 03:43:17AM +0100, h...@crypt.org wrote:
> > Michael G Schwern <sch...@pobox.com> wrote:
> > : CPANPLUS *need it*
> >
> > Agreed. I'm not happy about the name, but I'm not sure precisely why
> > and I don't have a better one.
> >
> > : Archive::Tar *need them*
> > : Archive::Zip
> > : Compress::Zlib
> >
> > I haven't thought about these. I'll take a look.
>
> Jarkko's concern was that we'd need to distribute zlib with the core.
> Size not a problem, the licensing is not a problem, but we aren't sure it's
> portable to all platforms that perl is, and Jarkko didn't want to impose the
> burden of asking the obscure porters to port it.
Why do we *need* to include the source of zlib. There are other extensions
that we distribute, but without the source for the libraries they depend
on.
Graham.
Vadim Konovalov
> > >now sometimes misbehave, because in 5.8 perl
> mis-interprets file position
> > >because of "\r\n"=>"\n" translation.
> >
> > That is not supposed to happen.
While trying some more today I've discovered that mis-interpreting of file
position is somewhat tricky to catch: on my data it is occured, on simple
data such as
join "\r\n", "a".."z";
it did not happen.
Moreover, I got absolutely strange behaviour.
Attached script "strange.pl" has line "#__END__" before a line with
__DATA__. If I uncomment it I get slightly different result!
Anyways, with that line __END__ commented program results in
[>
<TrU>
<CrD>04012002, 11:16:01
<CrU>VOVA
<UsC>0
<Seg L=EN_US>Cells are defined ass individual service areas.
<Seg L=RU_RU>Cells are defined as individual service areas.
</TrU>]
whereas it should output
[
<TrU>
<CrD>04012002, 11:16:01
<CrU>VOVA
<UsC>0
<Seg L=EN_US>Cells are defined ass individual service areas.
<Seg L=RU_RU>Cells are defined as individual service areas.
</TrU>]
I tried two different 5.8.0 perls, one built with MSVC++ and another built
with Borland C++.
Could you please someone look into that problem?
May be sending it to perlbug will be appropriate?
Best wishes,
Vadim.
Vadim Konovalov
\r\n=>\n translation on Win32. Namely, incorrect position is reported by
"tell" function when :crlf layer is used.
Here it is:
use 5.008;
$data = join "", map {"$_\r\n"} "a".."zzz";
my $fh;
my $fcontents = $data;
open $fh, "<:crlf", \$fcontents;
#open $fh, "<", \$fcontents;
$/ = "xxx";
$_ = <$fh>;
my $pos = tell $fh;
print STDERR "pos=$pos\n";
$/ = "yyy";
$_ = <$fh>;
seek $fh, $pos, 0;
my $s = <$fh>;
print "[".substr($s,0,40)."]";
============
Correct output will be achieved if the only commented line will be
uncommented and a line before it will be commented out.
If I'll not figure out something wise to do, I'll report this via perlbug
system...
Best wishes,
Vadim.
h...@crypt.org
:If I'll not figure out something wise to do, I'll report this via perlbug
:system...
Please report it anyway: even if it turns out not to be a bug in perl,
the analysis may well be useful to the next person that has a problem
with this.
Hugo
Nick Ing-Simmons
>On Mon, Aug 05, 2002 at 03:09:46PM +0200, Johan Vromans wrote:
>>
>> If, however, it is mandatory that users can maintain the data in a
>> friendly and intuitive matter, most of these formats are a bad
>> choice. I'd prefer lisp, your choice will be different. In any case, I
>> think it is a bad choice to have a data language that uses whitespace
>> to denote structure (Yes, I've had this discussion with Ingy
>> already). Just imagine two users that have a different setting for tab
>> width in their editors.
>
>
> Tab characters are not allowed in indentation unless a #TAB directive
> is used. If such a directive is used, each indentation tab is
> equivalent to a certain number of spaces determined by the specified
> tab policy.
>
>Users are encouraged to *not* use tab characters as indentation.
None the less it will find more favour with the Python crowd than here ;-)
after tabs characters the next gotcha is probably a mailer that helpfully
line wraps - but perhaps a YAML that wide is too complex anyway.
>
>-Colin.
Colin Meyer
>
> friendly and intuitive matter, most of these formats are a bad
> choice. I'd prefer lisp, your choice will be different. In any case, I
> think it is a bad choice to have a data language that uses whitespace
> to denote structure (Yes, I've had this discussion with Ingy
> already). Just imagine two users that have a different setting for tab
> width in their editors.
From the YAML spec:
Tab characters are not allowed in indentation unless a #TAB directive
is used. If such a directive is used, each indentation tab is
equivalent to a certain number of spaces determined by the specified
tab policy.
Users are encouraged to *not* use tab characters as indentation.
-Colin.
Bart Schuller
> None the less it will find more favour with the Python crowd than here ;-)
I for one only heard about it from this thread, but I intend to use it.
The readability improvements alone are worth it for me, compared with
Data::Dumper. I expect my huge windows with D::D dumps of SOAP
structures can be made a little smaller :-)
The writability is even greater improved.
It takes a while to get rid of the notion that there has to be a
connection between XML and YAML though.
Yes, the spec is big. But it also looks complete. I've seen too many
data formats that leave out little details like what character set they
use if they're "ASCII" based.
--
Bart.