RFC: Autoloading charnames

[Karl Williamson]

It has been proposed that charnames automatically load when a \N{} is
seen. That is easy to do, and I think it should load as if the :full
and :short options had both been specified.

Does anyone disagree with doing this?

If it is done, should the functions in the module also autoload. That
is, should a line like 'my $poo = charnames::viacode(0x1F4A9)' fail if
charnames hasn't been 'used' or 'required' or should it cause the module
to load ?

[David Golden]

On Tue, Nov 22, 2011 at 5:03 PM, Karl Williamson
<pub...@khwilliamson.com> wrote:
> It has been proposed that charnames automatically load when a \N{} is seen.
>  That is easy to do, and I think it should load as if the :full and :short
> options had both been specified.
>
> Does anyone disagree with doing this?

What are the pros and cons, specifically, of choosing either or both
of :full and :short? Would it be loaded at compile time or runtime?

> If it is done, should the functions in the module also autoload.  That is,
> should a line like 'my $poo = charnames::viacode(0x1F4A9)' fail if charnames
> hasn't been 'used' or 'required' or should it cause the module to load ?

I don't like general-purpose auto-loading. However, I can see the
case for \N{} (just I saw the case for autoloading IO::File when
methods are called on filehandles. If \N{} does load charnames, then
I would expect charnames::viacode(...) to work. However, without
prior \N{} (assuming compile-time loading), I would expect
charnames::viacode() to be a compilation error.

-- David

[Jesse Luehrs]

I personally wouldn't expect the presence or absence of a \N{} escape at
some arbitrary location in my code (possibly even in a different file)
to make the difference (on its own) of whether calling functions in
charnames:: worked or not. This seems like it would cause confusion.

-doy

[Karl Williamson]

On 11/22/2011 03:23 PM, Jesse Luehrs wrote:
> On Tue, Nov 22, 2011 at 05:17:41PM -0500, David Golden wrote:
>> On Tue, Nov 22, 2011 at 5:03 PM, Karl Williamson
>> <pub...@khwilliamson.com> wrote:
>>> It has been proposed that charnames automatically load when a \N{} is seen.
>>> That is easy to do, and I think it should load as if the :full and :short
>>> options had both been specified.
>>>
>>> Does anyone disagree with doing this?
>>
>> What are the pros and cons, specifically, of choosing either or both
>> of :full and :short? Would it be loaded at compile time or runtime?

\N{} is compile time, so this would be as well. The reason to choose
:full is that is what people generally expect. The universe of names
matched by it and :short is disjoint, as :short name matches require a
colon in them, and :full name matches forbid a colon. e.g.,
\N{greek:alpha} is matched by :short. I don't see any cons of having
both :full and :short be accepted. The pro of having :short auto loaded
is that it just happens without you having to worry about it. There's
no extra memory, etc. required to have it.

I had considered having :loose instead of :full, which is an extension
of :full allowing more sloppy name input that meets the Unicode
standard's recommendations. However, Perl traditionally has only had
:full, and I thought it could cause confusion, besides being slower.

>>
>>> If it is done, should the functions in the module also autoload. That is,
>>> should a line like 'my $poo = charnames::viacode(0x1F4A9)' fail if charnames
>>> hasn't been 'used' or 'required' or should it cause the module to load ?
>>
>> I don't like general-purpose auto-loading. However, I can see the
>> case for \N{} (just I saw the case for autoloading IO::File when
>> methods are called on filehandles. If \N{} does load charnames, then
>> I would expect charnames::viacode(...) to work. However, without
>> prior \N{} (assuming compile-time loading), I would expect
>> charnames::viacode() to be a compilation error.
>
> I personally wouldn't expect the presence or absence of a \N{} escape at
> some arbitrary location in my code (possibly even in a different file)
> to make the difference (on its own) of whether calling functions in
> charnames:: worked or not. This seems like it would cause confusion.

This was a concern of mine as well. Since subroutines are global, if
anyone loads the module, they get defined everywhere, unless I'm missing
something.

[Abigail]

On Tue, Nov 22, 2011 at 03:03:35PM -0700, Karl Williamson wrote:
> It has been proposed that charnames automatically load when a \N{} is
> seen. That is easy to do, and I think it should load as if the :full
> and :short options had both been specified.
>
> Does anyone disagree with doing this?

Not me. This has been one of my top wishes for years.

> If it is done, should the functions in the module also autoload. That
> is, should a line like 'my $poo = charnames::viacode(0x1F4A9)' fail if
> charnames hasn't been 'used' or 'required' or should it cause the module
> to load ?

Having done things automatically done for you seems to be the Perlish way.
It's quite clear what the programmer wants; I will not object to some DWIM.



Abigail

[David Golden]

On Tue, Nov 22, 2011 at 5:23 PM, Jesse Luehrs <d...@tozt.net> wrote:
>> I don't like general-purpose auto-loading.  However, I can see the
>> case for \N{} (just I saw the case for autoloading IO::File when
>> methods are called on filehandles.  If \N{} does load charnames, then
>> I would expect charnames::viacode(...) to work.  However, without
>> prior \N{} (assuming compile-time loading), I would expect
>> charnames::viacode() to be a compilation error.
>
> I personally wouldn't expect the presence or absence of a \N{} escape at
> some arbitrary location in my code (possibly even in a different file)
> to make the difference (on its own) of whether calling functions in
> charnames:: worked or not. This seems like it would cause confusion.

Let me clarify: *if* we say that \N{} autoloads charnames, then I
would expect things everywhere to behave just as if there were "use
charnames qw/:full :short/" as the statement before the statement with
\N{}.

It all comes down to how we document it. If we say that all the
Unicode character names are available for use with \N{}, without
specifying how that is implemented, then I could also support having
some private charnames::guts module be loaded behind the scenes to
provide the mapping. That would leave charnames.pm not loaded so that
charnames::viacode() continues to be an error. Then using \N{} just
has a one-time cost but DWIMs and you don't need
to know about charnames at all.

Put differently, we should *NOT* do either of these:

(a) *say* we autoload charnames on \N{} and then *not* have
charnames::viacode() be a valid function call after \N{}
(b) *don't say* anything about charnames, but have
charnames::viacode() be valid after \N{}

-- David

[Jesse Luehrs]

I can agree with these two statements. I think that not documenting the
implementation details makes the most sense - just say that all names
are available with \N{}, and don't mention charnames in connection with
that at all (other than as a "see also" or something).

-doy

[Johan Vromans]

Jesse Luehrs <d...@tozt.net> writes:

> I can agree with these two statements.

Despite this, you then say something different:

> ... and don't mention charnames in connection with
> that at all

One of the elegant aspects of Karl's proposal is that there's no trickery
involved. Upon the first encounter of \N, charnames is loaded with :full
and :short. Simple, elegant, clear.

-- Johan

[Jesse Luehrs]

On Wed, Nov 23, 2011 at 08:30:12AM +0100, Johan Vromans wrote:
> Jesse Luehrs <d...@tozt.net> writes:
>
> > I can agree with these two statements.
>
> Despite this, you then say something different:
>
> > ... and don't mention charnames in connection with
> > that at all

This is not different at all. charnames isn't mentioned at all, so the
first statement doesn't hold, and charnames::viacode isn't available
after \N{}, so the second statement doesn't hold.

> One of the elegant aspects of Karl's proposal is that there's no trickery
> involved. Upon the first encounter of \N, charnames is loaded with :full
> and :short. Simple, elegant, clear.

I'm not sure how this should be considered trickery. It's splitting the
two entirely separate pieces of functionality that charnames contains
into pieces that are actually separate, and avoiding confusing action at
a distance. Other than "because that's how it's currently implemented",
I don't see why using a \N{} sequence should have any impact on what
functions are available to use in my code.

-doy

[H.Merijn Brand]

On Tue, 22 Nov 2011 15:03:35 -0700, Karl Williamson
<pub...@khwilliamson.com> wrote:

> It has been proposed that charnames automatically load when a \N{} is
> seen. That is easy to do, and I think it should load as if the :full
> and :short options had both been specified.

/me is in favor!

> Does anyone disagree with doing this?

When I use charnames, it is always like

use charnames ":alias" => ":pro";

would autoloading charnames support a default like that? Guess not.
No problem, just to note ...

> If it is done, should the functions in the module also autoload. That
> is, should a line like 'my $poo = charnames::viacode(0x1F4A9)' fail if
> charnames hasn't been 'used' or 'required' or should it cause the module
> to load ?

No, as the functions are fully qualified, they should not fail

--
H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/
using 5.00307 through 5.14 and porting perl5.15.x on HP-UX 10.20, 11.00,
11.11, 11.23 and 11.31, OpenSuSE 10.1, 11.0 .. 11.4 and AIX 5.2 and 5.3.
http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/
http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/

[H.Merijn Brand]

On Tue, 22 Nov 2011 16:23:19 -0600, Jesse Luehrs <d...@tozt.net> wrote:

> > > If it is done, should the functions in the module also autoload.  That is,
> > > should a line like 'my $poo = charnames::viacode(0x1F4A9)' fail if charnames
> > > hasn't been 'used' or 'required' or should it cause the module to load ?
> >
> > I don't like general-purpose auto-loading. However, I can see the
> > case for \N{} (just I saw the case for autoloading IO::File when
> > methods are called on filehandles. If \N{} does load charnames, then
> > I would expect charnames::viacode(...) to work. However, without
> > prior \N{} (assuming compile-time loading), I would expect
> > charnames::viacode() to be a compilation error.
>
> I personally wouldn't expect the presence or absence of a \N{} escape at
> some arbitrary location in my code (possibly even in a different file)
> to make the difference (on its own) of whether calling functions in
> charnames:: worked or not. This seems like it would cause confusion.

I'd say lets make charnames behave at least like utf8:

$ perl -wlE'say utf8::is_utf8("\x{20ac}")'
1


See, no use and still available. If \N{} autoloads charnames (and I am
fully in favor), so should charnames::viacode

One more reason for doing so is that charnames cannot be backported and
is not available on CPAN. It is very tightly coupled to the current core

[Johan Vromans]

Jesse Luehrs <d...@tozt.net> writes:

> I don't see why using a \N{} sequence should have any impact on what
> functions are available to use in my code.

Compare:

$ perl -wE 'binmode STDOUT; warn File::Spec->catfile("foo","bar")'
Can't locate object method "catfile" via package "File::Spec" (perhaps you forgot to load "File::Spec"?) at -e line 1.

with:

$ perl -wE 'STDOUT->binmode; warn File::Spec->catfile("foo","bar")'
foo/bar at -e line 1.

-- Johan

[Tom Christiansen]

> I'd say lets make charnames behave at least like utf8:

> $ perl -wlE'say utf8::is_utf8("\x{20ac}")'
> 1

> See, no use and still available. If \N{} autoloads charnames (and I am
> fully in favor), so should charnames::viacode

> One more reason for doing so is that charnames cannot be backported and
> is not available on CPAN. It is very tightly coupled to the current core

That does seem a strong argument in support of the idea.

--tom

[Jesse Luehrs]

Sure, I think that's confusing too.

-doy

[Karl Williamson]

FWIW, I find this confusing as well, yet this is how things work today:

$ blead -E '
print charnames::viacode(0x41), "\n";

package foo {
use charnames();
}'
LATIN CAPITAL LETTER A

Loading charnames in a different scope, even package, enables use of the
functions at a distance.

Does this fact change anyone's opinion?

[Jesse Luehrs]

Well, this is just how the rest of perl works:

$ perl -E'warn File::Spec->catfile("foo", "bar"); package foo { use
File::Spec () }'

foo/bar at -e line 1.

since packages are global and use happens at compile time. The only
thing that is confusing to me is for this kind of thing to happen for
autoloaded stuff (when it's not necessary anyway - I'm not really
proposing for IO::File to stop using File::Spec). I feel like within a
given module, either use of anything in it should trigger autoloading,
or none of it should (and what "anything" means in this case depends on
whether charnames stays as one module or is split into charnames (for
the functions) and charnames::guts or whatever (for the \N{} stuff)).
I'll admit that this may be a bit overcomplicated on the implementation
side.

-doy

[Abigail]

I don't, and I really wish Perl did it more often. Every time I write

Some::Class::I::did::not::use -> foo ()

and Perl tells me "oh, you didn't load Some::Class::I::did::not::use",
I think "well, if you know what's missing, why don't you just do it
for me". [1]

But maybe I'm the only Perl user who appreciates DWIM.


Abigail
[1] I've no idea wether it's actually feasable, or how to do it if it were.

[Jesse Luehrs]

Well, loading a package can have arbitrary effects, so that's not
necessarily a safe thing to do (in case it actually was a typo). If it's
really what you want though, Class::Autouse is available on CPAN.

-doy

[Jesse Luehrs]

On Wed, Nov 23, 2011 at 10:48:30AM -0500, David Golden wrote:
> FWIW, I don't see why IO::File needs to load File::Spec anyway.

I don't see why either, but the same argument applies for
s/File::Spec/Carp/.

-doy

[David Golden]

FWIW, I don't see why IO::File needs to load File::Spec anyway.

[Tom Christiansen]

Karl Williamson <pub...@khwilliamson.com> wrote
on Wed, 23 Nov 2011 07:51:59 MST:

>>>> I don't see why using a \N{} sequence should have any impact on what
>>>> functions are available to use in my code.

>>> Compare:
>>>
>>> $ perl -wE 'binmode STDOUT; warn File::Spec->catfile("foo","bar")'

>>> Can't locate object method "catfile" via package "File::Spec" (perh<SNIP>

>>>
>>> with:
>>>
>>> $ perl -wE 'STDOUT->binmode; warn File::Spec->catfile("foo","bar")'
>>> foo/bar at -e line 1.

>> Sure, I think that's confusing too.

> FWIW, I find this confusing as well, yet this is how things work today:

> $ blead -E '
> print charnames::viacode(0x41), "\n";
> package foo {
> use charnames();
> }'
>
> LATIN CAPITAL LETTER A

> Loading charnames in a different scope, even package, enables
> use of the functions at a distance.

Ayup.

> Does this fact change anyone's opinion?

It may, but it doesn't change *mine*. This is just how things happen
in Perl, and there's no way to address it without redesigning the whole
language from the ground up; see Perl6.

And er, I've actually made intentional use of the sort of thing you show in
the quoted portion. Similarly with IO:: stuff. It's probably a dumb thing
to do, but I knew that was how Perl happens to work, so I did it. It's a
bit embarrassing but there it is. You use what's there.

Perl really isn't spec-based to allow one to adjudge a bug from a feature
based on a formal spec, so there's a lot of this sort of thing going around.

--tom

[Leon Timmermans]

On Wed, Nov 23, 2011 at 4:48 PM, David Golden <xda...@gmail.com> wrote:
>  FWIW, I don't see why IO::File needs to load File::Spec anyway.

Yeah, it seems it's a leftover from a previous version of IO::File
(c70704064b4c55c8116e7b43f88c86dc34819ee2), just as SelectSaver. Looks
like it can be safely deleted.

Leon

[Jesse Luehrs]

Right. And this is why I'd like to try to avoid adding more things like
this whenever possible - these are the sorts of things that makes the
language a lot harder to change in the future. Can we even make IO::File
stop loading File::Spec at this point? Someone may be relying on this
autoloading behavior, and their code would break. (To be clear, I think
the answer to this question is "yes", but the potential for breaking
existing code isn't something that should be ignored either.)

-doy

[Johan Vromans]

Jesse Luehrs <d...@tozt.net> writes:

> Can we even make IO::File
> stop loading File::Spec at this point? Someone may be relying on this
> autoloading behavior, and their code would break.

It is a fundamental principle of modular programming that when using a
module, it should not matter what that module does to accomplish its
documented behaviour. Note: Documented behaviour.

If the next version of IO::File no longer needs to load File::Spec than
that is a great optimalisation and any code that breaks because of this
deserves to break.

Now, where did I put my asbestos suit...

-- Johan

[Tom Christiansen]

Jesse Luehrs <d...@tozt.net> wrote on Wed, 23 Nov 2011 10:32:09 CST:

>> Perl really isn't spec-based to allow one to adjudge a bug from a feature
>> based on a formal spec, so there's a lot of this sort of thing going around.

> Right. And this is why I'd like to try to avoid adding more things like
> this whenever possible - these are the sorts of things that makes the
> language a lot harder to change in the future. Can we even make IO::File
> stop loading File::Spec at this point? Someone may be relying on this
> autoloading behavior, and their code would break. (To be clear, I think
> the answer to this question is "yes", but the potential for breaking
> existing code isn't something that should be ignored either.)

I think you're right that it could very well break code.

Whether this counts as breaking something that's already broken,
I don't know. Nor if it is, whether that justifies doing so.

Is the current situation actually hurting anyone?

--tom

[Tom Christiansen]

Abigail <abi...@abigail.be> wrote
on Wed, 23 Nov 2011 16:26:32 +0100:

>>>> I don't see why using a \N{} sequence should have any impact on what
>>>> functions are available to use in my code.

>>> Compare:
>>>
>>> $ perl -wE 'binmode STDOUT; warn File::Spec->catfile("foo","bar")'
>>> Can't locate object method "catfile" via package "File::Spec" (perhaps you forgot to load "File::Spec"?) at -e line 1.
>>>
>>> with:
>>>
>>> $ perl -wE 'STDOUT->binmode; warn File::Spec->catfile("foo","bar")'
>>> foo/bar at -e line 1.

>> Sure, I think that's confusing too.

> I don't, and I really wish Perl did it more often. Every time I write

> Some::Class::I::did::not::use -> foo ()

> and Perl tells me "oh, you didn't load Some::Class::I::did::not::use",
> I think "well, if you know what's missing, why don't you just do it
> for me". [1]

I've thought that, too. I have a memory-tickle that I even
tried to Make It So, once. Don't remember what came of that.

> But maybe I'm the only Perl user who appreciates DWIM.

*Larry* certainly does: it's a primary motivator behind the spirit of Perl.
Happens all over the place. The only caveat is that you always have to
realize that when Larry says "Do What I Mean", he means it. That is,
it's actually saying "Do What *Larry* Means" — but you are not Larry. :)

(Er, unless you are: Hi, Larry! :):)

> Abigail
>
> [1] I've no idea whether it's actually feasible,
> nor how to do it if it were.

Wouldn't UNIVERSAL::AUTOLOAD suffice to rapid-prototype this?

Seems like that'd be a quick enough way to get a feel for its
effects and side-effects, its benefits and detriments.

--tom

==========================================================

PS: More magical muttering about Perl's dwimmery follow.

+--------------------+
| ᴇx ʟɪʙʀɪs ᴍᴜʀᴀʟɪs: |
+--------------------+

dwimmer
ᴅᴡɪᴍ is an acronym for “Do What I Mean”, the principle that something
should just do what you want it to do without an undue amount of fuss.
A bit of code that does “dwimming” is a “dwimmer”. Dwimming can require
a great deal of behind-the-scenes magic, which (if it doesn’t stay
properly behind the scenes) is called a dweomer instead.

dweomer
An enchantment, illusion, phantasm, or jugglery. Said when Perl’s
magical dwimmer effects don’t do what you expect, but rather seem to
be the product of arcane dweomercraft, sorcery, or wonder-working.
[From Middle English dweomer-cræften, in Laȝamon]

magic
Technically speaking, any extra semantics attached to a variable
such as $!, $0, %ENV, or %SIG, or to any tied variable. Magical
things happen when you diddle those variables.

magical increment
An increment operator that knows how to bump up ᴀsᴄɪɪ alphabetics
as well as numbers.

magical variables
Special variables that have side effects when you access them or assign to
them. For example, in Perl, changing elements of the %ENV array also changes
the corresponding environment variables that subprocesses will use. Reading
the $! variable gives you the current system error number or message.

sigil
A glyph used in magic. Or, for Perl, the symbol in front of a variable name,
such as $, @, and %.

tie
The bond between a magical variable and its implementation class.

ᴡʏsɪᴡʏɢ
What You See Is What You Get. Usually used when something that appears
on the screen matches how it will eventually look, like Perl’s format
declarations. Also used to mean the opposite of magic because everything
works exactly as it appears, as in the three-argument form of open.

And also:

Some human endeavors require a disguise. Sometimes the intent is to
deceive, but more often, the intent is to communicate something true at a
deeper level. For instance, many job interviewers expect you to dress up
in a tie to indicate that you’re seriously interested in fitting in, even
though both of you know you’ll never wear a tie on the job. It’s odd when
you think about it: tying a piece of cloth around your neck can magically
get you a job. In Perl culture, the tie operator plays a similar role: it
lets you create a seemingly normal variable that, behind the disguise, is
actually a full-fledged Perl object that is expected to have an interesting
personality of its own. It’s just an odd bit of magic, like pulling
Bugs Bunny out of a hat.

In his poem "Errantry", Tolkien uses the old word "sigaldry",
which simply means "rune-magic":

| He sat and sang a melody,
| his errantry a tarrying,
| he begged a pretty butterfly,
| that fluttered by to marry him.
| She scorned him and she scoffed at him,
| she laughed at him unpitying,
| so long he studied wizardry,
-> | and sigaldry and smithying.

Clearly, Perl's ᴅᴡɪᴍmer involves sigaldry in its sigil-dweomer. :)

In its Second Edition of 1989, the ᴏᴇᴅ gives only three really old
citations for "sigaldry":

❧ ?c1225 (1200) Ancrene Riwle (Cleo. C.vi) (1972) 155
Sigaldrie [c1230 Corpus sygaldren]. false tollunges leuunge o nore
on swefne & alle wiche-creftes.

❧ a1400 K. Alis. (W.) 7015
Quede and harme he wil me spye,‥Gef he wot of this sygaldrye [Laud
trigoldrye] That this trowes [= trees] kan lye.

❧ ?a1500 Chester Pl., Crucifixion (Shaks. Soc.) II. 69
Burye hym wher thy wil be,
But look thou make no sigaldry
To raise him up agayne.

Tolkien also used sigaldry in Canto VII of his epic poem,
"The Lay of Leithian, whose lines 2068–2079 run:

| Not yet by Men enthralled adored,
| now was he Morgoth’s mightiest lord,
| Master of Wolves, whose shivering howl
| for echoed in the hills, and foul
-> | enchantments and dark sigaldry
| did weave and wield. In glamoury
| that necromancer held his hosts
| of phantoms and of wandering ghosts,
| of misbegotten or spell-wronged
| working his bidding dark and vile
| the werewolves of the Wizard’s Isle.

Somebody should send the ᴏᴇᴅ the Tolkien "sigaldry" citations, being
over four centuries more recent than the few they currently use.

Hm hm hm, maybe we should sneak "sigaldry" into the 4ᵗʰ Edition of
Programming Perl: I'd do believe I'd rather like an ᴏᴇᴅ citation. :)

Seems our existing "dweomer" and "dweomercraft" should suffice.

Gary Gygax and Gene Wolfe also resurrect words otherwise lost to modern use,
even if it's only as a mathom or other bric-à-brac passed around as a curio.

--tom

PS: There, *that* should stoke the googlers after weird old words, even
if my spell-checker does rather hate those spellings. And spells.

[Jesse Luehrs]

Well, if nothing else, it's a potentially unnecessary use of
~500kb/process.

-doy

[Tom Christiansen]

Jesse Luehrs <d...@tozt.net> wrote
on Wed, 23 Nov 2011 11:27:27 CST:

> Well, if nothing else, it's a potentially unnecessary use of
> ~500kb/process.

Ouch! That's good enough for me. Well, or used to be. Guido
recently told the Python folks that having to load 500k of Unicode
properties to get regexes to work right on Unicode data didn't
really bother him much, and I tend to agree with him.

Still, most people think nothing of loading in a megabyte of cruft to do
something just to get some syntactic sugar for already-loaded, built-in
functionality. These are with v5.14:

openbsd% perl -e 'system "ps v$$"; require IO::Handle; system "ps v$$"'
PID STAT TIME SL RE PAGEIN VSZ RSS LIM TSIZ %CPU %MEM COMMAND
9539 S+ 0:00.01 0 0 0 1244 1704 496036 940 0.0 0.3 perl -e system "ps v$
PID STAT TIME SL RE PAGEIN VSZ RSS LIM TSIZ %CPU %MEM COMMAND
9539 S+ 0:00.06 0 1 0 1476 2580 496036 940 1.0 0.5 perl -e system "ps v$

That's +232 VSZ, +876 RSS.

openbsd% perl -e 'system "ps v$$"; STDOUT->autoflush; system "ps v$$"'
PID STAT TIME SL RE PAGEIN VSZ RSS LIM TSIZ %CPU %MEM COMMAND
12282 S+ 0:00.01 0 0 0 1160 1712 496036 940 0.0 0.3 perl -e system "ps v$$"; STDOUT->au
PID STAT TIME SL RE PAGEIN VSZ RSS LIM TSIZ %CPU %MEM COMMAND
12282 S+ 0:00.09 0 1 0 1468 2840 496036 940 4.0 0.5 perl -e system "ps v$$"; STDOUT->au

That's +308 VSZ, +1228 RSS.

openbsd% perl -e 'system "ps v$$"; $| = 1; system "ps v$$"'
PID STAT TIME SL RE PAGEIN VSZ RSS LIM TSIZ %CPU %MEM COMMAND
2575 S+ 0:00.01 0 0 0 1244 1712 496036 940 0.0 0.3 perl -e system "ps v$$"; $| = 1; sy
PID STAT TIME SL RE PAGEIN VSZ RSS LIM TSIZ %CPU %MEM COMMAND
2575 S+ 0:00.01 0 0 0 1244 1740 496036 940 0.0 0.3 perl -e system "ps v$$"; $| = 1; sy

That's +0 VSZ, +28 RSS.

I know, I know: it's "only" a megabyte, and legibility is more important
than performance. Still. I've occasionally wondered whether the compiler
could catch that as a special case, but I end up deciding it's not worth
the code bloat in the compiler source such a hack would require. Never
stopped FORTRAN from optimizing its intrinsics though. Adding cheats to
the compiler can be worth more than they appear to cost, because it's a
rising tide that raises all ships. Look at all the compiler hacks for
detecting the standard Livermore Loops benchmark, for example. Nobody
thinks those should be removed for truth and beauty. I just wouldn't
want to have to be the one supporting them.

--tom

[(Johan Vromans)]

[Quoting Tom Christiansen, on November 23 2011, 10:54, in "Re: RFC: Autoloading"]

> Guido
> recently told the Python folks that having to load 500k of Unicode
> properties to get regexes to work right on Unicode data didn't
> really bother him much, and I tend to agree with him.

In this case, it serves a purpose.

-- Johan

[Konovalov, Vadim (Vadim)** CTR **]

> From: Karl Williamson [mailto:pub...@khwilliamson.com]

> It has been proposed that charnames automatically load when a \N{} is
> seen. That is easy to do, and I think it should load as if the :full
> and :short options had both been specified.
>

> Does anyone disagree with doing this?

I see 2 problems,

1) as everyone knows "threading" should make "use" under better control - namely after threads creation.
How this will be handled?

2) PAR packer already generates huge executables (compared to e.g 5.6) because it happens now to load Math::BigInt everytime, due to some unfortunate dependency on some $^O.
Under new circumstances, the problem will be even worse.

Maybe there are more.

I, personally, very much like for modules usage to be under better control, i.e. load what I demand.

I think instead of loading of something when \N{} is seen would be a proper advice from warning system ("you probably need to do "use charnames::blabla" module")

Regards,
Vadim.

[Leon Timmermans]

On Thu, Nov 24, 2011 at 9:40 AM, Konovalov, Vadim (Vadim)** CTR **
<vadim.k...@alcatel-lucent.com> wrote:
> 1) as everyone knows "threading" should make "use" under better control - namely after threads creation.
> How this will be handled?

Why would it be different than now?

> 2) PAR packer already generates huge executables (compared to e.g 5.6) because it happens now to load Math::BigInt everytime, due to some unfortunate dependency on some $^O.
> Under new circumstances, the problem will be even worse.

Again, I don't see this being different from now. If you need
charnames it should be bundled, otherwise not.

> I, personally, very much like for modules usage to be under better control, i.e. load what I demand.
>
> I think instead of loading of something when \N{} is seen would be a proper advice from warning system ("you probably need to do "use charnames::blabla" module")

Telling the user "we know what you wanted from us, be we're not doing
it" seems terribly user-unfriendly to me.

Leon

[Konovalov, Vadim (Vadim)** CTR **]

> From: Leon Timmermans

> On Thu, Nov 24, 2011 at 9:40 AM, Konovalov, Vadim wrote:
> > 1) as everyone knows "threading" should make "use" under better control - namely after threads creation.
> > How this will be handled?
>
> Why would it be different than now?

It is well-known adviced practice to use heavy "use something"
before any threads creation.

I am just wondered, this automated
use charnames; # :full or whatever
- will it be done before threads creation, or at what moment it will be
called WRT threads?

>
> > 2) PAR packer already generates huge executables (compared
> to e.g 5.6) because it happens now to load Math::BigInt
> everytime, due to some unfortunate dependency on some $^O.
> > Under new circumstances, the problem will be even worse.
>
> Again, I don't see this being different from now. If you need
> charnames it should be bundled, otherwise not.

Have you seen how modules dependency in PAR resolved?

There is some heuristic with false positives, and mostly the logic is to
package more modules than needed, to avoid any misses in packed
executables, so resulting in bigger executables than needed.

But with proposed new feature things will end up with adding
"use charnames;" to all PAR-packed executables, thus resulting
already oversized executables to be even bigger.

Needlessly bigger.
This is my concern.

>
> > I, personally, very much like for modules usage to be under
> better control, i.e. load what I demand.
> >
> > I think instead of loading of something when \N{} is seen
> would be a proper advice from warning system ("you probably
> need to do "use charnames::blabla" module")
>
> Telling the user "we know what you wanted from us, be we're not doing
> it" seems terribly user-unfriendly to me.

not to me :)

Yet, failed attempts to guess on what user needs is even worse.
Often this is unnoticeable waste, but in many situations this is hateful.

Regards,
Vadik.

[Leon Timmermans]

On Thu, Nov 24, 2011 at 2:35 PM, Konovalov, Vadim (Vadim)** CTR **
<vadim.k...@alcatel-lucent.com> wrote:
> It is well-known adviced practice to use heavy "use something"
> before any threads creation.
>
> I am just wondered, this automated
>  use charnames; # :full or whatever
> - will it be done before threads creation, or at what moment it will be
> called WRT threads?

I assume it'd be done during compile-time, so it shouldn't matter.

> Have you seen how modules dependency in PAR resolved?
>
> There is some heuristic with false positives, and mostly the logic is to
> package more modules than needed, to avoid any misses in packed
> executables, so resulting in bigger executables than needed.

No, but I don't think that avoiding PAR bugs should be a guiding
principle in perl's core design.

> But with proposed new feature things will end up with adding
> "use charnames;" to all PAR-packed executables, thus resulting
> already oversized executables to be even bigger.

Then PAR/Module::ScanDeps needs to add detection for \N{}.

Leon

[Konovalov, Vadim (Vadim)** CTR **]

> From: Leon Timmermans

> On Thu, Nov 24, 2011 at 2:35 PM, Konovalov, Vadim wrote:
> > It is well-known adviced practice to use heavy "use something"
> > before any threads creation.
> >
> > I am just wondered, this automated
> >  use charnames; # :full or whatever
> > - will it be done before threads creation, or at what
> moment it will be
> > called WRT threads?
>
> I assume it'd be done during compile-time, so it shouldn't matter.

good.

As long as you know better, I can rely on this knowledge.

>
> > Have you seen how modules dependency in PAR resolved?
> >
> > There is some heuristic with false positives, and mostly
> the logic is to
> > package more modules than needed, to avoid any misses in packed
> > executables, so resulting in bigger executables than needed.
>
> No, but I don't think that avoiding PAR bugs should be a guiding
> principle in perl's core design.

Look at this from the other side.

If perl this much full of such sudden behaviours,
it is much less maintainable.

Some of my application had only "perl.dll" with tiny scripts inside
char [] strings.

The further we go, the harder perl servers as scalable embeddable helper engine.
It will be always huge and never tiny.

So - it is not PAR bugs that bother me, but uncontrollable behaviour.

>
> > But with proposed new feature things will end up with adding
> > "use charnames;" to all PAR-packed executables, thus resulting
> > already oversized executables to be even bigger.
>
> Then PAR/Module::ScanDeps needs to add detection for \N{}.

That's not realistic.

this is why I want for better control.
Ask what you want and you will get it.

This simple.

[John Imrie]

On 23/11/2011 15:26, Abigail wrote:
> I don't, and I really wish Perl did it more often. Every time I write
>
> Some::Class::I::did::not::use -> foo ()
>
>

I'd be happy with this if it only worked under no strict 'refs'

John

[Reini Urban]

On Wed, Nov 23, 2011 at 11:54 AM, Tom Christiansen <tch...@perl.com> wrote:
> Jesse Luehrs <d...@tozt.net> wrote
>   on Wed, 23 Nov 2011 11:27:27 CST:
>
>> Well, if nothing else, it's a potentially unnecessary use of
>> ~500kb/process.
>
> Ouch!  That's good enough for me.  Well, or used to be.  Guido
> recently told the Python folks that having to load 500k of Unicode
> properties to get regexes to work right on Unicode data didn't
> really bother him much, and I tend to agree with him.
>
> Still, most people think nothing of loading in a megabyte of cruft to do
> something just to get some syntactic sugar for already-loaded, built-in
> functionality.

The problem is not the size, the problem is the time to load and
unload this tables.
It's a big difference to map a precompiled icu<xx>.so into the process space,
or when perl loads their handmade unicode tables, e.g. folding tables
on lc or m///i;

And then we also have to manually unload them, handling through the
refcounted SV's which is even worse, compared to unmap or forget. Hurts twice.

Why cannot we just precompile our tables into a XS lib, and access them
not via perl hashes, but pure functional, so its loaded and destroyed fast.
We autogenerate these tables anyway so it should be no big deal.

I'm not familiar with our implementation and I think that Collate
might partially
do such a thing, but the rest? There's a lot of perl data which should
just be a
XS without refcounted hashes.
--
Reini Urban
http://cpanel.net/   http://www.perl-compiler.org/

[Reini Urban]

Banging on my head. Encode does exactly this.

So I guess the ToFold and utf8_heavy tables should be done in
a similar fashion, right?

[Aristotle Pagaltzis]

* Ricardo Signes <perl...@rjbs.manxome.org> [2011-11-26 17:05]:
> My likely order of preference, best first:
>
> 1. using \N{} automatically loads the data needed to make it work,
> but no other functions magically become available

Reini’s suggestion lines up with this too.

[Karl Williamson]

On 11/26/2011 09:00 AM, Ricardo Signes wrote:
> * David Golden<xda...@gmail.com> [2011-11-22T18:51:35]
>> Put differently, we should *NOT* do either of these:
>>
>> (a) *say* we autoload charnames on \N{} and then *not* have
>> charnames::viacode() be a valid function call after \N{}
>> (b) *don't say* anything about charnames, but have
>> charnames::viacode() be valid after \N{}
>
> This summarized my feelings very well, too.
>
> I would prefer if \N{} did not cause charnames::viacode (etc) to just start
> working. That is a pretty weak preference, but I prefer that functions like
> that have their module loaded explicitly. \N{}, on the other hand, is (or
> ought to be) a much more basic language feature that should always work.
>
> This would require work to provide a way to only load charnames, and not
> everything else. I know that work might not get done right now, and I think
> it's more useful to have \N{} Just Work than delay for this small point.

>
> My likely order of preference, best first:
>
> 1. using \N{} automatically loads the data needed to make it work, but
> no other functions magically become available
>

> 2. using \N{} loads charnames normally, with documentation saying something
> like "at present, using \N{} will load the charnames package, but this
> is an implementation detail subject to change; if you're using functions
> from it, C<use charnames>"
>
> 3. using \N{} loads charnames normally, and we 'fess up to it
>

I had prototyped #1 some time ago. I used a simple solution to avoid
importing the functions, and that was to use undocumented mangled names
for the function definitions in charnames.pm (simply prefixing them with
an underscore, but further mangling could be done), and then in the
import method, doing
*charnames::viacode = *_viacode;
*charnames::vianame = *_vianame;
*charnames::string_vianame = *_string_vianame;

Is this acceptable, or is a sub-module really needed?

This is already done, whereas making it XS is more work, which can be
done later.

[Tom Christiansen]

> Won't this break (for example) Unicode::UCD, which does "use
> charnames ()" but then calls charnames::viacode?

I do that all the time, actually.

--tom

[Karl Williamson]

On 11/27/2011 03:43 PM, Ricardo Signes wrote:
> * Karl Williamson<pub...@khwilliamson.com> [2011-11-27T17:30:08]

>> I had prototyped #1 some time ago. I used a simple solution to
>> avoid importing the functions, and that was to use undocumented
>> mangled names for the function definitions in charnames.pm (simply
>> prefixing them with an underscore, but further mangling could be
>> done), and then in the import method, doing
>> *charnames::viacode = *_viacode;
>> *charnames::vianame = *_vianame;
>> *charnames::string_vianame = *_string_vianame;
>>
>> Is this acceptable, or is a sub-module really needed?
>

> Won't this break (for example) Unicode::UCD, which does "use charnames ()" but
> then calls charnames::viacode?
>

> That seems like a pattern we might find other places, too. grep.cpan.me is
> down, but I found more than zero results with:
>
> http://www.google.com/codesearch#search/&q=%22use%20charnames%20%28%29%22%20lang:perl&p=1&type=cs
>

Doh! It's been long enough since I wrote this prototype that I forgot I
never ran it through the test suite.

[Dr.Ruud]

On 2011-11-23 18:54, Tom Christiansen wrote:
> Jesse Luehrs<d...@tozt.net> wrote
> on Wed, 23 Nov 2011 11:27:27 CST:

>> Well, if nothing else, it's a potentially unnecessary use of
>> ~500kb/process.
>
> Ouch! That's good enough for me. Well, or used to be. Guido
> recently told the Python folks that having to load 500k of Unicode
> properties to get regexes to work right on Unicode data didn't
> really bother him much, and I tend to agree with him.

I vaguely remember having implemented it well within 100 kB.
(I assume that the '500kb' wasn't meant as '500 kilobits')
I will search and dig it up, and report if it still stands.

--
Ruud

[Karl Williamson]

On 11/26/2011 09:00 AM, Ricardo Signes wrote:
> * David Golden<xda...@gmail.com> [2011-11-22T18:51:35]
>> Put differently, we should *NOT* do either of these:
>>
>> (a) *say* we autoload charnames on \N{} and then *not* have
>> charnames::viacode() be a valid function call after \N{}
>> (b) *don't say* anything about charnames, but have
>> charnames::viacode() be valid after \N{}
>
> This summarized my feelings very well, too.
>
> I would prefer if \N{} did not cause charnames::viacode (etc) to just start
> working. That is a pretty weak preference, but I prefer that functions like
> that have their module loaded explicitly. \N{}, on the other hand, is (or
> ought to be) a much more basic language feature that should always work.
>
> This would require work to provide a way to only load charnames, and not
> everything else. I know that work might not get done right now, and I think
> it's more useful to have \N{} Just Work than delay for this small point.
>
> My likely order of preference, best first:
>
> 1. using \N{} automatically loads the data needed to make it work, but
> no other functions magically become available
>
> 2. using \N{} loads charnames normally, with documentation saying something
> like "at present, using \N{} will load the charnames package, but this
> is an implementation detail subject to change; if you're using functions
> from it, C<use charnames>"
>
> 3. using \N{} loads charnames normally, and we 'fess up to it
>

Sunder scenario #1, what should happen after someone does 'no
charnames'; should it just unload the functions, or also \N{}?

[Karl Williamson]

s/Sunder/Under/

And, you can't unload the functions

[Karl Williamson]

By commit fbb93542d6662666a88828fb4c15803f8ba377f0.

The functions aren't autoloaded; there is no unimport method.

[Karl Williamson]

On 12/20/2011 10:07 PM, Karl Williamson wrote:
> By commit fbb93542d6662666a88828fb4c15803f8ba377f0.
>
> The functions aren't autoloaded; there is no unimport method.
>

It occurred to me later that it might be useful to show the entire
commit message for this:

Autoload charnames for \N{name}

This autoloads charnames.pm when needed. It uses the :full and :short
options. :loose is not used because of its relative unfamiliarity in
the Perl community, and is slower. (If someone later added a typical
"use charnames qw(:full)", things that previously matched under :loose
would start to fail, causing confustion. If :loose does become more
common, we can change this in the future to use it; the converse isn't
true.)

The callable functions in the module are not automatically loaded. To
access them, an explicit "use charnames" must be provided.