is byteloaded code faster? could it be done faster yet?
Konovalov, Vadim (Vadim)** CTR **
I like the perl compiler and byteloader and am going to use it somewhat,
and would like to question some thoughts.
First, sometimes it comes to discussion that perl byteloader does not gain any speed, because usually compiling from source is even faster than byteloading.
How so?
Okay, for tiny "pm" modules (like strict.pm) this could be believable,
but is this considered so in general?
Further, looking at file.plc after issuing command
perl -MO=Bytecode,-H,-ofile.plc file.pl
it is clear that "use Byteloader;" is inserted into the start of compiled file and the source filter applied.
Good.
Now following idea comes into mind.
Given that all these .plc files use Byteloader, which is xs-loaded, speed optimization could be gained by following procedure
1) statically link byteloader into perl
2) call this perl somewhat different and pretend it is fed with plc file, say "perlc foo.plc"; using of filter and 'use Byteloader' line could be avoided, thus having speed gain.
3) (optionally) many pm files could be compiled into pmc files, much like python compiles its modules, and recognized by this modified perl binary
I think this idea is on the surface, so I pretend many people already know whether gain is not sufficient and why. Maybe someone already experimented with this?
to say, I am willing to play with this and want to start from point where someone before me left, thus avoiding repeating same initial steps.
Thanks in advance for sharing good ideas on the matter,
Vadim.
rurban
> I like the perl compiler and byteloader and am going to use it somewhat,
> and would like to question some thoughts.
>
> First, sometimes it comes to discussion that perl byteloader does not gain any speed, because usually compiling from source is even faster than byteloading.
>
> How so?
byteloader performance issues yet at all.
.pmc files are so much smaller, and I don't believe parse time is
faster then the ByteLoader switch
statement.
Maybe mmap'ing the .pmc file could improve performance.
> Okay, for tiny "pm" modules (like strict.pm) this could be believable,
> but is this considered so in general?
> Further, looking at file.plc after issuing command
> perl -MO=Bytecode,-H,-ofile.plc file.pl
> it is clear that "use Byteloader;" is inserted into the start of compiled file and the source filter applied.
>
> Good.
>
> Now following idea comes into mind.
>
> Given that all these .plc files use Byteloader, which is xs-loaded, speed optimization could be gained by following procedure
> 1) statically link byteloader into perl
> 2) call this perl somewhat different and pretend it is fed with plc file, say "perlc foo.plc"; using of filter and 'use Byteloader' line could be avoided, thus having speed gain.
Byteloader was in core until 5.10 and all of its users
compiled Byteloader statically.
You can still compile it in.
> 3) (optionally) many pm files could be compiled into pmc files, much like python compiles its modules, and recognized by this modified perl binary
Until the next axe falls on us. Byteloader was just
crippled by Jan Dubois on Windows, so it needs to be fixed for Win32.
cpan ByteCache
ByteCache - byte-compile modules when needed
use ByteCache;
use Other::Module;
This module causes any modules loaded after it to be loaded in
bytecode
compiled format. If a bytecode compiled version of the module does not
currently exist, ByteCache will call the compiler to create one and
then save it away.
> I think this idea is on the surface, so I pretend many people already know whether gain is not sufficient and why. Maybe someone already experimented with this?
>
> to say, I am willing to play with this and want to start from point where someone before me left, thus avoiding repeating same initial steps.
>
> Thanks in advance for sharing good ideas on the matter,
> Vadim.
Reini
Konovalov, Vadim (Vadim)** CTR **
> [mailto:perl-c...@googlegroups.com] On Behalf Of rurban
> I believe IO is the main bottleneck, but I haven't inspected
> byteloader performance issues yet at all.
> .pmc files are so much smaller, and I don't believe parse time is
> faster then the ByteLoader switch
> statement.
ok, good.
> Maybe mmap'ing the .pmc file could improve performance.
is mmap'ing available on Windows?
> > Given that all these .plc files use Byteloader, which is
> xs-loaded, speed optimization could be gained by following procedure
> > 1) statically link byteloader into perl
> > 2) call this perl somewhat different and pretend it is fed
> with plc file, say "perlc foo.plc"; using of filter and 'use
> Byteloader' line could be avoided, thus having speed gain.
>
> This was done in the late 1990-ies. It was called bperl.
> Byteloader was in core until 5.10 and all of its users
> compiled Byteloader statically.
> You can still compile it in.
I haven't found 'bperl' on CPAN - have it been lost?
>
> > 3) (optionally) many pm files could be compiled into pmc
> files, much like python compiles its modules, and recognized
> by this modified perl binary
>
> Sure. That's why core still accepts .pmc files.
cool! Good to know about this pmc thing :)
> Until the next axe falls on us. Byteloader was just
> crippled by Jan Dubois on Windows, so it needs to be fixed for Win32.
I saw that,
but I that particular change seems to me not a dangerous one - its harmless, IMO.
at least it could be worked-around and it reasonable.
I am much more bothered about inefficiency of loading some primary perl modules.
Some modules (like Archive::ZIP) include some IO::Handle which sometimes want to do long OR on bits and then Math::BigInt gets included.
Earlier, I was in favour of convenience of Math::BigInt, but then I realized its included at every single invocation, yet in PAR archives, thus bloating everything, which is bad thing...
>
> cpan ByteCache
>
> ByteCache - byte-compile modules when needed
>
> use ByteCache;
> use Other::Module;
>
> This module causes any modules loaded after it to be loaded in
> bytecode
> compiled format. If a bytecode compiled version of the module does not
> currently exist, ByteCache will call the compiler to create one and
> then save it away.
thanks for the pointer...
ByteCache looks tiny, and - if I correctly understood your note that core understands .pmc files - ByteCache even not needed if you'll go around and invoke
perlcc -B -o Foo.pmc Foo.pm
for evry single .pm file in @INC ?
Reini Urban
<vadim.k...@alcatel-lucent.com>:
>> From: perl-c...@googlegroups.com
>> [mailto:perl-c...@googlegroups.com] On Behalf Of rurban
>
>> I believe IO is the main bottleneck, but I haven't inspected
>> byteloader performance issues yet at all.
>> .pmc files are so much smaller, and I don't believe parse time is
>> faster then the ByteLoader switch
>> statement.
>
> ok, good.
>
>> Maybe mmap'ing the .pmc file could improve performance.
>
> is mmap'ing available on Windows?
Sure, CreateFileMapping, MapViewOfFile. See e.g. File::Map
http://cpansearch.perl.org/src/LEONT/File-Map-0.36/lib/File/Map.xs
>> > Given that all these .plc files use Byteloader, which is
>> xs-loaded, speed optimization could be gained by following procedure
>> > 1) statically link byteloader into perl
>> > 2) call this perl somewhat different and pretend it is fed
>> with plc file, say "perlc foo.plc"; using of filter and 'use
>> Byteloader' line could be avoided, thus having speed gain.
>>
>> This was done in the late 1990-ies. It was called bperl.
>> Byteloader was in core until 5.10 and all of its users
>> compiled Byteloader statically.
>> You can still compile it in.
>
> I haven't found 'bperl' on CPAN - have it been lost?
It was a perl executable Makefile target. It was abandoned when
ByteLoader became a source filter.
>> > 3) (optionally) many pm files could be compiled into pmc
>> files, much like python compiles its modules, and recognized
>> by this modified perl binary
>>
>> Sure. That's why core still accepts .pmc files.
>
> cool! Good to know about this pmc thing :)
>
>> Until the next axe falls on us. Byteloader was just
>> crippled by Jan Dubois on Windows, so it needs to be fixed for Win32.
>
> I saw that,
> but I that particular change seems to me not a dangerous one - its harmless, IMO.
> at least it could be worked-around and it reasonable.
Hopefully. I haven't tried yet to change the source filter to binary.
> I am much more bothered about inefficiency of loading some primary perl modules.
>
> Some modules (like Archive::ZIP) include some IO::Handle which sometimes want to do long OR on bits and then Math::BigInt gets included.
>
> Earlier, I was in favour of convenience of Math::BigInt, but then I realized its included at every single invocation, yet in PAR archives, thus bloating everything, which is bad thing...
Yep. Carp, unicode and much more uneccessary bloat if you are unlucky.
>>
>> cpan ByteCache
>>
>> ByteCache - byte-compile modules when needed
>>
>> use ByteCache;
>> use Other::Module;
>>
>> This module causes any modules loaded after it to be loaded in
>> bytecode
>> compiled format. If a bytecode compiled version of the module does not
>> currently exist, ByteCache will call the compiler to create one and
>> then save it away.
>
> thanks for the pointer...
>
> ByteCache looks tiny, and - if I correctly understood your note that core understands .pmc files - ByteCache even not needed if you'll go around and invoke
> perlcc -B -o Foo.pmc Foo.pm
> for evry single .pm file in @INC ?
So far yes. But I wouldn't trust it. p5p seems to want to get rid of
compiler support completely these days.
--
Reini Urban
http://phpwiki.org/ http://murbreak.at/