Oplibs, pmc libs, and function libs
Dan Sugalski
things it's time to address it.
We need parrot to be able to load in shared libraries, of course -- both
to get access to system libraries and to load up our own library code on
the fly. We'd also rather not have to have separate .so/.dll/.exe files
for each PMC class, opcode library, and Parrot function library, as that
can get unweildy very quickly, and potentially exhaust the file descriptor
pool on some OSes. (As well as being a massive headache to manage if a
high-level extension has multiple PMC classes written in C, an opcode
library or two, and some parrot functions written in C)
So, it looks like a shared library can have the following resources in it:
*) Native routines that must be actively imported
*) Parrot subs/methods
*) PMC classes
*) Opcode libraries
And, to make it more interesting, we don't want these things all
instantiated when we load in a library -- they need to be instantiated on
demand. So....
The logical solution seems to be that libraries should, when loaded,
register the parrot-aware stuff (parrot subs, pmc classes, and opcode
libraries) which can then be instantiated or imported as the loading
program needs. The already-specified behavior for library loading can be
leveraged for this, which is nice.
The sequence, then, is:
1) Code issues a library load call
2) Parrot dynaloads the shared library
3) The shared library init routine is called
3a) Shared library calls pmc_register to register each PMC class it has
3b) Shared library calls oplib_register to register each oplib it has
3c) Shared library calls class_register to register each class it has
4) The library loader makes sure the class/pmc/oplib it wants is
instantiated
A class, rather than a PMC class, is a HLL OO class or package. Basically
a namespace, set of functions and methods, and possibly some class
metadata.
Before I go any further, I want to make sure this makes sense to everyone.
Dan
Leopold Toetsch
> ... We'd also rather not have to have separate .so/.dll/.exe files
A small utility, that combines dynamic resources and emits a common
load routine, which calls the individual load routines could be enough
for this to achieve (modulo the same for init).
> And, to make it more interesting, we don't want these things all
> instantiated when we load in a library -- they need to be instantiated on
> demand. So....
This seems easily to be doable only for Parrot Subs / Classes.
- a dynamic PMC has its enum_class_<Foo>, which the assembler has
to know
- a dynamic oplib has its opnumber range. Again the assembler has to
load the lib to be able to spit out correct opcode numbers.
For both cases, when run from source files, the assembler does the
loadlib and the PMCs or oplibs are already installed. When running from
PBC, this happens on execution of the C<loadlib> opcode, which has to be
in the same sequence as the assembler has seen these C<loadlib> ops.
Sub/Classes i.e. parrot code is a little different. It has a different
opcode (load_bytecode). This can be PASM, PIR, or PBC. The former two
are compiled immediately, because the PMC Sub symbols must be installed
for global lookup. Predereferencing and or JITing happens on demand,
this is, when the first subroutine is invoked in that loaded bytecode.
> Before I go any further, I want to make sure this makes sense to everyone.
Yep.
> Dan
leo
Dan Sugalski
> Dan Sugalski <d...@sidhe.org> wrote:
>
> > ... We'd also rather not have to have separate .so/.dll/.exe files
>
> A small utility, that combines dynamic resources and emits a common
> load routine, which calls the individual load routines could be enough
> for this to achieve (modulo the same for init).
Yep, that's definitely in order as part of the extension building kit we
need to put together for Parrot's final release.
> > And, to make it more interesting, we don't want these things all
> > instantiated when we load in a library -- they need to be instantiated on
> > demand. So....
>
> This seems easily to be doable only for Parrot Subs / Classes.
> - a dynamic PMC has its enum_class_<Foo>, which the assembler has
> to know
> - a dynamic oplib has its opnumber range. Again the assembler has to
> load the lib to be able to spit out correct opcode numbers.
> For both cases, when run from source files, the assembler does the
> loadlib and the PMCs or oplibs are already installed. When running from
> PBC, this happens on execution of the C<loadlib> opcode, which has to be
> in the same sequence as the assembler has seen these C<loadlib> ops.
For PBC code, this is where metadata comes in. (Which means we can
potentially use it for on-the-fly assembled code as well) If a segment of
code is known to need an opcode library, then the bytecode loader should
make sure that library is loaded, instantiated, and that the opcode table
for the code segment is properly filled in.
Dan
Leopold Toetsch
> For PBC code, this is where metadata comes in. (Which means we can
> potentially use it for on-the-fly assembled code as well) If a segment of
> code is known to need an opcode library, then the bytecode loader should
> make sure that library is loaded, instantiated, and that the opcode table
> for the code segment is properly filled in.
Yes. When metadata -> Constant PMCs -> freeze/thaw -> VTABLE_traverse()
(sic!) is done, the bytecode loader can take care of loading needed
resources. This gives an extra bit of security, as the necessary
ordering of resources is always done in the assembler. If a C<loadlib>
opcode is found, the assembler prepares the metadata (which are in
correct order then) for runtime.
For on-the-fly running the metadata are prepared but unused, as the
loading has already been done (PMC and oplibs). So its always guaranteed
that dynamic resources are loaded in the same sequence, that is the
sequence that the assembler sees.
When it comes to multi-threaded programs that try to load different
PMCs (or oplibs) in their execution forks, we will have some fun...
> Dan
leo