IMCC problems with library loading
Jeff Clites
origin down inside the IMCC code:
1) External libraries are being loaded at parse time.
Inside of INS() in imcc/parser_util.c, Parrot_load_lib() is called at
parse-time when loadlib is encountered. This is causing libraries to be
loaded twice (once at parse-time and once at run-time), which is a
problem in its own right, but it also just seems like generally the
wrong behavior.
2) Code which tries to instantiate a dynamically-loaded PMC fails to
parse.
Code such as this:
loadlib P1, "foo"
new P0, .Foo
fails to parse, because ".Foo" is being interpreted as a macro. For
example:
% ./parrot dynclasses/dynfoo.pasm
error:imcc:unknown macro '.Foo'
in file 'dynclasses/dynfoo.pasm' line 7
The problem seems to be the following code, which interprets ".Blah" as
a macro if "Blah" isn't an already-registered type:
imcc/imcc.l line 337:
<emit,INITIAL>{DOT}{LETTER}{LETTERDIGIT}* {
int type = pmc_type(interp, string_from_cstring(interp,
yytext+1, 0));
if (type > 0) {
char *buf = malloc(16);
sprintf(buf, "%d", type);
valp->s = buf;
return INTC;
}
if (!expand_macro(valp, interp, yytext+1))
fataly(1, sourcefile, line, "unknown macro '%s'", yytext);
}
I can't find any definitive docs on the macro syntax, so I don't know
if this is pointing out a true ambiguity in the grammar (since
dynamically-loaded PMC types won't be known at parse-time, before the
library defining them is loaded), or if macro usages are always
supposed to have argument-syntax, such as ".Blah()", which would mean
that this is just a problem with the parser. There's no example in
t/op/macro.t without parentheses, so I'm thinking it's the latter.
JEff
Leopold Toetsch
> I've run into a couple of issue with library loading which have their
> origin down inside the IMCC code:
> 1) External libraries are being loaded at parse time.
> Inside of INS() in imcc/parser_util.c, Parrot_load_lib() is called at
> parse-time when loadlib is encountered. This is causing libraries to be
> loaded twice (once at parse-time and once at run-time), which is a
> problem in its own right, but it also just seems like generally the
> wrong behavior.
If the library registers itself correctly, its loaded once only. E.g. a
PMC library calls pmc_register() so that the classname is known.
> 2) Code which tries to instantiate a dynamically-loaded PMC fails to
> parse.
> Code such as this:
> loadlib P1, "foo"
> new P0, .Foo
How does "foo" look like? dynclasses/dynfoo.pasm has an example of
loading dynamic classes (and it works here).
> JEff
leo
Jeff Clites
> Jeff Clites <jcl...@mac.com> wrote:
>> I've run into a couple of issue with library loading which have their
>> origin down inside the IMCC code:
>
>> 1) External libraries are being loaded at parse time.
>
>> Inside of INS() in imcc/parser_util.c, Parrot_load_lib() is called at
>> parse-time when loadlib is encountered. This is causing libraries to
>> be
>> loaded twice (once at parse-time and once at run-time), which is a
>> problem in its own right, but it also just seems like generally the
>> wrong behavior.
>
> If the library registers itself correctly, its loaded once only. E.g. a
> PMC library calls pmc_register() so that the classname is known.
Actually, Parrot_load_lib calls Parrot_dlopen (inside if the call to
get_path), then later calls is_loaded and closes the library if it was
already loaded before. This is loading the library twice, from the
perspective of the operating system. But that's really a side issue.
My main point is that you can't do conditional library loading. This
code will try to load the "doesnt_exist" library, and I don't think it
should:
branch HERE
loadlib P1, "doesnt_exist"
HERE:
end
It's not because the branch fails--it's because the library is loaded
before the script starts running.
Or is this expected--that a loadlib instruction will cause a library to
be loaded, even if the instruction is never reached?
>> 2) Code which tries to instantiate a dynamically-loaded PMC fails to
>> parse.
>
>> Code such as this:
>
>> loadlib P1, "foo"
>> new P0, .Foo
>
> How does "foo" look like? dynclasses/dynfoo.pasm has an example of
> loading dynamic classes (and it works here).
dynclasses/dynfoo.pasm is the example I was referring to. It actually
works for me also--if I leave the code in imcc/parser_util.c which
causes the parse-time library loading mentioned above. (I had commented
that out.)
But rearranging the assembly slightly so that the execution order is
logically the same but the loadlib instruction is further down in the
source causes it to fail. That is:
null P0
branch one
two:
find_type I0, "Foo"
print I0
print "\n"
new P0, .Foo
print "ok 2\n"
branch three
one:
loadlib P1, "foo"
print "ok 1\n"
branch two
three:
set I0, P0
print I0
print "\n"
end
This again fails with the error:
error:imcc:unknown macro '.Foo'
This is because the parser is encountering ".Foo" before it has loaded
the lib at parse-time.
This brings me back to my original question of whether the parser
should think ".Foo" is a macro at all, or if rather that's not legal
macro syntax.
JEff
Leopold Toetsch
> My main point is that you can't do conditional library loading. This
> code will try to load the "doesnt_exist" library, and I don't think it
> should:
> branch HERE
> loadlib P1, "doesnt_exist"
> HERE:
> end
> It's not because the branch fails--it's because the library is loaded
> before the script starts running.
It depends. When you run it immediately, you are right: The library is
loaded at compile time. When you compile it as .pbc and run it, the lib
isn't loaded ar runtime.
*But*, when writing that stuff, one prelim was, that the compiler sees
all libs in the same sequence as runtime does. So above code (or your
second example) is illegal.
> dynclasses/dynfoo.pasm is the example I was referring to. It actually
> works for me also--if I leave the code in imcc/parser_util.c which
> causes the parse-time library loading mentioned above. (I had commented
> that out.)
The parser needs the information, that the lib provides, e.g. the
classname. We could have a
new P0, "className"
too, of course, where everything happens at runtime and the parser
doesn't know anything about that class.
> This brings me back to my original question of whether the parser
> should think ".Foo" is a macro at all, or if rather that's not legal
> macro syntax.
This doesn't change the problem per se.
> JEff
leo
Jeff Clites
> Jeff Clites <jcl...@mac.com> wrote:
>
>> My main point is that you can't do conditional library loading. This
>> code will try to load the "doesnt_exist" library, and I don't think it
>> should:
>
>> branch HERE
>> loadlib P1, "doesnt_exist"
>> HERE:
>> end
>
>> It's not because the branch fails--it's because the library is loaded
>> before the script starts running.
>
> It depends. When you run it immediately, you are right: The library is
> loaded at compile time. When you compile it as .pbc and run it, the lib
> isn't loaded ar runtime.
> *But*, when writing that stuff, one prelim was, that the compiler sees
> all libs in the same sequence as runtime does. So above code (or your
> second example) is illegal.
Okay for now then. But long-term it seems pretty obvious that this will
be too restrictive. Here are some specific problems:
1) For non-parrot libs (like the ncurses lib. used in Dan's example),
it's not necessary to load them at parse-time. It's a waste in terms of
performance, but also not all libraries can be unloaded on all
platforms, and some libraries will cause behavior changes that aren't
intended to take effect until later. This is an argument for the
distinction we used to have with different ops to load different types
of libraries. We may have been a bit hasty in unifying them all.
2) A program may act differently when run from source v. run from .pbc,
because different libraries may be loaded.
3) You can't do conditional loading of the form "if some condition is
met, load library A which implements the PMC I need, else load a
different library B which implements it". For instance, you might want
to choose at runtime between loading a debug and a non-debug version of
some library. Currently I'd imagine that we'd end up loading both at
compile-time, which it's probably guaranteed to not work correctly.
>> This brings me back to my original question of whether the parser
>> should think ".Foo" is a macro at all, or if rather that's not legal
>> macro syntax.
>
> This doesn't change the problem per se.
It does in that the current parser behavior blocks finding out what the
real issue is.
Also, it's still a legitimate question, so for the third time I'll ask
if ".Foo" is valid macro syntax? If not, then the current parser
behavior is misleading. I'm of the opinion that it should not be valid
macro syntax--that ".Foo()" should be necessary instead, because
otherwise the grammar is ambiguous.
JEff
Leopold Toetsch
> On Nov 17, 2003, at 11:07 AM, Leopold Toetsch wrote:
>> *But*, when writing that stuff, one prelim was, that the compiler sees
>> all libs in the same sequence as runtime does. So above code (or your
>> second example) is illegal.
> Okay for now then. But long-term it seems pretty obvious that this will
> be too restrictive. Here are some specific problems:
Yep. That's right.
> Also, it's still a legitimate question, so for the third time I'll ask
> if ".Foo" is valid macro syntax?
Its just the constant syntax. A macro can expand to many lines, while a
constant is just one substitution.
> JEff
leo