Serializing code
Yuval Kogman
Now with perl 6 compiling to javascript and perl 5 and what not, i
think there is a big future when you merge the two and remove the
details.
The way HTML::Prototype works is:
you get an OO interface, which is clean and simple
you use it to generate mostly functions
the functions can be thought of as closures, sometimes being
curried by your values
the functions are in a foreign language, and cannot be executed
- they are really strings that work in another runtime.
Now, in the foreseable future I think it's plausible that the Ajax
libraries will be written in the server side's native language. When
the code is serialized by a template, the template will say to what
language to compile it, and the result is the PIL code, or compiled
foreign language code that will work on the remote side.
What I'm saying is that in the future HTML::Prototype could be
implemented in Perl 6, and the code in question is compiled by the
web app (this is more than quasiquoting - the code is actually
parsed into the native language - full macros, code reuse and so
forth).
The perl 6 code will just be suited for the runtime that a browser
presents, including DOM manipulations, for example.
Now, one step further - ruby is compiled to PIL. Ruby guys maintain
HTML::Prototype, because it's their thing. It's written in ruby, and
perl 6 code gets to use the ruby code, and compile that code down to
javascript for IE, but to PIR for firefox with embedded parrot.
How's that for code reuse and lack of vendor lock-in?
So now that the skeptics can see why this is important, on the
design side I'd like to ask for ideas on how the code serialization
looks...
sub { $?DOM.document.write("<p>hello world!</p>") }.emit(
:runtime($browser_autodetect_object),
# this can be guessed by asking the $runtime object:
# :language<javascript>,
);
This is superficially nice, but here's what's missing:
- this is basically the interface to the code emitters - they
need lots of control in terms of optimization flags, and so
forth, and command line options (which are just like named
arguments) absolutely do not scale, as any makefile can tell
you.
- These need to be defined and stabilized:
- runtime (the virtual machine and more)
- version
- features
- target language (The actual code)
- architechture
- definition version
- environment
- browser
- command line
- web server
- ...
- system
- $?OS (this should not be "browser")
- ...
- compiler (the thing that makes your language into PIL)
- emitter (the thing that makes PIL into a language)
- handling included code (libraries could be compiled to static
JS files that are fetched)
- on the fly generation + caching
- precalculation
- standalone serialization (a single function has everything
it needs)
- what exactly is a code object?
- a wrapper for some PIL code
- that can be executed by the runtime
- what exactly is &code.emit
- also an object?
- comparable to other emissions?
- further transformable?
- when is it just text or bytes?
- handling of data
- types of:
- globals?
- closures?
- serialization? proxying?
- two examples of data sharing policies:
- the border between the webapp and the browser is
smudged by serialization and proxy objects
- the border between the webapp and the browser is clear
and distinct, and calls between the two are done by
explicitly invoking one runtime from the other
--
() Yuval Kogman <nothi...@woobling.org> 0xEBD27418 perl hacker &
/\ kung foo master: /me whallops greyface with a fnord: neeyah!!!!!!!
Ingo Blechschmidt
Yuval Kogman <nothingmuch <at> woobling.org> writes:
> So now that the skeptics can see why this is important, on the
> design side I'd like to ask for ideas on how the code serialization
> looks...
>
> sub { $?DOM.document.write("<p>hello world!</p>") }.emit(
> :runtime($browser_autodetect_object),
> # this can be guessed by asking the $runtime object:
> # :language<javascript>,
> );
>
> This is superficially nice,
> but here's what's missing:
> - what exactly is a code object?
> - a wrapper for some PIL code
> - that can be executed by the runtime
(think macros, use and BEGIN blocks), but...:
sub foo { $?DOM.document.write(...) }
BEGIN { foo() }; # error, there's no $?DOM object
# at compile-time!
> - what exactly is &code.emit
> - also an object?
object, even though it calls subs and methods from the
Perl6::Internals namespace (or somesuch).
> - handling of data
> - types of:
> - globals?
other variables, have any type you want them to have?
> - closures?
A subclass of Code, e.g. Bare.
> - two examples of data sharing policies:
> - the border between the webapp and the browser is
> smudged by serialization and proxy objects
> - the border between the webapp and the browser is
clear
> and distinct, and calls between the two are done by
> explicitly invoking one runtime from the other
--Ingo
Yuval Kogman
> Hi,
>
> Yuval Kogman <nothingmuch <at> woobling.org> writes:
> > So now that the skeptics can see why this is important, on the
> > design side I'd like to ask for ideas on how the code serialization
> > looks...
> >
> > sub { $?DOM.document.write("<p>hello world!</p>") }.emit(
> > :runtime($browser_autodetect_object),
> > # this can be guessed by asking the $runtime object:
> > # :language<javascript>,
> > );
> >
> > This is superficially nice,
>
> Indeed!
>
> > but here's what's missing:
> [...]
>
> > - what exactly is a code object?
> > - a wrapper for some PIL code
> > - that can be executed by the runtime
>
> Code objects may as well be executed at compile-time
> (think macros, use and BEGIN blocks), but...:
But it's still the runtime that runs them ;-)
Compilation has a runtime that runs the compiler, and compile-time
code. The output may be handed down to another runtime that actually
"runs".
>
> sub foo { $?DOM.document.write(...) }
> BEGIN { foo() }; # error, there's no $?DOM object
> # at compile-time!
Unless you're compiling in the browser ;-)
> Well, you call &code.emit above, so it's probably a Code
> object, even though it calls subs and methods from the
> Perl6::Internals namespace (or somesuch).
Not &code, but the return value of &code.emit
>
> > - handling of data
> > - types of:
> > - globals?
>
> I've probably misunderstood you, but can't globals, like all
> other variables, have any type you want them to have?
Not types in the "kind of value" type... Read it as:
handling of data... what types of data... global variables are
types of data
> > - closures?
>
> A subclass of Code, e.g. Bare.
How are they stored?
> I'd like to have a pragma to switch between these policies.
I doubt a pragma is enough - we need full introspection and fine
grained control for this, both in the lexical and the dynamic
scopes.
--
() Yuval Kogman <nothi...@woobling.org> 0xEBD27418 perl hacker &
/\ kung foo master: /me supports the ASCII Ribbon Campaign: neeyah!!!
Autrijus Tang
> > sub foo { $?DOM.document.write(...) }
> > BEGIN { foo() }; # error, there's no $?DOM object
> > # at compile-time!
>
> Unless you're compiling in the browser ;-)
Which... is possible, and that's how I plan to support eval().
Thanks,
/Autrijus/
Ingo Blechschmidt
Yuval Kogman <nothingmuch <at> woobling.org> writes:
> On Thu, Aug 18, 2005 at 12:24:40 +0000, Ingo Blechschmidt wrote:
> > Yuval Kogman <nothingmuch <at> woobling.org> writes:
> > > So now that the skeptics can see why this is important, on the
> > > design side I'd like to ask for ideas on how the code serialization
> > > looks...
> > >
> > > sub { $?DOM.document.write("<p>hello world!</p>") }.emit(
> > > :runtime($browser_autodetect_object),
> > > # this can be guessed by asking the $runtime object:
> > > # :language<javascript>,
> > > );
> > >
> > > This is superficially nice,
> >
> > Indeed!
> >
> > > but here's what's missing:
> > [...]
> >
> > > - what exactly is a code object?
> > > - a wrapper for some PIL code
> > > - that can be executed by the runtime
> >
> > Code objects may as well be executed at compile-time
> > (think macros, use and BEGIN blocks), but...:
>
> But it's still the runtime that runs them
>
> code. The output may be handed down to another runtime that actually
> "runs".
> > Well, you call &code.emit above, so it's probably a Code
> > object, even though it calls subs and methods from the
> > Perl6::Internals namespace (or somesuch).
>
> Not &code, but the return value of &code.emit
$template.param("globaljs") = $emitted.global_section;
$template.param("js_of_a_div") = $emitted.main_section;
Of course, these additional methods wouldn't be available for
all backends.
> > > - handling of data
> > > - types of:
> > > - globals?
> >
> > I've probably misunderstood you, but can't globals, like all
> > other variables, have any type you want them to have?
>
> Not types in the "kind of value" type... Read it as:
>
> handling of data... what types of data... global variables are
> types of data
like normal lexical variables, I don't see a big problem there.
WRT pads: Luckily, pads are constant, i.e. you can't inject
a new lexical variable at runtime (%MY::<new_var> = ...).
The backend does have to take care of read accesses, of course
(think $CALLER::, $::(...), %MY::, %OUR::, etc.).
FYI, this is how PIL2JS handles this currently:
my $foo; # is really
my $foo; %MY::<foo> := $foo;
sub foo {...}; # is really
sub foo {
my %MY:: = ();
push @*CURRENT_SUBPADS, %MY::;
LEAVE { pop @*CURRENT_SUBPADS };
...;
> > > - closures?
> >
> > A subclass of Code, e.g. Bare.
>
> How are they stored?
are access to outer lexical variables:
my $a = ...;
my $b = { ...$a... };
$b();
A backend can then either use the native closure support of the
runtime it targets (if existing) or it has to lambda lift, i.e.
my $b = { ...$a... }; # is really
my $b = -> $a { ...$a... };
$b(); # is really
$b($a);
> > I'd like to have a pragma to switch between these policies.
>
> I doubt a pragma is enough - we need full introspection and fine
> grained control for this, both in the lexical and the dynamic
> scopes.
--Ingo
--
Linux, the choice of a GNU | The computer revolution is over. The
generation on a dual AMD | computers won. -- Eduard Bloch
Athlon! |
Yuval Kogman
> > Not &code, but the return value of &code.emit
>
> Hm, Str? Or possibly a subtype of Str, allowing:
I would guess an AST, that is, any object, that implements
stringification.
the AST could just be the same PIL reblessed with some new
serialization magic, but I guess for most languages you want to make
a real AST to AST conversion, and only then serialize.
> Ah. Normal globals can probably be freezed more or less exactly
> like normal lexical variables, I don't see a big problem there.
The question is - should globals be frozen? Or should they
optimistically refer to values on the other side?
I think that
sub hello {
$*DOM.document.write("<p>Hello World!</p>");
}
should capture $*DOM in the same sense that a closure matches them,
and the global scope is implicitly the uber-parent lexical scope
type thingy.
Then, once we've unified, we can steal something from C and friends:
$*DOM is external;
sub hello {...} # $*DOM is not serialized, but will be resolved
# by the runtime on the other side
Anybody got ideas on how control is needed, and how it should be
specified?
> The backend does have to take care of read accesses, of course
> (think $CALLER::, $::(...), %MY::, %OUR::, etc.).
That's just the pads of the lexical structure and the dynamic stack,
and they're existence is implied by PIL. Adding the interface is
easy if the implementation isn't drunk, since things will tend to
just work out.
> The code itself isn't much a problem, much more problematic
> are access to outer lexical variables:
>
> my $a = ...;
> my $b = { ...$a... };
> $b();
Right... It is my assumption that actually serializing this is
trivial. The real question is whether we want to serialize, and
what parts we would like to serialize when we do.
--
() Yuval Kogman <nothi...@woobling.org> 0xEBD27418 perl hacker &
/\ kung foo master: /me sushi-spin-kicks : neeyah!!!!!!!!!!!!!!!!!!!!
Ingo Blechschmidt
> > > Not &code, but the return value of &code.emit
> >
> > Hm, Str? Or possibly a subtype of Str, allowing:
>
> I would guess an AST, that is, any object, that implements
> stringification.
>
> the AST could just be the same PIL reblessed with some new
> serialization magic, but I guess for most languages you want to make
> a real AST to AST conversion, and only then serialize.
But we should note that some backends don't generate meaningful
ASTs, simply because they don't convert PIL -> target language
AST -> target language, but PIL -> target language directly. I.e.
my $ast = $code.emit(..., :language<Foo>);
say keys $ast; # "FooCode"
say $ast<FooCode>; # ...
my $ast = { 3 + 4 }.emit(..., :language<PIL>);
say $ast.pBody.pStmt.pExpr.pLV.pArgs[1].pLit.pVal; # 4
> > Ah. Normal globals can probably be freezed more or less exactly
> > like normal lexical variables, I don't see a big problem there.
>
> The question is - should globals be frozen? Or should they
> optimistically refer to values on the other side?
>
> I think that
>
> sub hello {
> $*DOM.document.write("<p>Hello World!</p>");
> }
>
> should capture $*DOM in the same sense that a closure matches them,
> and the global scope is implicitly the uber-parent lexical scope
> type thingy.
>
> Then, once we've unified, we can steal something from C and friends:
>
> $*DOM is external;
> sub hello {...} # $*DOM is not serialized, but will be resolved
> # by the runtime on the other side
>
> Anybody got ideas on how control is needed, and how it should be
> specified?
There could be a module JavaScript::Browser or so, which would
export $*DOM. I.e.:
#!/usr/bin/pugs
$*DOM.document.write(...);
# Compile-time error: $*DOM not declared
#!/usr/bin/pugs -BParrot
use JavaScript::Browser <$*DOM>;
$*DOM.document.write(...);
# error: $*DOM does only work when running in a browser
#!/usr/bin/pugs -CJS
use JavaScript::Browser <$*DOM>;
$*DOM.document.write(...);
# fine now
This exported $*DOM object could then be a (proxy) object with
appropriate magic -- i.e. die when the current runtime is not
a browser and relay all calls to the respective native JavaScript
objects otherwise.
I think something like "$*DOM is exported" is too generic,
not sure...
> > The code itself isn't much a problem, much more problematic
> > are access to outer lexical variables:
> >
> > my $a = ...;
> > my $b = { ...$a... };
> > $b();
>
> Right... It is my assumption that actually serializing this is
> trivial. The real question is whether we want to serialize, and
> what parts we would like to serialize when we do.
specifically marked as objects which should not be freezed (e.g.
$*DOM).
--Ingo
--
Linux, the choice of a GNU | There are no answers, only
generation on a dual AMD | cross-references.
Athlon! |
Yuval Kogman
> Hi,
>
> Yuval Kogman <nothingmuch <at> woobling.org> writes:
> But we should note that some backends don't generate meaningful
> ASTs, simply because they don't convert PIL -> target language
> AST -> target language, but PIL -> target language directly. I.e.
>
> my $ast = $code.emit(..., :language<Foo>);
> say keys $ast; # "FooCode"
> say $ast<FooCode>; # ...
>
> my $ast = { 3 + 4 }.emit(..., :language<PIL>);
> say $ast.pBody.pStmt.pExpr.pLV.pArgs[1].pLit.pVal; # 4
>
In that case they just take the PIL tree and rebless it with the
code emitter as the stringifier.
> There could be a module JavaScript::Browser or so, which would
> export $*DOM. I.e.:
>
> #!/usr/bin/pugs
> $*DOM.document.write(...);
> # Compile-time error: $*DOM not declared
>
> #!/usr/bin/pugs -BParrot
> use JavaScript::Browser <$*DOM>;
> $*DOM.document.write(...);
> # error: $*DOM does only work when running in a browser
>
> #!/usr/bin/pugs -CJS
> use JavaScript::Browser <$*DOM>;
> $*DOM.document.write(...);
> # fine now
>
> This exported $*DOM object could then be a (proxy) object with
> appropriate magic -- i.e. die when the current runtime is not
> a browser and relay all calls to the respective native JavaScript
> objects otherwise.
>
> I think something like "$*DOM is exported" is too generic,
> not sure...
Yes, you're absolutely right.
--
() Yuval Kogman <nothi...@woobling.org> 0xEBD27418 perl hacker &
/\ kung foo master: /me spreads pj3Ar using 0wnage: neeyah!!!!!!!!!!!