Re: compile/eval work

[Matthew Wilson]

(transferring this thread to the mailing list)

Yes, I agree with all of the below... the "diffs" I sent you below actually already do what you describe...

On Thu, Apr 2, 2009 at 5:01 AM, Tim Macfarlane <timmac...@gmail.com> wrote:

Dude this is hot!

I've added you as a member to the project so you'll be able to commit commit away

I'm wondering if 'compile' should just be a method on string (like eval too) and 'eval' should be the syntactic extension?

so:

eval "print 2"

evaluates immediately, and is the same as:

"print 2".r:eval (load-stack-frame)

Which is the same as:

"print 2".r:compile (load-stack-frame) ()

Also wonder if 'load-stack-frame' should be 'current-stack-frame'? load is a verb, so would imply an action, like a function `load-stack-frame ()`. current-stack-frame is better as a variable and could even have a namespace: tycho:runtime:current-stack-frame, so as not to take up useful variable names. (don't think you can do this with the current syntax language stuff... which is all going to change soonish anyway...)

Also, we should be able to tell compile/eval which language to use too. Eventually (not too far away) we should be able to define languages as variables that we can use to load source files (as modules) and use in eval/compile too. Just something to think about.

Anyway, awesome stuff! Great to have you on the, um, "team" ;)

Tim.

2009/4/2 Matthew Wilson <diak...@gmail.com>

found a couple corrections; discuss online sometime.. :)

On Wed, Apr 1, 2009 at 5:50 PM, Matthew Wilson <diak...@gmail.com> wrote:

floob := (x) =>
  print x
flibbity = 2234234
flabbity = 2389482
closure-first = compile "floob (flibbity" + " + " + "flabbity)"

closure-first ()
// prints 4623716

(compile "closure-first ()")()
// prints 4623716

(compile "print (flibbity" + " + " + "flabbity)")()
// prints 4623716

"closure-first ()".r:eval (load-stack-frame)
// prints 4623716

floob := (x) =>
  print 999999

floob (1111111)
// prints 999999

closure-first ()
// prints 999999


I added a "load-stack-frame" keyword thus:

to expression.language, added:
parser expression
    ...
    compile "compile" expression
        tycho:runtime:compile (#expression, load-stack-frame)
    ...
    // keywords
    ...
    load-stack-frame "load-stack-frame"

to RuntimeFunctions.cs, added:
        [TychoFunction2("compile")]
        public static AnyObject Compile(string sourceCode, AnyObject contextStackFrame) {
            return RuntimeModule.CreateCompiledClosure(sourceCode, contextStackFrame);
        }

to RuntimeModules.cs, added:
        public static AnyObject CreateCompiledClosure(string evalSource, AnyObject contextStackFrame) {
            return new ExpressionLanguage(
                new ModuleSecurityObject(
                    Namespaces.User,
                    new ModuleFrameObject(contextStackFrame)),
                Namespaces.User)
            .CompileOperation(
                evalSource,
                "runtime generated",
                contextStackFrame);
        }

to TopLevel.cs, added:
        public AnyObject CompileOperation(string expression, string filename, AnyObject context) {
            FrameSymbolScope scope = BuildScope(context, new ModuleSymbolScope(ModuleManifest));
            return CompileOperation(expression, filename, scope, null, context, false);
        }

(and altered this one to be:)
        public AnyObject CompileOperation (string expression, string filename, AnyObject context, IModuleScopeLoader moduleLoader) {
            ModuleManifest = moduleLoader;
            FrameSymbolScope scope = BuildScope (context, new ModuleSymbolScope (moduleLoader));
            return CompileOperation (expression, filename, scope, null, context, false);
        }

To AliasSymbol in ExpressionCompiler.cs, added:
            LoadStackFrameSymbol = Namespaces.Parser.Get("load-stack-frame"),
To BuildExpressionTransform(), added:
            transform.AddTransform (LoadStackFrameSymbol, (context, term) =>
            {
                return ByteCodeCompiler.LoadStackFrame (
                    term.Term);
            });

To StringMethods.cs, added:
        [TychoMethod2("string", "compile")]
        public static AnyObject Compile(string sourceCode, AnyObject contextStackFrame) {
            return RuntimeModule.CreateCompiledClosure(sourceCode, contextStackFrame);
        }

        [TychoMethod2("string", "eval")]
        public static AnyObject Eval(string sourceCode, AnyObject contextStackFrame) {
            return RuntimeModule.CreateCompiledClosure(sourceCode, contextStackFrame).Invoke();
        }

[timmac...@gmail.com]

Just looking through some of the example uses of 'eval' you sent me
(see below).

Two things are worth pointing out with this eval functionality: The
stack frames we use in tycho are currently fully first class
reflectable objects so we can rebuild symbol tables on the fly at
runtime. We may not be able to do this when we target the CLR. Instead
we get the compiler to generate the symbol tables and store them as a
variable in the function that uses them (a function with an eval or
current-stack-frame reference.)

Also, the stack frame used in the eval must be mutable such that new
variables/constants can be declared and set in the eval code. Fine in
the current toy-interpreted target but another thing to think about
when targeting IL.

BUG: the runtime stack frames currently don't store any information
about consts and variables: everything is a variable (I think), so
'eval' code could be changing the values of constants. Not good but
fixable.

(here are the examples, `binding` is taken from Ruby)

import tycho:runtime as r

binding = method
() => binding (current-stack-frame)
stack-frame =>
object
method r:invoke (source)
source.r:to-string ().r:eval (stack-frame)

c := 4

d := binding ()
e := binding (load-stack-frame)

d ("c := 5")
print "inside d, c is " + d ("c")
print "inside e, c is " + e ("c")
print "outside, c is #c"

e ("c := 6")
print "inside d, c is " + d ("c")
print "inside e, c is " + e ("c")
print "outside, c is #c"

// Ruby Bindings to Tycho examples

// def f
// a = 22
// b = 33
// binding
// end

f := () =>
a := 22
b := 33
binding (load-stack-frame)

// f_vars = f()

f_vars = f ()

// eval "a", f_vars # => 22
// eval "b", f_vars # => 33
// eval "a = 101", f_vars
// eval "a", f_vars # => 101

print f_vars ("a") // => 22
print f_vars ("b") // => 33
print f_vars ("a := 101") // => 101
print f_vars ("a") // => 101
print f_vars ("zzzzz := 44") // => 44
print f_vars ("zzzzz") // => 44

g := 6
// newly *outside* defined variables appear in both bindings.
e ("say (g)")
d ("say (g)")
// so e() and d() actually contain references to the identical
binding.

// defining something in the child scope does *not* take effect in the
parent scope
d ("h := 7")
d ("say (h)")
// e ("say (h)") // fails! (h not defined in e())

// but it does take effect in the child scope when defined in a
capture of the parent scope.
e ("i := 8")
e ("say (i)")
d ("say (i)")

On Apr 2, 11:22 am, Matthew Wilson <diakop...@gmail.com> wrote:
> (transferring this thread to the mailing list)
>
> Yes, I agree with all of the below... the "diffs" I sent you below actually
> already do what you describe...
>

> On Thu, Apr 2, 2009 at 5:01 AM, Tim Macfarlane <timmacfarl...@gmail.com>wrote:
>
> > Dude this is hot!
> > I've added you as a member to the project so you'll be able to commit
> > commit away
>
> > I'm wondering if 'compile' should just be a method on string (like eval
> > too) and 'eval' should be the syntactic extension?
>
> > so:
>
> > eval "print 2"
>
> > evaluates immediately, and is the same as:
>
> > "print 2".r:eval (load-stack-frame)
>
> > Which is the same as:
>
> > "print 2".r:compile (load-stack-frame) ()
>
> > Also wonder if 'load-stack-frame' should be 'current-stack-frame'? load is
> > a verb, so would imply an action, like a function `load-stack-frame ()`.
> > current-stack-frame is better as a variable and could even have a namespace:
> > tycho:runtime:current-stack-frame, so as not to take up useful variable
> > names. (don't think you can do this with the current syntax language
> > stuff... which is all going to change soonish anyway...)
>
> > Also, we should be able to tell compile/eval which language to use too.
> > Eventually (not too far away) we should be able to define languages as
> > variables that we can use to load source files (as modules) and use in
> > eval/compile too. Just something to think about.
>
> > Anyway, awesome stuff! Great to have you on the, um, "team" ;)
>
> > Tim.
>

> > 2009/4/2 Matthew Wilson <diakop...@gmail.com>

>
> > found a couple corrections; discuss online sometime.. :)
>

[timmac...@gmail.com]

Oh yeah, and I guess you could rewrite `binding` as:

binding = method
() => binding (current-stack-frame)
stack-frame =>

source :: r:string =>
source.r:eval (stack-frame)

But then... if you were to use `binding` in any other function than
the one it was defined in it wouldn't capture the current stack frame,
so best to make a macro for `binding`:

macro binding > function-call "binding" "(" ")"
binding (current-stack-frame)

so when you write `binding ()` it's expanded to `binding (current-
stack-frame)`. Then you rewrite `binding` to be just:

binding = stack-frame =>
stack-frame =>
source.r:eval (stack-frame)

a = binding ()
a ('a ("print a")')