A proposal to simplify calling L# from C#
wanorris
CALLING L# FROM C#
Right now, it's much harder than it needs to be to call L# code
directly from C#. Since one obvious use of L# is as a scripting
language for .Net applications, it would be nice if it was really
simple. Of course, one good way to make this happen would be with an L#
compiler that produces a .Net assembly that can be called from any .Net
language. However, as useful as this would be, it still might not be
ideal for some application scripting scenarios. In some situations, it
may be more appropriate to use the interpreter (which already exists),
and simplify the interface to make it easier to call from C# and other
languages.
Suppose you have a lisp environment already running, and you want to
call a simple function you have defined. The easiest way to call it
right now is probably to create an L# string and evaluate it, like so:
object result = Runtime.EvalString("(my-function)",
new LSharp.Environment());
This isn't a very clean way to work with code, but it should work in
simple cases. In more elaborate cases, building correct strings gets to
be an increasingly elaborate problem. And really, even if it weren't
that hard, marshalling data to and from strings for interoperation
isn't an especially good practice.
The other way to call L# code from C# is to work with the actual L#
objects: symbols, functions, closures, and so on. Here is some simple
example code for calling the system function map and the user-defined
function (technically, a Closure object) concat from C#:
LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));
// build the arguments for map
ArrayList myarraylist = new ArrayList();
myarraylist.Add(env.GetValue(Symbol.FromName("double")));
Cons intCons = Cons.FromArray(new object[] { 1, 2, 3, 4, 5 });
myarraylist.Add(intCons);
// call map
Cons args = Cons.FromICollection(myarraylist).Reverse();
Function map = (Function)env.GetValue(Symbol.FromName("map"));
Console.WriteLine(map(args, env));
// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
args = Cons.FromArray(myarray);
Closure concat = (Closure)env.GetValue(Symbol.FromName("concat"));
Console.WriteLine(concat.Invoke(args));
The library that is being loaded, mylib.ls is simply:
; mylib.ls -- simple sample library
;
; Copyright (c) 2006, Andrew Norris
; Simple license: reuse this however you like.
(= double (fn (x) (* x 2)))
(= concat (fn (&rest items)
(let sb (new system.text.stringbuilder)
(foreach elem items
(call append sb elem))
(tostring sb ))))
When the above C# code executes, it will output:
(2 4 6 8 10)
foobar42baz
Using the L# interpreter objects directly is a better idiom in general,
because it doesn't require data structures to be marshalled as strings,
and because it allows for some things to be checked at compile-time.
However, it's obvious that this code is pretty cumbersome, if all you
want to do is call a couple of functions.
A SIDE NOTE: the ability to call a couple of trivial functions like
this generally isn't worth the trouble of embedding the L# interpreter
in your C# application. However, there are some things that may be much
easier to write in L#, such as list manipulation functions or code that
uses macros effectively.
1. ADD A STRING INDEXER TO ENVIRONMENT
It would be nice if it were easier to access symbol values in the L#
Environment. C# doesn't have symbols, so you have to create one from a
string. The GetValue call adds some an additional operation as well.
This seems like a good place to start in making it simpler to call L#.
The current method of dereferencing a symbol isn't that complex an
operation if it's called occasionally, but working with symbols in the
environment is one of the two most basic operations in L#, along with
calling a function. To be usable, it needs to be concise. A string
indexer makes this code much simpler:
LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));
// build the arguments for map
ArrayList myarraylist = new ArrayList();
myarraylist.Add(env["double"]);
Cons intCons = Cons.FromArray(new object[] { 1, 2, 3, 4, 5 });
myarraylist.Add(intCons);
// call map
Cons args = Cons.FromICollection(myarraylist).Reverse();
Function map = (Function)env["map"];
Console.WriteLine(map(args, env));
// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
args = Cons.FromArray(myarray);
Closure concat = (Closure)env["concat"];
Console.WriteLine(concat.Invoke(args));
As you can see, it doesn't affect the rest of the code, but it makes
recovering the functions from the environment much simpler.
Adding a basic string indexer to Environment is simple. The code looks
like this:
public object this[string s]
{
get { return GetValue(Symbol.FromName(s)); }
set { Assign(Symbol.FromName(s), value); }
}
Of course, this only gets things started. There are a lot of other
things we can improve as well.
2. SIMPLIFY THE INTERFACE FOR CONVERTING A .NET DATA STRUCTURE TO A
CONS
The interface for converting a collection of items to a Cons can
usefully be simplified. Cons contains some useful conversion functions,
and SpecialForms.The() does a good job of wiring them up. But, really,
anything that can be enumerated can be easily converted to a Cons, if
we add a new method. Cons.FromIEnumerable() is simple, and virtually
identical to Cons.FromICollection():
public static Cons FromIEnumerable(IEnumerable enumerable)
{
object list = null;
foreach (object o in enumerable)
{
list = new Cons(o, list);
}
return (Cons)list.Reverse();
}
This doesn't have a significant impact on the code right away. But it
will enable us to do some more important steps later on.
LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));
// build the arguments for map
ArrayList myarraylist = new ArrayList();
myarraylist.Add(env["double"]);
Cons intCons = Cons.FromIEnumerable(
new object[] { 1, 2, 3, 4, 5 });
myarraylist.Add(intCons);
// call map
Function map = (Function)env["map"];
Cons args = Cons.FromIEnumerable(myarraylist);
Console.WriteLine(map(args, env));
// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
args = Cons.FromIEnumerable(myarray);
Closure concat = (Closure)env["concat"];
Console.WriteLine(concat.Invoke(args));
It also enables you to convert any class that implements IEnumerable:
Cons cons = Cons.FromIEnumerable(someRandomThingWithAnEnumeration);
The advantage of this is that whenever something new comes along that
needs to be turned into a Cons, it can already be done if there is any
standard way to reference all the items.
3. MAKE IT EASY TO CONVERT NESTED .NET DATA STRUCTURES TO CONSES
After creating the code to consistently convert enumerable data
structures to Conses, one obvious problem is that it won't
automatically handle nested data structures. In the following code, the
array and the ArrayList have to be converted separately:
ArrayList myarraylist = new ArrayList();
myarraylist.Add(env["double"]);
Cons intCons = Cons.FromArray(new object[] { 1, 2, 3, 4, 5 });
myarraylist.Add(intCons);
Cons args = Cons.FromICollection(myarraylist).Reverse();
Since nested lists -- often, deeply nested ones -- are one of the most
basic building blocks of Lisp code, it would be nice if we could
convert the whole data structure at once. Forutunately, if we extend
the FromIEnumerable method we just built, there's a straightforward
solution:
public static Cons FromIEnumerable(IEnumerable enumerable,
bool isRecursive)
{
object list = null;
foreach (object o in enumerable)
{
if (isRecursive && o is IEnumerable) {
o = FromIEnumerable(o as IEnumerable, true);
}
list = new Cons(o, list);
}
return (Cons)list;
}
Now we can easily convert the data structure to a nice, Lispy nested
list in one step:
ArrayList myarraylist = new ArrayList();
myarraylist.Add(env["double"]);
int[] intArray = new object[] { 1, 2, 3, 4, 5 });
myarraylist.Add(intArray);
Cons args = Cons.FromIEnumerable(myarraylist, true);
This doesn't have a significant impact on our sample code here, but for
cases where there is data that is already in elaborate .Net data
structures and needs to be passed to L#, it will make things
significantly easier, and avoid the need to walk the data structure
tree.
4. SIMPLIFY BUILDING CONSES IN C#
Passing nested data to an L# function from C# is still fairly
cumbersome. While building arrays and ArrayLists is often more natural
for working with C# data, it's cumbersome for packaging up arguments,
as we saw in the last section when we built the arguments to pass to
map. Consider the code we've just been looking at:
ArrayList myarraylist = new ArrayList();
myarraylist.Add(env["double"]);
int[] intArray = new object[] { 1, 2, 3, 4, 5 });
myarraylist.Add(intArray);
Cons args = Cons.FromIEnumerable(myarraylist);
This could be done differently by building a Cons directly, of course:
Cons intCons = Cons.FromArray(new object[] { 1, 2, 3, 4, 5 });
Cons args = new Cons(intCons);
Cons args = new Cons(env["double"], args);
This is simpler, but still harder than it needs to be. Also, it either
involves performing the steps out of the usual order -- this is normal
in Lisp, but unusual in C# -- or adding a Reverse() operation each
time.
By contrast, in L#, the equivalent code is simply:
(map double '(1 2 3 4 5))
Obviously, if we could simplify the C# version, it would be a lot
easier to package arguments and call L# functions. For example, this
code would be much closer to ideal:
Cons args = Cons.Build(env["double"], Cons.Build(1, 2, 3, 4, 5));
That can be implemented relatively straightforwardly by creating a
Build method that can take any number of arguments and simply build a
list out of them:
public static Cons Build(params object[] items)
{
Object cons = null;
for (int i = items.Length - 1; i >= 0; i--)
{
cons = new Cons(items[i], cons);
}
return (Cons)cons;
}
This simplifies the example code to:
LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));
// call map
Cons args = Cons.Build(env["double"], Cons.Build(1, 2, 3, 4, 5));
Function map = (Function)env["map"];
Console.WriteLine((Cons)map(args, env));
// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
args = Cons.FromIEnumerable(myarray, true);
Closure concat = (Closure)env["concat"];
Console.WriteLine(concat.Invoke(args));
5. PROVIDE AN IDENTICAL INTERFACE FOR FUNCTIONS OR CLOSURES
Wouldn't it be nice if you could have one simple way to call an L#
operation, regardless of whether it was a system function, a
user-defined function (closure), or a special form? And wouldn't it be
nice if you could pass the operation whatever data structure you had
without converting it?
For example, here's what the code to call map and concat might look
like if you could do that:
Cons result = (Cons)map(args);
string sresult = (string)concat(myarray);
That's a lot simpler than the earlier example, right? Well, with
delegates, it's easy to produce that interface:
public delegate Object fn(IEnumerable arguments);
There's really only one problem with this delegate: it doesn't match
the signatures of Closure or Function, so you can't use it, at least
directly. Fortunately, there's a way around that problem:
6. BUILD AN IMPLEMENTATION CLASS THAT CAN MAKE FUNCTION CALLS SIMPLE
Although the interface to Closure and Function doesn't match the fn
delegate signature, it's straightforward to make an adapter class that
makes them match. Basically, the idea is to automatically translate
data structures to Conses, and automatically bind functions to the
current environment. The class then contains a method with the proper
signature for the fn delegate, and a property that will return it.
public class FunctionBinding
{
private Environment _env;
private Function _f;
private Closure _c;
private bool _isClosure;
public FunctionBinding(Environment env, Function f)
{
_f = f;
_env = env;
_c = null;
_isClosure = false;
}
public FunctionBinding(Closure c)
{
_f = null;
_env = null;
_c = c;
_isClosure = true;
}
private Object Impl(IEnumerable arguments)
{
// special handling for closures that take no
// arguments and functions with empty lists
if (arguments == null)
{
if (_isClosure)
return _c.Invoke();
else
return _f(new Cons(null), _env);
}
// build a cons out of the argument
Cons cons = null;
if (arguments is Cons)
cons = arguments as Cons;
else
cons = Cons.FromIEnumerable(arguments, true);
// invoke the closure or function
if (_isClosure)
return _c.Invoke(cons);
else
return _f(cons, _env);
}
public fn BoundFunction
{
get { return Impl; }
}
public static fn Bind(Environment env, Function f)
{
FunctionBinding fb = new FunctionBinding(env, f);
return fb.BoundFunction;
}
}
This makes it really simple to call L# functions, but complicates
retrieving them. The problem is that we have to build a FunctionBinding
object around the return value we get from the environment. At this
point, our code would work like this:
LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));
// call map
Cons args = Cons.Build(env["double"], Cons.Build(1, 2, 3, 4, 5));
fn map = (new FunctionBinding(env, (Function)env["map"]))
.BoundFunction;
Console.WriteLine(map(args));
// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
fn concat = (new FunctionBinding((Closure)env["concat"]))
.BoundFunction;
Console.WriteLine(concat(myarray));
7. WIRE THINGS UP SO FUNCTIONBINDINGS ARE CREATED AUTOMAGICALLY
Fortunately, we can simplify things. By modifying the string indexer we
built earlier, we can automatically bind functions and closures, and
return the easy-to-call bound functions.
public object this[string s]
{
get
{
object o = GetValue(Symbol.FromName(s));
if (o is Function)
o = (new FunctionBinding(this, (Function)o))
.BoundFunction;
else if (o is Closure)
o = (new FunctionBinding((Closure)o)).BoundFunction;
return o;
}
set { Assign(Symbol.FromName(s), value); }
}
This will also convert our call to the double function as well, which
means the interpreter needs to be able to handle fn delegates:
public static object Apply (object function, object arguments,
Environment environment)
{
if (function.GetType() == typeof(Function))
{
return ((Function) function) ((Cons)arguments,environment);
}
// If function is an LSharp Closure, then invoke it
if (function.GetType() == typeof(Closure))
{
if (arguments == null)
return ((Closure)function).Invoke();
else
return ((Closure)function).Invoke((Cons)arguments);
}
if (function is fn)
{
return ((fn)function)((IEnumerable)arguments);
}
else
{
// It must be a .net method call
return Call(function.ToString(),(Cons)arguments);
}
}
After this final change, our code simplifies to this:
LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));
// call map
Cons args = Cons.Build(env["double"], Cons.Build(1, 2, 3, 4, 5));
fn map = (fn)env["map"];
Console.WriteLine(map(args));
// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
fn concat = (fn)env["concat"];
Console.WriteLine(concat(myarray));
And if you want, you can simplify it even further, though it makes the
code a bit denser:
LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));
// call map
Cons args = Cons.Build(env["double"], Cons.Build(1, 2, 3, 4, 5));
Console.WriteLine(((fn)env["map"])(args));
// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
Console.WriteLine(((fn)env["concat"])(myarray));
CONCLUSION
Implementing these changes should make it much easier to call L# code
from inside C#, and without resorting to building evaluation strings.
Hopefully, this will make it more practical to take sections of code
that can be implemented much more easily in L# than in C# and build a
multiple-language implementation. For example, in a program that has a
section that requires elabortate list manipulation, but other sections
that need to be in C#.
With these changes, we were able to simplify our sample code from
LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));
// build the arguments for map
ArrayList myarraylist = new ArrayList();
myarraylist.Add(env.GetValue(Symbol.FromName("double")));
Cons intCons = Cons.FromArray(new object[] { 1, 2, 3, 4, 5 });
myarraylist.Add(intCons);
// call map
Function map = (Function)env.GetValue(Symbol.FromName("map"));
Cons args = Cons.FromICollection(myarraylist).Reverse();
Console.WriteLine(map(args, env));
// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
args = Cons.FromArray(myarray);
Closure concat = (Closure)env.GetValue(Symbol.FromName("concat"));
Console.WriteLine(concat.Invoke(args));
to
LSharp.Environment env = new LSharp.Environment();
Console.WriteLine(Functions.Load(new Cons("mylib.ls"), env));
// call map
Cons args = Cons.Build(env["double"], Cons.Build(1, 2, 3, 4, 5));
Console.WriteLine(((fn)env["map"])(args));
// call concat
object[] myarray = { "foo", "bar", 42, "baz" };
Console.WriteLine(((fn)env["concat"])(myarray));
Of course, to do the equivalent in L#, the code is
(load "mylib.ls")
(prl (map double '(1 2 3 4 5)))
(prl (concat "foo" "bar" 42 "baz"))
So L# functions still aren't nearly as easy to work with from C# as
they are natively. Even keeping this in mind, these changes should make
it a lot more practical to embed direct calls to L# code in C#.
The main thing this proposal hasn't touched on, of course, is macros.
If you have macros embedded in L# functions, this won't make any
difference -- the macros get expanded at read time, and your function
can be called from C# just like any other function. But what if you
want to call a macro directly? Or what if you want to embed a macro
call inside the arguments to the function you call?
The techniques discussed in this article won't handle macros correctly,
because by the time a function is being called, L# assumes all macros
have been expanded. In a future post, I'll discuss how to use similar
C# techniques with macros.
Finally, in the future, adding compiled code will be an important way
to make L# code more accessible from C#. That will allow L# libraries
to be attached at compile time and statically bound, rather than loaded
and called at runtime. While that will be a powerful technique for some
situations, the ability to easily call interpreted L# at runtime will
still have its place in many other scenarios.