How to (parse and) evaluate some code which is provided as a string

[Theodore Papamarkou]

I'm trying to do sth that I'm not sure it's doable in Julia (and if it is, I am not sure it is good Julia programming practice). Is it possible to parse and evaluate some code from an arbitrary string? For example, if I have the string "atype"*".args[2]"^2, is it possible somehow to evaluate the corresponding code atype.args[2].args[2] in order to access the nested field of the atype array? Sth along the lines @eval symbol("atype"*".args[2]"^2) or @eval $symbol("atype"*".args[2]"^2) is apparently wrong coding.

As side-questions (sorry to trouble you folks with several questions, just getting to understand how to do metaprogramming in Julia):

* is there a Julia function that takes an object as argument and returns a string with the name of the object? As a continuation of the above example, I am looking for a "name" function along the lines name(atype) so as to get back the string "atype".

* I was looking at the function _jl_pre_exec in base/process.jl as a means of understanding how Julia pointers work and noticed that C_NULL is the last element of ptrs... what does C_NULL do - why is it needed?

[Stefan Karpinski]

On Wed, May 8, 2013 at 11:15 AM, Theodore Papamarkou <theodore....@gmail.com> wrote:

I'm trying to do sth that I'm not sure it's doable in Julia (and if it is, I am not sure it is good Julia programming practice). Is it possible to parse and evaluate some code from an arbitrary string? For example, if I have the string "atype"*".args[2]"^2, is it possible somehow to evaluate the corresponding code atype.args[2].args[2] in order to access the nested field of the atype array? Sth along the lines @eval symbol("atype"*".args[2]"^2) or @eval $symbol("atype"*".args[2]"^2) is apparently wrong coding.

Yes, but unlike the "scripting languages" where the only way to generate and eval code is to build a string and then parse+evaluate it, Julia has a first-class representation of expressions, which are the preferred means of doing metaprogramming. You can, however, explicitly call the parse function to accomplish what you're talking about:

julia> ex = parse("z = 2x + y^2 + 1")

:(z = +(*(2,x),^(y,2),1))

Of course, that just gives you a expression object and doesn't evaluate it. You need to call eval for that:

julia> x,y = 2,3;

julia> eval(ex)

14

So the equivalent of eval(str) in Python (or Perl or Ruby) is eval(parse(str)) in Julia.

<aside>

When you write something like symbol("atype"*".args[2]"^2) you are creating a symbol with the very odd name "atype.args[2].args[2]", which is unlikely to be bound to anything when you evaluate it, although you could arrange for that to happen like this:

julia> eval(:($(symbol("atype"*".args[2]"^2)) = 1))

1

julia> eval(symbol("atype"*".args[2]"^2))

1

</aside>

In general, you're much better off manipulating Expr objects and Symbols than trying to build up strings for code and then parsing and evaluating them. It's much efficient, much more reliable, and you can do quite powerful, general transformations on expressions.

 

As side-questions (sorry to trouble you folks with several questions, just getting to understand how to do metaprogramming in Julia):

No problem.

 

* is there a Julia function that takes an object as argument and returns a string with the name of the object? As a continuation of the above example, I am looking for a "name" function along the lines name(atype) so as to get back the string "atype".

Objects don't have names, they're just values. Symbols have names and are bound to values, but many different names can be bound to the same value. You can sensibly say string(:atype) and you will get "atype" back, but you're asking for the string representation of the symbol :atype, not for the name of the value that atype refers to (it has no name).

<aside>

Then only exception to this is named functions, which do actually have a name:

julia> f() = 1

# methods for generic function f

f() at none:1

julia> g = f

# methods for generic function f

f() at none:1

The function object still knows its original name was "f".

</aside>

 

* I was looking at the function _jl_pre_exec in base/process.jl as a means of understanding how Julia pointers work and noticed that C_NULL is the last element of ptrs... what does C_NULL do - why is it needed?

The execvp function expects its second argument to be a NULL-terminated array of pointers to strings (which are themselves NULL-terminated arrays of bytes). The C_NULL has to be there so that execvp knows when it's done with the arguments to the program its going to exec.

[Theodore Papamarkou]

Thank you very much Stefan, this is really helpful. I understood your reply and educative comments, which by the way helped me get on with what I was doing. I wanted to access nested elements of an the args field of an expression as part of the tree traversal I have been working on, and your eval(parse()) suggestion resolved my problems:

julia> tmp01 = :(x+3y)

:(+(x,*(3,y)))

julia> eval(parse("tmp01.args[3].args[3]"))

:y

I wasn't aware of the powerful parse() function. Thanks a lot!

[Theodore Papamarkou]

P.S. I also noted your comment on avoiding building up strings and then parsing them, I will alter my approach so as to work with expressions directly. Cheers!

[Stefan Karpinski]

Excellent. Have fun! First-class metaprogramming is addictive.

[Edward Chen]

To follow up on this question, if I am using this eval(parse(foo)) formalism within a wrapper function, it seems that if I define the input arguments for foo (a first-class function object) within this wrapper function that the arguments can't be found. 

example:

function foo(arg)

return arg+3

end

function wrapper(bar)

arg = 3

output = eval(parse("foo"))

end

then I get an error saying that the "arg" is not defined when I run:

julia>> wrapper("foo")

Much appreciated,

Ed

[Edward Chen]

nvm, I just needed to define the variables to have a global scope

[Jameson Nash]

That will work, but it is often an indication that you are working down the wrong path here, and will keep finding little issues with `eval` that you have to alter your code handle (like that globals are slow and have to deal with choosing the right module scope).

For example, in your case here, simply passing `wrapper(foo)` (instead of putting foo in quotes)` would remove the need to call parse-eval.

[Edward Chen]

Hrm, okay you have my attention -- I do care about speed in this case. I am just trying to write a wrapper which tests different functions, such as foo and bar, in the psuedo code below.

What I'm trying to do is as follows:

function foo(arg1,arg2)

end

function bar(arg1)

end

function wrapper(input)

def arg1, arg2

if input == "bar"

eval(bar(arg1))

elseif input == "foo"

eval(foo(arg1,arg2))

end

end

[Jameson Nash]

i don't know how args is formed, so I can't give specifics, but either `wrapper(fun, args) = fun(args...)` or `wrapper(fun, nargs) = (local args; fun(args[1:nargs]...))` may be what you are looking for?