eval/binding question
Stefan Kaes
parameter, but I can't get it to work. The following test program shows
what is happening:
def test1
eval "x=25"
eval 'print "x=#{x}\n"'
end
test1
def test2
eval "x=25"
print "x=#{x}\n"
end
test2
produces:
x=25
test.rb:189:in `test2': undefined local variable or method `x' for
main:Object ( NameError)
from test.rb:192
It seems that changes to local variables are not reflected back to the
executing environment.
Does anyone out there know how to solve this problem?
-- stefan
Robert Klemme
"Stefan Kaes" <sk...@gmx.net> schrieb im Newsbeitrag
news:4236E882...@gmx.net...
You can't. Local variables are determined during parse phase and you
cannot access local variables that you create via eval other than via
eval. Consider
16:11:59 [robert.klemme]: ruby -e 'eval "x=10";puts x'
-e:1: undefined local variable or method `x' for main:Object (NameError)
16:13:51 [robert.klemme]: ruby -e 'x=nil;eval "x=10";puts x'
10
16:13:57 [robert.klemme]: ruby -e 'eval "x=10";eval "puts x"'
10
IOW, you can manipulate predefined locals but not those that spring into
existence during an eval.
Kind regards
robert
Yukihiro Matsumoto
In message "Re: eval/binding question"
on Tue, 15 Mar 2005 23:02:44 +0900, Stefan Kaes <sk...@gmx.net> writes:
|I tried to create local variables from a name=>value hash passed as a
|parameter, but I can't get it to work. The following test program shows
|what is happening:
local variables should be determined at compile time, thus local
variables defined first in the eval'ed string, can only be accessed from
other eval'ed strings. In addition, they will be more ephemeral in
Ruby2, so that these variables will not be accessed from outside.
In summary, I recommend you not to use local variables for your
purpose. They are wrong tool for it.
matz.
Stefan Kaes
>Hi,
>
>In message "Re: eval/binding question"
> on Tue, 15 Mar 2005 23:02:44 +0900, Stefan Kaes <sk...@gmx.net> writes:
>
>|I tried to create local variables from a name=>value hash passed as a
>|parameter, but I can't get it to work. The following test program shows
>|what is happening:
>
>local variables should be determined at compile time, thus local
>variables defined first in the eval'ed string, can only be accessed from
>other eval'ed strings. In addition, they will be more ephemeral in
>Ruby2, so that these variables will not be accessed from outside.
>
>
Well, I don't agree. From a language designers point of view x=5 and
eval "x=5" should do the same thing: modify the current binding by
introducing a new value-binding with name x and value 5. I don't know of
any language which behaves differently (e.g. LISP works like this
AFAIK). Of course, as a compiler writer, one might prefer to be able to
determine all local variables by looking at the source. But this is just
a wish to make the compiler simpler or enable better optimization.
>In summary, I recommend you not to use local variables for your
>purpose. They are wrong tool for it.
>
> matz.
>
Maybe I should explain why I tried this:
when using ERB one can compile a template into source, which can then be
eval'ed using eval. Local variables for use in the template can be set
up using the sort of eval given in my example. However, in this case the
'compiled' template code gets reparsed on each evaluation of the
template code. I wanted to speed up this process by defining a function
(eval "def fun; #{src}; end", aBinding), thereby pasring the code only
once. This works pretty well and gives about 25% increase in speed. The
performance gain will be much bigger, once ruby gets a real JIT. But, as
it turned out, local variables are not accessible when the defined
function is executed.
I saw two solutions: First, change the local variables to instance
variables on the class executing the function. This works, but does not
feel right (name space pollution). Second, put all locals in an instance
hash and set up local functions inside the defined function from the
hash. Which does not work.
Any other ideas?
Yukihiro Matsumoto
In message "Re: eval/binding question"
on Wed, 16 Mar 2005 01:17:55 +0900, Stefan Kaes <sk...@gmx.net> writes:
|>local variables should be determined at compile time, thus local
|>variables defined first in the eval'ed string, can only be accessed from
|>other eval'ed strings. In addition, they will be more ephemeral in
|>Ruby2, so that these variables will not be accessed from outside.
|
|Well, I don't agree. From a language designers point of view x=5 and
|eval "x=5" should do the same thing: modify the current binding by
|introducing a new value-binding with name x and value 5. I don't know of
|any language which behaves differently (e.g. LISP works like this
|AFAIK). Of course, as a compiler writer, one might prefer to be able to
|determine all local variables by looking at the source. But this is just
|a wish to make the compiler simpler or enable better optimization.
You don't have to agree here. Each language designer have different
point of view. I'm not going to change my mind by your "should".
|when using ERB one can compile a template into source, which can then be
|eval'ed using eval. Local variables for use in the template can be set
|up using the sort of eval given in my example. However, in this case the
|'compiled' template code gets reparsed on each evaluation of the
|template code. I wanted to speed up this process by defining a function
|(eval "def fun; #{src}; end", aBinding), thereby pasring the code only
|once. This works pretty well and gives about 25% increase in speed. The
|performance gain will be much bigger, once ruby gets a real JIT. But, as
|it turned out, local variables are not accessible when the defined
|function is executed.
I'm not sure I'm fully understand you. Can you show us your code?
matz.
Stefan Kaes
>I'm not sure I'm fully understand you. Can you show us your code?
>
>
Yes, but I need to clean up first.
-- stefan
George Moschovitis
>
> when using ERB one can compile a template into source, which can then
> be eval'ed using eval. Local variables for use in the template can be
set
> hash and set up local functions inside the defined function from the
Well, that's funny :)
I just finished working on a VERY similar bit of code.
FYI, I came up with a sligthly altered version of your first solution.
In my case, name space polution is not a problem.
The code will be available soon as part of Nitro's new Mailer
subsystem.
regards,
-g.
ps:
Stefan Kaes
>You don't have to agree here. Each language designer have different
>point of view. I'm not going to change my mind by your "should".
>
Let me rephrase: the semantics of eval should be defined in a way that
holds no surprises. I would expect that writing
...
eval "code"
...
and
...
code
...
should be identical.
-- stefan
Yukihiro Matsumoto
In message "Re: eval/binding question"
on Wed, 16 Mar 2005 03:29:53 +0900, Stefan Kaes <sk...@gmx.net> writes:
|Let me rephrase: the semantics of eval should be defined in a way that
|holds no surprises.
I know what you mean. But it's not good strategy to persuade me to
use "your expectation" or "your surprise". This "limitation" has a
lot of good aspects, such as better performance, better error
detection, etc. I'd love to pay the cost of small restriction for
these benefits as a language designer.
OK, I think this is enough. Let us focus on solving "your problem".
matz.
Yukihiro Matsumoto
In message "Re: eval/binding question"
on Wed, 16 Mar 2005 02:45:41 +0900, Stefan Kaes <sk...@gmx.net> writes:
|In the real environment, the names of the locally assigned variables can
|not be determined in advance, of course.
I'm sorry to say you can't pre-compile eRB allowing local variable set
not determined in advance. I know allowing this will make your
program 25% faster, but this also would make any other programs
slower.
If you have limited set of local variables, you can do something like:
require 'erb'
class Test
def initialize(path, *lv)
@erb = ERB.new(File.read(path), nil, '%')
@locals = {}
evalstr = "def run_erb\n" +
lv.collect{|k| "#{k} = @locals[\"#{k}\"]\n"}.join +
"#{@erb.src}\n_erbout\nend\n"
print evalstr, "\n\n"
eval evalstr, nil, path, 0
end
def values(hash)
@locals.update(hash)
hash.each {|k,v|
self.instance_variable_set("@#{k}", v)
}
end
end
t = Test.new("/tmp/test.rhtml", 'mumu')
t.values("mumu" => "HELLO")
print t.run_erb
matz.
Eric Hodel
> when using ERB one can compile a template into source, which can then
> be eval'ed using eval. Local variables for use in the template can be
> set up using the sort of eval given in my example. However, in this
> case the 'compiled' template code gets reparsed on each evaluation of
> the template code. I wanted to speed up this process by defining a
> function (eval "def fun; #{src}; end", aBinding), thereby pasring the
> code only once.
ERb already does this for you. See ERB::Defmethod. Unfortunately,
there are no docs for it, but you can dig around in Rails to see how it
is used.
--
Eric Hodel - drb...@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04
Eric Hodel
> On 15 Mar 2005, at 08:17, Stefan Kaes wrote:
>
>> when using ERB one can compile a template into source, which can then
>> be eval'ed using eval. Local variables for use in the template can be
>> set up using the sort of eval given in my example. However, in this
>> case the 'compiled' template code gets reparsed on each evaluation of
>> the template code. I wanted to speed up this process by defining a
>> function (eval "def fun; #{src}; end", aBinding), thereby pasring the
>> code only once.
>
> ERb already does this for you. See ERB::Defmethod. Unfortunately,
> there are no docs for it, but you can dig around in Rails to see how
> it is used.
Ok, I lied. ERB::DefMethod is both not used by Rails (does something
else) but it is also unsuitable for the task.
Anyhow, as my penance, here's my implementation, I did use def_method,
though:
http://segment7.net/projects/ruby/snippets/erb_cache.rb
Stefan Kaes
> On 15 Mar 2005, at 08:17, Stefan Kaes wrote:
>
>> when using ERB one can compile a template into source, which can then
>> be eval'ed using eval. Local variables for use in the template can be
>> set up using the sort of eval given in my example. However, in this
>> case the 'compiled' template code gets reparsed on each evaluation of
>> the template code. I wanted to speed up this process by defining a
>> function (eval "def fun; #{src}; end", aBinding), thereby pasring the
>> code only once.
>
>
> ERb already does this for you. See ERB::Defmethod. Unfortunately,
> there are no docs for it, but you can dig around in Rails to see how
> it is used.
>
That's not true. ERB::Defmethod does not work this way. And Rails uses
the less efficient method, which I wanted to improve on (and did). And I
wanted the local varaiable solution mainly for backwards compatibility
with rendering partials in Rails. But the question about eval/binding
semantics was posted to this list because I thought (and still think)
that the semantics of eval is not what it should be.
Stefan Kaes
> Ok, I lied. ERB::DefMethod is both not used by Rails (does something
> else) but it is also unsuitable for the task.
>
Why did you lie???
> Anyhow, as my penance, here's my implementation, I did use def_method,
> though:
>
> http://segment7.net/projects/ruby/snippets/erb_cache.rb
>
Thanks for the code, but I already knew this would also be a possible
solution. But probably less efficient than using instance variables,
though I have not checked that yet. Did you?
Stefan Kaes
>Hi,
>
>In message "Re: eval/binding question"
> on Wed, 16 Mar 2005 02:45:41 +0900, Stefan Kaes <sk...@gmx.net> writes:
>
>|In the real environment, the names of the locally assigned variables can
>|not be determined in advance, of course.
>
>I'm sorry to say you can't pre-compile eRB allowing local variable set
>not determined in advance. I know allowing this will make your
>program 25% faster, but this also would make any other programs
>slower.
>
>
I fail to see why it would make other code slower. I am convinced with a
proper compiler implementation there would'nt be any speed difference
for local variables introduced via explicit assignment. Access to
predeclared variables would use an optimized access method, whereas
access to variable names whose declaration point cannot be determined by
statical program analysis would use a less efficient access method.
>If you have limited set of local variables, you can do something like:
>
> require 'erb'
>
> class Test
> def initialize(path, *lv)
> @erb = ERB.new(File.read(path), nil, '%')
> @locals = {}
> evalstr = "def run_erb\n" +
> lv.collect{|k| "#{k} = @locals[\"#{k}\"]\n"}.join +
> "#{@erb.src}\n_erbout\nend\n"
> print evalstr, "\n\n"
> eval evalstr, nil, path, 0
> end
> def values(hash)
> @locals.update(hash)
> hash.each {|k,v|
> self.instance_variable_set("@#{k}", v)
> }
> end
> end
>
> t = Test.new("/tmp/test.rhtml", 'mumu')
> t.values("mumu" => "HELLO")
> print t.run_erb
>
> matz.
>
>
That's simlar to the way I "solved" "my problem" already. Thanks.
-- stefan
Robert Klemme
> Yukihiro Matsumoto wrote:
>
> >Hi,
> >
> >In message "Re: eval/binding question"
> > on Wed, 16 Mar 2005 02:45:41 +0900, Stefan Kaes <sk...@gmx.net>
writes:
> >
> >|In the real environment, the names of the locally assigned variables
can
> >|not be determined in advance, of course.
> >
> >I'm sorry to say you can't pre-compile eRB allowing local variable set
> >not determined in advance. I know allowing this will make your
> >program 25% faster, but this also would make any other programs
> >slower.
> >
> >
> I fail to see why it would make other code slower. I am convinced with a
> proper compiler implementation there would'nt be any speed difference
> for local variables introduced via explicit assignment. Access to
> predeclared variables would use an optimized access method, whereas
> access to variable names whose declaration point cannot be determined by
> statical program analysis would use a less efficient access method.
I wonder in which cases this would be useful. This code is artificial,
since one would never write test2 like you did:
def test2
eval "x=25"
print "x=#{x}\n"
end
If you write something similar but with a string coming from somewhere
else
def test2(s)
eval s
print "x=#{x}\n"
end
you know already that "s" must contain an assignment to "x". So it's not
a problem to predeclare it IMHO.
I don't really see which real world problems would be solved with a more
complex lookup for locals, which would make the second variant succeed
without the explicit declaration of "x".
Kind regards
robert
Stefan Kaes
This code is artificial because I only wrote it to demonstrate the
semantics of the current implementation.
>If you write something similar but with a string coming from somewhere
>else
>
> def test2(s)
> eval s
> print "x=#{x}\n"
> end
>
>you know already that "s" must contain an assignment to "x". So it's not
>a problem to predeclare it IMHO.
>
>
For this simple demonstration program you are absolutely right: there is
no reason to write it that way, other than to demonstrate current semantics.
>I don't really see which real world problems would be solved with a more
>complex lookup for locals, which would make the second variant succeed
>without the explicit declaration of "x".
>
>Kind regards
>
> robert
>
The current semantics introduces an artificial barrier between code
executed in *eval "code"* vs. just writing *code*, which makes the
semantics more difficult to explain and understand. Ideally, these two
forms should have identical semantics. As I have already explained, the
complexity of variable access increases only for variables whose
declaration point cannot be determined statically and therfore would not
have a negative performance impact on most programs.
The real world example can be found in Rails. It has a function
render_partial, which takes an erb template name (tn), a value (v) and a
name=>value hash (local_assigns). It makes the value v accessible as a
local variable named "tn" and all name=>value pairs in local_assigns
accessible by their name. This works, because the whole template source
is evaluated using an eval, but requires reparsing of the erb-code each
time the template gets interpreted. And, of course, the code that
implements this function cannot know the names of the local varaiables
in advance.
As soon as one tries to abstract the erb code into a function the whole
scheme breaks down, because there is no way to dynamically declare local
variables. I solved this problem by resorting to instance variables.
That is okay for me, I just rewrote my template code, but it breaks
backwards compatibility. So this valuable optimisation cannot be
included in Rails, unless everyone rewrites their templates. For this
reason, I have not proposed it to the Rails community.
regards, stefan
Robert Klemme
> The current semantics introduces an artificial barrier between code
> executed in *eval "code"* vs. just writing *code*, which makes the
> semantics more difficult to explain and understand. Ideally, these two
> forms should have identical semantics. As I have already explained, the
> complexity of variable access increases only for variables whose
> declaration point cannot be determined statically and therfore would not
> have a negative performance impact on most programs.
>
> The real world example can be found in Rails. It has a function
> render_partial, which takes an erb template name (tn), a value (v) and a
> name=>value hash (local_assigns). It makes the value v accessible as a
> local variable named "tn" and all name=>value pairs in local_assigns
> accessible by their name. This works, because the whole template source
> is evaluated using an eval, but requires reparsing of the erb-code each
> time the template gets interpreted. And, of course, the code that
> implements this function cannot know the names of the local varaiables
> in advance.
>
> As soon as one tries to abstract the erb code into a function the whole
> scheme breaks down, because there is no way to dynamically declare local
> variables. I solved this problem by resorting to instance variables.
> That is okay for me, I just rewrote my template code, but it breaks
> backwards compatibility. So this valuable optimisation cannot be
> included in Rails, unless everyone rewrites their templates. For this
> reason, I have not proposed it to the Rails community.
Hm, I'm not 100% sure I modeled this properly, but does this solve the
problem?
I omitted the template name and the value as these are just special cases
of values with fixed names. But you get the picture. And there is no
reparsing needed.
def local_eval(_values, _code)
_val = nil
_bnd = binding
p local_variables
_values.each {|name, val| eval "#{name}=_val", _bnd}
p local_variables
eval _code, _bnd
end
>> local_eval( {:x=>1}, "puts x" )
["_values", "_code", "_val", "_bnd"]
["_values", "_code", "_val", "_bnd", "x"]
nil
=> nil
Of course you get some more local variables than you would like to but
with proper naming that should not be a problem in practice.
Kind regards
robert
Stefan Kaes
>Hm, I'm not 100% sure I modeled this properly, but does this solve the
>problem?
>
>
No. _code gets parsed on every call to local_eval.
>I omitted the template name and the value as these are just special cases
>of values with fixed names. But you get the picture. And there is no
>reparsing needed.
>
>def local_eval(_values, _code)
> _val = nil
> _bnd = binding
> p local_variables
> _values.each {|name, val| eval "#{name}=_val", _bnd}
> p local_variables
> eval _code, _bnd
>end
>
>
>
>>>local_eval( {:x=>1}, "puts x" )
>>>
>>>
>["_values", "_code", "_val", "_bnd"]
>["_values", "_code", "_val", "_bnd", "x"]
>nil
>=> nil
>
>
-- stefan
Robert Klemme
> Robert Klemme wrote:
>
> >Hm, I'm not 100% sure I modeled this properly, but does this solve the
> >problem?
> >
> >
> No. _code gets parsed on every call to local_eval.
If you want to avoid that you'll need a two step approach. But then the
var names are needed beforehand... Ok, I see the problem (finally) :-)
Cheers
robert
Eric Hodel
> Eric Hodel wrote:
>
>> Ok, I lied. ERB::DefMethod is both not used by Rails (does something
>> else) but it is also unsuitable for the task.
>>
> Why did you lie???
I didn't bother to double-check first. My bad.
>> Anyhow, as my penance, here's my implementation, I did use
>> def_method, though:
>>
>> http://segment7.net/projects/ruby/snippets/erb_cache.rb
>>
> Thanks for the code, but I already knew this would also be a possible
> solution. But probably less efficient than using instance variables,
> though I have not checked that yet. Did you?
attr* methods are optimized for speed so you don't have all the
overhead of a regular method call, but I doubt this will be faster than
ivars.
An ivar implementation would be pretty easy too, use
instance_variable_set instead of send.
Here's how Ruby optimizes attr* methods:
$ parse_tree_show
class Example
attr_reader :x
def y() @y; end
end
[[:class,
:Example,
:Object,
[:defn, :x, [:ivar, :@x]],
[:defn, :y, [:scope, [:block, [:args], [:ivar, :@y]]]]]]
No scope setup is done for attr*.
samuel_x_...@hotmail.com
I have on a couple of occasions also been bit by this. It would be much
nicer (to the programmer, at any rate) for
code and eval "code"
to behave consistently with each other.
In particular, manipulating variable bindings is extremely difficult
without.
So I'd like to revive this request, if possible ...