Accessing top-level globals() from Python

[Jeroen Demeyer]

Hello,

I noticed that globals() does something different in Python and Cython.
In Cython, it allows to access the "top-level" globals, e.g. the globals
on the IPython command line. This is the reason that

sage: _ = var('y')
sage: y
y

works. However, this only works because the var() function is
implemented in Cython! Python's globals() are the globals of the module.
I am almost certain this is the source of #12446.

So the question is: is it possible to access the top-level globals from
Python? Implementing stuff in Cython just because globals() behaves
differently looks very fragile to me.

Jeroen.

[Volker Braun]

Cython doesn't really have globals dictionary. http://cython.readthedocs.org/en/latest/src/userguide/limitations.html says:

"The globals() builtin returns the last Python callers globals, not the current function’s locals. This behavior should not be relied upon, as it will probably change in the future."

So its somewhat by accident that this ends up injecting into the repl globals. Though at this point I don't really expect it to change in the future...

Plain IPython version:

sage: ip.ns_table['user_global']['foo'] = 'bar'

sage: foo

'bar'

sage: ip.ns_table['user_global'] is globals()

True

[Volker Braun]

On Wednesday, March 25, 2015 at 7:40:31 PM UTC+1, Volker Braun wrote:

sage: ip.ns_table['user_global']['foo'] = 'bar'

sage: foo

'bar'

sage: ip.ns_table['user_global'] is globals()

True

Forgot the "ip = get_ipython()" 

[Nils Bruin]

On Wednesday, March 25, 2015 at 11:40:31 AM UTC-7, Volker Braun wrote:

Cython doesn't really have globals dictionary. http://cython.readthedocs.org/en/latest/src/userguide/limitations.html says:

"The globals() builtin returns the last Python callers globals, not the current function’s locals. This behavior should not be relied upon, as it will probably change in the future."

So its somewhat by accident that this ends up injecting into the repl globals. Though at this point I don't really expect it to change in the future...

It has already changed! We must be setting some cython flag to turn this off for the compilation of var.pyx, but if I try:

cython("""
def gl():
    G = globals();
    G['uu'] = 200
"""); gl()

I will not get "uu" injected. If I change the relevant line to `G = __builtins__.globals()` then it does work.

The ipython approach is nice, but doesn't work in the notebook. A much mode robust solution on our end would be to implement a user_globals() that looks up a user global dictionary, adapting to ipython or notebook as necessary. Are there situations where there isn't a "top level globals" dictionary? in those cases it should raise an error (I guess to begin with: when you're not ostensibly in Ipython or the notebook)

[Volker Braun]

On Wednesday, March 25, 2015 at 9:34:24 PM UTC+1, Nils Bruin wrote:

Are there situations where there isn't a "top level globals" dictionary? 

I'd say that Python has no "top level globals" or truly global variables. Each module has its own module globals. A repl implementation may or may not use a module global dict as its user (commandline) namespace. 

[Jeroen Demeyer]

On 2015-03-25 19:40, Volker Braun wrote:
> Cython doesn't really have globals
> dictionary. http://cython.readthedocs.org/en/latest/src/userguide/limitations.html
> says:
>
> "The globals() builtin returns the last Python callers globals, not the
> current function’s locals. This behavior should not be relied upon, as
> it will probably change in the future."

This is about Cython 0.15 and the analogous document for Cython 0.22
doesn't have anything anymore about globals(). Which makes me guess that
the fact that var() works is a bug and not a feature.

[Volker Braun]

True, maybe I trusted the "latest" in the url too much ;-)

In src/setup.py we set Cython.Compiler.Options.old_style_globals = True, which causes Cython to fall back to the old behavior.

possibly related: http://trac.sagemath.org/ticket/12446

I think the right place for the global variable injection would be the frontend-specific rich representation backends that I recently added in sage.repl.rich_output. Its dependent on the repl implementation after all. Then we could also do away with the Cython hack.

[Ralf Stephan]

Isn't it high time for

http://trac.sagemath.org/ticket/17958

then?

[Jeroen Demeyer]

On 2015-03-26 00:17, Volker Braun wrote:
> True, maybe I trusted the "latest" in the url too much ;-)
>
> In src/setup.py we set Cython.Compiler.Options.old_style_globals = True,
> which causes Cython to fall back to the old behavior.

Yes, that should explain it. Strangely, this "old_style_globals" isn't
documented anywhere (the only Google hits are in source code).

[Jeroen Demeyer]

On 2015-03-25 21:34, Nils Bruin wrote:
> A much
> mode robust solution on our end would be to implement a user_globals()
> that looks up a user global dictionary, adapting to ipython or notebook
> as necessary.

I agree. The question is: how would this work for sagenb?

[Jeroen Demeyer]

Hello,

I noticed that globals() does something different in Python and Cython.
In Cython, it allows to access the "top-level" globals, e.g. the globals
on the IPython command line. This is the reason that

sage: _ = var('y')
sage: y
y

works. However, this only works because the var() function is
implemented in Cython! Python's globals() are the globals of the module.
I am almost certain this is the source of #12446.

So the question is: is it possible to access the top-level globals from
Python? Implementing stuff in Cython just because globals() behaves

differently seems too fragile.

Jeroen.

[Jeroen Demeyer]

[Nils Bruin]

On Wednesday, March 25, 2015 at 11:39:22 PM UTC-7, Jeroen Demeyer wrote:

> In src/setup.py we set Cython.Compiler.Options.old_style_globals = True,
> which causes Cython to fall back to the old behavior.
Yes, that should explain it. Strangely, this "old_style_globals" isn't
documented anywhere (the only Google hits are in source code).

After reading Robert's comment in the cython code at  https://github.com/sagemath/sage/blob/419f5ae5280eb124e56af80bee8328eb64f9cdc5/src/setup.py#L522 it's pretty obvious how to implement a python version (at least one that works on CPython):

import sys
def inject(s,v):
    G = sys._getframe(1).f_globals
    G[s]=v

This will inject a binding in the caller's globals dictionary, which is basically what the old style cython globals did. To make this a utility function we can use in "var" etc, we probably want to inject into sys._getframe(2).f_globals

It's still a bit of a hack, of course, but I think a more stable one than relying on deprecated cython behaviour (that cython kindly left accessible to sage).

[Jeroen Demeyer]

Thanks for all the suggestions in this thread. I implemented a proper
fix for the globals() issue at

http://trac.sagemath.org/ticket/12446