TL;DR:
  I have a function that makes a decorator, and only some of the names
  from an outer scope appear in the inner closure's locals().
  And I do not understand why at all.

Let me explain...

Environment: python 2.7.3 on MacOSX Mountain Lion from MacPorts.

Background: I have a decorator called "file_property" that watches a
file for changes and reloads the file at need. Otherwise it returns a
cached value. It has a lock and some sanity checks and works quite well.

Example method:

  class myclass(object):
    @file_property
    def rules(self):
      with open(self._rules_path) as rfp:
        R = parse(rfp)
      return R
  C = myclass()

and using C.rules fetches parses the rule file as needed and caches the
value until the file next changes.

I naively wrote file_property with a bunch of default parameters:

  def file_property(func, attr_name=None, unset_object=None, poll_rate=1):

and because I never overrode these failed to realise they're useless. Fine.

So, I'm rewriting file_property like this:

  def file_property(func):
    return make_file_property()(func)

i.e. it makes a "vanilla" file_property using the default internals.
Now I have make_file_property with the default parameters:

  def make_file_property(attr_name=None, unset_object=None, poll_rate=1):

which I might use as:

  class myclass(object):
    @make_file_property(poll_rate=3)
    def rules(self):
      with open(self._rules_path) as rfp:
        R = parse(rfp)
      return R

The inner function is the same, but it won't reload the file more often
that once every 3 seconds.

However, I can't make my make_file_property function work. I've stripped
the code down and it does this:

  [hg/css-mailfiler]fleet*1> python foo.py
  make_file_property(attr_name=None, unset_object=None, poll_rate=1): locals()={'attr_name': None, 'poll_rate': 1, 'unset_object': None}
  made_file_property(func=<function f at 0x10408b0c8>): locals()={'func': <function f at 0x10408b0c8>, 'unset_object': None}
  Traceback (most recent call last):
    File "foo.py", line 21, in <module>
      def f(self, foo=1):
    File "foo.py", line 4, in file_property
      return make_file_property()(func)
    File "foo.py", line 10, in made_file_property
      if attr_name is None:
  UnboundLocalError: local variable 'attr_name' referenced before assignment

Observe above that 'unset_object' is in locals(), but not 'attr_name'.
This surprises me.

The stripped back code (missing the internals of the file property
watcher) looks like this:

  import sys

  def file_property(func):
    return make_file_property()(func)

  def make_file_property(attr_name=None, unset_object=None, poll_rate=1):
    print >>sys.stderr, "make_file_property(attr_name=%r, unset_object=%r, poll_rate=%r): locals()=%r" % (attr_name, unset_object, poll_rate,locals())
    def made_file_property(func):
      print >>sys.stderr, "made_file_property(func=%r): locals()=%r" % (func, locals())
      if attr_name is None:
        attr_name = '_' + func.__name__
      lock_name = attr_name + '_lock'
      def getprop(self):
        with getattr(self, lock_name):
          # innards removed here
          pass
        return getattr(self, attr_name, unset_object)
      return property(getprop)
    return made_file_property

  @file_property
  def f(self, foo=1):
    print "foo=%r" % (foo,)

  @make_file_property(attr_name="_blah")
  def f2(self, foo=2):
    print "foo=%r" % (foo,)

Can someone explain what I'm doing wrong, or tell me this is a python
bug?
--
Cameron Simpson <c...@zip.com.au>

But 'unset_object' is in locals(). Why one and not the other?
Obviously there's something about closures here I'm missing.

| Pardon my brevity and some lack of trimming; I'm on a smartphone and in a
| rush.

No worries. Thansk for the rpely.
--
Cameron Simpson <c...@zip.com.au>

Can you elaborate a bit on that? The only place in my code that
unset_object is set is as a default parameter in make_file_property
(snippet):

  def make_file_property(attr_name=None, unset_object=None, poll_rate=1):
    print >>sys.stderr, "make_file_property(attr_name=%r, unset_object=%r, poll_rate=%r): locals()=%r" % (attr_name, unset_object, poll_rate,locals())
    def made_file_property(func):
      print >>sys.stderr, "made_file_property(func=%r): locals()=%r" % (func, locals())
      if attr_name is None:
        attr_name = '_' + func.__name__

and attr_name is set there also.

Is attr_name omitted from locals() in made_file_property _because_ I
have an assignment statement?

If that's the case, should I be doing this (using distinct names for the
closure variable and the function local variable):

  def make_file_property(attr_name=None, unset_object=None, poll_rate=1):
    print >>sys.stderr, "make_file_property(attr_name=%r, unset_object=%r, poll_rate=%r): locals()=%r" % (attr_name, unset_object, poll_rate,locals())
    def made_file_property(func):
      print >>sys.stderr, "made_file_property(func=%r): locals()=%r" % (func, locals())
      if attr_name is None:
        my_attr_name = '_' + func.__name__
      else:
        my_attr_name = attr_name
      lock_name = my_attr_name + '_lock'
      def getprop(self):
        with getattr(self, lock_name):
          pass
        return getattr(self, my_attr_name, unset_object)

i.e. deliberately _not_ assigning to attr_name as as to _avoid_ masking
the outer attr_name from the inner locals()?

BTW, doing that works. Is that The True Path for this situation?

If so, I think I now understand what's going on: Python has inspected
the inner function and not placed the outer 'attr_name' into locals()
_because_ the inner function seems to have its own local attr_name
in use, which should not be pre-tromped.

--
Cameron Simpson <c...@zip.com.au>

Aha. Good.

| > If that's the case, should I be doing this (using distinct names for the
| > closure variable and the function local variable):
| >
| >   def make_file_property(attr_name=None, unset_object=None, poll_rate=1):
[...]
| >       if attr_name is None:
| >         my_attr_name = '_' + func.__name__
| >       else:
| >         my_attr_name = attr_name
[...]
| > i.e. deliberately _not_ assigning to attr_name as as to _avoid_ masking
| > the outer attr_name from the inner locals()?
| >
| > BTW, doing that works. Is that The True Path for this situation?
|
| That's a perfectly good way to do it as long as you don't want to
| actually change the value of the outer attr_name.

Well, I don't need to - using a distinct local variable will do the job.  I
just hadn't realised I needed the extra level of naming.

| If you did, then
| you would either declare the variable as nonlocal (Python 3.x only)

... which is why I couldn't find such in the 2.7.3 doco ...

| or
| use a container (e.g. a 1-element list), which would allow you to
| modify the contents of the container without actually assigning to the
| variable.

Ah. Yeah, tacky; I've done that kind of thing in the past on occasion but
using a distinct local name is much cleaner here, and probably usually.

| > If so, I think I now understand what's going on: Python has inspected
| > the inner function and not placed the outer 'attr_name' into locals()
| > _because_ the inner function seems to have its own local attr_name
| > in use, which should not be pre-tromped.
|
| Exactly right.

Thanks for the explaination. Now I know a New Thing.

Cheers,
--
Cameron Simpson <c...@zip.com.au>