Lazy Attribute

[Andriy Kornatskyy]

A lazy attribute is an attribute that is calculated on demand and only once.

The post below shows how you can use lazy attribute in your Python class:

http://mindref.blogspot.com/2012/11/python-lazy-attribute.html

Comments or suggestions are welcome.

Thanks.

Andriy Kornatskyy

[Ian Kelly]

The name "attribute" is not very descriptive. Why not "lazy_attribute" instead?

I note that trying to access the descriptor on the class object
results in an error:

>>> from descriptors import attribute
>>> class Foo:
... @attribute
... def forty_two(self):
... return 6 * 9
...
>>> Foo().forty_two
54
>>> Foo.forty_two
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File ".\descriptors.py", line 33, in __get__
setattr(obj, f.__name__, val)
AttributeError: 'NoneType' object has no attribute 'forty_two'

If accessing the descriptor on the class object has no special
meaning, then the custom is to return the descriptor object itself, as
properties do.

>>> class Foo:
... @property
... def forty_two(self):
... return 6 * 9
...
>>> Foo().forty_two
54
>>> Foo.forty_two
<property object at 0x0280AD80>

[Ian Kelly]

On Thu, Nov 15, 2012 at 12:33 PM, Andriy Kornatskyy
<andriy.k...@live.com> wrote:
>
> A lazy attribute is an attribute that is calculated on demand and only once.
>
> The post below shows how you can use lazy attribute in your Python class:
>
> http://mindref.blogspot.com/2012/11/python-lazy-attribute.html
>
> Comments or suggestions are welcome.

I should add that I like the approach you're taking here. Usually
when I want a lazy property I just make an ordinary property of a
memoized function call:

def memoize(func):
cache = {}
@functools.wraps(func)
def wrapper(*args, **kwargs):
kwset = frozenset(kwargs.items())
try:
return cache[args, kwset]
except KeyError:
result = cache[args, kwset] = func(*args, **kwargs)
return result
return wrapper

class Foo:
def __init__(self):
self.times_called = 0

@property
@memoize # Alternatively, use functools.lru_cache
def forty_two(self):
self.times_called += 1
return 6 * 9


>>> foo = Foo()
>>> foo.times_called
0
>>> foo.forty_two
54
>>> foo.times_called
1
>>> foo.forty_two
54
>>> foo.times_called
1


Although you don't go into it in the blog entry, what I like about
your approach of replacing the descriptor with an attribute is that,
in addition to being faster, it makes it easy to force the object to
lazily reevaluate the attribute, just by deleting it. Using the
Person example from your blog post:

>>> p = Person('John', 'Smith')
>>> p.display_name
'John Smith'
>>> p.display_name
'John Smith'
>>> p.calls_count
1
>>> p.first_name = 'Eliza'
>>> del p.display_name
>>> p.display_name
'Eliza Smith'
>>> p.calls_count
2

Although in general it's probably better to use some form of reactive
programming for that.

[Andriy Kornatskyy]

Ian,

Thank you for the comments.

> The name "attribute" is not very descriptive. Why not "lazy_attribute" instead?

It just shorter and still descriptive.

> If accessing the descriptor on the class object has no special
> meaning, then the custom is to return the descriptor object itself, as
> properties do.

The lazy attribute, as a pattern, is designed to calculate something on demand, that being said means some `dynamic` nature must present, thus a class instance - object is a good candidate, while class itself is considered relatively `immutable`... of cause there might be extreme cases.

> If accessing the descriptor on the class object has no special
> meaning, then the custom is to return the descriptor object itself, as
> properties do.

If I would satisfy this, I will be forced to check for None 99.9% of the use cases (it is not None, being applied to an object). Thus it behaves as designed.

Thanks.

Andriy Kornatskyy


----------------------------------------
> From: ian.g...@gmail.com
> Date: Thu, 15 Nov 2012 15:24:40 -0700
> Subject: Re: Lazy Attribute
> To: pytho...@python.org

>
> On Thu, Nov 15, 2012 at 12:33 PM, Andriy Kornatskyy
> <andriy.k...@live.com> wrote:
> >
> > A lazy attribute is an attribute that is calculated on demand and only once.
> >
> > The post below shows how you can use lazy attribute in your Python class:
> >
> > http://mindref.blogspot.com/2012/11/python-lazy-attribute.html
> >
> > Comments or suggestions are welcome.
>

> --
> http://mail.python.org/mailman/listinfo/python-list

[Andriy Kornatskyy]

Ian,

Thank you for mentioning about this research, really appreciate that.

Thanks.

Andriy Kornatskyy


----------------------------------------
> From: ian.g...@gmail.com

> Date: Thu, 15 Nov 2012 15:46:19 -0700

> Subject: Re: Lazy Attribute
> To: pytho...@python.org
>
> On Thu, Nov 15, 2012 at 12:33 PM, Andriy Kornatskyy
> <andriy.k...@live.com> wrote:
> >
> > A lazy attribute is an attribute that is calculated on demand and only once.
> >
> > The post below shows how you can use lazy attribute in your Python class:
> >
> > http://mindref.blogspot.com/2012/11/python-lazy-attribute.html
> >
> > Comments or suggestions are welcome.
>

> --
> http://mail.python.org/mailman/listinfo/python-list

[Alex Strickland]

On 2012/11/16 09:49 AM, Andriy Kornatskyy wrote:

>> The name "attribute" is not very descriptive. Why not "lazy_attribute" instead?
>
> It just shorter and still descriptive.

Shorter, but not descriptive.

--
Regards
Alex

[Steven D'Aprano]

On Fri, 16 Nov 2012 10:49:07 +0300, Andriy Kornatskyy wrote:

> Ian,
>
> Thank you for the comments.
>
>> The name "attribute" is not very descriptive. Why not "lazy_attribute"
>> instead?
>
> It just shorter and still descriptive.

It is not descriptive. EVERYTHING accessed used dot notation obj.thing is
an attribute. What makes your "attribute" different from ordinary
attributes, properties, dynamic attributes calculated with __getattr__,
methods, etc?



--
Steven

[Rouslan Korneychuk]

On 11/16/2012 02:49 AM, Andriy Kornatskyy wrote:
>> If accessing the descriptor on the class object has no special
>> meaning, then the custom is to return the descriptor object itself, as
>> properties do.
>
> If I would satisfy this, I will be forced to check for None 99.9% of the use cases (it is not None, being applied to an object). Thus it behaves as designed.

That's not true. You can use a try-except block to return the descriptor
object when an AttributeError is raised.

[Rouslan Korneychuk]

Actually, never mind. I just realized the function has to be called
before the attribute can be set, which can not-only raise any exception,
but could potentially have undesired side-effects given a parameter it
doesn't expect.

[Andriy Kornatskyy]

from wheezy.core.descriptors import attribute as lazy

@lazy
def display_name...

Thanks.

Andriy Kornatskyy


----------------------------------------
> Date: Fri, 16 Nov 2012 09:56:41 +0200
> From: ss...@mweb.co.za
> To: pytho...@python.org
> Subject: Re: Lazy Attribute

>
> On 2012/11/16 09:49 AM, Andriy Kornatskyy wrote:
>
> >> The name "attribute" is not very descriptive. Why not "lazy_attribute" instead?
> >
> > It just shorter and still descriptive.
>

> Shorter, but not descriptive.
>
> --
> Regards
> Alex

> --
> http://mail.python.org/mailman/listinfo/python-list

[Andriy Kornatskyy]

Same applies to properties... they are seen as an object attributes.

Thanks.

Andriy


----------------------------------------
> From: steve+comp....@pearwood.info
> Subject: Re: Lazy Attribute
> Date: Fri, 16 Nov 2012 09:04:39 +0000
> To: pytho...@python.org

> --
> http://mail.python.org/mailman/listinfo/python-list

[Steven D'Aprano]

On Thu, 15 Nov 2012 15:46:19 -0700, Ian Kelly wrote:

> Although you don't go into it in the blog entry, what I like about your
> approach of replacing the descriptor with an attribute is that, in
> addition to being faster, it makes it easy to force the object to lazily
> reevaluate the attribute, just by deleting it.

You just lost me right there. That's a poor UI design -- it violates the
principle of least surprise. If I delete something, it should be deleted.
Consider your example:

>>>> del p.display_name
>>>> p.display_name
> 'Eliza Smith'

That's very surprising. I am not aware of any other name in Python where
deleting it does not remove the name from the namespace. (It is possible
with properties, but I haven't ever come across someone who does that.)

I don't have a good solution for invaliding such lazy attributes. Ideally
we could have a new statement:

refresh obj.attr # or some other name like "invalidate"

but that won't happen. Other alternatives like:

obj.attr.refresh()
refresh(obj.attr)

can't work because the function will see the result of the attribute
lookup, not the lazy attribute itself. This won't do:

obj.__class__.attr.refresh()

because it won't know which instance to invalidate, although this could
work:

obj.__class__.attr.refresh(obj) # but it's ugly

I'm very vaguely leaning towards this as the least-worst solution to
invalidating the cached value:

refresh(obj, 'attr') # pass the instance and the name


--
Steven

[Andriy Kornatskyy]

This is very minor use case. Unlikely useful to add any checks for None, or translate one exception to the other... with pretty much the same outcome: it makes sense in objects only.

Thanks.

Andriy

----------------------------------------
> From: rous...@msn.com
> Subject: Re: Lazy Attribute
> Date: Fri, 16 Nov 2012 05:12:11 -0500
> To: pytho...@python.org

> --
> http://mail.python.org/mailman/listinfo/python-list

[Stefan H. Holek]

On 16.11.2012, at 11:29, Steven D'Aprano wrote:

> I'm very vaguely leaning towards this as the least-worst solution to
> invalidating the cached value:
>
> refresh(obj, 'attr') # pass the instance and the name

This it exactly how lazy handles invalidation. http://lazy.readthedocs.org/en/latest/

Stefan

--
Stefan H. Holek
ste...@epy.co.at

[Andriy Kornatskyy]

I believe it is not valid relate a lazy attribute as something `cached` since it cause confusion (e.g. delete of attribute cause cached item to be re-evaluated...), `cached` and `lazy` have completely different semantic meaning... however might overlap, as we see.

Andriy


----------------------------------------
> From: steve+comp....@pearwood.info
> Subject: Re: Lazy Attribute
> Date: Fri, 16 Nov 2012 10:29:03 +0000
> To: pytho...@python.org

> --
> http://mail.python.org/mailman/listinfo/python-list

[Stefan H. Holek]

On 15.11.2012, at 20:33, Andriy Kornatskyy wrote:

> A lazy attribute is an attribute that is calculated on demand and only once.
>
> The post below shows how you can use lazy attribute in your Python class:
>
> http://mindref.blogspot.com/2012/11/python-lazy-attribute.html
>
> Comments or suggestions are welcome.

There is a ready made and well tested lazy decorator at http://pypi.python.org/pypi/lazy. I even has a better name. ;-)

Since people seem to come up with their own implementations on a bi-weekly basis, I am seriously wondering how to improve its visibility. Putting it on PyPI alone does not cut it, apparently.

[Stefan H. Holek]

On 16.11.2012, at 11:54, Andriy Kornatskyy wrote:

>> Subject: Re: Lazy Attribute
>> From: ste...@epy.co.at
>> Date: Fri, 16 Nov 2012 11:45:32 +0100
>> To: pytho...@python.org

>>
>> On 16.11.2012, at 11:29, Steven D'Aprano wrote:
>>

>>> I'm very vaguely leaning towards this as the least-worst solution to
>>> invalidating the cached value:
>>>
>>> refresh(obj, 'attr') # pass the instance and the name
>>

>> This it exactly how lazy handles invalidation. http://lazy.readthedocs.org/en/latest/
>

> @property is a solution to evaluate something that is dynamic. @attribute is good for immutable objects. Even if we assume refresh is a good idea... how I would know when it is valid to `refresh`? What is criteria?
>
> Andriy

I had to implement invalidation anyway in order to write tests. I decided to expose the mechanism to keep users from having to invent their own SHOULD the need arise. I was not advocating invalidation in any way with my reply. All I wanted was to confirm the "least bad" solution. ;-)

[Demian Brecht]

> There is a ready made and well tested lazy decorator at http://pypi.python.org/pypi/lazy. I even has a better name. ;-)

I was ignorantly unaware of this module. You've saved me a few lines of code every time I want to achieve lazy loading - thanks :)

> Since people seem to come up with their own implementations on a bi-weekly basis, I am seriously wondering how to improve its visibility. Putting it on PyPI alone does not cut it, apparently.

This was a good start ;)

Demian Brecht
@demianbrecht
http://demianbrecht.github.com