Is async object creation supported?

[Imran Geriskovan]

Or am I doing something wrong?
X does not run. Y runs.
Thanks for comments..

x = yield from X() # Does not work
y = Y() # Works..
yield from y.init()

class X:
def __init__():
yield from some async code

class Y:
def __init__():
some sync code

def init():
yield from some async code

[Luciano Ramalho]

The call X() invokes X.__new__() (a class method), and that class
method returns the instance which is then passed as the self argument
to __init__.

So asynchronous object creation should be done by a factory coroutine,
and not by calling the standard constructor machinery.

Best,

Luciano

--
Luciano Ramalho
| Author of Fluent Python (O'Reilly, 2015)
| http://shop.oreilly.com/product/0636920032519.do
| Professor em: http://python.pro.br
| Twitter: @ramalhoorg

[Imran Geriskovan]

> So asynchronous object creation should be done by a factory coroutine,
> and not by calling the standard constructor machinery.

It may be done by a factory or a second async helper function
method or by any other creative method. However, they all make the
the code more bureaucratic. Elegance of single point __init__ is lost.

May be developers can consider allowing

x = yield from X()

as async style will be more wide spread with python 3.5+
and resumable functions on C++.

Regards,
Imran

[Luciano Ramalho]

On Sat, May 16, 2015 at 11:50 AM, Imran Geriskovan
<imran.ge...@gmail.com> wrote:
> It may be done by a factory or a second async helper function
> method or by any other creative method. However, they all make the
> the code more bureaucratic. Elegance of single point __init__ is lost.

The "single point" you mention is an illusion. __new__ is always
invoked as part of object construction. If your class does not
implement it, the inherited __new__ is called to build the actual
instance. Always.

> x = yield from X()
>
> as async style will be more wide spread with python 3.5+
> and resumable functions on C++.

I agree the idea of an async constructor is intriguing, but I don't
think implementing an async factory is so bad.

If you'd like to promote your idea I think you should consider how
`yield from X()` will interact with __new__ and __init__, and whether
new special methods would be required besides __new__ and __init__.

Best,

Luciano

[Ben Darnell]

On Sat, May 16, 2015 at 10:50 AM, Imran Geriskovan <imran.ge...@gmail.com> wrote:

> So asynchronous object creation should be done by a factory coroutine,
> and not by calling the standard constructor machinery.

It may be done by a factory or a second async helper function
method or by any other creative method. However, they all make the
the code more bureaucratic. Elegance of single point __init__ is lost.

__init__ can't be a coroutine, but __new__ can (with the caveat that if __new__ is a coroutine, __init__ will not be called at all):

https://github.com/tornadoweb/tornado/issues/1161#issuecomment-91367819

Personally I'd rather use an explicit factory function than __new__ magic here, but it's possible to experiment with this style.

-Ben

[Imran Geriskovan]

> __new__ is a coroutine, __init__ will not be called at all):

> -Ben

It works! Thank you.
This style may be documented somewhere..

x = yield from X()

class X:
def __new__(cls):
self = object.__new__(cls)
yield from some async func
return self

Regards,
Imran

[Glyph]

On May 16, 2015, at 8:39 AM, Ben Darnell <b...@bendarnell.com> wrote:

Personally I'd rather use an explicit factory function than __new__ magic here, but it's possible to experiment with this style.

I can tell you from experiments in the Twisted community that this kind of "async object creation" without an explicit factory function is basically an antipattern, for the following reasons:

1. it interacts weirdly with inheritance and metaclasses; figuring out whose __new__ is going to be called and why always turns into an advanced calculus problem
2. it has surprising behavior with respect to __init__
3. it tightly couples a specific way of initializing a particular object with the residence of that object in memory. any object with an asynchronous initializer can't be deserialized normally, for example, because __new__ is now doing something surprising
4. it makes testing harder, because now you can't even create a simple test object without a whole event loop running

Fundamentally, object initialization is a local phenomenon.  It's a data structure that your program is using to think about the world.  When you "asynchronously initialize" something, you are doing two things: the remote gathering of information which will initialize the object, and the local construction and initialization.  The former phase should be a method (which can be a classmethod) and the latter is the object's constructor itself.

More broadly, having constructors do I/O is an anti-pattern for these same reasons; you just notice it faster in an async framework :).

-glyph

[Alex Gaynor]

Glyph's answer matches my experience. To add another reason though, remember back to when I was new to Python, I too wanted to figure out how to asynchronously instantiate an object, I think it's a common desire among new programmers. If the correct answer to that question requires a deep understanding of __init__, __new__, and god only knows what else ( ;-) ), the whole framework becomes more unapproachable. Making the correct answer "a factor function + boring __init__" requires far less deep understanding of Python, and pushes new folks towards good design in general.

Cheers,

Alex

--

"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero

GPG Key fingerprint: 125F 5C67 DFE9 4084

[Ludovic Gasc]

Hi Imran,

I don't have as much as experience like Glyph or Alex, however, for API-Hour, I've did the same pattern: I've two coroutines, simply "start" and "stop", launched when after the class is instancied or before to deleted: https://github.com/Eyepea/API-Hour/blob/master/api_hour/container.py#L40

It's simple to use and understand.

Regards.

--

Ludovic Gasc (GMLudo)

http://www.gmludo.eu/

[Imran Geriskovan]

> I can tell you from experiments in the Twisted community that this kind of
> "async object creation" without an explicit factory function is basically an
> antipattern, for the following reasons:

> More broadly, having constructors do I/O is an anti-pattern for these same
> reasons; you just notice it faster in an async framework :).
> -glyph

Thank you for sharing experience. However, anti-pattern is exactly
what I'm doing.
My experience is that whenever you try to push a concept into a predetermined
pattern, overhead and maintenance costs mostly surpass the cost of
developing the
concept itself. This is a major distraction.

The only patterns/frameworks to closely watch are the ones which are embedded
in the problem domain. And the language should be able to address them without
any "indirection". I found async style suprisingly fruitful in this area.

If a concept of creation is inherently async, let it be that way without any
intermediateries.

Regards,
Imran