Who use Trollius? Should we deprecate this project?

[Victor Stinner]

Hi,

Short story: I don't want to maintain Trollius anymore. If anyone is
using it, please say it. Otherwise, I will simply make it clear that
Trollius must not be used anymore.

Since almost no library support Trollius, I'm not sure that it's
possible to build an application on top of it. For example, there is
no HTTP client for Trollius!

--

Long story.

Two years ago, I wrote the Trollius project for OpenStack. I ported
asyncio to Python 2.6. When I wrote it, my main motivation was to
replace implicit task switching *and* port OpenStack to Python 3.
OpenStack uses eventlet and eventlet didn't support Python 3.

In the meanwhile, eventlet was ported to Python 3 (I helped to port it
;-)), and it became clear to me that OpenStack is too big to replace
eventlet with anything else (replacing eventlet with threads was also
discussed). I mean, it's too much work for a negligible gain, with a
high risk of regression, for such large project. For all these
reasons, I stopped to work on Trollius.

My goal was to support asyncio and trollius in a single code based for
the application code, but also for libraries. In practice, there are
important technical issues for that. asyncio coroutine uses "result =
yield from coro" whereas trollius coroutine uses "result = yield
From(coro)". I added "From()" to trollius to ease the transition from
Trollius to asyncio, the function is a no-op (it returns the coroutine
unchanged).

Since "yield from" raises a SyntaxError on Python 2, it's really hard
to write coroutines in a single file for Trollius (yield, Python 2)
and asyncio (yield from, Python 3).

Authors of recent libraries written for asyncio said clearly that they
don't want to support Trollius to have a clean code base. Some
libraries (like Pulsar) already supporting Twisted and Tornado added
support for trollius.

In short, very few libraries support trollius, and I don't think that
it's going to change. I only know trollius-redis, an explicit port of
an asyncio library (asyncio-redis) to Trollius.
https://trollius.readthedocs.org/libraries.html

--

Using asyncio with Python 3, it rocks ;-) It now has a wide choice of libraries!

Victor

[Tobias Oberstein]

Hi Victor,

Autobahn, a WebSocket/WAMP library supports running on top of Twisted or
asyncio (and Trollius for Py2):

https://github.com/crossbario/autobahn-python

It does that using 1 code base.

This is possible by virtue of txaio

https://github.com/crossbario/txaio#platform-support
https://travis-ci.org/crossbario/txaio/builds/104293274

(As you note, one code base is only possible when avoiding yield - which
we do inside Autobahn).

Autobahn has 25k downloads per month on PyPi, and is used in production
in different other software packages.

I don't know how many users are using it on Py2 with asyncio/Trollius -
but it's nice that users can choose any combination of Twisted, asyncio,
Py2 and Py3.

Thanks for Trollius!

Cheers,
/Tobias

[Victor Stinner]

Hi,

2016-01-25 21:57 GMT+01:00 Tobias Oberstein <tobias.o...@gmail.com>:
> Autobahn, a WebSocket/WAMP library supports running on top of Twisted or
> asyncio (and Trollius for Py2):

Yeah, it's mentioned in https://trollius.readthedocs.org/libraries.html

AutobahnPython, Pulsar and Tornado can be used with Trollius. Ok. But
I'm more interested by feedbak of final users, users building
applications with all these pieces.

I'm not sure that Trollius is used in practice :-)

Victor

[Tobias Oberstein]

Hi,

AutobahnPython supports running on Python 2 with asyncio - more
precisely, by using Trollius, a backport of asyncio to Python 2.

Same for txaio: https://github.com/crossbario/txaio#platform-support

Victor, creator of Trollius is thinking of depcrecating Trollius and is
asking if there is _real-world_ use of Trollius in _applications_.

Please see the thread here:

https://groups.google.com/forum/#!topic/python-tulip/mecVwhnVP0A

Since AutobahnPython depends on Trollius for Py2/asyncio support, I am
forwarding the question. If Trollius is deprecated, AutobahnPython will
(sooner or later) deprecate supporting that combination as well.

If you have an app using the combination, now would be a good time to
speak up;)

Cheers,
/Tobias

[Luca Sbardella]

Hi,

On 25 January 2016 at 21:06, Victor Stinner <victor....@gmail.com> wrote:

Hi,

2016-01-25 21:57 GMT+01:00 Tobias Oberstein <tobias.o...@gmail.com>:
> Autobahn, a WebSocket/WAMP library supports running on top of Twisted or
> asyncio (and Trollius for Py2):

Yeah, it's mentioned in https://trollius.readthedocs.org/libraries.html

AutobahnPython, Pulsar and Tornado can be used with Trollius. Ok. But

Pulsard does not use trollius anymore.

The last version working with python 2.7 (and trollius) was pulsar 0.9.2, released more than a year ago.

Pulsar 1.0.x and 1.1.x is only for python 3.4 and above.

Pulsar 1.2.x will be for python 3.5 only.

Regards,

--

http://lucasbardella.com

[Ben Darnell]

On Mon, Jan 25, 2016 at 4:45 PM, Victor Stinner <victor....@gmail.com> wrote:

Hi,

Short story: I don't want to maintain Trollius anymore. If anyone is
using it, please say it. Otherwise, I will simply make it clear that
Trollius must not be used anymore.

Since almost no library support Trollius, I'm not sure that it's
possible to build an application on top of it. For example, there is
no HTTP client for Trollius!

The Tornado bridge allows you to use any tornado-based library (and indirectly, twisted ones) on trollius:

```

from tornado.platform.asyncio import AsyncIOMainLoop, to_asyncio_future

from tornado.httpclient import AsyncHTTPClient

from trollius import coroutine, get_event_loop, From

@coroutine

def main():

    client = AsyncHTTPClient()

    resp = yield From(to_asyncio_future(client.fetch('http://www.google.com')))

    print(resp)

if __name__ == '__main__':

    AsyncIOMainLoop().install()

    get_event_loop().run_until_complete(main())

```

However, this only really makes sense to fill in the gaps between "native" trollius libraries. Since there are hardly any of those, I don't see any reason for anyone to use this mode (and I'm not aware of anyone using Tornado's support for trollius).

 

In short, very few libraries support trollius, and I don't think that
it's going to change.

I agree with this and think it makes sense to end support for trollius.

-Ben 

[Victor Stinner]

2016-01-25 23:27 GMT+01:00 Luca Sbardella <luca.sb...@gmail.com>:
> Pulsard does not use trollius anymore.
> The last version working with python 2.7 (and trollius) was pulsar 0.9.2,
> released more than a year ago.
>
> Pulsar 1.0.x and 1.1.x is only for python 3.4 and above.
> Pulsar 1.2.x will be for python 3.5 only.

Oh wow, dropping support for Python 2 is a brave choice for a
framework. Congrats :-)

Victor

[Luca Sbardella]

Thanks, to be honest the decision was quite straightforward at the end:

* Trollius was too slow on nested coroutines (no yield from)

* Pulsar does not have a huge number of end-users deploying it in production (I did a little survey)

* Most of the end-users, including myself, are on python 3

* Finally, moving to the new py 2.5 syntax in pulsar 1.2 should improve and speed up code even more

So I ditched python 2 altogether, and I feel fine ;-)

--

http://lucasbardella.com

[Victor Stinner]

Hi,

2016-01-26 10:27 GMT+01:00 Luca Sbardella <luca.sb...@gmail.com>:
> Thanks, to be honest the decision was quite straightforward at the end:
>
> * Trollius was too slow on nested coroutines (no yield from)

Ah yes, I'm aware of the issue. I documented it a little bit, and I
had to modify asyncio unit tests for increase the number of allowed
iterations :-)

Good to know that you confirm this in practice. So it's also a very
good feature of asyncio, an advantage compared to Twisted
@inlineCallbacks or Tornado coroutines (toro) ;-)

> * Pulsar does not have a huge number of end-users deploying it in production
> (I did a little survey)
> * Most of the end-users, including myself, are on python 3
> * Finally, moving to the new py 2.5 syntax in pulsar 1.2 should improve and
> speed up code even more

Yeah, we all love Python *2*.5, best syntax ever! :-D

(I guess that you are refering to Yury's PEP 492, await/async, right?)
https://www.python.org/dev/peps/pep-0492/

Victor

[Luca Sbardella]

On Tuesday, 26 January 2016, Victor Stinner <victor....@gmail.com> wrote:

Hi,

> * Finally, moving to the new py 2.5 syntax in pulsar 1.2 should improve and
> speed up code even more

Yeah, we all love Python *2*.5, best syntax ever! :-D

(I guess that you are refering to Yury's PEP 492, await/async, right?)
https://www.python.org/dev/peps/pep-0492/

Yes, 3.5, silly typo.

However I'll probably wait for Python 3.6 to be released when, hopefully, asyncio won't be provisional anymore.

--

http://lucasbardella.com

[Matthias Kirst]

Hi Victor,

We are using the autobahn/trollius framework for inter-process communication of a Python-COM-component within a .NET-GUI-Application with a HTML5 based presentation component.

That's great stuff, since even events can be sent easily.

We definitely have to stick to Python 2.7 for the next coming years.

So we really appreciate your work (and the autobahn folks as well ...) and hope that this technology can be maintained for a while.

IconoClust Download

Thank you very much for providing trollius,

Matthias Kirst,

Software Engineer

[Victor Stinner]

Hi,

2016-01-28 14:58 GMT+01:00 Matthias Kirst <ma...@clondiag.com>:
> We are using the autobahn/trollius framework for inter-process communication
> of a Python-COM-component within a .NET-GUI-Application with a HTML5 based
> presentation component.
> That's great stuff, since even events can be sent easily.
> We definitely have to stick to Python 2.7 for the next coming years.
> So we really appreciate your work (and the autobahn folks as well ...) and
> hope that this technology can be maintained for a while.

Hum, in fact I was not really prepared to such reply :-)

To be clear: I'm *not* interested to maintain Trollius anymore. I'm
not following anymore the latest changes made in asyncio, so it became
harder for me to update Trollius.

Trollius is a fork of the asyncio git repository. To update Trollius
to retrieve latest asyncio changes, you have to fix a lot of subtle
issues for Python 2 (Python 3 language is simpler!).

Victor

[Guido van Rossum]

So maybe you should turn this into a call for maintainers! Open source FTW!

--
--Guido van Rossum (python.org/~guido)

[Tobias Oberstein]

Hi,

>> We are using the autobahn/trollius framework for inter-process
communication
>> of a Python-COM-component within a .NET-GUI-Application with a HTML5 based
>> presentation component.

Nice!

OT: Curious: so on the WAMP side, you are using AutobahnJS and
AutobahnPython, but no native .NET WAMP client library (like eg
https://github.com/Code-Sharp/WampSharp)?

>> That's great stuff, since even events can be sent easily.
>> We definitely have to stick to Python 2.7 for the next coming years.
>> So we really appreciate your work (and the autobahn folks as well ...) and

Thanks;)

>> hope that this technology can be maintained for a while.
>
> Hum, in fact I was not really prepared to such reply :-)
>
> To be clear: I'm *not* interested to maintain Trollius anymore. I'm
> not following anymore the latest changes made in asyncio, so it became
> harder for me to update Trollius.
>
> Trollius is a fork of the asyncio git repository. To update Trollius
> to retrieve latest asyncio changes, you have to fix a lot of subtle
> issues for Python 2 (Python 3 language is simpler!).

I can understand Victor. As he pointed out, his original goal (moving
OpenStack to a more modern networking stack) didn't work out for
reasons. So motiviation (and I guess) time is the limit.

@Matthias: rgd AutobahnPython: if you are stuck on Python 2, and if
Trollius is should really go away, that will leave Twisted as the
underlying networking stack. You could either use Twisted fully at app
surface also (like when using @inlineCallbacks for co-routines), or you
could use txaio at the app level, write your code against these idioms,
and then the code will run anywhere: Py2 or 3, asyncio/Trollius or Twisted.

Of course there is a price: no co-routine style for example. But that's
how Autobahn itself is written under the hood (one code base). So it
works, and we'll be supporting txaio.

Probably you already knew all that - just wanted to mention in case not.

Cheers,
/Tobias

>
> Victor
>

[Matthias Kirst]

Hi Victor,

We really appreciate your previous work.
Since we are using Trollius via the Autobahn framework - we do have the choice to fallback to Twisted - but we lose some elegance and compactness (of necessary Python packages).
Besides I intended to facilitate the asyncio programming model to other parts of our Python 2.6+/3.3+ codebase - here I could stick to the asyncio-synchronuous features.

Ok, but it's still a pitty to see trollius die - because in my opinion asyncio is one of the greatest features of Python 3. And there are lots of folks out there who can't switch to Python 3.

Thank you again,

Matthias


-------

>>Hum, in fact I was not really prepared to such reply :-)

>>To be clear: I'm *not* interested to maintain Trollius anymore. I'm not following anymore the latest changes made in asyncio, so it became harder for me to update Trollius.

>>Trollius is a fork of the asyncio git repository. To update Trollius to retrieve latest asyncio changes, you have to fix a lot of subtle issues for Python 2 (Python 3 language is simpler!).

>>Victor
(Die gesetzlich vorgeschriebenen Email-Pflichtangaben finden Sie auf unserer Webseite http://alere-technologies.com/en/imprint.html <http://alere-technologies.com/en/imprint.html>. For company details required by German law, please refer to this website.)

[Victor Stinner]

Hi,

2016-01-29 17:07 GMT+01:00 Matthias Kirst <ma...@clondiag.com>:
> Ok, but it's still a pitty to see trollius die - because in my opinion asyncio is one of the greatest features of Python 3. And there are lots of folks out there who can't switch to Python 3.

(It's a pain to maintain trollius, whereas Python 3 is here and works
well. I don't understand why people keep using Python 2 in 2016, it's
not so hard to port code. IMHO it's a better compromise to port code
to Python 3 than trying to maintain backports.)

Well, I don't want to maintain Trollius. If anyone wants to maintain
it, I will give him/her all keys ;-) Otherwise, my plan is just to add
a huge warning in the doc, I don't plan to remove it from the Internet
;-)

Victor

[Victor Stinner]

2016-01-25 21:45 GMT+01:00 Victor Stinner <victor....@gmail.com>:
> Short story: I don't want to maintain Trollius anymore. If anyone is
> using it, please say it.

Ah, someone reported me that 3 OpenStack projects use Trollius and
that Trollius doesn't work on Python 3.5.

Is anyone interested to fix Python 3.5 support in Trollius?


Oslo Messaging: oslo_messaging/_executors/impl_aioeventlet.py:import trollius

=> This file can be dropped. I wrote this executor, it's optional and
not used by any project in OpenStack


Gnocchi: gnocchi/statsd.py: import trollius as asyncio

=> Oh, I have to discuss with upstream to see how to replace trollius.
Gnocchi works on Python 2 and Python 3, it's maybe time to drop
support of Python 2 also known as "legacy Python" :-) At least,
Gnocchi may use asyncio on Python 3. (I didn't check if they use yield
for coroutines.)

Zaqar:
* zaqar/transport/websocket/driver.py: import trollius as asyncio
* zaqar/transport/websocket/protocol.py: import trollius as asyncio

=> I wasn't aware of this project. I don't know how trolllius is used,
nor if it can be replaced with something else. Need investigation.

Victor