asyncio_tkinter doesn't work

[Victor Stinner]

Hi,

Ádám Szieberth contacted me privately to notify me that
"asyncio_tkinter" does not work. I host this project in this
repository:
https://bitbucket.org/haypo/asyncio_staging

The code was written by Dino Viehland for a talk, I just updated the
code for the new asyncio API (ex: import tulip => import asyncio).

Is there anyone interested to fix the code?

Adam wrote that the following line in tkapp.py :
self.button["command"] = self.do_count

must be replaced with:
self.button["command"] = lambda: asyncio.Task(self.do_count())
or
self.button["command"] = functools.partial(asyncio.Task, self.do_count())

But it's not enough.

guievents.py contains a class which inherits from AbstractEventLoop
and I see a lot of private code copied from asyncio, like
run_in_executor(). This code uses privates classes like Handle,
whereas Handle API has changed.

I'm not sure that I understood the design. asyncio event loop and Tk
main loop are both running in the same thread? The code uses at least
two pools of threads.

It looks like the Tk loop must run in the main loop. Why not using a
standard asyncio event loop in a dedicated thread with
call_soon_threadsafe()?

Victor

[Guido van Rossum]

I suspect it never worked, or only with a private variant of Tulip developed by Dino and Steve Dower, a long time ago, when Steve was arguing for yield instead of yield-from. I believe I have a version in a private scratch directory too somewhere. it would be good to revive this for real, but It would probably be a pretty big project, and Dino's code would at best be useful to glean some tricks from...

--
--Guido van Rossum (on iPad)

[Guido van Rossum]

I take back my speculation from last night; Dino's code did once work, and it uses yield from. I found the version Dino emailed me privately, cleaned up the tulip references, added the lambda helper you suggested, and then it worked. That version is definitely related to the one you checked in, but also pretty different; I will try to understand the differences and maybe I can help you. (You also need a Holmes.txt file; this one seems to work: http://www.gutenberg.org/cache/epub/1661/pg1661.txt.)

--
--Guido van Rossum (python.org/~guido)

[Guido van Rossum]

OK, I suppose you already figured this out, but the copied-and-pasted code has some bugs due to renames. Attached is a fix that makes the program actually work (if you can apply it that might save the next person who looks at that code a few minutes of debugging).

I noticed that on OSX I sometimes have to wiggle the mouse a bit so that events happen. I think this is an OSX-specific problem with communication between threads; I once figured this out with the help of the tkinter-discuss list (chiefly https://mail.python.org/pipermail/tkinter-discuss/2013-November/003520.html) -- the only thing that reliably passes events between threads is event_generate().

Regardless, this would still not be a real solution, due to the copy-paste nature of Dino's code. I expect that your proposal to run the IO loop in a second thread makes the most sense, but I also suspect that that would take away some of the (relative) elegance of using asyncio in the first place. Also,

Let me play with these ideas some more. It would be a neat demo if it worked.

--Guido

[Luciano Ramalho]

Reviving an old thread...

I just watched the video of Dino's talk at PyCon 2013 [1] and I am interested in updating their example code to run with asyncio 3.4 instead of tulip. I am writing to ask if anyone has done further work on this since may/2014 when Guido last wrote about it.

[1] http://lanyrd.com/2013/pycon/scdywd/

I believe their implementation of (a very small subset) of the asyncio event loop API on top of Tk is a great didactic example for understanding futures, the core of an event loop and the use of yield from for saner event oriented programming. As a teacher, I also think a GUI makes it easier to demonstrate a lot of the ideas in asyncio.

So if anyone has worked on this recently, or has any suggestions, any help will be most welcome!

I will report back as soon as I have something to show (or questions to ask).

Best,

Luciano

[Luciano Ramalho]

I just created a fork [1] of Victor's asyncio_staging repo and applied
the patch from Guido's last message. The example still doesn't work
but I will try to fix it tomorrow. I'll report back.

[1] https://bitbucket.org/ramalho/asyncio_staging

Cheers,

Luciano

--
Luciano Ramalho
Twitter: @ramalhoorg

Professor em: http://python.pro.br
Twitter: @pythonprobr

[Guido van Rossum]

Thank you. I have not found the time to work on this further myself, so I'd like to hear how you fare!

[Luciano Ramalho]

On Tue, Jan 20, 2015 at 2:38 AM, Guido van Rossum <gu...@python.org> wrote:
> Thank you. I have not found the time to work on this further myself, so I'd
> like to hear how you fare!

Hello, I made the example work: the essential Count Words button works
as advertised (as long as you have the holmes.txt file available).

After applying Guido's patch from 2014-05-17, I had to add touch only
three lines to make guievents.GuiEventLoop subclass BaseEventLoop
(previously it subclassed AbstractEventLoop). See changes in [1]

[1] https://bitbucket.org/ramalho/asyncio_staging/commits/8e7668cbd3e28c41f009e48800c9671e227b80e2

But there is still a bug I don't know how to fix: when I click quit
the Tk window closes by the program hangs. It's a threading issue (see
output and traceback at the end of this message).

I am now writing a tkapp2.py demo with three buttons, to show the
three different ways of counting words: sequentially (blocking), using
callbacks and coroutines.

I'd appreciate help with the threading issue on shutdown. I did some
research on the Web but did not find a solution yet.

Best,

Luciano

##################################### output of running tkapp.py
$ python3 tkapp.py
Starting on thread -1601961560
Loading words on -1601961560
110678 words loaded
Cleaning on -1601961560
107396 remain after cleaning
Counting on -1601961560
7947 distinct words after counting
Sorting on thread -1601961560
The ten most common words: the, and, i, to, of, a, in, that, it, you
^CException ignored in: <module 'threading' from
'/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py'>
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py",
line 1295, in _shutdown
t.join()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py",
line 1061, in join
self._wait_for_tstate_lock()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py",
line 1077, in _wait_for_tstate_lock
elif lock.acquire(block, timeout):
KeyboardInterrupt

[Luciano Ramalho]

I am still stuck with this, and it will be such an awesome example
when it works.

Right now the tkapp.py app works as advertised but hangs on exit. When
you click the Quit button the window closes but the program never
terminates. If you hit CTRL-C the code is invariably parked in that
lock.acquire(block, timeout) call in threading.py.

Meanwhile I wrote tkapp2.py implementing the three approaches for the
expensive processing function (sequential, nested callback and
coroutine), with a button for each. This is nice for doing demos. But
this second version has the same problem as the previous one: hangs on
exit.

Best,

Luciano

[Luciano Ramalho]

Alan Cristhian (@AlanCristhian on Github) fixed the hang-on-exit bug
by turning the asyncio event loop thread into a daemon; see
guievents.py, method GuiEventLoop._start_io_event_loop.

With this, the tkapp.py from Dino Viehland's PyCon 2013 time is ready
for prime time again!

As I said before, this example is outstanding because it demonstrates
three things:

- implementing an alternative asyncio.BaseEventLoop subclass on top of
the Tkinter event loop;
- leveraging futures and yield from to escape "callback hell";
- applying the asyncio abstractions to GUI programming instead of
network programming.

Besides updating the code so it runs on Python 3.4 with the standard
asyncio module, I added the tkapp2.py which makes it easier to run the
three demonstrations from Viehland's talk: sequential processing
(freezing the GUI), asynchronous with callbacks and asynchronous with
coroutines.

Thank you Guido and Alan for helping me with this, and Dino for the
awesome talk and example!

Victor: I made a pull request so you can merge this with the
haypo/asyncio_staging repo on Bitbucket.

Thanks!

Best,

Luciano

--
Luciano Ramalho
| Author of Fluent Python (O'Reilly, 2015)
| http://shop.oreilly.com/product/0636920032519.do
| Professor em: http://python.pro.br
| Twitter: @ramalhoorg

[Luciano Ramalho]

On Sun, Apr 5, 2015 at 10:12 AM, Luciano Ramalho <luc...@ramalho.org> wrote:
> Alan Cristhian (@AlanCristhian on Github) fixed the hang-on-exit bug
> by turning the asyncio event loop thread into a daemon; see
> guievents.py, method GuiEventLoop._start_io_event_loop.

Ah, yes, my code is in this repo, a fork of haypo/asyncio_staging:

https://bitbucket.org/ramalho/asyncio_staging

Cheers,

Luciano