exit ThreadPoolExecutor immediately

[Atul Johri]

I am looking for a way to stop a ThreadPoolExecutor immediately under the
assumption that I don't care about what's currently running or pending.

```
limit = 2
executor = ThreadPoolExecutor(10)
posts = itertools.islice(mygen(executor=executor, **kwargs), 0, limit)
for post in posts:
print(post)
executor.shutdown(wait=False)
```

Basically I have a generator, mygen, which is using the executor to submit
many tasks in parallel and yield the result one at a time. I would like to
be able to limit the generator and have the executor stop processing
immediately.

I was considering clearing the _work_queue or iterating over it and running
future.cancel() on each future but both seem to be quite hacky and didn't
work in my initial try.

https://github.com/python/cpython/blob/master/Lib/concurrent/futures/thread.py#L83

Any ideas?

- Al

[dieter]

I would implement my own class (inheriting from "ThreadPoolExecutor")
and give it appropriate API to realise what I need (using whatever
internal implementation details are necessary).

I should not be that difficult for tasks not yet started (i.e.
not yet run be a thread).
In earlier Python versions, it was not
possible from Python level to abort a thread; there was a C level
Python API function to abort a thread once it started to execute Python
code again. Thus, there was not a complete solution for your problem.
This might have changed with modern Python version, but I doubt it
(the main reason for the restriction has been the difficulty
to cleanly abort a thread in a platform independent way -- this
difficulty should remain, whatever the Python version).

[woo...@gmail.com]

Doug Hellmann has what looks like a similar example using a poison pill (2nd example at) https://pymotw.com/2/multiprocessing/communication.html#multiprocessing-queues Note that join() allows the processes to finish so it you don't care then don't "join()" them. You can also terminate a multiprocessing thread https://pymotw.com/2/multiprocessing/basics.html#terminating-processes