|Daniil Fajnberg 73aa93a9b7|
Dynamically manage pools of asyncio tasks
Full documentation available at RtD.
A task pool is an object with a simple interface for aggregating and dynamically managing asynchronous tasks.
With an interface that is intentionally similar to the
multiprocessing.Pool class from the standard library, the
TaskPool provides you such methods as
starmap to execute coroutines concurrently as
asyncio.Task objects. There is no limitation imposed on what kind of tasks can be run or in what combination, when new ones can be added, or when they can be cancelled.
For a more streamlined use-case, the
SimpleTaskPool provides an even more intuitive and simple interface at the cost of flexibility.
If you need control over a task pool at runtime, you can launch an asynchronous
ControlServer to be able to interface with the pool from an outside process or via a network, and stop/start tasks within the pool as you wish.
Generally speaking, a task is added to a pool by providing it with a coroutine function reference as well as the arguments for that function. Here is what that could look like in the most simplified form:
from asyncio_taskpool import SimpleTaskPool ... async def work(_foo, _bar): ... async def main(): pool = SimpleTaskPool(work, args=('xyz', 420)) pool.start(5) ... pool.stop(3) ... await pool.gather_and_close()
Since one of the main goals of
asyncio-taskpool is to be able to start/stop tasks dynamically or "on-the-fly", most of the associated methods are non-blocking most of the time. A notable exception is the
gather_and_close method for awaiting the return of all tasks in the pool. (It is essentially a glorified wrapper around the
For working and fully documented demo scripts see USAGE.md.
pip install asyncio-taskpool
Python Version 3.8+, tested on Linux
asyncio-taskpool is licensed under the GNU LGPL version 3.0 specifically.
© 2022 Daniil Fajnberg