generated from daniil-berg/boilerplate-py
made start
"non-async" using meta task
This commit is contained in:
@ -147,9 +147,11 @@ Or we could use a task pool:
|
||||
|
||||
Calling the :py:meth:`.map() <asyncio_taskpool.pool.TaskPool.map>` method this way ensures that there will **always** -- i.e. at any given moment in time -- be exactly 5 tasks working concurrently on our data (assuming no other pool interaction).
|
||||
|
||||
The :py:meth:`.gather_and_close() <asyncio_taskpool.pool.BaseTaskPool.gather_and_close>` line will block until **all the data** has been consumed. (see :ref:`blocking-pool-methods`)
|
||||
|
||||
.. note::
|
||||
|
||||
The :py:meth:`.gather_and_close() <asyncio_taskpool.pool.BaseTaskPool.gather_and_close>` line will block until **all the data** has been consumed. (see :ref:`blocking-pool-methods`)
|
||||
Neither :py:meth:`.apply() <asyncio_taskpool.pool.TaskPool.apply>` nor :py:meth:`.map() <asyncio_taskpool.pool.TaskPool.map>` return coroutines. When they are called, the task pool immediately begins scheduling new tasks to run. No :code:`await` needed.
|
||||
|
||||
It can't get any simpler than that, can it? So glad you asked...
|
||||
|
||||
@ -168,7 +170,7 @@ Let's take the :ref:`queue worker example <queue-worker-function>` from before.
|
||||
async def main():
|
||||
...
|
||||
pool = SimpleTaskPool(queue_worker_function, args=(q_in, q_out))
|
||||
await pool.start(5)
|
||||
pool.start(5)
|
||||
...
|
||||
pool.stop_all()
|
||||
...
|
||||
@ -193,9 +195,9 @@ This may, at first glance, not seem like much of a difference, aside from differ
|
||||
if some_condition and pool.num_running > 10:
|
||||
pool.stop(3)
|
||||
elif some_other_condition and pool.num_running < 5:
|
||||
await pool.start(5)
|
||||
pool.start(5)
|
||||
else:
|
||||
await pool.start(1)
|
||||
pool.start(1)
|
||||
...
|
||||
await pool.gather_and_close()
|
||||
|
||||
@ -228,6 +230,4 @@ The only method of a pool that one should **always** assume to be blocking is :p
|
||||
|
||||
One method to be aware of is :py:meth:`.flush() <asyncio_taskpool.pool.BaseTaskPool.flush>`. Since it will await only those tasks that the pool considers **ended** or **cancelled**, the blocking can only come from any callbacks that were provided for either of those situations.
|
||||
|
||||
In general, the act of adding tasks to a pool is non-blocking, no matter which particular methods are used. The only notable exception is when a limit on the pool size has been set and there is "not enough room" to add a task. In this case, :py:meth:`SimpleTaskPool.start() <asyncio_taskpool.pool.SimpleTaskPool.start>` will block until the desired number of new tasks found room in the pool (either because other tasks have ended or because the pool size was increased).
|
||||
|
||||
:py:meth:`TaskPool.apply() <asyncio_taskpool.pool.TaskPool.apply>` and :py:meth:`TaskPool.map() <asyncio_taskpool.pool.TaskPool.map>` (and its variants) will **never** block. Since they make use of "meta-tasks" under the hood, they will always return immediately. However, if the pool was full when one of them was called, there is **no guarantee** that even a single task has started, when the method returns.
|
||||
All methods that add tasks to a pool, i.e. :py:meth:`TaskPool.map() <asyncio_taskpool.pool.TaskPool.map>` (and its variants), :py:meth:`TaskPool.apply() <asyncio_taskpool.pool.TaskPool.apply>` and :py:meth:`SimpleTaskPool.start() <asyncio_taskpool.pool.SimpleTaskPool.start>`, are non-blocking by design. They all make use of "meta tasks" under the hood and return immediately. It is important however, to realize that just because they return, does not mean that any actual tasks have been spawned. For example, if a pool size limit was set and there was "no more room" in the pool when :py:meth:`.map() <asyncio_taskpool.pool.TaskPool.map>` was called, there is **no guarantee** that even a single task has started, when it returns.
|
||||
|
Reference in New Issue
Block a user