How to Use Asyncio wait() in Python
You can wait for asyncio tasks to complete via the asyncio.wait() function.
Different conditions can be waited for, such as all tasks to complete, the first task to complete, and the first task to fail with an exception.
In this tutorial, you will discover how to wait for asyncio tasks to complete in Python.
What is asyncio.wait()
The asyncio.wait() function can be used to wait for a collection of asyncio tasks to complete.
Recall that an asyncio task is an instance of the asyncio.Task class that wraps a coroutine. It allows a coroutine to be scheduled and executed independently, and the Task instance provides a handle on the task for querying status and getting results.
You can learn more about asyncio tasks in the tutorial:
The wait() function allows us to wait for a collection of tasks to be done.
The call to wait can be configured to wait for different conditions, such as all tasks being completed, the first task completed and the first task failing with an error.
Next, let’s look at how we might use the wait() function.
Run your loops using all CPUs, download my FREE book to learn how.
How to Use asyncio.wait()
The asyncio.wait() function takes a collection of awaitables, typically Task objects.
This could be a list, dict or set of task objects that we have created, such as via calls to the asyncio.create() task function in a list comprehension.
The asyncio.wait() will not return until some condition on the collection of tasks is met.
By default, the condition is that all tasks are completed.
The wait() function returns a tuple of two sets. The first set contains all task objects that meet the condition, and the second contains all other task objects that do not yet meet the condition.
These sets are referred to as the “done” set and the “pending” set.
Technically, the asyncio.wait() is a coroutine function that returns a coroutine.
We can then await this coroutine which will return the tuple of sets.
The condition waited for can be specified by the “return_when” argument which is set to asyncio.ALL_COMPLETED by default.
We can wait for the first task to be completed by setting return_when to FIRST_COMPLETED.
When the first task is complete and returned in the done set, the remaining tasks are not canceled and continue to execute concurrently.
We can wait for the first task to fail with an exception by setting return_when to FIRST_EXCEPTION.
In this case, the done set will contain the first task that failed with an exception. If no task fails with an exception, the done set will contain all tasks and wait() will return only after all tasks are completed.
We can specify how long we are willing to wait for the given condition via a “timeout” argument in seconds.
If the timeout expires before the condition is met, the tuple of tasks is returned with whatever subset of tasks do meet the condition at that time, e.g. the subset of tasks that are completed if waiting for all tasks to complete.
If the timeout is reached before the condition is met, an exception is not raised and the remaining tasks are not canceled.
Now that we know how to use the asyncio.wait() function, let’s look at some worked examples.
Confused by the asyncio module API?
Download my FREE PDF cheat sheet
Example of Waiting for All Tasks
We can explore how to wait for all tasks using asyncio.wait().
In this example, we will define a simple task coroutine that generates a random value, sleeps for a fraction of a second, then reports a message with the generated value.
The main coroutine will then create many tasks in a list comprehension with the coroutine and then wait for all tasks to complete.
The complete example is listed below.
Running the example first creates the main() coroutine and uses it as the entry point into the asyncio program.
The main() coroutine then creates a list of ten tasks in a list comprehension, each providing a unique integer argument from 0 to 9.
The main() coroutine is then suspended and waits for all tasks to complete.
The tasks execute. Each generates a random value, sleeps for a moment, then reports its generated value.
After all tasks have been completed, the main() coroutine resumes and reports a final message.
This example highlights how we can use the wait() function to wait for a collection of tasks to be completed.
This is perhaps the most common usage of the function.
Note, that the results will differ each time the program is run given the use of random numbers.
Next, let’s look at how we might wait for the first task to complete.
Free Python Asyncio Course
Download my asyncio API cheat sheet and as a bonus you will get FREE access to my 7-day email course on asyncio.
Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.
Example of Waiting for First Task
We can explore how to wait for the first task to complete using asyncio.wait().
In this example, we will reuse the same simple task coroutine that generates a random value, sleeps for a fraction of a second, then reports a message with the generated value.
The main coroutine will then create many tasks in a list comprehension with the coroutine as before. It will then wait for the first task to complete.
Once a task is completed, it is retrieved from the set of done tasks and its details are reported.
The complete example is listed below.
Running the example first creates the main() coroutine and uses it as the entry point into the asyncio program.
The main() coroutine then creates a list of ten tasks in a list comprehension, each providing a unique integer argument from 0 to 9.
The main() coroutine is then suspended and waits for the first task to complete
The tasks execute. Each generates a random value, sleeps for a moment, then reports its generated value.
As soon as the first task completes, the wait() function returns and the main() coroutine resumes.
The “done” set contains a single task that is finished, whereas the “pending” set contains all other tasks that were provided in the collection to the wait() function.
The single finished task is then retrieved from the “done” set and is reported. The printed status of the task confirms that indeed it is done.
The other tasks are not canceled and continue to run concurrently. Their execution is cut short because the asyncio program is terminated.
This example highlights how we can use the wait() function to wait for the first task to complete.
Note, that the results will differ each time the program is run given the use of random numbers.
Next, let’s look at how we might wait for the first task to fail with an exception.
Overwheled by the python concurrency APIs?
Find relief, download my FREE Python Concurrency Mind Maps
Example of Waiting for First Failure
We can explore how to wait for the first task to fail using asyncio.wait().
In this example, we will reuse the same simple task coroutine that generates a random value, sleeps for a fraction of a second, then reports a message with the generated value.
The task coroutine is updated so that conditionally it will fail with an exception if the generated value is below a threshold.
The main coroutine will then create many tasks. It will then wait for the first task to fail with an exception.
Once a task has failed, it is retrieved from the set of done tasks and its details are reported.
The complete example is listed below.
Running the example first creates the main() coroutine and uses it as the entry point into the asyncio program.
The main() coroutine then creates a list of ten tasks in a list comprehension, each providing a unique integer argument from 0 to 9.
The main() coroutine is then suspended and waits for the first task to complete
The tasks execute. Each generates a random value, sleeps for a moment, then reports its generated value.
Conditionally, some tasks raise an exception if their generated value is above a threshold value. It is possible that all tasks generate a value below the threshold and none raise an exception, although this situation is extremely unlikely.
As soon as the first task fails with an exception, the wait() function returns and the main() coroutine resumes.
The “done” set contains a single task that failed first, whereas the “pending” set contains all other tasks that were provided in the collection to the wait() function.
The single failed task is then retrieved from the “done” set and is reported. The printed status of the task confirms that it indeed failed with an exception in this case.
The other tasks are not canceled and continue to run concurrently. Their execution is cut short because the asyncio program is terminated.
This example highlights how we can use the wait() function to wait for the first task to fail.
Note, that the results will differ each time the program is run given the use of random numbers.