Parallel Requests

!!! warning This page documents some proposed functionality that is not yet released. See pull request #52 for the first-pass of an implementation.

HTTPX allows you to make HTTP requests in parallel in a highly efficient way,using async under the hood, while still presenting a standard threaded interface.

This has the huge benefit of allowing you to efficiently make parallel HTTPrequests without having to switch out to using async all the way through.

Making Parallel Requests

Let’s make two outgoing HTTP requests in parallel:

>>> with httpx.parallel() as parallel:
>>>     pending_one = parallel.get('https://example.com/1')
>>>     pending_two = parallel.get('https://example.com/2')
>>>     response_one = pending_one.get_response()
>>>     response_two = pending_two.get_response()

If we’re making lots of outgoing requests, we might not want to deal with theresponses sequentially, but rather deal with each response that comes backas soon as it’s available:

>>> with httpx.parallel() as parallel:
>>>     for counter in range(1, 10):
>>>         parallel.get(f'https://example.com/{counter}')
>>>     while parallel.has_pending_responses:
>>>         r = parallel.next_response()

Exceptions and Cancellations

The style of using parallel blocks ensures that you’ll always have welldefined exception and cancellation behaviours. Request exceptions are only everraised when calling either get_response or next_response, and any pendingrequests are cancelled on exiting the block.

Parallel requests with a Client

You can also call parallel() from a client instance, which allows you tocontrol the authentication or dispatch behaviour for all requests within theblock.

>>> client = httpx.Client()
>>> with client.parallel() as parallel:
>>>     ...

Async parallel requests

If you’re working within an async framework, then you’ll want to use a fullyasync API for making requests.

>>> client = httpx.AsyncClient()
>>> async with client.parallel() as parallel:
>>>     pending_one = await parallel.get('https://example.com/1')
>>>     pending_two = await parallel.get('https://example.com/2')
>>>     response_one = await pending_one.get_response()
>>>     response_two = await pending_two.get_response()

See the Async Client documentation for more details.