HTTPX vs Requests vs AIOHTTP

HTTPX is a modern HTTP client for Python that aims to provide a more enjoyable and powerful experience for making HTTP requests. It was created to add support for asynchronous programming and has quickly gained popularity in the Python community due to its feature-rich API and good performance. In this article, we will compare HTTPX to two other popular HTTP clients for Python: Requests and AIOHTTP.

How HTTPX compares to the Requests module

Requests is a widely-used Python library, especially for making HTTP requests. Its simple API and long existence make it the go-to choice for many developers. To learn more about Requests, read our comprehensive guide to Python Requests Library.

HTTPX is a newer and more feature-rich HTTP client for Python than the popular Requests library. Here are some of the key differences between HTTPX and Requests:

• Async compatibility: HTTPX has both async and sync compatibility, which means you can use it in both asynchronous and synchronous programs. The Requests library, on the other hand, is only synchronous.

• HTTP/2 support: HTTPX supports HTTP/2, the latest HTTP protocol version. The Requests library does not have this feature.

• Automatic decoding: HTTPX includes automatic decoding of JSON and other common formats, which can save developers time and effort when working with responses that contain structured data. 

• Size: HTTPX is larger than Requests, which may be a consideration for applications where size is a concern.

• Performance: HTTPX has better performance compared to Requests in most cases.

Below are syntax differences between the two. The following code uses the Requests library to send a GET request:

import requests
response = requests.get("https://example.com")
print(response.text)

The equivalent code uses the HTTPX library:

import httpx
response = httpx.get("https://example.com")
print(response.text)

Alternatively, you can create an httpx client object and use its get method:

client = httpx.Client()
response = httpx.get("https://example.com")

If you want to send a POST request using the Requests library, use the post method instead of get as follows:

response = requests.post(url, data= {"name": "John", "age": 30} )

The syntax using HTTPX is very similar:

client = httpx.Client()
response = client.post("https://httpbin.org/post",data={"name": "John", "age": 30} )

The two libraries are used almost identically in simple synchronous situations. The important difference is that HTTPX allows you to write asynchronous code as follows:

import httpx
import asyncio

async def main():
    async with httpx.AsyncClient() as client:
        response = await client.get("https://example.com")
        print(response.text)

asyncio.run(main())

As you can see, the syntax for making a request with HTTPX is slightly more verbose due to Python's async/await syntax. However, this is a small price to pay for the ability to make asynchronous HTTP requests.

HTTPX also includes automatic decoding of JSON and other common formats, which can save developers time and effort when working with responses that include structured data.

See the following example:

import httpx
response = httpx.get("http://httpbin.org/get")
data = response.json()
print(data["headers"]["User-Agent"])

The site http://httpbin.org/get returns a JSON response.

Do note that there is a new version of the HTTP protocol called HTTP/2, which provides far more efficient transport and better performance. If you want to read more about HTTP/2, see this detailed document. Importantly, HTTPX offers support for HTTP/2 while Requests doesn’t.

To utilize HTTP/2 with the HTTPX library, first, install the optional HTTP/2 component using the pip command:

$ pip install 'httpx[http2]'

Next, set the optional http2 flag to True when you create an instance of the client as follows:

client = httpx.Client(http2=True)
response = client.get("https://example.com")
print(response.http_version) # Prints HTTP/2

This snippet prints the currently used HTTP protocol version, which in this case is HTTP/2.

Features that make HTTPX stand out 

Several features make HTTPX stand out as a powerful and flexible HTTP client for Python. These features include support for async/await syntax, sync compatibility, HTTP/2 protocol, and support from streaming response.

If you need to download large amounts of data, you may want to use streaming responses that don't load the entire response body into memory at once. Thankfully, HTTPX has built-in support for streaming responses.

It’s worth noting that the similarly popular library AIOHTTP, offers a plethora of valuable options as well.

An overview of AIOHTTP

AIOHTTP is a client for Python that is designed for use in asynchronous programs. Similarly to HTTPX, it’s built on top of the asyncio library and supports async/await syntax meaning AIOHTTP is a good choice for developers who are already familiar with asyncio and want a lightweight library for making HTTP requests. To find out more, visit the official website.

Like HTTPX, AIOHTTP supports standard HTTP methods. It also includes support for cookies, redirects, and custom headers. AIOHTTP is particularly well-suited for making HTTP requests in high-concurrency environments due to its async-first design and efficient use of resources.

Here is an example of making an asynchronous GET request using AIOHTTP:

import aiohttp
import asyncio

async def main(): 
    async with aiohttp.ClientSession() as session: 
            async with session.get("https://example.com") as response: 
                print(await response.text())

asyncio.run(main())

AIOHTTP also supports making asynchronous POST requests and other HTTP methods similarly. Here is an example of making a POST request with AIOHTTP:

import aiohttp
import asyncio

async def main(): 
    async with aiohttp.ClientSession() as session: 
            async with session.post("http://httpbin.org/post", data={"key": "value"}) as response: 
                print(await response.text())

asyncio.run(main())

If you want to learn more about AIOHTTP, we highly recommend reading Asynchronous Web Scraping with AIOHTTP.

HTTPX vs AIOHTTP

When comparing HTTPX and AIOHTTP, one of the main differences is the scope of their libraries. HTTPX is a full-featured HTTP client that can be used for a wide range of applications, while AIOHTTP is more focused on providing a simple and efficient API primarily for making HTTP requests in async programs.

Another difference is that HTTPX supports synchronous and asynchronous programming, while AIOHTTP is strictly an async library. Furthermore, AIOHTTP doesn’t merely send HTTP requests but also allows the creation of HTTP servers.

Usually, HTTPX can be used in a broader range of projects, while AIOHTTP may be a better choice if you are working on async-only projects and want a library optimized for async programming.

Overall, HTTPX and AIOHTTP are powerful and flexible HTTP clients for Python that have a multitude of features and are well-suited for different projects. However, there are a few key differences between the two libraries:

• Async compatibility: HTTPX has both async and sync compatibility, while AIOHTTP is async-only.

• HTTP/2 support: HTTPX supports HTTP/2, the latest HTTP protocol version. AIOHTTP does not have this feature.

• Performance: AIOHTTP is generally considered to have excellent performance compared to HTTPX. See the next section for a detailed analysis.

Notably, the choice between the two will depend on your specific needs and requirements.

Performance Comparison of AIOHTTP and HTTPX

To compare the performance of AIOHTTP and HTTPX, we can build a simple program that sends multiple asynchronous GET requests to a website. Here is an example of such a program:

import asyncio
import time
import httpx
import aiohttp

async def main():
    # Create clients for both the library
    httpx_client = httpx.AsyncClient()
    aiohttp_client = aiohttp.ClientSession()

    try:
        # Send 100 asynchronous GET requests using HTTPX
        start_time = time.perf_counter()
        tasks = [httpx_client.get("https://example.com") for _ in range(100)]
        await asyncio.gather(*tasks)
        end_time = time.perf_counter()
        print(f"HTTPX: {end_time - start_time:.2f} seconds")

        # Send 100 asynchronous GET requests using AIOHTTP
        start_time = time.perf_counter()
        tasks = [aiohttp_client.get("https://example.com") for _ in range(100)]
        await asyncio.gather(*tasks)
        end_time = time.perf_counter()
        print(f"AIOHTTP: {end_time - start_time:.2f} seconds")
    finally:
        # Close client sessions
        await aiohttp_client.close()
        await httpx_client.aclose()

asyncio.run(main())

Here are the results of running this code on a mac with an M1 processor and gigabit internet:

HTTPX: 1.22 seconds

AIOHTTP: 1.19 seconds

If we run the same code for 1000 requests, the numbers change significantly:

HTTPX: 10.22 seconds

AIOHTTP: 3.79 seconds

As we can see, AIOHTTP is faster than HTTPX in this case. However, it’s important to note that the performance difference between the two libraries may vary depending on the specific use case and hardware.

Comparison of HTTPX vs Requests vs AIOHTTP

The following table shows a summary of comparison of HTTPX vs Requests vs AIOHTTP :