Async Python (asyncio)

After this lesson, you will be able to: Use asyncio + aiohttp/httpx for concurrent I/O; understand when async beats threads vs multiprocessing.

Async is how Python handles many concurrent I/O calls. Not faster for CPU work, different tool.

Prerequisites:Data Libraries

asyncio in 20 lines

async def + await are the keywords.

python

import asyncio
import httpx

async def fetch(url):
    async with httpx.AsyncClient() as client:
        r = await client.get(url, timeout=10)
        return r.text

async def main():
    urls = ["https://example.com", "https://example.org", "https://example.net"]
    results = await asyncio.gather(*(fetch(u) for u in urls))
    for u, html in zip(urls, results):
        print(u, len(html))

asyncio.run(main())
# All 3 fetches run concurrently; total time ≈ slowest one

Async vs threads vs multiprocessing

asyncio: many concurrent I/O calls (HTTP, DB) on one thread. Best for I/O-heavy workloads. threads (concurrent.futures.ThreadPoolExecutor): also I/O concurrency, but uses OS threads. GIL prevents real parallel CPU. multiprocessing (ProcessPoolExecutor): bypasses GIL for true parallel CPU work. Use for compute-heavy tasks. Rule: I/O-bound = async or threads. CPU-bound = multiprocessing.

Common patterns

Concurrency limit + timeouts + retries.

python

import asyncio, httpx

sem = asyncio.Semaphore(10)   # max 10 concurrent

async def fetch(url):
    async with sem:
        async with httpx.AsyncClient(timeout=10) as client:
            for attempt in range(3):
                try:
                    r = await client.get(url)
                    r.raise_for_status()
                    return r.text
                except httpx.HTTPError as e:
                    if attempt == 2: raise
                    await asyncio.sleep(2 ** attempt)   # exponential backoff

async def main():
    urls = ["..."] * 100
    results = await asyncio.gather(*(fetch(u) for u in urls), return_exceptions=True)
    successes = [r for r in results if not isinstance(r, Exception)]

asyncio.run(main())

Common mistakes only experienced async devs avoid

Calling sync code (requests.get, time.sleep) inside async. Blocks the event loop. Forgetting await. The coroutine doesn't run; you just have a reference. Mixing asyncio with threads naively. Use asyncio.to_thread() for blocking I/O. Async for CPU-heavy work. Use multiprocessing or rust extensions.

Quick Check

Your script makes 100 API calls sequentially and takes 200s. What's the simplest speedup?

Pick the canonical solution.

←Python for Data (pandas + NumPy)

Back to Python (Professional)

Packaging and Distribution→