Handling Rate Limits and Backoff When Polling Sensor APIs #

A rate limit is not an error condition; it is the contract. The failure mode worth designing against is not hitting the limit occasionally but spending your entire quota on rejected requests — a poller that retries immediately on a 429 collects less data than one that waits, because every rejected request consumes a slot that a successful one could have used. This guide builds a client that reads the limit from the response, governs itself below it, and backs off correctly when it is wrong. It is the throughput half of REST API polling and batch IoT ingestion; the pagination half is covered separately.

The Three Signals an API Gives You #

Most environmental data APIs — regulatory portals, commercial sensor clouds, weather services — expose some combination of three signals, and a robust client uses all of them.

Limit headers on every response. X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset (or RateLimit-* per the newer draft) tell you your budget before you exhaust it. Reading Remaining on each response and slowing down as it approaches zero is proactive; it means you never see a 429 at all under normal operation.

Retry-After on a 429 or 503. This is authoritative. When present, it overrides whatever your backoff algorithm would have chosen — the server knows when your window resets and your exponential curve does not. It comes as either a delay in seconds or an HTTP date, and clients that handle only the integer form break against the ones that send dates.

The 429 itself. Absent the other two, the status code alone is enough to drive an adaptive loop: slow down on rejection, speed up cautiously on a run of successes. This is the fallback, not the primary mechanism.

Production-Ready Implementation #

The governor is a token bucket, sized from the observed or documented limit. It is deliberately separate from the retry logic, because they solve different problems: the bucket prevents rejections, the backoff recovers from them.

# python 3.11 · httpx==0.27.0
from __future__ import annotations

import email.utils
import random
import time
from dataclasses import dataclass, field

import httpx

RETRYABLE = {408, 425, 429, 500, 502, 503, 504}


@dataclass
class TokenBucket:
    """Client-side governor: never issue more than `rate` requests per second."""

    rate: float                       # sustained requests per second
    capacity: float                   # burst size
    _tokens: float = field(default=0.0, init=False)
    _last: float = field(default_factory=time.monotonic, init=False)

    def __post_init__(self) -> None:
        self._tokens = self.capacity

    def take(self) -> None:
        """Block until a token is available, then consume it."""
        while True:
            now = time.monotonic()
            self._tokens = min(self.capacity, self._tokens + (now - self._last) * self.rate)
            self._last = now
            if self._tokens >= 1:
                self._tokens -= 1
                return
            time.sleep((1 - self._tokens) / self.rate)


def retry_after_seconds(response: httpx.Response) -> float | None:
    """Parse Retry-After in either of its two legal forms."""
    raw = response.headers.get("Retry-After")
    if raw is None:
        return None
    if raw.isdigit():
        return float(raw)
    parsed = email.utils.parsedate_to_datetime(raw)
    if parsed is None:
        return None
    return max(0.0, parsed.timestamp() - time.time())

The polling loop then combines the two, with full jitter on the backoff:

def get_with_backoff(
    client: httpx.Client,
    url: str,
    bucket: TokenBucket,
    *,
    params: dict | None = None,
    max_attempts: int = 6,
    base_delay: float = 1.0,
    max_delay: float = 120.0,
) -> httpx.Response:
    """GET with client-side rate governing and server-directed backoff.

    Raises on a non-retryable status immediately — a 400 will fail identically
    on every attempt, and retrying it only burns quota.
    """
    for attempt in range(max_attempts):
        bucket.take()
        response = client.get(url, params=params, timeout=30.0)

        if response.status_code < 400:
            _adapt(bucket, response)
            return response
        if response.status_code not in RETRYABLE:
            response.raise_for_status()

        server_wait = retry_after_seconds(response)
        if server_wait is not None:
            delay = server_wait                      # authoritative: use it verbatim
        else:
            ceiling = min(max_delay, base_delay * 2 ** attempt)
            delay = random.uniform(0, ceiling)       # full jitter
        time.sleep(delay)

    response.raise_for_status()
    return response


def _adapt(bucket: TokenBucket, response: httpx.Response) -> None:
    """Slow down as the remaining budget shrinks, using whatever headers exist."""
    remaining = response.headers.get("X-RateLimit-Remaining") or response.headers.get("RateLimit-Remaining")
    reset = response.headers.get("X-RateLimit-Reset") or response.headers.get("RateLimit-Reset")
    if remaining is None or reset is None:
        return
    try:
        left, window = float(remaining), max(1.0, float(reset))
    except ValueError:
        return
    bucket.rate = max(0.05, left / window * 0.9)     # aim at 90% of the safe rate

The _adapt step is what turns a static poller into one that survives a limit change. It reads the budget the server reports and re-targets the bucket at 90% of it, so a tightened quota shows up as slower polling rather than a wall of 429s.

Parameter Tuning Guide #

The third row points at the real fix for a strict limit: stop asking more often and start asking for more per request. An endpoint that accepts a time-window parameter turns ten requests per minute into one request per hour covering the same data, and the cursor pagination guide covers walking the result.

Verification and Testing #

Rate-limit handling is untestable against the real API — you cannot ask a vendor for a 429 on demand — so test against a stub that behaves badly on purpose.

# python 3.11 · httpx==0.27.0 · pytest==8.2.0 · respx==0.21.1
import httpx
import respx
import pytest


@respx.mock
def test_retry_after_header_is_obeyed_verbatim(monkeypatch):
    slept: list[float] = []
    monkeypatch.setattr(time, "sleep", slept.append)

    route = respx.get("https://api.example/readings")
    route.side_effect = [
        httpx.Response(429, headers={"Retry-After": "17"}),
        httpx.Response(200, json={"results": []}),
    ]

    bucket = TokenBucket(rate=100, capacity=100)     # bucket out of the way for this test
    r = get_with_backoff(httpx.Client(), "https://api.example/readings", bucket)

    assert r.status_code == 200
    assert slept == [17.0]                            # not an exponential guess


@respx.mock
def test_client_error_is_not_retried():
    respx.get("https://api.example/readings").mock(return_value=httpx.Response(422))
    bucket = TokenBucket(rate=100, capacity=100)
    with pytest.raises(httpx.HTTPStatusError):
        get_with_backoff(httpx.Client(), "https://api.example/readings", bucket, max_attempts=6)
    assert respx.calls.call_count == 1                # exactly one attempt

In production, the metric that tells you whether this is working is the ratio of 429 responses to total requests. Under a correctly sized bucket it should be near zero in steady state and spike only after the API changes its limits — at which point _adapt should bring it back down within a few minutes without intervention.

Gotchas #

Exponential backoff without jitter, across replicas. Two pollers that hit a limit at the same moment will retry at the same moments forever, because their delays are identical. Full jitter — sleeping a uniform random time between zero and the ceiling — decorrelates them.

Retrying a 401 as if it were transient. An expired token produces a 401 that no amount of waiting fixes; the retry loop turns an obvious authentication failure into a silent stall. Refresh the credential on 401 once, then fail.

Counting a 429 as a failed poll and re-fetching the same window. If the poller treats the rejection as “the window failed” and restarts it, it re-requests pages it already has. Track progress by cursor or window, not by attempt.

Ignoring Retry-After because the backoff already exists. The header is the server telling you exactly when its window resets. An exponential curve that guesses 4 seconds when the server said 60 will simply be rejected fourteen more times.

FAQ #

Related #

• REST API Polling and Batch IoT Ingestion — the polling ingestion stage this governs
• Cursor Pagination for Sensor History Endpoints — walking a large history feed once the request rate is under control
• Building a Local SQLite Fallback Buffer for Remote Sensors — where readings go when the upstream API is unavailable for hours