Tuning Poll Intervals for California BizFile Rate Limits #

This guide sits inside the Async Polling & Rate Limiting area of the Secretary of State Portal & API Ingestion framework, and it answers one narrow operational question: at what cadence should a worker poll California’s bizfile portal so that a full Statement of Information sweep clears without the portal throttling you into a stall? The parent area installs one token bucket per jurisdiction with a static refill rate. California is the jurisdiction where a static rate is not good enough — bizfile’s effective ceiling drifts across the day and collapses during renewal season — so this page replaces the fixed refill with a closed-loop controller that discovers the ceiling by probing it.

What This Page Covers #

The subject here is the steady-state pacing of a bulk poll against a single portal whose rate limit is undocumented and time-varying. It develops an additive-increase / multiplicative-decrease (AIMD) controller that treats the poll rate as the controlled variable and derives the sleep interval as its reciprocal, so the sweep continuously converges toward the highest cadence bizfile will tolerate at that moment. It deliberately excludes two adjacent concerns handled elsewhere: honouring a portal’s explicit Retry-After on a single retryable request, which belongs to Implementing Exponential Backoff for Secretary of State APIs, and deciding whether a given response is retryable at all, which belongs to the Error Categorization & Retry Logic taxonomy. This page assumes those are already in place and concerns itself only with how fast to go between requests.

The Suspension Clock Behind a BizFile 429 #

A 429 from bizfile is not a throughput inconvenience; it is a delay imposed on a request that is checking whether a California entity has met a statutory filing duty. The biennial (or, for many domestic stock corporations, annual) Statement of Information is required under Cal. Corp. Code § 1502, and an entity that fails to file faces a $250 penalty and, more consequentially, suspension or forfeiture of corporate powers under Cal. Corp. Code § 2205. A sweep that verifies filing status across a portfolio therefore runs against a hard consequence date: if throttling slows the sweep so much that some entities are never checked before their suspension exposure crystallises, the pacing algorithm has caused the very harm it exists to prevent. The controller below is bounded on both sides for exactly this reason — a floor rate that guarantees forward progress, and a completion guard that escalates the remainder of the queue the moment the current cadence can no longer finish in the time left.

Prerequisites #

• Python 3.10+ for match statements, X | Y unions, and modern asyncio.
• httpx 0.27+ for an async client with connection pooling and typed responses.
• Standard library asyncio, json, logging, time, dataclasses, enum, and datetime for the control loop and structured logging.
• A resolved list of California entity_id values to sweep and a UTC completion deadline derived from the entity’s § 1502 due date via the State Filing Deadline Calendars layer.
• An already-classified response contract: this controller reacts to a raw 429 as a throttle signal but leaves richer classification (CAPTCHA, empty body, schema drift) to the parent area’s error taxonomy.

Implementation: An AIMD Poll-Interval Controller #

The controlled quantity is the poll rate r in requests per second; the sleep interval is simply 1 / r. A window of clean polls nudges r upward by a fixed additive step — a gentle probe for more headroom — while a single 429 in the window halves r. Because the interval is the reciprocal of the rate, that asymmetry produces the classic AIMD signature: the interval eases down linearly as the sweep earns throughput and jumps up geometrically the instant bizfile pushes back. The result is a sawtooth that hovers just under the portal’s live ceiling instead of guessing at a safe constant. The diagram traces one control cycle.

The module is a single end-to-end controller. AimdConfig carries the tuning constants; WindowStats accumulates a control window; BizFileIntervalController owns the rate, exposes the derived interval, and drives an async sweep that adapts as it goes. The completion guard is the compliance-critical branch — it is what keeps the algorithm from politely crawling an entity past its § 2205 exposure.

from __future__ import annotations

import asyncio
import json
import logging
import time
from dataclasses import dataclass, field
from datetime import datetime, timezone
from enum import Enum
from typing import AsyncIterator, Iterable

import httpx

# Structured JSON logging — each control decision is a replayable pacing event.
logging.basicConfig(level=logging.INFO, format="%(message)s")
logger = logging.getLogger("ingestion.bizfile.aimd")


class WindowVerdict(str, Enum):
    CLEAN = "clean"          # no throttle in the window — safe to probe faster
    THROTTLED = "throttled"  # at least one 429 — back the rate off hard


@dataclass(frozen=True)
class AimdConfig:
    start_rate: float = 0.25   # cold-start at 1 poll / 4s; bizfile tolerates this from cold
    r_min: float = 0.10        # floor: never slower than 1 poll / 10s or the sweep never finishes
    r_max: float = 0.60        # ceiling: do not probe past ~1 poll / 1.7s even when clean
    ai_step: float = 0.02      # additive increase: +0.02 req/s per clean window (gentle probe)
    md_factor: float = 0.50    # multiplicative decrease: halve the rate on any 429 (sharp retreat)
    window: int = 20           # polls per control window before an adjustment is applied


@dataclass
class WindowStats:
    ok: int = 0
    throttled: int = 0
    started: float = field(default_factory=time.monotonic)

    def effective_rate(self) -> float:
        elapsed = max(1e-6, time.monotonic() - self.started)
        return (self.ok + self.throttled) / elapsed


class BizFileIntervalController:
    """AIMD controller for the poll interval against California's bizfile portal.

    The controlled variable is the poll rate r (req/s); the interval is 1/r. A clean
    window applies additive increase (probe for headroom); any 429 applies multiplicative
    decrease (retreat). The interval therefore eases down linearly and jumps up geometrically.
    """

    def __init__(self, client: httpx.AsyncClient, cfg: AimdConfig | None = None) -> None:
        self._client = client
        self._cfg = cfg or AimdConfig()
        self._rate = self._cfg.start_rate
        self._stats = WindowStats()

    @property
    def interval(self) -> float:
        return 1.0 / self._rate

    def _adjust(self, verdict: WindowVerdict) -> None:
        prev = self._rate
        if verdict is WindowVerdict.THROTTLED:
            self._rate = max(self._cfg.r_min, self._rate * self._cfg.md_factor)
        else:
            self._rate = min(self._cfg.r_max, self._rate + self._cfg.ai_step)
        logger.info(json.dumps({
            "event": "aimd_adjust", "verdict": verdict.value,
            "rate_prev": round(prev, 4), "rate_next": round(self._rate, 4),
            "interval_s": round(1.0 / self._rate, 3),
            "effective_rps": round(self._stats.effective_rate(), 4),
        }))
        self._stats = WindowStats()  # reset accumulation for the next window

    async def _poll_one(self, entity_id: str) -> WindowVerdict:
        url = f"https://bizfileonline.sos.ca.gov/api/entities/{entity_id}/soi-status"
        resp = await self._client.get(url, timeout=httpx.Timeout(15.0))
        if resp.status_code == 429:
            # A 429 is a pacing signal on a Cal. Corp. Code s.1502 status check, not a data verdict.
            self._stats.throttled += 1
            logger.warning(json.dumps({
                "event": "bizfile_throttle", "entity_id": entity_id,
                "retry_after": resp.headers.get("Retry-After"),
            }))
            return WindowVerdict.THROTTLED
        resp.raise_for_status()  # non-429 errors are the error-taxonomy's problem, not the pacer's
        self._stats.ok += 1
        return WindowVerdict.CLEAN

    async def sweep(
        self, entity_ids: Iterable[str], deadline_utc: datetime,
    ) -> AsyncIterator[tuple[str, str]]:
        """Poll every entity's SOI status, adapting the interval, before the s.2205 suspension clock."""
        queue = list(entity_ids)
        total = len(queue)
        window_verdict = WindowVerdict.CLEAN
        polled = 0

        for idx, entity_id in enumerate(queue):
            remaining = total - idx
            secs_left = (deadline_utc - datetime.now(timezone.utc)).total_seconds()
            # COMPLIANCE GUARD (Cal. Corp. Code s.2205): if the current cadence can no longer
            # clear the remaining checks before suspension exposure, stop crawling and escalate.
            if secs_left > 0 and remaining * self.interval > secs_left:
                logger.error(json.dumps({
                    "event": "sweep_deadline_risk", "remaining": remaining,
                    "interval_s": round(self.interval, 3), "secs_left": round(secs_left, 1),
                }))
                yield entity_id, "escalate"
                continue

            verdict = await self._poll_one(entity_id)
            if verdict is WindowVerdict.THROTTLED:
                window_verdict = WindowVerdict.THROTTLED  # one 429 taints the whole window
            polled += 1

            if polled >= self._cfg.window:
                self._adjust(window_verdict)
                window_verdict, polled = WindowVerdict.CLEAN, 0

            yield entity_id, verdict.value
            await asyncio.sleep(self.interval)  # the adapted pace, applied between polls


async def run_soi_sweep(entity_ids: list[str], deadline_utc: datetime) -> None:
    async with httpx.AsyncClient() as client:
        controller = BizFileIntervalController(client)
        async for entity_id, outcome in controller.sweep(entity_ids, deadline_utc):
            logger.info(json.dumps({"event": "soi_result", "entity_id": entity_id, "outcome": outcome}))

Two design choices are worth calling out. First, a single 429 taints the entire window rather than being averaged against the successes in it: throttling is a cliff, not a gradient, so the controller treats any evidence of the cliff as reason to retreat. Second, the guard uses the current interval as its estimate of future pace. That is intentionally pessimistic near a deadline — if the last window was throttled, the interval is already inflated, and the guard escalates sooner rather than betting on a recovery that may not come.

Configuration Reference #

Every tuning constant is a portal-behaviour or statutory decision, not an implementation detail, so each is externalised and justified.

Failure Modes and Fallback Routing #

Each failure maps onto the four-tier scheme in the parent area’s Error Categorization & Retry Logic — transient, statutory, data-validation, and system — and the controller responds to each differently.

1. Persistent throttling that drives the rate to r_min (transient, deadline-threatening). bizfile is genuinely saturated; multiplicative decrease has bottomed out at the floor. The controller does not go below r_min, because a slower cadence would guarantee the sweep misses its § 2205 window. If the completion guard then fires, the remaining entities are emitted as escalate and routed to a low-concurrency priority channel or diverted to the backoff engine for individual, Retry-After-honouring handling.
2. CAPTCHA or bot-mitigation interstitial on a burst (system / portal-blocked). bizfile is notorious for serving a challenge page under load, often with a 200 status. That is not a pacing signal and this controller must not read it as a clean window; the parent taxonomy classifies it as PORTAL_BLOCKED and hands it to the headless browser fallback chain rather than letting the AIMD loop speed up into a wall.
3. Empty or schema-invalid 200 body (data-validation). A 200 with no usable SOI payload counts as a successful request for pacing purposes but a failed observation. The controller records it as clean (it earned throughput), while the calling layer must still reject the body against its compliance metadata schemas before treating the entity as verified — never resolve status on an empty body.
4. A stale or oscillating ceiling (statutory-adjacent, tuning). If bizfile’s real limit swings every few minutes, a long window averages across two regimes and the controller lags the ceiling in both directions. Shorten window and lower ai_step so the sawtooth tracks the moving limit more tightly, and confirm the completion guard still leaves enough margin for the worst-case cadence before the deadline.

Frequently Asked Questions #

Related #

• Async Polling & Rate Limiting
• Implementing Exponential Backoff for Secretary of State APIs
• Error Categorization & Retry Logic
• State Filing Deadline Calendars
• Secretary of State Portal & API Ingestion