Benchmarking Priority Scoring Strategies: Risk-Weighted vs. Proximity vs. Entity-Tier #

This guide is part of the Priority Scoring Algorithms area within the Deadline Tracking & Routing Engines framework. It exists to settle an argument that recurs whenever a routing engine is capacity-constrained: given the same set of obligations and not enough hands to file them all before their deadlines, does it matter which single signal you sort by? The answer is measurable, and this page measures it — running three ordering strategies against one shared portfolio under one explicit metric, so the trade-off is a number rather than an opinion.

Scope of This Page #

This page is a controlled comparison, not a re-derivation. It takes three obligation-ordering strategies as given — risk-weighted (order by penalty exposure, largest first), deadline-proximity (order by soonest due date, the earliest-deadline-first heuristic), and entity-tier (order by blast radius, the most critical entity first) — and asks which one leaves the least money on the table when capacity runs out. It deliberately excludes the design of the composite scorer itself, which the Priority Scoring Algorithms guide already covers, and it excludes how penalty exposure is computed per jurisdiction, which is the job of Calculating Penalty Risk Scores Based on State Grace Periods. Here every obligation already carries a resolved due date, a modeled penalty figure, and an entity tier from Entity Taxonomy & Classification; the only question is the order those inputs get sorted into.

The Metric and the Constraint #

A benchmark is only meaningful if the objective is stated before the strategies run. The objective here is penalty dollars exposed under constrained-capacity dispatch: a team can file at most K obligations inside a planning window, and any obligation whose deadline falls inside that window but is not among the K dispatched accrues its full modeled penalty exposure. Obligations due after the window are deferred safely and cost nothing this cycle. Lower exposed dollars is better; the theoretical floor is the exposure an omniscient scheduler would leave by saving the K highest-dollar in-window obligations, which the harness computes as an oracle.

The dollar figures are not late fees alone. Delaware franchise tax accrues a $200 penalty plus 1.5% monthly interest under 8 Del. C. § 502 and § 510; a California Statement of Information carries a $250 penalty under Cal. Corp. Code § 2204 and puts corporate powers on a suspension path under § 2205; New York’s biennial statement under N.Y. BCL § 408 has no grace window; Texas franchise reports run under Tex. Tax Code § 171 with forfeiture of the right to transact business on the far side. Modeled penalty exposure folds the statutory penalty, accruing interest, and the downstream cost of losing good standing — a stalled financing, a blocked M&A signature — into one comparable number, because that is the quantity a constrained team is actually trading away when it picks what to file first.

Prerequisites #

• Python 3.10+ — for X | Y unions, Protocol, and the modern dataclass and typing surface used below.
• Standard library only: dataclasses, datetime, json, logging, typing. No third-party dependency is needed to reproduce the benchmark.
• A shared, frozen portfolio every strategy sees identically — the benchmark is worthless if the strategies are scored on different inputs.
• Upstream inputs already resolved: a timezone-neutral due_date from State Filing Deadline Calendars, a penalty_exposure from the grace-period scorer, and an entity_tier from entity taxonomy classification.

Implementation: One Portfolio, Three Strategies, One Harness #

The worked example is a single eight-obligation portfolio spanning Delaware, California, New York, and Texas, evaluated as of 2026-07-16 with a seven-day window (window_days = 6) and capacity for four filings (K = 4). Seven of the eight obligations fall inside the window; one — a regulated California advisory subsidiary — is not due for 45 days and is the deliberate trap that separates the strategies.

Because each strategy sorts by a different signal, each produces a different top-four dispatch set — and, crucially, a different first pick. Risk-weighted opens on the $8,500 Delaware parent; proximity opens on the California statement due today; entity-tier opens on the regulated subsidiary that is not due for six weeks. The matrix below shows every strategy’s ranking of every obligation side by side, with the dispatched cells shaded and the resulting penalty exposure totalled at the foot of each column.

The module below is the whole benchmark: the shared portfolio as frozen data, a common Strategy interface, the three concrete orderings, an oracle, and a harness that dispatches under capacity, scores each strategy on penalty dollars exposed, and emits a structured leaderboard. It runs with the standard library alone.

from __future__ import annotations

import json
import logging
from dataclasses import dataclass
from datetime import date
from typing import Callable, Protocol, Sequence

logging.basicConfig(level=logging.INFO, format="%(message)s")
logger = logging.getLogger("routing.priority_benchmark")


@dataclass(frozen=True)
class Obligation:
    obligation_id: str
    entity: str
    jurisdiction: str        # DE | CA | NY | TX
    due_date: date           # resolved upstream by the deadline calendar layer
    penalty_exposure: float  # modeled $ at risk: statutory penalty + interest + good-standing loss
    entity_tier: float       # blast-radius weight in [0, 1] from entity taxonomy


# ---- One shared portfolio: every strategy is scored on these identical inputs ----
AS_OF = date(2026, 7, 16)
PORTFOLIO: tuple[Obligation, ...] = (
    Obligation("O1", "Helix Holdings Parent",  "DE", date(2026, 7, 17), 8500.0, 0.9),
    Obligation("O2", "Coastal Operating LLC",  "CA", date(2026, 7, 16),  250.0, 0.4),
    Obligation("O3", "Meridian Realty NY",     "NY", date(2026, 7, 18),  500.0, 0.6),
    Obligation("O4", "Lone Star Fabrication",  "TX", date(2026, 7, 21), 1200.0, 0.5),
    Obligation("O5", "Dormant Holdco 7",       "DE", date(2026, 7, 20),  200.0, 0.1),
    Obligation("O6", "Regulated Advisory Sub", "CA", date(2026, 8, 30), 3000.0, 1.0),
    Obligation("O7", "Atlas Manufacturing",    "TX", date(2026, 7, 19), 4000.0, 0.8),
    Obligation("O8", "Bright Media NY",        "NY", date(2026, 7, 22),  600.0, 0.7),
)


class Strategy(Protocol):
    """Common interface: order the portfolio best-first. Pure and deterministic."""
    name: str

    def rank(self, obligations: Sequence[Obligation]) -> list[Obligation]: ...


@dataclass(frozen=True)
class SortStrategy:
    """A strategy expressed as one sort key over the shared obligation type."""
    name: str
    key: Callable[[Obligation], float]

    def rank(self, obligations: Sequence[Obligation]) -> list[Obligation]:
        # Stable sort keeps ties in portfolio order, so the benchmark reproduces bitwise.
        return sorted(obligations, key=self.key, reverse=True)


def _days_out(o: Obligation, as_of: date) -> int:
    return (o.due_date - as_of).days


def build_strategies(as_of: date) -> list[SortStrategy]:
    return [
        SortStrategy("risk_weighted", key=lambda o: o.penalty_exposure),
        # Proximity is earliest-deadline-first; negate days-out because SortStrategy sorts descending.
        SortStrategy("proximity", key=lambda o: -float(_days_out(o, as_of))),
        SortStrategy("entity_tier", key=lambda o: o.entity_tier),
    ]


def in_window(portfolio: Sequence[Obligation], as_of: date, window_days: int) -> list[Obligation]:
    """Obligations whose deadline falls inside the planning window — the ones actually at risk now."""
    return [o for o in portfolio if 0 <= _days_out(o, as_of) <= window_days]


def oracle_exposure(portfolio: Sequence[Obligation], as_of: date,
                    window_days: int, capacity: int) -> float:
    # Optimal for this metric: spend every slot on the highest-$ in-window obligation.
    at_risk = sorted(in_window(portfolio, as_of, window_days),
                     key=lambda o: o.penalty_exposure, reverse=True)
    return sum(o.penalty_exposure for o in at_risk[capacity:])  # what even the oracle must forfeit


@dataclass(frozen=True)
class BenchmarkResult:
    strategy: str
    dispatched: list[str]
    exposed: list[str]
    exposed_dollars: float
    regret_vs_oracle: float


def evaluate(strategy: Strategy, portfolio: Sequence[Obligation], as_of: date,
             window_days: int, capacity: int, oracle: float) -> BenchmarkResult:
    dispatched = strategy.rank(portfolio)[:capacity]        # constrained capacity: only K get filed
    dispatched_ids = {o.obligation_id for o in dispatched}
    at_risk = in_window(portfolio, as_of, window_days)
    # METRIC: penalty dollars exposed = in-window obligations we could not dispatch this window.
    exposed = [o for o in at_risk if o.obligation_id not in dispatched_ids]
    exposed_dollars = sum(o.penalty_exposure for o in exposed)

    logger.info(json.dumps({
        "event": "strategy_evaluated",
        "strategy": strategy.name,
        "top_pick": dispatched[0].obligation_id if dispatched else None,
        "dispatched": [o.obligation_id for o in dispatched],
        "exposed": [o.obligation_id for o in exposed],
        "exposed_dollars": round(exposed_dollars, 2),
        "regret_vs_oracle": round(exposed_dollars - oracle, 2),
    }))
    return BenchmarkResult(
        strategy.name,
        [o.obligation_id for o in dispatched],
        [o.obligation_id for o in exposed],
        exposed_dollars,
        exposed_dollars - oracle,
    )


def benchmark(portfolio: Sequence[Obligation] = PORTFOLIO, as_of: date = AS_OF,
              window_days: int = 6, capacity: int = 4) -> list[BenchmarkResult]:
    oracle = oracle_exposure(portfolio, as_of, window_days, capacity)
    results = [evaluate(s, portfolio, as_of, window_days, capacity, oracle)
               for s in build_strategies(as_of)]
    ranked = sorted(results, key=lambda r: r.exposed_dollars)  # lower exposure is the better strategy
    logger.info(json.dumps({
        "event": "benchmark_complete",
        "capacity": capacity, "window_days": window_days,
        "oracle_dollars": round(oracle, 2),
        "leaderboard": [(r.strategy, round(r.exposed_dollars, 2)) for r in ranked],
        "winner": ranked[0].strategy,
    }))
    return ranked


if __name__ == "__main__":
    for r in benchmark():
        print(f"{r.strategy:14s} exposed=${r.exposed_dollars:>7,.0f} "
              f"regret=${r.regret_vs_oracle:>7,.0f} dispatched={r.dispatched}")

Run it and the leaderboard is unambiguous on this portfolio: risk-weighted exposes $1,550, proximity $2,000, entity-tier $2,150, against an oracle floor of $950. None of the three single-signal strategies reaches the floor, and each misses it for a different, instructive reason.

Where each strategy fails. Risk-weighted comes closest because penalty dollars are the metric’s own currency — but it burns a slot on O6, the $3,000 regulated subsidiary, even though O6 is not due for 45 days and was never at risk this window; that wasted slot forfeits the $600 O8 filing it could have saved. Proximity is a clean earliest-deadline-first schedule and it never misses a soon obligation, but it is blind to magnitude: it spends two of its four slots on the $250 O2 and the $500 O3 while letting the $1,200 O4 and the $600 O8 lapse, because they happen to fall a day or two later. Entity-tier is the worst here — it leads with O6 purely on blast radius, protecting an entity with no imminent deadline and no large exposure, and in doing so strands the $1,200 O4 entirely. The oracle wins by combining both dimensions the single-signal strategies each ignore: it filters to in-window obligations first, then ranks them by dollars, which is precisely the shape of the composite scorer designed in Priority Scoring Algorithms.

Configuration Reference #

The benchmark’s verdict is a function of its parameters, so those parameters are data, not literals buried in the harness. Change them and the ranking can change — which is the point of running the benchmark against your portfolio rather than trusting this one.

Failure Modes and Fallback Routing #

The benchmark is a decision aid, not a dispatcher, and reading it wrong routes real filings badly. Each mode below maps to a concrete correction.

1. Reading a single-portfolio result as a universal ranking. Risk-weighted wins this portfolio; it does not win every portfolio. If your book is dominated by no-grace New York obligations under N.Y. BCL § 408, proximity and risk-weighted converge, and a portfolio thick with dormant shells rewards a heavier proximity weight. Re-run benchmark() on your own frozen portfolio before adopting any ordering as policy.
2. A stale or zeroed penalty_exposure skewing the metric. Because penalty dollars are the objective, one under-reported exposure silently demotes an obligation. Consume the figure from Calculating Penalty Risk Scores Based on State Grace Periods, never a cached copy, and fail an obligation with a missing exposure to maximum risk rather than zero so it cannot be quietly dropped from the top-K.
3. Capacity set to a planning target instead of true throughput. If K is optimistic, the benchmark under-states exposure and every strategy looks acceptable. Set K to demonstrated filings-per-window from the dispatch history in Multi-Entity Batch Orchestration, not to a target the team has never hit.
4. Adopting the winning single signal instead of the composite it points at. The honest takeaway is that no lone signal reaches the oracle; the oracle’s behavior — filter to the window, then rank by dollars, with entity tier as a tiebreak — is the composite scorer. Treat the benchmark as evidence for that weighted design, not as a mandate to hard-sort by penalty.

Frequently Asked Questions #

Related #

• Priority Scoring Algorithms
• Calculating Penalty Risk Scores Based on State Grace Periods
• Multi-Entity Batch Orchestration
• Entity Taxonomy & Classification
• Deadline Tracking & Routing Engines