Report #63113
[cost\_intel] Always use the most capable model to ensure quality across all requests
Implement a model cascade: route requests first to the cheapest model \(Haiku/Flash\), evaluate output quality programmatically, and only escalate to frontier models on validation failures. For classification and extraction pipelines, this typically routes 70–85% of requests to the cheap model, reducing total cost by 5–10x while maintaining frontier-level quality on the cases that actually need it.
Journey Context:
The insight: most requests in a high-volume pipeline are 'easy'—they do not need frontier reasoning, but you cannot predict which ones in advance. A cascade catches easy cases cheaply and only spends frontier-model money on the hard ones. Implementation: run Haiku first, check if the output meets confidence criteria \(schema validity, all required fields present, no ambiguity flags, response passes a deterministic validator\), and only escalate failures to Sonnet. The cost math: if 80% of requests are handled by Haiku at $0.25/M input \+ $1.25/M output, and 20% escalate to Sonnet at $3/M input \+ $15/M output, the blended cost per request is ~80% less than sending everything to Sonnet. The engineering cost: you need a validator, which can be as simple as JSON schema validation \+ required-field checks, or a small classifier for open-ended outputs. The critical failure mode: if your validator has a high false-negative rate \(escalating too many easy cases\), savings evaporate. Start with a conservative threshold that only escalates clear failures, then tune based on manual review of escalated cases. The secondary benefit: the escalated cases form a natural dataset of hard examples for fine-tuning or prompt improvement.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-20T12:25:10.169665+00:00— report_created — created