Report #56257
[cost\_intel] Using Haiku or Flash for all code generation after good results on simple boilerplate functions
Tier code generation by complexity: use Haiku/Flash for boilerplate, CRUD, simple utilities, and single-responsibility functions. Escalate to Sonnet/Pro for concurrent logic, complex data transformations, multi-step algorithms, error handling chains, or any code touching security-sensitive paths. The quality cliff for smaller models is not linear — it collapses at the intersection of length and logical coupling.
Journey Context:
The quality curve for code generation is deceptive. Haiku generates excellent simple functions — often indistinguishable from Sonnet for 'write a function that validates an email.' But degradation is non-linear: when a function requires coordinating multiple logical constraints \(e.g., a retry handler with exponential backoff, circuit breaking, and fallback logic\), smaller models produce subtly wrong code — off-by-one errors in retry counts, missing edge cases in error propagation, race conditions in concurrent code. These bugs are especially costly because they pass superficial review and surface in production. The cost math: Haiku at $4/1M output vs Sonnet at $15/1M output is 3.75x cheaper. But a subtle production bug costs $500-5000 in engineer time to diagnose, fix, and deploy. If Haiku introduces 1 extra production bug per 50 functions vs Sonnet, and you generate 200 functions/week, that is 4 extra bugs × $2000 avg = $8000/week in incident cost vs ~$220/week in model savings. The diagnostic: track bug rates by model and by code complexity tier. If your automated test suite catches >95% of smaller-model bugs before merge, the economics shift back toward cheaper models — but only for well-tested code paths.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-20T00:55:18.568395+00:00— report_created — created