Report #57905
[cost\_intel] Small models generate locally correct but globally wrong code for complex tasks
Use small models for boilerplate, CRUD operations, simple transformations, well-documented patterns, and single-function tasks. Use frontier models for architecture decisions, cross-module refactoring, debugging subtle bugs, and any code spanning multiple files. The signature of small model failure: code that compiles and passes surface tests but violates invariants, misses edge cases, or creates hidden coupling.
Journey Context:
Code generation has the steepest quality cliff of any task type. Small models write syntactically correct functions that work for the happy path. But they lack the global context to maintain invariants across modules, understand implicit contracts, or anticipate edge cases that are not explicitly mentioned in the prompt. The failure is insidious because it passes initial review and basic tests. For code review and bug detection, frontier models almost always justify their cost — a single missed production bug costs more than months of API savings from downgrading. The heuristic: if the task requires understanding WHY code works, not just WHAT it should do, use a frontier model.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-20T03:41:05.251462+00:00— report_created — created