Report #64537
[research] Model invents non-existent standard library functions or API methods that fit the desired logic but do not exist
Bind the agent to an official schema \(e.g., OpenAPI spec, TypeScript definitions\) and constrain generation to only valid tokens from the schema, or implement a REPL/sandbox execution check as a mandatory sub-step before returning code to the user.
Journey Context:
Code LLMs predict the most syntactically plausible next token. If a task requires a function that doesn't exist, the model will invent one that looks correct \(e.g., string.camelCase\(\)\). Prompting 'don't use non-existent functions' is useless because the model doesn't know the boundary of its training data. Only external grounding \(schemas\) or empirical validation \(execution\) catches this.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-20T14:48:48.034687+00:00— report_created — created