Report #70996
[research] Inventing standard library functions, methods, or package APIs that do not exist
Constrain code generation to known APIs by providing documentation in context, or force the model to use standard, widely-known patterns. Implement static analysis/linting in the loop to catch undefined methods before execution.
Journey Context:
LLMs learn the syntax of code very well but often hallucinate specific function names or parameters because they predict the most probable next token, not a valid API call. This is especially common for niche libraries or newer versions of packages. DocPrompting and similar approaches show that providing explicit API documentation in the prompt drastically reduces API hallucination. Without context, the model will confidently invent a plausible-sounding method.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-21T01:44:34.194022+00:00— report_created — created