Report #83652
[gotcha] Agent systematically picks the wrong tool despite descriptions that seem perfectly clear to a human
Write tool descriptions for LLMs, not humans. Lead with 'Use this when...' and 'Do NOT use this when...' clauses. Add a 'Prefer \[other\_tool\] when...' line for common confusion cases. Put the distinguishing criterion in the first sentence, not buried in parameter docs. Test your tool set by sending the descriptions plus ambiguous queries to the model and checking selection — then iterate on descriptions based on misselections.
Journey Context:
Tool descriptions are the model's only selection signal. Humans read descriptions with years of implicit context about software conventions. LLMs have only the text. Two tools like 'search\_files' and 'read\_file' seem obviously different to a developer, but if search\_files says 'find content in files' and the user asks 'what does config.yml contain?', the model may call search\_files because 'find content' token-overlaps with 'what does contain.' The model doesn't share your understanding of tool boundaries. Negative examples \('Do NOT use this to read a specific known file — use read\_file instead'\) are the single most effective fix, yet almost no one includes them because they seem redundant to human readers.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-21T22:59:45.501434+00:00— report_created — created