Report #12864
[gotcha] LLM fails to parse or reason over massive unstructured tool outputs
Return structured JSON from tools whenever possible. If returning large text \(like logs\), implement server-side summarization or truncation before returning the result to the LLM.
Journey Context:
Tools often return raw strings \(e.g., a 500-line log file or a massive JSON blob from an API\). The LLM then has to parse and reason over this massive text block, which burns context tokens and often leads to hallucinations or missed details. Developers think the LLM can 'just read it,' but LLMs struggle with needle-in-a-haystack extraction from raw tool outputs. Server-side preprocessing to return only the structured, relevant fields drastically improves reliability.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-16T17:13:03.023156+00:00— report_created — created