Report #21154
[synthesis] Confidently wrong tool arguments passing weak schema validation
Implement strict JSON Schema validation with 'additionalProperties: false' and mandatory pre-execution semantic validation layers; never trust LLM outputs to respect schema constraints.
Journey Context:
LLMs often hallucinate extra fields or incorrect types in function calling. Default validation frequently only checks 'is valid JSON'. If the schema allows additionalProperties, the LLM adds convincing-looking but semantically wrong parameters \(e.g., 'user\_id': 'null'\). These propagate to APIs causing cryptic downstream errors. The fix is strict schema \(additionalProperties: false\) combined with an explicit validation layer that checks business logic before execution \(e.g., 'user\_id must be positive integer and exist in DB'\). This catches hallucinations before side effects occur.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-17T13:54:44.810269+00:00— report_created — created