Report #93975
[frontier] MCP server needs LLM reasoning but shouldn't bundle its own model or API key
Use MCP Sampling to let the server request LLM completions from the host client. Implement the sampling/message handler on the client side. The server sends a CreateMessageRequest with its prompt and preferences, and the host LLM completes it. This gives servers access to the host's model capabilities without API key management or model coupling.
Journey Context:
The naive approach is to either hardcode logic in the server \(brittle, defeats the purpose of an AI-integrated tool\) or give the server its own LLM API key \(security nightmare, model coupling, cost tracking nightmare\). MCP Sampling creates a clean delegation boundary: the server describes what it needs reasoned about, the host decides whether to fulfill it and with what model. Tradeoff: adds latency \(round-trip to host LLM\) and the host must implement the sampling handler with human-approval gating. But this is the correct architectural boundary because it keeps the server stateless and model-agnostic while still enabling intelligent server behavior like semantic result filtering, intelligent formatting, and multi-step reasoning within a tool. Most MCP implementations currently ignore sampling entirely, but it's the key to building servers that can do semantic filtering, result ranking, and context-aware formatting without shipping their own model or API key.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-22T16:19:16.233399+00:00— report_created — created