Report #92881
[synthesis] Why does improving our model sometimes make the product experience worse
Co-optimize model and UX as a coupled system. When you change the model, simultaneously A/B test the UX affordances. Track the closed loop: model quality → user behavior → input distribution → model quality. Never ship a model change without also reconsidering the UX, and never change the UX without also reconsidering what inputs the model will receive. Build a 'coupling dashboard' that shows model metrics and user behavior metrics side by side.
Journey Context:
In traditional software, backend and frontend are decoupled: you can improve the database without changing the UI. In AI products, the model and UX are tightly coupled through a feedback loop that doesn't exist in deterministic systems. The UX determines what users ask, which determines the input distribution the model sees, which determines model performance, which determines user satisfaction, which determines UX engagement. A 'better' model \(by offline metrics\) can change user behavior in ways that shift the input distribution to something the model handles worse. Users may start asking harder questions because the model seems smarter, pushing it into failure modes it rarely encountered before. The synthesis of Google's AI design patterns with Seldon's model analysis tools reveals that AI products have an emergent coupling property: the model and UX form a dynamical system where local improvements to either component can cause global degradation. This is why model improvements must be tested in the full product context, not just in offline evals — and why the instinct to 'just make the model better' can paradoxically make the product worse.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-22T14:29:21.611855+00:00— report_created — created