Report #13254
[bug\_fix] ERROR: deadlock detected
Enforce a strict global lock-ordering convention \(e.g., always lock rows by primary key ascending\) so cycles cannot form, and implement application-level retry logic that catches 40P01 errors with exponential backoff.
Journey Context:
Your inventory microservice deadlocks under load testing: two concurrent checkout transactions each try to decrement stock for items \{A, B\}. Transaction 1 locks A then wants B; Transaction 2 locked B then wants A. PostgreSQL detects the cycle after 1 second \(deadlock\_timeout\) and kills Transaction 2 with 'deadlock detected'. You initially think to increase deadlock\_timeout, but that just delays the pain. Analyzing pg\_stat\_database confirms deadlocks are rising with user count. The root cause is inconsistent lock ordering: different API endpoints lock resources in different orders. The fix requires refactoring all 'reserve inventory' paths to sort item IDs before locking \(SELECT ... WHERE id IN \(ids\) ORDER BY id FOR UPDATE\). This guarantees a global total order, mathematically preventing cycles. For the remaining edge cases \(foreign key checks causing page locks\), you add a @retry\_on\_deadlock decorator that catches 40P01, rolls back, sleeps 50ms\*2^attempt, and retries up to 3 times, ensuring transient deadlocks don't surface to users.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-16T18:15:36.372466+00:00— report_created — created