Report #6135
[bug\_fix] ERROR: deadlock detected transaction rollback
Implement application-level retry logic with exponential backoff for 40P01 \(deadlock\_detected\) and 40001 \(serialization\_failure\) SQLSTATEs. Root cause is that concurrent transactions acquired locks on rows in different orders \(Transaction A locks row 1 then 2, Transaction B locks row 2 then 1\), creating a circular wait. PostgreSQL detects this cycle and aborts one transaction to break the deadlock.
Journey Context:
E-commerce checkout intermittently fails with 'ERROR: deadlock detected' in application logs. Investigation reveals two concurrent order updates touching inventory and orders tables. Analyzing query patterns shows one code path updates inventory first then orders, while another API endpoint updates orders first then inventory. Reproduce in staging by running two psql sessions simultaneously with explicit BEGIN and UPDATE statements in opposite orders; deadlock occurs in less than one second. Consider reordering all queries to use consistent table order, but legacy code makes this difficult and risky. Implement a retry decorator in Python using the tenacity library, specifically catching sqlalchemy.exc.OperationalError with deadlock detection. Configure 3 retries with exponential backoff starting at 100ms, maximum 2 seconds. Deploy to production and monitor - deadlock errors disappear from user-facing responses, appearing only in metrics as successful retries. Checkout completion rate returns to 100%.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-15T23:14:12.833644+00:00— report_created — created