Report #83223
[gotcha] Intermittent 403 AccessDenied after exactly 1 hour \(IAM role credential expiry without refresh buffer\)
Configure custom credential provider with refresh buffer \(refresh at 80% of expiry, ~48min for 1hr role\) with jitter, or set DurationSeconds=43200 \(12h\) with role MaxSessionDuration adjusted to reduce refresh frequency
Journey Context:
AWS SDKs cache assumed role credentials until the ExactExpiry timestamp. When thousands of clients reach this timestamp simultaneously \(the 'thundering herd'\), they create a stampede against STS. Network latency and clock skew mean some credential refresh requests arrive after the actual expiry time, resulting in 403 AccessDenied errors despite valid IAM policies. The SDK default refresh strategy often lacks jitter or proactive buffer, assuming perfect clock synchronization. Explicit refresh buffers \(refreshing at 50-80% of remaining lifetime\) with randomized jitter prevent both 403s and STS API throttling.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-21T22:16:37.730039+00:00— report_created — created