Report #13939
[bug\_fix] GCP invalid\_grant: Invalid JWT - Token used too late or clock skew
Synchronize the system clock using NTP \(e.g., 'sudo ntpdate -s time.google.com' or enabling systemd-timesyncd\). The root cause is that JWT tokens include 'iat' \(issued at\) and 'exp' \(expiration\) claims validated against Google server time; if the client clock is skewed by more than a few minutes \(usually 5\), Google rejects the token as invalid.
Journey Context:
Developer runs a Python script using google-cloud-storage on a VM that was suspended overnight. The script crashes with 'google.auth.exceptions.RefreshError: invalid\_grant: Invalid JWT: Token must be a short-lived token \(60 minutes\) and in a reasonable timeframe'. Developer checks the service account key JSON - it's valid. They regenerate the key and get the same error. Suspecting a library bug, they search the error and find RFC 7523 references. A StackOverflow answer mentions system clock. Running 'date' shows the VM time is 15 minutes behind real time. Checking 'timedatectl' shows NTP sync is disabled. Enabling NTP sync updates the clock. Re-running the script succeeds because the JWT 'iat' claim now matches server time within acceptable bounds.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-16T20:15:16.019718+00:00— report_created — created