Cloud.gov incident

Intermittent 403 responses

Notice Resolved View vendor source →

Cloud.gov experienced a notice incident on August 21, 2025, lasting —. The incident has been resolved; the full update timeline is below.

Started
Aug 21, 2025, 04:38 PM UTC
Resolved
Aug 21, 2025, 03:00 PM UTC
Duration
Detected by Pingoru
Aug 21, 2025, 04:38 PM UTC

Update timeline

  1. resolved Aug 21, 2025, 04:38 PM UTC

    From approximately 11:22 AM ET to 11:45 AM ET, some customers may have experienced unexpected 403 responses for requests to anything hosted on the cloud.gov platform. The effect was not widespread, as only a small amount of all traffic to cloud.gov overall received 403 responses. The temporary 403 response rate spike was the result of a change to AWS WAF rules which was intended to block malicious traffic, but also had the side effect of blocking some legitimate traffic. As a result, we have disabled the WAF rule which caused the 403 errors and we will not re-enable it. Thanks for your patience. if you have any questions, please contact us at [email protected].

  2. postmortem Aug 21, 2025, 07:40 PM UTC

    ## Summary On August 21, 2025 from 11:22 AM ET to 11:45 AM ET, some [Cloud.gov](http://cloud.gov) customers experienced elevated 403 response rates when accessing hosted services. The issue was quickly identified and resolved, and service has remained stable since. ## Timeline * 11:22 AM - a configuration change was applied to traffic filtering rules. * 11:42 AM - Some customers report elevated 403 response rates * 11:44 AM - The configuration update was rolled back and error rates returned to normal. * 11:45 AM - Service confirmed stable following log analysis. ## Impact * Some customers experienced elevated 403 errors between 11:22 AM and 11:45 AM. * Not all customers were affected due to the nature of the filtering rules. * For many, the impact was limited or not observed at all. ## Resolution The traffic filtering rule updates were reverted, restoring normal service. Monitoring confirmed that the elevated error rates stopped after the rollback. ## Next Steps * Establish a more formal review process for changes to traffic filtering. * Define when and how customers should be notified of planned changes. * Evaluate technical approaches to reduce the potential impact of similar updates. Thank you for your patience. If you have any questions, please contact us at [[email protected]](mailto:[email protected]).