Cloud.gov incident

Cannot create public s3 buckets

Major Resolved View vendor source →

Cloud.gov experienced a major incident on April 26, 2023 affecting Service - S3, lasting 7d 23h. The incident has been resolved; the full update timeline is below.

Started
Apr 26, 2023, 02:14 PM UTC
Resolved
May 04, 2023, 02:12 PM UTC
Duration
7d 23h
Detected by Pingoru
Apr 26, 2023, 02:14 PM UTC

Affected components

Service - S3

Update timeline

  1. identified Apr 26, 2023, 02:14 PM UTC

    Due to a change in the default configuration for new s3 buckets, cloud.gov users cannot currently create s3 buckets with the basic-public plan. Other plans are unaffected. The team has identified the issue and is implementing a fix.

  2. identified Apr 26, 2023, 02:15 PM UTC

    We are continuing to work on a fix for this issue.

  3. identified May 03, 2023, 04:03 PM UTC

    We are continuing to work on a fix for this issue.

  4. identified May 04, 2023, 01:59 PM UTC

    We are continuing to work on a fix for this issue.

  5. resolved May 04, 2023, 02:12 PM UTC

    The functionality to create publicly accessible S3 buckets using the basic-public and basic-public-sandbox plans in the S3 broker has been restored. The cause of the outage was a change made by AWS to create new S3 buckets with public access blocked by default. AWS also made a change to create new S3 buckets with ACLs disabled by setting object ownership to "BucketOwnerEnforced". You can read more details about these AWS changes in this blog post: https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/. To restore the ability to create public buckets using our S3 broker, we updated the broker to: - Delete the public access block when creating public buckets - Use the "ObjectWriter" setting for object ownership by default, so that ACLs can be used If you have any further issues with creating public S3 buckets using the broker, please contact us at [email protected].