Flatfile incident

Intermittent Bad Gateway Errors

Major Resolved View vendor source →

Flatfile experienced a major incident on June 10, 2025 affecting Spaces and Dashboard, lasting 9m. The incident has been resolved; the full update timeline is below.

Started
Jun 10, 2025, 08:52 PM UTC
Resolved
Jun 10, 2025, 09:01 PM UTC
Duration
9m
Detected by Pingoru
Jun 10, 2025, 08:52 PM UTC

Affected components

SpacesDashboard

Update timeline

  1. investigating Jun 10, 2025, 08:52 PM UTC

    We are investigating an issue where some users are seeing bad gateway errors.

  2. monitoring Jun 10, 2025, 08:57 PM UTC

    A fix has been implemented and we are monitoring the results.

  3. resolved Jun 10, 2025, 09:01 PM UTC

    This incident has been resolved.

  4. postmortem Jun 10, 2025, 09:28 PM UTC

    # Incident Overview **Date and Time of Incident:** June 10, 2025 from approximately 1:50pm to 2:05pm PDT **Nature of Incident:** Requests routing to an unavailable frontend pod **Services Affected:** Flatfile Platform Dashboard and Spaces ## Details of the Incident At approximately 1:50pm PDT on June 10, the Flatfile Platform Dashboard reported intermittent 503 status codes. This was due to an infrastructure change made earlier in the day. As a consequence 1 in every 4 requests made to the Platform dashboard returned a HTTP 503 status code. The engineering team were alerted to the incident as soon as the 503’s started appearing and manually scaled up the infrastructure resources. ## Impact Assessment The incident was intermittent and affected 1 in 4 requests made by users of the Platform frontend applications for approximately 15 minutes. The API was unaffected. ## Root Cause The root cause was due to an infrastructure scaling configuration made in the early of the morning. ## Resolution Flatfile engineering manually provisioned extra capacity in order for the resources to get allocated appropriately. Following this, further improvements will be made with how we serve traffic to end users. ## Security and Data Integrity Please be assured that this incident did not compromise the security or integrity of your data. Our commitment to data protection remains a top priority.