Flatfile experienced a minor incident on June 4, 2025 affecting UK Spaces, lasting 6h 15m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jun 04, 2025, 11:30 AM UTC
We are currently investigating reports of failures of file uploads in our UK platform region
- identified Jun 04, 2025, 11:35 AM UTC
The issue has been identified and a fix has been rolled out
- monitoring Jun 04, 2025, 11:39 AM UTC
The fix has been fully rolled out, we are currently monitoring the platform.
- resolved Jun 04, 2025, 05:46 PM UTC
This incident has been resolved.
- postmortem Jun 13, 2025, 01:30 PM UTC
# Incident Overview **Date and Time of Incident:** June 4, 2025 from approximately 4:57 AM to 5:31 AM PDT **Nature of Incident:** Requests routing to an unavailable worker pod **Services Affected:** Flatfile Platform UK Regional Environment ## Details of the Incident At approximately 4:57 AM PDT \(11:57 AM BST\) on June 4th, Flatfile UK Regional environment infrastructure experienced intermittent failures on file uploads due to the backend worker fleet restarting. This was due to a misconfiguration on the backend worker infrastructure which created a hard dependency on an internal observability component used by FlatFile for monitoring. The internal observability component had experienced some downtime which led to backend worker fleet restarts. The infrastructure team were alerted to the incident and issued a patch to remove the hard dependency on the worker fleet. ## Impact Assessment The incident impacted only half of the total worker fleet due as the infrastructure safe guards kicked and rolled out extra workers in order to meet minimum capacity threshold of 50%. The UK regional environment experienced disruption for a total of 33 minutes. The API was unaffected. ## Root Cause The root cause was due to a misconfiguration on the infrastructure worker observability configuration made in a previous release 3 weeks ago. ## Resolution Flatfile infrastructure team fixed the miconfiguration on the worker infrastructure components by removing the hard depedency on Flatfile’s internal observability stack. ## Security and Data Integrity Please be assured that this incident did not compromise the security or integrity of your data. Our commitment to data protection remains a top priority.