Filestack experienced a critical incident on February 10, 2021 affecting API and Conversions, lasting 5h 31m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Feb 10, 2021, 07:51 PM UTC
Our team is investigating an issue regarding abnormal traffic to the CDN. Once we have further information we will be updating the status of this incident.
- investigating Feb 10, 2021, 08:25 PM UTC
We are continuing to investigate network issues. We will be updating this page with more details as we have them.
- identified Feb 10, 2021, 09:17 PM UTC
AWS confirmed issue with their load balancers (https://status.aws.amazon.com/) that we are monitoring closely. In the meantime we have provisioned emergency load balancers cluster and directed CDN traffic through it. Delivery and transformations should be back to normal, however uploads are still affected.
- identified Feb 10, 2021, 09:46 PM UTC
US-EAST-1 uploads traffic was rerouted through our API servers in other regions. AWS confirmed that they have implemented fix and are deploying it to affected load balancers.
- identified Feb 10, 2021, 10:10 PM UTC
Uploads traffic in US-EAST-1 is back to normal. AWS is rolling out their fix to load balancers in affected zones. We continue serving traffic through our backup load balancers until we confirm that the fix is in place.
- monitoring Feb 10, 2021, 10:10 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Feb 11, 2021, 01:23 AM UTC
AWS engineers confirmed that their fix was fully rolled out in all affected availability zones. We have been closely monitoring our services for last couple hours and can confirm everything is back to normal. We have rerouted traffic back to our regular load balancers at AWS and will monitor them further.