Flxpoint experienced a critical incident on June 13, 2023 affecting Automation, lasting 1h 42m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Jun 13, 2023, 07:49 PM UTC
Our Cloud Hosting Provider; AWS, is experiencing a major outage that is causing our automation processes to not trigger off as expected. We've confirmed this issue is being investigated by AWS and the latest update shows : "Jun 13 12:36 PM PDT We are continuing to experience increased error rates and latencies for multiple AWS Services... We have identified the root cause as an issue with AWS Lambda, and are actively working toward resolution... We are actively working on full mitigation and will continue to provide regular updates." Our APIs still remain functional but our automation jobs leverage AWS services like Lambda which is currently failing. We hope to see this resolved quickly by AWS and will update this incident as progress is made.
- monitoring Jun 13, 2023, 08:47 PM UTC
While the AWS incident is still in effect, we are starting to see improvements with our processes. All of our recently scheduled automation processes triggered properly as we'd expect. We've reduced the severity of this incident while we continue to monitor to confirm the issue has been fully resolved.
- resolved Jun 13, 2023, 09:31 PM UTC
AWS has resolved their incident and all of our systems appear operational now. Below you will find a timeline of the impact for your reference : ~3:00pm EST : Our automation schedules stopped triggering off jobs as expected and would intermittently fail when manually scheduled ~4:40pm EST : Automation processes now work properly if manually triggered ~5:30pm EST : Automation schedules are now picking back up and triggering off jobs during their expected time-windows