Prismatic had 10 outages in the last 2 years totaling 9h 43m of downtime — averaging 0.4 incidents per month.
There were 10 Prismatic outages since October 7, 2024 totaling 9h 43m of downtime. Each is summarised below — incident details, duration, and resolution information.
Between 20:04 and 23:20 UTC on August 17, the OAuth2 service in eu-west-1 experienced intermittent periods of token refresh failures. The issue was corrected the same night and refreshes resumed normally.
A routine update caused a service interruption of the execution pipeline for some customers for approximately 5 minutes. A rollback was performed and service was restored. The Prismatic team is investigating the root cause and will ensure a subsequent update does not contain the same regression.
We experienced an outage impacting the Web App and Embedded UI in the GovCloud environment. The incident began at 9:19 a.m. CT on June 14, 2025 and was fully resolved at 8:02 a.m. CT on June 15, 2025. During the outage, users were unable to access the web app or embedded screens. Integration executions continued to run as expected and API access was not impacted. Commercial Cloud environments were not impacted.
At approximately 7:00AM Arizona the Prismatic team received several alert notifications for failed executions. Upon investigation it was discovered that a very large volume of executions had started, and autoscaling was not configured with a high enough upper threshold to accommodate. The Prismatic team took action to raise the thresholds and align them with other high-volume stacks, and service was fully restored by 7:20AM Arizona. We will audit and address any similar situations we discover on other high-volume stacks.
At approximately 1:47 PST the Prismatic team received an alert for an increase in error rates in the execution pipeline. We have started a rollback procedure that we expect will resolve the issue, and then we will address the underlying root cause.
We have completed rollback of all stacks the execution pipeline is back to fully operational status. We will continue our investigation to determine root cause and will keep this incident updated as we progress.
We are continuing to investigate this issue.
We have identified the root issue and are working on a fix.
This incident has been resolved.
A regression was introduced in the Embedded system that caused a Major disruption in service, specifically for organizations that use the i18n features. The issue was identified and a fix has been deployed. Prismatic is now confirming with impacted customers that the issue is resolved.
A fix has been implemented and we are monitoring the results.
Confirmed back to fully operational status.
At approximately 8:57am CT an update to the Prismatic web application was deployed that contained updated dependencies. Unfortunately there was a misconfiguration that was not problematic in development and CI environments, but caused crashes in the production environments. The Prismatic engineering team received alerts immediately upon release that there were problems and immediately started rolling back to the previous version. Full functionality was restored by 9:30am CT. The Prismatic engineering team is investigating root cause and will properly address this issue and further investigate how to improve the test coverage so as to minimize the likelihood and impact of similar situations in the future.
We are currently investigating an issue affecting the production database in the US-EAST-2 region. Performance has been degraded and has affected some executions as well as the ability to use the embedded marketplace and related services. The Prismatic team is actively investigating the issue and will provide continuous updates as we work towards resolution.
We have identified and mitigated the issue affecting the database in the US-EAST-2 region. All metrics have returned to normal operating levels. We will continue to monitor the situation.
This incident has been resolved.
The Prismatic team started seeing increased error rates in multiple services starting at 13:27 Mountain. The errors appear to originate in the underlying AWS services. We have reached out to our contacts at AWS for more information and to bring increased awareness to this issue. We will continue to monitor the situation and work towards resolution.
We are continuing to investigate this issue.
We continue to monitor the situation.
AWS has confirmed an issue with S3 and related services in their us-east-2 region. They are working towards resolution. We will continue to monitor the situation and provide updates as they are available.
It appears that the relevant issues have been identified and addressed by the AWS teams. We will continue to monitor for new additional problems and provide updates until we are confident that operation has been completely restored.
It appears the issue has been completely resolved and AWS is working to understand the root issue(s). We will open a new issue on this status page if any further problems emerge.