Trifacta incident

Performance degradation in UI and elevated number of 500 errors

Major Resolved View vendor source →

Trifacta experienced a major incident on May 5, 2022 affecting Dataprep by Trifacta, lasting 2d 2h. The incident has been resolved; the full update timeline is below.

Started
May 05, 2022, 04:19 PM UTC
Resolved
May 07, 2022, 06:20 PM UTC
Duration
2d 2h
Detected by Pingoru
May 05, 2022, 04:19 PM UTC

Affected components

Dataprep by Trifacta

Update timeline

  1. investigating May 05, 2022, 04:19 PM UTC

    We are seeing an increase in 500 errors and degradation in the performance of the UI for users. We are currently investigating this issue.

  2. investigating May 06, 2022, 01:23 PM UTC

    Google Cloud Platform has identified a problem and is working on a resolution. See https://status.cloud.google.com/incidents/4Qvmd4q81VnA9RirCMqV. This is affecting our services. The BigQuery API seems to be impacted as well. We are working toward identifying workarounds or mitigations.

  3. monitoring May 07, 2022, 01:08 AM UTC

    A fix for the UI performance issue and elevated 500 errors has been implemented. We are observing decreases in error rates in the affected services. We will continue monitoring the issue and will provide an update once we confirm the issues have been resolved.

  4. resolved May 07, 2022, 06:20 PM UTC

    On Friday, there was a service disruption at Google Cloud that affected some of our underlying services leading to performance degradation and instability for Trifacta. Google Cloud resolved these issues and our services are now operating as expected. We also detected other performance and stability issues due to increased load on our orchestration service. A fix was implemented on Friday afternoon and we have confirmed performance and service stability has been restored. We take incidents of this severity seriously and apologize for any impact this issue may have caused. If your workspace was impacted please reach out to our support team for additional information and updates.