Thought Industries incident

EU Reporting

Resolved after 23 hours 5 minutes, on June 10, 2024.

resolved Jun 13, 2024, 09:35 PM UTC

On June 11, 2024, 11:35 AM UTC the EU reporting database experienced intermittent issues in data ingress, resulting in varied delays in reporting accuracy for EU customers until June 11, 2024, 3:25 PM UTC, after which data flow was confirmed as restored. The root cause of this incident was determined to be a faulty state in the internal jobs system which caused ETL jobs to hang without completing or exiting properly, blocking future job runs. No data inconsistencies should be present as of the resolution of this issue, and our infrastructure team is implementing several solutions to improve data availability and more accurately detect and escalate any future issues.

Started
Jun 10, 2024, 11:30 AM UTC
Resolved
Jun 11, 2024, 10:35 AM UTC
Duration
23h 5m
Detected by Pingoru
Jun 10, 2024, 11:30 AM UTC

Update timeline

  1. resolved Jun 13, 2024, 09:35 PM UTC

    On June 11, 2024, 11:35 AM UTC the EU reporting database experienced intermittent issues in data ingress, resulting in varied delays in reporting accuracy for EU customers until June 11, 2024, 3:25 PM UTC, after which data flow was confirmed as restored. The root cause of this incident was determined to be a faulty state in the internal jobs system which caused ETL jobs to hang without completing or exiting properly, blocking future job runs. No data inconsistencies should be present as of the resolution of this issue, and our infrastructure team is implementing several solutions to improve data availability and more accurately detect and escalate any future issues.

Your product runs on more than Thought Industries

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →