Permutive experienced a major incident on December 15, 2021 affecting Permutive Analytics, lasting 2d 23h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Dec 15, 2021, 10:02 AM UTC
We have identified an issue where the insights reporting data is behind due to a backend technical outage. Our Engineering team is actively working on a fix to bring data up to date. We have confirmed that no data has been lost and all will be available once the processing catches up. We expect this to take approx 2-3 days and will provide additional updates as we progress toward resolution.
- monitoring Dec 15, 2021, 05:03 PM UTC
Our Engineering team has deployed a fix for this issue and data has begun to process into the insights tool. We estimate that it will take approximately 36 hours for all data to become available on the Platform. The data becomes live as soon as it is processed and we process it chronologically. This means that the delay in live data seen in the Insights too will diminish over this timeframe. We are continuing to monitor the situation and will provide additional updates as they become available.
- monitoring Dec 17, 2021, 04:48 PM UTC
Insights platform has now been stabilised. No data was lost during this incident. We are continuing to closely monitor and tune the system for a quick recovery. Data from the last 24hrs is being re-processed and the Insights dashboards will be up-to-date once again by the end of today.
- resolved Dec 18, 2021, 09:10 AM UTC
This incident has been resolved. Insights is once again showing current data. There has been no loss of data.