Torii experienced a minor incident on November 14, 2022 affecting Application Console and API Server, lasting 1d 2h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 14, 2022, 01:37 PM UTC
We are currently experiencing an outage with our Torii Application Console and API Server. Our Dev Team is actively investigating the issue and will continue to provide updates every 30 minutes.
- monitoring Nov 14, 2022, 02:04 PM UTC
Our Dev Team has identified the issue and deployed a fix. The Torii Application Console and API Server Outage are now operational, but we will continue to closely monitor the issue.
- monitoring Nov 14, 2022, 02:43 PM UTC
While closely monitoring the issue, our Dev Team identified an issue still occurring with the Torii Browser Extension. The browser extension is still actively collecting data (and no data will be lost), but it is currently not communicating with the Torii Application. Our Team is actively working on a fix. Once a fix is in place, all of the collected browser extension data will be synced with Torii accordingly. We will continue to provide updates.
- monitoring Nov 14, 2022, 03:20 PM UTC
We are currently experiencing an outage with our Torii Application Console and API Server. Our Dev Team is actively investigating the issue and will continue to provide updates every 30 minutes.
- monitoring Nov 14, 2022, 03:51 PM UTC
Our Dev Team continues to actively investigate an outage with our Torii Application Console and API Server. We will continue to provide updates every 30 minutes.
- monitoring Nov 14, 2022, 04:26 PM UTC
Our Dev Team continues to actively investigate an outage with our Torii Application Console and API Server. We will continue to provide updates every 30 minutes.
- monitoring Nov 14, 2022, 04:44 PM UTC
Our Torii Application Console is back up and operational. Our API Server continues to experience degraded performance with regard to the Torii Browser Extension. The browser extension is still actively collecting data (and no data will be lost), but it is currently not communicating with the Torii Application. Our Team is actively working on a fix. Once a fix is in place, all of the collected browser extension data will be synced with Torii accordingly. We will continue to provide updates.
- monitoring Nov 14, 2022, 06:16 PM UTC
Our Dev Team has determined that the fix for the Torii Browser Extension will have an unexpectedly longer ETA. The browser extension is still actively collecting data (and no data will be lost). Once the fix is in place, all of the collected browser extension data will be synced with Torii accordingly. We will provide another update at 5pm EST.
- monitoring Nov 14, 2022, 10:08 PM UTC
Our Dev Team is still working on the fix for the Torii Browser Extension. As they had informed earlier, the release of this fix will have a longer/unknown ETA. As a reminder, the browser extension is still actively collecting data (and no data will be lost). Once the fix is in place, all of the collected browser extension data will be synced with Torii accordingly. We will provide another update at 2 am EST.
- monitoring Nov 15, 2022, 06:16 AM UTC
Our Dev Team is monitoring the ingestion of collected browser extension data. We are seeing that most data is being successfully collected with some delays. A new version of the extension will be deployed in the next few days and will make sure all of the collected browser extension data is synced with Torii accordingly.
- monitoring Nov 15, 2022, 03:48 PM UTC
The API server and Application console are fully operational for the past 24 hours. Our team keeps monitoring the performance of the browser extension service.
- resolved Nov 15, 2022, 03:49 PM UTC
This incident has been resolved.