KnowledgeOwl experienced a critical incident on March 24, 2021 affecting Knowledge Bases and Web Application and 1 more component, lasting 5h 3m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 24, 2021, 11:26 AM UTC
We are currently investigating an issue with KnowledgeOwl. This affects both the knowledge base software, and published knowledge bases.
- monitoring Mar 24, 2021, 02:14 PM UTC
We've implemented a fix. Most traffic to knowledge bases, the KnowledgeOwl web application, and API should be unaffected, but we are continuing to monitor and tweak things.
- monitoring Mar 24, 2021, 03:13 PM UTC
We've rolled out an additional set of fixes that seem to have resolved issues for customers who were reporting issues. We'll continue to monitor for further alerts or issues.
- resolved Mar 24, 2021, 04:30 PM UTC
Sorry for interruption today, and thank you to all the customers who reported issues and for your patience while we got things sorted. One of our internal server monitoring systems went down and failed to alert us to an issue this morning. Because we did not receive the alert, it took us longer than normal to diagnose and fix the problem. We've since found and fixed the issue to restore service. To prevent this in the future, we are looking to improve our monitoring and notifications.
- postmortem Mar 24, 2021, 04:51 PM UTC
## Incident postmortem One of our internal server monitoring systems went down and failed to alert us to an issue this morning. Because we did not receive the alert, it took us longer than normal to diagnose and fix the problem. We've since found and fixed the issue to restore service. ## Next steps To prevent this in the future, we are looking to improve our monitoring and notifications.