Freckle experienced a major incident on March 14, 2022 affecting Student and Teacher and 1 more component, lasting 1h 4m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 14, 2022, 07:47 PM UTC
At approximately 3:26 ET, our primary database experienced a crash and recovery. The database is back online and downstream services are recovering, but we are still investigating the cause.
- resolved Mar 14, 2022, 08:51 PM UTC
Our systems are recovered and operating normally. We are still looking into the root cause and have escalated the issue to our database vendor for further investigation, but at this time are no longer considering this an active incident.
- postmortem Mar 15, 2022, 02:24 PM UTC
Our database vendor notified us that an underlying hardware failure required our database to be moved to another server. Such failures are extremely rare, but they do happen. We will continue to evaluate options which could improve our resiliency against hardware failures.