Learnosity incident
Investigating possible degraded performance affecting some asynchronous jobs in the US-West-2 Region
Learnosity experienced a minor incident on April 25, 2024 affecting AMER Data Centric (Summary) and Creation of Item Pools, lasting 1h 17m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Apr 25, 2024, 08:43 PM UTC
As of 20:30 UTC, we are investigating a possible issue affecting the processing of item pool and offline package jobs in the US-West-2 region and causing slower than normal processing times. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.
- identified Apr 25, 2024, 09:19 PM UTC
As of 21:15 UTC, the Learnosity Systems Engineering team has identified the issue and is making adjustments to restore normal jobs performance. Learnosity Support and Systems Engineering teams are continuing to work on the issue, and will follow on with an update and resolution as soon as possible.
- monitoring Apr 25, 2024, 09:30 PM UTC
As of 21:26 UTC, the jobs processing performance has returned to normal and all job queues are empty. Everything is currently performing as expected, we will continue to monitor for a time to ensure a full recovery before fully resolving the incident.
- resolved Apr 25, 2024, 10:00 PM UTC
As of 21:56 UTC, all jobs continue to be processed at a normal rate and all services are operational. We are marking the issue affecting the processing of item pool and offline package jobs in US-West-2 as resolved. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalised any next steps or preventative measures required. Please reach out if you have any questions or concerns.