Currents experienced a minor incident on August 14, 2023, lasting 23h 52m. The incident has been resolved; the full update timeline is below.
Update timeline
- investigating Aug 14, 2023, 11:22 PM UTC
We are experiencing intermittent parallelization-related issues. Arbitrary can fail to report their results. https://currents.instatus.com .
- identified Aug 14, 2023, 11:36 PM UTC
The root cause has been identified - the backed Redis cluster exceeded the allowed network bandwidth limits, causing some requests to drop. Performing cluster reconfiguration..
- monitoring Aug 15, 2023, 12:14 AM UTC
Reconfiguring the Redis configuration was completed and the # of errors dropped to 0. Working on follow-up tasks to ensure better scalability. Monitoring the errors. .
- resolved Aug 15, 2023, 06:36 AM UTC
This incident has been resolved..
- identified Aug 15, 2023, 04:38 PM UTC
Reverting the status back to "Identified." Scaling up the cluster to accommodate the increased demand. The backed Redis cluster exceeded the allowed network bandwidth limits, causing some requests to drop. .
- monitoring Aug 15, 2023, 05:39 PM UTC
Scaling up Redis cluster completed. We are currently monitoring the system performance..
- resolved Aug 15, 2023, 11:15 PM UTC
This incident has been resolved..