Grain experienced a major incident on June 2, 2026 affecting Grain Recorder, lasting 1d 1h. The incident has been resolved; the full update timeline is below.
Affected components
Grain Recorder
Update timeline
- investigating Jun 02, 2026, 03:20 PM UTC
Upstream provider is experiencing an outage causing out instability in our bot recorders including delayed joined times and recording failures. The team is investigating the scope and any potential mitigations.
- monitoring Jun 02, 2026, 06:29 PM UTC
The outage has subsided. The team has identified a set of recordings currently stuck in processing state that require manual recovery and will be working to recover them as quickly as possible.
- resolved Jun 03, 2026, 04:49 PM UTC
All recordings that were successfully captured have now been recovered. Future improvements around being more robust to upstream failures are being researched by the team.