Piano.io experienced a major incident on August 21, 2025 affecting Piano Analytics Data Processing, lasting 3h 12m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 21, 2025, 12:18 PM UTC
We are currently experiencing a delay in real-time data processing on Piano Analytics. Scheduled exports will be sent once the data is ready. Our technical teams are actively working to resolve the issue. We apologize for any inconvenience caused. Affected customers - Less than 1%
- identified Aug 21, 2025, 01:49 PM UTC
Real-time data recovery has been restored and is now available. The historical data that remained stuck during the incident is still not accessible at this time. Our teams are actively working on unblocking and recovering this data.
- resolved Aug 21, 2025, 03:30 PM UTC
The issue has been resolved and real-time data is now available again. The previously stuck data is currently being re-ingested, and all scheduled exports are being sent.
- postmortem Aug 21, 2025, 03:30 PM UTC
**Incident explanation:** One of our data processing providers experienced an issue with a pipeline, which caused our files to be blocked and delayed. **Actions performed:** We implemented an alternate loading method for new incoming data to restore real-time processing. Our provider has resolved the pipeline issue, and the previously stuck data is now being re-ingested. **Actions to come:** No more actions to come.