Zip experienced a major incident on April 21, 2024 affecting Zip Web App and Zip Requests and 1 more component, lasting 17h 27m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Apr 21, 2024, 06:00 PM UTC
During a routine database migration with no expected downtime, the Zip team found an issue with database replication. To prevent potential data inconsistency, the team decided to pause creating or updating any data in Zip. This includes any modification or writes such as: - The submission or updating of any Zip Request - Any changes to the Zip system - The ability for any logged-out users to log into Zip The engineering team is investigating and implementing changes to allow users to make changes and submissions in Zip once again, as its top priority.
- identified Apr 21, 2024, 06:00 PM UTC
We are continuing to work on a fix for this issue.
- identified Apr 22, 2024, 12:25 AM UTC
We are continuing to work on a fix for this issue.
- identified Apr 22, 2024, 05:51 AM UTC
We are continuing to work on a fix for this issue.
- identified Apr 22, 2024, 07:54 AM UTC
The team is in the final stages of mitigating the issue. This involves completing the re-indexing of a database table and deploying the new database. The Status Page will be updated with a more precise timeline as soon as possible.
- resolved Apr 22, 2024, 11:27 AM UTC
The team has fully implemented a resolution and will continue to monitor the situation to ensure no issues persist.