Exonet experienced a major incident on January 29, 2025, lasting –. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Jan 29, 2025, 12:28 PM UTC
Several servers on our platform went into a read-only state because a storage volume was full. At this moment it's unclear why that happened. The storage system did not indicate that the volume was (almost) full and a setting to automatically resize the volume was configured. After the volume was manually resized, we restarted the affected servers to prevent any further issues. We are currently investigating with our supplier how we can prevent this from happening in the future.
- postmortem Feb 04, 2025, 03:19 PM UTC
Together with our supplier we’ve identified a setting to adjust in our storage cluster to prevent this issue from happening in the future. We’ve also optimized our monitoring to identify any issues like these at an earlier stage.