Nebula experienced a minor incident on January 24, 2025 affecting Core Network and Dashboard and 1 more component, lasting 2d 19h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 24, 2025, 12:52 PM UTC
Our engineers are currently investigating reports of issues with apps and dashboards loading, and call connection. We will provide next update within 20 minutes.
- investigating Jan 24, 2025, 12:53 PM UTC
We are continuing to investigate this issue.
- identified Jan 24, 2025, 01:01 PM UTC
The issue has been identified and we're beginning to see traffic return to normal levels. We will continue to monitor and confirm once satisfied the incident is resolved.
- monitoring Jan 24, 2025, 02:11 PM UTC
All services had resumed normal operations at 13:06pm today. Our teams are continuing to monitor all areas of the platform and will mark this incident as resolved following successful review over the weekend.
- resolved Jan 27, 2025, 08:42 AM UTC
After a successful period of monitoring over the weekend our engineers are satisfied that this incident is resolved.
- postmortem Jan 27, 2025, 10:49 AM UTC
On January 24th at 12:40pm we rolled out a security patch which was needed to address a newly identified vulnerability with one of our provisioning services. This affected only our supporting systems such as the website and dashboard, and took around 8 minutes to complete, with a few minutes thereafter to clear the backlog. During this time there was no significant impact on our telephony system, although some customers did report difficulty initiating new calls due to the authentication process.