Scalepoint experienced a major incident on March 5, 2025 affecting All (Entire system affected) and All (Entire system affected), lasting 6h 40m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 05, 2025, 10:48 AM UTC
We are currently experiencing issues with degraded performance in Settlement. What does this mean? Claimhandlers may be unable to complete work in Settlement. The issue is being investigated with the highest priority, and we will provide an update as soon as we have additional information to provide or the issue has been resolved.
- investigating Mar 05, 2025, 11:46 AM UTC
It seems there are issues with a global service which is affecting multiple Scalepoint systems beyong Settlement Tool. We are investigating issue at highest priority.
- investigating Mar 05, 2025, 12:35 PM UTC
Work continues at the highest priority, in attempt to resovle this incident. User will experience issues across all Scalepoint systems and workflows.
- investigating Mar 05, 2025, 01:08 PM UTC
We are still working to resolve the issues on all systems. No ETA on solution yet.
- monitoring Mar 05, 2025, 01:40 PM UTC
A workaround to circumvent access issues to systems is being implemented. We hope to resolve the issue as soon as possbile by implementing a permanent fix.
- monitoring Mar 05, 2025, 01:48 PM UTC
Many systems are starting to reboot but users may still experience degraded performance. Work is stil ongoing to ensure full system availability again.
- monitoring Mar 05, 2025, 02:38 PM UTC
The incident has been resolved.
- monitoring Mar 05, 2025, 03:42 PM UTC
Minor changes have been applied to ensure claim creation works in all setups.
- resolved Mar 05, 2025, 05:28 PM UTC
Several minor services have been identified was identified as affected, after initial fix. All have since been patched. Regular weekly deployments are now able to be executed tonight. We recommend you reach out to have tomorrow morning if any issues should arise.
- postmortem Mar 06, 2025, 10:34 AM UTC
**What happened?** At 11:05 a misconfiguration of our DNS service, resulted in a failed DNS lookup. This means, that the host address where a service, e.g. ClaimShop or CORE, is usually found, no longer pointed to the service. **How was it resolved?** Changes were reverted and a fix was implemented on Scalepoint infrastructure to establish connection to services again. Main issue was resolved 14:48. Minor supporting services were brought back online over time up until 17:00 where final service was fixed.