Evalink Outage History

The previously observed API request issues have been resolved. Based on our investigation, the issue was likely related to the scheduled maintenance performed earlier. We are continuing to monitor the situation to ensure ongoing stability.

We're seeing infrastructure issues on some Farwest hosts and and working with them to get the service fully back up and running with full redundancy. Its currently affecting one of the two receiver sites and alarm transmission of dual path devices is still functional.

We've successfully repaired the data pipeline so that realtime replication is working nominal again - so the tables are all up-to-date with the latest data. We'll initiate a reload of the now missing data and give another update if the data is entirely correct again.

The issue was caused by a DNS problem at the underlying login provider, emergency login via https://talos.evalink.io/en/login2, remains possible. From what we can see the DNS started recovering around 10mins ago, and seems fully operation again by now. Only certain regions seemed to be affected. We're monitoring the situation to confirm that the system stays healthy.

We have identified the cause of the degraded performance affecting the device-service in Europe and have implemented mitigation measures. Platform performance has stabilized, and our team is continuing to monitor the service closely. Alarm transmission is not affected. We will provide a further update if necessary.

Between 12:12 and 12:15 CET one of our database clusters experienced sudden unavailability - we've very quickly noticed and remediated the issue with manual intervention and are now analysing the situation.

We're investigating briefly elevated API Latency in the EU region, which happened three (3) times for about 60s each on 08:55 and 09:30 CET, and seems to be caused by underlying infrastructure. We've deployed mitigation by increasing our compute capacity to spread the load about 20 minutes ago and are monitoring the situation.

We are continuing to monitor for any further issues.

We're seeing services starting to recover, Automated SMS seem to work again - Automated Calls and VoIP calls are still failing for the most part - we continue to monitor the situation.

Twilio Voice services have recovered, and seem to be operating normally again. SMS, Automated Calls and VoIP calls in evalink talos are all looking to be working as usual - we're still keep monitoring until the incident is fully resolved.