Resolved after 3 hours 30 minutes, on April 6, 2026.
The incident was caused by the number of concurrent active calls across tenants exceeding the capacity of a micro-service responsible for temporary call state storage. This situation was triggered by an unexpectedly high-volume campaign workload from a tenant, contributing to a significant increase in overall system load. While the underlying voice infrastructure had sufficient capacity to handle the traffic, the supporting micro-service reached its storage limit. This resulted in a processing blockage that prevented calls from being fully established. Corrective action plan: Migrate voice instance state storage from in-memory grain storage to blob-based storage.
Update timeline
- investigating Apr 06, 2026, 11:15 AM UTC
The System Reliability Engineering (SRE) team was alerted by the Geomant Helpdesk following a report from a partner that inbound calls were not reaching agents..
- investigating Apr 06, 2026, 11:52 AM UTC
After initial checks, SRE rebooted the first affected voice instance and performed test calls. The system did not recover. During the reboot, all Graia tenants were automatically failed over to the secondary instance..
- investigating Apr 06, 2026, 12:07 PM UTC
A broader system investigation observed an unusually high volume of outbound calls initiated by a campaign-based tenant, which was noted as a potential contributing factor and investigated further..
- investigating Apr 06, 2026, 12:40 PM UTC
Senior members of the development team were engaged for further investigation..
- identified Apr 06, 2026, 01:30 PM UTC
A blockage in the call processing flow was identified, and remediation steps were initiated.
- monitoring Apr 06, 2026, 01:43 PM UTC
The blockage was cleared, and calls began to be successfully processed. Monitoring continued..
- monitoring Apr 06, 2026, 01:57 PM UTC
The blockage reoccurred, again impacting call handling..
- monitoring Apr 06, 2026, 02:20 PM UTC
The underlying process responsible for the blockage was identified and stopped..
- resolved Apr 06, 2026, 02:45 PM UTC
This incident has been resolved..
- monitoring May 04, 2026, 09:37 AM UTC
The incident was caused by the number of concurrent active calls across tenants exceeding the capacity of a micro-service responsible for temporary call state storage. This situation was triggered by an unexpectedly high-volume campaign workload from a tenant, contributing to a significant increase in overall system load. While the underlying voice infrastructure had sufficient capacity to handle the traffic, the supporting micro-service reached its storage limit. This resulted in a processing blockage that prevented calls from being fully established. Corrective action plan: Migrate voice instance state storage from in-memory grain storage to blob-based storage.
Your product runs on more than Buzzeasy
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services