Mio Outage History

Following recent work to improve platform scalability, an additional optimization did not perform as expected and generated significantly higher traffic than anticipated. This created a processing backlog and caused some messages to be delivered late. We have increased processing capacity, are working through the backlog, and are addressing the behavior that caused the unexpected traffic. We continue to monitor the platform closely.

At approximately 9:00 AM UTC on August 10th, Mio experienced a brief service outage. Service has since been restored. We are currently processing the backlog of events in the order they occurred. As a result, end users may experience elevated message latency while the queue is worked through. No messages moving forward have been lost — this is a processing delay only. We will post another update once the event queue has been fully processed and confirm when we return to Operations Normal.

Mio has fixed the issue where end user messages were delayed from Microsoft to Google. This degraded service was caused by a deployment of multiple microservices, where one service was out of sync with the others, causing delays and failures of messages. The failure prevented our retry strategy from completing, which resulted in the delays and failures. We will resolve this issue after monitoring for the next few hours.

We are actively monitoring a Microsoft outage that occurred early on 2/17/2025 that is impacting Microsoft Teams. "Users may experience delays and failures when sending and receiving chat messages that include inline media (images, code snippets, videos)" - This incident can be tracked under the Microsoft ticket TM1233974 (admin.microsoft.com/#/MessageCenter/:/messages/TM1233974).

Mio has fixed the issue where end user messages were delayed or failing from Microsoft to Google. This degraded service was caused by a deployment of multiple microservices, where one service was out of sync with the others, causing delays and failures of messages. The failure prevented our retry strategy from completing, which resulted in the delays and failures. Older failures and delays should now be delivered, and new messages are being delivered as expected. We will resolve this issue after monitoring for the next few hours.

Mio is currently experiencing an intermittent issue that may impact an end users message delivery - the issue has been identified, fixed, and is now being monitored by our team, however we want to make sure you are aware that we were alerted of the issue by automated alerts. Mio will update this status after monitoring the fix that has already been released over the next few hours.

We have implemented changes intended to improve system performance and reduce message delivery delays. Early results are positive, and overall latency has decreased since these updates were applied. At this time, we are closely monitoring the system to confirm stability before marking the incident as resolved. We will provide another update once we have sustained healthy performance across all affected services.

As a result of AWS us-east-1 returning to operational status we are now seeing our own services returning to normal. We will continue to monitor operations closely to confirm full stability.