Bird App experienced a major incident on March 12, 2025 affecting Connectivity & Cellular, lasting 18h 26m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 12, 2025, 04:46 PM UTC
Vodafone is suffering an outage affecting multiple markets worldwide
- investigating Mar 12, 2025, 05:30 PM UTC
Message from Vodafone: We acknowledge that you experience issues with our service and we apologize for the inconvenience. We engaged our Network Engineering Team to further investigate the incident. We will get back to yoiu as soon as we receive a meaningful update.
- resolved Mar 13, 2025, 11:13 AM UTC
Dear Vodafone Enterprise IoT Customer, We would like to inform you that the following incident has now been resolved: Executive Summary Customers configured on Big IP experienced data degradation Incident Summary IoT Ops Service Support, through ongoing monitoring, identified that customers utilizing the Big IP platform were experiencing significant issues due to unresolved DNS queries. This issue was notably impacting traffic and causing substantial packet loss. In response, a bridge was established with all relevant teams for further investigation. To mitigate the immediate impact, traffic was switched from SPE7 (Milan) to SPE2 (Ratingen). However, despite this effort, packet loss continued to persist. Upon further troubleshooting, it was determined that the root cause of the problem was attributed to a F5 customer configured on MCC113. Once with the isolation of F5 traffic on MCC113, the Big IP platform stabilized. The remaining customers, except those configured on MCC113, were fully recovered at this point. To restore services for the customers on MCC113, a failover to MCC121 was performed. This led to service restoration for F5 customers configured on this MCC, in addition to those that were already working properly. Resolution Failover performed from MCC113 to MCC121 We apologise for the inconvenience.