Is StatusCast down?

As of August 1st, 2024 we have removed the link to the Legacy Incident Editor from the admin portal to ensure that users maintain a unified experience and are able to take advantage of all of our latest improvements to the incident creation process.

StatusCast engineers were alerted earlier that some users were experiencing sporadic issues attempting to connect to the status page and admin portal. Our hosting provider, Microsoft Azure, has alerted us via their status page that they are experiencing some network issues globally. We will provide an update as soon as more information is available.

Access to status pages has remained stable and Azure has updated their status indicating failover processes have been engaged to improve their service availability. StatusCast's engineers will continue to watch this closely and will post additional updates as necessary.

StatusCast's application has continued to remain stable. Our engineers will continue to watch the system closely as Microsoft has not fully closed out the event on their side. For more specific details on Azure's issue please refer to their status page. We will provide additional updates as necessary.

Microsoft has closed the issue on their side and StatusCast's platform continues to operate as expected. Once Microsoft has published more details on this we will provide here in the form of an RCA.

FROM MICROSOFT: Mitigation Statement - Azure Front Door Issues accessing a subset of Microsoft services Tracking ID: KTY1-HW8 What happened? Between approximately at 11:45 UTC and 19:43 UTC on 30 July 2024, a subset of customers may have experienced issues connecting to a subset of Microsoft services globally. Impacted services included Azure App Services, Application Insights, Azure IoT Central, Azure Log Search Alerts, Azure Policy, as well as the Azure portal itself and a subset of Microsoft 365 and Microsoft Purview services. What do we know so far? An unexpected usage spike resulted in Azure Front Door (AFD) and Azure Content Delivery Network (CDN) components performing below acceptable thresholds, leading to intermittent errors, timeout, and latency spikes. While the initial trigger event was a Distributed Denial-of-Service (DDoS) attack, which activated our DDoS protection mechanisms, initial investigations suggest that an error in the implementation of our defenses amplified the impact of the attack rather than mitigating it. How did we respond? Customer impact began at 11:45 UTC and we started investigating. Once the nature of the usage spike was understood, we implemented networking configuration changes to support our DDoS protection efforts, and performed failovers to alternate networking paths to provide relief. Our initial network configuration changes successfully mitigated majority of the impact by 14:10 UTC. Some customers reported less than 100% availability, which we began mitigating at around 18:00 UTC. We proceeded with an updated mitigation approach, first rolling this out across regions in Asia Pacific and Europe. After validating that this revised approach successfully eliminated the side effect impacts of the initial mitigation, we rolled it out to regions in the Americas. Failure rates returned to pre-incident levels by 19:43 UTC - after monitoring traffic and services to ensure that the issue was fully mitigated, we declared the incident mitigated at 20:48 UTC. Some downstream services took longer to recover, depending on how they were configured to use AFD and/or CDN. What happens next? Our team will be completing an internal retrospective to understand the incident in more detail. We will publish a Preliminary Post Incident Review (PIR) within approximately 72 hours, to share more details on what happened and how we responded. After our internal retrospective is completed, generally within 14 days, we will publish a Final Post Incident Review with any additional details and learnings. To get notified when that happens, and/or to stay informed about future Azure service issues, make sure that you configure and maintain Azure Service Health alerts – these can trigger emails, SMS, push notifications, webhooks, and more: https://aka.ms/ash-alerts. For more information on Post Incident Reviews, refer to https://aka.ms/AzurePIRs. Finally, for broader guidance on preparing for cloud incidents, refer to https://aka.ms/incidentreadiness.

StatusCast and 4me’s support team are ready to take the next step in migrating our support services from StatusCast’s legacy ticketing system to using 4me directly for all change requests and incidents. This next step in integrating StatusCast’s operations to 4me will ultimately provide you with much more insight into requests you submit and their status. What is changing? On Wednesday, July 24th 2024 StatusCast’s support portal will be shifted to https://4me-statuscast-portal.4me.com/, this self service portal will be your new resource to submit new tickets and view the status of any existing ones that you have submitted. This new portal will give you a more accurate status of the ticket, and in the event that you request a change or run into an issue that requires development work, you will have visibility into the status of that from a development to a QA, to a production perspective. Also, customers will be able to find answers to their questions by walking through the portal that contains a knowledge base which will be extended as we go along. We are very excited to bring this extended visibility to our customers as we believe it’s a major step forward in the support that we provide. As a result of this change, emailing [email protected] will no longer open tickets. You will need to access the self-service portal to submit and access support requests. A link to this self service portal has been added to StatusCast’s admin portal and you can access it by selecting the Settings > Help Center > Support Tickets Existing StatusCast administrators have been granted access to this new portal. To access the StatusCast Portal, go to https://xurrent-statuscast-portal.xurrent.com/ and start with ‘Forgot your 4me password?’(1) so that you can get sent an email to register your support portal password. If you do not get an email while doing this you may not have been granted automated access, in that case you can self register by clicking “Register as New User”(2): The StatusCast Portal requires you to use an email address and to set up a password with 18 characters.

In keeping with our commitment to security and transparency we wanted to inform you that our provider for SMS, Twilio, has alerted our security team to an incident that occurred. Below is the statement we received from Twilio which gives a detailed account of the incident and what has been done: —-------------------------------------- Twilio has been notified by one of our backup carriers, iBasis, that IdentifyMobile inadvertently exposed certain SMS-related data sent by iBasis publicly on the internet that included personal data. We are writing to inform you that some of your personal data and non-personal data (such as data related to marketing campaigns) was accessed by a security research group while it was publicly exposed by IdentifyMobile. Although this incident was outside Twilio’s control, we take it very seriously and are committed to helping you understand the full impact. What do you need to know? In order to deliver messages in specific regions, Twilio relies on numerous carriers to maximize deliverability to their final destinations. Twilio was notified that iBasis (a Twilio backup carrier) had used IdentifyMobile (iBasis's further backup carrier) who inadvertently enabled public access on an AWS S3 Bucket during development work. Information contained in this bucket was made public from May 10-15, 2024, and accessed between May 13-14, 2024. Based on a joint investigation between IdentifyMobile and Amazon AWS, we learned that a portion of this data was accessed by the Chaos Computing Club (CCC). CCC is a security research group that identifies security issues; CCC has confirmed that they are not holding any data downloaded from the AWS S3 Bucket. We do not have evidence that allows us to confirm that no other third party accessed the data. Twilio does not own this bucket, and none of our systems have been compromised in connection with this data exposure. This incident was the result of actions taken by IdentifyMobile and outside of Twilio’s control. While we continue collaborating with these carriers to bring you the most accurate information regarding this exposure, the portion of data exposed by IdentifyMobile related to SMS sent between January 1, 2024 and May 15, 2024, and included: Mobile number SMS message content SMS Sender ID SMS Timestamp What have we done so far? Twilio initiated our incident response process to rapidly investigate this matter. Twilio escalated this issue to the iBasis executive team; subsequently, we’ve done an analysis on the data logs that were compromised to provide you with as much information as possible. Out of an abundance of caution, we have ceased sending traffic to iBasis where possible. iBasis informed Twilio that it has stopped routing with IdentifyMobile. We will continue working with our 3rd party carriers to get you any additional details that may arise from this incident. What do you need to do? We recommend reviewing the SMS traffic you sent between January 1, 2024 and May 15, 2024, discussing the implications of an exposure with your internal team(s) and deciding if you need to engage with impacted individuals. If you need additional information regarding this incident, we are here to support you throughout this situation. We deeply regret any inconvenience this may cause and appreciate your understanding and cooperation. —-------------------------------------- What has StatusCast done? Once StautsCast’s security team was alerted an audit was performed to try and determine the potential impact this had on our service. Based on Twilio’s report it is possible that some notifications sent through StatusCast’s service would have been sent using the iBasis carrier. Based on the report data would have included the recipient's phone number as well as the SMS message. SMS notifications in StatusCast are designed to be brief and to drive users to the status page. SMS messages do not include recipients name or other personal identifiable information(PII). As always in StatusCast you have the right to choose whether SMS notifications are sent or not. SMS options can be controlled in the Settings and Integration sections of the application if you wish to review your configurations. At this time there are no additional action items for StatusCast to take in regards to its application or infrastructure, however we will maintain close communication with Twilio and if any additional information is released we will pass it along through our own status page. StautsCast has partnered with Twilio for years and they have maintained a strong commitment to security and transparency. We will take that as well as this incident into consideration as we continue to offer SMS notifications in the application. If you have any additional questions please reach out to us at [email protected]

Following the acquisition by 4me, we have been preparing to integrate StatusCast’s support operations into 4me’s infrastructure. By combining our support operations with 4me’s, our customers will have the advantage of 24x7 support and the number of available representatives will almost quadruple. To take the first step forward in officially merging these teams we will be removing the live chat option available in the admin portal starting on Friday June 7th, 2024. You can continue to be to communicate with support through email([email protected]) or by accessing our current support portal. In the coming months, you will notice several changes in how StatusCast's support is provided. This will involve the introduction of a new and improved support portal for tracking any change requests or issues that you have logged with support. Additionally, more tools and auditing information will be made available in the application, and there will be an increase in the help and knowledge resources accessible directly. We are very excited to be expanding our support offering with 4me and know that it will ultimately help our community of valued customers and users!