FBS experienced a major incident on December 11, 2019 affecting Flexmls Web, lasting 5h 24m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Dec 11, 2019, 05:09 PM UTC
We're currently investigating reports of upstream issues with our CDN provider. Affected users may experience intermittent failures or slowness while accessing flexmls.
- identified Dec 11, 2019, 05:15 PM UTC
We have verified that the issues are, in fact, a result of an upstream issue with our CDN provider, Edgecast Networks aka Verizon Digital Media (VDMS). Updates for that issue can also be found at their status page: https://status.verizondigitalmedia.com/ We're continuing to investigate other avenues to mitigate this issue without relying solely on VDMS's mitigation.
- monitoring Dec 11, 2019, 05:50 PM UTC
Verizon Digital Media Services (VDMS) is reporting that their issues have been at least partially mitigated. We've also taken proactive measures to side-step their issues. From our monitoring, we are seeing that the vast majority of our users have recovered, and are using our products without issues. We'll continue to monitor the situation, and update once we can safely say that it has been completely resolved.
- resolved Dec 11, 2019, 10:34 PM UTC
While our affected upstream CDN provider hasn't yet given the "all clear" for their services, we're confident that this incident has been completely resolved. We apologize for any inconvenience that was caused today, and we're implementing changes that will help prevent similar failures from impacting us in the future.