FloRecruit incident
Aug 17: Video chat rooms not loading for participants
FloRecruit experienced a critical incident on August 17, 2021 affecting Video Hosting, lasting 56m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 17, 2021, 01:00 PM UTC
We are currently investigating this issue. While we investigate, we recommend that clients edit their events and add in an external video conferencing solution under the option to add a Custom Link instead of auto-generating one.
- identified Aug 17, 2021, 01:31 PM UTC
We believe the issue has to do with an update released by our video chat provider. We are currently deploying the fix they recommended in order to solve the issue. We will not be sure if this fully solves the issue until the update is fully released. Customers should continue to insert external video chat providers in the Custom Link field for each Interview or Virtual Chat timeslot
- identified Aug 17, 2021, 01:32 PM UTC
We are continuing to work on a fix for this issue.
- monitoring Aug 17, 2021, 01:47 PM UTC
We believe we have resolved the issue. Please open all video chat rooms in Chrome's Incognito window when possible if the loading issue persists.
- resolved Aug 17, 2021, 01:56 PM UTC
All users are now able to enter into their video chat rooms. If a user reports that their video chat rooms are not loading, please instruct them to open their Chrome browser window in Incognito or clear cookies. To open an Incognito window in Chrome: 1. Click File 2. New Incognito Window 3. Paste your video chat link into that Incognito Window To clear cookies/cache if you do not have an Incognito Window option 1. Click the vertical three dots in the top right corner of your Chrome browser window 2. Scroll down to Privacy and Security 3. Click Clear Browsing Data and select last 7 days then Clear Data
- postmortem Aug 30, 2021, 10:25 PM UTC
**Event Summary** Flo Recruit utilizes [Daily.co](http://Daily.co) to provide video chat functionality on our platform. On the morning of August 17, Daily shipped a patch to address a separate issue with its custom theming on user pages. This patch unintentionally prevented users of certain older versions of Daily from successfully connecting and holding video calls. Flo Recruit was using one of the older versions impacted by this change, thus Flo users were unable to join video rooms. Additional technical details can be found on Daily’s site: [here](https://dailyco.notion.site/2021-08-17-v0-999-No-Longer-Compatible-64d872d68b6a47d5bcbd742f803eceb0). **Detection and Resolution** * 7:30am CT: Flo begins receiving notifications from customers of inability to access video rooms. Customer support team initiates troubleshooting and provided alternate room links when possible during impacted period * 8:04am CT: Flo confirms issue is not based on internally written code and alerts Daily * 8:44am CT: Flo, in concert with Daily, identifies and implements solution, allowing users to successfully join rooms and resume normal use of the product **Go-Forward** Flo Recruit and Daily are developing improved testing plans to prevent the issue from occurring in the future as additional updates are shipped by Daily. We are continually updating our internal testing, error monitoring, and remediation strategies as we refine and build our products.