Canary incident
Issues in video creation, notification delivery and mode changes.
Canary experienced a notice incident on October 3, 2017, lasting 2d 21h. The incident has been resolved; the full update timeline is below.
Update timeline
- investigating Oct 03, 2017, 02:06 PM UTC
We are currently investigating delays in mode changes, video processing and notifications being sent for created videos. Watch live services are not affected. We apologies for the continued degradation in Canary cloud services.
- identified Oct 04, 2017, 08:35 PM UTC
The underlying issue that's affecting video creation, notification delivery and mode changes still remains. Canary engineers are continuing to work towards resolving the underlying issue impacting these services. We expect this degraded system performance to continue into the evening. We are very sorry about the ongoing issues our customers have been experiencing with the Canary cloud.
- monitoring Oct 05, 2017, 12:03 PM UTC
There are no current delays or issues affecting Canary cloud services. Video creation, notification delivery and mode changes should all be working as expected. We will continue to closely monitor these services throughout the morning.
- monitoring Oct 05, 2017, 11:26 PM UTC
Canary engineers will be performing maintenance to our underlying services at 12:00AM EST. Video creation, notification delivery and mode changes are not expected to be impacted. We continue to closely monitor the Canary cloud for any instabilities and will next update customers Friday morning with the results of the system maintenance.
- resolved Oct 06, 2017, 11:07 AM UTC
Maintenance has been completed bringing further stability to the Canary cloud. Video creation, notification delivery and mode changes were not impacted during this time. We are very sorry about the previous issues customers have been experiencing with video event delivery.