7digital experienced a notice incident on July 14, 2020, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Jul 14, 2020, 11:54 AM UTC
Between 11:22 and 12:00 BST we saw a degradation of service on the platform. As of 12:00 BST full service has resumed but we're continuing to monitor the platform and will follow up to provide further details on the impact of the incident and the measures taken to resolve and prevent it reoccurring.
- postmortem Jul 17, 2020, 02:08 PM UTC
Our CDN provider \(Fastly\) has now confirmed that they have identified an issue with their service which directly caused the degradation see in this incident. A further explanation of the issue which occurred at our CDN provider is below: Starting at approximately 08:51 UTC, Fastly observed three global transit provider events affecting most Fastly data centers, leading to increased 5xx errors and latency. The first event occurred from 08:51 to 8:55 UTC, the second from 09:57 to 10:04 UTC, and a third event starting at 10:17 UTC. During the third event, at 10:21 UTC, a Fastly traffic engineering configuration change, designed to mitigate the provider issues, inadvertently led to additional customer impact in the form of errors, latency, and timeouts. Additionally, the Fastly portal and API were affected during this event. Fastly Engineering started mitigations at 10:25 UTC, gradually restoring services until the last repair was completed at 10:56 UTC.