Polly incident

DNS Misconfiguration

Major Resolved View vendor source →

Polly experienced a major incident on August 6, 2019 affecting Slack App and Slack Web App and 1 more component, lasting 8h 13m. The incident has been resolved; the full update timeline is below.

Started
Aug 06, 2019, 10:00 PM UTC
Resolved
Aug 07, 2019, 06:14 AM UTC
Duration
8h 13m
Detected by Pingoru
Aug 06, 2019, 10:00 PM UTC

Affected components

Slack AppSlack Web AppMicrosoft Teams AppMicrosoft Teams TabAPI

Update timeline

  1. monitoring Aug 07, 2019, 01:21 AM UTC

    A DNS misconfiguration was identified and up to 50% of all requests were being incorrectly routed and failing as a result. The configuration has been rolled back and the system will resume normal operations as DNS changes propagate through the network.

  2. monitoring Aug 07, 2019, 02:57 AM UTC

    Everything but API is back up and running at full speed. We expect that to take a few more hours as DNS changes propagate.

  3. resolved Aug 07, 2019, 06:14 AM UTC

    This incident has been resolved.