InTouch Receipting incident

DNS Outage

Major Resolved View vendor source →

InTouch Receipting experienced a major incident on December 1, 2014, lasting 10h 1m. The incident has been resolved; the full update timeline is below.

Started
Dec 01, 2014, 09:25 PM UTC
Resolved
Dec 02, 2014, 07:26 AM UTC
Duration
10h 1m
Detected by Pingoru
Dec 01, 2014, 09:25 PM UTC

Update timeline

  1. identified Dec 01, 2014, 10:00 PM UTC

    Our DNS provider is the victim of an ongoing DDoS attack. They are working with their network provider to increase network capacity to deal with the flood of traffic, as well as, shutting down the attackers.

  2. identified Dec 01, 2014, 10:59 PM UTC

    30 minute ETA from our DNS provider to begin scrubbing traffic in a location with capacity.

  3. identified Dec 01, 2014, 11:27 PM UTC

    The ETA from our DNS provider has been extended another 30 minutes while they work on getting the new higher capacity server online.

  4. identified Dec 01, 2014, 11:29 PM UTC

    As a result of today's DNS outage we are looking into automatic, secondary DNS failover services.

  5. monitoring Dec 02, 2014, 12:07 AM UTC

    Services are returning to normal. We are continuing to test all regions.

  6. resolved Dec 02, 2014, 07:26 AM UTC

    We have completed a transition to a new DNS provider. All services are operating normally. Thank you for your patience.