BlockChyp incident

EPX Outage

Major Resolved View vendor source →

BlockChyp experienced a major incident on March 10, 2021, lasting 2h 55m. The incident has been resolved; the full update timeline is below.

Started
Mar 10, 2021, 11:52 PM UTC
Resolved
Mar 11, 2021, 02:47 AM UTC
Duration
2h 55m
Detected by Pingoru
Mar 10, 2021, 11:52 PM UTC

Update timeline

  1. investigating Mar 10, 2021, 11:52 PM UTC

    The EPX platform appears to be having an outage. We are investigating and will update as we learn more.

  2. investigating Mar 11, 2021, 12:11 AM UTC

    We've identified the issue as a DNS configuration issue with the epx.com domain. We're working with EPX to help them resolve the issue.

  3. investigating Mar 11, 2021, 12:20 AM UTC

    We've manually overridden the DNS settings on our gateway servers to temporarily work around the outage. Transactions are starting to process normally and we'll update once proper DNS resolution is restored.

  4. investigating Mar 11, 2021, 12:58 AM UTC

    The issue has been traced to an issue with Oracle's Dyn DNS service, which handles DNS services for EPX. More details here: https://www.dynstatus.com/

  5. monitoring Mar 11, 2021, 01:09 AM UTC

    The DNS outage appears to be restored and EPX has closed the incident. We're monitoring Oracle's status page and will close our incident when Oracle's board turns green and we remove our workaround.

  6. monitoring Mar 11, 2021, 02:38 AM UTC

    Oracle has cleared the DNS issue. We're going to remove our temporary workaround and will update when we've confirmed the system operating normally in its standard configuration.

  7. resolved Mar 11, 2021, 02:47 AM UTC

    We've restored our system to its normal configuration and have confirmed that transactions are continuing to process normally. Thanks to all our partners and merchants for their patience with this issue.