Ticketmatic incident

Increased error rates

Minor Resolved View vendor source →

Ticketmatic experienced a minor incident on December 11, 2016, lasting 2h 6m. The incident has been resolved; the full update timeline is below.

Started
Dec 11, 2016, 06:09 PM UTC
Resolved
Dec 11, 2016, 08:15 PM UTC
Duration
2h 6m
Detected by Pingoru
Dec 11, 2016, 06:09 PM UTC

Update timeline

  1. investigating Dec 11, 2016, 06:09 PM UTC

    We're observing increased error rates due to a disruption in one of our services. A small minority of our users will be impacted and encounter an error dialog. A simple reload of the web page should fix things and allow you to continue. A fix is being worked on, apologies for the inconvenience.

  2. monitoring Dec 11, 2016, 07:04 PM UTC

    A fix has been deployed, service should no longer be disrupted. We're keeping a close eye on things but there shouldn't be any impact anymore.

  3. resolved Dec 11, 2016, 08:15 PM UTC

    Everything is back to normal. This disruption was caused by a recent change in our web socket connection handling code. A combination of factors led to a serious overload on one of our services, which in turn caused some connections to time out. We'll be updating our automated QA processes to ensure that this won't happen again in the future.