Honeybadger incident

API and Web App May be Unavailable to Some Users

Minor Resolved View vendor source →

Honeybadger experienced a minor incident on May 30, 2020 affecting Web Application and API, lasting 1h 33m. The incident has been resolved; the full update timeline is below.

Started
May 30, 2020, 11:56 AM UTC
Resolved
May 30, 2020, 01:29 PM UTC
Duration
1h 33m
Detected by Pingoru
May 30, 2020, 11:56 AM UTC

Affected components

Web ApplicationAPI

Update timeline

  1. investigating May 30, 2020, 11:56 AM UTC

    We have experienced a drop in external traffic which leads us to believe that some customers may not currently be able to access our service. At this time we suspect that the issue is caused by network problems outside of our own systems, and we are currently investigating to confirm this.

  2. monitoring May 30, 2020, 12:33 PM UTC

    The unavailability is being caused by an upstream SSL problem. Comodo (our CA) is being flagged as "untrusted" by clients. See this tweet for an example: https://twitter.com/aitorpazos/status/1266703889786691584 We are currently monitoring the situation, as well as investigating other solutions.

  3. monitoring May 30, 2020, 01:19 PM UTC

    We are in the process of issuing new certificates from a different CA, to work around Comodo's issue.

  4. resolved May 30, 2020, 01:29 PM UTC

    We have switched all affected systems to new certificates and everything is back to normal.