Patchman incident

Intermittent connectivity issues

Minor Resolved View vendor source →

Patchman experienced a minor incident on January 15, 2021 affecting Portal and Customer API and 1 more component, lasting 4h 23m. The incident has been resolved; the full update timeline is below.

Started
Jan 15, 2021, 06:02 PM UTC
Resolved
Jan 15, 2021, 10:26 PM UTC
Duration
4h 23m
Detected by Pingoru
Jan 15, 2021, 06:02 PM UTC

Affected components

PortalCustomer APIAgent APIDefinitions APILicensing APIData Processing Infrastructure

Update timeline

  1. identified Jan 15, 2021, 06:02 PM UTC

    Our infrastructure provider is experiencing some networking problems, which are leading to intermittent connectivity issues on our platform. All our APIs may experience packet loss and slow response times. We are working with our provider to resolve the problem as soon as possible.

  2. identified Jan 15, 2021, 06:15 PM UTC

    We are primarily seeing issues on the Agent API and data processing infrastructure at this time. The Portal and Licensing API may experience increased response times still.

  3. identified Jan 15, 2021, 06:23 PM UTC

    At this time, only the Licensing API and our Data Processing Infrastructure still seem to be impacted.

  4. identified Jan 15, 2021, 06:37 PM UTC

    On the Licensing API, we're currently running at lower capacity than normal, but we don't see any problems with the current load. On the Data Processing Infrastructure, the problems have mainly caused a small backlog in processed messages, which means the Portal may be running behind on some scan information. We expect the systems to catch up with the backlog soon.

  5. identified Jan 15, 2021, 08:08 PM UTC

    For now we are no longer seeing any issues on our platform, but our monitoring confirms that the actual networking issues have not yet been fully resolved. We continue to investigate.

  6. monitoring Jan 15, 2021, 09:06 PM UTC

    We are currently not seeing any remaining issues, but are continuing to monitor the situation closely for regressions. We are also in close contact with our infrastructure provider to confirm the resolutions are complete and in place.

  7. resolved Jan 15, 2021, 10:26 PM UTC

    This incident has been resolved.