Patchman experienced a minor incident on May 12, 2021 affecting Portal and Customer API and 1 more component, lasting 3h 31m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating May 12, 2021, 04:42 PM UTC
Various systems are currently reporting degraded performance due to networking problems in our DC. We are investigating the issue.
- investigating May 12, 2021, 04:51 PM UTC
We're continuing to investigate the root cause while we restore our services.
- investigating May 12, 2021, 04:59 PM UTC
We are seeing stabilization, but will keep the incident Open because the root cause has not yet been identified so we can't yet confirm that the problem has been completely addressed. Our investigation is ongoing.
- investigating May 12, 2021, 05:10 PM UTC
We are in touch with our infrastructure provider to investigate the root cause of the packet loss. Specifically Portal systems are reporting intermittent connectivity issues at this time.
- identified May 12, 2021, 05:31 PM UTC
The problems seem limited to intermittent timeouts on the Portal and its API. We have identified the root cause as packet loss in the network, and are relying on our infrastructure provider to address those issues. We will continue to update this incident as we get more information.
- identified May 12, 2021, 06:06 PM UTC
Our infrastructure provider has identified a problem in their routing. A solution is being implemented.
- monitoring May 12, 2021, 06:58 PM UTC
All services are stable. We are continuing to monitor the situation.
- resolved May 12, 2021, 08:14 PM UTC
This incident has been resolved.