Increased Access Manager error rates and latency in our US East PoP

Update timeline

1. resolved Dec 09, 2022, 08:31 PM UTC

   Between 19:12 - 19: 20 UTC, Customers using PAM grant in our US East PoP may have experienced increased error rates and latency. They may have experienced a small impact to other services as well.

2. postmortem Dec 12, 2022, 05:02 PM UTC

   ### **Problem Description, Impact, and Resolution** On December 9, 2022 at 19:12 UTC, we observed an increase in errors and latency for Access Manager API calls in our US East PoP. The errors and latency occurred shortly after initiating a deployment. We responded by stopping the deployment, and rerouted affected traffic. The issue was resolved at 19:20 UTC. This issue occurred because of a misconfiguration in an infrastructure deployment. **Mitigation Steps and Recommended Future Preventative Measures** To prevent a similar issue from occurring in the future, we will fix the misconfiguration and update our system to better handle failures.