IT Portal experienced a minor incident on May 27, 2022 affecting US API and Portal KBs and 1 more component, lasting 58m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified May 27, 2022, 03:15 AM UTC
Access to multiple cloud regions is being affected by an Issue at Cloudflare. The issue seems to be sporadic and access is limited at times. See the video below for a detailed explanation. https://www.loom.com/share/9c7e6ff845034fbcae8786414b74cd54
- monitoring May 27, 2022, 04:09 AM UTC
A fix has been implemented and we are monitoring the results.
- monitoring May 27, 2022, 04:12 AM UTC
We have worked with Cloudflare and made changes to our setup of DNS records and the issue now seems resolved. We speculate an issue on amazon's DNS servers. We switched our cluster DNS records from amazon CNAMEs to the IP of the cluster and the issue had resolved itself. BEFORE www.siportal.net is a CNAME pointing to us-active.siportal.net us-active.siportal.net is a CNAME pointing to wc04.siportal.net wc04.siportal.net was a CNAME pointing to ec2-34-232-208-181.compute-1.amazonaws.com AFTER www.siportal.net is a CNAME pointing to us-active.siportal.net us-active.siportal.net is a CNAME pointing to wc04.siportal.net wc04.siportal.net is now pointing to A 34.232.208.181
- resolved May 27, 2022, 04:14 AM UTC
This incident has been resolved.