Go Guardian experienced a major incident on June 5, 2026 affecting Admin: Dashboard, lasting 1h 33m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jun 05, 2026, 01:09 PM UTC
https://cortana.goguardian.com/api/policies/186176/sites/filters?includeBlockCounts=true is returning a 500. UI is working, no website urls within a policy are viewable. https://console.cloud.google.com/logs/query;cursorTimestamp=2026-06-05T12:55:11.650154444Z;duration=PT5M;lfeCustomFields=jsonPayload%252Ferror;query=logName%3D%22projects%2Fgoguardian-gcp%2Flogs%2Fecs.go-cortana.prod%22%0Aseverity%3E%3DERROR%0Aerror_group%2528%22CIua9_Cc6d71pwE%22%2529%0Atimestamp%3D%222026-06-05T12:55:11.650154444Z%22%0AinsertId%3D%22lf3mk8fgi9q3k%22?project=goguardian-gcp { "insertId": "lf3mk8fgi9q3k", "jsonPayload": { "source": "stdout", "user_agent": "grpc-go/1.80.0", "admin_id": 15834408, "route": "/gocortana.GoCortana/GetPolicySiteFilters", "container_id": "02b2570d43a3c41d29b926e2617a95cc6a0f7269fd266b89d9e2072ca60c8533", "error": "error getting site filter blocks count by policy id with nobunaga: rpc error: code = Unknown desc = failed to executed query: SELECT admin_filter_id, COUNT(*) FROM blocked_sites_v3 WHERE admin_policy_id = ? AND reason = ? AND org_id = ? AND submission_time >= ? AND submission_time <= ? AND time_shard_key IN (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) GROUP BY admin_filter_id: Error 1146: Table 'ent_auth.blocked_sites_v3' doesn't exist", "org_id": 8017, "container_name": "/ecs-go-cortana-prod-178-go-cortana-aca1c4c4a3cfbde85500", "log": "error getting site filter blocks count by policy id", "timestamp": "2026-06-05T12:55:11.649559702Z", "request_id": "d47d28b0-4da4-444f-9626-d33a157a124e" }, "resource": { "type": "aws_ec2_instance", "labels": { "region": "aws:us-west-2a", "instance_id": "i-05d732ddaba3c237e", "aws_account": "476415093293", "project_id": "goguardian-gcp" } }, "timestamp": "2026-06-05T12:55:11.650154444Z", "severity": "ERROR", "labels": { "ec2.amazonaws.com/resource_name": "df46b326a5f5" }, "logName": "projects/goguardian-gcp/logs/ecs.go-cortana.prod", "receiveTimestamp": "2026-06-05T12:55:15.606143589Z", "errorGroups": [ { "id": "CIua9_Cc6d71pwE" } ] }
- investigating Jun 05, 2026, 01:27 PM UTC
per Efrain: the blocked_sites_v3 view was dropped from memSQL prod staging looks okay. I checked to verify. there was a handful of memSQL schema changes that went out yesteday for adding new columns for the time spent project possibly related He pinged dev-ops we're going to need a hand from TF folks
- investigating Jun 05, 2026, 01:44 PM UTC
We are currently investigating reports of an issue within GoGuardian's Admin product that is causing the user interface to return a 500 error or to show policies as if they are "blank." We have confirmed that this is only an issue with the display; ALL policies are still actively functioning and filtering as expected. We will keep this page up-to-date as we learn more. We apologize for any inconvenience.
- monitoring Jun 05, 2026, 02:02 PM UTC
We have identified and fixed the bug that was causing our GoGuardian Admin product to display as if policies were "blank" and/or to show an error 500. This fix has been released and we will continue monitoring for any further concerns. However, we have confidence that this issue has been fully resolved. We greatly apologize for the inconvenience.
- resolved Jun 05, 2026, 02:42 PM UTC
The issue with GoGuardian Admin is fully resolved. All applications are fully functioning. Thank you.