Amity Outage History
Amity major outage · 1 active incident View live status →Pingoru has recorded 51 incidents for Amity: 3 in the last 12 months and 1 in the last 90 days.
Show as a table
| Month | Incidents | Hours with an open incident |
|---|---|---|
| April 2026 | 0 | 0 of 720 (0%) |
| May 2026 | 0 | 0 of 744 (0%) |
| June 2026 | 1 | 7 of 720 (1%) |
| July 2026 | 0 | 0 of 744 (0%) |
| August 2026 | 0 | 0 of 744 (0%) |
| September 2026 | 0 | 0 of 720 (0%) |
| October 2026 | 1 | 31 of 528 (6%) |
Incidents, newest first
Minor June 7, 2026 Elevated errors on core and API systems (US) 7h
Started Jun 07, 2026, 12:00 AM UTC· resolved Jun 07, 2026, 07:00 AM UTC
- resolved Jun 08, 2026, 04:40 AM UTC
We have identified an issue which is resulting in parts of core and API system being degraded. A fix has been implemented and we are monitoring the results.
- postmortem Jun 08, 2026, 07:04 AM UTC
**Elevated errors on core and API systems \(US\)** Date: Jun 7, 2026 · Window: 06:58 – 07:11 AM GMT\+07:00 \(~13 minutes\) **What happened** For approximately 13 minutes, parts of our core and API systems in the US region were degraded, and clients experienced elevated error rates. A backend database reached its storage capacity, which briefly prevented it from completing write operations. **Root cause** An atypical workload consumed database storage at a rate faster than our systems had previously encountered. While automated scaling did respond, it did not add capacity quickly enough to keep pace with the abnormal rate before storage was exhausted. **Resolution** Additional storage capacity was brought online, after which error rates returned to normal and all systems fully recovered.
Major January 21, 2026 Partially Degraded Performance [US region] 1h 58m Core Services (US)
Started Jan 21, 2026, 02:00 PM UTC· resolved Jan 21, 2026, 03:58 PM UTC
- investigating Jan 21, 2026, 03:35 PM UTC
We're investigating an alert on our social and realtime performance. You may experience delay in social and realtime connection or response time.
- monitoring Jan 21, 2026, 03:37 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jan 21, 2026, 03:58 PM UTC
This incident has been resolved.
- postmortem Jan 22, 2026, 02:32 PM UTC
**Incident Date:** 2026-01-21 **Impact:** System degradation and intermittent downtime. **Primary Cause:** Infrastructure **resource exhaustion** triggered by an unprecedented high-volume traffic surge. ## 1. Summary On January 21, an unprecedented surge in traffic, peaking at **450,000 requests per minute \(~56.25x baseline\)**. While application servers autoscaled successfully, the Core Database became the bottleneck. Despite two manual vertical scaling interventions, the system experienced two periods of degradation before stabilizing as database capacity finally matched the demand. ## 2. Root Cause The root cause of the incident was **infrastructure resource exhaustion** resulting from insufficient database overhead to accommodate a sudden traffic spike. * **Traffic Volume:** An unprecedented surge in external demand drove platform traffic significantly beyond predicted growth, increasing from a baseline of **8,000 req/min** to a peak of **450,000 req/min \(a ~56.25x increase\)**. * **Scaling Operation Time:** Vertical scaling of the Core Database required a **10–30 minute operation time** per event. During these intervals, the system remained degraded as incoming demand outpaced both available capacity and recovery speed. ## 3. Optimizations & Corrective Actions Based on the investigation, we will implement the following technical safeguards: #### **A. Transition Impacted Queries to Secondary Nodes** * **Action:** Reconfigure remaining database queries to target Secondary \(Read\) Replicas rather than the Primary node. * **Goal:** Offload significant pressure from the Primary database. By reducing the load on the Primary node, we ensure it retains enough resource overhead to improve the scaling and recovery time. This prevents the Primary from being choked by contention, allowing it to complete vertical scaling operations much faster during a surge. #### **B. Optimize Autoscaling Performance \(Server & Database\)** * **Action:** Review and tune autoscaling policies for both the App Tier and Database Tier to specifically reduce operation time. * **Goal:** Decrease the "Time-to-Ready" for new resources. By optimizing scaling triggers and resource warm-up procedures, we ensure capacity is provisioned more rapidly, improving the system's overall recovery time during a sudden spike.
Major December 30, 2024 Partially Degraded Performance [SG region] 34s Core Services (SG)
Started Dec 30, 2024, 05:05 PM UTC· resolved Dec 30, 2024, 05:06 PM UTC
- investigating Dec 30, 2024, 05:05 PM UTC
We're investigating an alert on our websocket performance. You may experience delay in websocket connection or response time.
- resolved Dec 30, 2024, 05:06 PM UTC
This incident has been resolved.