Hang incident

Degraded performance in loyalty app

Major Resolved View vendor source →

Hang experienced a major incident on July 31, 2025, lasting –. The incident has been resolved; the full update timeline is below.

Started
Jul 31, 2025, 03:30 PM UTC
Resolved
Jul 31, 2025, 03:30 PM UTC
Duration
–
Detected by Pingoru
Jul 31, 2025, 03:30 PM UTC

Update timeline

  1. resolved Aug 04, 2025, 03:47 PM UTC

    Degraded performance in Hang loyalty app - Timeouts are being seen loading the loyalty app home page

  2. postmortem Aug 04, 2025, 03:49 PM UTC

    **Incident Report: Logging Infrastructure Impact on App Performance** **Summary:** Hang deployed a new logging infrastructure that inadvertently introduced latency in calls made from the front end to Hang’s API. The issue was identified following alerts about timeouts from the loyalty application. After investigation, the team pinpointed the root cause and deployed a fix. **Impact:** The incident resulted in slow loading times for some users. A subset of users experienced timeouts that prevented the loyalty app home page from loading. **Resolution:** The issue was resolved by removing all synchronous logging calls to ensure that slow response times from downstream providers \(e.g., New Relic, Datadog, Sentry\) no longer impact web traffic. **Preventive Measures:** Hang has updated its logging infrastructure to use only asynchronous calls. This prevents logging-related latency from affecting API performance and ensures that future slowdowns from external observability tools do not impact user experience. **Timeline of Events:** * **10:14 AM:** The team was alerted to irregular server autoscaling activity and began investigating. * **10:35 AM:** Slow response times began affecting live users. Some users experienced app timeouts. The root cause was identified as synchronous logging blocking request flow. * **10:55 AM:** A fix was fully deployed to production. User-facing issues were resolved, and app performance returned to normal.