ChatDaddy incident

Delay in Live Messaging

Minor Resolved View vendor source →

ChatDaddy experienced a minor incident on November 11, 2024 affecting Instant Messaging Service and Keyword Reply Service and 1 more component, lasting 55m. The incident has been resolved; the full update timeline is below.

Started
Nov 11, 2024, 02:28 AM UTC
Resolved
Nov 11, 2024, 03:24 AM UTC
Duration
55m
Detected by Pingoru
Nov 11, 2024, 02:28 AM UTC

Affected components

Instant Messaging ServiceKeyword Reply ServiceFlow service

Update timeline

  1. investigating Nov 11, 2024, 02:28 AM UTC

    We're investigating. Please refresh the page to review live data

  2. resolved Nov 11, 2024, 03:24 AM UTC

    This incident has been resolved.

  3. postmortem Dec 01, 2024, 01:51 PM UTC

    This was the same issue as the one on Oct 31. We finally managed to track the source of the delay, that being the incorrect reading of “high memory usage” in our queuing system that prevented more events from being added. We’ve added more memory to counteract this as well as increase the threshold to high memory usage. Furthermore, the internal alarm setup on Oct 31 helped catch this incident early & resolve it much faster than the incident on the 31st.