Chameleon.io incident

Redis instance connection timeouts

Resolved after 1 hour 48 minutes, on February 6, 2024.

resolved Feb 06, 2024, 05:32 PM UTC

This incident has been resolved. The analytics data received for the time between Feb 6th, 15:31 UTC and Feb 6th, 16:28 UTC were not processed in real time. Our team in the process of re-running any data received during this time frame including experience analytics. Additionally many background jobs during this will also be re-run including data exports. If you do not receive your data export by the end of the day today please re-export the data or reach out to support

Started
Feb 06, 2024, 03:43 PM UTC
Resolved
Feb 06, 2024, 05:32 PM UTC
Duration
1h 48m
Detected by Pingoru
Feb 06, 2024, 03:43 PM UTC

Update timeline

  1. investigating Feb 06, 2024, 03:43 PM UTC

    We are currently investigating this issue.

  2. monitoring Feb 06, 2024, 04:07 PM UTC

    Our main redis instance had a failover to its secondary and is catching back up with traffic. This redis instance handles background processing and will take some time to catch back up with realtime. Customers may experience issues opening Chameleon's Builder in their applications, exporting reports, or loading certain Chameleon Dashboard pages.

  3. resolved Feb 06, 2024, 05:32 PM UTC

    This incident has been resolved. The analytics data received for the time between Feb 6th, 15:31 UTC and Feb 6th, 16:28 UTC were not processed in real time. Our team in the process of re-running any data received during this time frame including experience analytics. Additionally many background jobs during this will also be re-run including data exports. If you do not receive your data export by the end of the day today please re-export the data or reach out to support

Your product runs on more than Chameleon.io

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →