Datacake experienced a minor incident on May 27, 2024, lasting 22h 15m. The incident has been resolved; the full update timeline is below.
Update timeline
- investigating May 27, 2024, 10:31 AM UTC
Our system is experiencing processing delays. No data has been lost, and the system should be caught up shortly..
- identified May 27, 2024, 12:16 PM UTC
We are continuing to work on a fix for this incident..
- investigating May 27, 2024, 02:06 PM UTC
We are currently investigating this incident..
- identified May 27, 2024, 05:17 PM UTC
We are still working on a mitigation of the delayed processing of device data..
- monitoring May 27, 2024, 09:47 PM UTC
Queue processing is largely complete, with only outgoing MQTT and webhooks remaining to be finished..
- resolved May 28, 2024, 08:47 AM UTC
Dear customers, We are informing you that some of our services on the Datacake IoT platform have recently experienced delays. Please be assured that there was no downtime and no data was lost. The protection and integrity of your data remain our top priority. The issue affected a subset of devices, causing delays of up to two hours for certain tasks. While some devices continued to operate at full performance, others were impacted by the delays. The cause was increased wait times for specific task processing, including Payload Decoder and API requests. Although the platform's core API remained stable, queues became overwhelmed, leading to longer processing times. To ensure the platform's stability, we have temporarily disabled some non-essential features and are monitoring the situation closely. We are taking measures to optimize internal processes and will keep you informed of any changes. Thank you for your understanding and patience during this time. Best regards, The Datacake Team.