365 Retail Markets incident
January 11, 2024 - Service Outage Root Cause Analysis
365 Retail Markets experienced a notice incident on January 12, 2024, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Jan 12, 2024, 02:49 PM UTC
Subject: January 11, 2024 - Service Outage Root Cause Analysis Dear Valued Customer Yesterday, we experienced an unexpected database impairment with our AWS service at approximately 12:10PM Eastern Time. While most active kiosks continued operating in local mode, cloud services such as order ahead were unavailable during this 20–30–minute event. A reboot process was initiated to correct that database impairment which took approximately 20 minutes to complete. Once the reboot process was completed normal cloud services were restored and all kiosks came out of local mode. All transactions data was synced back to the server. Our development and database engineers are actively engaged with AWS experts on this issue. Based on their suggestions, we performed an emergency maintenance on the database yesterday evening to prevent this from occurring in the future. We apologize for any inconvenience this may have caused. If you have any further questions, please contact us through our support team or your account representative. Sincerely, 365 Retail Markets Support Team