KDS not displaying orders

Update timeline

1. investigating Oct 19, 2022, 11:52 AM UTC

   We are currently facing a network traffic issue. Orders are being delayed coming into the system and there will be delays when using the KDS. We have been experiencing this issue since 12:00 and our team is investigating the issue as a top priority. We will provide updates as soon as we have them

2. identified Oct 19, 2022, 12:21 PM UTC

   We are continuing to investigate this issue as a top priority. We have Increased capacity on databases, increased resources and time outs on the processes that allow us to process orders. We are observing an improvement in performance but are continuing our investigation, whilst monitoring performance.

3. monitoring Oct 19, 2022, 12:39 PM UTC

   The changes that we have made to capacity, resources and timeouts on the relevant processes are having the desired impact on performance. Performance on the KDS should be back to normal. We are continuing to actively monitor performance and to investigate the root cause.

4. resolved Oct 19, 2022, 04:16 PM UTC

   We have now identified the root cause of the issue and implemented a fix. Performance on the KDS will have returned to normal and there should be no further issues. Update: - The root cause of the issue was actually isolated to a package within a subsystem that drives Order History and the KDS - An update was made to this package in the release this morning, which caused our database querying and communications to slow down - The issue had initially presented itself as a network issue - which prompted the action to increase capacity and resources to the subsystem. This allowed us to recover KDS performance and identify the actual root cause.