LivePOS incident

We are investigating an issue where some customers are experiencing Red Logos and Admin Site "Ooops" errors

Minor Resolved View vendor source →

LivePOS experienced a minor incident on December 24, 2015, lasting 4h 21m. The incident has been resolved; the full update timeline is below.

Started
Dec 24, 2015, 09:10 PM UTC
Resolved
Dec 25, 2015, 01:31 AM UTC
Duration
4h 21m
Detected by Pingoru
Dec 24, 2015, 09:10 PM UTC

Update timeline

  1. investigating Dec 24, 2015, 09:10 PM UTC

    We are working with AWS to identify the issue, we will post updates here every 30min.

  2. investigating Dec 24, 2015, 09:39 PM UTC

    We are still identifying the problem with AWS. will keep you posted ASAP

  3. identified Dec 24, 2015, 10:11 PM UTC

    We have Identified the issue with AWS Support and working on resolving the problem. will continue updating here.

  4. identified Dec 24, 2015, 11:16 PM UTC

    We are still working on the problem with AWS. Red Logos may come and go as the issue is resolved.

  5. identified Dec 25, 2015, 12:02 AM UTC

    We received word that Red Logos are slowly disappearing and Sync is slowly working. AWS is still trying to pin point the problem.

  6. monitoring Dec 25, 2015, 12:36 AM UTC

    We are seeing improvements and diminishing red logos. will update soon.

  7. resolved Dec 25, 2015, 01:31 AM UTC

    AWS identified the issue as an instance (hardware) problem, and as such we are required to transfer some of our servers onto new hardware. We will do so during tonight and tomorrow morning before the stores reopen again.