Mactera Prizewall incident

Prizewall Frontend Unavailable

Resolved after 18 hours 49 minutes, affecting Prizewall Application, on July 20, 2025.

postmortem Jul 20, 2025, 08:10 PM UTC

After user reports that Prizewall was unavailable, we looked to immediately resolve the issue. The issue was that Prizewall was being run through Screens, a piece of software that emulates an active terminal connection, even when a remote terminal is not connected. This was how Prizewall was run from 0.1.0 to 0.5.0, **without documentation of this practice.** This process, regardless of documentation was poor practice. When Prizewall was upgraded to 0.5.1, we failed to run it through screens, but **maintained an active connection to the application through a remote terminal.** As a result, we were entirely unaware of any issues present. Once the remote terminal disconnected from the session \(like when a laptop is closed, for example\), the Prizewall application stopped running. We were unaware of this until a user notified us. As a result of this, we changed how the application stack is deployed and executed. Prizewall is now executed in such a way that persists upon reboot, and without maintaining a virtual terminal. This may have the added bonus of a slight performance boost. We have also improved our notification processes to ensure that Prizewall notifies key stakeholders through multiple means of communication, to reduce downtime resulting from application failure.

Started
Jul 20, 2025, 12:00 AM UTC
Resolved
Jul 20, 2025, 06:50 PM UTC
Duration
18h 49m
Detected by Pingoru
Jul 20, 2025, 12:00 AM UTC

Affected components

Prizewall Application

Update timeline

  1. monitoring Jul 20, 2025, 04:09 PM UTC

    During the deployment of release 0.5.1, we encountered an issue with how the application was executed. Previously, in version 0.5.0, the Prizewall application was run with screens, a service that maintains a constant terminal connection, even when a remote client is not connected. However, after the latest update, the application was run directly from the server connection. This caused the application to disconnect once the session was terminated. We have implemented a fix, and have improved how Prizewall is both run and deployed in the future. We will be monitoring the service to ensure that no further issues are presented.

  2. monitoring Jul 20, 2025, 04:11 PM UTC

    We are continuing to monitor for any further issues.

  3. resolved Jul 20, 2025, 06:50 PM UTC

    This incident has been resolved.

  4. postmortem Jul 20, 2025, 08:10 PM UTC

    After user reports that Prizewall was unavailable, we looked to immediately resolve the issue. The issue was that Prizewall was being run through Screens, a piece of software that emulates an active terminal connection, even when a remote terminal is not connected. This was how Prizewall was run from 0.1.0 to 0.5.0, **without documentation of this practice.** This process, regardless of documentation was poor practice. When Prizewall was upgraded to 0.5.1, we failed to run it through screens, but **maintained an active connection to the application through a remote terminal.** As a result, we were entirely unaware of any issues present. Once the remote terminal disconnected from the session \(like when a laptop is closed, for example\), the Prizewall application stopped running. We were unaware of this until a user notified us. As a result of this, we changed how the application stack is deployed and executed. Prizewall is now executed in such a way that persists upon reboot, and without maintaining a virtual terminal. This may have the added bonus of a slight performance boost. We have also improved our notification processes to ensure that Prizewall notifies key stakeholders through multiple means of communication, to reduce downtime resulting from application failure.

Your product runs on more than Mactera Prizewall

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →