Singlewire Software incident

Cloud to On-prem Notification Delivery Affected

Notice Resolved View vendor source →

Singlewire Software experienced a notice incident on January 15, 2025, lasting —. The incident has been resolved; the full update timeline is below.

Started
Jan 15, 2025, 04:20 PM UTC
Resolved
Jan 15, 2025, 04:20 PM UTC
Duration
—
Detected by Pingoru
Jan 15, 2025, 04:20 PM UTC

Update timeline

  1. resolved Jan 22, 2025, 09:15 PM UTC

    Notifications initiated from the web, mobile apps, or any cloud-side inbound integration may have failed to deliver to on-prem recipients between approximately 10:21 AM and 10:33AM on 1/15/2025.

  2. postmortem Jan 22, 2025, 09:15 PM UTC

    During this window, one of our engineers was executing a scheduled InformaCast deployment. During this deployment, we encountered an unexpected error. The established procedure during such an event is to immediately escalate to a senior engineer to assess the error and its potential impact. This process was not followed and remediation steps were taken instead, which temporarily left InformaCast with too few copies of a critical service for cloud to on-premises notification to function. Steps are being taken to reinforce training around deployment communications, escalating to senior engineers or management when the unexpected occurs, and the importance of following established procedures.