Stripo incident

Stripo Plugin is down

Critical Resolved View vendor source →

Stripo experienced a critical incident on October 8, 2026 affecting API Gateway Service (plugin), lasting 58m. The incident has been resolved; the full update timeline is below.

Started
Oct 08, 2026, 10:20 PM UTC
Resolved
Oct 08, 2026, 11:18 PM UTC
Duration
58m
Detected by Pingoru
Oct 08, 2026, 10:20 PM UTC

Affected components

API Gateway Service (plugin)

Update timeline

  1. investigating Oct 08, 2026, 11:32 PM UTC

    Started: 2026-10-08 22:20 UTC Impact: Plugin is unavailable for customers — the editor does not load / does not respond. We have received reports from several customers. What happened: At 22:13 UTC Zabbix reported the agent on the Plugin DB host (pg-plugins.stripo.email, 172.31.13.64) as unavailable; at 22:40 UTC the PostgreSQL monitor for live-plugins failed as well. The Plugin stopped working for customers at ~22:20 UTC. DevOps is investigating. Next update here as soon as we have more information.

  2. resolved Oct 08, 2026, 11:33 PM UTC

    The Plugin is fully operational again. DevOps restored the Plugin DB host; we are monitoring and have seen no further errors since 23:18 UTC. A postmortem with root cause, timeline, and action items is being prepared and will be published shortly.

  3. postmortem Oct 09, 2026, 09:14 AM UTC

    **SUMMARY** On 8 October 2026, between 22:20 and 23:18 UTC, the Stripo Plugin was unavailable: the editor did not load or respond. The outage lasted 58 minutes. No customer data was lost, and other Stripo products were not affected. **WHAT HAPPENED** The database server behind the Plugin lost time synchronization. Two time-sync services were running on the server at once: the standard daemon, which adjusts the clock gradually, and a legacy scheduled job that steps the clock abruptly. A misconfigured schedule made the legacy job run almost every minute. The repeated clock jumps destabilized the server until the Plugin's database became unavailable. **TIMELINE \(UTC, 8 October 2026\)** 21:44 - Clock instability begins on the Plugin database server. 22:13 - Our monitoring raises the first alert for the server. 22:20 - The Plugin stops loading for customers. 22:40 - Monitoring reports the Plugin database as unavailable. 23:00 - First customer reports are received; the on-duty engineer is called in. 23:18 - Service restored. The Plugin is fully operational. **RESOLUTION** Our engineers identified the time-sync conflict and restored the server. The Plugin returned to normal operation at 23:18 UTC and has been under extended monitoring since. **WHAT WE ARE DOING TO PREVENT THIS** * Removing the legacy time-sync job so that this server, and every other production server, uses a single time source. * Adding monitoring for clock drift so that this kind of fault is caught before it affects service. * Changing alert escalation. Our monitoring alerted seven minutes before the outage, but the alert did not page an engineer. Critical alerts on production database servers will now page the on-duty engineer immediately, at any hour. We apologize for the disruption and thank you for your patience. If you were affected and have questions, please contact our support team.