Paubox incident

MX1 Mail Delay affecting limited group of customers

Minor Resolved View vendor source →

Paubox experienced a minor incident on September 12, 2023 affecting Inbound Security and Hosted Mail, lasting 14h 53m. The incident has been resolved; the full update timeline is below.

Started
Sep 12, 2023, 02:55 PM UTC
Resolved
Sep 13, 2023, 05:48 AM UTC
Duration
14h 53m
Detected by Pingoru
Sep 12, 2023, 02:55 PM UTC

Affected components

Inbound SecurityHosted Mail

Update timeline

  1. investigating Sep 12, 2023, 02:55 PM UTC

    We are investigating a mail delay on the MX1 server cluster. Only customers on this cluster have been affected.

  2. investigating Sep 12, 2023, 04:03 PM UTC

    While customers on MX1 may be experiencing mail delays up to ~40min, we are seeing the queue recover. We're continuing to investigate.

  3. identified Sep 12, 2023, 04:46 PM UTC

    Engineering team has identified the cause of the bottleneck leading to slower than usual mail processing time. The root cause is currently being confirmed. Customers on MX1 are experiencing mail delays of ~9 min and falling.

  4. monitoring Sep 12, 2023, 07:21 PM UTC

    Changes have been applied to the inbound security pipeline. The engineering team is closely monitoring the results. Delays have been resolved.

  5. resolved Sep 13, 2023, 05:48 AM UTC

    This incident has been resolved.