Particle incident

Connectivity Outage

Critical Resolved View vendor source →

Particle experienced a critical incident on January 16, 2025 affecting Cellular Connectivity and Ether SIM, lasting 7h 48m. The incident has been resolved; the full update timeline is below.

Started
Jan 16, 2025, 03:03 PM UTC
Resolved
Jan 16, 2025, 10:51 PM UTC
Duration
7h 48m
Detected by Pingoru
Jan 16, 2025, 03:03 PM UTC

Affected components

Cellular ConnectivityEther SIM

Update timeline

  1. investigating Jan 16, 2025, 03:03 PM UTC

    Particle is investigating reports of a rolling connectivity outage; update on status is incoming.

  2. investigating Jan 16, 2025, 03:19 PM UTC

    Particle has identified an increase in handshake failures, impacting devices that are attempting to reconnect. We are actively investigating with the highest priority and escalating to our cellular partners.

  3. identified Jan 16, 2025, 03:33 PM UTC

    The issue has been identified as a network infrastructure issue. Our partners are enaged in pursuing a fix. We will post an update ASAP. Only a subset of EtherSIM devices are impacted, specifically those with short reconnection timelines.

  4. identified Jan 16, 2025, 04:42 PM UTC

    Work is ongoing to rectify this issue, which we have escalated with the highest priority. We will continue to provide updates with regularity.

  5. identified Jan 16, 2025, 06:38 PM UTC

    Work continues to rectify this issue, no affirmative ETA as of yet. To reiterate, this issue affects EtherSIM devices primarily in the US, specifically during the reconnect process.

  6. identified Jan 16, 2025, 07:32 PM UTC

    Particle is beginning to observe a steady drop in failed reconnect attempts, pointing to progress being made with respect to resolution.

  7. identified Jan 16, 2025, 07:48 PM UTC

    Particle is continuing to observe an improvement in connectivity conditions. To reiterate, an issue experience by one of our network providers has caused an interruption in EtherSIM traffic, specifically interrupting a device's ability to reconnect to the Cloud. We will continue to post live updates as the situation improves.

  8. identified Jan 16, 2025, 08:50 PM UTC

    Improvement is still ongoing, we've seen a 60% reduction in problematic connections with the impacted network provider over the past 90 minutes and we are seeing data throughput volumes converge to normal levels. The incident remains, nonetheless, at our highest stage of escalation. We will provide another update as progress continues.

  9. monitoring Jan 16, 2025, 09:55 PM UTC

    Particle has observed a significant reduction in the reconnection issue affecting EtherSIM devices. We have received word from our impacted partner that their resolution work is finishing in a satisfactory manner. We will keep a very close eye on the stability of the system for the next 60 minutes before making any further assessments.

  10. resolved Jan 16, 2025, 10:51 PM UTC

    Particle has assessed a complete and sustained return of traffic to normal levels. We will continue to monitor the recovery closely, but at this moment will mark the incident resolved. We will append an RCA to this incident notification at the moment we receive it from the impacted provider.