WellSaid Labs incident

Cloud provider core infrastructure unavailability

Resolved after 2 hours 13 minutes, affecting Editor and Developer API and 1 more, on June 12, 2025.

monitoring Jun 12, 2025, 08:32 PM UTC

Services are now operational, though some requests may still be slower than normal. Clip creation is functioning, but clips generated during the outage remain unavailable. Our team is continuing to monitor the system and investigating options to recover the affected clip data.

Started
Jun 12, 2025, 06:33 PM UTC
Resolved
Jun 12, 2025, 08:46 PM UTC
Duration
2h 13m
Detected by Pingoru
Jun 12, 2025, 06:33 PM UTC

Affected components

EditorDeveloper APIAPIGoogle Cloud Platform Google Kubernetes EngineClip DownloadingGoogle Cloud Platform Google Cloud StorageGoogle Cloud Platform Identity & Access Management

Update timeline

  1. investigating Jun 12, 2025, 06:33 PM UTC

    We are currently experiencing issues with our Cloud provider's core infrastructure, which is affecting a large portion of network traffic. We are monitoring this outage at the PaloAltoNetworks level https://status.paloaltonetworks.com/incidents/l4zv6n67r8x6

  2. identified Jun 12, 2025, 06:33 PM UTC

    The issue has been identified as outside our infrastructure and reported by our cloud providers.

  3. identified Jun 12, 2025, 06:40 PM UTC

    We are continuing to work on a fix for this issue.

  4. identified Jun 12, 2025, 07:05 PM UTC

    Google Cloud has reported a major service outage across IAM affecting the entire platform https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW

  5. identified Jun 12, 2025, 07:45 PM UTC

    Our cloud provider has reported that most locations have fully recovered. However, some features—such as clip interactions—remain unavailable. We’re actively working to restore full functionality and will provide further updates as they become available.

  6. monitoring Jun 12, 2025, 08:17 PM UTC

    Services are beginning to recover, and clip creation is now fully operational. However, clips that were created during the outage remain unavailable. Our engineering team is actively investigating options to recover those affected clips.

  7. monitoring Jun 12, 2025, 08:32 PM UTC

    Services are now operational, though some requests may still be slower than normal. Clip creation is functioning, but clips generated during the outage remain unavailable. Our team is continuing to monitor the system and investigating options to recover the affected clip data.

  8. resolved Jun 12, 2025, 08:46 PM UTC

    Services are now fully operational, and all systems are performing as expected.

Your product runs on more than WellSaid Labs

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →