Gigalixir incident

GCP us-central1 Degraded Performance

Minor Resolved View vendor source →

Gigalixir experienced a minor incident on July 15, 2024, lasting 3h 58m. The incident has been resolved; the full update timeline is below.

Started
Jul 15, 2024, 12:57 PM UTC
Resolved
Jul 15, 2024, 04:55 PM UTC
Duration
3h 58m
Detected by Pingoru
Jul 15, 2024, 12:57 PM UTC

Update timeline

  1. investigating Jul 15, 2024, 12:57 PM UTC

    ## **Scope** Some applications in the us-central1 region experienced degraded packet delivery from approximately 13:57 UTC (08:57 CDT) to 14:11 UTC (09:11 CDT), with most applications recovering by 14:04 UTC (9:04 CDT). [gigalixir.com](http://gigalixir.com), the [Gigalixir console](https://console.gigalixir.com/) and use of the Gigalixir CLI and APIs were also affected during this time..

  2. resolved Jul 15, 2024, 04:55 PM UTC

    This incident has been resolved. ## **Timeline** ### **16:55 UTC / 11:55 CDT** The root cause has been identified as a flood of malformed requests into load balancers. We are investing measures to help prevent similar issues in the future. If you have any questions, feel free to contact support at [[email protected]](mailto:[email protected]). ### **14:11 UTC / 09:11 CDT** At this time we show everything is operational and we are continuing to monitor the situation. Please report back if you are still having any issues. Contact us at [[email protected]](mailto:[email protected]) if you have any questions. ### **13:57 UTC / 08:57 CDT** We are experiencing delayed packet delivery and increased packet loss in the us-central1 region. At this time we are investigating the issue and will provide updates as we have them..