Wikimedia incident

Database issues in the Dallas TX datacenter

Major Resolved View vendor source →

Wikimedia experienced a major incident on October 24, 2024 affecting Reading and Editing, lasting 3h. The incident has been resolved; the full update timeline is below.

Started
Oct 24, 2024, 11:02 AM UTC
Resolved
Oct 24, 2024, 02:02 PM UTC
Duration
3h
Detected by Pingoru
Oct 24, 2024, 11:02 AM UTC

Affected components

ReadingEditing

Update timeline

  1. investigating Oct 24, 2024, 11:02 AM UTC

    Reading and editing is affected. We are investigating the issue.

  2. investigating Oct 24, 2024, 11:33 AM UTC

    We shifted read traffic to the secondary datacenter, which helped with the overloaded database. We are still investigating the root cause.

  3. monitoring Oct 24, 2024, 01:49 PM UTC

    We have repooled the primary datacenter for read traffic and are monitoring the situation while investigating the root cause.

  4. monitoring Oct 24, 2024, 01:52 PM UTC

    We are continuing to monitor for any further issues.

  5. resolved Oct 24, 2024, 02:02 PM UTC

    Traffic back to baseline, root cause investigation is still ongoing but the incident is resolved.