GitLab incident

Background processing delays

Minor Resolved View vendor source →

GitLab experienced a minor incident on May 7, 2026 affecting Background Processing and GitLab agent server for Kubernetes (Google Compute Engine), lasting 4d 8h. The incident has been resolved; the full update timeline is below.

Started
May 07, 2026, 03:44 PM UTC
Resolved
May 11, 2026, 11:56 PM UTC
Duration
4d 8h
Detected by Pingoru
May 07, 2026, 03:44 PM UTC

Affected components

Background ProcessingGitLab agent server for Kubernetes (Google Compute Engine)

Update timeline

  1. investigating May 07, 2026, 03:44 PM UTC

    We are investigating an issue background jobs being processed with delay. The problem is being tracked in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22017. Back to current status Status History Subscribe to receive status updates by email Close Subscribe Manage Subscription Manage Existing Subscription Create New Subscription Subscribe to receive status updates by webhook Each status update will POST a JSON payload to this URL Email address for managing webhook Close Subscribe Manage Subscription Manage Existing Subscription Create New Subscription Subscribe to receive status updates in Microsoft Teams Enter your Microsoft Teams webhook. View Instructions Email address for managing subscription Close Subscribe Manage Subscription Manage Existing Subscription Create New Subscription Subscribe to receive status updates via RSS RSS Feed Close Follow our status updates on Twitter/X @gitlabstatus Close Subscribe to receive status updates in Slack Slack channel ID Find the channel ID: Select the channel in your Slack workspace. The channel ID is displayed in the browser URL. Example: https://app.slack.com/client/T04SJBK1C/ C03SKGJ1P Email address Close Manage Subscription Manage Existing Subscription Create New Subscription Subscribe to receive status updates via iCalendar iCalendar Feed Close

  2. investigating May 07, 2026, 03:44 PM UTC

    We are investigating an issue background jobs being processed with delay. The problem is being tracked in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22017.

  3. identified May 07, 2026, 04:45 PM UTC

    We have identified the cause of the background job processing delays. We are working to identify mitigating measures.

  4. identified May 07, 2026, 04:45 PM UTC

    We have identified the cause of the background job processing delays. We are working to identify mitigating measures.

  5. monitoring May 07, 2026, 05:52 PM UTC

    The background job backlog has mostly cleared and merge train delays should be significantly reduced. We're keeping this incident in monitoring while we implement performance improvements. More: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22017

  6. monitoring May 07, 2026, 05:52 PM UTC

    The background job backlog has mostly cleared and merge train delays should be significantly reduced. We're keeping this incident in monitoring while we implement performance improvements. More: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22017

  7. resolved May 11, 2026, 11:56 PM UTC

    Background job processing has returned to normal and this issue is resolved. More information available at https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22017