Travis CI incident

Slow booting Linux builds for both open source and private repositories

Minor Resolved View vendor source →

Travis CI experienced a minor incident on July 2, 2019 affecting Linux and Windows Builds, lasting 11h 29m. The incident has been resolved; the full update timeline is below.

Started
Jul 02, 2019, 03:16 PM UTC
Resolved
Jul 03, 2019, 02:45 AM UTC
Duration
11h 29m
Detected by Pingoru
Jul 02, 2019, 03:16 PM UTC

Affected components

Linux and Windows Builds

Update timeline

  1. investigating Jul 02, 2019, 03:16 PM UTC

    We are receiving reports of slow booting Linux builds. We are currently looking into it.

  2. investigating Jul 02, 2019, 04:25 PM UTC

    We are currently hitting the API rate-limit when using our Cloud Provider’s API that is preventing us from creating/deleting VMs. We are looking into getting the system back under the API rate-limit threshold. Thank you for your patience.

  3. identified Jul 02, 2019, 05:22 PM UTC

    We’ve made progress in being able to stay under the API rate-limit and things are looking better on travis-ci.com. A higher backlog of jobs remains on travis-ci.org. We’ll keep you posted on the status of each backlog on a timely basis.

  4. identified Jul 02, 2019, 09:07 PM UTC

    Sorry for the lack of updates. It's been challenging to remain below the API rate-limit and the backlogs haven't been decreasing at a steady pace as previously mentioned. We now confirm that the backlog on .com is decreasing at a comfortable pace while the backlog on .org is over its peak. Thanks for hanging in there with us.

  5. monitoring Jul 02, 2019, 10:21 PM UTC

    Backlog on travis-ci.com has cleared around 21:55 UTC. We are monitoring travis-ci.org's backlog closely.

  6. resolved Jul 03, 2019, 02:45 AM UTC

    Backlog on travis-ci.org has cleared has of 2019-07-03 2:14 UTC. Thanks again for your continued patience during this incident.