Semaphore Outage History

Semaphore is up right now

Pingoru has recorded 56 incidents for Semaphore: 44 in the last 12 months and 19 in the last 90 days.

Source: status.semaphoreci.com

56incidents recorded
44last 12 months
19last 90 days
4 hourstypical duration
Incidents per month
048AprMayJunJulAugSepOct6
Share of hours with an open incident, any severity
0%50%100%AprMayJunJulAugSepOct4%
Pingoru began tracking Semaphore in April 2026, so earlier months are not shown. Hover a bar for its value.
Show as a table
MonthIncidentsHours with an open incident
April 202600 of 720 (0%)
May 2026213 of 744 (2%)
June 202622 of 720 (0%)
July 2026411 of 744 (1%)
August 2026826 of 744 (3%)
September 2026532 of 720 (4%)
October 2026619 of 524 (4%)

Incidents, newest first

Minor October 22, 2026 Delays in Job Processing 2h 15m Semaphore

Started Oct 22, 2026, 03:08 PM UTC· resolved Oct 22, 2026, 05:23 PM UTC

  1. investigating Oct 22, 2026, 03:08 PM UTC

    We are currently investigating this issue.

  2. investigating Oct 22, 2026, 04:03 PM UTC

    We are continuing to investigate this issue.

  3. identified Oct 22, 2026, 05:01 PM UTC

    The issue has been identified and a fix is being implemented.

  4. monitoring Oct 22, 2026, 05:12 PM UTC

    A fix has been implemented and we are monitoring the results.

  5. resolved Oct 22, 2026, 05:23 PM UTC

    This incident has been resolved.

Read the full incident report →

Notice October 20, 2026 Delays in jobs scheduling 2h 1m Semaphore

Started Oct 20, 2026, 09:58 PM UTC· resolved Oct 20, 2026, 11:59 PM UTC

  1. identified Oct 20, 2026, 09:58 PM UTC

    The issue has been identified and a fix is being implemented.

  2. identified Oct 20, 2026, 10:27 PM UTC

    We are continuing to work on a fix for this issue.

  3. identified Oct 20, 2026, 10:44 PM UTC

    We're starting to see improvements in job scheduling times, although some delays may still occur. We're continuing to monitor the situation closely and will provide further updates as we make progress.

  4. monitoring Oct 20, 2026, 11:10 PM UTC

    Job scheduling times have returned to normal. We're continuing to monitor the situation closely to ensure service continues running as expected.

  5. monitoring Oct 20, 2026, 11:30 PM UTC

    We are continuing to monitor the situation.

  6. resolved Oct 20, 2026, 11:59 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor October 14, 2026 Delays in job processing 2h 7m Semaphore

Started Oct 14, 2026, 03:10 PM UTC· resolved Oct 14, 2026, 05:17 PM UTC

  1. investigating Oct 14, 2026, 03:10 PM UTC

    Jobs are currently experiencing delays during initialization. We are investigating the issue and will share updates as we have more information.

  2. identified Oct 14, 2026, 03:55 PM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring Oct 14, 2026, 04:19 PM UTC

    A fix has been implemented and we are monitoring the results.

  4. monitoring Oct 14, 2026, 05:09 PM UTC

    We are continuing to monitor for any further issues.

  5. resolved Oct 14, 2026, 05:17 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor October 13, 2026 Intermittent network connectivity issues 8h 36m Semaphore

Started Oct 13, 2026, 01:53 PM UTC· resolved Oct 13, 2026, 10:30 PM UTC

  1. investigating Oct 13, 2026, 01:53 PM UTC

    We are currently investigating this issue.

  2. investigating Oct 13, 2026, 02:32 PM UTC

    We are continuing to investigate this issue.

  3. identified Oct 13, 2026, 02:40 PM UTC

    The issue has been identified and a fix is being implemented.

  4. identified Oct 13, 2026, 03:42 PM UTC

    We are continuing to work on a fix for this issue.

  5. investigating Oct 13, 2026, 04:34 PM UTC

    We identified that the intermittent network connectivity issues are likely related to our infrastructure provider's recent maintenance. Although the maintenance is complete, we're still experiencing intermittent connection interruptions and slow network performance.

  6. investigating Oct 13, 2026, 06:16 PM UTC

    We are continuing to investigate this issue.

  7. identified Oct 13, 2026, 06:31 PM UTC

    Our investigation has identified intermittent connectivity issues between our infrastructure provider’s network and certain Google services. This may affect artifact uploads and downloads, as well as jobs that depend on Google-hosted resources. We’ve gathered diagnostic information and are escalating the issue to our infrastructure provider for further investigation. We’ll continue monitoring the situation and share updates as significant new information becomes available.

  8. monitoring Oct 13, 2026, 09:45 PM UTC

    Network connectivity is returning to normal, and the affected services should no longer be impacted. We will continue monitoring the situation closely. If you continue to experience similar issues, please contact our support team at [email protected].

  9. resolved Oct 13, 2026, 10:30 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor September 17, 2026 Intermittent cache connection errors 38m Semaphore

Started Sep 17, 2026, 01:33 PM UTC· resolved Sep 17, 2026, 02:12 PM UTC

  1. investigating Sep 17, 2026, 01:33 PM UTC

    We are currently investigating this issue.

  2. identified Sep 17, 2026, 01:52 PM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring Sep 17, 2026, 01:56 PM UTC

    A fix has been implemented and we are monitoring the results.

  4. resolved Sep 17, 2026, 02:12 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor September 2, 2026 Periodic authentication failure with GitHub repositories 1d Semaphore

Started Sep 02, 2026, 04:08 PM UTC· resolved Sep 03, 2026, 04:41 PM UTC

  1. identified Sep 02, 2026, 04:08 PM UTC

    The wider situation impacting HTTPS operations, which is also affecting Semaphore and our users, has started occurring again: https://github.com/orgs/community/discussions/206581 As a workaround, you can configure git to use only HTTP/1.1, using the following command: git config --global http.version HTTP/1.1 We are continuing to monitor the situation on our end and will share further updates as they become available.

  2. identified Sep 02, 2026, 04:35 PM UTC

    The wider situation impacting HTTPS operations, which is also affecting Semaphore and our users, has started occurring again: https://github.com/orgs/community/discussions/206581 As a workaround, you can configure git to use only HTTP/1.1, using the following command: git config --global http.version HTTP/1.1 We are continuing to monitor the situation on our end and will share further updates as they become available.

  3. identified Sep 02, 2026, 05:04 PM UTC

    The wider situation impacting HTTPS operations, which is also affecting Semaphore and our users, has started occurring again: https://github.com/orgs/community/discussions/206581 As a workaround, you can configure git to use only HTTP/1.1, using the following command: git config --global http.version HTTP/1.1 We are continuing to monitor the situation on our end and will share further updates as they become available.

  4. identified Sep 02, 2026, 05:32 PM UTC

    The wider situation impacting HTTPS operations, which is also affecting Semaphore and our users, has started occurring again: https://github.com/orgs/community/discussions/206581 As a workaround, you can configure git to use only HTTP/1.1, using the following command: git config --global http.version HTTP/1.1 We are continuing to monitor the situation on our end and will share further updates as they become available.

  5. identified Sep 02, 2026, 06:02 PM UTC

    The wider situation impacting HTTPS operations, which is also affecting Semaphore and our users, has started occurring again: https://github.com/orgs/community/discussions/206581 As a workaround, you can configure git to use only HTTP/1.1, using the following command: git config --global http.version HTTP/1.1 We are continuing to monitor the situation on our end and will share further updates as they become available.

  6. identified Sep 02, 2026, 06:31 PM UTC

    The wider situation impacting HTTPS operations, which is also affecting Semaphore and our users, has started occurring again: https://github.com/orgs/community/discussions/206581 As a workaround, you can configure git to use only HTTP/1.1, using the following command: git config --global http.version HTTP/1.1 We are continuing to monitor the situation on our end and will share further updates as they become available.

  7. identified Sep 02, 2026, 07:00 PM UTC

    The wider situation impacting HTTPS operations, which is also affecting Semaphore and our users, has started occurring again: https://github.com/orgs/community/discussions/206581 As a workaround, you can configure git to use only HTTP/1.1, using the following command: git config --global http.version HTTP/1.1 We are continuing to monitor the situation on our end and will share further updates as they become available.

  8. identified Sep 02, 2026, 07:32 PM UTC

    The wider situation impacting HTTPS operations, which is also affecting Semaphore and our users, has started occurring again: https://github.com/orgs/community/discussions/206581 As a workaround, you can configure git to use only HTTP/1.1, using the following command: git config --global http.version HTTP/1.1 We are continuing to monitor the situation on our end and will share further updates as they become available.

  9. identified Sep 02, 2026, 08:00 PM UTC

    The wider situation impacting HTTPS operations, which is also affecting Semaphore and our users, has started occurring again: https://github.com/orgs/community/discussions/206581 As a workaround, you can configure git to use only HTTP/1.1, using the following command: git config --global http.version HTTP/1.1 We are continuing to monitor the situation on our end and will share further updates as they become available.

  10. identified Sep 02, 2026, 08:30 PM UTC

    The wider situation impacting HTTPS operations, which is also affecting Semaphore and our users, has started occurring again: https://github.com/orgs/community/discussions/206581 As a workaround, you can configure git to use only HTTP/1.1, using the following command: git config --global http.version HTTP/1.1 We are continuing to monitor the situation on our end and will share further updates as they become available.

  11. identified Sep 02, 2026, 09:00 PM UTC

    The wider situation impacting HTTPS operations, which is also affecting Semaphore and our users, has started occurring again: https://github.com/orgs/community/discussions/206581 As a workaround, you can configure git to use only HTTP/1.1, using the following command: git config --global http.version HTTP/1.1 We are continuing to monitor the situation on our end and will share further updates as they become available.

  12. identified Sep 02, 2026, 09:30 PM UTC

    The wider situation impacting HTTPS operations, which is also affecting Semaphore and our users, has started occurring again: https://github.com/orgs/community/discussions/206581 As a workaround, you can configure git to use only HTTP/1.1, using the following command: git config --global http.version HTTP/1.1 We are continuing to monitor the situation on our end and will share further updates as they become available.

  13. identified Sep 02, 2026, 10:07 PM UTC

    Our engineers are actively monitoring the ongoing situation affecting HTTPS operations: https://github.com/orgs/community/discussions/206581. We’ll share the next update once we have new information from GitHub. As a workaround, you can configure git to use only HTTP/1.1, using the following command: git config --global http.version HTTP/1.1

  14. identified Sep 03, 2026, 12:45 PM UTC

    Our engineers are actively monitoring the intermittent issue affecting Git operations over HTTPS. We will share another update when more information becomes available. To prevent affected builds from stalling while waiting for credentials, our engineering team has implemented fail-fast behavior. Affected builds will now fail promptly, reducing unnecessary runtime. If the issue persists after rerunning the job, add the following command to the job before the affected Git operation to force Git to use HTTP/1.1: git config --global http.version HTTP/1.1

  15. resolved Sep 03, 2026, 04:41 PM UTC

    Semaphore is currently operating normally, which is why we’re marking this incident as resolved. GitHub confirmed that the behavior affecting unauthenticated Git operations is the result of an intentional change on their end rather than a bug. You can read their latest statement here: https://github.com/orgs/community/discussions/206581#discussioncomment-18269083 For builds affected by this behavior, the previously shared workaround remains available. Configure Git to use HTTP/1.1 before running the affected Git operation: git config --global http.version HTTP/1.1 We will add this instruction and further context to our documentation. Any additional guidance from Semaphore will be shared through in-app notifications. For further assistance, contact our support team at [email protected].

Read the full incident report →

Critical August 27, 2026 Platform Outage 47m Semaphore

Started Aug 27, 2026, 06:43 PM UTC· resolved Aug 27, 2026, 07:30 PM UTC

  1. identified Aug 27, 2026, 06:43 PM UTC

    The issue has been identified and a fix is being implemented.

  2. monitoring Aug 27, 2026, 07:00 PM UTC

    A fix has been implemented and we are monitoring the results.

  3. monitoring Aug 27, 2026, 07:15 PM UTC

    We are continuing to monitor for any further issues.

  4. resolved Aug 27, 2026, 07:30 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor August 25, 2026 Degraded performance on r1-standard machines 6h 11m Semaphore

Started Aug 25, 2026, 08:35 AM UTC· resolved Aug 25, 2026, 02:47 PM UTC

  1. investigating Aug 25, 2026, 08:35 AM UTC

    We are currently investigating this issue.

  2. identified Aug 25, 2026, 08:44 AM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring Aug 25, 2026, 09:19 AM UTC

    A fix has been implemented and we are monitoring the results.

  4. monitoring Aug 25, 2026, 12:57 PM UTC

    We are continuing to monitor for any further issues.

  5. identified Aug 25, 2026, 02:00 PM UTC

    We have observed continued impact while monitoring. The issue remains identified, and our team is working on a resolution.

  6. identified Aug 25, 2026, 02:17 PM UTC

    We are continuing to work on a fix for this issue.

  7. resolved Aug 25, 2026, 02:47 PM UTC

    This incident has been resolved.

Read the full incident report →

Major August 21, 2026 Issues with Bitbucket integration 1h 33m Semaphore

Started Aug 21, 2026, 07:14 PM UTC· resolved Aug 21, 2026, 08:47 PM UTC

  1. investigating Aug 21, 2026, 07:14 PM UTC

    We are currently investigating this issue.

  2. identified Aug 21, 2026, 07:26 PM UTC

    The issue has been identified and a fix is being implemented.

  3. identified Aug 21, 2026, 08:03 PM UTC

    We are continuing to work on a fix for this issue.

  4. monitoring Aug 21, 2026, 08:27 PM UTC

    A fix has been implemented and we are monitoring the results.

  5. resolved Aug 21, 2026, 08:47 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor August 18, 2026 Delays in job processing - F1 machines 2h 40m Semaphore

Started Aug 18, 2026, 03:07 PM UTC· resolved Aug 18, 2026, 05:47 PM UTC

  1. investigating Aug 18, 2026, 03:07 PM UTC

    We are currently investigating this issue.

  2. investigating Aug 18, 2026, 03:15 PM UTC

    Jobs configured to run on F1 machines may take longer than usual to start.

  3. identified Aug 18, 2026, 03:18 PM UTC

    The issue has been identified and a fix is being implemented.

  4. monitoring Aug 18, 2026, 03:24 PM UTC

    A fix has been implemented and we are monitoring the results.

  5. identified Aug 18, 2026, 03:28 PM UTC

    The issue is still ongoing, and the engineering team is actively working on a fix.

  6. identified Aug 18, 2026, 04:28 PM UTC

    We are continuing to work on a fix for this issue.

  7. monitoring Aug 18, 2026, 05:16 PM UTC

    A fix has been implemented and we are monitoring the results.

  8. resolved Aug 18, 2026, 05:47 PM UTC

    This incident has been resolved.

Read the full incident report →

Major August 4, 2026 Bitbucket integration may currently be experiencing issues 1h 25m Semaphore

Started Aug 04, 2026, 10:38 AM UTC· resolved Aug 04, 2026, 12:04 PM UTC

  1. identified Aug 04, 2026, 10:38 AM UTC

    The issue has been identified and a fix is being implemented.

  2. monitoring Aug 04, 2026, 11:30 AM UTC

    A fix has been implemented and we are monitoring the results.

  3. resolved Aug 04, 2026, 12:04 PM UTC

    This incident has been resolved.

Read the full incident report →

Major July 21, 2026 Some jobs may have issues authenticating with GitHub 5h 10m Semaphore

Started Jul 21, 2026, 08:36 AM UTC· resolved Jul 21, 2026, 01:46 PM UTC

  1. investigating Jul 21, 2026, 08:36 AM UTC

    We are currently investigating this issue.

  2. investigating Jul 21, 2026, 08:49 AM UTC

    As a workaround to get more consistent builds during this time, please try adding the --use-cache flag to your checkout command. Read more about it here: https://docs.semaphore.io/reference/toolbox#cache-full-clone

  3. investigating Jul 21, 2026, 09:06 AM UTC

    Our engineering team is still investigating and working on this issue, but as a current workaround, you can regenerate your GitHub deployment key. You can find the instructions on how to do that here: https://docs.semaphore.io/using-semaphore/connect-github#deploy-key

  4. monitoring Jul 21, 2026, 09:25 AM UTC

    Our team is monitoring the situation. If you are affected, please regenerate your deploy key by following the instructions on this page: https://docs.semaphore.io/using-semaphore/connect-github#deploy-key

  5. monitoring Jul 21, 2026, 10:53 AM UTC

    We are continuing to monitor the situation, and you can follow the current GitHub incident at the following link: https://www.githubstatus.com/incidents/g40zcbvchny4

  6. monitoring Jul 21, 2026, 12:18 PM UTC

    GitHub confirmed that the incident on their end has been resolved, and services should now be returning to normal. We are continuing to monitor the situation. You can review the GitHub update here: https://www.githubstatus.com/incidents/g40zcbvchny4

  7. resolved Jul 21, 2026, 01:46 PM UTC

    GitHub confirmed that the incident on their end has been resolved. You can review the GitHub update here: https://www.githubstatus.com/incidents/g40zcbvchny4

Read the full incident report →

Major July 17, 2026 Some features may currently be unresponsive 2h 53m Semaphore

Started Jul 17, 2026, 10:32 AM UTC· resolved Jul 17, 2026, 01:25 PM UTC

  1. investigating Jul 17, 2026, 10:32 AM UTC

    We are currently investigating this issue.

  2. monitoring Jul 17, 2026, 10:51 AM UTC

    A fix has been implemented and we are monitoring the results.

  3. resolved Jul 17, 2026, 01:25 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor June 24, 2026 Elevated error rates 24m Semaphore

Started Jun 24, 2026, 09:01 PM UTC· resolved Jun 24, 2026, 09:26 PM UTC

  1. investigating Jun 24, 2026, 09:01 PM UTC

    We are currently investigating this issue.

  2. identified Jun 24, 2026, 09:07 PM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring Jun 24, 2026, 09:16 PM UTC

    A fix has been implemented and we are monitoring the results.

  4. resolved Jun 24, 2026, 09:26 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor June 10, 2026 Elevated Error Rates 1h 9m Semaphore

Started Jun 10, 2026, 03:12 PM UTC· resolved Jun 10, 2026, 04:22 PM UTC

  1. identified Jun 10, 2026, 03:12 PM UTC

    We're experiencing an elevated level of 500's errors and are currently looking into the issue.

  2. identified Jun 10, 2026, 03:20 PM UTC

    The issue has been identified, and our team is implementing a fix.

  3. monitoring Jun 10, 2026, 04:14 PM UTC

    A fix has been implemented and we are monitoring the results.

  4. resolved Jun 10, 2026, 04:22 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor May 20, 2026 Delays when checking out GitHub repos via SSH 6h 47m Semaphore

Started May 20, 2026, 08:44 AM UTC· resolved May 20, 2026, 03:32 PM UTC

  1. identified May 20, 2026, 08:44 AM UTC

    We identified slow communication between Semaphore jobs and GitHub. We are continuing to work with our upstream provider to resolve this. For impacted jobs, please replace the normal checkout command with the following: retry --times 5 "timeout 40 bash -lc 'source ~/.toolbox/toolbox; checkout'" And afterwards, use cd to change the directory to the cloned repository (replace with your project).

  2. monitoring May 20, 2026, 12:48 PM UTC

    A fix has been implemented and we are monitoring the results.

  3. identified May 20, 2026, 01:59 PM UTC

    We identified slow GitHub operations related to problems with the network, and we deployed a fix to re-route those operations to a healthier server. Some HTTPS operations started to fail because of that. To disable the fix, you should use the command: sudo sed -i 's/^\([0-9].*github\.com\)/#\1/' /etc/hosts And to re-enable, you can use: sudo sed -i 's/^#\([0-9].*github\.com\)/\1/' /etc/hosts

  4. monitoring May 20, 2026, 03:02 PM UTC

    A fix has been implemented, and everything should be back to normal. The previously shared snippets can be removed.

  5. resolved May 20, 2026, 03:32 PM UTC

    This incident has been resolved.

Read the full incident report →

Major February 24, 2026 Jobs Not Starting and Processing Delays 1h 42m Semaphore

Started Feb 24, 2026, 08:49 AM UTC· resolved Feb 24, 2026, 10:32 AM UTC

  1. investigating Feb 24, 2026, 08:49 AM UTC

    We are currently investigating this issue.

  2. investigating Feb 24, 2026, 09:06 AM UTC

    We are continuing to investigate this issue.

  3. investigating Feb 24, 2026, 09:34 AM UTC

    We are continuing to investigate job start failures and processing delays. We will continue to update while investigating this.

  4. identified Feb 24, 2026, 09:54 AM UTC

    The issue has been identified and we are working on a fix. We will provide our next update within 30 minutes or sooner if things change. We thank you for your patience while we continue to work to resolve this issue.

  5. monitoring Feb 24, 2026, 10:05 AM UTC

    A fix has been implemented and we are monitoring the results. We will provide our next update within 30 minutes or sooner if things change.

  6. resolved Feb 24, 2026, 10:32 AM UTC

    This incident has been resolved.

Read the full incident report →

Minor February 3, 2026 Delivering webhooks delays between Github and Semaphore 4h 3m Semaphore

Started Feb 03, 2026, 03:46 PM UTC· resolved Feb 03, 2026, 07:50 PM UTC

  1. investigating Feb 03, 2026, 03:46 PM UTC

    We are currently investigating this issue.

  2. identified Feb 03, 2026, 05:34 PM UTC

    Webhook delivery delays between GitHub and Semaphore appear to be caused by degraded performance on GitHub’s side. GitHub is currently reporting an active incident that appears related. We are monitoring the situation closely and will provide updates as GitHub resolves the issue.

  3. monitoring Feb 03, 2026, 07:11 PM UTC

    GitHub has posted an update indicating recovery of the aforementioned issue. We continue monitoring Semaphore closely as things stabilize. (GitHub Status: https://www.githubstatus.com/incidents/f314nlctbfs5)

  4. resolved Feb 03, 2026, 07:50 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor January 27, 2026 Delays in job processing on Semaphore 3h 36m Semaphore

Started Jan 27, 2026, 03:34 PM UTC· resolved Jan 27, 2026, 07:10 PM UTC

  1. identified Jan 27, 2026, 03:34 PM UTC

    The issue has been identified and a fix is being implemented.

  2. identified Jan 27, 2026, 04:35 PM UTC

    We are continuing to work on a fix for this issue.

  3. identified Jan 27, 2026, 05:36 PM UTC

    We are continuing to work on a fix for this issue.

  4. monitoring Jan 27, 2026, 06:49 PM UTC

    A fix has been implemented and we are monitoring the results.

  5. resolved Jan 27, 2026, 07:10 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor November 12, 2025 Network connectivity issues towards us-east 3h 14m Semaphore

Started Nov 12, 2025, 02:55 PM UTC· resolved Nov 12, 2025, 06:10 PM UTC

  1. investigating Nov 12, 2025, 02:55 PM UTC

    We are currently investigating the issue. Implementing the retry command should help mitigate the issue: https://docs.semaphore.io/reference/toolbox#retry

  2. investigating Nov 12, 2025, 03:28 PM UTC

    We are continuing to investigate this issue.

  3. identified Nov 12, 2025, 04:24 PM UTC

    We’ve identified that these issues are being caused by inconsistent network routing between Europe and the United States on upstream providers. Semaphore’s systems remain healthy, but affected routes can cause slower file transfer and occasional connection timeouts.

  4. monitoring Nov 12, 2025, 05:34 PM UTC

    The connection seems stable and we keep monitoring it.

  5. resolved Nov 12, 2025, 06:10 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor November 6, 2025 Network congestion on route to AWS ECR us-east region 14h 24m Semaphore

Started Nov 06, 2025, 04:51 PM UTC· resolved Nov 07, 2025, 07:15 AM UTC

  1. investigating Nov 06, 2025, 04:51 PM UTC

    Operations towards the AWS US-East region might present some slowness. We are currently investigating this issue.

  2. investigating Nov 06, 2025, 05:22 PM UTC

    We are continuing to investigate this issue.

  3. investigating Nov 06, 2025, 05:50 PM UTC

    We are continuing to investigate this issue.

  4. investigating Nov 06, 2025, 06:33 PM UTC

    Some operations in the AWS US-East region may be experiencing slow performance. Our team is still investigating.

  5. investigating Nov 06, 2025, 07:05 PM UTC

    We are continuing to investigate this issue.

  6. investigating Nov 06, 2025, 07:58 PM UTC

    We are continuing to investigate this issue.

  7. investigating Nov 06, 2025, 08:49 PM UTC

    We have observed slight improvements in the operations towards the AWS US-East region. We are continuing to work on this issue.

  8. investigating Nov 06, 2025, 09:02 PM UTC

    Connections to us-west, ap-northeast, and GCP are performing normally. If your workflows are affected, pushing/pulling data to and from these regions should help mitigate the issue. We are continuing to monitor the AWS US-East region slowness.

  9. investigating Nov 06, 2025, 09:40 PM UTC

    Our team is still monitoring the AWS US-East region slowness. Connections to us-west, ap-northeast, and GCP are performing normally. If your workflows are affected, pushing/pulling data to and from these regions should help mitigate the issue.

  10. investigating Nov 06, 2025, 10:31 PM UTC

    Performance in AWS US-East region is showing gradual recovery. Our team continues to monitor the situation. Connections to us-west, ap-northeast, and GCP are performing normally. If your workflows are affected, pushing/pulling data to and from these regions should help mitigate the issue.

  11. investigating Nov 06, 2025, 11:02 PM UTC

    We are continuing to monitor the AWS US-East region slowness. Connections to us-west, ap-northeast, and GCP are performing normally. If your workflows are affected, pushing/pulling data to and from these regions should help mitigate the issue.

  12. investigating Nov 06, 2025, 11:30 PM UTC

    We are continuing to follow up the AWS US-East region slowness issue. Connections to us-west, ap-northeast, and GCP are performing normally. If your workflows are affected, pushing/pulling data to and from these regions should help mitigate the issue.

  13. investigating Nov 07, 2025, 12:03 AM UTC

    Our team is following up the slight recovery in AWS US-East region. Connections to us-west, ap-northeast, and GCP are performing normally. If your workflows are affected, pushing/pulling data to and from these regions should help mitigate the issue.

  14. investigating Nov 07, 2025, 12:36 AM UTC

    Our team is still monitoring the AWS US-East region performance. Connections to us-west, ap-northeast, and GCP are performing normally. If your workflows are affected, pushing/pulling data to and from these regions should help mitigate the issue.

  15. investigating Nov 07, 2025, 01:07 AM UTC

    Increased network congestion detected in the AWS US-East region. Connections to us-west, ap-northeast, and GCP are performing normally. If your workflows are affected, pushing/pulling data to and from these regions should help mitigate the issue.

  16. investigating Nov 07, 2025, 01:48 AM UTC

    We are continuing to monitor the AWS US-East region slowness. Connections to us-west, ap-northeast, and GCP are performing normally. If your workflows are affected, pushing/pulling data to and from these regions should help mitigate the issue.

  17. investigating Nov 07, 2025, 02:29 AM UTC

    We are continuing to monitor the AWS US-East network congestion. Connections to us-west, ap-northeast, and GCP are performing normally. If your workflows are affected, pushing/pulling data to and from these regions should help mitigate the issue.

  18. resolved Nov 07, 2025, 07:15 AM UTC

    We have observed connection times recover and stay healthy for the affected regions.

Read the full incident report →

Minor October 29, 2025 Slow job processing 4h 49m Semaphore

Started Oct 29, 2025, 11:38 AM UTC· resolved Oct 29, 2025, 04:28 PM UTC

  1. identified Oct 29, 2025, 11:38 AM UTC

    The issue has been identified and a fix is being implemented.

  2. identified Oct 29, 2025, 12:24 PM UTC

    We are continuing to work on a fix for this issue.

  3. identified Oct 29, 2025, 12:53 PM UTC

    We are continuing to work on a fix for this issue.

  4. monitoring Oct 29, 2025, 01:30 PM UTC

    A fix has been implemented and we are monitoring the results.

  5. monitoring Oct 29, 2025, 02:22 PM UTC

    We are continuing to monitor for any further issues.

  6. monitoring Oct 29, 2025, 03:22 PM UTC

    We are continuing to monitor for any further issues.

  7. monitoring Oct 29, 2025, 03:58 PM UTC

    We are continuing to monitor for any further issues.

  8. resolved Oct 29, 2025, 04:28 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor October 21, 2025 Delays in job completion on Semaphore 1h Semaphore

Started Oct 21, 2025, 08:58 AM UTC· resolved Oct 21, 2025, 09:58 AM UTC

  1. investigating Oct 21, 2025, 08:58 AM UTC

    We are currently investigating this issue.

  2. identified Oct 21, 2025, 09:35 AM UTC

    The issue has been identified, and we are working on a fix.

  3. monitoring Oct 21, 2025, 09:44 AM UTC

    A fix has been implemented and we are monitoring the results.

  4. resolved Oct 21, 2025, 09:58 AM UTC

    This incident has been resolved.

Read the full incident report →

Major October 20, 2025 Docker Hub outage affecting image push/pulls in Semaphore 16h 15m Semaphore

Started Oct 20, 2025, 08:44 AM UTC· resolved Oct 21, 2025, 01:00 AM UTC

  1. identified Oct 20, 2025, 08:44 AM UTC

    Docker Hub is experiencing an outage, which is affecting image push/pulls in CI. We are also monitoring broader AWS service issues, which could indirectly affect some users' workloads.

  2. identified Oct 20, 2025, 10:29 AM UTC

    Image push/pulls are still being affected by the Docker Hub outage. We are closely monitoring the situation. We are continuing to monitor broader AWS service issues, which could still indirectly affect some users' workloads.

  3. identified Oct 20, 2025, 12:10 PM UTC

    We are continuing to monitor the situation: - Image push/pulls are still being affected by the Docker Hub outage. - Broader AWS service issues could still indirectly affect some users' workloads.

  4. identified Oct 20, 2025, 02:04 PM UTC

    We are continuing to monitor the situation: - Image push/pulls are still being affected by the Docker Hub outage. - Broader AWS service issues could still indirectly affect some users' workloads.

  5. monitoring Oct 20, 2025, 02:40 PM UTC

    Issues with image push/pulls seem to be resolved. We are still closely monitoring the situation.

  6. identified Oct 20, 2025, 04:01 PM UTC

    Image push/pulls seem to be affected again by the ongoing Docker Hub outage. We continue to monitor the situation.

  7. identified Oct 20, 2025, 06:01 PM UTC

    Image push/pulls seem to be affected again by the ongoing Docker Hub outage. We continue to monitor the situation.

  8. identified Oct 20, 2025, 07:44 PM UTC

    Image push/pulls seem to be affected by the ongoing Docker Hub outage. We keep monitoring the situation.

  9. identified Oct 20, 2025, 09:38 PM UTC

    Some image push/pulls seem to be affected by the ongoing Docker Hub outage. Our team is monitoring the situation.

  10. identified Oct 20, 2025, 11:52 PM UTC

    Image push/pulls may be affected by the ongoing Docker Hub outage. We continue to monitor the situation.

  11. monitoring Oct 21, 2025, 12:44 AM UTC

    Issues with image push/pulls seem to be resolved. We are still closely monitoring the situation.

  12. resolved Oct 21, 2025, 01:00 AM UTC

    This incident has been resolved.

Read the full incident report →

Minor October 2, 2025 Slow job processing on e2 machine types 3h 38m Semaphore

Started Oct 02, 2025, 06:48 PM UTC· resolved Oct 02, 2025, 10:26 PM UTC

  1. investigating Oct 02, 2025, 06:48 PM UTC

    We are currently investigating this issue.

  2. investigating Oct 02, 2025, 07:27 PM UTC

    We are continuing investigating this issue.

  3. identified Oct 02, 2025, 08:00 PM UTC

    The issue has been identified and a fix is being implemented.

  4. identified Oct 02, 2025, 08:46 PM UTC

    We are continuing to work on a fix for this issue.

  5. identified Oct 02, 2025, 09:18 PM UTC

    We're finalizing a fix for this issue.

  6. identified Oct 02, 2025, 09:52 PM UTC

    We are continuing to work on a fix for this issue.

  7. monitoring Oct 02, 2025, 09:56 PM UTC

    A fix has been implemented and we are monitoring the results.

  8. resolved Oct 02, 2025, 10:26 PM UTC

    This incident has been resolved.

Read the full incident report →