Semaphore had 28 outages in the last 2 years totaling 84h 22m of downtime — averaging 1.2 incidents per month.
There were 28 Semaphore outages since February 24, 2025 totaling 84h 22m of downtime. Each is summarised below — incident details, duration, and resolution information.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
We are continuing to investigate this issue.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we have observed an improvement in job processing. We are continuing to monitor the situation.
This incident has been resolved.
We are currently investigating this issue.
As a workaround to get more consistent builds during this time, please try adding the --use-cache flag to your checkout command. Read more about it here: https://docs.semaphore.io/reference/toolbox#cache-full-clone
Our engineering team is still investigating and working on this issue, but as a current workaround, you can regenerate your GitHub deployment key. You can find the instructions on how to do that here: https://docs.semaphore.io/using-semaphore/connect-github#deploy-key
Our team is monitoring the situation. If you are affected, please regenerate your deploy key by following the instructions on this page: https://docs.semaphore.io/using-semaphore/connect-github#deploy-key
We are continuing to monitor the situation, and you can follow the current GitHub incident at the following link: https://www.githubstatus.com/incidents/g40zcbvchny4
GitHub confirmed that the incident on their end has been resolved, and services should now be returning to normal. We are continuing to monitor the situation. You can review the GitHub update here: https://www.githubstatus.com/incidents/g40zcbvchny4
GitHub confirmed that the incident on their end has been resolved. You can review the GitHub update here: https://www.githubstatus.com/incidents/g40zcbvchny4
We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We're experiencing an elevated level of 500's errors and are currently looking into the issue.
The issue has been identified, and our team is implementing a fix.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We identified slow communication between Semaphore jobs and GitHub. We are continuing to work with our upstream provider to resolve this. For impacted jobs, please replace the normal checkout command with the following: retry --times 5 "timeout 40 bash -lc 'source ~/.toolbox/toolbox; checkout'" And afterwards, use cd to change the directory to the cloned repository (replace with your project).
A fix has been implemented and we are monitoring the results.
We identified slow GitHub operations related to problems with the network, and we deployed a fix to re-route those operations to a healthier server. Some HTTPS operations started to fail because of that. To disable the fix, you should use the command: sudo sed -i 's/^\([0-9].*github\.com\)/#\1/' /etc/hosts And to re-enable, you can use: sudo sed -i 's/^#\([0-9].*github\.com\)/\1/' /etc/hosts
A fix has been implemented, and everything should be back to normal. The previously shared snippets can be removed.
This incident has been resolved.
We are currently investigating this issue.
We are continuing to investigate this issue.
We are continuing to investigate job start failures and processing delays. We will continue to update while investigating this.
The issue has been identified and we are working on a fix. We will provide our next update within 30 minutes or sooner if things change. We thank you for your patience while we continue to work to resolve this issue.
A fix has been implemented and we are monitoring the results. We will provide our next update within 30 minutes or sooner if things change.
This incident has been resolved.
Webhook delivery delays from GitHub led to a backlog of workflows being triggered all at once as delivery resumes. This caused a temporary spike in demand for some Semaphore machines. We are actively working to reduce the impact and restore normal capacity. We continue monitoring both GitHub’s incident and Semaphore infrastructure closely, and we will provide further updates as conditions stabilize.
We’re seeing considerable improvement in internal performance metrics, and we will continue monitoring until conditions fully stabilize.
This incident has been resolved.
We are currently investigating this issue.
Webhook delivery delays between GitHub and Semaphore appear to be caused by degraded performance on GitHub’s side. GitHub is currently reporting an active incident that appears related. We are monitoring the situation closely and will provide updates as GitHub resolves the issue.
GitHub has posted an update indicating recovery of the aforementioned issue. We continue monitoring Semaphore closely as things stabilize. (GitHub Status: https://www.githubstatus.com/incidents/f314nlctbfs5)
This incident has been resolved.
The issue has been identified and a fix is being implemented.
We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating the issue. Implementing the retry command should help mitigate the issue: https://docs.semaphore.io/reference/toolbox#retry
We are continuing to investigate this issue.
We’ve identified that these issues are being caused by inconsistent network routing between Europe and the United States on upstream providers. Semaphore’s systems remain healthy, but affected routes can cause slower file transfer and occasional connection timeouts.
The connection seems stable and we keep monitoring it.
This incident has been resolved.
The issue has been identified and a fix is being implemented.
We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
A fix has been implemented and we are monitoring the results.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
This incident has been resolved.
We are currently investigating this issue.
The issue has been identified, and we are working on a fix.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
We are continuing investigating this issue.
The issue has been identified and a fix is being implemented.
We are continuing to work on a fix for this issue.
We're finalizing a fix for this issue.
We are continuing to work on a fix for this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
The issue has been identified and a fix is being implemented.
We are continuing to work on a fix for this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
We are continuing to investigate this issue.
The issue has been identified and a fix is being implemented.
We are continuing to work on a fix for this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
The issue has been identified and a fix is being implemented.
We are continuing to work on a fix for this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
Semaphore is now operating normally. We’re currently monitoring the situation to ensure continued stability.
This incident has been resolved.
Some users may see incorrect account or billing messages in the UI. CI pipelines might be affected. We're investigating and will provide updates shortly.
A fix has been implemented and we are monitoring the results. Some users may still see outdated messages due to caching, but functionality is unaffected.
This incident has been resolved.
We are currently investigating this issue.
The issue has been identified and a fix is being implemented.
We are continuing to work on a fix for this issue.
Job delays on the Ubuntu Noble (24.04) image.
A fix was deployed for e2 machines. We are continuing work on f1 machines.
We have deployed a fix for f1 machines, and we are monitoring the results.
This incident has been resolved.