Codeberg Outage History
Codeberg is up right nowPingoru has recorded 66 incidents for Codeberg: 9 in the last 12 months and 0 in the last 90 days.
Show as a table
| Month | Incidents | Hours with an open incident |
|---|---|---|
| April 2026 | 0 | 0 of 720 (0%) |
| May 2026 | 1 | 0 of 744 (0%) |
| June 2026 | 3 | 179 of 720 (25%) |
| July 2026 | 1 | 22 of 744 (3%) |
| August 2026 | 0 | 329 of 744 (44%) |
| September 2026 | 0 | 0 of 720 (0%) |
| October 2026 | 0 | 0 of 744 (0%) |
| November 2026 | 0 | 0 of 47 (0%) |
Incidents, newest first
Major July 31, 2026 Performance degradation for unauthenticated traffic 61d 17h
Started Jul 31, 2026, 04:27 PM UTC· resolved Oct 01, 2026, 10:04 AM UTC
No vendor updates were posted for this incident.
Minor June 28, 2026 Recovered after power outage 3d 8h
Started Jun 28, 2026, 05:36 AM UTC· resolved Jul 01, 2026, 01:55 PM UTC
No vendor updates were posted for this incident.
Minor June 27, 2026 Power Outage 6h 13m
Started Jun 27, 2026, 11:20 PM UTC· resolved Jun 28, 2026, 05:34 AM UTC
No vendor updates were posted for this incident.
Minor June 23, 2026 High false positive rate 4d 10h
Started Jun 23, 2026, 12:52 PM UTC· resolved Jun 27, 2026, 11:06 PM UTC
No vendor updates were posted for this incident.
Notice May 5, 2026 We're no longer using incidents 3s
Started May 05, 2026, 09:23 PM UTC· resolved May 05, 2026, 09:23 PM UTC
- investigating May 05, 2026, 09:23 PM UTC
Please find recent status messages on our Mastodon account (https://social.anoxinon.de/@codebergstatus).
Minor November 17, 2025 Hardware update –
Started Nov 17, 2025, 10:00 PM UTC· resolved Nov 17, 2025, 10:00 PM UTC
- investigating Nov 17, 2025, 10:00 PM UTC
We are rebooting and upgrading BIOS and Firmware on the server that hosts mosts services. This will take at most 10 minutes. We have rebooted this server. We are working on getting pages back online.
Notice November 12, 2025 Hardware maintenance –
Started Nov 12, 2025, 06:43 PM UTC· resolved Nov 12, 2025, 06:43 PM UTC
- investigating Nov 12, 2025, 06:43 PM UTC
Impact: * Several short interruptions of Forgejo ("Codeberg.org") service in the next 24 hours * As a consequence, interruptions or transient failures in connected services (e.g. hosted CI/CD, Weblate) * Potential performance degradation Status: * The planned changes have been conducted and the affected servers are up and running again. We're still monitoring the systems in case of follow-up issues. * The hardware maintenance has continued at 17.55 UTC. Server replacement is ongoing. * The first round of failover was successfully completed. Disruption was longer than expected due to human error. Background: We are replacing 1 of 3 physical servers powering Codeberg operations, upgrading computing capacity and replacing a machine with a broken mainboard. To complete the operation, several services need to be moved across servers, leading to periods of unavailability. Also, computing capacity is temporarily reduced due to the hardware constraints or blocked by backfilling operations (data replication from MariaDB/Galera database and Ceph storage clusters). We aim to keep the impact as minimal as possible. Unfortunately, our current deployment of Forgejo only supports exactly one primary instance, and each failover / reconfiguration currently involves a downtime of about 1 minute.
Notice November 11, 2025 Codeberg.org update –
Started Nov 11, 2025, 06:28 AM UTC· resolved Nov 11, 2025, 06:28 AM UTC
- investigating Nov 11, 2025, 06:28 AM UTC
Codeberg.org will be down for a few minutes to deploy spam fighting features.
Minor November 4, 2025 Pages maintenance –
Started Nov 04, 2025, 08:40 PM UTC· resolved Nov 04, 2025, 08:40 PM UTC
- investigating Nov 04, 2025, 08:40 PM UTC
We are performing scheduled maintenance on Codeberg Pages. While we didn't expect any impact on the live operation, we are currently investigating "502 Bad Gateway" responses on some sites.
Notice October 27, 2025 Codeberg server upgrade –
Started Oct 27, 2025, 07:15 PM UTC· resolved Oct 27, 2025, 07:15 PM UTC
- investigating Oct 27, 2025, 07:15 PM UTC
21:26 UTC: We have upgraded our last server to Debian Trixie. We are bringing up all services up and monitoring the situation. 20:44 UTC: We are upgrading our last server to Debian Trixie, the upgrade with the other server went very well and had minimal downtime. This servers hosts codeberg.org and will be unavailable during the upgrade. 19:59 UTC: The server upgrade is done we are bringing all services up and monitoring the situation. We are expecting to upgrade one of our servers to Debian Trixie this evening (CEST). This server hosts Codeberg CI (Woodpecker), Codeberg Translate and Codeberg pages will therefore be unavailable during the upgrade. It is possible that other services are impacted, our filesystem (Ceph) will be put into maintenance mode during the upgrade.
Minor October 25, 2025 Fixed: Database performance degradation –
Started Oct 25, 2025, 02:30 PM UTC· resolved Oct 25, 2025, 02:30 PM UTC
- investigating Oct 25, 2025, 02:30 PM UTC
Update: The cause for the performance degradation was identified to be unprecendented AI crawling on our servers. We have had to do more aggressive proof-of-work challenging for various expensive URLs. A novelty is that AI crawlers not only access links that are actually generated by our web frontend, but they seem to convert the URLs into a format that bypassed existing filter rules. https://codeberg.org/Codeberg-Infrastructure/scripted-configuration/commit/e4aaceeb434a82d2b8c610284006d3d0a3cbe665 We are currently investigating high traffic on Codeberg that has degraded the performance of our database cluster. We are still investigating. Some attempts to remediate the problem did not yield the required results.
Minor October 21, 2025 Forgejo is upgrading –
Started Oct 21, 2025, 04:23 PM UTC· resolved Oct 21, 2025, 04:23 PM UTC
- investigating Oct 21, 2025, 04:23 PM UTC
We're scheduling a Forgejo upgrade. It will result in an expected downtime of a few minutes. You can join the upgrade party via https://lecture.senfcall.de/ott-zml-1vs-qcc if you want to watch the progress.
Notice October 14, 2025 Maintenance: E-Mail server migration –
Started Oct 14, 2025, 04:18 PM UTC· resolved Oct 14, 2025, 04:18 PM UTC
- investigating Oct 14, 2025, 04:18 PM UTC
We are currently performing scheduled maintenance on the email setup. Please expect * delay in email notification * temporary shutdowns of services during reconfiguration * unexpected issues due to human mistakes
Notice October 1, 2025 Hardware failure on Kampenwand –
Started Oct 01, 2025, 01:13 PM UTC· resolved Oct 01, 2025, 01:13 PM UTC
- investigating Oct 01, 2025, 01:13 PM UTC
The following services are temporarily degraded: * Codeberg Pages * Hosted Woodpecker CI * Blog, Documentation and Design assets The reason is a failing SSD on our host Kampenwand.
Minor September 23, 2025 Reduced availability of codeberg.org 44m
Started Sep 23, 2025, 02:49 AM UTC· resolved Sep 23, 2025, 03:33 AM UTC
- investigating Sep 23, 2025, 02:49 AM UTC
We are currently investigating the reduced availability of codeberg.org. This means it might be down for several minutes or have very poor performance. 03:00 UTC: We made adjustments to bring the availability back to manageable levels, we are monitoring the situation. 03:30 UTC: We are seeing large influx of targeted traffic that is is impacting the availability of codeberg.org. We are disabling certain routes temporarily.
Notice September 11, 2025 Hardware maintenance 1h 37m
Started Sep 11, 2025, 04:06 PM UTC· resolved Sep 11, 2025, 05:43 PM UTC
- investigating Sep 11, 2025, 04:06 PM UTC
We are performing scheduled hardware maintenance in the datacenter, starting 17.00 UTC / 19.00 CEST. There is no expected disruption of any services. We are expanding SSD capacity for our Ceph storage cluster. 19:30 CEST: We've successfully expanded the SSD capacity of our Ceph storage cluster, no downtime or performance degradation have been noticed so far and we will continue to monitor the situation.
Major September 9, 2025 Codeberg.org down –
Started Sep 09, 2025, 02:17 AM UTC· resolved Sep 09, 2025, 02:17 AM UTC
- investigating Sep 09, 2025, 02:17 AM UTC
Codeberg.org will be down until the early morning (CEST).
Notice September 4, 2025 Deploying update –
Started Sep 04, 2025, 01:03 AM UTC· resolved Sep 04, 2025, 01:03 AM UTC
- investigating Sep 04, 2025, 01:03 AM UTC
We are deploying a small update to codeberg.org
Minor September 3, 2025 Performance degradation –
Started Sep 03, 2025, 12:57 AM UTC· resolved Sep 03, 2025, 12:57 AM UTC
- investigating Sep 03, 2025, 12:57 AM UTC
We are investigating degraded performance on our Ceph filesystem.
Notice August 14, 2025 Updating –
Started Aug 14, 2025, 11:54 PM UTC· resolved Aug 14, 2025, 11:54 PM UTC
- investigating Aug 14, 2025, 11:54 PM UTC
We're deploying a small update to codeberg.org, should take a few minutes.
Notice July 30, 2025 Maintenance on Forgejo –
Started Jul 30, 2025, 02:28 PM UTC· resolved Jul 30, 2025, 02:28 PM UTC
- investigating Jul 30, 2025, 02:28 PM UTC
Today (30 July 2025), we are performing scheduled maintenance on our Forgejo instance at 20.00 CEST / 18.00 UTC. We expect a short disruption of services, but please be prepared for a potentially longer downtime in case of issues.
Minor July 22, 2025 Upgrade to Forgejo v12 18h 57m
Started Jul 22, 2025, 09:28 PM UTC· resolved Jul 23, 2025, 04:26 PM UTC
- investigating Jul 22, 2025, 09:28 PM UTC
Following the upgrade to Forgejo v12, we have noticed several problems. | Problem | Status | Reason | | --- | --- | --- | | Registration not possible | ✅ Fixed | A small mistake in porting our custom patches to the new Forgejo version | | Issue search doesn't work properly | ✅ Fixed | | | OAuth login to some third-party apps (like translate.codeberg.org) is not working | Workarounds have been applied. | Please follow [forgejo/forgejo#8634](https://codeberg.org/forgejo/forgejo/issues/8634) for details. |
Notice July 3, 2025 Network outage 39m
Started Jul 03, 2025, 09:36 AM UTC· resolved Jul 03, 2025, 10:16 AM UTC
- investigating Jul 03, 2025, 09:36 AM UTC
Our systems are currently not reachable, including for us. Since the website of our rackspace provider is also unreachable, we assume it is a problem on their end. We'll keep you updated as soon as we know more. Update: We investigated the problem and figured out that all services from the datacenter are offline, so it seems like a larger outage. However, the good news is that it then relies on paid staff and not volunteers to fix the issue. We hope it will be resolved soon. Update: Codeberg is back online. It looks like there was a power outage at the datacenter, but we have no confirmed information about the background yet.
Minor May 17, 2025 Hardware failure 2d 3h
Started May 17, 2025, 09:47 AM UTC· resolved May 19, 2025, 01:41 PM UTC
- investigating May 17, 2025, 09:47 AM UTC
Status 2025-05-19: We are currently working on the affected machine, resulting in downtime for Codeberg Pages, Hosted Woodpecker CI, Weblate/Codeberg Translate and meta pages (blog, docs, ...). This might take a while. One of our machines is operating at degraded performance due to hardware failure (an SSD disappeared from the RAID). We have moved Forgejo to another 'mountain' (we name our servers after mountains). Some secondary services remain on the affected machine and might operate with degraded performance (Codeberg Pages, Hosted Woodpecker CI, Weblate/Codeberg Translate). We have had issues with an SSD in that slot some months ago, where it was suddenly gone, but reappeared after a server reboot. Some weeks ago, the issue occured again, but the SSD was entirely dead. We replaced it earlier this week, but only after a few days of operation, the replacement disappeared again. We suspect a hardware issue with the nvme slot, which unfortunately also seems to entirely kill the drives that are inserted into it.
Notice May 12, 2025 Scheduled downtime 2025-05-13 07:10-07:40 UTC –
Started May 12, 2025, 12:59 PM UTC· resolved May 12, 2025, 12:59 PM UTC
- investigating May 12, 2025, 12:59 PM UTC
We will replace a failed NVMe SSD which is not hot-pluggable which should not take more than 30 minutes.