Flightcontrol Outage History
Flightcontrol is up right nowPingoru has recorded 195 incidents for Flightcontrol: 195 in the last 12 months and 75 in the last 90 days.
Show as a table
| Month | Incidents | Hours with an open incident |
|---|---|---|
| April 2026 | 13 | 79 of 720 (11%) |
| May 2026 | 29 | 114 of 744 (15%) |
| June 2026 | 22 | 100 of 720 (14%) |
| July 2026 | 18 | 34 of 744 (5%) |
| August 2026 | 33 | 34 of 744 (5%) |
| September 2026 | 26 | 25 of 720 (3%) |
| October 2026 | 13 | 30 of 572 (5%) |
Incidents, newest first
Major September 2, 2026 API background job queue failure –
Started Sep 02, 2026, 07:44 AM UTC· resolved Sep 02, 2026, 07:44 AM UTC
- resolved Sep 02, 2026, 08:10 AM UTC
This incident has been resolved.
- monitoring Sep 02, 2026, 08:10 AM UTC
A fix has been implemented and we are monitoring the results.
- investigating Sep 02, 2026, 08:10 AM UTC
We are investigating an issue with the background job runner for our API. Actions that require a background job, such as creating apps, assigning IP addresses, or creating/renewing certificates, may fail at this time.
Notice August 31, 2026 HTTP/2 traffic disruptions –
Started Aug 31, 2026, 03:30 PM UTC· resolved Aug 31, 2026, 03:30 PM UTC
- resolved Aug 31, 2026, 04:11 PM UTC
A configuration update caused temporary failures for incoming HTTP/2 traffic for Fly Machines located on a subset of hosts for a few minutes. This incident has since been resolved. Managed Postgres depends on HTTP/2 and some control plane ops may have been affected as well. However, downstream Postgres connections were unlikely to have been affected by this since they do not use the HTTP2 handler.
Minor August 31, 2026 Packet loss in ORD 59m
Started Aug 31, 2026, 07:10 AM UTC· resolved Aug 31, 2026, 08:10 AM UTC
- investigating Aug 31, 2026, 07:10 AM UTC
Due to an upstream provider, we are seeing ~50% packet loss on a subset of hosts in ORD. Some MPG clusters in ORD are slow to replicate as a result.
- monitoring Aug 31, 2026, 08:09 AM UTC
Packet loss in ORD is improving and impacted services are recovering; we’re continuing to monitor for intermittent issues
- resolved Aug 31, 2026, 09:10 AM UTC
This incident has been resolved.
Notice August 30, 2026 Sprite deletion jobs failing –
Started Aug 30, 2026, 10:45 PM UTC· resolved Aug 30, 2026, 10:45 PM UTC
- resolved Aug 30, 2026, 10:45 PM UTC
We saw Sprite deletion jobs failing between 21:18 and 22:05 UTC. This issue has been resolved.
Minor August 28, 2026 Networking Issues in GRU 26m
Started Aug 28, 2026, 10:10 PM UTC· resolved Aug 28, 2026, 10:36 PM UTC
- investigating Aug 28, 2026, 10:10 PM UTC
We are investigating networking issues impacting some hosts in GRU (São Paulo, Brazil) region. Some apps in GRU may experience increased latency or packet loss.
- monitoring Aug 28, 2026, 10:10 PM UTC
Networking performance in GRU has normalized and we are no longer seeing issues. We are continuing to monitor to ensure a full recovery.
- identified Aug 28, 2026, 11:11 PM UTC
Our upstream provider has implemented a fix. Network performance in GRU has normalized.
- identified Aug 28, 2026, 11:11 PM UTC
We are seeing a recurrance in networking issues in GRU. Some apps in the region may experience increased latency or packet loss. We are working with our upstream networking provider to resolve.
- resolved Aug 28, 2026, 11:11 PM UTC
This incident has been resolved.
Minor August 28, 2026 Increased packet loss 1h 13m
Started Aug 28, 2026, 09:09 AM UTC· resolved Aug 28, 2026, 10:22 AM UTC
- investigating Aug 28, 2026, 09:09 AM UTC
We are currently investigating this issue.
- resolved Aug 28, 2026, 11:12 AM UTC
This incident has been resolved.
Notice August 26, 2026 WireGuard gateway issues –
Started Aug 26, 2026, 06:43 PM UTC· resolved Aug 26, 2026, 06:43 PM UTC
- investigating Aug 26, 2026, 07:09 PM UTC
We are investigating issues with our WireGuard gateways. Some CLI commands like `flyctl ssh console` or `flyctl proxy` may not work at this time. Apps continue to run.
- resolved Aug 26, 2026, 07:09 PM UTC
This incident has been resolved.
- monitoring Aug 26, 2026, 07:09 PM UTC
Our testing and monitoring indicates gateways should be back to normal; if you are still having problem using `flyctl ssh console`, try restarting the `flyctl` agent by `flyctl agent restart`.
- monitoring Aug 26, 2026, 07:09 PM UTC
A fix has been implemented and we are monitoring the results.
Minor August 24, 2026 Metrics in some regions are lagging behind 2h 15m
Started Aug 24, 2026, 11:09 AM UTC· resolved Aug 24, 2026, 01:24 PM UTC
- investigating Aug 24, 2026, 11:09 AM UTC
We are currently experiencing some metrics lag on servers in some regions. We are provisioning more metric processing instances to accommodate the backlog and catch up.
- monitoring Aug 24, 2026, 01:10 PM UTC
All hosts have caught up with metrics and we're monitoring the situation
- resolved Aug 24, 2026, 02:10 PM UTC
This is now resolved
Major August 23, 2026 Network Issues in LAX Region 1m
Started Aug 23, 2026, 02:09 AM UTC· resolved Aug 23, 2026, 02:10 AM UTC
- investigating Aug 23, 2026, 02:09 AM UTC
We are investigating network issues in the Los Angeles region. Apps may experience higher latency or be unreachable at this time.
- monitoring Aug 23, 2026, 02:09 AM UTC
Upstream networking issues have resolved.
- resolved Aug 23, 2026, 03:06 AM UTC
This incident has been resolved.
Notice August 20, 2026 Temporary DNS resolution failure –
Started Aug 20, 2026, 07:30 PM UTC· resolved Aug 20, 2026, 07:30 PM UTC
- resolved Aug 20, 2026, 08:15 PM UTC
A BGP configuration error caused our Anycast DNS to route to some nodes without the proper DNS infrastructure. The issue was temporary and was resolved as soon as we removed that node from BGP.
Notice August 20, 2026 Oauth/Macaroon Errors from flyctl 8m
Started Aug 20, 2026, 02:08 PM UTC· resolved Aug 20, 2026, 02:16 PM UTC
- monitoring Aug 20, 2026, 02:08 PM UTC
A fix has been deployed and this error should no longer be occurring. We're monitoring to ensure full recovery.
- identified Aug 20, 2026, 02:08 PM UTC
We have identified an issue causing authentication errors for some operations from `flyctl`. These operations are failing with an error like: `This endpoint no longer accepts legacy OAuth tokens (starting with `fo1_`). Please use a macaroon token (starting with `fm2_`) instead. We have identified the issue and are rolling out a fix
- resolved Aug 20, 2026, 03:13 PM UTC
This incident has been resolved.
Major August 20, 2026 MPG (v1) partially down in ORD –
Started Aug 20, 2026, 07:57 AM UTC· resolved Aug 20, 2026, 07:57 AM UTC
- investigating Aug 20, 2026, 08:11 AM UTC
We had an issue with the ord-0 Fly Kubernetes cluster, and many MPG clusters are failing to restart. Our MPG team is actively working on it.
- investigating Aug 20, 2026, 08:11 AM UTC
We are continuing to investigate this issue.
- monitoring Aug 20, 2026, 08:11 AM UTC
A fix has been implemented and we are monitoring the results.
- resolved Aug 20, 2026, 08:11 AM UTC
This incident has been resolved.
Notice August 19, 2026 6PN Networking issue in YYZ –
Started Aug 19, 2026, 12:00 AM UTC· resolved Aug 19, 2026, 12:00 AM UTC
- resolved Aug 19, 2026, 02:10 AM UTC
6PN networking issues between some machines in YYZ during a rollout which was rolled back once we noticed errors. During this time some machines were unable to talk to internal resources like other DBs, other apps or MPG clusters.
Notice August 17, 2026 No capacity in ARN 2h 21m Deployments
Started Aug 17, 2026, 01:19 PM UTC· resolved Aug 17, 2026, 03:41 PM UTC
- investigating Aug 17, 2026, 01:19 PM UTC
New machines may fail to create in ARN because we lack capacity.
- resolved Aug 17, 2026, 03:41 PM UTC
The capacity issue in the ARN region has been resolved.
Major August 14, 2026 Secrets service outage 1h 3m Machines API
Started Aug 14, 2026, 07:30 PM UTC· resolved Aug 14, 2026, 08:33 PM UTC
- identified Aug 14, 2026, 07:30 PM UTC
We are working to recover our secrets service after a failed deployment. Apps continue to run, but it is not possible to create new apps or update secrets at this time.
- monitoring Aug 14, 2026, 08:08 PM UTC
We have failed over the secrets database to a replica, and the Machines API appears healthy now. We are monitoring for any further issues.
- resolved Aug 14, 2026, 08:33 PM UTC
This incident has been resolved.
Minor August 13, 2026 IPv6 Networking Issues 1h 56m Customer Applications
Started Aug 13, 2026, 05:45 PM UTC· resolved Aug 13, 2026, 07:41 PM UTC
- investigating Aug 13, 2026, 05:45 PM UTC
We are currently investigating degraded ipv6 networking on a subset of hosts
- investigating Aug 13, 2026, 05:51 PM UTC
We are continuing to investigate this issue.
- identified Aug 13, 2026, 06:14 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Aug 13, 2026, 06:36 PM UTC
A fix has been implemented and we are monitoring the results.
- monitoring Aug 13, 2026, 06:37 PM UTC
We are continuing to monitor for any further issues.
- resolved Aug 13, 2026, 07:41 PM UTC
This incident has been resolved.
Major August 9, 2026 Increased app-not-found errors 4h 20m Machines API
Started Aug 09, 2026, 02:50 AM UTC· resolved Aug 09, 2026, 07:10 AM UTC
- investigating Aug 09, 2026, 02:50 AM UTC
We are currently investigating app-not-found errors returned by Machines API calls made shortly after creating new applications.
- identified Aug 09, 2026, 03:50 AM UTC
We have identified the issue as failed insertions in a subset of Corrosion batches. These failures trigger retries, which can cause timeouts for other batches.
- identified Aug 09, 2026, 04:58 AM UTC
We’ve applied a mitigation to reduce the impact from Corrosion batch insertion retries and are continuing to monitor while affected nodes catch up.
- identified Aug 09, 2026, 06:03 AM UTC
We’ve deployed an additional mitigation to further reduce Corrosion retry pressure and are seeing improvement; we’re continuing to monitor while remaining affected nodes catch up.
- monitoring Aug 09, 2026, 06:42 AM UTC
A fix has been implemented and we are monitoring the results
- resolved Aug 09, 2026, 07:10 AM UTC
This incident has been resolved.
Minor August 5, 2026 MPG IAD data plane degraded for new clusters 15m Management Plane - IAD
Started Aug 05, 2026, 08:26 PM UTC· resolved Aug 05, 2026, 08:41 PM UTC
- identified Aug 05, 2026, 08:26 PM UTC
High CPU pressure on a shared etcd instance is causing lags on MPG creation in the IAD region
- resolved Aug 05, 2026, 08:41 PM UTC
Etcd is stable. Services are back to normal.
Major August 4, 2026 MPG creation is failing in GRU due to lack of capacity 8h 19m Management Plane - GRU
Started Aug 04, 2026, 07:03 PM UTC· resolved Aug 05, 2026, 03:22 AM UTC
- identified Aug 04, 2026, 07:03 PM UTC
New MPG clusters may fail to create in GRU because we lack capacity.
- monitoring Aug 04, 2026, 08:59 PM UTC
We tweaked hosts to allow for more machine allocation. We'll be monitoring the region over the next hours.
- resolved Aug 05, 2026, 03:22 AM UTC
This incident has been resolved.
Minor August 4, 2026 Certificate issuance delays 9h 4m SSL/TLS Certificate Provisioning
Started Aug 04, 2026, 02:53 PM UTC· resolved Aug 04, 2026, 11:58 PM UTC
- investigating Aug 04, 2026, 02:53 PM UTC
We're currently investigating issues related to certificate issuance. Certificates may be delayed for new custom domains.
- identified Aug 04, 2026, 05:02 PM UTC
We believe we have identified the issue and are releasing a fix.
- identified Aug 04, 2026, 06:26 PM UTC
We have the size of TLS certificate issuance backlog under control, but are still seeing some remaining issues and are currently working to clean up the edge cases.
- monitoring Aug 04, 2026, 08:36 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Aug 04, 2026, 11:58 PM UTC
This incident has been resolved.
Critical August 3, 2026 Inbound connection failure to Fly Apps –
Started Aug 03, 2026, 06:50 PM UTC· resolved Aug 03, 2026, 06:50 PM UTC
- resolved Aug 03, 2026, 03:16 PM UTC
A BGP misconfiguration while provisioning new edge capacity caused most traffic from Europe endpoints to be dropped, between 14:50 UTC and 15:02 UTC. The misconfiguration has been fixed and we are implementing safeguards against this kind of issue in the future.
Minor August 1, 2026 Managed Postgres v2 control plane issues in iad 1h 28m Management Plane - IAD
Started Aug 01, 2026, 05:01 PM UTC· resolved Aug 01, 2026, 06:30 PM UTC
- investigating Aug 01, 2026, 05:01 PM UTC
We are investigating an issue with the control plane for Managed Postgres v2 in the IAD region. Creating new v2 clusters in the IAD region may fail at this time. Existing clusters continue to run, but may experience lagging backups.
- monitoring Aug 01, 2026, 05:26 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Aug 01, 2026, 06:30 PM UTC
This incident has been resolved.
Notice July 31, 2026 Capacity issues in CDG 3h 27m CDG - Paris, France
Started Jul 31, 2026, 04:44 PM UTC· resolved Jul 31, 2026, 08:11 PM UTC
- investigating Jul 31, 2026, 04:44 PM UTC
Creating machines in CDG region may fail at this time with an "no capacity available in cdg" message. Existing apps continue to run.
- monitoring Jul 31, 2026, 07:02 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jul 31, 2026, 08:11 PM UTC
This incident has been resolved.
Minor July 31, 2026 Outbound email issues 1h 16m Dashboard
Started Jul 31, 2026, 01:34 PM UTC· resolved Jul 31, 2026, 02:51 PM UTC
- investigating Jul 31, 2026, 01:34 PM UTC
Our dashboard is failing to send outbound email. Emails such as new account verification or password reset may fail to send at this time.
- monitoring Jul 31, 2026, 02:38 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jul 31, 2026, 02:51 PM UTC
This incident has been resolved.
Minor July 29, 2026 Increased API latency 44m Dashboard, Machines API, Deployments
Started Jul 29, 2026, 12:41 PM UTC· resolved Jul 29, 2026, 01:26 PM UTC
- investigating Jul 29, 2026, 12:41 PM UTC
We are investigating some database issues causing high latency on some API endpoints and dashboard operations. You may experience intermittent "503 service unavailable" errors at this time. Currently deployed apps continue to run.
- resolved Jul 29, 2026, 01:26 PM UTC
This incident has been resolved.