Aptible Outage History

Started Oct 22, 2026, 12:24 AM UTC· resolved Oct 22, 2026, 12:24 AM UTC

1. resolved Oct 22, 2026, 12:24 AM UTC

   We are aware of a security incident involving compromised eMudhra Certificate Authority (CA) certificates that could affect HTTPS security on systems with outdated certificate trust stores. The Aptible platform is not directly impacted by this issue. However, as a precaution, we recommend that customers: - Update operating systems and web browsers to their latest security-patched versions to ensure certificate trust stores are current. - Upgrade to Aptible CLI v1.0.5, which includes additional mitigations for systems whose OS trust stores have not yet been updated: https://www.aptible.com/docs/reference/aptible-cli/overview For additional background, see Cloudflare's security advisory: https://blog.cloudflare.com/emudhra-certificate-compromise

Read the full incident report →

Started Jul 27, 2026, 10:31 AM UTC· resolved Jul 27, 2026, 03:12 PM UTC

1. investigating Jul 27, 2026, 10:31 AM UTC

   AWS is experiencing an issue with their APIs in us-east-1, and many services are impacted at this time. This may lead to failed operations. To avoid further issues, we have disabled operations in this region while we understand the full impact.

2. investigating Jul 27, 2026, 12:55 PM UTC

   Operations remain disabled in us-east-1 while we continue to investigate AWS networking issues in the region. We are testing mitigations and will restore operations as soon as it is safe to do so. We will provide another update no later than 10:00 AM Eastern Time.

3. monitoring Jul 27, 2026, 01:43 PM UTC

   We have applied a mitigation and re-enabled operations in us-east-1. We are monitoring for stability and will resolve this incident once we confirm conditions remain stable.

4. resolved Jul 27, 2026, 03:12 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jul 16, 2026, 10:31 PM UTC· resolved Jul 17, 2026, 05:04 PM UTC

1. identified Jul 16, 2026, 10:31 PM UTC

   We are aware of delayed and timed-out requests to the LLM Gateway. We have made some changes to help minimize the impact while engineers continue to address the root cause.

2. monitoring Jul 17, 2026, 12:02 AM UTC

   We have increased resource capacity, and we are monitoring the results.

3. monitoring Jul 17, 2026, 12:14 PM UTC

   After a regression in performance starting at 9:57 UTC (5:57 EDT) and lasting until 11:12 UTC (7:12 EDT), we've deployed a potential fix and updated our service scaling. We are continuing to monitor whether these changes fully resolve the issue before closing this incident.

4. resolved Jul 17, 2026, 05:04 PM UTC

   Following the fixes deployed per the previous update, we have seen stable performance on the LLM Gateway since 7:12 EDT and are resolving the issue. The root cause of this issue was a severe performance degradation in 3 specific endpoints: • `GET /models` • `GET /v1/models` • `GET /metadata/models` ...all of which return a list of available models. The performance issue on these endpoints was severe enough to cause significant request queueing and degraded performance for other unaffected endpoints. The fix we deployed added caching on this endpoint to ensure fast responses. Since those fixes have been deployed, we are no longer seeing request queueing or impacts to other LLM Gateway API endpoints.

Read the full incident report →

Started Jun 08, 2026, 06:28 PM UTC· resolved Jun 23, 2026, 09:48 PM UTC

1. investigating Jun 08, 2026, 06:28 PM UTC

   As part of our security policy and ongoing commitment to protecting customer workloads, we are applying patches for a newly disclosed Systemd vulnerability (CVE-2026-40226) across our platform. This patching will be rolled out automatically over the next several days. During this process, you may experience a momentary network disruption to running applications and databases. Customers configured for high-availability with horizontally-scaled services should experience little to no disruption from this update. No action is required on your part. If you have any questions or concerns, please reach out to our Support team at https://contact.aptible.com/.

2. resolved Jun 23, 2026, 09:48 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jun 03, 2026, 04:02 PM UTC· resolved Jun 03, 2026, 07:16 PM UTC

1. investigating Jun 03, 2026, 04:02 PM UTC

   There is some partial service degradation with log drains configured to send to Mezmo where not all log messages are being sent. Mezmo is rejecting logs over a certain size. We are actively investigating this issue with the upstream provider.

2. resolved Jun 03, 2026, 07:16 PM UTC

   As of 18:42 UTC the upstream provider began to accept log messages again consistently and this issue is now remediated.

Read the full incident report →

Started May 08, 2026, 05:02 AM UTC· resolved May 08, 2026, 03:35 PM UTC

1. monitoring May 08, 2026, 05:02 AM UTC

   Aptible is aware of “Dirty Frag,” a recently disclosed Linux kernel vulnerability that may allow local privilege escalation under certain conditions [0]. We have taken action across the Aptible platform to protect all customer workloads against this vulnerability. No customer action is required at this time. We will continue to monitor upstream kernel, distribution, and security vendor guidance related to this vulnerability and will take any additional action if needed. [0] https://www.openwall.com/lists/oss-security/2026/05/07/8

2. resolved May 08, 2026, 03:35 PM UTC

   This incident has been resolved.

Read the full incident report →

Started May 08, 2026, 01:05 AM UTC· resolved May 09, 2026, 03:35 AM UTC

1. investigating May 08, 2026, 01:05 AM UTC

   Due to an underlying issue in AWS we are experiencing instability in a certain availability zone in us-east-1. We are investigating this issue and will follow up with recommendations shortly

2. identified May 08, 2026, 01:39 AM UTC

   We have blocked operations from provisioning into the specified region

3. monitoring May 08, 2026, 02:02 PM UTC

   We have unblocked operations from the specified region as we have seen successful provisioning and maintenance of new instance in that availability zone. Instances that were affected have not yet seen recovery, and the upstream AWS alert indicates they may still take hours of recovery.

4. resolved May 09, 2026, 03:35 AM UTC

   An upstream AWS issue affected infrastructure in a single Availability Zone in us-east-1. During the event, we blocked provisioning into the affected zone and monitored AWS’s recovery efforts. AWS has since recovered enough of the affected availability zone for the impacted Aptible infrastructure to recover. We have also completed recovery of any Aptible-managed resources that required follow-up action on our side. Any customers with directly affected resources have been notified of resolution. We are continuing normal platform monitoring, but we are marking this incident as resolved on our end.

Read the full incident report →

Started Apr 29, 2026, 10:40 PM UTC· resolved Apr 29, 2026, 10:40 PM UTC

1. resolved Apr 29, 2026, 10:40 PM UTC

   Aptible is aware of CVE-2026-31431, also known as “Copy Fail,” a recently disclosed Linux kernel vulnerability that may allow local privilege escalation under certain conditions [0]. We have taken action across the Aptible platform to protect all customer workloads against this vulnerability. No customer action is required at this time. We will continue to monitor upstream kernel, distribution, and security vendor guidance related to this CVE and will take any additional action if needed. [0] https://copy.fail/

Read the full incident report →

Started Apr 19, 2026, 02:03 AM UTC· resolved Apr 19, 2026, 03:10 AM UTC

1. investigating Apr 19, 2026, 02:03 AM UTC

   We are investigating an incident impacting api.aptible.com. Viewing current resources and running operations are currently impacted, however running applications and databases are unaffected

2. monitoring Apr 19, 2026, 02:54 AM UTC

   A fix has been implemented and operations that were failing to enqueue during the availability outage have been retried. We will continue monitoring the situation.

3. resolved Apr 19, 2026, 03:10 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Mar 17, 2026, 10:33 PM UTC· resolved Mar 17, 2026, 11:33 PM UTC

1. investigating Mar 17, 2026, 10:33 PM UTC

   We are currently observing intermittent DNS resolution failures affecting infrastructure in us-east-1. These resolver errors may cause some Aptible Deploy operations (such as deploys, restarts, SSH, or container provisioning) to fail or time out. Applications running on Aptible Deploy that rely on network hostname resolution may also experience intermittent connection failures related to DNS resolution. We are actively working with our infrastructure providers to investigate the issue and will provide updates here as they are available.

2. monitoring Mar 17, 2026, 11:02 PM UTC

   We have observed DNS resolution error rates return to normal levels, and all dependent activities should be functioning normally. We will continue to monitor for an additional period to ensure the issue is fully resolved before closing the incident. Further updates will be posted here if we observe any recurrence.

3. resolved Mar 17, 2026, 11:33 PM UTC

   This incident has been resolved.

4. postmortem Mar 18, 2026, 04:42 PM UTC

   Additional detail about timeline: we observed a relative increase in DNS resolver errors in us-east-1 beginning at approximately 2026-03-17 18:30 UTC. The elevated error rate was intermittent rather than constant, with the most visible clusters occurring from roughly 18:30-18:55 UTC and again from about 20:55-21:40 UTC. Error rates returned to normal shortly after 21:40 UTC.

Read the full incident report →

Started Dec 31, 2025, 10:48 AM UTC· resolved Dec 31, 2025, 02:49 PM UTC

1. investigating Dec 31, 2025, 10:48 AM UTC

   We are aware of degraded performance affecting auth.aptible.com. This affects all actions that interact with our APIs including login, viewing and managing resources, etc.

2. monitoring Dec 31, 2025, 11:33 AM UTC

   Performance has improved since our mitigations. We are still monitoring the situation for continued degradation.

3. resolved Dec 31, 2025, 02:49 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Dec 30, 2025, 02:14 AM UTC· resolved Dec 30, 2025, 05:15 PM UTC

1. monitoring Dec 30, 2025, 02:14 AM UTC

   The Aptible Security Team is aware of the recently disclosed vulnerability CVE-2025-14847 (https://github.com/advisories/GHSA-4742-mr57-2r9j) affected all MongoDB versions. The vulnerability could allow an attacker with network access to a MongoDB database to exfiltrate data, including sensitive data and/or credentials, without authenticating to the database. In response to the vulnerability, we have updated our supported MongoDB versions to prevent exploitations of this vulnerability. In addition, we have proactively restarted databases matching either of the following criteria, to ensure they are running on the latest protected versions: * All databases in shared-tenancy stacks, and * All databases with customer-created public endpoints that do not have access restricted to an IP allow list Since Aptible databases run on private networks by default, most Aptible managed MongoDB databases are not accessible from the internet or by other Aptible customers, and so are not vulnerable to CVE-2025-14847. As such, we did not proactively restart these databases. Customers may restart their databases at any time to update to the latest protected versions.

2. resolved Dec 30, 2025, 05:15 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Dec 05, 2025, 02:39 PM UTC· resolved Dec 08, 2025, 04:52 PM UTC

1. investigating Dec 05, 2025, 02:39 PM UTC

   We are aware of the recently disclosed critical vulnerability CVE-2025-55182 (https://react.dev/blog/2025/12/03/critical-security-vulnerability-in-react-server-components) affecting React Server Components, which could allow remote code execution under certain conditions. This vulnerability affects certain React 19.x and frameworks that implement React Server Components, including Next.js. *Aptible Platform Impact:* Aptible has reviewed all infrastructure components that we manage and has confirmed that all are unaffected by this vulnerability. *Customer Application Impact:* If you are running applications on Aptible that use React Server Components, you may be affected. We have seen active exploitation of this vulnerability, and we recommend upgrading to the patched versions immediately. Specifically, applications using: - React 19.0.0, 19.1.0, 19.1.1, or 19.2.0 with react-server-dom-webpack, react-server-dom-turbopack, or react-server-dom-parcel - Next.js versions 14.3.0-canary.77 and all subsequent 14.3.x canary releases, 15.0.0, 15.0.1, 15.0.2, 15.0.3, 15.0.4, 15.1.0, 15.1.1, 15.1.2, 15.1.3, 15.1.4, 15.1.5, 15.1.6, 15.1.7, 15.1.8, 15.2.0, 15.2.1, 15.2.2, 15.2.3, 15.2.4, 15.2.5, 15.3.0, 15.3.1, 15.3.2, 15.3.3, 15.3.4, 15.3.5, 15.4.0, 15.4.1, 15.4.2, 15.4.3, 15.4.4, 15.4.5, 15.4.6, 15.4.7, 15.5.0, 15.5.1, 15.5.2, 15.5.3, 15.5.4, 15.5.5, 15.5.6, 15.6.0-canary.0 through 15.6.0-canary.57, 16.0.0, 16.0.1, 16.0.2, 16.0.3, 16.0.4, 16.0.5, 16.0.6 - Other frameworks implementing React Server Components including Vite, Parcel, React Router, RedwoodSDK, Waku *Additional Resources:* React Security Advisory: https://github.com/facebook/react/security/advisories/GHSA-fv66-9v8q-g76r Next.js Security Advisory: https://github.com/vercel/next.js/security/advisories/GHSA-9qr9-h5gf-34mp We will continue to update this incident page as more information becomes available.

2. resolved Dec 08, 2025, 04:52 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Nov 05, 2025, 03:59 PM UTC· resolved Nov 05, 2025, 04:07 PM UTC

1. identified Nov 05, 2025, 03:59 PM UTC

   We are investigating an issue causing degraded performance for the Aptible Platform API. As part of the mitigation, a temporary operation block across all regions is in place which prevents new operations from starting. This includes deployments, scaling actions, SSH sessions, and database tunnels. Running apps and databases are not affected. Next update We will provide an update by 11:30 ET or sooner.

2. monitoring Nov 05, 2025, 04:05 PM UTC

   A fix has been implemented and we are monitoring the results. The operation block has been lifted.

3. resolved Nov 05, 2025, 04:07 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 29, 2025, 09:17 PM UTC· resolved Oct 29, 2025, 09:50 PM UTC

1. investigating Oct 29, 2025, 09:17 PM UTC

   We are currently experiencing failures when provisioning new hosts across all regions. This impacts operations that require new capacity, including but not limited to: app deploys, scaling events, and new database creation. Existing running workloads are not affected. Our team is actively investigating and working to restore normal operation performance. We will provide an update by no later than 5:45pm EST.

2. monitoring Oct 29, 2025, 09:33 PM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Oct 29, 2025, 09:50 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 28, 2025, 05:09 PM UTC· resolved Oct 28, 2025, 09:08 PM UTC

1. investigating Oct 28, 2025, 05:09 PM UTC

   We are seeing significantly increased provisioning times in the AWS us-east-1 region. This is affecting operations that require new capacity, including deployments, scaling, and creation of new resources. These operations may time out and fail. Our team is actively investigating and working to mitigate the impact. We will provide our next update by 2 pm EST.

2. monitoring Oct 28, 2025, 05:32 PM UTC

   AWS has confirmed the issue with EC2 provisioning issues in us-east-1. Meanwhile, we have been able to implement a mitigation for affected customers. We will continue to leave this issue open and monitor its progress, and will provide any relevant updates as needed.

3. identified Oct 28, 2025, 05:53 PM UTC

   We are now seeing additional broader EC2 provisioning issues in the AWS us-east-1 region that are outside the scope of our mitigation. Additional delays/failures on operations that require new capacity in us-east-1 may still occur at this time. We will provide our next update by 2:30 pm EST.

4. monitoring Oct 28, 2025, 06:35 PM UTC

   We are not currently seeing provisioning issues in AWS us-east-1. We are modifying the incident status to Monitoring, and will provide any relevant updates as needed.

5. resolved Oct 28, 2025, 09:08 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 20, 2025, 07:47 AM UTC· resolved Oct 20, 2025, 09:46 AM UTC

1. identified Oct 20, 2025, 07:47 AM UTC

   AWS is experiencing an issue with their APIs in us-east-1, and many services are impacted at this time. This may lead to failed operations, but there appears to be no impact to availability for apps and databases running on Aptible at this time.

2. monitoring Oct 20, 2025, 08:31 AM UTC

   Since the AWS issues seem serious, and to avoid any operation failures causing resources to get stuck in a transitional state, we are taking action to block all new operations on the platform at this time. We have identified no impact to acceibilty or preformance of running apps and database containers at this time.

3. monitoring Oct 20, 2025, 09:30 AM UTC

   AWS has indicated a fix has been applied, and we are re-enabling operations. We will continue to monitor for additional impact, as both Dockerhub and Quay.io were offline due to the AWS issue.

4. resolved Oct 20, 2025, 09:46 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 12, 2025, 02:30 AM UTC· resolved Oct 12, 2025, 03:40 AM UTC

1. investigating Oct 12, 2025, 02:30 AM UTC

   One of our internal queuing systems observed a failure that has affected several operations enqueued in us-east-1. Our Reliability Team is restoring the queueing system and investigating the root cause of the failure.

2. monitoring Oct 12, 2025, 02:42 AM UTC

   A fix has been implemented and we are monitoring the results.

3. monitoring Oct 12, 2025, 03:40 AM UTC

   No operation queueing issues have been observed since the last update. If you observed any operations that failed or were stuck in a "pending" state, please retry the operations and they should run without errors.

4. resolved Oct 12, 2025, 03:40 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 10, 2025, 03:53 PM UTC· resolved Oct 10, 2025, 03:53 PM UTC

1. resolved Oct 10, 2025, 03:53 PM UTC

   Aptible has reviewed and addressed the Redis vulnerability described in https://redis.io/blog/security-advisory-cve-2025-49844/. We have confirmed that all internal Redis instances used in delivering the Aptible platform are secure and have no exploitation path related to this CVE. For Aptible customers who do not have their Redis databases publicly exposed, a path to exploitation is similarly unlikely. Additionally, for customers using Redis 6.2 databases on Aptible, a patched version will be available by Oct 10th at 5pm EST, and additional releases (7.2) are upcoming; see aptible.com/changelog for ongoing release updates. To ensure you are running the latest minor version when available, please run the cli command: aptible db:reload for each Redis database. For more information on using this command, visit https://www.aptible.com/docs/reference/aptible-cli/cli-commands/cli-db-reload. If you have any specific questions or concerns related to this CVE, please contact us.

Read the full incident report →

Started Aug 07, 2025, 01:43 PM UTC· resolved Aug 07, 2025, 02:21 PM UTC

1. investigating Aug 07, 2025, 01:43 PM UTC

   Our online documentation at aptible.com/docs is temporarily unavailable. We are working with our upstream provider to resolve the issue and will update this incident when it is resolved.

2. resolved Aug 07, 2025, 02:21 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jun 05, 2025, 07:34 PM UTC· resolved Jun 05, 2025, 08:32 PM UTC

1. investigating Jun 05, 2025, 07:34 PM UTC

   We are investigating an increased error rate in our API which may be causing failed operations

2. monitoring Jun 05, 2025, 08:03 PM UTC

   The errors have been resolved, however we are still monitoring

3. resolved Jun 05, 2025, 08:32 PM UTC

   This incident has been resolved.

Read the full incident report →

Started May 29, 2025, 10:04 PM UTC· resolved May 29, 2025, 10:24 PM UTC

1. monitoring May 29, 2025, 10:04 PM UTC

   We've noticed that some Operations are failing due to Route53 record changes not propagating within the 10 minute time limit allowed by our platform. Running App and Databases are not impacted, but creation or deletion of Databases or Endpoints, as well as scaling services to/from zero containers may be impacted. We'll continue to monitor the situation and provide updates as we have any additional information to shre.

2. resolved May 29, 2025, 10:24 PM UTC

   Route 53 record propagation appears to have returned to normal.

Read the full incident report →

Started Apr 04, 2025, 10:54 PM UTC· resolved Apr 04, 2025, 11:26 PM UTC

1. identified Apr 04, 2025, 10:54 PM UTC

   We are currently experiencing issues with operations being delayed for stacks hosted in eu-central-1. Our Engineering team is currently working to restore normal functionality.

2. resolved Apr 04, 2025, 11:26 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Mar 26, 2025, 04:53 PM UTC· resolved Mar 26, 2025, 05:10 PM UTC

1. investigating Mar 26, 2025, 04:53 PM UTC

   We are currently experiencing issues with operations being delayed. Our Engineering team is currently investigating.

2. monitoring Mar 26, 2025, 05:02 PM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Mar 26, 2025, 05:10 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Feb 20, 2025, 12:01 AM UTC· resolved Feb 20, 2025, 02:37 AM UTC

1. monitoring Feb 20, 2025, 12:01 AM UTC

   We are experiencing intermittent failures in App and Database operations due to issues with an upstream provider. This issue only affects Apps and Databases with endpoints. Retrying the operation may resolve the issue. We are actively monitoring the situation and will provide updates once the problem is fully resolved.

2. resolved Feb 20, 2025, 02:37 AM UTC

   This incident has been resolved.

Read the full incident report →