Chartnote Outage History
Chartnote is up right nowPingoru has recorded 9 incidents for Chartnote: 6 in the last 12 months and 2 in the last 90 days.
Incidents, newest first
Minor September 29, 2026 Delayed or stuck note generation 49m AI Scribe & Voice Chart
Started Sep 29, 2026, 03:48 PM UTC· resolved Sep 29, 2026, 04:38 PM UTC
- identified Sep 29, 2026, 03:48 PM UTC
We're seeing delays and failures in AI note generation. Some notes may stay in a processing state or fail to complete. This is caused by an outage at one of our third-party AI providers, which began around 7:20 AM PT (14:20 UTC). Service is recovering, and many notes are now completing normally. If a note appears stuck, please try generating it again. We're monitoring closely and will post updates here as things improve.
- monitoring Sep 29, 2026, 04:22 PM UTC
A fix has been implemented, and we're monitoring the results.
- resolved Sep 29, 2026, 04:38 PM UTC
This incident has been resolved.
Major August 2, 2026 Service Disruption on Mobile App 1h 1m Mobile App (iOS), Mobile App (android)
Started Aug 02, 2026, 12:54 PM UTC· resolved Aug 02, 2026, 01:55 PM UTC
- investigating Aug 02, 2026, 12:54 PM UTC
We are currently investigating the issue.
- identified Aug 02, 2026, 01:05 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Aug 02, 2026, 01:53 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Aug 02, 2026, 01:55 PM UTC
The incident has been resolved.
Major May 28, 2026 Elevated errors on Chartnote 1h 40m AI Scribe & Voice Chart
Started May 28, 2026, 06:45 PM UTC· resolved May 28, 2026, 08:25 PM UTC
- investigating May 28, 2026, 06:45 PM UTC
We are currently investigating the issue
- identified May 28, 2026, 07:03 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring May 28, 2026, 07:18 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved May 28, 2026, 08:25 PM UTC
This incident has been resolved.
Minor February 25, 2026 Slow processing of AI notes 1h 57m AI Scribe & Voice Chart
Started Feb 25, 2026, 04:35 PM UTC· resolved Feb 25, 2026, 06:33 PM UTC
- identified Feb 25, 2026, 04:35 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Feb 25, 2026, 04:37 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Feb 25, 2026, 06:33 PM UTC
This incident has been resolved.
Major January 20, 2026 MFA SMS Delivery Interruption 17h 21m Website, Mobile App (iOS), Mobile App (android), Chrome Extension
Started Jan 20, 2026, 06:55 PM UTC· resolved Jan 21, 2026, 12:16 PM UTC
- identified Jan 20, 2026, 06:55 PM UTC
We identified an issue with SMS delivery for MFA and have temporarily disabled SMS-based MFA while we work to restore service.
- identified Jan 21, 2026, 09:50 AM UTC
We are continuing to work on a fix for this issue.
- resolved Jan 21, 2026, 12:16 PM UTC
SMS-based MFA has been restored and is operating normally.
Critical October 29, 2025 Users unable to log in. 29m Website, Mobile App (iOS), Mobile App (android), Chrome Extension
Started Oct 29, 2025, 04:11 PM UTC· resolved Oct 29, 2025, 04:40 PM UTC
- investigating Oct 29, 2025, 04:11 PM UTC
We are currently investigating this issue.
- identified Oct 29, 2025, 04:18 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Oct 29, 2025, 04:31 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Oct 29, 2025, 04:40 PM UTC
This incident has been resolved.
- postmortem Oct 29, 2025, 08:00 PM UTC
Database Storage Capacity Incident Incident Overview: * **Date/Time:** 2025-10-29, 16:11-16:40 UTC * **Duration:** 29 minutes * **Impact:** Users unable to log in across all platforms \(Website, Mobile Apps for iOS and Android, Chrome Extension\) * **Suspected Cause:** Database disk storage reached capacity limit * **Resolution Method:** Increased database storage capacity limits Incident Details: At approximately 16:11 UTC, multiple user reports indicated authentication failures across all platforms. Initial testing confirmed the issue, with staff members also unable to log in to the system. Investigation revealed that the MariaDB database had reached its 100GB storage capacity limit, causing it to enter a special Amazon state for full disks. The database was originally provisioned with a 100GB limit when initially created, which had been sufficient until today. Technical analysis suggests that database indexes, which enable fast data retrieval, likely occupy a significant portion of the storage space rather than the actual data itself. Response Actions: * **16:11 UTC:** Initial reports received about users unable to log in; investigation began * **16:18 UTC:** Issue identified as database storage capacity reached its limit * **16:18-16:31 UTC:** Remediation implemented by doubling the database storage limits * **16:31 UTC:** Fix implemented and monitoring began * **16:40 UTC:** Incident resolved; all systems confirmed operational The response team quickly identified the root cause and implemented an immediate solution by increasing the database storage capacity, which promptly restored system availability. Retrospective & Recommendations: **Findings:** * The MariaDB database reached its 100GB storage capacity limit without prior warning * No proactive monitoring was in place to alert the team before reaching critical storage thresholds * The timing was fortunate as it occurred during business hours rather than during a critical deployment period **Challenges:** * Initial concern that the issue might be related to ongoing development work caused momentary panic * Current alerting system focuses on activity metrics and traffic spikes rather than infrastructure capacity * Existing alerts are not well-tuned, resulting in notification fatigue and unclear prioritization **Recommendations:** * Configure disk space threshold alerts at 80% capacity for all database instances * Review and increase storage limits for other database instances to prevent similar incidents * Refine the current alerting system to reduce noise and prioritize critical notifications * Implement a more comprehensive monitoring strategy for infrastructure resources **Follow-Up:** * Team to implement disk space monitoring alerts as a priority task * Review storage requirements for all database instances * Evaluate and improve the current alerting system to ensure important notifications receive appropriate attention Additional Notes: This incident was resolved quickly due to prompt identification of the root cause. The timing was fortunate as it occurred before a planned deployment rather than during or after it, which could have complicated diagnosis.
Minor May 28, 2025 Speech-to-text dictation unavailable 6h 56m Website, Mobile App (iOS), Mobile App (android), Chrome Extension
Started May 28, 2025, 09:36 PM UTC· resolved May 29, 2025, 04:33 AM UTC
- investigating May 28, 2025, 09:36 PM UTC
We are currently investigating this issue.
- identified May 29, 2025, 12:18 AM UTC
The issue has been identified and a fix is being implemented.
- monitoring May 29, 2025, 02:48 AM UTC
A fix has been implemented and we are monitoring the results.
- resolved May 29, 2025, 04:33 AM UTC
This incident has been resolved.
Major January 10, 2025 Elevated errors on Chartnote AI 2d 18h AI Scribe & Voice Chart
Started Jan 10, 2025, 11:00 PM UTC· resolved Jan 13, 2025, 05:08 PM UTC
- investigating Jan 11, 2025, 12:25 PM UTC
We are currently investigating the issue.
- investigating Jan 11, 2025, 12:26 PM UTC
We are continuing to investigate this issue.
- identified Jan 11, 2025, 12:27 PM UTC
The issue has been identified and an issues is being implemented.
- monitoring Jan 11, 2025, 12:41 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jan 13, 2025, 05:08 PM UTC
This incident has been resolved.
Critical December 12, 2024 Chartnote Copilot Facing Issues 12h 50m Copilot
Started Dec 12, 2024, 12:41 AM UTC· resolved Dec 12, 2024, 01:32 PM UTC
- investigating Dec 12, 2024, 12:41 AM UTC
We are currently investigating the issue and will provide more updates shortly.
- resolved Dec 12, 2024, 01:32 PM UTC
Between 3:16pm PST and 7:38pm PST on December 11, Copilot services were unavailable. We began to see recovery starting at approximately 5:40pm PST. We mitigated the issues at 7:38 pm PST, and all services are now fully operational.