Issues with metered billing report generation

Update timeline

1. investigating Mar 31, 2026, 01:47 PM UTC

   We are investigating reports of impacted performance for some GitHub services.

2. investigating Mar 31, 2026, 01:56 PM UTC

   We're seeing issues related to metered billing reports, intermittently affecting metered usage graphs and reports on the billing page. We have identified an issue with a data store, and are working on mitigations.

3. investigating Mar 31, 2026, 02:39 PM UTC

   We're continuing to see high failure rates on billing report generation, and are working on mitigations for a data store related to billing reports.

4. investigating Mar 31, 2026, 02:56 PM UTC

   We are seeing a high number of 500s due to timeouts across GitHub services. We are redeploying some of our core services and we expect that this allow us to recover.

5. investigating Mar 31, 2026, 02:59 PM UTC

   We have applied mitigations to a data store related to billing reports, and are seeing partial recovery to billing report generation. We continue to monitor for full recovery.

6. monitoring Mar 31, 2026, 03:01 PM UTC

   The degradation has been mitigated. We are monitoring to ensure stability.

7. resolved Mar 31, 2026, 03:10 PM UTC

   On March 31, 2026, between 06:15 UTC and 15:30 UTC, the GitHub billing usage reports feature was degraded due to reduced server capacity. Customers requesting billing usage reports and loading the top usage by organization and repository on the billing overview and usage pages were impacted. The average error rate for usage report requests was 15%, peaking at 98% over an eight-minute window. For the billing pages, an average of 56% of requests failed to load the top usage cards. The root cause was an increase in billing usage report requests with large datasets, which exhausted the capacity of the nodes responsible for reporting data. There was no impact on billing charges. We mitigated the incident by adjusting our auto-scaling thresholds to better meet our capacity needs. We are working to improve our metrics to reduce time to detection and mitigation for similar issues in the future.