Notabene experienced a major incident on January 16, 2025, lasting 55s. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Jan 16, 2025, 09:56 PM UTC
For exactly 33 mins between 20:26 UTC and 20:59 UTC a misonfigured load balancer rate limit caused API requests to be rejected resulting in customers being unable to access the API, dashboard and authentication to fail. The load balancer distributing traffic to our API has a rate limiter that was configured incorrectly causing requests to be rejected. The rate limit was a legacy configuration that applied rate limits across an incorrect source IP header causing requests to be rejected globally.