Nanonets incident

Failures in Instant learning model on app.nanonets.com

Major Resolved View vendor source →

Nanonets experienced a major incident on February 4, 2025 affecting API, lasting 5h 48m. The incident has been resolved; the full update timeline is below.

Started
Feb 04, 2025, 04:48 PM UTC
Resolved
Feb 04, 2025, 10:37 PM UTC
Duration
5h 48m
Detected by Pingoru
Feb 04, 2025, 04:48 PM UTC

Affected components

API

Update timeline

  1. investigating Feb 04, 2025, 04:48 PM UTC

    We are currently investigating the issue

  2. investigating Feb 04, 2025, 04:57 PM UTC

    We are continuing to investigate this issue.

  3. investigating Feb 04, 2025, 04:58 PM UTC

    We are continuing to investigate this issue.

  4. identified Feb 04, 2025, 05:02 PM UTC

    The issue has been identified and a fix is being implemented.

  5. monitoring Feb 04, 2025, 05:06 PM UTC

    A fix has been implemented and we are monitoring the results.

  6. resolved Feb 04, 2025, 10:37 PM UTC

    This incident has been resolved.

  7. postmortem Feb 05, 2025, 07:15 AM UTC

    ### **Incident Summary:** Between 16:30 UTC and 22:30 UTC 4th Feb, users experienced high response times and some failures when using Instant Learning models on [app.nanonets.com](https://app.nanonets.com). This was caused by unusually high latency in our secondary database when processing certain requests. Our other models on [app.nanonets.com](http://app.nanonets.com) are working fine and our other regions \(EU, EU-Open, IN\) are operational without any issues. ### **Resolution:** Our engineering team identified the issue and implemented a fix to restore normal response times. We are planning to schedule a maintenance upgrade for our secondary database to prevent similar incidents in the future. We apologize for the inconvenience and appreciate your patience.