Delayed file processing for all ILM models on app.nanonets.com

Update timeline

1. investigating Mar 10, 2026, 02:43 PM UTC

   We are currently investigating this issue.

2. identified Mar 10, 2026, 03:19 PM UTC

   The issue has been identified and a fix is being implemented.

3. monitoring Mar 10, 2026, 03:34 PM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved Mar 10, 2026, 03:58 PM UTC

   This incident has been resolved.

5. postmortem Mar 10, 2026, 04:50 PM UTC

   On March 10 between 1:10 PM and 3:30 PM UTC, we experienced a service degradation that impacted Instant Learning Models \(ILM\) on [app.nanonets.com](http://app.nanonets.com/). **Impact**: During this period, our vector database experienced unusually high load due to the query planner selecting a suboptimal execution path for certain queries. This led to elevated database response times and eventually triggered a database restart. As a result, overall system throughput temporarily decreased. Customers may have observed the following effects during this window: * Asynchronous uploads for ILM models experiencing delayed file processing. * Synchronous uploads for ILM models encountering intermittent failures. Only **Instant Learning Models on** [**app.nanonets.com**](http://app.nanonets.com) were affected. Other services and model types remained fully operational. **Resolution:** Our engineering team identified the root cause related to query planning behaviour in the vector database and implemented targeted customizations to ensure the optimal execution path is used. This stabilized database performance and restored normal processing capacity. **Next Steps:** System is stable now and we are actively working on additional database optimizations and long-term improvements to further strengthen system stability and prevent similar issues from recurring. These enhancements will be rolled out in the coming days. We sincerely apologize for the inconvenience caused and appreciate your patience while we worked to resolve the issue. Please feel free to reach out to our support team incase if you have any questions.