Keka Outage History

ATS recovered

Attendance recovered

Payroll recovered

keka API recovered

Keka API recovered

The Keka application is currently experiencing significant slowness, with pages taking an unusually long time to load. We have identified the issue related to Redis configuration changes.

The earlier database availability issue has been resolved. Azure reported a transient platform event that caused temporary unavailability between 06:03 AM and 06:11 AM IST. Our monitoring system alerted our team immediately, and services have been stable since recovery. The system is now fully operational.

All resources are restored

CIN03 environment is not loading due to partial slowness,the issue has been mitigated, and the CIN03 environment is now functioning normally.

Current Status: The issue has been identified. The Hire team recently updated the API URL for Global Admin and Hire Admin, which inadvertently caused the Report Builder functionality to fail across all regions. The frontend logic missed handling this change, leading to the disruption. Impact: Users across all regions were unable to generate reports in all regions, affecting critical reporting workflows and operational decision-making. All backend systems are working and operational.

The root cause was database connection pool exhaustion triggered by a sudden increase in concurrent requests to the Mobile and Web API services. The devops team scaled up the database connections to handle the increased load. System performance stabilized after the scale-up completed. Continuous monitoring was enabled to validate stability. Actions Taken: Short-term - Verified successful scale-up and confirmed service recovery. - Enabled real-time monitoring on database connection utilization. - Validated API stability post scale-up using Better Stack Long-term - Optimize database pooling configuration to better handle high concurrency. - Implement proactive alerts for connection pool exhaustion and sudden traffic spikes. - Fine-tune auto-scaling thresholds to trigger earlier under high load. - Review and upgrade service plan or database tier to improve responsiveness during scaling events.

The issue was resolved on Oct 8, 2025, at 9:56 AM. The Mobile API environment experienced intermittent timeouts and failures in displaying the Remote Clock-in card. Investigation revealed that the issue occurred during autoscaling, when new backend instances were being provisioned. Due to memory and storage limitations, these instances required a brief warm-up period, during which some requests were temporarily dropped for approximately two minutes.

We are currently investigating an issue where certain clients are unable to access the Keka application due to a Content Delivery Network (CDN) caching and propagation anomaly. This appears to be affecting tenant-specific assets or login endpoints, causing partial or complete service disruption for impacted users. Our engineering team has identified that some edge nodes may be serving stale or corrupted content due to improper cache invalidation or TTL (time-to-live) configuration. 🛠 Next Steps: We are performing a CDN cache purge and forcing revalidation of content across all PoPs (Points of Presence). DNS-level inspection and propagation checks are underway to ensure edge resolution consistency. We will continue to provide updates as we mitigate the issue. Thank you for your patience and understanding.