Cornerstone experienced a notice incident on November 5, 2025, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Nov 05, 2025, 08:43 PM UTC
Between 6:10 PM PT on November 3 and 2:24 AM PT on November 4, webhooks were unavailable due to a service disruption. Webhook functionality has been fully restored, and all systems are now operating normally.
- postmortem Dec 03, 2025, 07:46 PM UTC
**Incident Summary:** On November 3rd, Cornerstone’s internal monitoring systems detected a failure in the Webhooks service impacting customers hosted in the US region across the Production, Pilot, and Stage environments. During this period, Webhooks delivery was intermittently disrupted, resulting in delays or failures in outbound event notifications. **Root Cause:** The incident was triggered by auto-scaling activities performed on the underlying messaging infrastructure that supports Webhooks processing. These scaling operations were initiated to accommodate increased service usage. However, the activity introduced instability in the Webhooks message flow, which caused interruptions and ultimately led to the Webhooks service failure in the affected environments. **Corrective Action:** Cornerstone engineering teams took immediate remediation steps to restore normal Webhooks operations: * Service Restoration: Webhooks services were actively restored across the impacted Production, Pilot, and Stage environments. * Infrastructure Stabilization: Scaling adjustments were made to the Webhooks infrastructure to stabilize processing queues and resume consistent event delivery. * Post-Recovery Validation: Engineering teams validated message throughput and platform stability before confirming full restoration of service. **Preventive Measures:** To mitigate recurrence, Cornerstone is taking the following actions: * Introduce controlled deployment and impact assessment before scaling activities in shared systems. * Implement enhanced monitoring and alerting to detect early degradation in Webhooks dependencies. * Establish coordination and change-validation protocol between teams to evaluate downstream impact prior to infrastructure changes.