Proposify incident

AWS outage in N.East region is causing service disruptions

Major Resolved View vendor source →

Proposify experienced a major incident on December 7, 2021 affecting Proposal Previews and Proposal Editing and 1 more component, lasting 8h. The incident has been resolved; the full update timeline is below.

Started
Dec 07, 2021, 05:46 PM UTC
Resolved
Dec 08, 2021, 01:46 AM UTC
Duration
8h
Detected by Pingoru
Dec 07, 2021, 05:46 PM UTC

Affected components

Proposal PreviewsProposal EditingThird Party IntegrationsEmail System

Update timeline

  1. identified Dec 07, 2021, 05:46 PM UTC

    Amazon Web Services which hosts our application is suffering from a major outage in the N.East region. We are working to mitigate the outage as much as possible while they work to resume service https://status.aws.amazon.com/

  2. identified Dec 07, 2021, 05:47 PM UTC

    We are continuing to work on a fix for this issue.

  3. identified Dec 07, 2021, 08:02 PM UTC

    The root cause of this issue is an impairment of several network devices in the N.East Region. We are pursuing multiple mitigation paths in parallel, have seen some signs of recovery, and continue to work towards a full recovery.

  4. identified Dec 07, 2021, 10:20 PM UTC

    AWS has executed a mitigation which is showing significant recovery in the N.East Region. We are continuing to closely monitor the incident and they expect to continue to make progress towards full recovery.

  5. identified Dec 07, 2021, 11:25 PM UTC

    AWS has mitigated the underlying issue that caused some network devices in the N.East Region to be impaired. We are seeing improvement in availability across most AWS services. Many services have already recovered, however we are working towards full recovery across services. Engineers are actively working on resolving impact to these services.

  6. monitoring Dec 07, 2021, 11:26 PM UTC

    A fix has been implemented and we are monitoring the results.

  7. resolved Dec 08, 2021, 01:46 AM UTC

    This incident has been resolved.