Crossbeam incident

Issues with HubSpot data syncs

Major Resolved View vendor source →

Crossbeam experienced a major incident on April 28, 2022 affecting Background Processing, lasting 1d 1h. The incident has been resolved; the full update timeline is below.

Started
Apr 28, 2022, 03:53 PM UTC
Resolved
Apr 29, 2022, 05:41 PM UTC
Duration
1d 1h
Detected by Pingoru
Apr 28, 2022, 03:53 PM UTC

Affected components

Background Processing

Update timeline

  1. identified Apr 28, 2022, 03:53 PM UTC

    Our team has identified an issue with our incoming HubSpot data syncs. If you connected your HubSpot CRM into Crossbeam, you are most likely affected. We've identified a root cause and are working towards a solution. We will be providing updates as we progress. Apologies for the inconvenience!

  2. identified Apr 28, 2022, 05:12 PM UTC

    Our team has identified an issue with rate limits against the HubSpot API and are working towards a solution. In the interim, we have paused HubSpot data syncs for users that were affected. We will provide an update once we have a solution in place.

  3. monitoring Apr 28, 2022, 07:12 PM UTC

    Our team has uncovered some unclear rate limiting that started occurring with the HubSpot API this morning around 9:24am ET. We have paused our HubSpot data syncs through tomorrow to allow it to reset. We will be monitoring our data syncs tomorrow as we bring them back online. Thank you for your patience.

  4. monitoring Apr 28, 2022, 07:33 PM UTC

    We are continuing to monitor for any further issues.

  5. monitoring Apr 29, 2022, 01:20 PM UTC

    HubSpot informed us that they were having issues with their Cloudflare servers inadvertently rate limiting users, and Crossbeam was affected. It should be resolved today, and we will be re-enabling HubSpot data syncs and closely monitoring them. Thank you for your patience!

  6. resolved Apr 29, 2022, 05:41 PM UTC

    All HubSpot data syncs have been re-enabled and are operating normally.