Omatic incident

Omatic Cloud long-running external API calls - 45601

Minor Resolved View vendor source →

Omatic experienced a minor incident on November 14, 2023 affecting Omatic Cloud US and Omatic Cloud UK and 1 more component, lasting 19h 16m. The incident has been resolved; the full update timeline is below.

Started
Nov 14, 2023, 08:29 PM UTC
Resolved
Nov 15, 2023, 03:46 PM UTC
Duration
19h 16m
Detected by Pingoru
Nov 14, 2023, 08:29 PM UTC

Affected components

Omatic Cloud USOmatic Cloud UKOmatic Cloud AUSOmatic Cloud CA

Update timeline

  1. identified Nov 14, 2023, 08:29 PM UTC

    Omatic engineers have identified an issue with Long running API calls affecting the Omatic Cloud product resulting in some related jobs failing to complete. The first occurrence of the issue appears to be Nov 11, 2023 and could impact ingestion of data as well as sending to destination. The team is seeking a workaround or resolution to the issue. A limited number of formula jobs ingesting data from large data sources could be affected. The issue has only been detected in the US Region. Updates are expected twice daily or on a change of status. For more detail, you can subscribe to updates or contact Omatic Support through normal methods.

  2. identified Nov 14, 2023, 09:57 PM UTC

    Omatic engineers have identified an issue with Long running API calls affecting the Omatic Cloud product resulting in some related jobs failing to complete. The first occurrence of the issue appears to be Nov 10, 2023 and could impact ingestion of data as well as sending to destination. The team is creating and testing a HotFix for the issue. A limited number of formula jobs ingesting data from large data sources could be affected. Users may see a Red failure banner on certain jobs while this condition continues. After the hotfix is in place, impacted jobs can be retried and will continue as normal. Updates are expected twice daily or on a change of status. For more detail, you can subscribe to updates or contact Omatic Support through normal methods.

  3. resolved Nov 15, 2023, 03:46 PM UTC

    At approximately 9:50 AM Eastern US Time on Wednesday, November 15, 2023 The Omatic team applied and successfully verified a HotFix that resolved the issue with Long running API calls affecting the Omatic Cloud product resulting in some related jobs failing to complete. Users affected by this issue may see a Red error message when viewing the status of recurring jobs in the web interface. Users should take advantage of the "Dismiss/Retry" option on that error so that their jobs will reconnect and run normally. For more detail, or if you are unsure if you are impacted, please contact Omatic Support through normal methods.