Technolutions incident

Background worker optimization in progress

Notice Resolved View vendor source →

Technolutions experienced a notice incident on January 15, 2020 affecting Slate, lasting 11h 9m. The incident has been resolved; the full update timeline is below.

Started
Jan 15, 2020, 05:50 PM UTC
Resolved
Jan 16, 2020, 05:00 AM UTC
Duration
11h 9m
Detected by Pingoru
Jan 15, 2020, 05:50 PM UTC

Affected components

Slate

Update timeline

  1. monitoring Jan 15, 2020, 05:50 PM UTC

    We continue to optimize and adjust the quantity of background workers to achieve an appropriate balance between the frequency with which these workers run and the throughput they can achieve and during-the-day transactional activities, which these background workers (used for rule executions, scheduled exports, batched imports, etc.) may block, resulting in suboptimal latency levels for transactional activities. These optimizations will continue for the remainder of the week. We will seek to avoid any sustained impact to transactional database activity and latency as we achieve this balance.

  2. monitoring Jan 15, 2020, 07:03 PM UTC

    We are observing increased latency at this time and are working to identify the specific source of this increased latency to return levels to within normal operating parameters.

  3. monitoring Jan 15, 2020, 07:40 PM UTC

    We've isolated the source of the increased latency, and all systems have returned to operating within normal latency parameters and performance levels are steady at this time. We are continuing to monitor these systems.

  4. resolved Feb 11, 2020, 02:56 AM UTC

    This incident has been resolved.