PocketSmith incident

Forecast recalculation and bank feed processing delays

Major Resolved View vendor source →

PocketSmith experienced a major incident on December 22, 2020 affecting Web and Feeds, lasting 2h 10m. The incident has been resolved; the full update timeline is below.

Started
Dec 22, 2020, 09:14 AM UTC
Resolved
Dec 22, 2020, 11:25 AM UTC
Duration
2h 10m
Detected by Pingoru
Dec 22, 2020, 09:14 AM UTC

Affected components

WebFeeds

Update timeline

  1. investigating Dec 22, 2020, 09:14 AM UTC

    Forecast recalculations, bank feed jobs, file upload processing and other queued work is currently non-operational. We've investigating now.

  2. identified Dec 22, 2020, 09:20 AM UTC

    We've identified the cause of the background job processing delays, and have resolved the issue. The backlog is being worked through now. We'll post updates here as queues clear.

  3. monitoring Dec 22, 2020, 09:35 AM UTC

    With the exception of bank feeds, all other background job queues have now caught up and are back to normal. This means that forecast recalcuations and file imports will now be processing as expected. We'll update here again as bank feed processing returns to normal.

  4. monitoring Dec 22, 2020, 09:47 AM UTC

    The bank feed queues are now processing new jobs as normal again. We're performing remedial actions on some bank feeds which could be stuck syncing, and will update here when completed.

  5. resolved Dec 22, 2020, 11:25 AM UTC

    Everything is back to normal now. We're sorry for the inconvenience this glitch has caused. Please get in touch with us if you notice anything out of the ordinary.