YCharts incident

[Model Portfolio] Issue with subset of equities being removed/delisted from Model Portfolios

Notice Resolved View vendor source →

YCharts experienced a notice incident on January 27, 2023 affecting YCharts Application, lasting 2d 22h. The incident has been resolved; the full update timeline is below.

Started
Jan 27, 2023, 03:10 PM UTC
Resolved
Jan 30, 2023, 01:13 PM UTC
Duration
2d 22h
Detected by Pingoru
Jan 27, 2023, 03:10 PM UTC

Affected components

YCharts Application

Update timeline

  1. identified Jan 27, 2023, 03:10 PM UTC

    We have been made aware that a large number of Model Portfolios have had securities delisted in our overnight processing though they should not have been removed. We are actively investigating the issue and will provide updates here as they come in. We apologize for the inconvenience this causes our users.

  2. identified Jan 27, 2023, 04:33 PM UTC

    We have identified the issue from an upstream data provider that has given us stock data from 2018. We actively working with them to fix this issue upstream.

  3. identified Jan 27, 2023, 06:33 PM UTC

    We are continuing to work with our upstream provider to fix their issue. We will update as soon as we get an update from them.

  4. identified Jan 28, 2023, 05:35 PM UTC

    We have identified a solution with our upstream provider and are actively working on a fix. The estimate for resolution right now is by end of day.

  5. identified Jan 28, 2023, 07:29 PM UTC

    Our upstream data provider is still actively working on a fix. We are ready to fix our data pipeline as soon as we get the green light from our upstream data provider.

  6. identified Jan 29, 2023, 04:16 PM UTC

    We received confirmation that our data provider resolved issues with their securities data. We are actively reviewing and will be implementing a fix immediately afterward. The fix will solve affected securities first, then tertiary securities, such as Model Portfolios, afterward. The estimate for resolution is by the end of day. We will send out updates as we progress. Thank you for your patience.

  7. identified Jan 29, 2023, 08:59 PM UTC

    We have addressed the underlying issue impacting our stock and ETF universe. We are actively backfilling individual stock and ETF data and refreshing our Model Portfolio universe. During our investigation, we recovered all information except news articles and tick data for those affected securities on Thursday and Friday. The estimated time to resolution remains end of day today.

  8. monitoring Jan 30, 2023, 12:38 AM UTC

    We have addressed the issue impacting our stock, ETF, and Model Portfolio universe. These securities should be fully functional and we are monitoring performance closely. We will send a final notice when we are confident that the security universe has been resolved properly.

  9. resolved Jan 30, 2023, 01:13 PM UTC

    We resolved the issue impacting our stock, ETF, and Model Portfolio universe. These securities should now be fully functional.