Performance Degradation in saas.whitesourcesoftware.com

Update timeline

1. investigating Sep 01, 2021, 10:10 AM UTC

   Dear customers and partners, We are experiencing 2 issues in saas.whitesourcesoftware.com: 1) Logins randomly fail with an "Internal Server Error" message 2) Scan processing time is unusually long Kindly note that we are still investigating both of these issues and will continue to post updates here as we progress. We apologize for this temporary service disruption. Thank you all very much for your patience and understanding Avinoam Weitman Support Manager WhiteSource Software Inc.

2. investigating Sep 01, 2021, 02:43 PM UTC

   Dear customers and partners, I would like to update that issue #2 in my previous update, slow scan processing, has been resolved. We're still investigating issue #1, random SAML login failures, and will continue to provide updates until the service has been fully restored. Thank you, Avinoam Weitman Support Manager

3. monitoring Sep 02, 2021, 09:32 AM UTC

   Dear customers and partners, We have recently performed a few network changes in the SaaS infrastructure, to avoid the login issues altogether. We haven't seen any related login failures since 08:40 AM (UTC), however we're currently in the monitoring phase of this incident. Expect another update in the next few hours. Thank you all for your patience here! Avinoam Weitman Support Manager

4. monitoring Sep 02, 2021, 02:11 PM UTC

   Hello everybody, 1. The login issue in saas.whitesourcesoftware.com has been resolved. You may need to clean up your browser (cache and cookies) one last time, however after that you are expected to be logged in successfully, consistently. If you continue to face issues, don't hesitate to inform our friendly support team. 2. We noticed another spike related to scan processing, which might result in longer-than-usual scan processing times. We're still investigating this issue and will continue to post updates here. Thank you! Avinoam Weitman Support Manager

5. monitoring Sep 03, 2021, 04:38 AM UTC

   Hello everybody, I would like to update that the scan queue was cleared a few hours ago and we have since been noticing a significant improvement in scan processing times, in saas.whitesourcesoftware.com. We are still monitoring it as well as investigating the root cause. We will update if the status changes again and keep posting updates about the findings of our investigation. Thank you and have a nice day, Avinoam Weitman Support Manager

6. investigating Sep 03, 2021, 12:17 PM UTC

   Dear customers and partners, We have seen another spike impacting the scan processing times, since 08:00 UTC. We are still investigating the issue and will continue to update here with every new development. Thank you, Avinoam Weitman Support Manager

7. investigating Sep 03, 2021, 09:17 PM UTC

   Dear Customers and Partners, Our Engineering team continues to work on this investigation with very high priority. We have made improvements that have reduced the current impact and at the moment we are not seeing any delays. We will continue updating you as we reach a final root cause and resolution. We thank you again for your patience and understanding during this investigation Sincerely, Jorge Ramirez Director Technical Support - Americas WhiteSource

8. investigating Sep 05, 2021, 10:48 AM UTC

   Dear Customers and Partners, Following the recurring issue of long scan processing times since August 30th, we have rolled back the changes performed as part of the upgrade to 21.8.1 in our saas.whitesourcesoftware.com environment from August 29th. Users logging into that environment will notice that the application version is now 21.7.2 again and not 21.8.1. We will monitor the behavior and performance of the environment in the next few days and update regarding the next steps, based on our investigation and findings. Thank you all for your patience and understanding during the past week. Best regards, Avinoam Weitman Support Manager WhiteSource

9. investigating Sep 05, 2021, 10:49 AM UTC

   We are continuing to investigate this issue.

10. resolved Sep 09, 2021, 11:42 AM UTC

    Dear Customers and Partners, We've been monitoring the performance of saas.whitesourcesoftware.com and are glad to say that it's been stable since the rollback to 21.7.2 on Sunday (September 5th). The incident itself is going to be closed now, but will remain in a postmortem analysis state, until we conclude that part of the investigation. Thank you all very much for being patient and attentive to the status page updates since the incident was announced. Avinoam Weitman Support Manager WhiteSource