LMS365 incident
Course Catalog, Course Homepage, and My Training Dashboard Web Parts not loading
LMS365 experienced a notice incident on December 6, 2024 affecting Learn365 Government Community Cloud and Learn365 Government Community Cloud High, lasting 4h 47m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Dec 06, 2024, 04:00 PM UTC
We have identified an issue that is affecting the loading of the web parts in SharePoint. At the moment the issue is no longer affecting our customers. We will add more details as the investigation proceeds.
- resolved Dec 06, 2024, 04:58 PM UTC
This incident has been fully resolved. A full postmortem will follow.
- postmortem Dec 09, 2024, 12:16 PM UTC
### Incident Overview During a one-time renewal deployment process for front-end assets, a global configuration update of Azure Front Door was required across both the **general Azure platform** and the **US GCC platform**. While the changes were successfully applied to the general Azure platform, the **US GCC platform encountered issues** due to platform-specific configuration differences. **It is important to emphasize that service degradation was not expected** during this process. However, incomplete propagation of configuration changes on the US GCC platform necessitated additional manual adjustments, resulting in temporary disruption loading the Course Catalog, Course Homepage and My Training Dashboard web parts ### Resolution Manual adjustments were made to complete the deployment on the US GCC platform, and the issue has been resolved. Full services have been restored, and the affected platform is operating normally. ### Follow-Up Actions 1. **Process Improvement:** Deployment processes have been revised to account for platform-specific differences, ensuring smoother execution in the future. 2. **Documentation Update:** Comprehensive updates have been made to include detailed guidance for handling deployments across multiple Azure platforms. 3. **Mitigation for Future Incidents:** While this was a one-time process that will not recur, the lessons learned will be applied to similar scenarios to ensure no unexpected service degradation occurs in the future. We sincerely apologize for any inconvenience caused and remain committed to delivering reliable services.