Altmetric experienced a minor incident on February 15, 2023 affecting Altmetric Explorer and Data Processing, lasting 5d 2h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Feb 15, 2023, 10:53 AM UTC
We are currently investigating issues with the data refresh for the Altmetric Explorer. Customers may not be able to see the latest available data in the Altmetric Explorer while we identify and resolve the underlying causes. Explorer data is expected to be current up to Feb 14th 11AM GMT.
- identified Feb 15, 2023, 02:31 PM UTC
The issues preventing a successful Altmetric Explorer data refresh have been identified and we are currently working on a resolution. We will update further once a solution has been put in place and we have an estimate on when the data refresh will be completed.
- identified Feb 15, 2023, 04:01 PM UTC
We are continuing to work on a solution for the underlying issues preventing the Altmetric Explorer data refresh from completing successfully and will issue a further update at 5:30PM GMT.
- identified Feb 15, 2023, 05:48 PM UTC
Unfortunately the investigation so far has revealed that the issues preventing the Altmetric Explorer data refresh are more complex than initially estimated and will require further infrastructure work. As this will involve periods of interruption to Altmetric Explorer functionality, an emergency maintenance window has been scheduled for tomorrow morning, Feb 16th, beginning at 9:00 AM GMT time. We will post further updates at that time and through the course of the maintenance window, with the aim being to restore full Altmetric Explorer functionality by end of business hours on Feb 16th.
- identified Feb 16, 2023, 09:23 AM UTC
The remediation work aimed to address the data refresh issues affecting the Altmetric Explorer has started as scheduled. Short interruptions to Altmetric Explorer functionality are to be expected throughout the day until this emergency maintenance is complete, so we ask all users to plan accordingly. We expect to post a further update at 11AM GMT.
- identified Feb 16, 2023, 11:11 AM UTC
Remediation work is continuing as planned at this time. We will release a further update at 1PM GMT.
- identified Feb 16, 2023, 01:05 PM UTC
The majority of the remediation infrastructure work planned has now been completed without incident. We are continuing to work on minor configuration changes that will allow us to re-enable the data refresh process for the Altmetric Explorer and will update again at 3PM GMT.
- identified Feb 16, 2023, 03:25 PM UTC
Unfortunately we have encountered further issues trying to restore functionality to the data refresh process and continue to work on a remediation pathway. The Altmetric Explorer remains functional with data current up to February 14th. We will post a further update at 5:30 PM GMT
- identified Feb 16, 2023, 05:32 PM UTC
Our teams continue to work on remediation for the issues affecting the data refresh process for the Altmetric Explorer. At this time the Altmetric Explorer is stable but further work is still required to bring the data up to date. None of the other Altmetric products are currently affected by these or any other issues. Although an additional operational plan has been put together and work continues in the background we no longer expect a resolution to these issues today and will resume incident communication at 9:30AM GMT tomorrow.
- identified Feb 17, 2023, 09:47 AM UTC
Remediation work has been resumed on the issues affecting the Altmetric Explorer data refresh. We are currently in a position where we have been able to start the data refresh process and are currently monitoring its progress. We will post a further update at 11AM GMT.
- identified Feb 17, 2023, 12:54 PM UTC
The data refresh process is currently running for the Altmetric Explorer and we expect it to complete before 2PM GMT. We will update again once the Explorer data is up to date and to advise on further steps that we will be taking before closing the incident.
- identified Feb 17, 2023, 04:04 PM UTC
The data refresh process for the Altmetric Explorer has now successfully finished and the Altmetric Explorer data is now fully up to date. We are currently running the subsequent cache warming processes to improve the performance of the system when accessing the new data and will mark the systems as fully operational once this final step is complete.
- monitoring Feb 18, 2023, 01:32 PM UTC
All data refresh and cache warming operations are now running as scheduled. We continue to monitor the situation for any further issues.
- resolved Feb 20, 2023, 01:34 PM UTC
We are now satisfied that the data refresh issues initially encountered have been fully resolved and the data refresh process has remained stable. We are therefore closing the incident at this time and will be following up internally to identify any improvements and safeguards applicable to prevent these issues from re-occurring.