DataCite incident
High load amongst content negotiation service.
DataCite experienced a minor incident on January 11, 2022 affecting Content Resolver API and Citation Formatter, lasting 12d 22h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 11, 2022, 11:00 AM UTC
During the past week or so we've noticed a huge surge of increased traffic to the Content Negotiation Service. This has unfortunately caused various problems with requests not being resolved. This is currently affecting both the Content Negotiation service but also the Citation Formatter service. We've made some initial improvements to support the increased load on the service, but it is unfortunately still causing issues. We will continue to investigate.
- identified Jan 19, 2022, 03:06 PM UTC
A large number of requests has been identified that are specifically attempting to resolve Crossref DOIs via the DataCite content negotiation service, we are having on-going conversations about this use-case. To remedy we've implemented some timeout logic and additional rate limiting, this however may have a knock on effect those attempting Content Negotiation for Crossref DOIs via DataCite Content Negotiation (Crossref Content Negotiation or via doi.org is unaffected) may receive a 404. Requests for DataCite DOIs via Content Negotiation should now start to be resolving.
- monitoring Jan 19, 2022, 06:56 PM UTC
The service appears stable now, and we are processing requests. We are monitoring now with the changes we've made. As mentioned in the previous update, do note requests for Crossref DOIs via DataCite Content Negotiation may still return a 404 and this is expected behaviour at present.
- resolved Jan 24, 2022, 09:20 AM UTC
We are still receiving a high load of requests but we are now fufilling due to increased capacity, so this issue is being marked resolved. Requests for DataCite DOIs should be unaffected going forward, requests for Crossref DOIs against DataCite Content negotiation directly may occasionally return 404s when it is unable to process. It is preferred if possible to use Content Negotiation via doi.org and you will be redirected as appropriate to the registration agencies content negotiation service as appropriate.