implio incident
Issue retrieving duplicate items and loading images in web app
implio experienced a notice incident on April 23, 2023 affecting implio core and implio web app, lasting 13h 3m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Apr 23, 2023, 04:42 PM UTC
There is an ongoing issue affecting manual moderation in the web app: - An "Error retrieving duplicated items" message sometimes appears in moderation queues. - Some images do no load. We have identified the source of the problem and we are working to resolve it.
- monitoring Apr 23, 2023, 05:54 PM UTC
A hot fix was deployed about an hour ago (17:55 UTC), fixing the issue. We're still addressing the underlying issue to prevent any further problem.
- resolved Apr 24, 2023, 05:46 AM UTC
This issue was resolved on Apr 23 at 17:55 UTC. All follow-up actions were implemented. Closing this incident. We apologize for any inconvenience this issue may have caused.
- postmortem Apr 24, 2023, 11:08 AM UTC
One of our OpenSearch clusters entered a rebalancing state after an automated software update. This, in turn, caused some data nodes to become overloaded and deny some of the inbound search queries. We will be increasing the capacity of this cluster to avoid any similar problem in the future.