Shelf.io Outage History

Started Jun 23, 2026, 12:00 PM UTC· resolved Jun 23, 2026, 07:00 PM UTC

1. resolved Jul 10, 2026, 08:04 AM UTC

   Between 23 Jun 12:00 UTC and 25 Jun 19:00 UTC, we experienced intermittent errors in core resource fetching. This caused a small percentage of requests to fail, which in some cases resulted in users seeing a white screen instead of the app. The incident affected a small number of customers globally. Due to a configuration issue, impact may have been higher for some customers than others. The issue has been resolved, and service has returned to normal. We apologize for the disruption.

Read the full incident report →

Started Nov 03, 2025, 02:15 PM UTC· resolved Nov 03, 2025, 06:43 PM UTC

1. investigating Nov 03, 2025, 04:47 PM UTC

   Some customer Self-Service Portal homepages were showing other customers' homepage content. Other pages (search, article, product pages) remained functional. Workaround: avoid the homepage. Append a language code, or add a query character such as ?, to force a fresh render.

2. identified Nov 03, 2025, 05:07 PM UTC

   Root cause narrowed to CDN cache contamination triggered by a configuration rollback. The contamination affected homepage edge cache only.

3. monitoring Nov 03, 2025, 05:10 PM UTC

   Self-Service Portal homepages are now returning correct content after cache purges. We are monitoring edge cache consistency and user reports for recurrence.

4. resolved Nov 03, 2025, 06:43 PM UTC

   Duration: ~2h55m. Cause: CDN cache contamination of homepage caches. Impact limited to US and EU Self-service portal homepages. All systems normal.

Read the full incident report →

Started Oct 20, 2025, 02:16 PM UTC· resolved Oct 21, 2025, 06:18 AM UTC

1. investigating Oct 20, 2025, 02:16 PM UTC

   We are currently investigating this issue.

2. investigating Oct 20, 2025, 02:21 PM UTC

   We are continuing to investigate this issue.

3. monitoring Oct 20, 2025, 02:26 PM UTC

   A fix has been implemented and we are monitoring the results.

4. identified Oct 20, 2025, 03:01 PM UTC

   The issue has been identified and a fix is being implemented.

5. monitoring Oct 20, 2025, 03:17 PM UTC

   A fix has been implemented and we are monitoring the results.

6. monitoring Oct 20, 2025, 03:35 PM UTC

   We're still observing occasional timeouts in Agent Assist searches for certain real-time suggestions.

7. identified Oct 20, 2025, 04:13 PM UTC

   Reopened. Our earlier resolution was premature. Our cloud infrastructure provider continues to report widespread degradation in their US‑East region (about 80 services affected, several in a brownout state). This is causing intermittent errors, elevated latency, and timeouts across multiple Shelf services in the US region, Answer Assist and Search, and integrations. EU and CA regions remain healthy. We are working with the provider and have applied mitigations to reduce impact. Next update within 30 minutes.

8. identified Oct 20, 2025, 05:46 PM UTC

   Update - This remains a brownout, not a full outage. Most requests in the US region are succeeding, but ongoing upstream degradation is causing intermittent errors and latency. Authentication, Viewing Content & Search, Self‑Service Portals, Emails & Notifications, and Making Changes to Content are largely available; Integrations and Insights & Analytics are the most affected with higher delays/timeouts. We continue to mitigate and monitor.

9. monitoring Oct 20, 2025, 08:08 PM UTC

   Update - Error rates are trending down. Over the last few minutes the US region is averaging about 0.4% 5xx responses. Core paths (Authentication, Viewing Content & Search, Self‑Service, and content edits) are each ≤0.5%, while Integrations and Insights & Analytics fluctuate slightly higher but remain under 1%. We'll keep mitigations in place and continue to monitor.

10. monitoring Oct 20, 2025, 10:01 PM UTC

    In the last three hours we observed a single brief spike in 5xx responses around 20:45 UTC in the US region; since then, 5xx has held at 0% across monitored endpoints. For transparency, we'll keep the incident open in Monitoring with no components marked as affected through the end of the day while we continue to observe.

11. resolved Oct 21, 2025, 06:18 AM UTC

    This incident has been resolved.

Read the full incident report →

Started Oct 20, 2025, 06:48 AM UTC· resolved Oct 20, 2025, 10:14 AM UTC

1. investigating Oct 20, 2025, 07:13 AM UTC

   We are investigating reports of sign‑in failures in our US region. The first errors began at approximately 06:48 UTC. EU and CA regions remain healthy. We'll share more as we learn. Next update within 30 minutes.

2. identified Oct 20, 2025, 07:24 AM UTC

   We have identified that the sign‑in failures correlate with a regional incident at our cloud infrastructure provider in their US‑East region. Customers served from our US region may see "incorrect password" messages or slow/failed requests. Our EU and CA regions remain healthy.

3. identified Oct 20, 2025, 08:28 AM UTC

   The issue persists. Our cloud infrastructure provider continues to experience a region-wide incident in their US‑East region. We are actively working with the provider to reduce impact.

4. monitoring Oct 20, 2025, 09:29 AM UTC

   US region services have returned to normal operation as of 09:22 UTC. This incident stemmed from a region‑wide issue at our cloud infrastructure provider and caused intermittent errors and latency across multiple US services. Impact window: 06:48-09:22 UTC (2 hours 34 minutes). No data was lost. EU and CA regions were not affected. Thank you for your patience.

5. resolved Oct 20, 2025, 10:14 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 17, 2025, 03:22 PM UTC· resolved Oct 17, 2025, 04:03 PM UTC

1. investigating Oct 17, 2025, 03:34 PM UTC

   We are currently investigating increased error rates affecting search and content discovery in our US region. Some users may experience slower response times or intermittent errors when searching for content, viewing announcements, or submitting feedback. Other platform capabilities, including content creation and editing, remain fully operational.

2. monitoring Oct 17, 2025, 04:03 PM UTC

   We have identified and addressed the underlying issue affecting search and content discovery performance in the US region. Our engineering team has implemented corrective measures and is actively monitoring system behavior. During peak impact, approximately 20-40% of search and content discovery requests experienced errors. We continue to observe system performance to ensure full stability.

3. resolved Oct 19, 2025, 06:29 PM UTC

   This incident has been resolved. All search, announcement, and feedback services in the US region have returned to normal operation. System monitoring confirms stable performance with no residual errors. We apologize for any disruption this may have caused.

4. postmortem Oct 19, 2025, 07:47 PM UTC

   On October 17, 2025, between 15:22 and 16:03 UTC, users in our US region experienced intermittent errors when searching for content, viewing announcements, and submitting feedback. At the peak of the incident, approximately 40% of requests to these services returned errors, while other platform capabilities remained fully operational. This event occurred during an ongoing maintenance process to upgrade our search relevance engine - work aimed at delivering more accurate, higher‑quality results to users. We ran this upgrade path in the background for several days at minimal load to avoid disruption. Near the end of the week, a configuration error caused the process to place more load on the system than intended. Coupled with a spike in search activity, this created temporary capacity pressure and intermittent errors in the US region. The outage was partial: error rates varied over the window and peaked around 40%, with the majority of requests continuing to succeed. We've already applied lessons from a prior search incident and rolled out the upgraded search infrastructure in the EU and CA regions; the US region is scheduled next. We'll continue with this upgrade, and will better communicate maintenance windows in advance, align change timing to lower‑traffic periods, and add redundancy specific to search operations to further protect performance during these necessary maintenance activities.

Read the full incident report →

Started Sep 12, 2025, 04:30 PM UTC· resolved Sep 12, 2025, 08:00 PM UTC

1. identified Sep 12, 2025, 05:04 PM UTC

   Starting at 16:30 UTC, an infrastructure issue caused elevated 5xx errors for application and API requests. We have identified the cause and are rolling out a fix. Impact affects a subset of customers across multiple regions.

2. identified Sep 12, 2025, 05:14 PM UTC

   We are seeing recovery in the CA and EU regions, and a fix is being rolled out to the remaining regions. A subset of customers outside those regions may still be impacted.

3. identified Sep 12, 2025, 05:21 PM UTC

   We are continuing to work on a fix for this issue.

4. identified Sep 12, 2025, 05:30 PM UTC

   We are continuing to work on a fix for this issue.

5. identified Sep 12, 2025, 05:57 PM UTC

   Search has recovered across regions. Feedback and CPW remain degraded for a subset of customers while the fix completes rollout. Next update within 30 minutes.

6. monitoring Sep 12, 2025, 06:05 PM UTC

   We’ve deployed a fix and are seeing recovery for application and API requests across regions. If you encountered errors, please try again or refresh your browser.

7. monitoring Sep 12, 2025, 06:10 PM UTC

   All services have recovered. Error rates and latency are back to normal across all regions. We’ll continue to monitor closely for the next hour. If you still encounter issues, please refresh your browser and try again.

8. resolved Sep 12, 2025, 08:00 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Sep 10, 2025, 02:38 PM UTC· resolved Sep 10, 2025, 04:35 PM UTC

1. investigating Sep 10, 2025, 04:06 PM UTC

   Starting at 14:38 UTC, we observed a spike in SSO authentication failures. Affected users may be unable to sign in and see "access_denied (Unauthorized)" after returning from their IdP. We are actively investigating with our identity provider. Next update within 30 minutes.

2. identified Sep 10, 2025, 04:26 PM UTC

   We have deployed a fix to our authentication flow at 16:15 UTC. Users should begin to see successful sign-ins resume across all regions. Please retry logging in; starting a new browser session (or incognito) may help clear a failed attempt. We’re closely monitoring error rates and will provide another update within 30 minutes.

3. resolved Sep 10, 2025, 04:35 PM UTC

   SSO sign-ins have been restored. Between 14:38-16:27 UTC a configuration regression in our authentication layer caused the authorization code exchange to fail, resulting in "access_denied (Unauthorized)" after IdP login. We corrected the configuration and refreshed services; metrics show recovery across all regions.

Read the full incident report →

Started Sep 02, 2025, 06:26 PM UTC· resolved Sep 02, 2025, 06:56 PM UTC

1. investigating Sep 02, 2025, 07:16 PM UTC

   We are investigating an issue where users could experience intermittent errors and delays when searching or viewing content; content creation and edits remain available.

2. identified Sep 02, 2025, 07:16 PM UTC

   We have identified a performance issue and are working to restore normal search and content viewing.

3. monitoring Sep 02, 2025, 07:17 PM UTC

   Mitigation is in place and performance is improving for search and content viewing; we will provide another update by 18:56 UTC.

4. resolved Sep 02, 2025, 07:18 PM UTC

   The issue affecting search and content viewing has been resolved and service is operating normally.

5. postmortem Sep 02, 2025, 07:19 PM UTC

   Between 18:26 - 18:56 UTC in the US region, approximately 20% of search and content viewing requests failed outright. The remaining successful requests experienced elevated latency, with p99 around 17 seconds. Content creation and edits were not affected.

Read the full incident report →

Started Aug 22, 2025, 09:27 AM UTC· resolved Aug 22, 2025, 06:19 PM UTC

1. investigating Aug 22, 2025, 09:27 AM UTC

   We're currently experiencing an issue with a third-party service provider that may impact some users' experience. Our team is actively working with the provider to resolve the situation as quickly as possible.

2. investigating Aug 22, 2025, 09:27 AM UTC

   We are continuing to investigate this issue.

3. investigating Aug 22, 2025, 10:42 AM UTC

   We have identified that the issue is limited to gem listing updates. Specifically, some gems may continue to appear in their previous location after being moved or archived. Our team is actively working on a fix to ensure updates are reflected correctly. We will share another update as soon as we have progress to report. Thank you for your understanding.

4. identified Aug 22, 2025, 12:48 PM UTC

   Core issue is isolated, we are working on the fix

5. identified Aug 22, 2025, 03:10 PM UTC

   Our Team is actively working on a resolution. We will provide further updates as soon as more information becomes available. Thank you for your patience.

6. identified Aug 22, 2025, 03:43 PM UTC

   We have begun applying fixes to address the issue. While progress has been made, the issue is not yet fully resolved. Our team continues to work toward a complete resolution, and we will share updates as they become available.

7. identified Aug 22, 2025, 03:50 PM UTC

   Some functionalities have been successfully restored. We are continuing to work on bringing full service back online and will provide updates as progress continues.

8. resolved Aug 22, 2025, 06:19 PM UTC

   The issue has been fully resolved and all functionalities are now restored. Thank you for your patience throughout this incident.

Read the full incident report →

Started Jul 03, 2025, 03:26 PM UTC· resolved Jul 03, 2025, 03:46 PM UTC

1. identified Jul 03, 2025, 03:48 PM UTC

   The issue has been identified and a fix is being implemented.

2. resolved Jul 03, 2025, 03:51 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jun 27, 2025, 12:40 PM UTC· resolved Jun 27, 2025, 02:51 PM UTC

1. resolved Jul 03, 2025, 03:53 PM UTC

   On 27 June 2025, between 14:40 UTC and 14:51 UTC, some users in the US region were unable to retrieve search results after selecting a library. Other areas of the platform continued to operate normally.

Read the full incident report →

Started Apr 29, 2025, 03:37 PM UTC· resolved Apr 29, 2025, 04:03 PM UTC

1. identified Apr 29, 2025, 04:03 PM UTC

   We are investigating reports of users being unable to view Gem content.

2. identified Apr 29, 2025, 04:04 PM UTC

   We have identified the root cause related to a deployment configuration and are deploying a correction.

3. resolved Apr 29, 2025, 04:05 PM UTC

   The configuration issue has been corrected, and Gem content access via KMS is restored.

4. postmortem Apr 29, 2025, 06:36 PM UTC

   Between approximately 15:37 UTC and 16:03 UTC on April 29, 2025, some users experienced an issue where content pages \('Gems'\) would not load when accessed directly from the main Shelf dashboard or search results. This temporary issue stemmed from the premature activation of a configuration change during preparations for our planned user interface update, scheduled for April 30th. **Impact & Workaround:** During this time, content remained accessible via the Gem Preview feature. Other platform areas like Agent Assist, Self-Service Portals, and Content Intelligence were unaffected. **Resolution:** The issue was identified and fully resolved by 16:03 UTC. All systems are now operating normally. We apologize for any inconvenience caused.

Read the full incident report →

Started Feb 25, 2025, 05:20 PM UTC· resolved Feb 25, 2025, 08:56 PM UTC

1. identified Feb 25, 2025, 07:53 PM UTC

   We are currently experiencing a disruption to our Content Intelligence service in the US region that began at 17:20 UTC. Users may be unable to view analytics dashboards and charts. All data remains secure, and any changes made during this period will be properly processed once service is fully restored. Our engineering team has identified the root cause as an underlying infrastructure issue with our service provider and is actively implementing mitigation measures, including routing operations to redundant systems. We are in direct communication with AWS support engineers via a priority incident channel and collaboratively working toward resolution. This issue affects only the US region; EU and CA regions remain fully operational. We will provide updates as the situation progresses.

2. monitoring Feb 25, 2025, 08:12 PM UTC

   The Content Intelligence service in the US region has been fully restored as of 20:00 UTC. Our engineering team successfully implemented a failover solution using a recent system backup taken moments before the disruption. All user data is intact and operations performed during the outage have been properly stored. Some users may experience brief periods of reduced performance as our systems complete the scaling process. We continue to monitor the service closely during this stabilization period and appreciate your patience throughout this event.

3. resolved Feb 25, 2025, 08:56 PM UTC

   The disruption to Content Intelligence service in the US region has been fully resolved. Our engineering team successfully restored all services to their normal state. The situation has been closely monitored, and performance metrics have stabilized without further issues. We appreciate your patience and understanding during this time.

Read the full incident report →