Eptura Workplace incident
S1 - Inability to Access iOffice for Specific Sites
Eptura Workplace experienced a major incident on October 12, 2022 affecting System Status, lasting 20h 13m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Oct 12, 2022, 07:31 PM UTC
Our engineering team has investigated and identified an issue for customers specifically regarding those that use ricohtrac.co.uk. We found the root cause of this issue to be related externally and are working with partners on a resolution. We will have another update at 2:30pm mst.
- identified Oct 12, 2022, 09:27 PM UTC
Our engineering team has investigated and identified an issue for customers specifically regarding those that use ricohtrac.co.uk. We found the root cause of this issue to be related externally and are working with partners on a resolution. We will not have another update till 12pm MST, tomorrow, 10/13/2022.
- resolved Oct 13, 2022, 03:44 PM UTC
This incident has been resolved.
- postmortem Oct 21, 2022, 07:12 PM UTC
**Preliminary Event Analysis \(RCA\) – Severity 1 – October 11, 2022** **Description:** _On October 11, 2022, at approximately 12:44pm MDT, our engineering team reported potential issues for Ricoh customers on sites_ [_ricohtrac.co.uk_](http://ricohtrac.co.uk)_. These customers may or may not experience issues accessing their site._ **Type of Event:** _System Disruption \(non-iOffice issue\)_ **Services\\Modules Impacted:** _All Ricoh customers on sites_ [_ricohtrac.co.uk_](http://ricohtrac.co.uk)_._ **Remediation:** _Transfer all domain registrations from GoDaddy to Cloudflare or_ [_101domains.com_](http://101domains.com) **Timeline:** _**October 11, 2022**_ _12:44pm MDT: Our engineering team reported potential issues for Ricoh customers on sites_ [_ricohtrac.co.uk_](http://ricohtrac.co.uk)_. Immediately our engineers contacted Cloudflare because we use them as a nameserver and around. Please note,_ [_ricohtrac.co.uk_](http://ricohtrac.co.uk) _remained functional._ _4pm MDT: We were pointed to GoDaddy as the root of our problems by Cloudfare. We were advised that issues happened on GoDaddy’s backend that broke our Nameservers and would take up to 24-48 hours to resolve._ _11:35pm MDT: The DNS still hasn’t propagated but_ [_ricohtrac.co.uk_](http://ricohtrac.co.uk) _continues to remain functional._ _**October 12, 2022**_ _4:21am MDT: Ricoh Support reports multiple Ricoh UK sites down._ _6:31am MDT: Engineering confirmed from backend, this is an issue and_ [_ricohtrac.co.uk_](http://ricohtrac.co.uk) _sites are down_ _7:40am MDT: iOffice Support sent communication to_ [_ricohtrac.co.uk_](http://ricohtrac.co.uk) _via email advising this is an issue we found which is caused by external partner, GoDaddy_ _1:31pm MDT: iOffice Support created incident for Teem status page_ _3:11pm MDT: iOffice Support reached out to Ricoh UK and advised that we are waiting on Nominet and that they open, October 13 8am UK \(12am MDT\). I advised that we are hopeful to have an update by Oct 13, 12pm MDT or sooner_ _**October 13, 2022**_ _1am MDT: Engineering engaged w/ GoDaddy and Nominet to make necessary step to transfer all domains from GoDaddy to Cloudflare._ _9:37am MDT: Engineering teams confirmed that_ [_ricohtrac.co.uk_](http://ricohtrac.co.uk) _sites are up and running._ _9:44am MDT: Teem status page was marked as resolved_ **Total Duration of Event:** _1day_ _21hrs 0mins_