Google Calendar events are not appearing on devices

Update timeline

1. investigating Nov 08, 2019, 06:59 AM UTC

   We are seeing that Google calendar events are not saving on user devices/tablets. This appears to be affecting all Google calendar events. We are in early stages of investigating the issue. Next update: 1AM MT/3AM ET

2. resolved Nov 08, 2019, 07:46 AM UTC

   The issue is resolved. We have rolled back code changes made earlier this evening. After rolling back the changes, we confirmed Google calendar events have began saving on devices/tablets as normal. If you continue to see any issue, please open a Support ticket.

3. postmortem Nov 11, 2019, 07:36 PM UTC

   On the evening of Thursday, 7th of November, Teem conducted a routine code release. In this release, an optimization with Google Calendar syncing was introduced, that is designed to reduce the overall load on Teem servers as well as limit the number of API requests made to Google during calendar sync. These changes worked as anticipated, and gave us the expected improvements, handing a smaller event set to other areas of our application for storage. That limited event set however caused the storage phase to delete already retrieved events, making the calendar seem blank. Because the sync and storage both completed, no errors were raised or tests failed. Early in the morning of 8th of November, due to this unexpected behavior, Teem reverted the release that had happened hours before, going back to the previous known good state, and then removed the above mentioned changes before rolling code forward again on the afternoon of the 8th of November. Going forward, further changes will be made to the storage phase of calendar syncing to appropriately handle the conditions outlined above. In addition the testing surrounding this area of the application will be increased.