Eleos Technologies incident

Intermittent duplicate outbound messages

Eleos Technologies experienced a minor incident on November 3, 2023 affecting API and Mobile Apps, lasting 19m. The incident has been resolved; the full update timeline is below.

Started: Nov 03, 2023, 10:21 PM UTC
Resolved: Nov 03, 2023, 10:41 PM UTC
Duration: 19m
Detected by Pingoru: Nov 03, 2023, 10:21 PM UTC

Affected components

APIMobile Apps

Update timeline

identified Nov 03, 2023, 10:21 PM UTC

We've identified an issue that can cause outbound messages sent using the API to appear as duplicates in the mobile app. In addition, change updates (delivered via the inbound message API) for these messages may be delivered multiple times with a single client handle, but with different message UUIDs, one per duplicate. This change specifically affects messages that are submitted to the API more than once. This should normally be permitted without resulting in duplicates, but a recent infrastructure change caused this functionality to not work correctly resulting in duplicate messages seen by users within the app. We are rolling back the change.
resolved Nov 03, 2023, 10:41 PM UTC

As of 22:18 UTC we completed the rollback of the infrastructure change that caused the errant behavior, and have confirmed that messages submitted to the API more than once behave as expected. The rollback of this infrastructure change should not cause any unexpected behavior or changes in functionality of the Eleos Mobile Platform.