tawk.to incident

Delays in email deliveries to @gmail.com mailboxes

Minor Resolved View vendor source →

tawk.to experienced a minor incident on November 17, 2023 affecting Ticketing and Emails, lasting 42d 6h. The incident has been resolved; the full update timeline is below.

Started
Nov 17, 2023, 08:39 AM UTC
Resolved
Dec 29, 2023, 03:37 PM UTC
Duration
42d 6h
Detected by Pingoru
Nov 17, 2023, 08:39 AM UTC

Affected components

TicketingEmails

Update timeline

  1. identified Nov 17, 2023, 08:39 AM UTC

    We've identified issue in our email infrastructure that is causing deliveries to @gmail.com domains to be delayed

  2. identified Nov 23, 2023, 11:45 AM UTC

    Our team has identified source of large volume of unsolicited emails being generated that caused queue buildup. Offending senders have been suspended indefinitely and pending emails form those senders has been dropped from outbound email queue.

  3. identified Nov 23, 2023, 11:48 AM UTC

    We've deployed additional outbound email servers to increase sending capacity

  4. identified Nov 23, 2023, 11:54 AM UTC

    We're still experiencing heavy rate-limiting when sending to GMail based domains and GMail itself. Our engineering team is in contact with our google account managers looking for potential solutions. In the mean time we're restricting deliveries of chat transcript emails to @gmail.com domains to prioritize ticket deliveries

  5. identified Nov 23, 2023, 12:02 PM UTC

    Our engineers have updated handling of DMARC records to help improve and separate reputation of individual property specific sender domains.

  6. identified Dec 18, 2023, 07:40 AM UTC

    Our engineers are working to establish a new sender domain for transactional emails such as tickets. By doing so we are looking to separate sender reputations for notifications and transactional emails.

  7. monitoring Dec 18, 2023, 07:49 AM UTC

    We are starting to slowly roll out new sender domain for transactional emails as we continue work on improving our sender reputation.

  8. monitoring Jan 10, 2024, 06:41 PM UTC

    Through continued efforts of our engineers, we have narrowed down the issue to a handful of affected sender domains and we're continuing to work with affected property owners on solution on a case-by-case basi

  9. monitoring Jan 10, 2024, 06:44 PM UTC

    General deliveries to gmail are back to normal. We continue to monitor the situation and work with a handful of affected property owners who still are seeing issues.

  10. resolved Jan 10, 2024, 06:50 PM UTC

    We're no longer seeing general issues delivering to gmail based domains. If you are still seeing issues, please contact us in live chat or at [email protected]