TPG Telecom experienced a major incident on July 24, 2024 affecting Web Portal, lasting 1d 1h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jul 24, 2024, 09:39 PM UTC
We are currently experiencing issues navigating the web portal and contacts are not populating. Customers are advised to upload a CSV to send to campaigns or broadcasts while the engineers work to resolve the issue. Impact: - Contacts requests may fail to complete - Campaigns using contacts are also likely to fail Thank you for your patience as we work to resolve these issues.
- identified Jul 24, 2024, 11:06 PM UTC
We are continuing to work on a fix for this issue. Users may encounter token errors when sending messages directly from the web portal using contacts. As a workaround, please upload a CSV file to send your campaigns or broadcasts. Impact: - Contacts requests may fail to complete. - Campaigns using contacts are also likely to fail. - Web portal server errors appearing on campaigns. Our engineers are actively working to resolve the issue. Thank you for your patience.
- identified Jul 25, 2024, 12:38 AM UTC
We are continuing to investigate this contact web portal issue. We have identified the issue and are implementing a fix. Impact: - Web portal server errors appearing on campaigns. - Unable to update contact records or create new contacts Users may encounter token errors when sending messages directly from the web portal using contacts. As a workaround, please upload a CSV file to send your campaigns or broadcasts. Our engineers are actively working to resolve the issue. Thank you for your patience.
- identified Jul 25, 2024, 01:40 AM UTC
We have identified the issue with the contact web portal and are actively implementing a fix. Impact: - Web portal server errors appearing on campaigns. - Inability to update contact records or create new contacts. - Contacts Segment and List creation and management is blocked. - Contact Syncing is blocked. Users may encounter token errors when sending messages directly from the web portal using contacts. To bypass this, please upload a CSV file to send your campaigns or broadcasts. Customer who are sending through our API are unaffected. Our engineers are working to resolve the issue. Thank you for your patience
- identified Jul 25, 2024, 02:33 AM UTC
We are continuing to work on a fix for this issue.
- identified Jul 25, 2024, 03:33 AM UTC
We are continuing to work on a fix for this issue.
- identified Jul 25, 2024, 04:53 AM UTC
We are continuing to work on a fix for this issue.
- identified Jul 25, 2024, 05:40 AM UTC
We have identified the issue with the contact web portal and are actively implementing a fix. Impact: - Inability to update contact records or create new contacts. - Contacts Segment and List creation and management is blocked. - Contact Syncing is blocked. Customer who are sending through our API are unaffected. Our engineers are working to resolve the issue. Thank you for your patience
- identified Jul 25, 2024, 06:58 AM UTC
We are continuing to work on a fix for this issue.
- identified Jul 25, 2024, 08:05 AM UTC
We are continuing to work on a fix for this issue.
- identified Jul 25, 2024, 09:03 AM UTC
We are continuing to work on a fix for this issue.
- identified Jul 25, 2024, 10:08 AM UTC
We are continuing to work on a fix for this issue.
- identified Jul 25, 2024, 11:05 AM UTC
We are actively investigating the incident and are making continuous efforts to implement a resolution.
- identified Jul 25, 2024, 12:15 PM UTC
We are actively investigating the incident and are making continuous efforts to implement a resolution
- resolved Jul 25, 2024, 10:59 PM UTC
This incident has been resolved.