Dalenys incident

INCIDENT PROCESSING | Perturbations plateforme de paiement / Payment platform disruptions

Critical Resolved View vendor source →

Dalenys experienced a critical incident on March 27, 2025 affecting Processing e-commerce / E-commerce processing and Authentification 3DS / 3DS authentication and 1 more component, lasting 1h 31m. The incident has been resolved; the full update timeline is below.

Started
Mar 27, 2025, 03:28 PM UTC
Resolved
Mar 27, 2025, 04:59 PM UTC
Duration
1h 31m
Detected by Pingoru
Mar 27, 2025, 03:28 PM UTC

Affected components

Processing e-commerce / E-commerce processingAuthentification 3DS / 3DS authenticationPaiement en magasin - CB2A / Instore paymentPaiement en magasin - NEXO / Instore paymentMoyens de paiements alternatifs / Alternative payment methodsProcessing e-terminal / E-terminal processing

Update timeline

  1. investigating Mar 27, 2025, 03:21 PM UTC

    FR Nous avons identifié des difficultés sur la plateforme de paiement. L'incident est en cours d'analyse. EN We have identified ongoing difficulties on the payment platform. An investigation is in progress.

  2. monitoring Mar 27, 2025, 03:28 PM UTC

    FR Un fix est en cours. La situation est en train de revenir à la normale. Nous continuer de monitorer le service. EN A fix has been implemented and we are monitoring the results.

  3. monitoring Mar 27, 2025, 03:38 PM UTC

    TSR-1532 - Début / Start : 27/03/2025 16h11 CET - Fin / End : 27/03/2025 16h23 CET - Catégorie / Category : Production - Responsabilité / Responsibility : Payplug - Priorité / Priority : P1 FR La situation est revenue à la normale. Nous continuons de monitorer le service. EN The situation has now returned to normal. We are continuing to monitor the service.

  4. monitoring Mar 27, 2025, 04:57 PM UTC

    TSR-1532 - Début / Start : 27/03/2025 16h11 CET - Fin / End : 27/03/2025 16h23 CET - Catégorie / Category : Production - Responsabilité / Responsibility : Payplug - Priorité / Priority : P1 FR L'incident est maintenant résolu et le service est rétabli. EN Incident is now resolved and service restored.

  5. resolved Mar 27, 2025, 04:59 PM UTC

    This incident has been resolved.

  6. postmortem Mar 31, 2025, 07:39 AM UTC

    # _English version below_ # Post Mortem **Référence incident** TSR-1532 **Service concerné** Paiements e-commerce et magasin. **Impact client** Aucun paiement durant 13 minutes. **Synthèse de l’incident** * **16h11 :** mise en production d’une évolution sur un composant réseau \(ingress\). **Début de l’incident.** * **16h16 :** cellule incident majeur ouverte. * **16h21 :** déploiement du rollback de la mise en production. * **16h21 :** communication status page. * **16h24 :** Service rétabli. **Fin de l’incident.** **Contexte** N/A **Root cause** Dans le cadre d’une mise à jour d’un composant réseau \(ingress\), les serveurs correspondants ont été vu down par les load balancer, ce qui a bloqué l’ensemble des flux entrant sur la plateforme. **Actions à entreprendre par Payplug** | **Symptômes** | **Actions** | | --- | --- | | Mise en production en erreur. | Investigation de la mise en production avant toute tentative de redéploiement ultérieure. | | Test des composants réseau \(ingress externals\) ne permettant pas de valider qu’ils étaient fonctionnels après le changement en environnement de test. | Créer des tests dédiés afin de valider isolément le fonctionnement des ingress external des ingress internal. | ==============ENGLISH VERSION============== # Post Mortem **Incident reference** TSR-1532 **Payment services affected by the incident** E-commerce and instore payments.. **Client impact** No payment during 13 minutes. **Incident Overview** * **4h11 PM :** Production deployment of an upgrade on a network component \(ingress\). **Incident begins.** * **4h16 PM :** Major incident response team activated. * **4h21 PM :** Rollback deployment of the production change. * **4h21 PM :** Status page communication. * **4h24 PM :** Service restored. **Incident resolved.** **Context** N/A **Root cause** As part of an update to a network component \(ingress\), the corresponding servers were seen as down by the load balancers, which blocked all incoming traffic to the platform. **Actions to be taken by Payplug** | **Symptoms** | **Actions** | | --- | --- | | Production deployment failed. | Investigation of the deployment before any further redeployment attempts. | | Testing of network components \(external ingress\) did not confirm whether they were functional after the change in the test environment. | Create dedicated tests to validate the functionality of external ingresses separately from internal ingresses. |