IdenTrust: Intermitent interruptions to DNS service

IdenTrust reported an incident involving intermittent DNS service interruptions that it linked to the expiration of its IdenTrust Root DST X3 (DST X3) on September 30, 2021. IdenTrust system engineers were notified of large volumes of network traffic causing intermittent site connectivity, slow responses, and timeouts for customers attempting OCSP validations and other certificate lifecycle events. IdenTrust also logged intermittent CRL validation failures beginning around 5:00 pm MT on 10/05/2021 for traffic routed to a DR system via a round-robin method. IdenTrust determined the CRLs used by the DR system were not updated according to normal production protocol and were outdated, and after testing it pushed new CRLs to the DR system at 9:18 am MT on 10/06/2021, confirming resolution. In parallel, IdenTrust moved approximately 40% of traffic to its secondary site to alleviate bottlenecks and stabilize response times. IdenTrust stated that continued monitoring found no further DNS service interruptions and requested the bug be resolved as “Fixed,” with the bug status shown as RESOLVED and resolution FIXED.

1. 2021-09-30 IdenTrust Root DST X3 (DST X3) expired as expected, triggering increased customer traffic to download the new certificate chain.
2. 2021-10-05 Intermittent CRL validation failures were logged for traffic routed to the DR system via round-robin.
3. 2021-10-06 IdenTrust identified outdated CRLs on the DR system and pushed updated CRLs to the DR system, confirming resolution.

1. 2021-10-08 IdenTrust Services, LLC — IdenTrust disclosed the incident, explaining how DST X3 expiration led to intermittent connectivity and CRL validation failures on the DR system, and provided a timeline of remediation actions including traffic redistribution and CRL updates.
2. 2021-10-15 IdenTrust Services, LLC — IdenTrust reported continued monitoring with no further DNS service interruptions and stated no additional tasks were pending.
3. 2021-10-25 IdenTrust Services, LLC — IdenTrust reported volumes had subsided with no further DNS service interruptions and requested the bug be resolved as “Fixed.”