IdenTrust: Unavailable CRL and OCSP Responders
IdenTrust reported an incident on February 4, 2022, where it experienced an issue for about 8 hours that prevented consistent CRL and OCSP online responses within the B.R. 4.10.2 service availability guidelines. IdenTrust said it first became aware of the problem through system monitoring alerts indicating connectivity issues affecting customers. The CA stated that high-volume traffic hitting its firewalls contributed to the firewalls being unable to pass traffic, and that it worked with a third-party vendor to identify a known firmware problem on the firewalls. IdenTrust said it restored validation services by restarting services so that renewed OCSP responder certificates would be recognized, after services were initially confirmed operational but only issuance had been restored. In response, IdenTrust reported working with the vendor to bypass the problematic firewall functionality, upgrading the firewall firmware the following day, and deploying additional monitoring to detect OCSP responder certificate renewals in advance of expiration. IdenTrust later stated that the CRL and OCSP responders referenced in the incident report had been consistently available without interruptions and requested formal closure; Mozilla indicated the incident could be closed. The bug is resolved as FIXED.
- IdenTrust experienced an ~8-hour outage affecting consistent CRL and OCSP online responses due to firewall connectivity issues.
- Mozilla asked about whether training/documentation contributed and which services/devices required restarting.
- IdenTrust reported the referenced CRL/OCSP responders had been consistently available since the incident and requested formal closure.
- Mozilla indicated the incident could be closed.
- IdenTrust Services, LLC — IdenTrust described the February 4, 2022 incident, its cause and remediation steps, and stated it would deploy additional monitoring to prevent recurrence.
- Mozilla representative — Mozilla asked whether lack of training/documentation caused the service not to be restarted and which service/device needed restarting.
- IdenTrust Services, LLC — IdenTrust responded that the responsible individual was trained and documented procedures exist, and that services tied to the Commercial Root needed restarting to restore validation.
- Mozilla representative — Mozilla stated it had no further questions.
- IdenTrust Services, LLC — IdenTrust reported consistent availability of the referenced CRL/OCSP responders since 2022-02-04 19:59 MST and requested formal closure.
- IdenTrust Services, LLC — IdenTrust stated there were no pending actions other than including the incident in the annual WebTrust audit.
- IdenTrust Services, LLC — IdenTrust confirmed there were no other pending activities for the incident besides including it in the WebTrust annual audit report.
- Mozilla representative — Mozilla indicated the incident could be closed and would do so on or about 13-April-2022.