Certainly: Serving Expired OCSP Responses

Certainly LLC experienced an incident where expired OCSP responses were being served due to a failure in their OCSP updater service. The issue was traced back to a configuration change made during a Boulder release, which caused the updater to panic and stop generating new responses. The problem was identified on May 24, 2022, and was resolved by rolling back to a previous Boulder release on May 25, restoring normal service. A total of approximately 10,647 certificates were affected during the downtime. The CA has since implemented a remediation plan to enhance monitoring and prevent future occurrences.

1. 2022-05-17 OCSP updater service begins to panic.
2. 2022-05-24 Incident declared after expired OCSP responses were discovered.
3. 2022-05-25 Service restored after rolling back Boulder to a prior release.