Certainly: Early CRL Entry Removal
Certainly reported a potential compliance issue involving CRL generation. The initial incident report stated that on 2025-02-13 Certainly deployed a Boulder version with a logic bug that could remove entries for some revoked certificates from the CRL before the certificates had expired, which would violate Baseline Requirements section 4.10.1 and RFC 5280 section 3.3. Certainly said it became aware of the issue at 16:23 UTC on 2025-03-18 after a member of the Let’s Encrypt team directed it to a Boulder pull request containing the fix, and that a fix was deployed. In the full incident report, Certainly described an impact assessment (including a total of 77,467 certificates and zero “remaining valid” certificates) and stated that issuance was not stopped because the incident did not affect issuance. After further review, Certainly revised its evaluation and concluded it was not actually impacted because its configuration did not enable the feature flags needed to trigger the buggy code path, and it confirmed it had not issued incomplete CRLs. Based on this revised evaluation, Certainly requested the bug be closed as invalid, and the bug status is RESOLVED with resolution INVALID.
- Certainly deployed a Boulder release that included a CRL generation logic bug.
- Let’s Encrypt notified Certainly about the upstream Boulder fix, prompting Certainly’s investigation.
- Certainly deployed the upstream fix to address the CRL generation bug.
- A preliminary incident report was published in the bug.
- Certainly revised its evaluation after further review and requested closure as invalid.
- Fastly representative — Wayne Thayer posted a preliminary incident report describing the suspected CRL entry removal issue and stated a fix had been deployed with a full report to follow.
- Fastly representative — David Jeffery posted the full incident report with a detailed timeline, impact assessment, and root cause analysis, and noted they would continue monitoring.
- Fastly representative — David Jeffery revised the incident evaluation, stated Certainly was not actually impacted due to configuration/feature flags, confirmed CRLs were complete, and requested the bug be closed as invalid.