Google Trust Services: SXG certificates issued without correctly checking CAA restrictions
Google Trust Services (GTS) disclosed an incident involving the issuance of SXG certificates. GTS identified a corner case introduced during refactoring where SXG issuance did not correctly check additional CAA requirements specified in its CP for SXG certificates. Specifically, GTS required the presence of an "issue" or "issuewild" CAA property to authorize issuance, but it did not properly handle cases where no applicable "issue"/"issuewild" records were present while other CAA records were present (e.g., only an "iodef" property, or a non-wildcard FQDN with only an "issuewild" property). During incident handling, GTS deployed a fix, stopped SXG issuance for about 2 hours and 28 minutes, and revoked all affected active certificates within 24 hours of discovery. GTS reported that 58 SXG certificates were issued due to the CAA bug from July 19, 2022 to June 14, 2024, with 12 certificates (9 active at discovery plus 3 additional misissued during handling) revoked within 24 hours. GTS also listed and completed a set of action items (including adding SXG deny tests to caatestsuite.com) and requested the bug be closed; Mozilla indicated it would close it around July 31, 2024.
- First SXG certificate affected by the CAA-checking bug was issued.
- GTS discovered a potential CAA bug affecting SXG certificates and began remediation steps.
- First unsuccessful rollout attempt to fix the issue was started.
- GTS confirmed the CAA bug affecting SXG certificates and reported the incident.
- GTS proposed a PR to add SXG deny tests to caatestsuite.com, completing the last remaining action item.
- GTS stated all action items were complete and requested closure.
- Google representative — GTS reported it was investigating SXG certificate issuance without correctly verifying CAA record parameters, said a fix was deployed, and stated affected certificates would be revoked within 24 hours.
- Google representative — GTS posted a full incident report describing the SXG-specific CAA corner case, the impact (58 issued; 12 revoked), and the remediation and timeline, including that TLS CAA checks were unaffected.
- Mm representative — A commenter asked for examples clarifying how the "cansignhttpexchanges" requirement was and was not enforced, and suggested adding SXG checks to caatestsuite.com.
- Google representative — GTS provided a table of CAA record sets showing expected TLS vs SXG outcomes and the faulty-code outcomes, and discussed practices for handling draft requirements.
- Google representative — GTS said it was monitoring the bug and listed remaining action items with due dates, requesting the NextUpdate field be set to July 12.
- Google representative — GTS reported completing formal analysis of RFC 8659, RFC 8657, and the SXG draft specification and said identified tests/code changes were made and would reach production in the next rollout.
- Google representative — GTS reported it proposed a PR to add SXG deny tests to caatestsuite.com, completing the last remaining action item.
- Google representative — GTS stated all action items were complete and requested the bug be closed if no further comments or questions.
- Mozilla representative — Mozilla said it would close the bug on or about July 31, 2024 unless additional comments or questions were raised.