Netlock incident: AIA CA Issuers endpoints served PEM instead of DER
This case tracks Netlock’s incident report about Authority Information Access (AIA) CA Issuers endpoints that returned issuer certificates in PEM format instead of the DER encoding required by RFC 5280 section 4.2.2.1. Netlock opened the bug on 2025-12-08 and said the issue had been identified through an external community report received on 2025-12-06. During the investigation, Netlock first reported several affected hierarchies, later acknowledged that its initial production deployment on 2026-01-08 did not fully resolve the problem, and said remediation required software changes in its publishing component. On 2026-09-24, Netlock said an external re-verification found all endpoints in its published inventory returned DER but that some legacy-host paths still returned PEM, and that its automated validation had not covered every `ca=` value the CGI would answer. On 2026-10-02, Netlock said the PEM encoding defect was resolved, reported zero PEM responses across 97 `ca=` values on five affected virtual hosts, and corrected its earlier scope from two affected values to 12. Netlock also said those 12 values were changed from dynamically produced responses to static DER files and that no certificate re-issuance was required. In follow-up comments on 2026-10-09 and 2026-10-14, Netlock said repeated checks of all 12 affected `ca=` values on aia1, aia2, and aia3 continued to return DER, but it missed promised dates for an updated full incident report and set a new target of 2026-10-21 12:00 UTC. The bug remains open and assigned while Netlock says the DER fix is holding and additional report updates are still in progress.
- Non-compliant AIA configuration was introduced for the pdvca hierarchy.
- Non-compliant AIA configuration was introduced for the trustev3 and qtrustev3 hierarchies.
- Non-compliant AIA configuration was introduced for the DVCA hierarchy.
- Netlock said an external community report identified the AIA encoding non-compliance.
- Netlock said its initial production deployment was completed.
- Netlock staff and an external observer reported that some AIA endpoints were still serving PEM-encoded content.
- Netlock said testing identified and corrected an implementation defect, and re-testing began.
- Netlock said auditor procedures were updated so AIA CA Issuers endpoints are explicitly retrieved and their encoding verified during audit testing.
- Netlock said some legacy-host AIA paths still returned PEM and that its automated validation had not covered every `ca=` value the CGI would answer.
- Netlock said the PEM encoding defect was resolved and that all 97 tested `ca=` values returned DER.
- Netlock said re-measurement of all 12 affected `ca=` values on aia1, aia2, and aia3 again showed DER responses on every host.
- Netlock said another check of all 12 affected `ca=` values on aia1, aia2, and aia3 again found only DER responses.
- Netlock — Netlock opened the bug with a preliminary incident report stating that a third party had reported an AIA CA Issuers URI returning PEM content instead of DER.
- Community commenter — A community commenter said Netlock had not provided a satisfactory full incident report within 14 days and listed additional PEM-serving AIA URLs.
- Netlock — Netlock posted a full incident report describing affected hierarchies, dates, root causes, and a planned production release.
- Netlock — Netlock said demo validation had been completed and production deployment was still planned for 2026-01-09.
- Netlock — Netlock said production deployment had been completed on 2026-01-08 and that a closure summary would follow.
- Google representative — Google reported that several endpoints were still serving PEM-encoded certificates after the claimed deployment.
- Community commenter — A commenter said Netlock’s report did not follow CCADB incident-reporting guidance and included unsupported boilerplate claims.
- Apple representative — Apple asked Netlock to explain why similar prior bugs were not listed as related incidents and whether Netlock reviews other CAs’ incident reports.
- Netlock — Netlock updated its full incident report, acknowledged that PEM content was still being served from some endpoints, and added a second corrective-fix timeline.
- Netlock — Netlock said it limited Related Incidents to its own operations and described plans for an internal knowledge base.
- Apple representative — Apple said Netlock’s interpretation of Related Incidents was incompatible with the CCADB Incident Reporting Guidelines and requested a separate incident for that process non-compliance.
- Mozilla representative — Mozilla reiterated that Netlock should review public incident documentation from other CAs and file a separate incident report for the reporting non-compliance.
- Netlock — Netlock said it had opened the corresponding ticket to analyze and address the incident-reporting non-compliance referenced by the community.
- Netlock — Netlock posted updated action items and added a list of related incidents from other bugs.
- Netlock — Netlock explained that the remediation required a new software release because the affected URLs were generated by an integrated publishing component.
- Netlock — Netlock said testing had identified an implementation defect, that it had been corrected, and that re-testing was underway.
- Netlock — Netlock re-baselined its remaining action items, marking the additional corrective fix and automated validation as in progress with updated dates still to be determined.
- Netlock — Netlock said the corrected implementation remained under validation and preventive measures were progressing in parallel.
- Netlock — Netlock said no new blocking issues had been identified and that validation was ongoing.
- Netlock — Netlock said testing and verification continued and updated target dates would follow after validation and release planning.
- Netlock — Netlock said the remaining action items were still in progress and validation continued.
- Netlock — Netlock said there were no material changes and that validation and preventive actions remained underway.
- Netlock — Netlock said validation and release preparation for the remaining action items were ongoing.
- Netlock — Netlock said the additional corrective fix, automated AIA validation, and auditor-scope clarification were still in progress.
- Netlock — Netlock said no additional action item had been completed and that revised target dates for the remaining open items were still being finalized.
- Netlock — Netlock said another round of testing found no new defect, that remaining action items were still in progress, and that updated target dates and closure evidence were being prepared.
- Netlock — Netlock said no new issues had surfaced, the remaining action items were still in progress, and no additional action item had been completed during the latest reporting period.
- Netlock — Netlock reported that auditor-scope clarification was completed on 2026-07-15 and set 2026-08-13 target dates for the additional corrective fix and automated AIA validation.
- Netlock — Netlock said the additional corrective fix and automated AIA validation remained in progress, that both still targeted 2026-08-13, and that verification evidence and a closure summary were being prepared.
- Netlock — A Netlock representative posted an update intended for a different OCSP incident bug.
- Community commenter — A commenter said the 2026-08-10 update appeared to have been posted to the wrong bug and pointed to bug 2051459.
- Netlock — Netlock apologized and said the prior comment was posted in error, belongs to bug 2051459, and should be disregarded in this case.
- Netlock — Netlock said the 2026-08-13 targets passed without a report, disclosed that some legacy-host AIA paths still returned PEM, and set new target dates of 2026-10-01 for endpoint correction or retirement and 2026-10-15 for rebuilding validation coverage.
- Community commenter — A commenter asked where Netlock’s promised assessment or new incident was and noted that Netlock had missed both the seven-day response expectation and its own update commitment.
- Netlock — Netlock said the PEM encoding defect was resolved, reported zero PEM responses across 97 `ca=` values on five affected vhosts, corrected its earlier scope from two affected values to 12, and said missed update dates would be tracked as a recurrence under bug 2052541.
- Netlock — Netlock said the fix reported on 2026-10-02 had held, acknowledged another missed promised update, and said an updated full incident report would be filed by 2026-10-14 12:00 UTC.
- Netlock — Netlock said it missed the 2026-10-14 deadline for the updated full incident report, set a new date of 2026-10-21 12:00 UTC, and said repeated checks still showed DER responses for all 12 affected values.