NETLOCK: CRL Error on CRL Watch of NETLOCK DVCA CRL

This case reports a CRL error detected via CRL watch for NETLOCK DVCA CRL. Netlock said it became aware of the problem after receiving an email from Ben Wilson on 2023-07-12 and began investigation the same day. Netlock reported that, in response to a client request, it generated a new CRL and copied it to the application server by hand, but the colleague copying the file did not set the file rights correctly, which prevented the automatic task from replacing the file when the new CRL was generated. Netlock also stated that while the monitoring system detected the issue, the alert rule contained a misspelling, which meant it did not receive the notification. As remediation, Netlock introduced an extra internal procedure step to check and set file rights correctly, notified colleagues, corrected the internal monitoring misconfigurations, and continued expanding monitoring to use external resources. Netlock later updated that it implemented an external monitoring tool (UptimeRobot) to check crlwatch, ocspwatch, and crt.sh data and send push notifications, and the bug was set to RESOLVED with resolution FIXED.

1. 2023-07-12 Netlock received an email about a CRL watch error and started investigating the CRL issue.
2. 2023-07-13 Netlock identified the root cause and fixed the problem, and began a problem review.
3. 2023-07-18 Netlock provided an update that it was testing third-party monitoring solutions and developing additional monitoring modules.
4. 2023-07-28 Netlock reported implementation of an external monitoring tool (UptimeRobot) for faster alerts.
5. 2023-09-29 Mozilla closed the case after no further questions were raised.

1. 2023-07-12 Netlock — Horváth opened an initial bug report and said a full incident report would be submitted by end of day Friday.
2. 2023-07-14 Netlock — Banyai submitted an incident report describing the CRL generation/copying issue, the file-rights problem, and that an alert rule misspelling prevented notification; they also described remediation steps including internal procedure changes and monitoring corrections.
3. 2023-07-14 Mozilla representative — Ben asked whether there was a timeline for remaining remediation tasks.
4. 2023-07-18 Netlock — Horváth responded with a timeline for testing and deploying external monitoring solutions and module development.
5. 2023-07-18 Mozilla representative — Ben thanked Horváth for the update.
6. 2023-07-28 Netlock — Horváth reported that Netlock implemented an external monitoring tool (UptimeRobot) checking crlwatch, ocspwatch, and crt.sh data with push notifications.
7. 2023-09-26 Mozilla representative — Ben stated he intended to close the bug on 2023-09-29 if there were no additional questions.