SwissSign: OCSP responder unreachable

The case reports that SwissSign’s OCSP responder became unreachable, with internal monitoring detecting that the OCSP service was down. SwissSign’s initial investigation started after the OCSP outage, and IT Ops identified an invalid BGP configuration that had been in use. SwissSign began reconfiguring BGP and routing, which restored routing and brought the OCSP service back up, but it then went down again due to a hardware failure on external firewalls. SwissSign later reported that it confirmed a massive DDoS attack and worked with its ISPs to defend against it, including deploying rules to prevent the DDoS attack patterns. The thread includes further updates that OCSP was down again due to DDoS during the night, and that additional external DDoS protection by Akamai was implemented to protect the OCSP service. SwissSign reported that Akamai protection was performing well and that heavy attacks did not have significant impact, and Mozilla proposed closing the bug if there were no further questions. The bug is marked RESOLVED with resolution FIXED.

1. 2020-08-31 SwissSign’s OCSP responder became unreachable and the service went down.
2. 2020-08-31 SwissSign reconfigured BGP/routing, restoring OCSP service, followed by another outage due to external firewall hardware failure.
3. 2020-09-01 SwissSign identified a massive DDoS attack and worked with ISPs to deploy mitigation measures.
4. 2020-09-02 OCSP experienced instability and outages during ongoing heavy attacks.
5. 2020-09-03 SwissSign implemented additional external DDoS protection via Akamai to protect OCSP.
6. 2020-09-10 SwissSign reported Akamai protection was performing well and requested closure if no questions remained.

1. 2020-08-31 SwissSign AG — Reported that internal monitoring detected the OCSP responder was not reachable and provided a timeline including BGP configuration issues, reconfiguration, and subsequent outages due to hardware failure.
2. 2020-09-01 SwissSign AG — Updated the timeline, stated OCSP was up with minor issues, confirmed a massive DDoS attack, and described ISP coordination and DDoS mitigation rules.
3. 2020-09-04 SwissSign AG — Provided further updates that OCSP was down again due to DDoS, later up but not stable, and that Akamai-based protection was implemented.
4. 2020-09-10 SwissSign AG — Reported Akamai protection was performing well, with heavy attacks not having significant impact, and asked to close the ticket if there were no questions.
5. 2020-09-10 Mozilla representative — Asked whether there were additional questions and proposed closing the bug on or about 14-Sept-2020.