We made a stuff-up over the weekend. Historically we have different SSL certificates for different services in our realm, and last Saturday the certificate for the main website expired. Of course we noticed at that point, but we should have had an internal notification earlier and somehow that had failed. Fixed, but it would have been much better if the front-end hadn’t been temporarily inaccessible. It was, because of HTTPS Strict Transport Security (HSTS). Any browser that had previously talked with our website (rightfully) refuses to talk to it if it doesn’t see a valid certificate. Going back to non-HTTPS is not an option either, for this reason as well as others mentioned below. However, we do have different certificates for different services, so it was only our frontend website that was affected (bad enough), the various other services for our clients fortunately were unaffected.
Going forward though, keeping up-to-date with the certificates and automatically renewing them is much easier now than it used to be. Let’s Encrypt® has been around for a while, but a few months ago they started supporting wildcard certificates. With non-wildcard, one of the ways Let’s Encrypt can verify that you own the site is by doing a challenge/response on port 443 of the website address; Certbot will temporarily listen there and give the appropriate answers. For a wildcard, that doesn’t work, because you can have an infinite number of subdomains and Let’s Encrypt needs to be certain that you actually control these. So in v2 of the API there’s support for DNS based validation. Through special TXT records for which Let’s Encrypt provides the token on every domain request, you can prove that you are in control of the DNS servers for the domain. That’s pretty slick!
There are integrations for many hosting providers as well as Cloudflare, which through a secure mechanism allow Let’s Encrypt to update those records in your DNS, and then validate. As Let’s Encrypt certificates are only valid for 3 months, this is important for the automation. If you run your own DNS servers, you can still automate the DNS based verification process by setting up RFC-2136 remote updates in your DNS server (Bind9 can do it, it’s been around for many years – that said, being an older system, it can be rather finicky to set up).
Let’s Encrypt’s Certbot can take care of the entire updating process, including reloading your webserver’s or reverse proxy’s config. Do make sure you use a recent Certbot, as all the appropriate support is quite recent. We had to grab Certbot from Github the first time as the Debian release hadn’t updated quite far enough yet – it has now.
We think that the EFF has done brilliantly with setting up Let’s Encrypt, and helping everyone move towards a fully encrypted web. Particularly with the cost-factor removed, there’s no reason to not offer HTTPS to users – whether for a website, or an API. Respecting one’s users and their online privacy is really a must-do. Companies that don’t, increasingly look bad. See it this way: going fully HTTPS is an opportunity to make a good first impression. And did you know it also affects your ranking in search engines? Now there’s a good incentive for the PHB…
Do you need an EV certificate? Probably not, as they actually have very little meaning – and even less so with various CAs having distinctly flawed verification processes.
Do you need a site seal from your CA (Certificate Authority)? Really not. It just advertises the CA, and actually enables them to track your users – if you get the seal served from the CA’s URL, that’s every single user. Not cool. So just don’t.
Final hint: if you do get a wildcard certificate from Let’s Encrypt, make sure you include both the wildcard and non-wildcard in the certificate domain names, otherwise it won’t work. So, *.example.com as well as example.com. You may not have noticed that your wildcard certificate always contains these, as many CAs automatically include the appropriate extra item. Certbot just does exactly what you tell it to, so it’s something to be aware of.