To end up in this post, you probably are in some way involved with an entity that has a web presence, either that be a website, a service or a product. This entity of yours needs to be discovered by your clients/users through the World Wide Web and most of the times using a web browser directly (that is also true for mobile devices). To make this happen you took your path together with your team and designed and developed a kind of web program that aims to make the world a better place.

At some point depending on your role, you came across the SSL or HTTPS thing that hopefully people said is a must for all websites independently of purpose and content. They are right in what they said and you should believe them.

What happened after that is, either with an automatic commercial setup or a hosting service cross-sell, you purchased the magic certificate that guarantees safety, freedom from Google's restless eye and a, not so stylish now, lock badge on the visitor's address bar 🔒

What Can Go Wrong ?

When you bought your certificate and the process has gone smoothly, your server gained the capability to encrypt between your site and your visitors. This certificate is also used to authenticate that you are capable and authorized to claim this security standard.

This certification is not printed once and valid forever though. The authority organization behind these certificates have decided an expiration date of 2 years maximum. As browser vendors really care about serving secure information around the web in the case that your certificate is expired, or has any kind of "issue", your web visitors will be prompted by the dreaded screen of a faulty SSL certificate.

Yes, this is the screen layout that the browser will prompt to your users. In some cases they will need to explicitly state they want to enter this web space or in some cases they are not allowed at all. What could that translate to ?

By now you are probably are shaken enough about not forgetting to renew your SSL certificate but also to detect issues with it as fast as possible. One thing you can do now is send a message to your webmaster to check on the certificate expiration date. The next best thing to do is to schedule a regular check about when is the required time to renew the certificate and also monitor incidents similar in nature.

Can we automate that ?
We sure can!

Here Comes Web Automation

You decided to automate this process of checking your certificate. Good for you and web automation tools have you covered. To showcase how we can do that we will be using Puppeteer as it has the utility of providing that kind information out of the box, but probably any CDP capable solution can do.

If you are not sure how to start of with Puppeteer, you can have a look at Recipe #1 and come back.

Looking through the documentation, we can see that Puppeteer's API exposes directly information about the security details of a specific network response through the SecurityDetails interface. This interface exposes really neat information, like certificate issuer and the ending date of the certificate validity, that we will be using for our demonstration. Let's jump in!

Into The Code

After you have done your casual Puppeteer setup, the first thing to do is enable request interception for the newly created Page object.

await page.setRequestInterception(true);

With that, you have now gained access to some page events that will allow you to listen to the request/response cycle of every request on the pages you navigate.

// We do not need to do anything on the Request event, just let it move forward
page.on("request", (request) => request.continue());

page.on("response", (response) => {
  // Here the magic will occur
});

await page.goto("https://www.thehomeofwebautomation.com/")

The request that we need to cater for in our case is the initial DOC type request that is done to fetch the first HTML content of our page. That request carries all the information about the certificate we own.

/* Inside the response handler */
const contentType = response.headers()["content-type"]; // Get the content-type of the response

if (contentType.match("text/html")) { // Check for an HTML specific response
  /* Retrieve the security details */
  const securityDetails = response.securityDetails();
  
  /* Authority that issued the certificate */
  const certificateIssuer = securityDetails.issuer();

  /* SecurityDetails.validTo() returns a Unix Timestamp so we need to convert it */
  const validToDate = new Date(securityDetails.validTo() * 1000);

  /* ... */
}

Run this code and log some things on the console to get a fill for the data you can retrieve! For the next part we would aim to notify the responsible individuals on an upcoming certificate expiration date, so that they can take action accordingly.

/* Continue inside the conditional */
const diffInDays = computeDateDiffInDays(new Date(), validToDate); // Calculate the difference in days

if(diffInDays < 90){ // If the expiry is in less than 90 days
  notify(diffInDays, certificateIssuer); // Send a notification
}

/* The notify function can be something like */
function notify(daysRemaining, message, whoToContact = "some channel or some email address"){
  // sendSlackNotification(...) https://blog.nodeswat.com/simple-node-js-and-slack-webhook-integration-d87c95aa9600
  // sendMailToServiceManagement(...) https://blog.mailtrap.io/sending-emails-with-nodemailer/
}

With that, we are mostly done! What I would advise you to do is schedule this as a job to run every day and leave it running for one or for all the domains you own and take care of.

Some Additional Checks

In many scenarios, there more than one requests that responds with HTML content (fitting the content-type conditional), but for our needs, the first one will suffice. What you can do is introduce a simple boolean flag.

/* Higher scope from the response handler */
let initialHtmlFound = false;

/* Inside the response handler */
if (contentType.match("text/html") && !initialHtmlFound) {
  // ...
  initialHtmlFound = true;
}

if (contentType.match("text/html") && !initialHtmlFound) {
 /* You should also check for possible redirects using response.status() >= 300 && response.status() < 400 */
}

There are different kinds of SSL errors that might come up and they would throw an error right at the navigation step. To be on the safe side you can also take care of those.

// Add this simple check for the rest of the certificate error cases
await page.goto("https://expired.badssl.com/").catch(err => {
  if(err.message.match("net::ERR_CERT_")){
    notify(0, "Now we need to worry")
  }
  // Do whatever you want here
});

The final special case that I would mention here is the possibility of the SecurityDetails being null. When does this happen ? Most often when we intercept a response that does not have any security details to expose. That can happen if you decide to navigate to a page that has no SSL certificate at all e.g. http://example.com

Now you are really ready!

Side Note

If For Some Reason you have lost the plot at some point and a charlatan convinced you that you do not need an SSL certificate for your so special case, please let the next thing you do today is go to any vendor (Let's Encrypt is free) and just set up the goddamn thing. If you do not believe us here, I will leave the experts do the talking and Mr. Troy Hunt is the real deal.

Closing

Thanks for reading this recipe and I hope your learned one or two useful things for your web automation efforts. Stay safe on the web and do not allow your users to see one of these embarrassing screens again.