Tag: SaaS

Practical SaaS Security Assessment Checklist for 2024

Håkon OlsenLeave a comment

We all use web application for a lot of the business computing we do. That means that we need to care about the security of the applications we use, but this is not always so easy to assess. The traditional approach with sending long security questionnaires won’t get you very far. That’s why I developed a practical checklist approach described below – and there’s a template too for subscribers to this blog!

In 2021 Daniel Miessler had a great blog post on the failings of security questionnaires, and what to do instead, that I also commented on this blog: Vendor Security Management: how to decide if tech is safe (enough) to use. The essence of that thinking is that questionnaires won’t help much, and we should instead worry about whether there is a security program in place, and how they handled the last breach. We can take that though one step further, and create a practical assessment process for SaaS apps we are considering using. The great thing about SaaS apps is we get to test some of the security by using the tech, not only readying claims from others.

By using a checklist and giving it some scores based on security controls we think should be in place, we get a practical approach to assess the security. This won’t give you a complete answer, but it will relatively quickly give you a way to sort the bad from the potentially good.

google sheet with security assessment

The way we built this checklist is by dividing our checks into 6 categories. We could have used more, and it is a good idea to tailor the controls you check to what’s important for you. In this example we have used the following categories:

  • Identity: most breaches happen at the user account level. This is important.
  • Integrations: API keys leaking and kneeling applications due to DDoS are not fun. Do some checks.
  • Backups: You definitely want backups.
  • Data protection: how do you make sure other SaaS users can’t access your data? And what about the SaaS provider?
  • Logging: if things go wrong, you want to be able of seeing that. If you are blind, you have no security. Logs are critical.
  • Privacy: not only a legal issue, it is also important for everyone using the app. Colleagues and customers alike.

Let’s take a look at the identity checklist. I have organized each checklist with just a few checkpoints I find important into different sheets in a Google Sheet.

Identity checklist

For each line there is a checkpoint, some guidance on how to check, and a dropdown where you can choose the rating “good, weak, or bad”. You can also set it to “not applicable” if you think for some reason that a particular control is not interesting for the current use case. There is also a cell to jot down some notes about your assessment. Below the table I have added some extra assessment advice to make it easier for the user to evaluate what’s more important in the checklist.

For each category, an overall score as a percentage. I don’t think you should use this as a hard threshold but low scores are worse than high scores. I used the following point scale to calculate the overall score: 

SCORE = -(number of bad items) + 0.5 x (number of weak items) + (number of good items) / (number of applicable items)

This is not a scientific formula, but it seems to give reasonable spread of the scores. The score is punished by bad results, you get a little bit of credit for weak results, and the “best score” is still 100%.

The Google sheet is free to anyone subscribing to this blog – enjoy 🙂

For subscribers: here’s the checklist: Free SaaS security evaluation template.

Do you consider security when buying a SaaS subscription?

Håkon Olsen2 Comments

tl;dr;  SaaS apps often have poor security. Before deciding to use one do a quick security review. Read privacy statements, ask for security docs, and test authentication practices, crypto and console.log information leaks before deciding if you want to trust the app or not. This post gives you a handy checklist to breeze through your SaaS pre-trial security review.

Over the last year I’ve been involved in buying SaaS access for lots of services in a corporate setting. My stake in the evaluation has been security. Of course, we can’t do active attacks or the like to evaluate if we are going to buy a software, but we can read privacy statements, look for strange things in the HTML and test their login process if they allow you to set up a free account or a trial account (most of them do). Here’s what I’ve generally found of challenges:

  • Big name players know what they are doing, but they have so many options for setup that you need to be careful what you select, especially if you have specific compliance needs.
  • Smaller firms (including startups)  don’t offer a lot of documentation, and quite often when talking to them they don’t really have much in place in terms of security management (they may still make great software, it is just harder to trust it in an enterprise environment)
  • Authentication blunders are very common. From HR support systems to payroll to IT management tools, we’ve found a lot of bad practices:
    • Ineffective password policies (5 digits and digits only, anyone?)
    • A theoretical password policy in place but validation is only performed client-side (that means it is easy to trick the server into setting a very weak password, or even no password in same cases, if you should so desire)
    • Lack of cookie security for session cookies (missing HTTPOnly and Secure flags, allowing for cookie theft by XSS attacks or man-in-the-middle attacks)
    • Poor password reset processes. In one case I found the supposedly random string used for a password reset link to be a base 64 encoding of my username…. how hard is that to abuse?
  • When you ask about development practices and security testing, many firms will lack such processes but try to explain that they implicitly have them because their developers are so smart (even with the authentication blunders mentioned above).

Obviously, at some point, security will be so bad that you have to say “No, we cannot buy this shiny thing, because it is rotten on the inside”. This is not a very popular decision in some cases, because the person requesting the service probably has some reason to request it.

egg power fear hammer
Don’t put your data and business at risk by using a SaaS product you can’t trust. By doing a simple due dilligence job you can at least identify services you definitely should be avoiding!

In order to evaluate SaaS security without spending too much time on it I’ve come up with a process I find works pretty well. So, here’s a simple way to sort the terrible from the average!

  1. Read the privacy statement and the terms and conditions. Not the whole thing, that is what lawyers are for (if you have some), but scan it and look for anything security related. Usually they will try to explain how they protect your personal data. If it is a clear and understandable explanation, they usually know what they are doing. If not, they usually don’t.
  2. Look for security or compliance documentation, or request it from their sales/support team. Specific questions to consider are:
    1. Do they offer encryption at rest (if this is reasonable to expect)?
    2. Do they explain how they store passwords?
    3. Do they use trustworthy data centers, and have some form of compliance proof for good practice for their data centers? SOC-2 reports, ISO certificates, etc?
    4. Do they guarantee limited access for insiders?
    5. Do they explain what cryptographic controls they are using for TLS, signatures and at-rest encryption? That means how they perform key management and exchange, what ciphers they allow, and the key strength they require?
    6. Do they say anything about vulnerability management and security testing?
    7. Do they say anything about incident handling?
  3. Perform some super-lightweight testing. Don’t break the law but do your own due diligence on the app:
    1. Create a free account if possible, and test if the authentication practices seem sound:
      • Test the “I forgot my password functionality” to see if the password reset link has an expiry time, and if it is a unguessable” link
      • Try changing your password to something really bad, like “password” or “dog”. Try to replay post requests if stopped by client-side validation (this can be done directly from the dev tools in Firefox, no hacker tool necessary)
      • Try logging in many times with the wrong password to see what happens (warning, lockout,… or most likely, nothing)
    2. Check their website certificate practices by running sslscan, or by using the online version at ssllabs.com. If they support really weak ciphers, it is an indicator that they are not on the ball. If they support SSL (pre TLS 1.0) it is probably a completely outdated service.
    3. Check for client-side libraries with known vulnerabilities. If they are using ancient versions of jQuery and other client side libraries, they are likely at risk of being hacked. This goes for WordPress sites as well, including their plugins.
    4. Check for information leaks by opening the console (Ctrl + I in most browsers). If the console logs a lot of internal information coming from the backend, they probably don’t have the best security practices.

So, should you dump a service if any of these tests “fail”? Probably not. Most sites have some weak points, and sometimes that is a tradeoff for some reason that may be perfectly legitimate. But if there are a lot of these “bad signs” in the air and you would be running some critical process or storing your company secrets in the app, you will be better off finding another service to suit your needs – or at least you will be able to sleep better at night.