We all use web application for a lot of the business computing we do. That means that we need to care about the security of the applications we use, but this is not always so easy to assess. The traditional approach with sending long security questionnaires won’t get you very far. That’s why I developed a practical checklist approach described below – and there’s a template too for subscribers to this blog!
In 2021 Daniel Miessler had a great blog post on the failings of security questionnaires, and what to do instead, that I also commented on this blog: Vendor Security Management: how to decide if tech is safe (enough) to use. The essence of that thinking is that questionnaires won’t help much, and we should instead worry about whether there is a security program in place, and how they handled the last breach. We can take that though one step further, and create a practical assessment process for SaaS apps we are considering using. The great thing about SaaS apps is we get to test some of the security by using the tech, not only readying claims from others.
By using a checklist and giving it some scores based on security controls we think should be in place, we get a practical approach to assess the security. This won’t give you a complete answer, but it will relatively quickly give you a way to sort the bad from the potentially good.
The way we built this checklist is by dividing our checks into 6 categories. We could have used more, and it is a good idea to tailor the controls you check to what’s important for you. In this example we have used the following categories:
- Identity: most breaches happen at the user account level. This is important.
- Integrations: API keys leaking and kneeling applications due to DDoS are not fun. Do some checks.
- Backups: You definitely want backups.
- Data protection: how do you make sure other SaaS users can’t access your data? And what about the SaaS provider?
- Logging: if things go wrong, you want to be able of seeing that. If you are blind, you have no security. Logs are critical.
- Privacy: not only a legal issue, it is also important for everyone using the app. Colleagues and customers alike.
Let’s take a look at the identity checklist. I have organized each checklist with just a few checkpoints I find important into different sheets in a Google Sheet.
For each line there is a checkpoint, some guidance on how to check, and a dropdown where you can choose the rating “good, weak, or bad”. You can also set it to “not applicable” if you think for some reason that a particular control is not interesting for the current use case. There is also a cell to jot down some notes about your assessment. Below the table I have added some extra assessment advice to make it easier for the user to evaluate what’s more important in the checklist.
For each category, an overall score as a percentage. I don’t think you should use this as a hard threshold but low scores are worse than high scores. I used the following point scale to calculate the overall score:
SCORE = -(number of bad items) + 0.5 x (number of weak items) + (number of good items) / (number of applicable items)This is not a scientific formula, but it seems to give reasonable spread of the scores. The score is punished by bad results, you get a little bit of credit for weak results, and the “best score” is still 100%.
The Google sheet is free to anyone subscribing to this blog – enjoy 🙂
For subscribers: here’s the checklist: Free SaaS security evaluation template.
safecontrols 