Two-factor auth for your Node project, without tears

  • How 2fa works with TOTP
  • Code
  • Adding bells and whistles
  • How secure is OTP really?

How 2fa works with TOTP

TOTP is short for time-based one-time password. This is a commonly used method for a second factor in two-factor authentication. The normal login flow for the user is then: first log in with username and password. Then you are presented with a second login form, where you have to enter a one-time password. Typically you will have an app like Google Authenticator or Microsoft Authenticator that will provide you with a one-time code that you can enter. If you have the right code, you will be authenticated and you gain access.

How does the web service know what the right one-time code is?

Consider a web application using two-factor authentication. The user has Google Authenticator on his or her phone to provide one-time passwords – but how does the app know what to compare with?

That comes from the setup: there is a pre-shared secret that is used to generate the tokens based on the current time. These tokens are valid for a limited time (typically 30, 60 or 120 seconds). The time here is “Unix time” – the number of seconds since midnight 1 January 1970. TOTP is a special case for a counter-based token created with the HOTP algorithm (HOTP = HMAC based one-time password). Both of these are described in lengthy detail in RFC documents, and the one for TOTP is RFC 6238. The main point is: both token generator (phone) and validator (web server) needs to know the current unix time and they need a pre-shared secret. Then the token can be calculated a function of the time and this secret.

A one-time password used for two-factor authentication is a function of the current time and a pre-shared key.

Details in RFC 6238
The basics of multi-factor authentication: something you know + something you have

How do I get this into my app?

Thanks to open source libraries, creating a 2fa flow for your app is not hard. Here’s an example made on Glitch for a NodeJS app: https://aerial-reward.glitch.me/login.

The source code for the example is available here: https://glitch.com/edit/#!/aerial-reward. We will go through the main steps in the code to make it easy to understand what we are doing.

Step 1: Choose a library for 2FA.

Open source libraries are great – but they also come with a risk. They may contain vulnerabilities or backdoors. Doing some due diligence up front is probably a good idea. In this case we chose speakeasy because it is popular and well-documented, and running npm audit does not show any vulnerabilities for the library although it hasn’t been updated in 4 years.

Step 2: Activate MFA using a QR code for the user

We assume you have created a user database, and that you have implemented username- and password based login (in a safe way). Now to the MFA part – how can we share the pre-shared secret with the device used to generate the token? This is what we use a QR code for. The user will then log in, be directed to a profile page where “Activate MFA” is an option. Clicking this link shows a QR code that can be scanned with an authenticator app. This shares the pre-shared key with the app. Hence: the QR code is sensitive data, so it should only be available when setting up the app, and not be stored permanently. The user should also be authenticated in order to see the QR code (using username and password).

In our example app, here’s the route for activating MFA after logging in.

app.get('/mfa/activate', (req, res) => {
  if (req.session.isauth) {
    var secret = speakeasy.generateSecret({name: 'Aerial Reward Demo'})
    req.session.mfasecret_temp = secret.base32;
    QRCode.toDataURL(secret.otpauth_url, function(err, data_url) {
      if (err) {
        res.render('profile', {uname: req.session.username, mfa: req.session.mfa, qrcode: '', msg: 'Could not get MFA QR code.', showqr: true})
      } else {
        console.log(data_url);
        // Display this data URL to the user in an <img&gt; tag
        res.render('profile', {uname: req.session.username, mfa: req.session.mfa, qrcode: data_url, msg: 'Please scan with your authenticator app', showqr: true}) 
      }
    })
  } else {
    res.redirect('/login')
  }
})

What this does is the following:

  • Check that the user is authenticated using a session variable (set on login)
  • Create a temporary secret and store as a session variable, using the speakeasy library. This is our pre-shared key. We won’t store it in the user profile before having verified that the setup worked, to avoid locking out the user.
  • Generate a QRCode with the secret. To do this, you need to use a qrcode library, and we used the one qrcode, which seems to do the job OK. The speakeasy library generates an otpauth_url that can be used in the QR code. This otpauth_url contains the pre-shared secret.
  • Finally we are rending a template (the profile page for the user) and supplying the QR code data url to a template (res.render).

For rendering this to the end user we are using a Pug template.

html
  head
    title Login
    link(rel="stylesheet" href="/style.css")
  body
    a(href="/logout") Log out
    br
    h1 Profile for #{uname}
    p MFA: #{mfa}
    unless mfa
      br
      a(href="/mfa/activate") Activate multi-factor authentication
    if showqr
      p= msg
      img(src=qrcode)
      p  When you have added the code to your app, verify that it works here to activate.
      a(href="/mfa/verify") VERIFY MFA CODE
    if mfa
      img(src="https://media.giphy.com/media/81xwEHX23zhvy/giphy.gif")
      p Security is important. Thank you for using MFA!

The QR code is shown in the profile when the right route is used and the user is not already using MFA. This presents the user with a QR code to scan, and then he or she will need to enter a correct OTP code to verify that the setup works. Then we will save the TOTP secret in the user profile.

How it looks for the user

The profile page for the user “safecontrols” with the QR code embedding the secret.

Scanning the QR code on an authenticator app (many to choose from, FreeOTP from Red Hat is a good alternative), gives you OTP tokens. Now the user needs to verify by entering the OTP. Clicking the link “VERIFY MFA CODE” to do this brings up the challenge. Entering the code verifies that you have your phone. When setting things up, the verification will store the secret “permanently” in your user profile.

How do I verify the token then?

We created a route to verify OTP’s. The behavior depends on whether MFA has been set up yet or not.

app.post('/mfa/verify', (req, res) => {
  // Check that the user is authenticated
  var otp = req.body.otp
  if (req.session.isauth && req.session.mfasecret_temp) {
    // OK, move on to verify 2fa activation
    var verified = speakeasy.totp.verifyDelta({
      secret: req.session.mfasecret_temp,
      encoding: 'base32',
      token: otp,
      window: 6
    })
    console.log('verified', verified)
    console.log(req.session.mfasecret_temp)
    console.log(otp)
    if (verified) {
      db.get('users').find({uname: req.session.username}).assign({mfasecret: req.session.mfasecret_temp}).write()
      req.session.mfa = true
      req.session.mfarequired = true
      res.redirect('/profile')
    } else {
      console.log('OTP verification failed during activation')
      res.redirect('/profile')
    }
  } else if (req.session.mfarequired) {
    // OK, normal verification
    console.log('MFA is required for user ', req.session.username)
    var verified = speakeasy.totp.verifyDelta({
      secret: req.session.mfasecret,
      encoding: 'base32',
      token: otp,
      window: 6
    })
    console.log(verified)
    if (verified) {
      req.session.mfa = true
      res.redirect('/profile')  
    } else {
      // we are pretty harsh, thrown out after one try
      req.session.destroy(() => {
        res.redirect('/login')
      })
    }
  } else {
    // Not a valid 2fa challenge situation
    console.log('User is not properly authenticated')
    res.redirect('/')
  }
})

The first path is for the situation where MFA has not yet been set up (this is the activation step). This is checked that the user is authenticated and that there is a temporary secret stored in a session variable. This happens when the user clicks the “VERIFY…” link on the profile page after scanning the QR code, so this session variable will not be available in other cases.

The second path checks if there is a session variable mfarequired set to true. This happens when the user authenticates, if an MFA secret has been stored in the user profile.

The verification itself is done the speakeasy library functions. Note that you can use speakeasy.totp.verify (Boolean) or speakeasy.totp.verifyDelta (gives a time delta). The former did not work for some reason, whereas the Delta version did, which is the only reason for this choice in this app.

How secure is this then?

Nothing is unhackable, and this is no exception to that rule. The security of the OTP flow depends on your settings, as well as other defense mechanisms. How can hackers bypass this?

  • Stealing tokens (man-in-the-middle or stealing the phone)
  • Phishing with fast use of tokens
  • Brute-forcing codes has been reported as a possible attack on OTP’s but this depends on configuration

These are real attacks that can happen, so how to protect against them?

  • Always use https. Never transfer tokens over insecure connections. This protects against man-in-the-middle.
  • Phishing: this is more difficult, if someone obtains your password and a valid token and can use them on the real page before the token expires, they will get in. Using meta-data to calculate a risk-score can help: sign-in from new device requires confirmation by clicking a link sent in email, force password reset after x failed logins, etc. None of that is implemented here. That being said, OTP-based 2FA protects against most phishing attacks – but if you are a high-value target for organized crime of professional spies you probably should think about more secure patterns. Alternatives include push notifications or hardware tokens that avoid typing something into a form.
  • Brute-force: trying many OTP’s until you get it right is possible if the “window” is too long for when a code is considered valid, and you are not logged out after trying 1 or more wrong codes. In the code above the window parameter is set to 6, which is very long and potentially insecure, but the user is logged out if the OTP challenge fails, so brute-force is still not possible.

Keeping your conversations private in the age of supervised machine learning and government snooping

Most of us would like to keep our conversations with other people private, even when we are not discussing anything secret. That the person behind you on the bus can hear you discussing last night’s football game with a friend is perhaps not something that would make you feel uneasy, but what if employees, or outsourced consultants, from a big tech firm are listening in? Or government agencies are recording your conversations and using data mining techniques to flag them for analyst review if you mention something that triggers a red flag? That would certainly be unpleasant to most of us. The problem is, this is no longer science fiction.

You are being watched.

Tech firms listening in

Tech firms are using machine learning to create good consumer products – like voice messaging that allows direct translation, or digital assistants that need to understand what you are asking of them. The problem is that such technologies cannot learn entirely by themselves, so your conversations are being recorded. And listened too.

Microsoft: https://www.vice.com/en_us/article/xweqbq/microsoft-contractors-listen-to-skype-calls

Google: https://www.theverge.com/2019/7/11/20690020/google-assistant-home-human-contractors-listening-recordings-vrt-nws

Amazon: https://www.bloomberg.com/news/articles/2019-04-10/is-anyone-listening-to-you-on-alexa-a-global-team-reviews-audio

Apple: https://www.theguardian.com/technology/2019/jul/26/apple-contractors-regularly-hear-confidential-details-on-siri-recordings

All of these systems are being listened in to in order to improve speech recognition, which is hard for machines. They need some help. The problem is that users have not generally been aware that they conversations or bedroom activities may be listened in to by contractors in some undisclosed location. It certainly doesn’t feel great.

That is probably not a big security problem for most people: it is unlikely that they can specifically target you as a person and listen in on everything you do. Technically, however, this could be possible. What if adversaries could bribe their way to listen in to the devices of decision makers? We already know that tech workers, especially contractors and those in the lower end of the pay scale, can be talked into taking a bribe (AT&T employee installing malware on company servers allowing unauthorized unlocking of phones (wired.com), Amazon investigating data leaks for bribe payments). If you can bribe employees to game the phone locking systems, you can probably manipulate them into subverting the machine learning QA systems too. Because of this, if you are a target of high-resource adversaries you probably should be skeptical about digital assistants and what you talk about around them.

Governments are snooping too

We kind of knew it already but not the extent of it. Then Snowden happened – confirming that governments are using massive surveillance program that will capture the communications of everyone and make it searchable. The NSA got heavily criticized for their invasive practices in the US but that did not stop such programs from being further developed, or the rest of the world to follow. Governments have powers to collect massive amounts of data and analyze it. Here’s a good summary of the current US state of phone record collection from Reuters: https://www.reuters.com/article/us-usa-cyber-surveillance/spy-agency-nsa-triples-collection-of-u-s-phone-records-official-report-idUSKBN1I52FR.

The rest of the world is likely not far behind, and governments are using laws to make collection lawful. The intent is the protection of democracy, freedom of speech, and the evergreen “stopping terrorists”. The only problem is that mass surveillance seems to be relatively inefficient at stopping terrorist attacks, and it has been found to have a chilling effect on freedom of speech and participation in democracy, and even stops people from seeking information online because they feel somebody is watching them. Jonathan Shaw wrote an interesting comment on this on Harvard Magazine in 2017: https://harvardmagazine.com/2017/01/the-watchers.

When surveillance makes people think “I feel uneasy researching this topic – what if I end up on some kind of watchlist?” before informing themselves, what happens to the way we engage, discuss and vote? Surveillance has some very obvious downsides for us all.

If an unspoken fear of being watched is stopping us from thinking the thoughts we otherwise would have had, this is a partial victory for extremists, for the enemies of democracy and for the planet as a whole. Putting further bounds on thoughts and exploration will also likely have a negative effect on creativity and our ability to find new solutions to big societal problems such as climate change, poverty and even religious extremism and political conflicts, the latter being the reason why we seem to accept such massive surveillance programs in the first place.

But isn’t GDPR fixing all this?

The GDPR is certainly a good thing for privacy but it has not fixed the problem. It does apply to the big tech firms and the adtech industry but it really hasn’t solved the problem, at least not yet. As documented in this post from Cybehave.no, privacy statements are still too long, too complex, and too hidden for people to care. We all just click “OK” and remain subject to the same advertising driven surveillance as before.

The other issue we have here is that the GDPR does not apply to national security related data collection. And for that sort of collection, the surveillance state is still growing with more advanced programs, more collection, and more sharing between intelligence partners. In 2018 we got the Australian addition with their rather unpleasant “Assist and access” act allowing for government mandated backdoors in software, and now the US wants to backdoor encrypted communications (again).

Blocking the watchers

It is not very difficult to block the watchers, at least not from advertisers, criminals and non-targeted collection (if a government agency really wants to spy on you as an individual, they will probably succeed). Here’s a quick list of things you can do to feel slightly less watched online:

  • Use an ad-blocker to keep tracking cookies and beacons at bay. uBlock origin is good.
  • Use a VPN service to keep your web traffic away from ISP’s and the access of your telephone company. Make sure you look closely at the practices of your VPN supplier before choosing one.
  • Use end-2-end encrypted messaging for your communications instead of regular phone conversations and text messages. Signal is a good choice until the US actually does introduce backdoor laws (hopefully that doesn’t happen).
  • Use encrypted email, or encrypt the message you are sending. Protonmail is a Swiss webmail alternative that has encryption built-in if you send email to other Protonmail users. It also allows you to encrypt messages to other email services with a password.

If you follow these practices it will generally be very hard to snoop on you.

Vacation’s over. The internet is still a dumpster fire.

This has been the first week back at work after 3 weeks of vacation. Vacation was mostly spent playing with the kids, relaxing on the beach and building a garden fence. Then Monday morning came and reality came back, demanding a solid dose of coffee.

  • Wave of phishing attacks. One of those led to a lightweight investigation finding the phishing site set up for credential capture on a hacked WordPress site (as usual). This time the hacked site was a Malaysian site set up to sell testosteron and doping products… and digging around on that site, a colleague of mine found the hackers’ uploaded webshell. A gem with lots of hacking batteries included.
  • Next task: due diligence of a SaaS vendor, testing password reset. Found out they are using Base64 encoded userID’s as “random tokens” for password reset – meaning it is possible to reset the password for any user. The vendor has been notified (they are hopefully working on it).
  • Surfing Facebook, there’s an ad for a productivity tool. Curious as I am I create an account, and by habit I try to set a very weak password (12345). The app accepts this. Logging in to a fancy app, I can then by forced browsing look at the data from all users. No authorization checks. And btw, there is no way to change your password, or reset it if you forget. This is a commercial product. Don’t forget to do some due diligence, people.

Phishing for credentials?

Phishing is a hacker’s workhorse, and for compromising an enterprise it is by far the most effective tool, especially if those firms are not using two-factor authentication. Phishing campaigns tend to come in bursts, and this needs to be handled by helpdesk or some other IT team. And with all the spam filters in the world, and regular awareness training, you can reduce the number of compromised accounts, but it is still going to succeed every single time. This is why the right solution to this is not to think that you can stop every malicious email or train every user to always be vigilant – the solution is primarily: multifactor authentication. Sure, it is possible to bypass many forms of it, but it is far more difficult to do than to just steal a username and a password.

Another good idea is to use a password manager. It will not offer to fill in passwords on sites that aren’t actually on the domain they pretend to be.

To secure against phishing, don’t rely on awareness training and spam filters only. Turn on 2FA and use a password manager for all passwords. #infosec

You do have a single sign-on solution, right?

Password reset gone wrong

The password reset thing was interesting. First on this app I registered an account with a Mailinator email account and the password “passw0rd”. Promising.. Then trying the “I forgot” on login to see if the password recovery flow was broken – and it really was in a very obvious way. Password reset links are typically sent by email. Here’s how it should work:

You are sent a one-time link to recover your password. The link should contain an unguessable token and should be disabled once clicked. The link should also expire after a certain time, for example one hour.

This one sent a link, that did not expire, and that would work several times in a row. And the unguessable token? Looked something like this: “MTAxMjM0”. Hm… that’s too short to really be a random sequence worth anything at all. Trying to identify if this is a hash or something encoded, the first thing we try is to decode from Base64 – and behold – we can a 6-digit number (101234 in this case, not the userID from this app). Creating a new account, and then doing the same reveals we get the next number (like 101235). In other words, using the reset link of the type /password/iforgot/token/MTAxMjM0, we can simply Base64 encode a sequence of numbers and reset the passwords for every user.

Was this a hobbyist app made by a hobbyist developer? No, it is an enterprise app used by big firms. Does it contain personal data? Oh, yes. They have been notified, and I’m waiting for feedback from them on how soon they will have deployed a fix.

Broken access control

The case with the non-random random reset token is an example of broken authentication. But before the week is over we also need an example of broken access control. Another web app, another dumpster fire. This was a post shared on social media that looked like an interesting product. I created an account. Password this time: 12345. It worked. Of course it did…

This time there is no password reset function to test, but I suspect if there had been one it wouldn’t have been better than the one just described above.

This app had a forced browsing vulnerability. It was a project tracking app. Logging in, and creating a project, I got an URL of the following kind: /project/52/dashboard. I changed 52 to 25 – and found the project goals of somebody planning an event in Brazil. With budgets and all. The developer has been notified.

Always check the security of the apps you would like to use. And always turn on maximum security on authentication (use a password manager, use 2FA everywhere). Don’t get pwnd. #infosec

Securing media stored in cloud storage buckets against unauthorised access

Insecure direct object reference (IDOR) is a common type of vulnerability online. Normally we think of this as a vulnerable parameter in a URL or a form that allows forced browsing, but file downloads can also be an issue here. For a general background on IDOR and how to secure against it, see this cheatsheet from OWASP.

Our case is a bit different. Consider storing files in a cloud storage bucket (Google Cloud Storage, Amazon S3, etc). This may be for a file sharing site for example, where users are allowed to upload documents that are then stored in a bucket. We only want the users with the right authorisation to have access to these files. What are our options?

  1. Use cloud identity management and bucket security rules to manage access. This may be impractical as we don’t necessarily want to give app users IAM users in the cloud environment, but where applicable it is a direct solution to our little security problem.
  2. Allow full access to the bucket from the app and manage user permissions in the app.
  3. Make the object public but use non-descriptive and random filenames so unauthorised users cannot easily guess the right path. Maintain the link to contextual data in the backend code to not expose it publicly.
  4. Same as 3 but with a signed URL – a temporary ‘secret’ URL where permissions can be controlled without creating specific IAM users.

Google has made a list of best practices for cloud storage here. In our use case we want the shared object to have permanent permissions. Let us consider how to achieve acceptable security using option 2.

A simple architecture for sharing files securely

For this set-up there are a few things we need to take care of:

  1. For uploaded files do not expose the actual bucket meta data or file names to the user in the frontend. Create a reference in the database that maps to the object name in the bucket
  2. Manage access to objects through the database references, for example by adding a “shared with” key containing user ID’s for all users who are going to have read access to the object.
  3. Do not make the object publicly accessible. Instead use a service account IAM user for the application and allow the permissions you need. Download content to the app, and relay this to the frontend using the mapping described above to avoid exposing the actual object name.

What are the threat vectors to this method for securing shared files?

This is a relatively simple setup that avoids making a bucket, or objects in that bucket, publicly available. It is still possible to exploit to gain unauthorised access but this is no longer as easy as finding an unsecured bucket.

Identity spoofing: a hacker can take on the identity of a user of the application, and thus get access to the files this user has access to. To avoid this, make sure to follow good practices for authentication (strong passwords, two-factor authentication). Also keep identity secrets on the client side hard to get at by securing the frontend against cross-site scripting (XSS), turning on security headers and setting parameters on cookies to avoid easy exposure.

Database server: A hacker may try to guess the database credentials directly, either using a connection string or through the management plane of a cloud provider. Make sure to use multiple layers of defence. If using a cloud accessible database, make sure the management plane is sufficiently secured. Use IP whitelisting or cloud security groups to limit access to the database, and use a strong authentication secret.

Bucket security: Hackers will look for publicly available buckets. Make sure the bucket is not accessible from the internet. limit accessibility to the relevant cloud security group, or from whitelisted IP addresses if accessed from outside the cloud.

Monitoring: turn on monitoring of file access in the application, and consider also logging access on database and bucket level. Regularly review logs to look for unauthorised access or unusual behaviour.

CCSK Domain 5: Information governance

Information governance is the management practices we introduce to enusre that data and information complies with organizational policies, standards and strategy, including regulatory, contractual and business objectives. 

There are several aspects of cloud storage of data that has implications for information governance. 

Public cloud deployments are multi-tenant. That means that there will be other organizations also storing their information in the same datacenter, on the same hardware. The security features for account separation will thus be an important part of achieving information compliance in most cases. 

As data is shared across cloud infrastructure, so is the responsibility for securing the data. To define a working governance structure it is important to define data ownership and who the data custodian is. The difference between the two, is that the former is who actually owns the data (and is accountable for its governance), and the latter who manages the data (and is responsible for ensuring compliance in practice). 

When we host third-party data in the cloud, we are introducing a third-party into the governance model. This third-party is the cloud provider; the information governance now depends on the provider’s management practices and technologies offered by the cloud provider. This complicates the regulatory compliance considerations we need to make and should be taken into account when designing a project’s regulatory compliance matrix. First, legal requirements may change because the cloud stores, or makes data available, in more geographical regions that would otherwise be the case. Compliance, regulations, and in particular privacy, should be carefully reviewed with regard to how governance is managed in the cloud for customer data. Further, one should ensure that customer requirements to deletion (destruction) of data is possible to satisfy given the technical offerings from the cloud provider. 

Moving data to the cloud provides a welcome opportunity to review and perhaps redesign information architectures. In many organizations information architectures have evolved over a long time, perhaps with little planning, and may have resulted in a fractured model where it is hard to manage compliance. 

Cloud information governance domains

Cloud computing can have an effect on multiple aspects of data governance. The following list defined issues the CSA has described as affected by cloud artifacts: 

Information classification. Often tied to storage and handling requirements, that may include limitations on access, location. Storing information in an S3 bucket will require a different method for access control than using a file share on the local network. 

Information management practices. How data is managed based on classification. This should include different cloud deployment models (or SPI tiers: SaaS, PaaS, IaaS). You need to decide what can be allowed where in the cloud, with which products and services and with which security requirements. 

Location and jurisdiction policies. You need to comply with regulations and contractual obligations with respect to data storage, data access. Make sure you understand how data is processed and stored, and the contractual instruments in place to manage regulatory compliance. One primary example here is personal data under the GDPR, and how data processing agreements with cross-border transfer clauses can be used to manage foreign jurisdictions. 

Authorizations. Cloud computing does not typically require much changes to authorizations but the data security lifecycle will most likely be impacted. The way authorization controls are implemented may also change (e.g. IAM practices of the cloud vendor for account level authorization). 

Ownership. The organization owns its data and this is not changed when moving to cloud. One should be careful with reviewing the terms and conditions of cloud providers here, in particular SaaS products (especially those targeting the consumer market).

Custodianship. The cloud provider may fully or partially become the custodian, depending on the deployment model. Encrypted data stored in a cloud bucket is still under custody of the cloud provider. 

Privacy. Privacy needs to be handled in accordance with relevant regulations, and the necessary contractual instruments such as data processing agreements must be put in place. 

Contractual controls. Contractual controls when moving data and workloads to control will be different from controls you employ in an on-premise infrastructure. There will often be limited access to contract clause negotiations in public cloud environments. 

Security controls. Security controls are different in cloud environments than in on-premise environments. Main concepts are security groups and access control lists.

Data Security Lifecycle

A data security lifecycle is typically different from information lifecycle. A data security lifecycle has 6 phases: 

  • Create: generation of new digital content, or modification of existing content
  • Store: committing digital data to storage, typically happens in direct sequence with creation. 
  • Use: data is viewed, processed or otherwise used in some activity that does not include modification. 
  • Share: Information is made accessible to others, such as between users, to customers, and to partners or other stakeholders. 
  • Archive: data leaves active use and enters long-term storage. This type of storage will typically have much longer retrieval times than data in active storage. 
  • Destroy. Data is permanently destroyed by physical or digital means (cryptoshredding)

The data security lifecycle is a description of phases the data passes through, without regard for location or how it is accessed. The data typically goes through “mini lifecycles” in different environments as part of these phases. Understanding the physical and logical locations of data is an important part of regulatory compliance. 

In addition to where data lives and how it is transferred, it is important to keep control of entitlements; who accesses the data, and how can they access it (device, channels)? Both devices and channels may have different security properties that may need to be taken into account in a data governance plan. 

Functions, actors and controls

The next step in assessing the data security lifecycle is to review what functions can be performed with the data, by a given actor (personal or system account) and a particular location. 

There are three primary functions: 

  • Read the data: including creating, copying, transferring.
  • Process: perform transactions or changes to the data, use it for further processing and decision making, etc. 
  • Store: hold the data (database, filestore, blob store, etc)

The different functions are applicable to different degrees in different phases. 

An actor (a person or a system/process – not a device) can perform a function in a location. A control restricts the possible actions to allowed actions. The key question is: 

What function can which actor perform in which location on a given data object?

An example of data modeling connecting actions to data security lifecycle stages.

CSA Recommendations

The CSA has created a list of recommendations for information governance in the cloud: 

  • Determine your governance requirements before planning a transition to cloud
  • Ensure information governance policies and practices extent to the cloud. This is done with both contractual and security controls. 
  • When needed, use the data security lifecycle to model data handling and controls. 
  • Do not lift and shift existing information architectures to the cloud. First, review and redesign the information architecture to support the current governance needs, and take anticipated future requirements into account. 

CCSK Domain 4 – Compliance and Audit Management

This section on the CCSK domains is about compliance management and audits. This section goes through in some detail aspects one should think about for a compliance program when running services in the cloud. The key issues to pay attention to are:

  • Regulatory implications when selecting a cloud supplier with respect to cross-border legal issues
  • Assignment of compliance responsibilities
  • Provider capabilities for demonstrating compliance

Pay special attention to: 

  • The role of provider audits and how they affect customer audit scope
  • Understand what services are within which compliance scope with the cloud provider. This can be challenging, especially with the pace of innovation. As an example, AWS is adding several new features every day. 

Compliance 

The key change to compliance when moving from an on-prem environement to the cloud is the introduction of a shared responsibility model. Cloud consumers must typically rely more on third-party auudit reports to understand compliance arrangement and gaps than they would in a traditional IT governance case. 

Many cloud providers certify for a variety of standards and compliance frameworks to satisfy customer demand in various industries. Typical audit reports that may be available include: 

  • PCI DSS
  • SOC1, SOC2
  • HIPAA
  • CSA CCM
  • GDPR
  • ISO 27001

Provider audits need to be understood within their limitations: 

  • They certify that the provider is compliant, not any service running on infrastructure provided by that provider. 
  • The provider’s infrastructure and operations is then outside of the customer’s audit scope, relying on pass-through audits. 

To prove compliance in a servicec built on cloud infrastructure it is necessary that the internal parts of the application/service comply with the regulations, and that no non-compliant cloud services or components are used. This means paying attention to audit scopes is important when designing cloud architectures. 

There are also issues related to jurisdictions involved. A cloud service typically will let you store and process data across a global infrastructure. Where you are allowed to do this depends on the compliance framework, and you as cloud consumer have to make the right choices in the management plane. 

Audit Management

The scope of audits and audit management for information security is related to the fulfillment of defined information security practices. The goal is to evaluate the effectiveness of security management and controls. This extends to cloud environments. 

Attestations are legal statements from a third party, which can be used as a statement of audit findings. This is a key tool when working with cloud providers. 

Changes to audit management in cloud environments

On-premise audits on multi-tenant environments are seen as a security risk and typically not permitted. Instead consumers will have to rely on attestations and pass-through audits. 

Cloud providers should assist consumers in achieving their compliance goals. Because of this they should publish certifications and attestations to consumers for use in audit management. Providers should also be clear about the scope of the various audit reports and attestations they can share. 

Some types of customer technical assessments, such as vulnerability scans, can be liimted in contracts and require up-front approval. This is a change to audit management from on-prem infrastructures, although it seems most major cloud providers allow certain penetration testing activities without prior approval today. As an example, AWS has published a vulenrability anpenetration testing policy for customers here: https://aws.amazon.com/security/penetration-testing/

In addition to audit reports, artifacts such as logs and documentation are needed for compliance proof. The consumer will in most cases need to set up the right logging detail herself in order to collect the right kind of evidence. This typically includes audit logs, activity reporting, system configuration details and change management details. 

CSA Recommendations for compliance and audit management in the cloud

  1. Compliance, audit and assurance should be continuous. They should not be seen as point-in-time activities  but show that compliance is maintained over time. 
  2. Cloud providers should communicate audit results, certifications and attestations including details on scope, features covered in various locations and jurisdictions, give guidance to customers for how to build compliant services in their cloud, and be clear about specific customer responsibilities. 
  3. Cloud customer should work to understand their own compliance requirements before making choices about cloud providers, services and architectures. They should also make sure to understand the scope of compliance proof from the cloud vendor, and ensure they understand what artifacts can be produced to support the management of compliance in the cloud. The consumer should also keep a register of cloud providers and services used. CSA recommends the Cloud control matrix is used to support this activity (CCM).

CCSK Domain 3: Legal and contractual issues

This is a relatively long post. Specific areas covered:

3.1 Overview

3.1.1 Legal frameworks governing data protection and privacy

Conflicting requirements in different jurisdictions, and sometimes within the same jurisdiction. Legal requirements may vary according to

  • Location of cloud provider
  • Location of cloud consumer
  • Location of data subject
  • Location of servers/datacenters
  • Legal jurisdiction of contract between the parties, which may be different than the locations of those parties
  • Any international treaties between the locations where the parties are located

3.1.1.1 Common themes

Omnibus laws: same law applicable across all sectors

Sectoral laws

3.1.1.2 Required security measures

Legal requirements may include prescriptive or risk based security measures.

3.1.1.3 Restrictions to cross-border data transfer

Transfer of data across borders can be prohibited. The most common situation is a based on transferring personal data to countries that do not have “adequate data protection laws”. This is a common theme in the GDPR. Other examples are data covered by national security legislation.

For personal data, transfers to inadequate locations may require specific legal instruments to be put in place in order for this to be considered compliant with the stricter region’s legal requirements.

3.1.1.4 Regional examples

Australia

  • Privacy act of 1988
  • Australian consumer law (ACL)

The privacy act has 13 Australian privacy principles (APP’s) that apply to all sectors including non-profit organizations that have an annual turnover of more than 3 million Australian dollars.

In 2017 the Australian privacy act was amended to require companies to notify affected Australian residents and the Australian Information Commissioner of breaches that can cause serious harm. A security breach must be reported if:

  1. There is unauthorized access or disclosure of personal information that can cause serious harm
  2. Personal information is lost in circumstances where disclosure is likely and could cause serious harm

The ACL protects consumers from fraudulent contracts and poor conduct from service providers, such as failed breach notifications. The Australian Privacy Act can apply to Australian customers/consumers even if the cloud provider is based elsewhere or other laws are stated in the service agreement.

China

China has introduced new legislation governing information systems over the last few years.

  • 2017: Cyber security law: applies to critical information infrastructure operators
  • May 2017: Proposed measures on the security of cross-border transfers of personal information and important data. Under evaluation for implementation at the time of issue of CCSP guidance v. 4.

The 2017 cybersecurity law puts requirements on infrastructure operators to design systems with security in mind, put in place emergency response plans and give access and assistance to investigating authorities, for both national security purposes and criminal investigations.

The Chinese security law also requires companies to inform users about known security defects, and also report defects to the authorities.

Regarding privacy the cybersecurity law requires that personal information about Chinese citizens is stored inside mainland China.

The draft regulations on cross-border data transfer issued in 2017 go further than the cybersecurity law.

  • New security assessment requirements for companies that want to send data out of China
  • Expanding data localization requirements (the types of data that can only be stored inside China)

Japan

The relevant Japanese legislation is found in “Act on the Protection of Personal Information (APPI). There are also multiple sector specific laws.

Beginning in 2017, amendments to the APPI require consent of the data subject for transfer of personal data to a third party. Consent is not required if the receiving party operates in a location with data protection laws considered adequate by the Personal Information Protection Commission.

EU: GDPR and e-Privacy

The GDPR came into force on 25 May 2018. The e-Privacy directive is still not enforced. TechRepublic has a short summary of differences between the two regulations (https://www.techrepublic.com/article/gdpr-vs-epPRrivacy-the-3-differences-you-need-to-know/):

  1. ePrivacy specifically covers electronic communications. It is evolved from the 2002 ePrivacy directive that focused primarily on email and sms, whereas the new version will cover electronic communications in general, including data communication with IoT devices and the use of social media platforms. The ePrivacy directive will also cover metadata about private communications.
  2. ePrivacy includes non-personal data. The focus is on confidentiality of communications, that may also contain non-personal data and data related to a legal person.
  3. The have different legal precedents. GDPR is based on Article 8 in the European Charter of Human Rights, whereas the ePrivacy directive is based on Article 16 and Article 114 of the Treaty on the Functioning of the European Union – but also Article 7 of the Charter of Fundamental Rights: “Everyone has the right to respect for his or her private and family life, home and communications.”

The CSA guidance gives a summary of GDPR requirements:

  • Data processors must keep records of processing
  • Data subject rights: data subjects have a right to information on how their data is being processed, the right to object to certain uses of their personal data, the right to have data corrected or deleted, to be compensated for damages suffered as a result of unlawful processing, and the right to data portability. These rights significantly affect cloud relationships and contracts.
  • Security breaches: breaches must be reported to authorities within 72 hours and data subjects must be notified if there is a risk of serious harm to the data subjects
  • There are country specific variations in some interpretations. For example, Germany required that an organization has a data protection officer if the company has more than 9 employees.
  • Sanctions: authorities can use fines up to 4% of global annual revenue, or 20 million EUR for serious violations, whichever amount is higher.

EU: Network information security directive

The NIS directive is enforced since May 2018. The directive introduces a framework for ensuring confidentiality, integrity and availability of networks and information systems. The directive applies to critical infrastructure and essential societal and financial functions. The requirements include:

  • Take technical and organizational measures to secure networks and information systems
  • Take measures to prevent and minimize impact of incidents, and to facilitate business continuity during severe incidents
  • Notify without delay relevant authorities
  • Provide information necessary to assess the security of their networks and information systems
  • Provide evidence of effective implementation of security policies, such as a policy audit

The NIS directive requires member states to impose security requirements on online marketplaces, cloud computing service providers and online search engines. Digital service providers based outside the EU but that supply services within the EU are under scope of the directive.  

Note: parts of these requirements, in particular for critical infrastructure, are covered by various national security laws. The scope of the NIS directive is broader than national security and typically requires the introduction of new legislation. This work is not yet complete across the EU/EEC area. Digital Europe has an implementation tracker site set up here: https://www.digitaleurope.org/resources/nis-implementation-tracker/.

Central and South America

Data protection laws are coming into force in Central and South American countries. They include security requirements and the need for a data custodian.

North America: United States

The US has a sectoral approach to legislation with hundreds of federal, state and local regulations. Organizations doing business in the United States or that collect or process data on US residents or often subject to multiple laws, and identification of the regulatory matrix can be challenging for both cloud consumers and providers.

Federal law

  • The Gramm-Leach-Bliley Act (GLBA)
  • The Health Insurance Portability and Accountability Act, 1996 (known as HIPAA)
  • The Children’s Online Privacy Protection Act of 1998 (COPPA)

Most of these laws require companies to take precautions when hiring subcontractors and service providers. They may also hold organizations responsible for the acts of subcontractors.

US State Law

In addition to federal regulations, most US states have laws relating to data privacy and security. These laws apply to any entity that collect or process information on residents of that state, regardless of where the data is stored (the CSA guidance says regardless of where within the United States, but it is likely that they would apply to international storage as well in this case).

Security breach disclosure requirements

Breach disclosure requirements are found in multiple regulations. Most require informing data subjects.

Knowledge of these laws is important for both cloud consumers and providers, especially to regulate the risk of class action lawsuits.

In addition to the state laws and regulations, there is the “common law of privacy and security”, a nickname given to a body of consent orders published by federal and state government agencies based on investigations into security incidents.

Especially the FTC (Federal Trade Commission) has for almost 20 years the power to conduct enforcement actions against companies whose privacy and security practices are inconsistent with claims made in public disclosures, making their practices “unfair and deceptive”. For cloud computing this means that when a certain way of working changes, the public documentation of the system needs to be updated to make sure actions are not in breach of Section 4 of the FTC Act.

1.3.2 Contracts and Provider Selection

In addition to legal requirements, cloud consumers may have contractual obligations to protect the personal data of their own clients, contacts or employees, such as securing the data and avoiding other processing that what has been agreed. Key documents are typically Terms and Conditions and Privacy Policy documents posted on websites of companies.

When data or operations are transferred to a cloud, the responsibility for the data typically remains with the collector. There may be sharing of responsibilities when the cloud provider is performing some of the operations. This also depends on the service model of the cloud provider. In any case a data processing agreement or similar contractual instrument should be put in place to regulate activities, uses and responsibilities.

3.1.2.1 Internal due diligence

Prior to using a cloud service both parties (cloud provider and consumer) should identify legal requirements and compliance barriers.

Cloud consumers should investigate whether it has entered into any confidentiality agreements or data use agreements that could limit the use of a cloud service. In such cases consent from the client needs to be in place before transferring data to a cloud environment.

3.1.2.3 External due diligence

Before entering into a contract, a review of the other party’s operations should be done. For evaluating a cloud service, this will typically include a look at the applicable service level, end-user and legal agreements, security policies, security disclosures and compliance proof (typically an audit report).

3.1.2.4 Contract negotiations

Cloud contracts are often standardized. An important aspect is the regulation of shared responsibilities. Contracts should be reviewed carefully also when they are presented as “not up for negotiation”. When certain contractual requirements cannot be included the customer should evaluate if other risk mitigation techniques can be used.

3.1.2.5 Reliance on third-party audits and attestations

Audit reports could and should be used in security assessments. The scope of the audit should be considered when used in place of a direct audit.

3.1.3 Electronic discovery

In US law, discovery is the process by which an opposing party obtains private documents for use in litigation. Discovery does not have to be limited to documents known to be admissible as evidence in court from the outset. Discovery applies to all documents reasonably held to be admissible as evidence (relevant and probative). See federal rules on civil procedure: https://www.federalrulesofcivilprocedure.org/frcp/title-v-disclosures-and-discovery/rule-26-duty-to-disclose-general-provisions-governing-discovery/.

There have been many examples of litigants having deleted or lost evidence that caused them to lose the case and be sentenced to pay damages to the party not causing the data destruction. Because of this it is necessary that cloud providers and consumers plan for how to identify and extract all relevant documents relevant to a case.

3.1.3.1 Possession, custody and control

In most US jurisdictions, the obligation to produce relevant information to court is limited to data within its possession, custody or control. Using a cloud provider for storage does not remove this obligation. Some data may not be under the control of the consumer (disaster recovery, metadata), and such data can be relevant to a litigation. The responsibility of a cloud provider to provide such data remains unclear, especially in cross-border/international cases.

Recent cases of interest:

  • Norwegian police against Tidal regarding streaming fraud
  • FBI against Microsoft (Ireland Onedrive case)

3.1.3.2 Relevant cloud applications and environment

In some cases, a cloud application or environment itself could be relevant to resolving a dispute. In such circumstances the artefact is likely to be outside the control of the client and require a discovery process to served on the cloud provider directly, where such action is enforceable.

3.1.3.3 Searchability and e-discovery tools

Discovery may not be possible using the same tools as in traditional IT environments. Cloud providers do sometimes provide search functionality, or require such access through a negotiated cloud agreement.

3.1.3.4 Preservation

Preservation is the avoidance of destruction of data relevant to a litigation, or that is likely to be relevant to a litigation in the future. There are similar laws on this in the US, Europe, Japan, South Korea and Singapore.

3.1.3.5 Data retention laws and record keeping obligations

Data retention requirements exist for various types of data. Privacy laws put restrictions on retention. In the case of conflicting requirements on the same data, this should be resolved through guidance and case law. Storage requirements should be weighed against SLA requirements and costs when using cloud storage.

  • Scope of preservation: a requesting party is only entitled to data hosted in the cloud that contains data relevant to the legal issue at hand. Lack of granular identifiability can lead to a requirement to over-preserve and over-share data.
  • Dynamic and shared storage: the burden of preserving data in the cloud can be relevant if the client has space to hold it in place, if the data is static and the people with access is limited. Because of the elastic nature of cloud environments this is seldom the case in practice and it may be necessary to work with the cloud provider on a plan for data preservation.
  • Reasonable integrity: when subject to a discovery process, reasonable steps should be taken to secure the integrity of data collection (complete, accurate)
  • Limits to accessibility: if a cloud customer cannot access all relevant data in the cloud. The cloud consumer and provider may have to review the relevance of the request before taking further steps to acquire the data.

3.1.3.7 Direct access

Outside cloud environments it is not common to give the requesting party direct access to an IT environment. Direct hardware access in cloud environments if often not possible or desirable.

3.1.3,8 Native production

Cloud providers often store data in proprietary systems that the clients do not control. Evidence is typically expected to be delivered in the form of PDF files, etc. Export from the cloud environment may be the only option, which may be challenging with respect to the chain of custody.

3.1.3.9 Authentication

Forensic authentication of data admitted into evidence. The question here is whether the document is what it seems to be. Giving guarantees on data authenticity can be hard, an a document should not inherently be considered more or less admissible due to storage in the cloud.

3.1.3.10 Cooperation between provider and client in e-discovery

e-Discovery cooperation should preferably be regulated in contracts and be taken into account in service level agreements.

3.1.3.11 Response to a subpoena or search warrant

The cloud agreement should include provisions for notification of a subpoena to the client, and give the client time to try to fight the order.

3.2 Recommendations

The CSA guidance makes the following recommendations

  • Cloud customers should understand relevant legal and regulatory frameworks, as well as contractual requirements and restrictions that apply to handling of their data, and the conduct of their operations in the cloud.
  • Cloud providers should clearly disclose policies, requirements and capabilities, including its terms and conditions that apply to the services they provide.
  • Cloud customers should perform due diligence prior to cloud vendror selection
  • Cloud customers should understand the legal implications of the location of physical operations and storage of the cloud provider
  • Cloud customers should select reasonable locations for data storage to make sure they comply with their own legal requirements
  • Cloud customers should evaluate and take e-discovery requests into account
  • Cloud customers should understand that click-through legal agreements to use a cloud service do not negate requirements for a provider to perform due diligence