Terraform and story of storing sensitive information
Table of Contents
The intro
Terraform is a basic DSC (Desired State Configuration) engine, that does this very well.
Terraform is super easy to learn, easy to understand and it’s easy to get going.
Terraform stores all the information about the resources it manages in a statefile called terraform.tfstate (when using local state). You can also use blob storage or a few other storage options for storing state.
The problem
In essence, my main pain point with using terraform in production is that it it saves all of its managed state, in plain-text.
This includes api-keys, connection-strings, passwords etc. If the resource exports the property, it will be stored in the statefile — in plain text. This looks to be referenced first in a github issue from 2014.
So why is this a problem?
If your account gets compromised, or your storage account gets compromised — the attacker now has access to ALL of the resources that is in that statefile.
The usual 4 proposed solutions;
- Use encrypted blobstorage.
- Use the terraform enterprise plan.
- Use a tool like Terrahelp.
- Use git-crypt.
Use encrypted blob-storage like S3 or Azure blob;
Sure, also plausible, but if your account is compromised, you are still out of luck unfortunately.
Use terraform enterprise;
I am unsure, but by the looks of it, its the same as using S3, with some nice management interface on top. So same issue as with storing directly in blob.
Use a tool like terrahelp;
This is a good proposal, the reason it does not work for me is that now the burden is on the user to remember to encrypt, decrypt before checking state into git (or uploading it somewhere) — Higher risk of detached state because some guy forgot to `git pull` before changing something.
Use gitcrypt;
Yes, would work for the security issue, it does however not fix the git pull/push process.
My proposed solution
Use a Http backend as a proxy for the blob-storage.
The idea is, that you then let that proxy remove all the sensitive information from the statefile being saved. You could even do it as an azure function / aws lambda.
This lets you control what gets saved and how.
Unless you use the api-keys/connection-strings as references in your terraform modules, it’s not a problem. Terraform will not complain.
Say we use an azure function and blob storage to keep state in;
I will provide hopefully a full list of sensitive properties (in the hashicorps managed providers) and some code to create the http backend in a later blogpost.
The actual solution
I’ve looked at the source code, and what needs to happen is that terraform needs a flag, to remove or encrypt all properties marked with the “sensitive” attribute (which is already a part of all the resources) when it saves state.
