I have a terraform project that accesses a google cloud bucket. All pull requests are done through github. However, I'm not sure where I'm supposed to securely store my bucket credentials? Of course I don't want to upload them to github but I'm not sure where they should be kept. This is my variables file
variable "credentials_filepath" {
default = "../../../creds.json"
}
And this is my main file
provider "google" {
credentials = file(var.credentials_filepath)
project = var.project
region = "europe-west2"
zone = "europe-west2-a"
}
The main idea is to store the secrets securely in a secrets manager and to use a wrapper that makes the secrets available as environment variables only for the duration of the wrapper process.
Among the tools that I can recommend: pass, gopass, summon.
For example, once the secrets are stored in GPG and you have the gpg-agent configured, you can run:
TF_VAR_secret=$(pass gc/myproject) terraform ...
This will tell the shell to set the environment variable TF_VAR_secret to the output of pass gc/myproject.
That command tells pass to use gpg and gpg-agent to read the value of the secret stored at gc/myproject.
secret is a Terraform variable and TF_VAR_secret tells Terraform to fill that variable from that environment variable. (See Terraform documentation).
Related
I am setting up Actions in Github, some of them require a Token for authentication. This is the process I follow to generate them, which is detailed in the Actions Docs:
Go to my Account, generate a PAT
Go to the project and add a Secret using that PAT value
Add the variable name in the Action yml file, for example:
env:
# used by semantic-release to authenticate at github and write to master
# I used the developer tab to generate the token and then paste it to several projs
# as a secret
GH_TOKEN: ${{secrets.GH_TOKEN}}
# used by semantic release to authenticate when publishing to npm
# Generate it in NPM (you just need to be registered at npm which is simple)
NPM_TOKEN: ${{secrets.NPM_TOKEN}}
Now, this is rather tedious, even having a single PAT, I'd still need to create a secret per project.
I have noticed though, that if you create an organization, you can create secrets at the ORG level.
Can you do something similar without having an organization? Can you create user secrets? Or maybe there is a way to use the user settings in the Project's Action?
So I'm creating a website in a GitHub repo, on which I'm trying to implement a Discord Login via OAuth2. For all that to work I need to set an environment variable for the client id and secret. How do I set an environment variable in a GitHub repo?
How do I set an env var in a github repo
Env vars are only relevant to the machine actually running the bot (or website).
Most hosting companies today give you a way to edit your environment variables. If you are using github as a webhost, maybe see this github article.
Let's say you have a string which needs to be in a variable token. For your local development, you create a file that simply sets that variable. Make sure to exclude it from git by including it in your .gitignore!
And in your code, you only use that file IF the env var isn't set. e.g.:
// load auth & other tokens
var token = null;
if (process.env.hasOwnProperty('TOKEN')) {
token = process.env.TOKEN;
}
else {
var auth = require('./discordauth.json');
token = auth.token;
}
From there you just have to learn how to set env vars in your particular hosting. Create an env var named TOKEN (to match process.env.TOKEN in the code example). Example:
Contents of file discordauth.json:
{
"token": "XXXXXXXXXXXXX",
}
Value of env var TOKEN
XXXXXXXXXXXXX
If you don't want to expose secrets in your Github repo, you could use dotenv to manage them locally. You store the secrets in a .env file at the root of your project. Make sure to remember to include .env in your .gitignore file so it doesn't get sent to version control.
It's perfectly fine to store different client IDs and secrets for each environment inside of a GitHub repo (assuming that the repo is private). However, ideally you would store your client IDs and secrets for each of your environments inside of a database table. Then in your application itself you would simply reference said table (using the connection string for each environment) to retrieve the ID / secret pertaining to that specific environment.
VSTS build definition has the option to create a secret variable. How secret is that variable? Is it safe to store the user credentials which is specific to a set of users? Can other users (who are not authorized to do it) can decrypt that variable?
I came across this article.
Assuming users have build modification access then is it possible to decrypt the variable?
Variables stored are as secure as the agent that runs the build and the integrity of your build definition.
Like you said, if a user can modify the Build Definition and has access to the secret they can pass it to a PowerShell or a Curl task etc. Or if the user can take control over a Build Task's script they can iterate all available secrets (build tasks are considered trusted by the Build System).
Consider that everyone who has write-access over the work directories of the agent can access all secrets that are available to the Build Definitions that execute on the build agent. They can change the scripts used by Build Tasks and thus gain the same level of trust. Any build that runs after this change and until a new version of the task is pushed to the agent will be compromised in this scenario. In theory can every build definition "infect" the _tasks folder of the agent as well. Best way to protect against this is to use the Hosted Pool or to regularly reset your agent's VMs.
YAML build definitions combined with Pull-Requests give you more control over the Change/approval process of build definitions.
Using a Variable Library you can reduce the number of people who can add secret variables to their Build Definition.
You must secure the Agent Pools and the Variable Libraries/Build Definitions in such ways that only limited and trusted users can access these resources. Optionally use single-use passwords that expire after a short time or temporarily grant these permissions.
Remember that all changes to Build Definitions and Variable Libraries and Scripts in the Git Repository are tracked.
The alternate ways to get access to the secrets do not apply to Azure DevOps as none have access to the Application Tier in Azure and access is strictly monitored by Microsoft.
To encrypt GitHub tokens to use them in your .travis.yml you seem to need the Travis CI CLI tool - which I can't install right now.
Therefore I didn't put the GitHub token in my .travis.yml but rather created a new environment variable GITHUB_TOKEN in the repository settings at https://travis-ci.org and use it like this:
github_token: $GITHUB_TOKEN
Now I can use the token to deploy to GitHub releases, for example.
Is this safe? Or are my environment variables in Travis CI somehow publicly accessible?
I found out by now: No, in general, they're not public. Yes, if you follow some rules, it seems perfectly safe to use them for sensitive information. To quote directly from the docs at https://docs.travis-ci.com/user/environment-variables#Defining-Variables-in-Repository-Settings about environment variables:
Define variables in the Repository Settings that (...) contain sensitive data, such as third-party credentials.
Some reasonable rules to keep in mind:
Most importantly: Hide them from the logs with the ON/OFF switch in the settings.
By default, the value of these new environment variables is hidden from the export line in the logs. This corresponds to the behavior of encrypted variables in your .travis.yml. The variables are stored encrypted in our systems, and get decrypted when the build script is generated.
Define them correctly with " if necessary.
These values are used directly in your build, so make sure to escape special characters (for bash) accordingly. In particular, if a value contains spaces, you should put quotes around that value. E.g. my secret passphrase should be written "my secret passphrase".
Also important to know: even if your environment variables are hidden in the logs, someone could still submit a pull request to your repository with malicious code and access the variable this way. Therefore, the variables are not available within builds from pull requests:
Similarly, we do not provide these values to untrusted builds, triggered by pull requests from another repository.
I have a case where the RunDeck scripts do need some credentials in order to run. Obviously we do not want to store these in the job definitions because these are visible and also stored in SCM.
While I was able to use the Key Storage vault to put these secrets in, I was not able to find a way to access them from the job itself.
Rundeck 2.6.2 (released 2015-12-02) allows you to specify key storage secrets as default values for secure job options. See Secure Options using Key Storage