Everyday I must launch new EC2 instances (or any other server with public IP). I'm provisioning it with Chef, creating vhosts, uploading databases etc.
I need to clone there a couple of private repos from GitHub. What would be the best way to do this?
I could manually generate an ssh key, and add it for each GitHub repo I need, then run the script - but it's a lot of work.
I could go for git clone git://user:password#github.com/*****/*****.git, but obviously I don't want to store my password this way
What else?
Is there any way to:
store a private key (or password?) in a recipe/cookbook or
generate new key, and synchronize it via API with GitHub (but this would lead to hundreds of keys in my GitHub account)
Store your key in an S3 bucket and use IAM roles/policies to control access. Citadel makes this easy to integrate with Chef. See my post about secrets management with Chef for a summary of other options.
Shameless plug: the deploy_key cookbook.
I created this cookbook with this precise use case in mind. It manages the entire lifecycle of deploy keys in GitHub, BitBucket and GitLab. It creates a key locally (so that it never has to be sent over the network), adds it to the repo as a deploy_key (read-only, so that these keys don't ever push changes to the repo), and can be used to delete the key files and remove the keys from the repo.
All actions are idempotent so if you're afraid your repos will be flooded with too many deploy_keys you can either remove the key from the repo after use (via Chef, :remove action), or have a periodic clean up task to delete all deploy_keys. Next time Chef runs, it will notice that the key is absent and re-add it.
The only secret you need to protect are the credentials to the repo, which can be protected in the same way you do other secrets.
Related
We have a private repository on GitHub for our company's project. Initially i started off the repository, therefore current owner is me. But as the team grew we started to feel an urge to create a organization on GitHub which enables roles etc.
Current situation:
several distinct development machines are connected to the repository
by 'personal access tokens'. So their authentication is cached globally.
Live (production) server that is connected to the repository, with an 'SSH key' .
Test (development) server also connected to the repository. But it asks for authentication each time (username and password/PAT)
In this case, if a transfer the ownership of the repository from my account to company's organization: what I'll possibly break ? I've read that GitHub will redirect old URL to the repository will point to the new URL. But still, i think SSH key on the live server is likely to fail.
Can you please inform me about possible outcomes ? Thank you.
Edit 1: Results
Here is the results if anyone stumbles upon this post.
Old remote URL worked just fine, as stated in GitHub docs on transferring ownership. But we decided it's for the best to change the remote URL to new one, just in case.
Development machines with PAT worked just fine. No issue has been encountered.
We updated Live (production) server's origin remote URL right away so there won't be any funny business. SSH key had to be updated to comfort the new URL.
Test is still using old URL without any issue.
Generally, personal access tokens work based on the owner's access permissions, so if you continue to retain access to the repository, then those will continue to work. The same is true of SSH keys associated with the account. Similarly, other users who have their own PATs or SSH keys and continue to retain access will have those PATs or keys continue to work.
Ideally for most automated system purposes, you'll want to use SSH with deploy keys, because those are associated with the repository, so if someone leaves the company, things won't break. If you need to have individual users log into a server and pull, just have them forward their SSH agent.
If you need PATs for automated processes, such as for making API calls, you'll want to make a bot account and grant it access to the repository. That's much easier than trying to use individual people's PATs and again, it prevents employees who leave the company from breaking things when they lose access.
In the myhome/.ssh/ directory are local SSH keys for my personal GitHub account, the directory has the id_rsa and id_rsa.pub files that allow me to authenticate with GitHub.
I am setting up a server (compute engine on GCP), and this server needs these keys because it needs to authenticate to my GitHub to pull a repo. Should I either:
transfer over / SCP the currently-existing id_rsa, `id_rsa.pub that I have locally onto the server, and use those on the server.
create a brand new SSH key from on the server, and use this key. Add it to my GitHub profile.
it doesn't matter, either (1) or (2) is fine.
or something else?
(2) seems like the right approach, but we are not certain.
Indeed option 2. As a best practice, you should not share the same private key.
Go ahead and generate a new SSH key following the docs: https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh
For a public repository, in github actions, assume my action runs on a compute in azure. I am confused on how i can protect the azure auth details if the CI pipeline has to run in azure.
Lets say to use this action, i have to use a secret and i set an environment variable's value to be the secret - have I not lost the point of having a secret? A malicious user can send a PR that prints the value of the environment variable :
user_password: {{secret.USER_PASSWORD}}
User code:
print(os.environment['user_password'])
The malicious user does not have to guess since the workflow is public and he knows which env var has the secret.
Where am i wrong?
GitHub, like other CI providers, redacts most secrets from the logs. It considers a variety of formats and tries to scrub them. However, in general, you should be careful to avoid printing them to the logs because no system is foolproof and not every possible encoding can be considered.
If you're worried about forked repositories trying to access your secrets, they can't; that's specifically not allowed for the reason you describe. So if someone opens a PR against your repository, they won't be able to access the secrets unless the branch is in your repository (in which case, don't grant that person write access to your repo). It's presumed that you'll perform basic code review to catch any malicious code before merging, so a bad actor won't be able to run any code with the secrets for your repository.
In general, though, using environment variables as a way to pass secrets to programs is a best practice, assuming the running system and programs are trusted. Other users on the system cannot see the environment, and in a CI system the system and programs are assumed to be trusted.
This one is rather complicated to describe. I have two github accounts, I will refer to them as github A and B.
I had created a local repo X and tried to connect it to a new remote repo I created on github account A. However, my default github login credetials were set for my github account B. I went through this tutorial and was able to add a second key created in my .ssh to allow me access to acct. A at will. This method worked and allowed me to add the new remote repo on github account A to my local repo X. I could then push origin to master just fine.
I then made a clone of the same repo X on github account A and placed it in a new local location (same drive, different folder) which I will refer to as local repo Y. I then tried to push changes to the same remote repo on github A and it denies me saying I'm trying to access using my username for my remote github B account.
I tried re-adding the remote url via the same special .ssh key previously created for repo X and use it for the local repo Y but it says it already exists.
I'm looking for a simple solution here, hopefully one exists. I would appreciate your help on this.
You can test what actual key is used with:
ssh -Tv <yourSSHConfigEntry>
Make sure to use an ssh URL like yourSSHConfigEntry:/ (no need to prefix it with git#: the User should be in your ~/.ssh/config file)
The idea is: your SSH config file should use the right private key, whose public key has been registered to the right GitHub account.
Check the remote URL: git remote -v. If it starts with git#github.com, it won't use the ssh config entry.
Type:
git remote set-url origin <my-github-acct-A>:<user>/<repo>
That will use the SSH config entry, and make sure the User in it is "git".
I'm pretty sure the issues you're running into are a result of managing multiple accounts on the same computer, regardless of the repos being duplicates.
If you haven't already, I'd suggest you take a look at the very detailed answers provided here.
I have been through this trouble many times and have resolved it by setting the ssh config rules or by enforcing the ssh-agent to have the correct ssh key alone.
I have documented the entire process and Freecodecamp published the article.
Here's the link! Hope this will help whenever you come across this use case. :)
I'm trying to pull an existing github repo made on my local machine onto a vm running on EC2 that will be used by multiple people. I have some concerns with using an ssh key without a password, so I was wondering if there was any way to pull directly onto the VM either anonymously, or by providing the username and password of the account that originally pushed the repo, so that my personal information won't have to be stored on the vm, and there's no security risk in having someone get ahold of a password-less ssh key for the vm. Is this possible?
Currently running Ubuntu 12.04
I recommend generating a new key and adding it as deployment key to your specific repo.
These keys are linked to a specific repo, not your account.
Alot of options are also available here.
https://help.github.com/articles/managing-deploy-keys