UPDATE - It happened AGAIN!!!
Ok, so this just happened AGAIN! MAN is this frustrating!!! But this time I dug a little deeper and found that for some reason, my private keys were unloaded.
Specifically, when I call this...
ssh-add -l -E md5
I get this...
The agent has no identities.
However, if I then run this...
ssh-add /Users/[username]/.ssh/[private key]
Everything works again! SourceTree connects just as it's supposed to.
The question is why do I have to keep running the 'ssh-add' command?! Why does it keep forgetting my keys?!
As mentioned elsewhere, not sure if this makes a difference, but I'm running a MacBook Pro with High Sierra, although this happens on Sierra too.
Original Post:
This one has me both stumped, and annoyed as heck!! SourceTree (or ssh or something!) keeps forgetting/not applying/ignoring my SSH keys every day! I don't know why.
Note: Updated to use BitBucket's info instead of GitHub.
Here's the relevant portion of my current config file
# --- Sourcetree Generated ---
Host MarqueIV-Bitbucket
HostName bitbucket.org
User MarqueIV
PreferredAuthentications publickey
IdentityFile /Users/MarqueIV/.ssh/MarqueIV-Bitbucket
UseKeychain yes
AddKeysToAgent yes
# ----------------------------
Here's a 'ls' of my ~/.ssh folder (truncated)
-rw-r--r--# 1 MarqueIV staff 421 Dec 14 11:25 config
-rw-r--r--# 1 MarqueIV staff 1808 Dec 9 14:20 known_hosts
-rw------- 1 MarqueIV staff 3243 Dec 6 23:33 MarqueIV-Bitbucket
-rw-r--r-- 1 MarqueIV staff 781 Dec 6 23:33 MarqueIV-Bitbucket.pub
Here's my known_hosts file (keys redacted)
bitbucket.org,104.192.143.3 ssh-rsa [redacted]
bitbucket.com,104.192.143.9 ssh-rsa [redacted]
104.192.143.2 ssh-rsa [redacted]
Note: Not sure if this matters, but you can see lines 1 and 2 seem to be duplicates.
And here's the output of ssh -Tv git#bitbucket.org
OpenSSH_7.6p1, LibreSSL 2.6.2
debug1: Reading configuration data /Users/MarqueIV/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: Connecting to bitbucket.org port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file /Users/MarqueIV/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/MarqueIV/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/MarqueIV/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/MarqueIV/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/MarqueIV/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/MarqueIV/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/MarqueIV/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/MarqueIV/.ssh/id_ed25519-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.6
debug1: Remote protocol version 2.0, remote software version conker_1.0.315-a08d059 app-153
debug1: no match: conker_1.0.315-a08d059 app-153
debug1: Authenticating to bitbucket.org:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256-etm#openssh.com compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256-etm#openssh.com compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:zzXQOXSRBEiUtuE8AikJYKwbHaxvSc0ojez9YXaGp1A
debug1: Host 'bitbucket.org' is known and matches the RSA host key.
debug1: Found key in /Users/MarqueIV/.ssh/known_hosts:1
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering public key: RSA SHA256:h+6zCXg32Uw4fYxSUMwYst3zee8RFb9Z47H1QUTz58E /Users/MarqueIV/.ssh/MarqueIV-GitHub
debug1: Authentications that can continue: publickey
debug1: Trying private key: /Users/MarqueIV/.ssh/id_rsa
debug1: Trying private key: /Users/MarqueIV/.ssh/id_dsa
debug1: Trying private key: /Users/MarqueIV/.ssh/id_ecdsa
debug1: Trying private key: /Users/MarqueIV/.ssh/id_ed25519
debug1: No more authentication methods to try.
git#bitbucket.org: Permission denied (publickey).
See how it doesn't appear to be applying the key defined in config and known_hosts? Seems like that would be a problem, no?
Note: I'm using macOS Sierra, and I have updated my path to include /usr/bin before /usr/local/bin as outlined here. If I don't do that, I get an error saying ssh doesn't recognize UseKeychain yes in the config.
As a result, almost daily here's the routine I have to go through. I'll use GitHub as my example.
I open SourceTree and try to pull the latest from GitHub. It fails with a 'git#github.com: Permission denied (publickey).' message.
I remove my GitHub account from SourceTree.
I delete both the public and private keys for GitHub from the .ssh folder on my machine.
I go to GitHub and delete my old public key from my account.
Back in SourceTree, I log into GitHub again using my username and password.
Once logged in, using SourceTree, I generate a new SSH key-pair for GitHub.
I copy my public key to the SSH area in my GitHub account. (Sometimes I notice it adds it for me, but I like to be safe and double-check.)
Now I can push and pull again just fine.
I go home for the day and log on at home. It fails again. Repeat all of the steps above.
How do I get SourceTree/ssh/whatever to remember my da*n keys so I don't have to keep doing this every time I change locations?! What step am I missing???
So can anyone offer suggestions on how to make my SSH keys 'stick'?
Ok, I think I have all the parts figured out.
To help people get what they're after, here's the solution right up front:
Make sure the keys you want to work with are secured with a password or else they will not add to Keychain.
Make sure the keys you want to auto-load are configured in your config file and have the UseKeychain and AddKeysToAgent set
Make sure to connect to those config-defined hosts from terminal!!
Create a LaunchAgent to run ssh-add -A to automatically reload your Keychain-stored keys
Ok now that you know what to do, here's the 'why'.
The Meat
As explained in my question, lately, whenever I rebooted, I (incorrectly) thought the system was losing my private keys. It wasn't losing them, it was just ignoring them. This was because of a bunch of things that all came together in a perfect storm of confusion for someone like me who never uses the terminal for GIT.
In the latest versions of macOS, Apple changed how it's implemented SSH so that It better matches the implementation of OpenSSH
As a result of #1, ssh-add -K [privateKey] no longer stores the keys in the keychain (it essentially ignores the -K.) While they do get added to ssh for that session--and thus your connections will work again--as soon as you reboot, they will no longer work. (This is what's been driving me mad!)
Even for keys that are in the Keychain, Apple no longer loads them automatically meaning you manually have to call ssh-add -A from the terminal to reload them every time you reboot.
However, as stated above, ssh-add -K [privateKey] no longer adds the keys to keychain, so ssh-add -A is pointless anyway for keys added that way. (They can be added to Keychain another way. More on that in a minute.)
Because of the above, any keys manually added with the -K option prior to upgrading your OS will still be in your Keychain. However, keys added after Apple's change are not.
That said, Apple does still have the ability to store keys in the keychain, but not from ssh-add anymore. It now only works for hosts defined in your config file.
This is now the only way to get your keys in your Keychain.
Again, here's my config:
Host MarqueIV-Bitbucket
HostName bitbucket.org
User git <-- Make sure this is 'git', not what SourceTree puts here
PreferredAuthentications publickey
IdentityFile /Users/MarqueIV/.ssh/MarqueIV-Bitbucket
UseKeychain yes <-- Note here
AddKeysToAgent yes <-- ...and here
But wait! If you look in my config file, it does have those values set! So why didn't it work?
Two things.
I don't use Terminal, ever. I use SourceTree which doesn't use the host entry in that file
Apple technically only adds (and stores) the key on demand when that host is accessed, not when the file is (re)loaded meaning unless you explicitly access that host, nothing happens.
In my case, adding the keys via SourceTree would add them for that initial session, but as soon as I rebooted, the keys would again not be loaded and thus all connections would fail. ssh-add -A wouldn't fix it either because again, they weren't in the keychain, meaning I was back to manually adding each one on the command line with ssh-add [privateKey]. What a pain!!
Then it occurred to me... if that setting is in the config file, and that entry can be used from the command line, then shouldn't I be able to directly connect to that host, thus adding the keys to my keychain? Let's find out! I typed this...
ssh -T MarqueIV-BitBucket
And sure enough, not only was the key added to ssh, but it was also again added to my Keychain! I confirmed this by checking Keychain Access directly and it was there.
To further test, I ran this...
ssh-add -D
which deleted all my keys. Sure enough, my SourceTree connections all failed again.
Then I ran this...
ssh-add -A
and the keychain-stored keys magically came back and connections started working again! WOOT!!
Ok, almost there, but not quite! What about reboots? Again, Apple no longer automatically loads keys from Keychain. Sure, it's just a quick jaunt now to terminal to type ssh-add -A, but again, I shouldn't have to do that!
Enter LaunchAgents!
LaunchAgents and LaunchDaemons are beyond the discussion of this post, but in short, they allow you to execute something on reboot, on a schedule, when changes happen to the system, etc.
In my case, I wanted something that would run when I logged onto my mac, so a LaunchAgent was the best choice.
Here's my plist defining how to execute ssh-add --apple-load-keychain every time I logged into my account (even if I never touched Terminal):
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>ssh-add-a</string>
<key>ProgramArguments</key>
<array>
<string>ssh-add</string>
<string>--apple-load-keychain</string>
</array>
<key>RunAtLoad</key>
<true/>
</dict>
</plist>
Since I only want this for my particular user, I stored it here:
~/Library/LaunchAgents/com.mydomain.ssh-add-a.plist
Then I ran this to make it executable
chmod +x ~/Library/LaunchAgents/com.mydomain.ssh-add-a.plist
Note: Make sure to change the permissions to allow it to be executed, or it won't start!
Sure enough, on reboot, all my keys came back and were active! Connections all worked, children played, grown men cried, and it was a good day in the Code-dom!
So to recap:
Apple changed how their SSH worked
Keys were no longer added to Keychain from the command line
Apple also no longer auto-loaded keys that were stored in the keychain
Using terminal to connect to config-defined hosts fixed #2
Using a LaunchAgent fixed #3
Hope this helps! Now time to go get some Icy-Hot for my sore shoulder that I've been patting myself on so hard for figuring this all out! ;)
First, install the latest Git for Windows release (the 2.15.1.2 one, by simply uncompressing the archive PortableGit-2.15.1.2-64-bit.7z.exe anywhere you want, and adding it to your PATH)
Second, make sure your SourceTree is using the System Git
Third, test in command-line if your ssh key is recognized:
ssh -T git#github.com
Hi username! You've successfully authenticated,
but GitHub does not provide shell access.
Finally, make sure that SourceTree / Tools / Option uses as SSH client the OpenSSH one (not putty)
Then SourceTree should have nop problem reusing your ssh key.
From your logs, the ~/.ssh/config generate is wrong: it mentions as User your username.
Whenever you establish an SSH connection to github.com/bitbucket.org, it is never as "you". It is always as git.
Host MarqueIV-Bitbucket
HostName bitbucket.org
User MarqueIV
PreferredAuthentications publickey
IdentityFile /Users/MarqueIV/.ssh/MarqueIV-Bitbucket
UseKeychain yes
AddKeysToAgent yes
Test it with ssh -Tv MarqueIV-Bitbucket
That ~/.ssh/config excerpt is only applicable for the host MarqueIV-Bitbucket. If your SSH remotes are listed as MarqueIV-Bitbucket:owner/repo then SSH and SourceTree should respect that config; you can confirm this with ssh -Tv MarqueIV-Bitbucket and by updating one or more of the remotes to the MarqueIV-Bitbucket:owner/repo.git format.
Related
This is my first time dealing with ssh keys and github as a new developer.
Every time I've tried to connect an ssh key to my Mac Terminal, this message pops up:
The authenticity of host 'github.com (140.82.114.3)' can't be established.
ECDSA key fingerprint is SHA256:p2QAMXNIC1TJYWeIOttrVc98/R1BUFWu3/LiyKgUfQM.
Are you sure you want to continue connecting (yes/no)?
The fingerprint pasted in the GitHub website never matches the fingerprint in the message.
I followed all of the instructions as I saw them.
Any ideas on what I could do to fix this?
If the output of ssh-keyscan github.com matches the official GitHub's SSH key fingerprints, you can go ahead and type:
ssh-keyscan github.com >> ~/.ssh/known_hosts
That way, SSH should not query you anymore about the authenticity of host 'github.com.
From there, any SSH key added to your GitHub account profile should work, which you can check with:
ssh -Tv git#github.com
I know there are lots of links about this topic and some official ones. I have tried several SO answers too but I am still at a loss.
I have created a ED25519 SSH private key (PPK file) and public key (PUB file).
My key has a passphrase.
I have added my public key to GitHub.
I have added my PPK key to Putty Pageant.
I have changed the repository to use the SSH key URL as identified on GitHub.
But it will not push and came up with a port 22 issue. After research I found and answer that said to create a config file and put this in it:
Host github.com
Hostname ssh.github.com
Port 443
then, I tried this command:
ssh -vT git#github.com
It found my key and asked for the passphrase. It worked, although it does say this:
Hi xxxxx! You've successfully authenticated, but GitHub does not provide shell access.
debug1: channel 0: free: client-session, nchannels 1
Transferred: sent 2160, received 2304 bytes, in 0.2 seconds
Bytes per second: sent 9143.6, received 9753.2
debug1: Exit status 1
It has that comment about the shell? Anyway, I then go into TortoiseGit Git settings and confirm it is using my "noreply" email address, has the SSH git URL and that I have selected the PPK PuttyLink file.
Despite all of this, I can't push to the remote server. Tried for 3 days. :(
What step am I missing?
The configuration file .ssh/config is not honored by PuTTY. Switch to OpenSSH or configure a session in PuTTY or use the url ssh://git#ssh.github.com:443/....
cf. https://tortoisegit.org/support/faq/#howsshdifferentport
(Note: I know that a personal access token will work, but external reasons require me to do this via an SSH Deploy Key. Both the source repo and the target repo are private.)
I need to use CircleCI to push every commit from the source repo to the target repo. Assume the repos are named source and target. I am configuring CircleCI to run my custom push script but it's saying that the key is read-only.
What I've done:
Created a new key pair with ssh-keygen on my PC and compress the private key.
Uploaded the public key id_rsa.pub to the target repo as a Deploy Key, with "allow push access with this key" ticked.
Put the compressed private key in the repository Environment Variables on CircleCI
Wrote this script:
#!/bin/bash
set -e
if [ -z "$SSH_KEY_E" ]; then
echo "No SSH key found in environment, set it as \$SSH_KEY_E" >&2
exit 1
fi
echo "$SSH_KEY_E" |
base64 -d |
gunzip -c > ~/.ssh/m.id_rsa
set -x # debug
cat >> ~/.ssh/config << EOF
Host GHMirror
HostName github.com
User git
Port 22
IdentityFile ~/.ssh/m.id_rsa
EOF
git remote add mirror GHMirror:iBug/circleci-target.git
git push mirror +master
The output log indicates that the key restored from environment is valid, but it doesn't seem like it's used to push to GitHub.
Some points I'd like to point out:
At the beginning, I overwrote ~/.ssh/id_rsa and used directly git#github.com:iBug/target.git as the remote URL for mirror, but it didn't work, saying the key is read-only
Then, thinking that the default key (by CircleCI) might be located at somewhere else, I changed the remote host to GHMirror and wrote this rule to ~/.ssh/config, as seen in the shell script. Still it complains that the key is read-only
I changed the key to another path ~/.ssh/m.id_rsa, but no luck.
I have verified that everything by running the script locally, and it successfully pushed to the target repository, so there must be something on CircleCI that I'm missing.
Update 1
I added the environment variable GIT_SSH_COMMAND="ssh -vv" and got this result:
debug1: key_load_public: No such file or directory
debug1: identity file /home/circleci/.ssh/id_rsa type -1
...
debug2: key: (0xREDACTED), agent
debug2: key: /home/circleci/.ssh/id_rsa ((nil))
debug2: key: /home/circleci/.ssh/id_dsa ((nil))
debug2: key: /home/circleci/.ssh/id_ecdsa ((nil))
debug2: key: /home/circleci/.ssh/id_ed25519 ((nil))
However, ls -l ~/.ssh/id_rsa shows that the file is there, with permission 0600.
I believe the problem you are experiencing is due to the ssh-agent offering the CircleCI key, which is read-only. I've hit this problem as well in the past. To debug you can use the following:
export GIT_SSH_COMMAND="ssh -vv"
This will print out details about which key is being used.
I was able to fix the problem with something like this:
# Disable the ssh-agent
export SSH_AUTH_SOCK=none
# Tell ssh to use the specific SSH key
export GIT_SSH_COMMAND="ssh -i path/to/key"
Also make sure that you chmod 0600 path/to/key. SSH will not use keys if they are readable by other users.
I got a new Windows computer and installed putty on it. I exported the registry keys from the old computer and imported them to recover the Putty connections and avoid adding them manually.
I also copied the rsa public key .ppk to the new computer and selected it in the Auth section in putty, but I get the following error:
Unable to use key file "c:\path\to\id_rsa.ppk" (unable to open file)
I'm temped to regenerate the key pair again in the VPS server but I thought public keys could be used by any computer!
Am I missing something? Any help is appreciated.
Thanks!
Edit:
Plink output (plink -v username#hostname -i "c:\path\to\id_rsa.ppk") as requested by #Martin Prikryl:
We claim version: SSH-2.0-PuTTY_Release_0.70
Server version: SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.1
Using SSH protocol version 2
Doing ECDH key exchange with curve Curve25519 and hash SHA-256
Server also has ssh-ed25519/ecdsa-sha2-nistp256 host keys, but we don't know any of them
Host key fingerprint is:
ssh-rsa 2048 24:2a:c3:65:b9:9a:3e:5f:59:fc:dc:15:42:31:0b:a0
Initialised AES-256 SDCTR client->server encryption
Initialised HMAC-SHA-256 client->server MAC algorithm
Initialised AES-256 SDCTR server->client encryption
Initialised HMAC-SHA-256 server->client MAC algorithm
Reading key file "D:\path\to\id_rsa.ppk"
Using username "user".
Offered public key
Offer of public key accepted
Authenticating with public key "imported-openssh-key"
Sent public key signature
Access granted
Opening session as main channel
Opened main channel
Allocated pty (ospeed 38400bps, ispeed 38400bps)
Started a shell/command
Edit 2:
putty.exe -v username#hostname -i "c:\path\to\id_rsa.ppk" throws a timeout error.
It seems that you didn't "select" the key on the Connection > SSH > Auth page.
You have rather typed key name there only, without a path. So PuTTY cannot find it.
You need to use a full path to the key file, like C:\path\to\id_rsa.ppk.
Following this link:
You used Putty Private Key format (.ppk)
Convert your ppk to Openssh Key File (id_rsa)
And here is an explanation how to do it. Hope it helps.
I have recently changed my GitHub username and when I try to push with Authentication > Use local SSH agent selected, GitKraken push fails with following error:
Push Failed
Configured SSH key is invalid.
Please confirm that it is properly associated with your git provider.
Unselecting the option reveals proper private/public key pair located at
~/.ssh/id_rsa
~/.ssh/id_rsa.pub
and allows the push to be done.
I can also push from inside Eclipse, and via command line without error. The URL of the repository point to my new username.
So far I have checked several things:
ssh -T github.com
gives permission denied
Fingerprint of id_rsa.pub matches to a key existing in Github account.
The following command suggests that ssh tool tries rsa private key, but is rejected:
$ ssh -vT github.com
debug1: Offering RSA public key: /home/<user>/.ssh/id_rsa
debug1: Authentications that can continue: publickey
debug1: Trying private key: /home/<user>/.ssh/id_dsa
debug1: Trying private key: /home/<user>/.ssh/id_ecdsa
debug1: Trying private key: /home/<user>/.ssh/id_ed25519
debug1: No more authentication methods to try.
Permission denied (publickey).
What is causing ssh -vT to fail in this case, and why doesn't it prevent git push/pull but fails when GitKraken tries pushing?
For me this was caused by an upgrade to OSX Sierra. I had to execute
ssh-add -k ~/.ssh/id_rsa
(you might have to replace that with the path of the key you are using). The command adds the ssh key to the git agent. For more information see the GitHub Help Page about adding SSH Keys.
You might have to add the command to some sort of setup file which is run at boot, because it seems like the ssh agent doesn't remember the key (at least for me).
For people who will come here in the future, my case was that pull/push working on vsc but gitkraken was giving me the message : no auth sock variable.
I went into file > preferences > SSH > unmarked use local SSH agent then it automatically found my id_rsa, ida_rsa.pub (you can browse and locate them manually if needed)
and now everything working fine.
OS: Arch,
gitkraken version: 7.7.2
What helped me resolve the issue was to go to GitKraken preferences and connect both Bitbucket and Github and authorize them.
After that tick the box in "general" that says use local ssh agent. (might be
If you have already set-up your local SSH agent, and your Git in terminal works, but Gitkraken does not, this might help:
First, start gitkraken from your terminal, instead of a desktop shortcut:
Open your favorite terminal
Run gitkraken
See if "Pull" works, or any other authenticated operation
It it works, you need to edit your Gitkraken desktop shortcut:
In my scenario, this is where my desktop shortcut is located, so: sudo vim /usr/share/applications/GitKraken.desktop
Change Exec=gitkraken %f to Exec=env SSH_AUTH_SOCK="${XDG_RUNTIME_DIR}/ssh-agent.socket" gitkraken %f