I'm trying to deploy a project using Capistrano. My development machine is running Windows 7; the server I'm deploying onto is Linux.
In the deploy.rb script, the following is set:
ssh_options[:forward_agent] = true
The Capistrano script starts off by running a git command locally, at which point I'm prompted for the passphrase for my SSH key:
* executing `deploy:update_code'
executing locally: "git ls-remote git#github.com:pathto/gitproject.git develop"
Enter passphrase for key '/c/Users/Sam/.ssh/id_rsa':
command finished in 6999ms
(I've removed the real git path from the above as you can probably see.)
I enter my passphrase and this works fine. However, Capistrano then tries to do something similar on the remote machine (renamed to staging-server.com in the below) and I get an error -- see the final line of this extract:
[staging-server.com] executing command
[staging-server.com] sh -c 'git clone git#github.com:pathto/gitproject.git /home/perstest/releases/20120412074500 && cd
/home/perstest/releases/20120412074500 && git checkout -b deploy 50eaf06d06d66fd20c3e55038276f420d8c308a8 && (echo 50eaf06d06d66fd20c3e55038
276f420d8c308a8 > /home/perstest/releases/20120412074500/REVISION)'
** [staging-server.com :: out] Initialized empty Git repository in /home/perstest/releases/20120412074500/.git/
** [staging-server.com :: err] Error reading response length from authentication socket.
If I comment out the forward_agent line in deploy.rb, this install succeeds, presumably using the key from the deployment server itself.
My SSH key seems to be working - after all, the initial git command works, I can use it in git push/pull and ssh -T git#github.com reports what it should (as recommended on github:help).
I assumed that my machine was not serving keys when asked for them by the remote server. After an awful lot of Googling, I found the recommendations at http://hustoknow.blogspot.co.uk/2011/06/ssh-agent-and-agent-forwarding-on.html, which explain how to get ssh-agent running on startup. I was doing all this from the command-prompt, so I then switched to git-bash and set up my .bash_profile for that to include the script I've linked to.
After adding this, I was prompted for my passphrase when first launching git bash and I could then connect to github using the SSH key (without having to re-enter passphrase). Furthermore, ssh-add -L reported that there was indeed a key available. So everything seems to be set up correctly!
However, I still get the "Error reading response length from authentication socket." error.
Any ideas? This works for my colleague (same deploy.rb file) whose development machine is Linux.
This is a bug from net-ssh as described here
Related
Have you seen and do you have a solution for or investigation ideas for this problem:
jenkins pipeline jobs using Pipeline from GitHub SCM intermittently hang on clone/checkout
nothing reported in jenkins logs about this situation
only seems to happen when we have a lot of activity (usually right before sprint demos)
clears after some period of time - haven't been able to detect much a pattern yet
Checking out git git#github.com:MyTeam/myproject into
/var/lib/jenkins/jobs/myproject-dev-deploy/workspace#script to read deploy/dev-Jenkinsfile
git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
git config remote.origin.url git#github.com:MyTeam/myproject # timeout=10
Fetching upstream changes from git#github.com:MyTeam/myproject
git --version # timeout=10
using GIT_SSH to set credentials Deploy key for myproject
git fetch --tags --progress git#github.com:MyTeam/myproject +refs/heads/:refs/remotes/origin/```
A horrible intermittent problem in a java app? To the JVM settings we go. Luckily CloudBees wrote a great article on this Tuning Jenkins GC For Responsiveness and Stability with Large Instances
Turns out this wasn't the prime factor. The real cause was network configuration and impact of a backup job on saturation. Still, an excellent article.
Here's how we debugged it:
Github support gave us this https://github-debug.com/ script and we augmented it thus
`
#!/usr/bin/env bash
echo Clearing Previous Run
rm -rf /tmp/debug-repo-http
rm -rf /tmp/debug-repo-ssh
echo clone1: `date`
time git clone https://github.com/github/debug-repo /tmp/debug-repo-http
echo clone2: `date`
time git clone git#github.com:github/debug-repo /tmp/debug-repo-ssh
echo Ping: `date`
time ping -c 10 github.com
#echo traceroute: `date`
#traceroute github.com
Setup jenkins job to attempt checkout, fail and report if it took longer than 30s - once via pipeline + plugin and once via shell script
Began running various tests: dump all docker images and re-pull, s3 pull backup, s3 push backup etc.
Repeat tests in another network area and compare results
I had meet this issue by git clone command and it's cost me 2 days to resolved this problems.
If you run git command by windows shell/batch on jenkins, it maybe cause by git ssh key issue...
Jenkins windows shell or batch command will use 'system' account to process git command and its ssh key setting is different form your gerrit plugin or login account.
A. Different git ssh key path
The git ssh key of 'system' account is setting in following path:
C:\Windows\System32\config\systemprofile\.ssh
C:\Windows\SysWOW64\config\systemprofile\.ssh
You can check ssh key in these paths for system account.
B. Register key in know_host file
Jenkins will not auto import new key of known host to your know_host file.
You can do it mannually by following steps:
Login your local account in windows
SSH this git remote path directly ssh <git remote web site> -p <port>, type yes to register the key
Copy the registered key form local account know_host (C:\Users\.ssh) you will see the bottom of file likes:
[gerrit.cartography.com]:29168,[10.25.174.78]:29418 ssh-rsa AAAAB3NzaC1yc2EAASSSSAQABAAAAgQDPHbDKCHL3oPbzDn3qaVZR8S1EZx/rb500Th/fjQYg5jrJ7PBMVVUyNTpJ2moje5hLYjNsVtXtcoNEZjImGul2dl3f+eHrCB27X5198jr8Wj66I/xsQCLEngd7l3velsdfgd3432fsdDiFaE9INjDjgngfNzlFA+CYpFsnaorZ6XSPQ==
Paste to 'system' know_host file.
C:\Windows\System32\config\systemprofile\.ssh
C:\Windows\SysWOW64\config\systemprofile\.ssh
I think you will resolve your problem now !
Enjoy it!
Anybody has any problems deploying with Laravel's envoy when using private Github repos?
When manually cloning my repo from the production server, the ssh key seems to be accessible but when using Envoy, I always get a "Permission denied (publickey) error.
Thanks
It is probably because the ssh key on your remote server requires a password.
If you change the Envoy.blade.php to perform some other task you should be able to establish whether you are connecting to your remote correctly.
#servers(['web' => 'user#domain.com'])
#task('deploy')
cd /path/to/site
git status
#endtask
Should return something like:
[user#domain.com]: On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
If you are connecting using a Mac or Linux you probably don't have to enter your password because your terminal is using ssh-agent which silently handles your authentication.
Wikipedia article on ssh-agent
When connecting over ssh, ssh-agent isn't running and the script is being prompted for a password which is where it is failing.
To get around this you could to generate a new key on the remote machine that doesn't use a password.
If you want to restrict the ssh key to a single repository on GitHub have a look at deploy keys
You need to pass the -A (as per the man page it - Enables forwarding of the authentication agent connection. This can also be specified on a per-host basis in a configuration file) in you ssh string.
You will also need add your ssh key for agent forwarding (on the machine which can access the git remote which I assume be your localhost)
ssh-add -K ~/.ssh/your_private_key
Something like this
#servers(['web' => '-A user#domain.com'])
#task('deploy')
cd /path/to/site
git status
#endtask
Git remote commands should now work.
So after installing the Github plugin to Jenkins CI and attempting to add a job for my Github project to compile automatically after a Github push to master, Jenkins reports that .git/config doesn't exist.
I went to /var/lib/jenkins and created .git and the config file and chowned them to jenkins:nogroup.
The error persisted. I then went and did this in my own VPS home directory. Error persists.
I then made a home folder for the jenkins user with no shell and did the same steps. Error persists.
Which .git/config is it referring to?!
Edit:
The actual error I'm referring to is:
Failed to connect to repository : Command "git config --local credential.helper store --file=\"/tmp/git7406572387997126825.credentials\"" returned status code 255:
stdout:
stderr: error: could not lock config file .git/config: No such file or directory
Solved...switched to SSH.
Steps:
Assume control of Jenkins user. su jenkins
(You may need to set a password for it first (make it good!) sudo passwd jenkins.
Go home: cd ~.
Generate key: ssh keygen -t rsa -C "email". (No passcode)
Copy $JENKINS_HOME/.ssh/id_rsa.pub to Github.
Set Jenkins credentials to authenticate to Github using the private key.
Works fine.
Found a neat workaround here: https://groups.google.com/forum/#!topic/git-users/pyv6ldFxUio
mkdir ~/.git
git config --global user.email "m...#school.com"
https://groups.google.com/forum/#!topic/git-users/pyv6ldFxUio
here's my situation.
I'm running Cruise Control as a Windows Service and trying to get it to connect to a Mercurial Repository on BitBucket over SSH.
I'm pretty sure that everything's configured OK (PuttyGen, Pagaent, etc). I'm remoting onto the server using the same account that I am using to run the service and if I issue hg pull -b ssh://#bitbucket.org// from a command line everything works. I added -v to the ssh configuration in mercurial.ini and I can see all of the steps that are taken.
If I run CC.NET from a command prompt then it builds fine. In the console window I can see the same logging from the SSH operation.
However, if I run CC.NET as a service (using the same user account that I'm logged in on) the call to BitBucket times out. I can find no way to work out why either. The build log doesn't help and neither do ccnet.log or ccnet.trace in the temp directory. I was expecting one of them to contain the logging from the SSH operation, but they don't.
Can anyone help? Is it that running as a service prevents it from connecting to Pagaent (I've started Pagaent by adding it to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run). When I did the pull from the command line I had to OK a dialog, but only once. Is it waiting on the same dialog now that it's running as a service?
Getting close to my wits end here.
Thanks
I did get it working in the end. The trick was to create the public key without a passphrase. When running as a service the solution has to be completely non-interactive and the passphrase option with pagaent.exe just isn't.
Here are the steps:
Use PutTTygen to generate a secure key WITHOUT a passphrase. If you really do need one then you can add it to the mercurial.ini file, but defeats the point for me as it's in plain sight anyway.
Copy a mercurial.ini to two locations: C:\Windows\System32\config\systemprofile and C:\Windows\SysWOW64\config\systemprofile. Probably only one of these was really necessary, but I didn't have the time to experiment. The first is the home directory for the system user when running 64 bit apps, the SysWOW64 location for 32 bit. Make sure that if you do the same as me then keep both files in sync - or go one further and work out which is the correct location.
Add something like this line under the [ui] key in both files:
ssh = "D:\Program Files\TortoiseHg\TortoisePlink.exe" -ssh -2 -C -batch -v -i "[Path to your ppk file]"
Add the passphrase to the end of the command if one was created in step 1.
Make sure that TortoisePlink.exe is specified, not Plink.exe. They should both be in the same directory.
Download psexec from http://technet.microsoft.com/en-gb/sysinternals/bb842062.aspx
Run d:\PSTools\PsExec.exe -s -i cmd.exe. This will open a command line as the system account in interactive mode.
Now do an hg pull, or hg clone or whatever.
A dialog should pop up with a confirmation message. This is a one time thing and the reason that you have to do the PsExec step. OK the dialog.
Now cc.net should be able to be run as a service under the local system account using SSH!
Here is my setup:
Jenkins is running on my linux machine as 'jenkins' user.
I have generated a ssh key-pair as described in Linux - Setup Git, for the 'jenkins' user.
When I sudo su jenkins and try ssh -vT git#github.com, I am always asked my passphrase, but I am always eventually authenicated. (the verbose option shows which key is used, among others).
I could clone my repo from GitHub using jenkins:
Thusly:
jenkins#alpm:~/jobs/test git/workspace$ git pull
Enter passphrase for key '/var/lib/jenkins/.ssh/id*_rsa':
Already up-to-date.
Up to this point I have followed the instructions to the letter. The problem is that the Jenkins job fails with the following error:
status code 128:
stdout:
stderr: Permission denied (publickey).
fatal: The remote end hung up unexpectedly
This is same error as I get when I typo the passphrase (but of course, Jenkins does not ask me for the passphrase). The following pages:
GitHub - SSH Issues
Using SSH Agent Forwarding
indicate to me that ssh-agent could help remember the passphrase, which it does when I am using my own user, but not the jenkins id. Note that while running as my normal user yields:
echo "$SSH_AUTH_SOCK"
/tmp/keyring-nQlwf9/ssh
While running the same command as my 'jenkins' yields nothing (not even permission denied)
My understanding of the problem is that the passphrase is not remembered.
Do you have any idea?
Shall I start a ssh-agent or key ring manager for the jenkins user? How?
Or is ssh forwarding suitable when forwarding to the same machine?
Any brighter idea?
ps: I never sudo gitted, I always used jenkins or my user account (as mentioned in this SO post - Ubuntu/GitHub SSH Key Issue)
Since nobody wrote the answer from the comments for several months, I will quickly do so.
There are 2 possible problems/solutions:
id_rsa created with wrong user
Create id_rsa as the jenkins user (see hudson cannot fetch from git repository)
Leave passphrase empty
To summarise what must be done on the Jenkins server:
# 1. Create the folder containing the SSH keys if necessary
if [ ! -e ~jenkins/.ssh ]; then mkdir ~jenkins/.ssh; fi
cd ~jenkins/.ssh/
# 2. Create the SSH pair of keys
# The comment will help to identify the SSH key on target systems
ssh-keygen -C "jenkins" -f ~jenkins/.ssh/id_rsa -P ""
# 3. Assign the proper access rights
chown -R jenkins ~jenkins/.ssh/
chmod 700 ~jenkins/.ssh
chmod 600 ~jenkins/.ssh/*
Remember:
Please keep the default "id_rsa" name when generating the keys, as other such as "id_rsa_jenkins" won't work, even if correctly set up.
Do not use a passphrase for your key
Check that the public key (id_rsa.pub) has been uploaded on the git server (GitHub, Bitbucket, etc). Once done, test your SSH key by running: ssh -vvv git#github.com (change address according to your git server)
I got around this problem by simply leaving the passphrase empty when creating the keys.
I would add that if you created the keys by hand, they might still be owned by you and not readable by jenkins, try:
sudo chown jenkins -R /var/lib/jenkins/.ssh/*
To check are the following:
if the right public key (id_rsa.pub) is uploaded to the git-server.
jenkins user will access to github -> to CHECK if the right private key (id_rsa) is copied to /var/lib/jenkins/.ssh/
if the known_hosts file is created inside ~/.ssh folder. Try ssh -vvv git#github.com to see debug logs. If thing goes well, github.com will be added to known_hosts.
if the permission of id_rsa is set to 755 (chmod 755 id_rsa)
After all checks -> try ssh -vvv git#github.com
Dont try to do config in jenkins until ssh works!
If you are running jenkins as a service in windows, you need to verify the user running the service. If you created the keys using the user "MACHINENAME\user", change the service so the user running it can match
For Mac users, the issue can be solved by removing the existing keys and creating new Private and Public Keys by following these steps:
1.Remove all Public and Private keys located at /Users/Username/.ssh
2.Remove all the credentials saved under the Credentials tab in Jenkins.
3.Remove the existing Public SSH keys defined in the Github Repository Settings.
4.Create new SSH keys (private and public: id_rsa and id_rsa.pub) by following the steps from https://confluence.atlassian.com/bitbucketserver/creating-ssh-keys-776639788.html#CreatingSSHkeys-CreatinganSSHkeyonLinux&MacOSX
5.Set the newly created public SSH key (id_rsa.pub) in Github or an equivalent Repository Settings.
6.In Jenkins,create new credentials by adding the private SSH key(id_rsa) for your Github username.
7.The Error should be removed now.
keys need to generated from jenkins user.
sudo su jenkins
ssh-keygen
once the key is generated, it should be added as ssh key in bitbucket or github.