I am trying to create run my rails application on ec2 using saltstack and capistrano.
Here's what I have successfully completed so far.
Using salt cloud and salt master I am able to create a new minion instance and setup everything required for the application to run i.e. ruby, rails, unicorn, mysql etc.
I have done proper configuration for capistrano. when I try to deploy I see the following error.
DEBUG [ed84c6ab] Command: ( GIT_ASKPASS=/bin/echo GIT_SSH=/pathto/git-ssh.sh /usr/bin/env git ls-remote -h git#github.com:somehost/somerepo.git )
DEBUG [ed84c6ab] Warning: Permanently added 'github.com,ip' (RSA) to the list of known hosts.
DEBUG [ed84c6ab] Permission denied (publickey).
DEBUG [ed84c6ab] fatal: Could not read from remote repository.
DEBUG [ed84c6ab]
DEBUG [ed84c6ab] Please make sure you have the correct access rights
DEBUG [ed84c6ab] and the repository exists.
DEBUG [ed84c6ab] Finished in 12.600 seconds with exit status 128 (failed).
So this means that from my local capistrano is able to connect to the minion but when it tries to checkout git repo it fails.
I know this is happening because the ssh public key of the minion is not added to the github.
so the goal is.
run salt cloud to create instance
run salt highstate to install everything required for app
run capistrano deploy to start the application
I would like to automate github authorization process too. I mean once the minion is created the minion should be able to clone git repo without any manual intervention.
I am confused as to this can be done through capistrano or saltstack.
I used github ssh forwarding to achieve this.
Here's the changes I made.
Steps to enable ssh forwarding for github
Then in capistrano deploy.rb file configure ssh forwarding by adding forward_agent: true
set :ssh_options, {
user: 'user',
auth_methods: %w(publickey),
port: <some port>,
forward_agent: true
}
Related
Hi i'm having trouble pulling in a remote endpoint.
command to connect from remote google compute engine:
ssh -i /path/.ssh/google_compute_engine NAME#REMOTEIP "cd /home/repo/; git pull origin dev"
Connection via ssh to remote endpoint is working.
When i do manually my git pull is working but when i try to pass commands line as arguments it fail :
git#github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Also if i try to add my ssh:
ssh -i /path/.ssh/google_compute_engine NAME#IP "cd /home/repo; ssh-add ~/.ssh/rsa_pub"
it fail:
Could not open a connection to your authentication agent.
ssh -i /path/.ssh/google_compute_engine NAME#IP "ls /home; cd /home/repo; eval ssh-agent;"
return :
SSH_AGENT_PID=311; export SSH_AGENT_PID;
echo Agent pid 311;
I have to use it like this to make a script for production.
I don't find a way to do it, and also avoid entering passphrase so it will automatically pull as i launch of the command below.
Can i somehow pass my the rsa_pub as parameter ?
Servers from some providers like AWS will not log you in as root but rather as a user with limited permissions (i.e. ubuntu#ip) try adding sudo before git pull on the server.
Have you seen and do you have a solution for or investigation ideas for this problem:
jenkins pipeline jobs using Pipeline from GitHub SCM intermittently hang on clone/checkout
nothing reported in jenkins logs about this situation
only seems to happen when we have a lot of activity (usually right before sprint demos)
clears after some period of time - haven't been able to detect much a pattern yet
Checking out git git#github.com:MyTeam/myproject into
/var/lib/jenkins/jobs/myproject-dev-deploy/workspace#script to read deploy/dev-Jenkinsfile
git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
git config remote.origin.url git#github.com:MyTeam/myproject # timeout=10
Fetching upstream changes from git#github.com:MyTeam/myproject
git --version # timeout=10
using GIT_SSH to set credentials Deploy key for myproject
git fetch --tags --progress git#github.com:MyTeam/myproject +refs/heads/:refs/remotes/origin/```
A horrible intermittent problem in a java app? To the JVM settings we go. Luckily CloudBees wrote a great article on this Tuning Jenkins GC For Responsiveness and Stability with Large Instances
Turns out this wasn't the prime factor. The real cause was network configuration and impact of a backup job on saturation. Still, an excellent article.
Here's how we debugged it:
Github support gave us this https://github-debug.com/ script and we augmented it thus
`
#!/usr/bin/env bash
echo Clearing Previous Run
rm -rf /tmp/debug-repo-http
rm -rf /tmp/debug-repo-ssh
echo clone1: `date`
time git clone https://github.com/github/debug-repo /tmp/debug-repo-http
echo clone2: `date`
time git clone git#github.com:github/debug-repo /tmp/debug-repo-ssh
echo Ping: `date`
time ping -c 10 github.com
#echo traceroute: `date`
#traceroute github.com
Setup jenkins job to attempt checkout, fail and report if it took longer than 30s - once via pipeline + plugin and once via shell script
Began running various tests: dump all docker images and re-pull, s3 pull backup, s3 push backup etc.
Repeat tests in another network area and compare results
I had meet this issue by git clone command and it's cost me 2 days to resolved this problems.
If you run git command by windows shell/batch on jenkins, it maybe cause by git ssh key issue...
Jenkins windows shell or batch command will use 'system' account to process git command and its ssh key setting is different form your gerrit plugin or login account.
A. Different git ssh key path
The git ssh key of 'system' account is setting in following path:
C:\Windows\System32\config\systemprofile\.ssh
C:\Windows\SysWOW64\config\systemprofile\.ssh
You can check ssh key in these paths for system account.
B. Register key in know_host file
Jenkins will not auto import new key of known host to your know_host file.
You can do it mannually by following steps:
Login your local account in windows
SSH this git remote path directly ssh <git remote web site> -p <port>, type yes to register the key
Copy the registered key form local account know_host (C:\Users\.ssh) you will see the bottom of file likes:
[gerrit.cartography.com]:29168,[10.25.174.78]:29418 ssh-rsa AAAAB3NzaC1yc2EAASSSSAQABAAAAgQDPHbDKCHL3oPbzDn3qaVZR8S1EZx/rb500Th/fjQYg5jrJ7PBMVVUyNTpJ2moje5hLYjNsVtXtcoNEZjImGul2dl3f+eHrCB27X5198jr8Wj66I/xsQCLEngd7l3velsdfgd3432fsdDiFaE9INjDjgngfNzlFA+CYpFsnaorZ6XSPQ==
Paste to 'system' know_host file.
C:\Windows\System32\config\systemprofile\.ssh
C:\Windows\SysWOW64\config\systemprofile\.ssh
I think you will resolve your problem now !
Enjoy it!
Anybody has any problems deploying with Laravel's envoy when using private Github repos?
When manually cloning my repo from the production server, the ssh key seems to be accessible but when using Envoy, I always get a "Permission denied (publickey) error.
Thanks
It is probably because the ssh key on your remote server requires a password.
If you change the Envoy.blade.php to perform some other task you should be able to establish whether you are connecting to your remote correctly.
#servers(['web' => 'user#domain.com'])
#task('deploy')
cd /path/to/site
git status
#endtask
Should return something like:
[user#domain.com]: On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
If you are connecting using a Mac or Linux you probably don't have to enter your password because your terminal is using ssh-agent which silently handles your authentication.
Wikipedia article on ssh-agent
When connecting over ssh, ssh-agent isn't running and the script is being prompted for a password which is where it is failing.
To get around this you could to generate a new key on the remote machine that doesn't use a password.
If you want to restrict the ssh key to a single repository on GitHub have a look at deploy keys
You need to pass the -A (as per the man page it - Enables forwarding of the authentication agent connection. This can also be specified on a per-host basis in a configuration file) in you ssh string.
You will also need add your ssh key for agent forwarding (on the machine which can access the git remote which I assume be your localhost)
ssh-add -K ~/.ssh/your_private_key
Something like this
#servers(['web' => '-A user#domain.com'])
#task('deploy')
cd /path/to/site
git status
#endtask
Git remote commands should now work.
I have code repository on GitHub and CI server (Ubuntu) with Jenkins behind firewall.
So I set proxy settings to jenkins and it works fine (e.g. i can see the availvable plugins).
And I configured proxy settings in .gitconfig.
Also i have id_rsa in jenkins home/.ssh directory.
But jenkins can't perform git clone during job execution because of an error:
ssh: connect to host github.com port 22: Connection timed out
Jenkins user successfully performs proxychains git clone operation in terminal.
Is it possible to force jenkins works through proxychains? Or could anybody advice me some alternatives?
I have solved my problem wih this:
http://blog.paulbetts.org/index.php/2008/04/08/getting-ssh-to-connect-through-a-socks-proxy/
I have solved similar problem using STDIO proxy tunnel.
update ~/.ssh/config:
Host github.com
HostName github.com
User git
ProxyCommand gitproxy %h %p
create ~/bin/gitproxy script with this content:
#!/bin/sh
exec socat STDIO PROXY:proxy.com:$1:$2,proxyport=8080,proxyauth=user:password
I'm trying to deploy a project using Capistrano. My development machine is running Windows 7; the server I'm deploying onto is Linux.
In the deploy.rb script, the following is set:
ssh_options[:forward_agent] = true
The Capistrano script starts off by running a git command locally, at which point I'm prompted for the passphrase for my SSH key:
* executing `deploy:update_code'
executing locally: "git ls-remote git#github.com:pathto/gitproject.git develop"
Enter passphrase for key '/c/Users/Sam/.ssh/id_rsa':
command finished in 6999ms
(I've removed the real git path from the above as you can probably see.)
I enter my passphrase and this works fine. However, Capistrano then tries to do something similar on the remote machine (renamed to staging-server.com in the below) and I get an error -- see the final line of this extract:
[staging-server.com] executing command
[staging-server.com] sh -c 'git clone git#github.com:pathto/gitproject.git /home/perstest/releases/20120412074500 && cd
/home/perstest/releases/20120412074500 && git checkout -b deploy 50eaf06d06d66fd20c3e55038276f420d8c308a8 && (echo 50eaf06d06d66fd20c3e55038
276f420d8c308a8 > /home/perstest/releases/20120412074500/REVISION)'
** [staging-server.com :: out] Initialized empty Git repository in /home/perstest/releases/20120412074500/.git/
** [staging-server.com :: err] Error reading response length from authentication socket.
If I comment out the forward_agent line in deploy.rb, this install succeeds, presumably using the key from the deployment server itself.
My SSH key seems to be working - after all, the initial git command works, I can use it in git push/pull and ssh -T git#github.com reports what it should (as recommended on github:help).
I assumed that my machine was not serving keys when asked for them by the remote server. After an awful lot of Googling, I found the recommendations at http://hustoknow.blogspot.co.uk/2011/06/ssh-agent-and-agent-forwarding-on.html, which explain how to get ssh-agent running on startup. I was doing all this from the command-prompt, so I then switched to git-bash and set up my .bash_profile for that to include the script I've linked to.
After adding this, I was prompted for my passphrase when first launching git bash and I could then connect to github using the SSH key (without having to re-enter passphrase). Furthermore, ssh-add -L reported that there was indeed a key available. So everything seems to be set up correctly!
However, I still get the "Error reading response length from authentication socket." error.
Any ideas? This works for my colleague (same deploy.rb file) whose development machine is Linux.
This is a bug from net-ssh as described here