What I have done so far:
Deployed a tomcat6.0 server to c:\www
Downloaded and deployed the Hudson.war in the c:\www\webapps folder
Installed the Github plugin
Created a private repository on Github
On the server, with hudson installed, generated ssh keys.
Environment variable %HOME% is set to c:\Documents and settings[username] (there is a .ssh directory in there with the keys)
Environment variable %HUDSON_HOME% set to c:\www\webapps\hudson
In hudson I have the following configurations:
Github Project: https://github.com/[my organization]/[project name]
Source Code Management: Git
URL of Repository: git#github.com:[my organization]/[project name].git
Branch specifier: **
Repository Browser: (Auto)
When I run a build, and click the console output link I see this --
Started by user anonymous
Checkout:workspace / C:\www\webapps\hudson\jobs\[project name] (git)\workspace - hudson.remoting.LocalChannel#2e8f6d20
Using strategy: Default
Checkout:workspace / C:\www\webapps\hudson\jobs\[project name] (git)\workspace - hudson.remoting.LocalChannel#2e8f6d20
Fetching changes from the remote Git repository
Fetching upstream changes from git#github.com:[organization name]/[project name].git
... at which point it hangs. When I cancel the build, the following are added --
ERROR: Problem fetching from origin / origin - could be unavailable. Continuing anyway
ERROR: (Underlying report) : Error performing command: git.exe fetch -t git#github.com:[organization name]/[project name].git +refs/heads/*:refs/remotes/origin/*
null
ERROR: Could not fetch from any repository
FATAL: Could not fetch from any repository
hudson.plugins.git.GitException: Could not fetch from any repository
at hudson.plugins.git.GitSCM$3.invoke(GitSCM.java:796)
at hudson.plugins.git.GitSCM$3.invoke(GitSCM.java:754)
at hudson.FilePath.act(FilePath.java:756)
at hudson.FilePath.act(FilePath.java:738)
at hudson.plugins.git.GitSCM.gerRevisionToBuild(GitSCM.java:754)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:540)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1180)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:506)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:422)
at hudson.model.Run.run(Run.java:1362)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:145)
First off, since you are starting off with a new CI instance, I would highly recommend that you install the Jenkins fork instead (since it is actively maintained by most of the original developers of Hudson).
Secondly, install the DumpInfo Wrapper plugin and re-run the build again. This plugin prints the system properties and environmental variables in effect during the build and allows you to verify them.
Update:
That plugin should log the system properties and environmental variables, I am surprised you do not see them. As for the passphrase, I suggest you generate a separate private key (from your existing pair) that is not passphrase-protected instead, otherwise you may need to consider the solution offered for: Why git can't remember my passphrase under Windows. I have verified that it works (when I set up my own CI on windows) but I don't feel it is worth the trouble (there are other nuances including installing and running the tomcat instance as the currently logged in user, and NOT as local service, so that pageant will work with it properly), hence I would recommend the first option.
In my case it turned out to be the problem with Git client: I was using v1.6.0 which was triggering
fatal: https://github.com/dmak/jaxb-xew-plugin.git/info/refs download error - The requested URL returned error: 403
error message in Hudson. First it looked like this issue, but strace analysis of git run shown that is was Nginx WebServer (which GitHub is running on) returning 403, not the proxy.
When I've updated to v1.7.3 the problem was gone. So general advise: don't use old clients with GitHub.
P.S. I have tested the cloning both with "Branch Specifier (blank for default): origin/master" in Hudson job configuration and also with blank (default): in both cases Git was able to discover the master branch (origin/master) correctly and used it for cloning.
Related
We have a Flutter app which uses a private repository as a dependency.
The SSH key has been added to CircleCI, and the remote access to the repository works just fine locally using this same key.
The config has been added to the .circleci/config.yml:
- add_ssh_keys:
fingerprints:
- "84:1a:so:me:ke:y:14:31:0f"
But CircleCI keeps failing to access the private dependency repo , giving the following error:
" Running "flutter pub get" in project... Git error. Command: git clone --mirror git#bitbucket.org:our_account/priv_repo.git /home/circleci/development/flutter/.pub-cache/git/cache/priv_repo-3456accd54b38ec5b3820944f77e90ce2ddc9887
stdout: stderr: Cloning into bare repository
'/home/circleci/development/flutter/.pub-cache/git/cache/priv_repo-3456accd54b38ec5b3820944f77e90ce2ddc9887'...
Warning: Permanently added the RSA host key for IP address
'18.205.93.1' to the list of known hosts. Unauthorized fatal: Could
not read from remote repository.
Please make sure you have the correct access rights and the repository
exists. exit code: 128 "
Has anyone successfully created a CircleCI deployment which includes a private repo dependency?
What might possibly be missing to cause this issue?
OK - there were a couple of things I had wrong,
The main one was that I had the add_ssh_keys line in the wrong place.
It really needs to be the first step, or at least be before the flutter/install_sdk_and_pub step.
eg. This works (but if the add_ssh_keys step was at the bottom of the list of 4 steps here then it fails):
steps:
- add_ssh_keys:
fingerprints:
- "84:1a:so:me:ke:y:14:31:0f"
- checkout
- aws-cli/setup:
profile-name: example
- flutter/install_sdk_and_pub:
flutter_version: 2.5.3
In addition to that it is worth noting that I added my SSH key as an "Additional SSH Keys" type key (adding a "User Key" broke the deploy) (that is under Project Settings > SSH Keys),
and I set the Hostname for the key to "bitbucket.org".
So CircleCI is now successfully pulling in my private repo dependency.
It is failing on versioning mismatch stuff, but that is another issue, and shall be solved at another time.
I want to clarify some things first.
The main one was that I had the add_ssh_keys line in the wrong place.
It really needs to be the first step, or at least be before the flutter/install_sdk_and_pub step.
Yes, it's because each step runs in a different session, so they don't share the configs from each other. Your ssh-key only works because the environment variables defined on the Project Settings, are used across all the steps.
eg. This works (but if the add_ssh_keys step was at the bottom of the list of 4 steps here then it fails):
Yes, and if you want to use this fingerprint in another step, you need to paste it there too.
There are some things that you can do to fix your problem, you can leave the main ssh-key on the Project Settings and you can create an environment variable with the public key content (from another ssh-key).
After that, before using that second key, you need to run some commands like:
- run:
name: Step I'm using the second key
command: |
echo -e $MY_ENV_VAR | base64 -d > key
chmod 400 key
eval $(ssh-agent -s)
ssh-add key
git clone...
docker build...
Doing that, you will be able to clone the second repo.
Two things that worked for me:
Adding checkout step before updating the submodules
Adding the User public key(from CircleCI) to Bitbucket both repositories,the submodule repository and the parent repository that uses the submodule.
Checkout
As mentioned here https://circleci.com/docs/configuration-reference#checkout:
Note: CircleCI does not check out submodules. If your project requires submodules, add run steps with appropriate commands as shown in the following example:
- checkout
- run: git submodule sync
- run: git submodule update --init
User Key
The "User Key" public key is not added automatically to Bitbucket. Here is an explanation on how to add it https://circleci.com/docs/bitbucket-integration#create-a-bitbucket-user-key.
After you copy the key from the "Developer Tools" you need to add it to "Access keys" in Bitbucket.
I want to use Weblate to help with the localization of an open source project that is developed on GitHub. I don't have write access to the repositories of the maintainers and have to submit pull requests for the improvements I make. It look as if Weblate supports creating pull requests, but I can't get it to work.
What I've done so far:
Run Weblate 3.4 (in a Docker container) and setup the SSH keys for GitHub access.
Set Docker environment variable WEBLATE_GITHUB_USERNAME to my GitHub username.
Opened a shell session in the Docker container and run hub clone octocat/Spoon-Knife (both as root and the weblate user; this asks for username and password and the clones the repository; the token is also still there after I restart the Docker containers)
Set up a Weblate project.
Set up a component in the project.
Source code repository = upstream GitHub repo (git#github.com:maintainer/repo)
Repository branch = master
Repository push URL = my forked GitHub repo (git#github.com:me/repo)
Version Control System: GitHub (EDIT: added on Feb 10 2019)
EDIT (Feb 10 2019):
After finding out that you have to specify "GitHub" as Version Control System when creating a component, Weblate now tries to push changes to the fork and create a pull request. Authentication of the hub tool doesn't work. When I click on "push" in Repository Maintance I get an error. Somewhere in the stack trace there is this error:
'github.com username: github.com password for (never stored): '
'Error forking repository: Unauthorized (HTTP 401)\nRequires authentication'
I strongly suspect that Python can't open ~/.config/hub. The file definitely exists. Maybe this has to to with how ~ is resolved by Python?
I've figured it out by myself:
The problem is that '~/.config/hub' is resolved to '/root/.config/hub' when Weblate calls the GitHub hub program. However, the user Weblate is running as is weblate, so it doesn't have access to the root's home directory.
I've fixed this by hardcoding '/home/weblate/.config/hub' as the location of the Hub's config file and by making sure the config file is created at this location.
Introduction:
Main Repo: Repo1.
After forking it, my repo is : ForkedRepo1
In Eclipse-Git, I cloned my ForkedRepo1.
Until Thursday, I was able to commit & push without any issue.
Then suddenly, on Thursday, I got the error:
Cannot get remote repository refs
Reason:
ssh://<forkedrepo1: session.connect:
java.netSocketTimeoutException: Read timed out
I have a SSH key.
Nothing as altered in my system, so I am not sure what happened.
To make matters worse, I deleted my Remote(origin) in eclipse-git and when I try to add it back, I get the same error - java.net.SockettimeoutException: Read timed out.
However, via git bash, I can commit a file and push it to the local branch in GitHub.
I hope this info is enough to give me some idea.
FYI: I am working on a existing project, so when I push a file up for review, it worked successfully. It is just that I cannot now do it in eclipse.
Then try, as described here, to switch to an HTTPs URL.
You can then cache your GitHub credentials (GitHub username/password) in a credential helper, generally already configured (see git config credential.helper)
It is probably caused by the removal of weak cryptographic standards.
You need a recent Java 8 revision (i.e. 1.8.0_161), a recent version of EGit (such as 4.10.0.201712302008-r) and you probably need also to restart (just once) Eclipse with the -clean parameter.
When I am trying to give the Git URL in Jenkins source code management, I am getting this error:
Failed to connect to repository : Error performing command: git.exe ls-remote -h https://github.com/shivnathr/DevOPS1 HEAD
How can I avoid it?
Make sure your Jenkins is running as your account, and not as a service with a technical account.
That way, Jenkins will benefit from the same configuration as your account.
But first, check if you can, with your account, in a simple CMD shell session, execute that same command successfully:
git.exe ls-remote -h https://github.com/shivnathr/DevOPS1 HEAD
When you configure this in Jenkins, it gives this error. But if you ignore this error, does it then work for the build? Have you configured git tools in Jenkins? I am not sure if having git in the path for the user running Jenkins will work (it might), but if you configure a git tool, it should use the first tool in the list to find git. BUT, if the first tool is set to auto-install, then this problem can occur. Make sure the first git tool is just pointing to an installation on the machine. I believe this is a bug. I found this on one of my installations, but I have yet to open an issue for it.
I tried to follow the instructions in this article:
http://developer.cloudbees.com/bin/view/DEV/GitHub+Commit+Hooks+HOWTO
But I have two problems:
It doesn't work. When I push to github nothing happens on the jenkins side. On Jenkins the GitHub Hook log says 'Polling has not run yet.'
The possible cause for this is that I can't specify the git repository as described in this image:
There is no textfield for a public key, nor the other fields. Instead I have the fields
Repository URL, Name, Refspec
Also I can specify a http url (and everything but the triggering works). But if I specify the git/ssh url as in the image I get the following error:
Failed to connect to repository : Command "git ls-remote -h
git#github.com:schauder/degraph.git HEAD" returned status code 128:
stdout: stderr: Warning: Permanently added
'github.com,207.97.227.239' (RSA) to the list of known hosts.
Permission denied (publickey). fatal: Could not read from remote
repository.
Please make sure you have the correct access rights and the repository
exists.
I copied the url from github so a typo is somewhat unlikely.
I verified that I have the github plugin installed (upgraded to version 1.5)
Any hints how to fix or at least debug the problem?
Update answering questions in the comments:
Yes I checked the checkbox 'Build when a change is pushed to GitHub' in the job configuration.
And Yes I setup the webhook in GitHub. When I click the test button there, nothing happens. When I put the url from the webhook in my browser I get a http 500 saying
Not intended to be browsed interactively (must specify payload parameter)
Plus some lengthy stacktrace.
Another Update
I checked the GIT related plugins installed on the Jenkins side. Everything showing their with GIT in the text is installed and at the most current version
Based on Git error message above, I assume you can't be authenticated. Did you add the CloudBees public SSH key on GitHub as described here?
Clarification by OP
The image referenced in the question seemed to suggest that one puts a public key from GitHub into CloudBees. But actually it is the other way round, and also all the labels seem to be changed, so it is kind of difficult to match that screenshot to what you see in CloudBees.
So what I did is this:
in the Job configuration at Cloudbees there is a Public Key (generated by CloudBee). I it in the CloudBees DEV#cloud Authorization section and labeled CloudBees Public Key. Copy that Key.
Hop over to GitHub and go to Settings > Deploy Keys. Add the key there.
go to Settings > Service Hooks > WebHook URLs. Click on the test button. You are not going to see much on the GitHub side.
Hop back to CloudBees Jenkins. From the main page of the job go to the ´GitHub Hook Log´. There you should see something like:
Started on Mar 15, 2013 8:55:14 AM
Using strategy: Default
[poll] Last Built Revision: Revision 581acde89a62317fd8dabaa3f4c6025d1c9dd413 (origin/master)
Fetching changes from the remote Git repositories
Polling for changes in
Done. Took 0.34 sec
Changes found
And your job should start.
If the last line says No Changes this means the hook worked, but you already build the current version so the build is not triggered again.
If you still have problems, have look at https://wiki.jenkins-ci.org/display/JENKINS/Github+Plugin#GitHubPlugin-TriggerabuildwhenachangeispushedtoGitHub under 'trouble shooting' it tells you how to set up logging on the jenkins side. With this I get all kind of stuff in the Jenkins SystemLog.
I've just published a tutorial on continuous integration for Play applications hosted on GitHub using CloudBees. This is intended to provide a comprehensive guide:
http://ics-software-engineering.github.io/play-example-continuous-integration/