Setting up GitHub webhook with Jenkins on EC2 - github

I have read through almost all of the guides by now on how to set up Jenkins with GitHub, but none of them seem to solve my problem. My job currently builds fine, but I want it to build every time I push to GitHub. I also want to manually set up my hook URLs (not let Jenkins auto-manage them). Currently my Jenkins server is running on EC2 on port 8080 (http://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:8080/). The sys admin says the server is set up to listen to all incoming requests on that port.
Under the GitHub Web Hook area of the Configure System area, I have Manually manage hook URLs checked off.
In my project's configuration, I have the following setup:
GitHub Project: https://github.com/username/projectname/
Source Code Management > Git
Repository URL: git#github.com:username/projectname.git
Branches to Build - Branch Specifier (blank for default): master
Build Triggers > Build when a change is pushed to GitHub
In the GitHub Hook Log it displays the message:
Last GitHub Push
Polling has not run yet.
I have also created a user with which GitHub can connect. The user has a role with the following global permissions:
Overall: Read
Job: Read
Job: Discover
Job: Build
On GitHub, in Settings > Service Hooks I have added https://USERNAME:PASSWORD#ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:8080/github-webhook/, where USERNAME and PASSWORD are the username and password for the GitHub account. I added the link to BOTH the WebHook URLs area and the Jenkins (GitHub plugin) area. When I click Test Hook under former area no message of confirmation shows up on the GitHub page, but when I click Test Hook under the latter it says Payload Deployed. Both, however, are registering in my System log file, and they are generating similar errors:
winstone.WinstoneException: Error URI Line: <Garbled Text Here>
at winstone.HttpListener.parseURILine(HttpListener.java:349)
at winstone.HttpListener.parseURI(HttpListener.java:258)
at winstone.RequestHandlerThread.run(RequestHandlerThread.java:83)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
Jun 7, 2013 6:16:31 PM winstone.Logger logInternal
SEVERE: Error within request handler thread
Any ideas about what could be causing this?

If this does not appear in Winstone’s access log, and the problem persists without SSL, try nc -l -p 8080 to see what the actual request being sent from GitHub is. (Or try a different servlet container.)

Related

Github enterprise webhook - localhost not recheable over Internet error

After upgrading githubEnterprise from 2.19.4 to 3.0.25 one issue appeared.
When I try to create git webhooks with POST request from my local instance or from my CI machine I receive response from git:
{"message":"Validation Failed","errors":[{"resource":"Hook","code":"custom","message":"Sorry, the URL host localhost is not supported because it isn't reachable over the public Internet"}],"documentation_url":"https://docs.github.com/enterprise/3.0/rest/reference/repos#create-a-repository-webhook"}
I know the root cause - creating webhook with localhost callback, but I would like to keep it as it worked before.
I've tried to find some setting/property to set in the config to force github instance to accept that url but I haven't found anything.
Do you know how to tell github to accept this callback urls or any workaround?I don't want to change my tests and CI. I want to configure github that way.
P.S. I've found POST with explanation this behaviour but there is no solution.

SCM Sync configuration plugin - Jenkins GIT prompt issue

Jenkins version: 1.642.3
SCM Sync Configuration Plugin version: 0.0.10
I created a private repository for storing Jenkins job's configuration (which is the git url that I'll use in Jenkins's Global configuration page while configuring this plugin). As I created a private repo (which is not open to anyone), I had to add any collaborators (jenkins service user which is running Jenkins master process on the Jenkins master server). If in your case, it's different, look for user name user.name under Manage Jenkins > System Information for the user which is running Jenkins process otherwise, you'll get an error that OS user doesn't have access to the repository or permission denied (Public Key). One other solution is to generate ssh key (ssh-keygen -t rsa and once the Key files are generated, putting the .pub public key in your Github user's SSH settings section: https://git.instance.yourcomany.com/settings/ssh).
One Important Note: While configuring the logger, if you accidentally pick
hudson.plugins.scm_sync_configuration.ScmSyncConfigurationPlugin as it shows up in the list of available options (vs using the correct value which is hudson.plugins.scm_sync_configuration) then logger will NOT log anything.
The above setting will make this plugin not log anything in the logger, so make sure you set the logger to the correct value.
AFTER this, everything is looking good so far in Jenkins main Global configuration page for configuring this plugin (no errors there) and at the right bottom of Jenkins home page, I can see "SCM Sync status : Last operation # Tue Jul 25 11:55:43 PDT 2017"
I created a dummy job (and did echo something && sleep 60 in the build step) and tried to make few changes to this build step for changing the sleep time to some random values.
My expection was I should have been prompted for making any changes to the Jenkins job and that Jenkins job should be checked-in in the given GIT repo (as now the pub key for jenkins user is all set). PS: I'm succesfully able to clone from this repo in another test jenkins job (running on master node) + I'm also able to successfully git clone + git push to this repo from Jenkins master server (just added README.md myself as jenkins user). In the logger's log, I can see:
Jul 25, 2017 11:53:37 AM FINE hudson.plugins.scm_sync_configuration.SCMManipulator checkinFiles
Checking in SCM files ...
Jul 25, 2017 11:53:37 AM FINE hudson.plugins.scm_sync_configuration.SCMManipulator checkinFiles
Checked in SCM files !
Jul 25, 2017 11:53:37 AM FINEST hudson.plugins.scm_sync_configuration.ScmSyncConfigurationBusiness processCommitsQueue
Commit Commit hudson.plugins.scm_sync_configuration.model.Commit#5d951ea :
Author : asangal
Comment : asangal: Jenkins configuration files updated
Change performed by asangal
Changeset :
A hudson.scm.SubversionSCM.xml
A hudson.plugins.ansicolor.AnsiColorBuildWrapper.xml
A hudson.plugins.git.GitTool.xml
A hudson.scm.CVSSCM.xml
A hudson.tasks.Mailer.xml
A hudson.plugins.gradle.Gradle.xml
A hudson.tasks.Ant.xml
A hudson.plugins.git.GitSCM.xml
A hudson.plugins.groovy.Groovy.xml
A jenkins.plugins.publish_over_ssh.BapSshPublisherPlugin.xml
A hudson.triggers.SCMTrigger.xml
A hudson.maven.MavenModuleSet.xml
A jenkins.mvn.GlobalMavenConfig.xml
A jenkins.model.ArtifactManagerConfiguration.xml
A hudson.tasks.Shell.xml
A scm-sync-configuration.xml
A hudson.plugins.copyartifact.TriggeredBuildSelector.xml
A config.xml
A hudson.tasks.Maven.xml
A jenkins.metrics.api.MetricsAccessKey.xml
A hudson.plugins.timestamper.TimestamperConfig.xml
A jenkins.model.JenkinsLocationConfiguration.xml
A hudson.plugins.build_timeout.operations.BuildStepOperation.xml
A hudson.plugins.mercurial.MercurialInstallation.xml
A hudson.plugins.emailext.ExtendedEmailPublisher.xml
A hudson.plugins.throttleconcurrents.ThrottleJobProperty.xml
pushed to SCM !
Jul 25, 2017 12:49:48 PM FINEST hudson.plugins.scm_sync_configuration.ScmSyncConfigurationBusiness queueChangeSet
Queuing commit Commit hudson.plugins.scm_sync_configuration.model.Commit#5511458b :
Author : asangal
Comment : asangal: Job [dummy_job] configuration updated
Change performed by asangal
Changeset :
A jobs/dummy_job/config.xml
to SCM ...
Jul 25, 2017 12:49:48 PM FINEST hudson.plugins.scm_sync_configuration.ScmSyncConfigurationBusiness processCommitsQueue
Processing commit : Commit hudson.plugins.scm_sync_configuration.model.Commit#5511458b :
Author : asangal
Comment : asangal: Job [dummy_job] configuration updated
Change performed by asangal
Changeset :
A jobs/dummy_job/config.xml
Jul 25, 2017 12:49:48 PM FINE hudson.plugins.scm_sync_configuration.SCMManipulator checkinFiles
Checking in SCM files ...
Jul 25, 2017 12:49:48 PM FINE hudson.plugins.scm_sync_configuration.SCMManipulator checkinFiles
Checked in SCM files !
Jul 25, 2017 12:49:48 PM FINEST hudson.plugins.scm_sync_configuration.ScmSyncConfigurationBusiness processCommitsQueue
Commit Commit hudson.plugins.scm_sync_configuration.model.Commit#5511458b :
Author : asangal
Comment : asangal: Job [dummy_job] configuration updated
Change performed by asangal
Changeset :
A jobs/dummy_job/config.xml
pushed to SCM !
Questions:
1) Why I didn't get prompted when I made changes to Jenkins job?
2) Why I'm not seeing any commits (which the logger is showing successfully checked-in / pushed, in the log) to the my target git repo? My github repo just shows README.md file so far.
3) Where exactly did this plugin CHECKED-IN the files and pushed?
Resolution / steps followed to properly check and implement this plugin:
Make sure:
1) Before you setup SCM Sync configuration, you have a repository / project created in Git
2) Find the user (which is running jenkins process). Usually it's jenkins user. If required, work with your system team and have them create a service/ldap user account named jenkins so that you can add jenkins user (or whatever user running the jenkins process) in Git/Github as one of the Collaborator with write access. If that's not possible, then go to Jenkins Master server and sudo su - jenkins and make sure you create (ssh keys). Once created, grab the public key and upload the key to Github repositories under Settings > Deploy Keys.
3) For this user, create a ~/.gitconfig file and put the following contents in this file.
user]
name = enter_theuserid
email = enter_theemailid#company.com
[core]
editor = vim
[color]
ui = auto
4) Now configure SCM Sync plugin in Jenkins global settings configuration (As per the Plugin's documentation). As soon as you provide the git#git..:org/your-repo.git, you'll notice that it'll not error out by saying the OS user running Jenkins is not able to do anything.
5) Create a new LOGGER (make sure Logger value is: hudson.plugins.scm_sync_configuration)
6) Go to Jenkins master server under: $JENKINS_HOME/scm-sync-configuration/checkoutConfiguration folder. Look for .git folder. If that's not present, that means, SCM Sync plugin didn't perform any checkout. To find if there are any issues with this plugin able to check out the Git repo, you can create another new logger where the Logger value should be hudson.plugins.scm_sync_configuration.ScmSyncConfigurationPlugin from the list box available options.
7) If you didn't find .git in bullet 5th folder, then remove checkoutConfiguration folder (leaf folder only) and in Jenkins Global configuration for SCM Sync plugin, change version control from Git to None (radio button), save it. Then go back and select Git (radio button) and provide git#git...:org/repo-name.git URL again and Save.
8) Keep an eye on the logs. This time if the user (running the Jenkins process) is listed in Git repository as a valid collaborator with WRITE access OR if it's SSH pub key is uploaded to Github repository, then it'll successfully CHECKOUT the repository to $JENKINS_HOME/scm-sync-configuration/checkoutConfiguration folder. Inside this folder, you'll see all valid .git/* files.
9) For testing purpose, you can make a small change in a file under $JENKINS_HOME/scm-sync-configuration/checkoutConfiguration folder, and run git add thatfilename; git commit -m "test change" $_; git pull && git push
10) If bullet 9 succeeds, at this point you'll have a working SCM Sync plugin where when you make a change to a Jenkins job, as soon as you press SAVE button, it'll first PROMPT via a popup modal window and request the user to enter some comment.
Goto Jenkins -> Manage Jenkins -> Configure System.
Under SCM Sync Configuration section look for "Never bother me with commit messages" configuration.
If you want Jenkins to prompt for commit messages when Job has been modified, Uncheck the option "Never bother me with commit messages".

Github - Failed to clone/remote sync or clone or any github activity

I have github on Windows-7. The github doesn't seem to allow me to check in code as something is messed up.
I did try changing the credentials & so forth by looking up online but nothing seems to work.
I still see the Bad credentials error alongside some wamp developer errors.
I don't know how wamp developer is related to GitHub.
I did have WAMP developer once upon a time on the PC.
The log file for the attempt is here: Github log file.
The error:
2016-03-22 12:44:50.7329|
ERROR|thread: 5|StartupLogging| MISSING PATH!!: 'C:\WampDeveloper\Components\Apache\bin'
That simply means your %PATH% currently reference one non-existent path: you could clean-up your environment variable PATH which is currently quite large.
This is not-blocking for GitHub Desktop though.
The other error is linked to a key previously used for:
Logged user r... off of host 'https://<server_url>'
When that key "C:\Users\ffgr.ghjk\.ssh\github_rsa" is used to authenticate to github.com does not work.
Make sure that key (the public one github_rsa.pub) is added to your GitHub account: "Adding a new SSH key to your GitHub account"

How trigger Jenkins to build a project?

I have the task to run a build task whenever the source code in github is updated. However I am very new with Jenkins and I have a hard time to accomplish this.
My understanding
github.com will send a POST message to a specific URL that I specify. As an example let's use:
http://mywebsite/src-updated
So the source code get's update, github sends the POST message to mywebsite/src-updated. Since HTTP runs on port 80, Apache receives this message.
____________LAN____________
| |
| .......... .......... |
| :JENKINS : :APACHE : | POST message to: ..............
| :Listen : :Listen :<--|<----http://mywebsite/src-updated--- : github.com :
| :on 8080 : :on 80 : | :............:
| :........: :........: |
|___________________________|
My frustration
Now what?
Jenkins sits there like a loser with no-one wanting to play with him. How will Mr. Jenkins get the message? Is there some module I have to install on Apache so that it notifies him? All this sounds very different from the sparse information I read so far so I feel that I am totally off track.
I also tried to use the Github plugin but I am totally lost on how it's supposed to work (terrible documentation if you're new to the whole thing).
Any help?
Please check this link on configuring Jenkins with Apache. Besides, i also found this note on the GitHub plugin page:
Jenkins inside a firewall:
In case your Jenkins run inside the firewall and not directly reachable from the internet, this plugin lets you specify an arbitrary endpoint URL as an override in the automatic mode. The plugin will assume that you've set up reverse proxy or some other means so that the POST from GitHub will be routed to the Jenkins.
As far as running builds is concerned whenever source code in GitHub is updated, it's very simple to configure in Jenkins. There is a polling option present in the job's/project's configuration page. Go to the configuration section of the job. Search for Build Triggers section. You will find a check-box named Poll SCM. Enabling this option tells Jenkins to initiate a build as soon as it finds a change in the repository (in this case, GitHub) you specified. You will have to specify some interval after which it will check GitHub for changes:
For example,
# every fifteen minutes (perhaps at :07, :22, :37, :52)
H/15 * * * *
For more options and details on the above, don't forget to look for the help section '?'

Enable jenkins services in bitbucket account

What is the value of "END POINT URL". I am using Jenkins URL="http://localhost:8080".
Can you please explain all steps.How to we trigger Jenkins Job when we commit(PUSH) the code in repository.
Read the manual.
You will find/configure End Point in Manage Jenkins > Configure System.
it's correct change your url below
localhost replace as ip Address instead by http://198.162.1.25
Second image Source by:Jesly Varghese
To trigger a build ensure that your job can successfully checkout your project first. Then under Build Triggers check Poll SCM
In the text box, set your build trigger to fire within 3 min(or so) of the check-in. This is to reduce CPU load: