Jenkins version: 1.642.3
SCM Sync Configuration Plugin version: 0.0.10
I created a private repository for storing Jenkins job's configuration (which is the git url that I'll use in Jenkins's Global configuration page while configuring this plugin). As I created a private repo (which is not open to anyone), I had to add any collaborators (jenkins service user which is running Jenkins master process on the Jenkins master server). If in your case, it's different, look for user name user.name under Manage Jenkins > System Information for the user which is running Jenkins process otherwise, you'll get an error that OS user doesn't have access to the repository or permission denied (Public Key). One other solution is to generate ssh key (ssh-keygen -t rsa and once the Key files are generated, putting the .pub public key in your Github user's SSH settings section: https://git.instance.yourcomany.com/settings/ssh).
One Important Note: While configuring the logger, if you accidentally pick
hudson.plugins.scm_sync_configuration.ScmSyncConfigurationPlugin as it shows up in the list of available options (vs using the correct value which is hudson.plugins.scm_sync_configuration) then logger will NOT log anything.
The above setting will make this plugin not log anything in the logger, so make sure you set the logger to the correct value.
AFTER this, everything is looking good so far in Jenkins main Global configuration page for configuring this plugin (no errors there) and at the right bottom of Jenkins home page, I can see "SCM Sync status : Last operation # Tue Jul 25 11:55:43 PDT 2017"
I created a dummy job (and did echo something && sleep 60 in the build step) and tried to make few changes to this build step for changing the sleep time to some random values.
My expection was I should have been prompted for making any changes to the Jenkins job and that Jenkins job should be checked-in in the given GIT repo (as now the pub key for jenkins user is all set). PS: I'm succesfully able to clone from this repo in another test jenkins job (running on master node) + I'm also able to successfully git clone + git push to this repo from Jenkins master server (just added README.md myself as jenkins user). In the logger's log, I can see:
Jul 25, 2017 11:53:37 AM FINE hudson.plugins.scm_sync_configuration.SCMManipulator checkinFiles
Checking in SCM files ...
Jul 25, 2017 11:53:37 AM FINE hudson.plugins.scm_sync_configuration.SCMManipulator checkinFiles
Checked in SCM files !
Jul 25, 2017 11:53:37 AM FINEST hudson.plugins.scm_sync_configuration.ScmSyncConfigurationBusiness processCommitsQueue
Commit Commit hudson.plugins.scm_sync_configuration.model.Commit#5d951ea :
Author : asangal
Comment : asangal: Jenkins configuration files updated
Change performed by asangal
Changeset :
A hudson.scm.SubversionSCM.xml
A hudson.plugins.ansicolor.AnsiColorBuildWrapper.xml
A hudson.plugins.git.GitTool.xml
A hudson.scm.CVSSCM.xml
A hudson.tasks.Mailer.xml
A hudson.plugins.gradle.Gradle.xml
A hudson.tasks.Ant.xml
A hudson.plugins.git.GitSCM.xml
A hudson.plugins.groovy.Groovy.xml
A jenkins.plugins.publish_over_ssh.BapSshPublisherPlugin.xml
A hudson.triggers.SCMTrigger.xml
A hudson.maven.MavenModuleSet.xml
A jenkins.mvn.GlobalMavenConfig.xml
A jenkins.model.ArtifactManagerConfiguration.xml
A hudson.tasks.Shell.xml
A scm-sync-configuration.xml
A hudson.plugins.copyartifact.TriggeredBuildSelector.xml
A config.xml
A hudson.tasks.Maven.xml
A jenkins.metrics.api.MetricsAccessKey.xml
A hudson.plugins.timestamper.TimestamperConfig.xml
A jenkins.model.JenkinsLocationConfiguration.xml
A hudson.plugins.build_timeout.operations.BuildStepOperation.xml
A hudson.plugins.mercurial.MercurialInstallation.xml
A hudson.plugins.emailext.ExtendedEmailPublisher.xml
A hudson.plugins.throttleconcurrents.ThrottleJobProperty.xml
pushed to SCM !
Jul 25, 2017 12:49:48 PM FINEST hudson.plugins.scm_sync_configuration.ScmSyncConfigurationBusiness queueChangeSet
Queuing commit Commit hudson.plugins.scm_sync_configuration.model.Commit#5511458b :
Author : asangal
Comment : asangal: Job [dummy_job] configuration updated
Change performed by asangal
Changeset :
A jobs/dummy_job/config.xml
to SCM ...
Jul 25, 2017 12:49:48 PM FINEST hudson.plugins.scm_sync_configuration.ScmSyncConfigurationBusiness processCommitsQueue
Processing commit : Commit hudson.plugins.scm_sync_configuration.model.Commit#5511458b :
Author : asangal
Comment : asangal: Job [dummy_job] configuration updated
Change performed by asangal
Changeset :
A jobs/dummy_job/config.xml
Jul 25, 2017 12:49:48 PM FINE hudson.plugins.scm_sync_configuration.SCMManipulator checkinFiles
Checking in SCM files ...
Jul 25, 2017 12:49:48 PM FINE hudson.plugins.scm_sync_configuration.SCMManipulator checkinFiles
Checked in SCM files !
Jul 25, 2017 12:49:48 PM FINEST hudson.plugins.scm_sync_configuration.ScmSyncConfigurationBusiness processCommitsQueue
Commit Commit hudson.plugins.scm_sync_configuration.model.Commit#5511458b :
Author : asangal
Comment : asangal: Job [dummy_job] configuration updated
Change performed by asangal
Changeset :
A jobs/dummy_job/config.xml
pushed to SCM !
Questions:
1) Why I didn't get prompted when I made changes to Jenkins job?
2) Why I'm not seeing any commits (which the logger is showing successfully checked-in / pushed, in the log) to the my target git repo? My github repo just shows README.md file so far.
3) Where exactly did this plugin CHECKED-IN the files and pushed?
Resolution / steps followed to properly check and implement this plugin:
Make sure:
1) Before you setup SCM Sync configuration, you have a repository / project created in Git
2) Find the user (which is running jenkins process). Usually it's jenkins user. If required, work with your system team and have them create a service/ldap user account named jenkins so that you can add jenkins user (or whatever user running the jenkins process) in Git/Github as one of the Collaborator with write access. If that's not possible, then go to Jenkins Master server and sudo su - jenkins and make sure you create (ssh keys). Once created, grab the public key and upload the key to Github repositories under Settings > Deploy Keys.
3) For this user, create a ~/.gitconfig file and put the following contents in this file.
user]
name = enter_theuserid
email = enter_theemailid#company.com
[core]
editor = vim
[color]
ui = auto
4) Now configure SCM Sync plugin in Jenkins global settings configuration (As per the Plugin's documentation). As soon as you provide the git#git..:org/your-repo.git, you'll notice that it'll not error out by saying the OS user running Jenkins is not able to do anything.
5) Create a new LOGGER (make sure Logger value is: hudson.plugins.scm_sync_configuration)
6) Go to Jenkins master server under: $JENKINS_HOME/scm-sync-configuration/checkoutConfiguration folder. Look for .git folder. If that's not present, that means, SCM Sync plugin didn't perform any checkout. To find if there are any issues with this plugin able to check out the Git repo, you can create another new logger where the Logger value should be hudson.plugins.scm_sync_configuration.ScmSyncConfigurationPlugin from the list box available options.
7) If you didn't find .git in bullet 5th folder, then remove checkoutConfiguration folder (leaf folder only) and in Jenkins Global configuration for SCM Sync plugin, change version control from Git to None (radio button), save it. Then go back and select Git (radio button) and provide git#git...:org/repo-name.git URL again and Save.
8) Keep an eye on the logs. This time if the user (running the Jenkins process) is listed in Git repository as a valid collaborator with WRITE access OR if it's SSH pub key is uploaded to Github repository, then it'll successfully CHECKOUT the repository to $JENKINS_HOME/scm-sync-configuration/checkoutConfiguration folder. Inside this folder, you'll see all valid .git/* files.
9) For testing purpose, you can make a small change in a file under $JENKINS_HOME/scm-sync-configuration/checkoutConfiguration folder, and run git add thatfilename; git commit -m "test change" $_; git pull && git push
10) If bullet 9 succeeds, at this point you'll have a working SCM Sync plugin where when you make a change to a Jenkins job, as soon as you press SAVE button, it'll first PROMPT via a popup modal window and request the user to enter some comment.
Goto Jenkins -> Manage Jenkins -> Configure System.
Under SCM Sync Configuration section look for "Never bother me with commit messages" configuration.
If you want Jenkins to prompt for commit messages when Job has been modified, Uncheck the option "Never bother me with commit messages".
Related
I have an action, that automatically indexes all files in the repository and creates a csv. Currently this action always creates their own pull request. This creates the annoying need to approve two pull requests per change (the first one with the change itself and the second one with the change in the index.csv file created by the action minutes later).
What I tried now, is creating a Github App, which is added to the "Allow specified actors to bypass required pull requests" in the branch protection and using tibdex/github-app-token#v1 to create a token in the context of the GitHub app.
Unfortunately, it still does not work. I get the following error:
Run echo "Hello World" >> HelloWorld.txt
[main f8445ab] Add unncessary file for testing
1 file changed, 1 insertion(+)
create mode 100644 HelloWorld.txt
remote: error: GH006: Protected branch update failed for refs/heads/main.
remote: error: Required status check "*****" is expected. At least 1 approving review is required by reviewers with write access.
To https://github.com/***/***.git
! [remote rejected] main -> main (protected branch hook declined)
error: failed to push some refs to 'https://github.com/***/***.git'
Error: Process completed with exit code 1.
The code I'm trying to run in the action is the following:
echo "Hello World" >> HelloWorld.txt
git config --global user.email "****"
git config --global user.name "****"
git add .
git commit -m "Add unncessary file for testing"
git push "https://action-name:${{ steps.generate_token.outputs.token }}#github.com/***/***.git"
I know that it is bad practice and breaks the permission concept to be able to do this, but in this case, it is fine for many reasons which go beyond the discussion at this point.
I'm happy for any suggestions on how to achieve this.
Thanks so much in advance
I just started using an alt account for IJ and things immediately goes out of hand.
At the start of the day, I had two accounts (foo, the intended / bar, from last sprint) in ...
Version Control > Github
Then I tried to push, with the intension of using the foo account
19:23:59.917: [foo-repo] git -c http.sslBackend=schannel -c core.quotepath=false -c log.showSignature=false push --progress --porcelain origin refs/heads/master:master --set-upstream
remote: Permission to foo/foo-repo.git denied to bar.
After one Google search I udpated my default account according to https://www.jetbrains.com/help/idea/github.html
Still the same message, weird. I tried each of the following and nothing changed.
Remove literally all accounts in Version Control > Github
Double check both the project and global gitconfig doesn't specify anything related to [user]
Appearance & Behavior > System Settings > Password clear keepass database.
Ditto, but switched to Do not save, forget passwords after restart. Invalidate IJ cache & restart
Logout from all other Github related apps running on my PC
I am really scared about my privacy by this point since it seems that IJ remains access to my second account bar against all my best intensions and efforts (Like, it shouldn't even have any legitimate means to access bar by this point`). I would really love to get rid of anything related to Github from IJ. Where should I lookup to?
Turns out using credentials helper AND Chrome autofill was the problem, nothing to do with IJ iteslf.
I have a jenkins job that runs some tests on a mercurial repo, and if successful tags the local repo with a 'stable' tag and then pushes this back to the main repo. The issue I'm having is that if someone pushes changesets while the build is running, then I cannot push the 'stable' tag.
I was wondering if there was a way to set the remote repo to read-only while the build is running, then make it 'push-able' once the build finishes?
Thanks,
Vackar
Preventing the push is probably not what you want (and it's almost pretty much impossible). The promise of a DVCS like Mercurial or git is that there's no locking -- it's a step forward.
Have you considered having Jenkins just pull and update before it merges? You can still tag the proper revision. Something like this:
jenkins checks out the code and notes the revision id it's building
jenkins does the build, runs the tests, etc. and everything goes well
jenkins does a hg pull to get the latest from the server
jenkins does a hg tag -m "build number $BUILD_NUMBER" --revision X --force stable
jenkins does a hg push
Then there's (almost) no time between that final pull, tag, and push, but the tag still goes on the revision that was actually build -- because you saved that revision hash id from when you first pulled.
I've just been looking for something similar. In our case, Jenkins is performing a merge, running an extensive suite of tests and once they all pass, pushing the merged code back to the repository. So it takes ~1 hour and fails if a developer pushes while the job is executing (it can't do the final push).
I couldn't find a ready-made solution, so ended up writing a mercurial hook which checks whether the job is building (using the REST API) before allowing the push.
You'll need access to your remote mercurial repository, but other than that, it's not too complex.
Add the following to your-remote-repo/.hg/hgrc:
[hooks]
pretxnchangegroup.DisablePushDuringJenkinsBuild= python:.hg/disable_push_if_building_hook.py:check_jenkins
[jenkins]
url=http://path-to-jenkins
jobs=jenkins-job-name[,comma-separated, for-multiple, jobs]
And make sure this python script is in your-remote-repo/.hg/
import json, urllib2
from mercurial import util
TEN_SECONDS = 10
def check_jenkins(ui, repo, node, **kwargs):
jenkins_url = ui.config('jenkins', 'url', default=None, untrusted=False)
jenkins_jobs = ui.config('jenkins', 'jobs', default=None, untrusted=False)
if not jenkins_url:
raise util.Abort('Jenkins hook has not been configured correctly. Cannot find Jenkins url in .hg/hgrc.')
if not jenkins_jobs:
raise util.Abort('Jenkins hook has not been configured correctly. Cannot find Jenkins jobs in .hg/hgrc.')
jenkins_jobs = [x.strip() for x in jenkins_jobs.split(',')]
for job in jenkins_jobs:
job_url = jenkins_url + '/job/' + job + '/lastBuild/api/json'
ui.write('Checking if job is running at URL: %s\n' % job_url)
try:
job_metadata = json.load(urllib2.urlopen(job_url, timeout = TEN_SECONDS))
if 'building' in job_metadata and job_metadata['building']:
raise util.Abort('Jenkins build "%s" is in progress. Pushing is disabled until it completes.' % job_metadata['fullDisplayName'])
except urllib2.URLError, e:
raise util.Abort('Error while trying to poll Jenkins: "%s"' % e)
return False # Everything is OK, push can be accepted
I have read through almost all of the guides by now on how to set up Jenkins with GitHub, but none of them seem to solve my problem. My job currently builds fine, but I want it to build every time I push to GitHub. I also want to manually set up my hook URLs (not let Jenkins auto-manage them). Currently my Jenkins server is running on EC2 on port 8080 (http://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:8080/). The sys admin says the server is set up to listen to all incoming requests on that port.
Under the GitHub Web Hook area of the Configure System area, I have Manually manage hook URLs checked off.
In my project's configuration, I have the following setup:
GitHub Project: https://github.com/username/projectname/
Source Code Management > Git
Repository URL: git#github.com:username/projectname.git
Branches to Build - Branch Specifier (blank for default): master
Build Triggers > Build when a change is pushed to GitHub
In the GitHub Hook Log it displays the message:
Last GitHub Push
Polling has not run yet.
I have also created a user with which GitHub can connect. The user has a role with the following global permissions:
Overall: Read
Job: Read
Job: Discover
Job: Build
On GitHub, in Settings > Service Hooks I have added https://USERNAME:PASSWORD#ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:8080/github-webhook/, where USERNAME and PASSWORD are the username and password for the GitHub account. I added the link to BOTH the WebHook URLs area and the Jenkins (GitHub plugin) area. When I click Test Hook under former area no message of confirmation shows up on the GitHub page, but when I click Test Hook under the latter it says Payload Deployed. Both, however, are registering in my System log file, and they are generating similar errors:
winstone.WinstoneException: Error URI Line: <Garbled Text Here>
at winstone.HttpListener.parseURILine(HttpListener.java:349)
at winstone.HttpListener.parseURI(HttpListener.java:258)
at winstone.RequestHandlerThread.run(RequestHandlerThread.java:83)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
Jun 7, 2013 6:16:31 PM winstone.Logger logInternal
SEVERE: Error within request handler thread
Any ideas about what could be causing this?
If this does not appear in Winstone’s access log, and the problem persists without SSL, try nc -l -p 8080 to see what the actual request being sent from GitHub is. (Or try a different servlet container.)
I'm having a hard time pushing my commits to a remote repository with libgit2sharp. Using the git bash it works fine.
The remote is addressed via UNC like "//computer_name/remote.git". So it's a folder on a machine in the local network which has an accessible folder.
Cloning it to a local repo with libgit2sharp worked just fine and constructing a remote was successful too with
Remote remote = localrepo.Network.Remotes["origin"];
Now when I try pushing to the remote with:
localrepo.Network.Push(remote, "HEAD", "origin");
I get the exception in git_push_add_refspec() in Proxy.cs
An error was raised by libgit2. Category = Invalid (Error).
Not a valid reference 'origin'
So then I tried:
repo.Network.Push(remote, "HEAD", #"refs/remotes/origin/master");
and got the exception in git_push_finish() in Proxy.cs
An error was raised by libgit2. Category = Net (Error).
Remote transport doesn't support push.
Is there a right way to do this or is there a support problem for my usecase?
Thanks in advance!
[UPDATE]
Now my command looks like this
repo.Network.Push(remote, "HEAD", #"refs/remote/origin/master", pushStErrHnd, null);
The PushStatusErrorHandler has only one line of code in which it should write the PushErrorStatus to console. But console remains blank and then the above exception occurs.
[UPDATE 2]
The static variable Repository.Version was 0.9.5 when I experienced the above.
As nulltoken has already mentioned, local push has just recently been added. You will need to make sure that you have a recent LibGit2Sharp build (you will need LibGit2Sharp containing commit 547a6bd, committed on March 12)
Also, there is a slight mistake in the API usage. The destination reference should be the reference to update on the remote (e.g. #"refs/remote/origin/master" should probably be #"refs/heads/master").
Push to a local repository has been recently added to libgit2 (see PR #1406) and eventually embedded in LibGit2Sharp. However, this feature hasn't been properly tested yet.
In order to try and help you, could you please update your questions with the answers to the few questions below:
Push accept an additional parameter to give more information about issues: onPushStatusError. Could you please provide the output of each potential PushStatusErrors?
You state "Cloning it to a local repo worked". How did you perform the clone? Through git? Through LibGit2Sharp?
What happens if you change the url of the remote to a file URI format (eg. file://computer_name/remote.git)?
UPDATE
The amazing #yorah is working on a Pull Request to enhance the test coverage regarding your scenario.
Cloning from a local repository
Adding a new Commit
Pushing the newly created commit
Retrieving the list of the remote references