I have created a single job in Jenkins to be triggered by commit on any branches in the GitHub using Webhook.
I want to create a custom workspace for each job trigger with respective to the branch commit.
I Tried following options and could not achieve it. Appreciate for any help.
Job Name : Test_Clone
Used advance options in general section and tried below options to name the workspace dynamically.
Test_Clone_${GIT_BRANCH} - always gives the branch name from previous build.
Test_Clone_${ref} - tried to use ref as defined in GitHub webhook payload, with no result.
To achieve: Ex: Commit from GitHub branch release-2.13.0, Jenkins should create workspace with name Test_Clone_release-2.13.0.
You might try and setup
first a payload variable in your Jenkins build, as described in "How to process a github webhook payload in Jenkins?"
then a webhook url (on GitHub side) of:
http://<<yourserver>>/job/<<yourjob>>/buildWithParameters?token=<<yourtoken>>
Then your Jenkins job would have the xml webhook payload in it, which you can analyze (with jq) and extract the 'ref' which includes the branch name.
From there, I would call (chain) a second job with a "branch" parameter (unless you can write a pipeline DSL with Jenkins 2) with that branch name, in order for the second job to use a custom workspace name based on that parameter
Related
I have created a custom PR status check to validate my PR follows a conventional commit like pattern. This is in Azure Devops Git, not Github. To do this, I created an Azure Function App and setup a status check in Azure Devops. Here is the configuration:
I enabled the PR status check for my branch. Here is the config for that:
The PR status check appears and actually works....when I invoke the call manually. I can use postman to invoke my function (with a PAT I generated for my account) and it will update the status on the PR. But if I commit to the branch, the step sits there on the validation step even though I have the checkbox checked to "Reset status when there are new changes".
This is what it looks like after I invoke the function manually through postman
I would expect the system (AZDO in this case) to invoke my function every time a new iteration was created (i.e. when a new commit is pushed to the branch). Can someone point out what I'm missing? Thanks!
Ended up coming up with solution. I don't have privs to add a an authenticate an application to run the pipeline because of the way our org is setup. So I created a node script to accomplish what the function app is supposed to do.
Azure dev-ops pipelines have predefined varialbles related to github pull requests. I can use SYSTEM_PULLREQUEST_PULLREQUESTNUMBER for getting PR number that triggered my pipeline. However I get no value from SYSTEM_PULLREQUEST_PULLREQUESTNUMBER when my pipeline is triggered again as a result of merging this PR in the main repo.
My use case is to identify the list of files that were changed in the original PR.
I looked into Azure user predefined variable document but could not see if there is any variable available to get this information.
When the pipeline is ran the second time when it is merged, it is considered to have the trigger type CI not Pull Request. Therefore the PR number is unavailable in this context because there was no PR.
You could also try to save the pull request number to a variable group in the previous run triggered by a pull request.
https://learn.microsoft.com/en-us/azure/devops/pipelines/scripts/cli/pipeline-variable-group-secret-nonsecret-variables?view=azure-devops
You could try and steal the pull request number from the commit message. If your PR merge type is set to 'squash commit' you can write some regex to pull the PR number out of the commit message.
https://learn.microsoft.com/en-us/azure/devops/repos/git/merging-with-squash?view=azure-devops#squash-merge
Environment:
BitBucket
Concourse 3.14.0
Wondering is it possible to configure Concourse pipeline with Git webhook which will check if new commit has happened and it would trigger a pipeline build based on that trigger? I looked at https://concourse-ci.org/resources.html#resource-webhook-token, but it does not tell me how to get a webhook token from Concourse and if it does support what I am asking.
Any feedback is very much appreciated.
Concourse resources usually pull any new versions every minute or so. Whenever this frequency doesn't suit your needs, you can modify it with the check_every resource property. But values lower that 1m (one minute) are typically considered aggressive. Github implements quotas for API calls and when you have many pipelines, you don't want them to fail because you've hit some quota limits.
In case you want Concourse to immediately react on published new versions for the pipeline resources, you need to reverse the pattern. Instead of Concourse pulling any new versions at some defined frequency, you start pushing the information to Concourse that some new versions are to be pulled. This reversed “push” pattern involves triggering “resource checks” whenever new versions are created on the resource.
Trigger immediate resource checks
Every Concourse resource can enable a resource-check triggering URL with the webhook_token resource property. This URL includes the webhook_token secret in its query string, and is supposed to receive a mere POST HTTP request.
With Github repositories, you can POST to this URL with a Github workflow, relying on a standard Github action from the marketplace (recommended, first choice), or a Github webhook (second choice).
Using a Github workflow
You need to commit and push a YAML file in the .github/workflows folder of your Github repository, in order to define your workflow. Refer to the documentation of the “Trigger Concourse resource-check” action for detailed examples. It's very easy, as only five simple inputs need to be configured.
Using a Github webhook
With this alternative, you can manually setup a Github webhook in your repository. The URL depends on the resource for which an immediate check is to be triggered, so you can't set it up at your Github organization level. The webhook_token secret in appended in clear-text to the URL set up for the webhook, and can't be stored as a Github secret. Github webhook don't support fetching any Github secret.
And in case you're bored of manually set up webhooks, automated setup is possible with the github-webhook resource. You can even trigger the webhook recreation whenever the webhook_token secret changes in Credhub, with the help of the Credhub resource. I've done some working code implementing this idea, see those example jobs and those example resource definitions.
But I definitely recommend using a Github workflow with the “Trigger Concourse resource-check” action as a first choice.
I think you are looking for this resource - https://github.com/concourse/git-resource
It automatically checks for any new commit in your git repository and you can run other jobs based on that.
Example pipeline.yml:
resources:
- name: git-repo
type: git
source:
uri: git#github.com:concourse/git-resource.git
branch: master
private_key: {{GIT_KEY}}
jobs:
- name: run-on-new-commit
- get: git-repo
trigger: true
- task: do-something-else
I'm trying to analyse my source code with Sonar using Jenkins pipelines. To ask Sonar to notify Github with the results I need to specify the Pull Request ID.
How can I get this Pull Request ID from Jenkins Pipelines?
We are using GitHub Organization Folder Plugin to build pull requests, not GitHub pull request builder plugin. That's why $ghprbPullId is not working for me. Any ideas how to get the pull request id in a different way?
Jenkins exposes a global variable named CHANGE_ID:
For a multibranch project corresponding to some kind of
change request, this will be set to the change ID, such as a pull
request number.
This variable is only populated for pull request builds, so you have to disable branch builds and enable PR builds in your pipeline's configuration for branch sources:
My pipeline step then looks like this:
def PULL_REQUEST = env.CHANGE_ID
stage('Analysis') {
withCredentials([[$class: 'StringBinding', credentialsId: '***', variable: 'GITHUB_ACCESS_TOKEN']]) {
withSonarQubeEnv('Sonar') {
withMaven(maven: 'M3') {
sh "mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar " +
"-Dsonar.analysis.mode=preview " +
"-Dsonar.github.pullRequest=${PULL_REQUEST} " +
"-Dsonar.github.oauth=${GITHUB_ACCESS_TOKEN}"
}
}
}
}
You get the PR number through for example env.BRANCH_NAME.
if (env.BRANCH_NAME.startsWith('PR-')) {
def prNum = env.BRANCH_NAME.replace(/^PR-/, '')
...
}
In the case that Thomas' answer doesn't work or apply to you, you may also (possibly) use the branch name to get the Pull Request number by querying the Github REST API. All you need is an API token and the branch name, lookup the pull requests in order of date updated DESC, and find the first PR that matches your branch name. That will have the Pull Request number.
This only works if you have a unique branch name for each pull request (such as a JIRA issue ticket number).
I am familiar with Jenkins Pull Request Builder and I had set up a freestyle job with it to build my project based on the comment that authorized user put. (For example test in prod) in the past.
Now I am trying to use a Jenkins 2.0 with github organization plugin for one of my project.
this is the scenario:
A User is making a PR to master(or some other sensitive branch)
A test is going to get run automatically.
After the test past, an authorized user needs to go to the PR and put a comment Deploy to test environment and then a jenkinsfile that was waiting for this input needs to get trigger.
I just dont know how to do the step 3. how do I make jenkins pipeline job listen for comments in github repo pull requests? the Jenkins documentation is not really clear about the input from user part.
I read this thread answer but the documentation about the Gates approval is really limited.
I know this is super late, but here's some info for future Googlers:
I have a Github webhook that sends the event to a Lambda function that will parse the event for a specific comment string, then create an HTTP POST request for the Jenkins job, which is configured to allow builds to be triggered remotely.
So: open PR > comment on PR 'Deploy to test environment' > webhook sends to AWS APIGateway > AWS SNS topic > AWS Lambda > parse the event for comment > If comment matches, create HTTP POST > Jenkins receives request and runs job
There's a lot of documentation on this, but none of it together, so here are the resources that I used:
Regarding allowing jobs to be triggered remotely:
https://wiki.jenkins-ci.org/display/JENKINS/Remote+access+API
Using Github to trigger Lambda function:
https://aws.amazon.com/blogs/compute/dynamic-github-actions-with-aws-lambda/
Github API. You will want to pay particular attention to the Issues API:
https://developer.github.com/webhooks/