I need a solution to display the results of sonar in the azure pull request.
I tried to do it with a status check by selecting the sonar pipeline in branch policy. It is showing success/fail and redirecting to sonar portal on click.
Is it really possible to show the actual results(vulnarabilities,duplications,etc.,) in the pull requets itself?
please help.
Thanks
After got the result of Sonarqube, you could use DevOps REST API to update the result to Azure pull request.
The flow is : a new pull request created > trigger a pipeline > run REST API to update the pull request description or title.
Add a Powershell task in the pipeline with follow script to update the pull request description and title. You could also refer to the document above to update other properties of the pull request. Please pay attention to PAT, the result of Sonarqube, organization name, project name, repository ID. Here we could use $(System.PullRequest.PullRequestId) to get the pull request ID, thus, the build will fail if it was not triggered by pull request.
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
$connectionToken="<PAT>"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($connectionToken)"))
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", "Basic $base64AuthInfo")
$headers.Add("Content-Type", "application/json")
$body = '{"description": "<the result of Sonarqube>","title": "<the result of Sonarqube>"}'
$response = Invoke-RestMethod 'https://dev.azure.com/<organization name>/<project name>/_apis/git/repositories/<repository ID>/pullrequests/$(System.PullRequest.PullRequestId)?api-version=5.0' -Method 'PATCH' -Headers $headers -Body $body
$response | ConvertTo-Json
After configure the pipeline, please enable Build Validation for the branch in project setting >> repositories >> your repo >> policies >> branch >> Build Validation. Then, every time a new pull request created for that branch will trigger the pipeline. You could also find the repository ID in the URL.
Related
I'm looking on a way to trigger a Azure pipeline ONLY on successful (or attempted) pull request merge.
Now I have :
trigger:
branches:
include:
- DEV
steps:
- script: FOO
But this runs EVERY time there is a change on the DEV branch and I would like to avoid that.
Besides, I want a programmatic response not going trough the UI each time.
EDIT:
A weird thing is happnening
condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest'))
gets:
Expanded: and(True, eq('IndividualCI', 'PullRequest'))"
When doing a PR, and thus doesn't work as intented
I'm looking on a way to trigger a Azure pipeline ONLY on successful (or attempted) pull request merge.
There is no such out of box way to achieve this at this moment.
We could only set the CI trigger on the target branch, but we could set the condotion for the pipeline to avoid build any task:
and(succeeded(), eq(variables['Build.Reason'], 'PullRequest'))
For example:
trigger:
branches:
include:
- DEV
steps:
- script: FOO
condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest'))
Or you could set the condition for the stage, job and so on.
Please check the document Specify conditions for some more details.
If there is a change on the DEV branch and it would be avoided by the condition.
Note: With above way, the pipeline will be triggered, but no task will be executed.
And if you even do not want the pipeline be triggered. You could add new pipeline with powershall task to invoke REST API to trigger above pipeline and set the condition to the powershell task.
In this way, the pipeline will only triggered when the commit comes from the PR.
Update:
Doing a PR on the DEV branch results in : "Expanded: and(True,
eq('IndividualCI', 'PullRequest'))"
Yes,you are correct. That because azure devops does not have the feature to trigger the pipeline after the PR completed. Pull request trigger and Build Validation both trigger the pipeline when the PR starts.
To resolve this request, we could try create a service hook to monitor PR status. If the PR status changes, the pipeline is triggered through API or Application, you could check this document for some more details.
And another way to achieve is using the REST API.
The main idea is：
create a pipeline and set it as Build validation, but not set it as Required, should set it as Optional:
Add powershell task in above pipeline to invoke REST API to monitor the PR status until it complated, and add another task to invoke the REST API to trigger your current pipeline.
So, you could remove the:
trigger:
branches:
include:
- DEV
in your current pipeline.
The trigger you have set is a CI trigger, it will work whenever the target branch has a new commit.
Currently, there isn't a trigger that works when a pull request is completed.
The feature closest to your needs is PR triggers and build validation branch policy.
They will work when a pull request is created or when it has been changed.
If you are using Azure Repos Git, please use branch policy for build validation. If you are using GitHub or Bitbucket Could, please use pr triggers. Click the documents for the detailed information.
Besides, you can use branch policy to prevent the direct commits. When you set the branch policy of any type, only users with "Bypass policies" permission can commit to the branch directly. The rest of the users must commit the branch through a pull request.
How to create branch policy: Branch policies and settings.
How to set "Bypass policies" permission: Set Git repository permissions.
I would put my questions through following points, hope it's make clear now:
The application source code is in application_code repo.
The pipeline code(YAMLs) are in pipeline_code repo. Because I'd like to version it and don't like to keep in application_code repo. Just to avoid giving control to Dev team to manage it.
Problem statement:
The pipeline YAML won't be triggered unless it's in the source code repository based on the events pr, commit etc.
Can we trigger or execute YAML file which is in pipeline_repo whenever there's event triggered in application_code repo?
I've tried achieving above using Classic pipeline and YAML template but this don't work together. As I can execute a YAML template from a YAML pipeline only not from a classic pipeline like below:
#azure-pipeline.yaml
jobs:
- job: NewJob
- template: job-template-bd1.yaml
Any ideas or better solution than above?
The feature Multi-repository support for YAML pipelines will be available soon for azure devops service. This feature will support for triggering pipelines based on changes made in one of multiple repositories. Please check Azure DevOps Feature Timeline or here. This feature is expected to be rolled out in 2020 Q1 for azure devops service.
Currently you can follow below workaround to achieve above using Build Completion(the pipeline will be triggered on the completion of another build).
1, Setup the triggering pipeline
Create an empty classic pipeline for application_code repo as the triggering pipeline, which will always succeed and do nothing.
And check Enable continuous integration under Triggers tab and setup Bracnh filters
2, setup the triggered pipeline
In the pipeline_code repo using Checkout to Check out multiple repositories in your pipeline. You can specifically checkout the source code of application_code repo to build. Please refer below example:
steps:
- checkout: git://MyProject/application_code_repo#refs/heads/master # Azure Repos Git repository in the same organization
- task: TaskName
...
Then in the yaml pipeline edit page, click the 3dots on the top right corner and click Triggers. Then click +Add beside Build Completion and select above triggering pipeline created in step 1 as the triggering build.
After finishing above two steps, when changes made to application_code repo, the triggering pipeline will be executed and completed with success. Then the triggered pipeline will be triggered to run the real build job.
Update:
Show Azure DevOps Build Pipeline Status in Bitbucket.
you can add a python script task at the end of the yaml pipeline to update the Bitbucket build status. You need to set a condtion: always() to always run this task even if other tasks are failed.
You can get the build status with env variable Agent.JobStatus. For below example:
For more information, please refer to document Integrate your build system with Bitbucket Cloud, and also this thread.
- task: PythonScript#0
condition: always()
inputs:
scriptSource: inline
script: |
import os
import requests
# Use environment variables that your CI server provides to the key, name,
# and url parameters, as well as commit hash. (The values below are used by
# Jenkins.)
data = {
'key': os.getenv('BUILD_ID'),
'state': os.getenv('Agent.JobStatus'),
'name': os.getenv('JOB_NAME'),
'url': os.getenv('BUILD_URL'),
'description': 'The build passed.'
}
# Construct the URL with the API endpoint where the commit status should be
# posted (provide the appropriate owner and slug for your repo).
api_url = ('https://api.bitbucket.org/2.0/repositories/'
'%(owner)s/%(repo_slug)s/commit/%(revision)s/statuses/build'
% {'owner': 'emmap1',
'repo_slug': 'MyRepo',
'revision': os.getenv('GIT_COMMIT')})
# Post the status to Bitbucket. (Include valid credentials here for basic auth.
# You could also use team name and API key.)
requests.post(api_url, auth=('auth_user', 'auth_password'), json=data)
I am trying to find the configuration required for using VSTS external services to make a VSTS release required to be successful before the PR to master can be completed.
The workflow is:
Create PR -> Triggers Build -> On Success Triggers Release -> On Success will flag the PR as OK.
Is there a way to do this using external services to post a successful status?
For the release definition, you can triggerred by the PR validation build artifacts, so you can get the pull request id by the pre-definied variable Release.Artifacts.{alias}.SourceBranch.
Assume the artifact alias for the release definition is prvalid, then you can get the pull request id (as the variable $id below) with below script:
$id="$(Release.Artifacts.prvalid.SourceBranch)".Split('/')
echo $id
$id=$id[2]
And then you can get the source branch and target branch by the REST API:
GET https://acount.visualstudio.com/DefaultCollection/_apis/git/repositories/repoID/pullRequests/PRid?api-version=3.0
Then you can merge the source branch into tartget branch, and the PR status will be completed.
I have a pull request trigger for Github in VSTS. I also want to add this trigger to the required checks in Github and show build status on pull request page like below.
I also checked branch protection page on Github but there are no status checks available.
Is it possible to do this in VSTS or do I need to create a PR status server mentioned here ?
I checked Advanced settings => Report build status option and VSTS automatically sends commit status to Github.
Configuration for enabling the GitHub commit status checks in Azure DevOps seems to have changed.
Ensure Azure Pipelines is installed for your organization or repository
Edit your Azure DevOps Build (Pipeline)
Click on the Get sources step
Under the GitHub configuration, select Report build status
Save (& queue, if you wish) your updated configuration
If someone on the DevOps team sees this, reporting commit status should be enabled by default!
There isn’t such setting in VSTS, you can refer to this workflow to do it:
Get a commit sha
Create a status check context through REST API
Post: https://api.github.com/repos/[owner]/[repository]/statuses/[commit sha]
Body(application/json):
{
"state": "success",
"target_url": "XXX",
"description": "Build verify",
"context": "continuous-integration/vsts"
}
Then check the related status check in branch protect page:
Note: the target_url can be badge URL (Check Badge enabled in Options of build definition)
Create a build definition to create status through REST API (The same as step 2: change commit sha and body) in VSTS continuous integration (Enable continuous integration) for current commit
Create a build definition to update status of current commit through REST API in VSTS (Enable pull request validation)
From Jenkins, Pull-Request Status can be created/updated from pipeline
script {
pullRequest.createStatus(status: "success",
context: "validate-profiles",
description: "Profiles file validated successfully!",
targetUrl: "$RUN_DISPLAY_URL")
}
Tons of other things can be done from pipeline avoiding explicit calls to GitHub API
Make a comment on Pull-Request
pullRequest.comment("Your service-profile request is received. Please track ticket progress here: "+ticketData['_links']['web'])
Create & Add Labels to Pull-Request
pullRequest.addLabel(env.TICKET_ID)
Update Title for the Pull-Request
pullRequest.setTitle("["+env.TICKET_ID+"] Profile Review Request for "+env.CHANGE_TARGET)
I'm trying to analyse my source code with Sonar using Jenkins pipelines. To ask Sonar to notify Github with the results I need to specify the Pull Request ID.
How can I get this Pull Request ID from Jenkins Pipelines?
We are using GitHub Organization Folder Plugin to build pull requests, not GitHub pull request builder plugin. That's why $ghprbPullId is not working for me. Any ideas how to get the pull request id in a different way?
Jenkins exposes a global variable named CHANGE_ID:
For a multibranch project corresponding to some kind of
change request, this will be set to the change ID, such as a pull
request number.
This variable is only populated for pull request builds, so you have to disable branch builds and enable PR builds in your pipeline's configuration for branch sources:
My pipeline step then looks like this:
def PULL_REQUEST = env.CHANGE_ID
stage('Analysis') {
withCredentials([[$class: 'StringBinding', credentialsId: '***', variable: 'GITHUB_ACCESS_TOKEN']]) {
withSonarQubeEnv('Sonar') {
withMaven(maven: 'M3') {
sh "mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar " +
"-Dsonar.analysis.mode=preview " +
"-Dsonar.github.pullRequest=${PULL_REQUEST} " +
"-Dsonar.github.oauth=${GITHUB_ACCESS_TOKEN}"
}
}
}
}
You get the PR number through for example env.BRANCH_NAME.
if (env.BRANCH_NAME.startsWith('PR-')) {
def prNum = env.BRANCH_NAME.replace(/^PR-/, '')
...
}
In the case that Thomas' answer doesn't work or apply to you, you may also (possibly) use the branch name to get the Pull Request number by querying the Github REST API. All you need is an API token and the branch name, lookup the pull requests in order of date updated DESC, and find the first PR that matches your branch name. That will have the Pull Request number.
This only works if you have a unique branch name for each pull request (such as a JIRA issue ticket number).