AzureDevops Api call for pull-request doesn't work - azure-devops

I'm referring to this page, to create a pull request during the build process in Azure DevOps via API call. I'm using authorization as Bearer using $(System.AccessToken) instead of PAT credentials. In my previous steps, I used Bearer with $(System.AccessToken) to get the buildNumber via API call which worked seamlessly. However, if I use the same in my below task for the POST mechanism to create a pull request it doesn't work and gives me 400 error code.
Can anyone suggest to me, how can I make this work?
- task: Bash#3
inputs:
targetType: 'inline'
script: |
echo 'Started createing pull-request'
url="https://dev.azure.com/{Organization name}/{Project name}/_apis/git/repositories/$(Build.Repository.ID)/pullrequests?api-version=6.0"
echo $url
ret=$(curl -d '{"sourceRefName": $(BUILD.SOURCEBRANCH), "targetRefName": "refs/heads/devlopment", "title": "test", "description": "test" }' -X POST -H "Authorization:Bearer $(System.AccessToken)" -H "Content-Type:application/json" ${url} --write-out "%{http_code}" --output response.json)
echo $ret
cat response.json

The error was in JSON. Fixed and reformatted JSON all working now. This is the task now looks like with some extra addition:
Title and Description are parameterized.
- task: Bash#3
displayName: Creating Pull request
inputs:
targetType: 'inline'
script: |
url="$(System.TeamFoundationCollectionUri)/$(System.TeamProject)/_apis/git/repositories/$(Build.Repository.ID)/pullrequests?api-version=6.0"
ret=$(curl -X POST \
--silent \
-H "Authorization:Bearer $(System.AccessToken)" \
-H "Content-Type:application/json" \
${url} \
--write-out "%{http_code}" \
--output response.json \
-d'{
"sourceRefName":"$(BUILD.SOURCEBRANCH)",
"targetRefName":"refs/heads/<branch-name>",
"title":"$(Title)",
"description":"$(Description)"
}' )
if [[ $ret -ne 201 ]];
then
message=$(cat response.json | jq '.message' --raw-output)
echo "##vso[task.logissue type=error;] ERROR MESSAGE: ${message}"
exit 1
fi
continueOnError: false

Did you check your URL?
In your example:
url="https://dev.azure.com/{Organization name}/{Project name}/_apis/git/REPOSITORIE/$(Build.Repository.ID)/pullrequests?api-version=6.0"
should be
url="https://dev.azure.com/{Organization name}/{Project name}/_apis/git/REPOSITORIES/$(Build.Repository.ID)/pullrequests?api-version=6.0"

GIT Pullrequest
I was trying to use your code, yet I had a problem
MESSAGE: TF401027: You need the Git 'PullRequestContribute' permission to perform this action. Details: identity 'Build\c5f6a8a4-de0a-4081-bff4-928a38d232c4', scope 'repository'.
The solution comes from
https://developercommunity.visualstudio.com/t/tf401027-you-need-the-git-pullrequestcontribute-pe/1441618
Navigate to Project Settings >> Repositories >> select Security tab >> Type Project Collection Build Service (organizationName) in the search box >> check if the Contribute to pull requests permission is set to Allow.
You need to search for this 'Project Collection Build Service' and only then it would appear on your list. There is no add button, only 'search'.
https://dev.azure.com/{owner}/{project}/_settings/repositories?_a=permissions
(example) https://dev.azure.com/jmusz/iospoc/_settings/repositories?_a=permissions
My code was based on yours, an example of a plain curl without an error handling is
- task: Bash#3
displayName: Creating Pull request
inputs:
targetType: 'inline'
script: |
url="$(System.TeamFoundationCollectionUri)$(System.TeamProject)/_apis/git/repositories/$(Build.Repository.ID)/pullrequests?api-version=6.0"
echo $url
curl -X POST \
-v \
-H "Authorization: Bearer $(System.AccessToken)" \
-H "Content-Type: application/json" \
${url} \
--write-out "%{http_code}" \
--output response.json \
-d'{
"sourceRefName": "$(BUILD.SOURCEBRANCH)",
"targetRefName": "refs/heads/dest",
"title": "Title",
"description": "Description"
}'
cat response.json
As you might guess, when I could not find PullRequestContribute (took me a while), I had a 'Plan B' that was to use a Personal Access Token...
Turns out, there is a difference with Authorization header:
$(System.AccessToken) - you can use it directly, with -H "Authorization: Bearer $(System.AccessToken)"
while PAT must be extended with a (username:PAT) that is encoded with BASE64, and Basic not Bearer so you use -H "Authorization: Basic ${b64pat}" \
The code for PAT is:
- task: Bash#3
displayName: Creating Pull request
inputs:
targetType: 'inline'
script: |
url="$(System.TeamFoundationCollectionUri)$(System.TeamProject)/_apis/git/repositories/$(Build.Repository.ID)/pullrequests?api-version=6.0"
echo $url
#works
b64pat=`echo -n 'jmusz:ljfcuww24sa7ywmgvlvuwrn3v7znmuvbnfymfh4xtnt3tbo6bk' | base64`
#works (empty username)
b64pat=`echo -n ':ljfcuww24sa7ywmgvlvuwrn3v7znmuvbnfymfh4xtnt3tbo6bk' | base64`
#not working (missing `:` )
#b64pat=`echo -n 'ljfcuww24sa7ywmgvlvuwrn3v7znmuvbnfymfh4xtnt3tbo6bk' | base64`
#echo $b64pat
#-H "Authorization: Bearer $(System.AccessToken)" \ #require PullRequestContribute
#-H "Authorization: Basic ${b64pat}" \ # user:PAT | base64
ret=$(curl -X POST \
-v \
-H "Authorization: Basic ${b64pat}" \
-H "Content-Type: application/json" \
${url} \
--write-out "%{http_code}" \
--output response.json \
-d'{
"sourceRefName": "$(BUILD.SOURCEBRANCH)",
"targetRefName": "refs/heads/dest",
"title": "Title",
"description": "Description"
}' )
if [[ $ret -ne 201 ]];
then
message=$(cat response.json | jq '.message' --raw-output)
echo "##vso[task.logissue type=error;] ERROR MESSAGE: ${message}"
#cat response.json
exit 1
fi
PAT with powershell
- task: PowerShell#2
displayName: Creating Pull request - PS
inputs:
targetType: 'inline'
script: |
$url = "https://dev.azure.com/jmusz/iospoc/_apis/git/repositories/iospoc/pullrequests?api-version=6.0"
$sourceBranch = "refs/heads/source"
$targetBranch = "refs/heads/dest"
$title = "title"
$description = "description"
$MyPat = 'ljfcuww24sa7ywmgvlvuwrn3v7znmuvbnfymfh4xtnt3tbo6bk'
$B64Pat = [Convert]::ToBase64String([System.Text.Encoding]::Unicode.GetBytes(":$MyPat")) #Note : !!!!
$B64Pat
$header = #{ Authorization = "Basic $B64Pat" }
$body = #{
sourceRefName = "$sourceBranch"
targetRefName = "$targetBranch"
title = "$title"
description = "$description"
}
$jsonBody = ConvertTo-Json $body
try {
$response = Invoke-RestMethod -Uri $url -Method Post -Headers $header -Body $jsonBody -ContentType "application/json;charset=UTF-8"
}
catch {
Write-Error $_
Write-Error $_.Exception.Message
}
$response
GIT repo clone with Token
I had to pass credentials to repo to a task in pipeline, I decided the best would be to use $(System.AccessToken) (plan B was with PAT).
I faced the same problem as above - Bearer $(System.AccessToken) vs Basic ${b64pat}
Please note that username is optional when used with PAT (for git clone) while it is not optional when used for API interaction
- stage: Lint
jobs:
- job: SwiftLint
steps:
- bash: |
b64pat=`echo -n jmusz:ljfcuww24sa7ywmgvlvuwrn3v7znmuvbnfymfh4xtnt3tbo6bk |base64`
echo 1 not working
git clone $(System.TeamFoundationCollectionUri)$(System.TeamProject)/_git/$(Build.Repository.ID) ||true
#rm -rf libraries ||true
echo 2 not working
#not working
git -c http.extraHeader="Authorization: Basic $(System.AccessToken)" clone $(System.TeamFoundationCollectionUri)$(System.TeamProject)/_git/$(Build.Repository.Name) ||true
rm -rf iospoc ||true
echo 2.1 working
#working
git -c http.extraHeader="Authorization: Bearer $(System.AccessToken)" clone $(System.TeamFoundationCollectionUri)$(System.TeamProject)/_git/$(Build.Repository.Name) ||true
rm -rf iospoc ||true
echo 3 working
git -c http.extraHeader="Authorization: Basic ${b64pat}" clone $(System.TeamFoundationCollectionUri)$(System.TeamProject)/_git/$(Build.Repository.Name) ||true
#git -c http.extraHeader="Authorization: Basic $(System.AccessToken)" clone https://dev.azure.com/jmusz/iospoc/_git/libraries/ ||true
rm -rf libraries ||true
echo 4 not working
git clone "https://jmusz:$(System.AccessToken)#dev.azure.com/jmusz/iospoc/_git/libraries" ||true
rm -rf libraries ||true
echo 4.01 not working
git clone "https://$(System.AccessToken)#dev.azure.com/jmusz/iospoc/_git/libraries" ||true
rm -rf libraries ||true
echo 4.1 working
git clone "https://jmusz:ljfcuww24sa7ywmgvlvuwrn3v7znmuvbnfymfh4xtnt3tbo6bk#dev.azure.com/jmusz/iospoc/_git/libraries" ||true
rm -rf libraries ||true
echo 4.2 working
git clone "https://ljfcuww24sa7ywmgvlvuwrn3v7znmuvbnfymfh4xtnt3tbo6bk#dev.azure.com/jmusz/iospoc/_git/libraries" ||true
rm -rf libraries ||true
#not working
echo 5 not working
git clone https://${b64pat}#dev.azure.com/jmusz/iospoc/_git/libraries ||true
rm -rf libraries ||true
#not working
echo 6 not working
git clone https://$(System.AccessToken)#dev.azure.com/jmusz/iospoc/_git/libraries ||true
rm -rf libraries ||true
#working
echo 7 working
git config --global --add http.https://dev.azure.com/jmusz/iospoc/_git/libraries.extraHeader "AUTHORIZATION: Basic ${b64pat}"
git clone https://dev.azure.com/jmusz/iospoc/_git/libraries/ ||true
rm -rf libraries ||true
#not working?
echo 8 not working
git config --global --add http.https://dev.azure.com/jmusz/iospoc/_git/libraries.extraHeader "AUTHORIZATION: Bearer $(System.AccessToken)"
git clone https://dev.azure.com/jmusz/iospoc/_git/libraries/ ||true
rm -rf libraries ||true
Interesting that 2.1 works while 8 does not

Related

Configure YAML Pipeline to run databricks repos update Error:JSONDecodeError

I am trying to configure a devops pipeline to get Databricks workspace to automatically pull the latest version of the repo when master branch is updated.
I am able to generate the DB_PAT but I cannot seem to get this into the DATABRICKS_TOKEN variable properly hence the databricks repos update fails with the following error:
Error: JSONDecodeError: Expecting value: line 1 column 1 (char 0)
##[debug]Exit code 1 received from tool '/usr/bin/bash'
##[debug]STDIO streams have closed for tool '/usr/bin/bash'
##[error]Bash exited with code '1'.
This is my YAML file that I ran in Azure Devops.
trigger:
- master
jobs:
- job: PullReposUpdate
steps:
- task: AzureCLI#2
name: DbTokenGen
inputs:
azureSubscription: $(azureSubscription)
scriptType: bash
scriptLocation: inlineScript
inlineScript: |
ADBWORKSPACENAME="adb-$(clientName)-$(stageName)"
echo "$ADBWORKSPACENAME"
TENANT_ID=$(az account show --query TENANT_ID --output tsv)
WORKSPACE_ID=$(az resource show --resource-type Microsoft.Databricks/workspaces --resource-group $RGNAME --name $ADBWORKSPACENAME --query id --output tsv)
TOKEN=$(az account get-access-token --resource 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d | jq --raw-output '.accessToken')
AZ_TOKEN=$(az account get-access-token --resource https://management.core.windows.net/ | jq --raw-output '.accessToken')
DB_PAT=$(curl --silent https://$(region).azuredatabricks.net/api/2.0/token/create \
--header "Authorization: Bearer $TOKEN" \
--header "X-Databricks-Azure-SP-Management-Token:$AZ_TOKEN" \
--header "X-Databricks-Azure-Workspace-Resource-Id:$WORKSPACE_ID" \
--data '{ "lifetime_seconds": 1200, "comment": "Azure DevOps pipeline" }' \
| jq --raw-output '.token_value')
echo "##vso[task.setvariable variable=DB_PAT]$DB_PAT"
failOnStandardError: true
displayName: 'Generate Token for Databricks'
- script: |
pip install --upgrade databricks-cli
displayName: 'Install dependencies'
- script: |
databricks repos update --path $(STAGING_DIRECTORY) --branch "$(branchName)"
env:
DATABRICKS_HOST: $(databricks-url)
DATABRICKS_TOKEN: $(DbTokenGen.DB_PAT)
displayName: 'Update Production Databricks Repo'
How can I pass the DB_PAT successfully to the databricks cli environment? any suggestion is highly appreciated.
Thank you.

Using new github tokens to list and create comments on pull requests

After I switched to the new Github Token format, I am not able anymore to perform a call for this github API:
https://docs.github.com/en/rest/issues/comments#list-issue-comments
My script does the following:
[[ -z "$GITHUB_REPO_SLUG" ]] && export GITHUB_REPO_SLUG="$TRAVIS_REPO_SLUG"
GITHUB_API_BASE_URL="https://api.github.com/repos/${GITHUB_REPO_SLUG}"
GITHUB_AUTH_HEADER="Authorization: token $GITHUB_TOKEN"
comment_ids=$(curl -s -H "$GITHUB_AUTH_HEADER" "$GITHUB_API_BASE_URL/issues/$pull_request_id/comments" \
| jq --arg USER "$GITHUB_USER" -r '. | map(select(.user.login==$USER)) | map(.id) | .[]')
for comment_id in $(echo $comment_ids); do
jq -n -r --arg message "$message" '{ body: $message }' \
| curl -s -H "$GITHUB_AUTH_HEADER" "$GITHUB_API_BASE_URL/issues/comments/$comment_id" -X PATCH --data #- > /dev/null
echo "Edited comment with id $comment_id of PR $pull_request_id"
done
The api call that list comments ids responds:
{
"message": "Not Found",
"documentation_url": "https://docs.github.com/rest/reference/issues#list-issue-comments"
}
This still works using tokens in the old format, but doesn't with tokens in the new format.
Is there any way to use the GHP to call that api?
The github user associated with the github token must be provided writing access to the repository.
Since this was missing API responded 404 (default behaviour set for Github APIs).

How can a Github workflow read a file on another private repository hosted on the same GitHub account?

I have a private git repository like:
Github.com/acct/repo1
I have a test.json file in another private repository in the same GitHub subscription and account like:
Github.com/acct/repo2/test.json
How can a workflow in repo1 have http get access to the url Github.com/acct/repo2/test.json?
All I need is a simple way to read the content of test.json in a workflow running on repo2.
It works the best for me to read the file using a curl get request.
First of all, you need to get an access token from GitHub. This is described here.
And add it to the secrets for the repository in which the Github Action will run. In your case, this is repo1. Read more about setting secrets.
After that, you can execute the request using curl.
To do this, run the following command:
curl -H 'Authorization: token ${{ secrets.GITHUB_TOKEN }}' \
-H 'Accept: application/vnd.github.v4.raw' \
-L https://api.github.com/repos/OWNER_HERE/REPO_NAME_HERE/contents/FOLDER_OR_FILE_HERE
After execution, the contents of the file will be displayed on the console.
If the contents of the file need to be saved to a variable, then use the following syntax:
VARIABLE_NAME="$(curl -H 'Authorization: token ${{ secrets.GITHUB_TOKEN }}' \
-H 'Accept: application/vnd.github.v4.raw' \
-L https://api.github.com/repos/OWNER_HERE/REPO_NAME_HERE/contents/FOLDER_OR_FILE_HERE)"
Add the -O flag to save the content to a file
curl -H 'Authorization: token ${{ secrets.GITHUB_TOKEN }}' \
-H 'Accept: application/vnd.github.v4.raw' \
-O \
-L https://api.github.com/repos/OWNER_HERE/REPO_NAME_HERE/contents/FOLDER_OR_FILE_HERE
In your case, the step in the .yml file to get the content would look like this:
#...
-name: Get file test.json
run: |
curl -H 'Authorization: token ${{ secrets.GITHUB_TOKEN }}' \
-H 'Accept: application/vnd.github.v4.raw' \
-O \
-L https://api.github.com/repos/acct/repo2/contents/test.json
#...
Please note that the path /contents/ must remain after REPO_NAME_HERE in the request, and after that you can specify the path to the FOLDER_OR_FILE_HERE file.

How to create the pull request in Bitbucket from command line

what is the command to create a new pull request in Bitbucket.
I am automating the check-in the code to Bitbucket and create pull request.
I referred many documents and found the curl command. but it is also not working and I don't know the what is the use of each module in that command.
see the below command that I have tried.
curl -u username:Password -H "Content-Type: application/jso https://bitbucket.server.com/rest/api/1.0/projects/myProject/repos/myRepo/pull-requests -X POST --data #req.json
created the req.json file, copied the content to file
see the below content that I copied.
{"title":"test","description":"test","fromRef":{"id":"refs/heads/del","repository":{"slug":"BB_repo","name":null}},"toRef":{"id":"refs/heads/master","repository":{"slug":"BB_repo","name":null}}}
can anyone suggest me the better way to create pull request? I need some explanation as I'm new to Bitbucket
Using a bitbucket personal access token, you could create an alias or adapt these to a script.
Global Config
[bitbucket]
url = https://bitbucket.<domain>/rest/api/latest/projects/<proj>/repos/<repo>/pull-requests
token = <Enter PAT Here>
[alias "pr"]
m = "!f() { B=$(git branch-name);D=$(git pr.data $B master);U=$(git config --get bitbucket.url);J=$(git pr.json);T=$(git pr.token);C=$(echo curl -X POST $U -H $T -H $J -d $D;); eval $C; }; f"
d = "!f() { B=$(git branch-name);D=$(git pr.data $B develop);U=$(git config --get bitbucket.url);J=$(git pr.json);T=$(git pr.token);C=$(echo curl -X POST $U -H $T -H $J -d $D;); eval $C; }; f"
json = "!echo '\"Content-Type: application/json\"'"
token = "!echo '\"Authorization: Bearer '$(git config --get bitbucket.token)'\"'"
data = !sh -c 'echo -e \"\\x27{\\\"title\\\":\\\"$1 - $2\\\",\\\"fromRef\\\":{\\\"id\\\":\\\"refs/heads/$1\\\"},\\\"toRef\\\":{\\\"id\\\":\\\"refs/heads/$2\\\"""}}\\x27\"' -
[alias]
branch-name = !git rev-parse --abbrev-ref HEAD
Usage:
git pr.m
git pr.d

Using Travis to deploy to GitHub and build Tag

I am struggling for some time on how to use Travis-CI to build and deploy a tag to GitHub.
I find some information about that on this question, but it is not very clear and it is also not working for me.
I created a new Release version 0.0.0 on gitHub and I am using Scala with Play Framework and my .travis.yml file is as follows:
language: scala
scala:
- 2.10.4
jdk:
- openjdk7
services:
- postgresql
env:
- PLAY_VERSION=2.0.2 DATABASE_USER=postgres DATABASE_PWD='' DATABASE_URL=jdbc:postgresql:testdb BUILD_KEY=xxxxxxxxxxxxxxxxxxxxxxx
before_script:
- psql -c 'create database testdb;' -U postgres
- wget http://download.playframework.org/releases/play-${PLAY_VERSION}.zip
- unzip -q play-${PLAY_VERSION}.zip
- sudo apt-get install jq
script:
- sbt test
after_success:
- play-${PLAY_VERSION}/play dist
- 'ASSETID=$(curl -s -H "Authorization: token ${BUILD_KEY}" "https://api.github.com/repos/Company/spinsurstaging/releases/456729/assets" | jq ".[0].id")'
- 'curl -XDELETE -s -H "Authorization: token ${BUILD_KEY}" "https://api.github.com/repos/Company/spinsurstaging/releases/assets/$ASSETID"'
- 'curl -XPOST -s -H "Authorization: token ${BUILD_KEY}" -H "Content-Type: application/zip" --data-binary #./spinsurstaging-0.0-SNAPSHOT.zip "https://uploads.github.com/repos/Company/spinsurstaging/releases/456729/assets?name= spinsurstaging.zip"'
notifications:
email: false
deploy:
provider: heroku
api_key: "${HEROKU_KEY}"
app: spinsurstaging
Main question is: How to make it work? Is something wrong?
And also: I didn't understand how I will post the file spinsurstaging-0.0-SNAPSHOT.zip as a release to github. Where is this file coming from? Is there a standard for this?
Is there a better way to do that?
Thank you in advance for helping me out with this issue.
I just figured out how to solve it.
I made a mistake on the line of code:
- 'curl -XPOST -s -H "Authorization: token ${BUILD_KEY}" -H "Content-Type: application/zip" --data-binary #./spinsurstaging-0.0-SNAPSHOT.zip "https://uploads.github.com/repos/Company/spinsurstaging/releases/456729/assets?name= spinsurstaging.zip"'
The correct code is:
- 'curl -XPOST -s -H "Authorization: token ${BUILD_KEY}" -H "Content-Type: application/zip" --data-binary #./spinsurstaging-0.0-SNAPSHOT.zip "https://uploads.github.com/repos/Company/spinsurstaging/releases/456729/assets?name= spinsurstaging .zip"'
Note that I change the snapshot filename. This snapshot is created when running the command play-${PLAY_VERSION}/play dist
Another important thing is that I have to change the PLAY_VERSION variable to 2.2.4 because the older version didn't had a repository for the same version I was using for sbt.
So I believe I could answer all my questions. Hope it helps someone else. Here are the complete solution:
language: scala
scala:
- 2.10.4
jdk:
- openjdk7
services:
- postgresql
env:
- PLAY_VERSION=2.2.4 DATABASE_USER=postgres DATABASE_PWD='' DATABASE_URL=jdbc:postgresql:testdb BUILD_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
before_script:
- psql -c 'create database testdb;' -U postgres
- wget http://downloads.typesafe.com/play/${PLAY_VERSION}/play-${PLAY_VERSION}.zip
- unzip -q play-${PLAY_VERSION}.zip
- sudo apt-get install jq
script:
- sbt test
after_success:
- play-${PLAY_VERSION}/play dist
- cd target/universal/
- 'ASSETID=$(curl -s -H "Authorization: token ${BUILD_KEY}" "https://api.github.com/repos/Company/spinsurstaging/releases/456729/assets" | jq ".[0].id")'
- 'curl -XDELETE -s -H "Authorization: token ${BUILD_KEY}" "https://api.github.com/repos/Company/spinsurstaging/releases/assets/$ASSETID"'
- 'curl -XPOST -s -H "Authorization: token ${BUILD_KEY}" -H "Content-Type: application/zip" --data-binary #./spinsurstaging-1.0-SNAPSHOT.zip "https://uploads.github.com/repos/Company/spinsurstaging/releases/456729/assets?name= spinsurstaging'
notifications:
email: false
deploy:
provider: heroku
api_key: "${HEROKU_KEY}"
app: spinsurstaging