Is there a way to script repetitive tasks in Azure DevOps? - azure-devops

We have a number of tasks that we carry out every time we create a new GIT repository in our project, and I would like to know if there's a way to script (PowerShell or any other method) these out. for e.g. every these are the steps we follow everytime we create a new repo
Create a new GIT repo
Create a Build pipeline for Build validations during
pull request
Add branch policies to Master including a step to validate build using the above build
Create a Build pipeline for releases
Create a Release pipeline

Is there a way to script repetitive tasks in Azure DevOps?
Of course yes! As Daniel said in comment, just use REST API can achieve these all. But since the steps you want to achieve are little much, the script might be little complex.
Create a new GIT repo
If you also want to use API to finish this step, it needs 3 steps to finish that( Since this does not be documented in doc, I will described it very detailed ):
Step1: Create the validation of importing repository
POST https://dev.azure.com/{org name}/{project name}/_apis/git/import/ImportRepositoryValidations?api-version=5.2-preview.1
Request body:
{
"gitSource":
{
"url":"${ReposURL}",
"overwrite":false
},
"tfvcSource":null,
"username":"$(username}"/null,
"password":"${pw}"/"${PAT}"/null
}
Step2: Create the new repos name
POST https://dev.azure.com/{org name}/{project name}/_apis/git/Repositories?api-version=5.2-preview.1
Request body:
{
"name":"${ReposName}",
"project":
{
"name":"{project name}",
"id":"{this project id}"
}
}
Step3: Import repos
POST https://dev.azure.com/{org name}/{project name}/_apis/git/repositories/{the new repos name you create just now}/importRequests?api-version=5.2-preview.1
Request body:
{
"parameters":
{
"deleteServiceEndpointAfterImportIsDone":true,
"gitSource":
{
"url":"${ReposURL}",
"overwrite":false
},
"tfvcSource":null,
"serviceEndpointId":null
}
}
In these script, you can set variables in Variable tab, then use ${} to get them in the script.
Create a Build pipeline for Build validations during pull request
This step you'd better finish manually, because you can configure more about tasks and trigger with UI. If still want use API, refer to this doc: create build definition. There has detailed sample you can try with.
Add branch policies to Master including a step to validate build using the above build
This API still be documented in doc: create build policy. Just refer to that, and ensure use the correct policy type and the corresponding buildDefinitionId.
Create a Build pipeline for releases
This still recommend you finish manually, same with the step3 you mentioned.
Create a Release pipeline
See this doc: create release.
Note: For some parameter which will be used many times, you can set it as variable. For the parameter which need get from previous API response, you can define a variable to get its value then pass this variable into the next API to use.For e.g. :
$resultT= $result.Headers.ETag
Write-Host "##vso[task.setvariable variable=etag;]$resultT"
Now, you can directly use the $(etag) in the next API.

Related

How to pull code from different branches at runtime and pass parameter to NUnit.xml file?

We recently moved Java(TestNG) to C#.Net (NUnit). Sametime migrated to Jenkins to Team-city. Currently we are facing some challenges while we configuring the new build pipeline in Team-City.
Scenario 1: Our project has multiple branches, we generally pull code from different Git-branches then trigger the automation.
In Jenkins we used to create build-parameter(list), when user try to execute the Job, he/she select the branch-name from the list (build-parameters), then git will pull code user selected branch then trigger execution.
Can you please help how to implement a similar process in Team-City?
How to configure the default value in the list parameter?
Scenario 2: In Jenkins build-parameter use used to pass to (TestNG.xml). eg: browser, environment. When the user select browser and environment from build parameters, when execution trigger TestNG pull those values and initiate the regression.
How should create build parameters (browser, envi.) and pass those
values to NUnit/ config file?
Thanks
Raghu

Is there a way in Terraform Enterprise to read the payload from VCS?

I have configured a webhook between github and terraform enterprise correctly, so each time I push a commit, the terraform module gets executed. Why I want to achieve is to use part of the branch name where the push was made and pass it as a variable in the terraform module.
I have read that the value of a variable can be a HCL code, but I am unable to find the correct object to access the payload (or at least, the branch name), so at this moment I think it is not possible to get that value directly from the workspace configuration.
if you get a workaround for this, it may also work from me.
At this point the only idea I get is to call the terraform we hook using an API Call
Thanks in advance
Ok, after several try and error I found out that it is not possible to get any information in the terraform module if you are using the VCS mode. So, in order to be able to get the branch, I got these options:
Use several workspaces
You can configure a workspace for each branch, so you may create a variable a select that branch in each workspace. The problem is you will be repeating yourself with this option
Use Terraform CLI and a GitHub action
I used these fine tutorial from Hashicorp for creating a Github action that uses Terraform Cloud. It gets you done the 99% of the job. For passing a varible you must be aware that there are two methods, using a file or using an enviromental variable (check that information on the Hashicorp site here). So using a:
terraform apply -var="branch=value"
won't work. In my case I used the tfvars approach, so in my Github Action I put this snippet:
- name: Setup Terraform variables
id: vars
run: |-
cat > terraform.auto.tfvars <<EOF
branch = "${GITHUB_REF#refs/*/}"
EOF
I defined a variable within terraform called branch, I was able to get and work with this value

Azure DevOps - get custom Task Reference ID

I want to update a Pipeline with the Definitions - Update REST API call.
That works fine, but when I want to add a custom task (self made build pipeline task extension) then I struggeling to find the correct task reference id:
Invoke-RestMethod : {"$id":"1","innerException":null,"message":"The pipeline is not valid. A task is missing. The pipeline references a task called '7f1fe94f-b811-4ba1-9d6a-b6c27de758d7'. This
usually indicates the task isn't installed, and you may be able to install it from the Marketplace: https://marketplace.visualstudio.com. (Task version 1.*, job 'Job_1', step ''.),Job Job_1: Step
has an invalid task definition reference. A valid task definition reference must specify either an ID or a name and a version specification with a major version
specified.","typeName":"Microsoft.TeamFoundation.DistributedTask.Pipelines.PipelineValidationException,
Microsoft.TeamFoundation.DistributedTask.WebApi","typeKey":"PipelineValidationException","errorCode":0,"eventId":3000}
I check out the registrationId of my custom task with the Installed Extensions - List REST API call. But it is not the correct one. (7f1fe94f-b811-4ba1-9d6a-b6c27de758d7)
I also add the custom task manually to a pipeline and read out the correct task refernce id with the Definitions - Get REST API call. I could find the id in:
$pipeline.process.phases.steps.task.id -> 2c7efb3e-3267-4ac6-addc-86e88a6dab34
But how can I read out this id without adding the custom task manually?
This id is obviously dynamic and changes everytime when the custom task get installed, so there must be a way to get this refernce.
The task id has not changed every time when the custom task gets installed, but he existed in task.json of the task:
{
"id": "2f159376-f4dk-4311-a49c-392f9d534113",
"name": "TaskName",
"friendlyName": "Task Name",
Another option is to use this api:
https://dev.azure.com/{organiztion}/_apis/distributedtask/tasks
You will get a long list of all the tasks, search your task and you will see the id.

Azure DevOps: Can we identify Builds triggered from a changeset number?

I have certain builds set for Continuous Integration, i.e., build for every check-in.
I have an automated method to perform code merge and check-ins; now I want to get the list of builds triggered for a particular changeset created. Is there any way we could get this information?
I would use the the REST API so you could check for builds that were run:
GET https://dev.azure.com/{organization}/{project}/_apis/build/builds
will return all builds that you could then go through and check for more details. You can also have more filters in the request (for example based on the build definition).
The build specifics you could then get via:
GET https://dev.azure.com/{organization}/{project}/_apis/build/builds/<buildid>
This will return you the infos like:
"triggerInfo": {
"ci.sourceBranch": "refs/heads/master",
"ci.sourceSha": "0fcb5a27ca2f73561dde0a066a1ec1781128fa81",
"ci.message": ""
},
...
"sourceBranch": "refs/heads/master",
"sourceVersion": "0fcb5a27ca2f73561dde0a066a1ec1781128fa81",
for builds queued from a git repository or
{ ...
"sourceBranch": "$/Build Test",
"sourceVersion": "93",
... }
for TFVC repositories. It actually also would contain a trigger info but I don't have any build around that was triggered automatically based on TFVC.
The sourceVersion in git will be the commit hash, where in TFVC it's the changeset.
More details on the REST API can be found in the Microsoft Docs

Can I skip an AWS CodePipeline build?

I am currently developing a personal project on master. Every time I push to origin master a build is triggered on CodePipeline. As I am the only developer working on this project and don't want to bother with branches at this stage it would be nice to skip unnecessary builds. I wouldn't mind pushing to another branch, but it's a small annoyance.
CodeShip allows you to skip a build by including --skip-ci in your commit message. Is something like this possible with CodePipeline?
None of my Google searches have yielded results. The CodePipeline documentation makes no mention of such a feature either.
A valid reason for not wanting to build a certain commit is when you use CodeBuild to generate a commit for you. For example, I have some code on the master branch which passes all the tests. I then want to update the changelog, package.json version and create a git tag on a new commit and push it back to the CodeCommit repo.
If I do this on Codebuild, the version-commit triggers another build! Given the contents of the commit does not materially change the behaviour of the code, there is no need to build & test this commit.
Besides all of this, Amazon should be looking at the features in the marketplace and attempting to provide at least feature-parity. Adding a RegEx check for "skip-ci" to the CodeBuild trigger-code would take a few hours to implement, at most.
By default codepipeline creates a cloudwatch event which triggers your pipeline on all changes of the specific branch.
What you can do is to set this cloudwatch event to trigger a lambda function. This function can check whether it is necessary to build this commit and start your CodePipeline.
Here is an example of how to achieve this:
https://aws.amazon.com/blogs/devops/adding-custom-logic-to-aws-codepipeline-with-aws-lambda-and-amazon-cloudwatch-events/
Here is a simple example for lambda function. It checks if the last commit has no [skip-CI] in its message and after that executes the pipeline.
Keep in mind, that this code checks only the last commit if your change was a series of commits you might want to check everything between oldCommitId and commitId.
const AWS = require('aws-sdk');
const codecommit = new AWS.CodeCommit();
const codepipeline = new AWS.CodePipeline();
exports.handler = async (event) => {
const { detail: { repositoryName, commitId, oldCommitId } } = event
const { commit } = await codecommit.getCommit({
commitId,
repositoryName
}).promise()
if(commit.message.search(/\[skip-CI\]/) === -1) {
const { pipelineExecutionId } = await codepipeline.startPipelineExecution({
name: 'your-pipeline-name'
}).promise()
console.log(`Pipeline have started. Execution id: ${pipelineExecutionId}!`)
} else {
console.log('Pipeline execution is not required')
}
return;
};
This isn't a feature offered by CodePipeline at this time.
I'd be curious to know why you view some builds as unnecessary. Do you push a sequence of commits and only want a build against the last commit? This may be more applicable in a team environment, but I would tend to want a build for every commit so I don't find myself in a situation where I push code and pick up and build someone else's broken code for the first time in my build.
I would suggest a manual review stage in your pipeline before it builds. Then you can just approve it when you are ready to build.