In Azure DevOps, we can organize our Build/Release definitions into high-level folders:
Example: for every pipeline that resides in the Framework folder, I want to conditionally execute a certain task. The pre-defined Build and Release variables provide a plethora of ways to discover information about the underlying file system, but seemingly nothing for this internal path information.
During a pipeline run, is it possible to determine the folder/path that it resides in?
You can check it with Rest API - Builds - Get:
GET https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}?api-version=6.0
In the response you get the definition details including the path:
"definition": {
"drafts": [
],
"id": 13,
"name": "TestBuild",
"url": "https://dev.azure.com/xxxxx/7fcdafd5-b891-4fe5-b2fe-xxxxxxx/_apis/build/Definitions/13?revision=1075",
"uri": "vstfs:///Build/Definition/13",
"path": "\\Test Folder",
"type": "build",
"queueStatus": "enabled",
"revision": 1075,
"project": {
"id": "7fcdafd5-b891-4fe5-b2fe-9b9axxxxx",
"name": "Sample",
"url": "https://dev.azure.com/xxxx/_apis/projects/7fcdafd5-b891-4fe5-b2fe-9xxxxxx",
"state": "wellFormed",
"revision": 97,
"visibility": "private",
"lastUpdateTime": "2021-03-22T10:25:39.33Z"
}
},
So:
Add a simple PS script that invokes the rest API (with the $(Build. BuildId) pre-defined variable)
Check the value of the path property
If it contains the Framework folder set a new variable with this command:
Write-Host "##vso[task.setvariable variable=isFramework;]true"
Now, in the task add a custom condition:
and(succeeded(), eq(variables['isFramework'], 'true'))
Related
I would like to create a test environment for my app running in BC18, but I have trouble downloading the necessary symbols. I created a new workspace along with a .vscode/launch.json and an app.json-file, which contains the following properties:
"dependencies": [
{
"id": "65d313b7-f655-41af-9325-3ade0d3489e9",
"name": "Main App",
"publisher": "My Company",
"version": "1.0.1.0"
}
],
"test": "17.0.0.0"
However, when trying to download the symbols, I always get the following error:
[2021-05-18 15:39:06.44] The request for path /BC/dev/packages?publisher=Microsoft&appName=Test&versionText=17.0.0.0&tenant=default failed with code NotFound. Reason: No published package matches the provided arguments.
[2021-05-18 15:39:06.44] The following dependencies will be queried for propagated dependencies:
Application by Microsoft (18.0.0.0)
[2021-05-18 15:39:06.44] Sending request to http://ll-bc-at:7049/BC/dev/packages?publisher=Microsoft&appName=Application&versionText=18.0.0.0&tenant=default
[2021-05-18 15:39:06.45] Could not download reference symbols. Please ensure that:
1. The correct server name and instance are specified in the launch.json file.
2. The correct application version is specified in the app.json file.
3. The dependencies are correctly specified in the app.json file.
I've tried reinstalling the container, making sure the test toolkit is included and confirmed it is working properly by accessing the default test suite in the Test Tool, but nothing so far has been successful. Using different version for test hasn't had an effect either.
The test property has been deprecated. You will need to add the dependecies for the various test extensions you want to use.
I normally use these dependencies as a starting point:
{
"id": "e7320ebb-08b3-4406-b1ec-b4927d3e280b",
"name": "Any",
"publisher": "Microsoft",
"version": "17.0.0.0"
},
{
"id": "23de40a6-dfe8-4f80-80db-d70f83ce8caf",
"name": "Test Runner",
"publisher": "Microsoft",
"version": "17.0.0.0"
},
{
"id": "dd0be2ea-f733-4d65-bb34-a28f4624fb14",
"name": "Library Assert",
"publisher": "Microsoft",
"version": "17.0.0.0"
},
{
"id": "9856ae4f-d1a7-46ef-89bb-6ef056398228",
"name": "System Application Test Library",
"publisher": "Microsoft",
"version": "17.0.0.0"
},
{
"id": "5d86850b-0d76-4eca-bd7b-951ad998e997",
"name": "Tests-TestLibraries",
"publisher": "Microsoft",
"version": "17.0.0.0"
}
I'm developing a release pipeline where I would like to do some analysis on a given artifact. I will not use the artifact itself for anything later, I'm only interested in reading certain properties such as version number. I, therefore, think that it is unnecessary to download an artifact that I won't do any changes to or to publish. From my point of view, the most time and resource-efficient way would be to not have to download the artifact, but I'm not sure if this is possible. Or are there workarounds, where you can download a "lite" version of the artifact? I'm planning on using a task for this.
If you want to get metadata for your build/pipeline artifact, you can't do this as there is no such thing. You can call this endpoint:
https://dev.azure.com/{{organization}}/{{project}}/_apis/build/builds/6179/artifacts?api-version=6.1-preview.5
but you will only get something like this:
{
"count": 1,
"value": [
{
"id": 1095,
"name": "drop",
"source": "12f1170f-54f2-53f3-20dd-22fc7dff55f9",
"resource": {
"type": "Container",
"data": "#/6799617/drop",
"properties": {
"localpath": "D:\\a\\1\\a",
"artifactsize": "1330651"
},
"url": "https://dev.azure.com/thecodemanual/4fa6b279-3db9-4cb0-aab8-e06c2ad550b2/_apis/build/builds/6179/artifacts?artifactName=drop&api-version=6.1-preview.5",
"downloadUrl": "https://dev.azure.com/thecodemanual/4fa6b279-3db9-4cb0-aab8-e06c2ad550b2/_apis/build/builds/6179/artifacts?artifactName=drop&api-version=6.1-preview.5&%24format=zip"
}
}
]
}
If you want to get details of you package in Azure Artifact, which is a feed, you have some option in REST API. For instance: Artifact Details - Get Package Version
In Azure DevOps, I have created a Task Group that runs Postman tests using the newman CLI. As inputs, users can pass in the paths to the Postman collection and environment files.
As the newman CLI is a requirement, the first task in the Task Group is to install it. However, in scenarios where several collections are run, there is no need to keep installing the CLI over and over, so I would like to offer a checkbox and then conditionally run the install task depending on the value of that checkbox.
As the UI for Task Groups is pretty lacking in useful options, I started exploring the API. I'm able to add additional inputs, but setting the obvious type option to checkbox yields only an additional text (string) input.
POST https://dev.azure.com/{org}/{project}/_apis/distributedtask/taskgroups?api-version=5.1-preview.1
{
...
"inputs": [
{
"aliases": [],
"options": {},
"properties": {},
"name": "Rbt.Cli.Install",
"label": "Install 'newman' CLI?",
"defaultValue": true,
"required": false,
"type": "checkbox",
"helpMarkDown": "Choose whether or not to install the 'newman' CLI. You only need to install it if it hasn't already been installed by a previos task running on this job.",
"groupName": ""
},
...
],
...
}
Looking more closely at the documentation, there is a definition for inputs - TaskInputDefinition. However, it looks as though whoever was tasked with writing that documentation left early one day and never got around to it. There are no descriptions at all, making it impossible to know valid values for properties in the definition.
How can I add a checkbox to my Task Group?
I have now found that Task Groups offer picklist as an input type. This has allowed be to present a yes/no option to the user, and based on their answer I am able to conditionally run a task.
I would still prefer to have a checkbox though, should anyone know how to do that.
{
"aliases": [],
"options": {
"yes": "Yes - install CLI",
"no": "No - the CLI has already been installed"
},
"properties": {},
"name": "Postman.Cli.Install",
"label": "Install 'newman' CLI?",
"defaultValue": "yes",
"required": true,
"type": "picklist",
"helpMarkDown": "Choose whether or not to install the 'newman' CLI. You only need to install it if it hasn't already been installed by a previos task running on this job.",
"groupName": ""
}
You can add checkbox to pipeline task easily by setting the type as boolean
{
"name": "Rbt.Cli.Install",
"type": "boolean",
"label": "Install 'newman' CLI?"
}
And also control the visibility of other controls based on the check box state as following:
{
"name": "someOtherField",
"type": "string",
"label": "Some other field",
"visibleRule": "Rbt.Cli.Install = true"
},
I have a Task Group with two tasks:
Set BinLogFlag
Compile Solution
The first task is actually another TG that sets BinLogFlag="/bl:...", if binary log is requested. It basically run the following Powershell code:
"##vso[task.setvariable variable=BinLogFlag]..."
Now Compile Solution is simply the MSBuild task with the following command line:
/m /v:m $(BinLogFlag)
So, to recap - Set BinLogFlag creates the build variable BinLogFlag and Compile Solution uses it. Of course, if BinLogFlag is empty, then no binary log is generated.
The problem is that the moment I add $(BinLogFlag) to the command line, Azure DevOps adds it to the parameters of the TG I develop:
But it is an internal implementation detail. I do not want it exposed. My current workaround is fragile - I updated/generate Directory.Build.rsp file before compiling and delete/restore it afterwards.
I do not like it at all. Is it possible for Compile Solution to use the BinLogFlag build variable created by Set BinLogFlag without exposing it as yet another parameter of the parent Task Group?
Is it possible for Compile Solution to use the BinLogFlag build variable created by Set BinLogFlag without exposing it as yet another parameter of the parent Task Group?
The answer is Yes.
There is a feature for Task Group: Export after we create the task group:
Export it, you will get the .json file and open it with notepad or Visual Studio, you will get following code segment about parameter :
"iconUrl": "https://cdn.vsassets.io/v/M154_20190723.10/_content/icon-meta-task.png",
"friendlyName": "TestTG",
"description": "",
"category": "Build",
"definitionType": "metaTask",
"author": "YourCount",
"demands": [],
"groups": [],
"inputs": [
{
"aliases": [],
"options": {},
"properties": {},
"name": "",
"label": "",
"defaultValue": "",
"required": true,
"type": "string",
"helpMarkDown": "",
"groupName": ""
}
],
"satisfies": [],
Then, we delete the content in the inputs to [], so it looks like:
"groups": [],
"inputs": [],
"satisfies": [],
Finally, we import this .json back to the task group and rename it with a new name Set BinLogFlag V2.
When we add this task group to the pipeline to create the another task group Compile Solution, the BinLogFlag build variable created by set BinLogFlag will not be exposed:
I use the command line task instead of MSBuild in your demo to test if the value of the variable BinLogFlag can be passed correctly. And it works fine as test.
Hope this helps.
I need a work around pretty quickly - this was a late surprise in the dev process when we added an Az function to our development ADF pipeline.
When you use a function app in ADF V2, when you generate the ARM template, it does not parameterize the key references unlike in other linked services. Ugh!
So for CI/CD scenarios, when we deploy we now have a fixed function app reference. What we'd like to do is the same as other linked services - override the key parameters to point to the correct Dev/UAT /Production environment versions of the functions.
I can think of dirty hacks using powershell to overwrite (does powershell support ADF functions yet? don't know - in January they didn't).
Any other ideas on how to override function app linked service settings?
the key parameters are under typeProperties (assuming the function key is in keyvault):
{"functionAppUrl:="https://xxx.azurewebsites.net"}
{"functionkey":{"store":{"referenceName"="xxxKeyVaultLS"}}}
{"functionkey":{"secretName"="xxxKeyName"}}
Right now these are hard coded from the UI settings - no parameter and no default.
ok, eventually got back to this.
The solution looks a lot but it is pretty simple.
In my devops release, I create a Powershell task after both the data factory ARM template has been deployed and the powershell task for deployment.ps1 with the "predeployment=$false" setting has run (see ADF CI/CD here.)
I have a json file for each environment (dev/uat/prod) in my git repo (I actually use a separate "common" repo to store scripts apart from the ADF git repo and its alias in DevOps is "_Common" - you'll see this below in the -File parameter of the script).
The json file to replace the deployed function linked service is a copy of the function linked service json in ADF and looks like this for DEV:
(scripts/Powershell/dev.json)
{
"name": "FuncLinkedServiceName",
"type": "Microsoft.DataFactory/factories/linkedservices",
"properties": {
"annotations": [],
"type": "AzureFunction",
"typeProperties": {
"functionAppUrl": "https://myDEVfunction.azurewebsites.net",
"functionKey": {
"type": "AzureKeyVaultSecret",
"store": {
"referenceName": "MyKeyvault_LS",
"type": "LinkedServiceReference"
},
"secretName": "MyFunctionKeyInKeyvault"
}
},
"connectVia": {
"referenceName": "MyintegrationRuntime",
"type": "IntegrationRuntimeReference"
}
}
}
...and the PROD file would be like this:
(scripts/Powershell/prod.json)
{
"name": "FuncLinkedServiceName",
"type": "Microsoft.DataFactory/factories/linkedservices",
"properties": {
"annotations": [],
"type": "AzureFunction",
"typeProperties": {
"functionAppUrl": "https://myPRODfunction.azurewebsites.net",
"functionKey": {
"type": "AzureKeyVaultSecret",
"store": {
"referenceName": "MyKeyvault_LS",
"type": "LinkedServiceReference"
},
"secretName": "MyFunctionKeyInKeyvault"
}
},
"connectVia": {
"referenceName": "MyintegrationRuntime",
"type": "IntegrationRuntimeReference"
}
}
}
then in the devops pipeline, I use a Powershell script block that looks like this:
Set-AzureRMDataFactoryV2LinkedService -ResourceGroup "$(varRGName)" -DataFactoryName "$(varAdfName)" -Name "$(varFuncLinkedServiceName)" -File "$(System.DefaultWorkingDirectory)/_Common/Scripts/Powershell/$(varEnvironment).json" -Force
or for Az
Set-AzDataFactoryV2LinkedService -ResourceGroupName "$(varRGName)" -DataFactoryName "$(varAdfName)" -Name "$(varFuncLinkedServiceName)" -DefinitionFile "$(System.DefaultWorkingDirectory)/_Common/Scripts/Powershell/Converter/$(varEnvironment).json" -Force
Note:
the $(varXxx) are defined in my pipeline variables e.g.
varFuncLinkServiceName = FuncLinkedServiceName.
varEnvironment = "DEV", "UAT", "PROD" depending on the target release
Force is used because the Linked service must already exist in the Data Factory ARM deployment and then we need to force the overwrite of just the function linked service.
Hopefully MSFT will release a function app linked service that uses parameters but until then, this has got us moving with the release pipeline.
HTH. Mark.
Update: Added the Az cmdlet version of the AzureRM command and changed to Set ("New-Az..." worked but in the new Az - there is only Set- for V2 linked services).