Checking Functions exist on FunctionApp in Azure via DevOps Pipeline - powershell

On Azure DevOps I have a build/deploy YAML pipeline which builds my Function App and deploys this to a Linux Function App in Azure (hosted on an app-service plan).
However, recently I noticed an issue where the pipeline was showing the function app was successfully deployed, but when going into the FunctionApp in Azure and clicking on the Functions tab on the left, it shows "No results", but there should be 4 functions in there.
I want to have a step at the end of the deploy pipeline which checks that 4 functions exist in this FunctionApp, and to fail the pipeline run if this is not the case.
I know this will most likely be a task in the pipeline using Azure CLI or Powershell, but not sure how I would go around writing the script. Any help would be immensely appreciated.
Thanks all,

You could use an Azure CLI task and call the Rest API to list the functions withint the FunctionApp. I don't think the native Az CLI and Azure PowerShell functions expose the functions within the FunctionApp/
URI = "https://management.azure.com/subscriptions/<subscriptionId>/resourceGroups/<resourceGroupName>/providers/Microsoft.Web/sites/<functionAppName>/functions?api-version=2015-08-01"
az rest -u $URI --method get | jq '.value[].name'

steps:
- powershell: |
#get token
$TENANTID="xxx"
$APPID="xxx"
$PASSWORD="xxx"
$result=Invoke-RestMethod -Uri https://login.microsoftonline.com/$TENANTID/oauth2/token?api-version=1.0 -Method Post -Body #{"grant_type" = "client_credentials"; "resource" = "https://management.core.windows.net/"; "client_id" = "$APPID"; "client_secret" = "$PASSWORD" }
$token=$result.access_token
##set Header
$Headers=#{
'authorization'="Bearer $token"
'host'="management.azure.com"
}
$functions = Invoke-RestMethod -Uri "https://management.azure.com/subscriptions/<subcription id>/resourceGroups/<resource group name>/providers/Microsoft.Web/sites/<function app name>/functions?api-version=2015-08-01" -Headers $Headers -ContentType "application/json" -Method GET
if($functions.value.Count -eq 4) {
# make pipeline to succeed
Write-Host 'Function deployment success.'
exit 0
}
else {
Write-Host 'Function deployment failed.'
exit 1
}
displayName: 'Check whether the function app deployment completed.'
If you have concerns, let me know.

Related

How to create resources in Azure Portal using Devops CI/CD pipeline and ARM Template

I am working on creating qnamaker based Chatbot and for this I have prepared ARM template for creating and deploying resources.
PowerShell script is being used to create knowledge base once all the resources are created.
This process works just fine when I am executing PowerShell script from local PowerShell tool.
Now I want to do CI/CD for this deployment process so that deployment process can be automated
Creating resources and deploying to azure through Pipeline is quite possible through Azure ARM Template deployment task but I am not sure how to execute PowerShell script that is responsible for creating knowledge base once resources are created based on recently created qnamaker service
Any help greatly appreciated
You can refer to Create a CI/CD pipeline for .NET with the Azure DevOps Project for step by step lab to create the pipelines.
If you need help on ARM templates (if you are stuck somewhere), you can check out A quick guide on writing your Azure Resource Manager (ARM) Templates. and get started quickly.
Please clarify that your issue is creating and training the QnA Maker KB after the actual resources have been created by ARM? There are probably multiple ways to do this, but here is what I have done. First of all, I am using the Azure CLI Task version 2.* which simplifies some of the initial steps. You could probably use regular PowerShell for some of the subsequent steps. I have selected all as PowerShell script types and inline scripts. Each of these segments I do as a separate task but you could probably do all as one. I like being able to see which step fails if something happens, though.
First you need to get the key for your cognitive service
$QNAKEY= & az cognitiveservices account keys list -g "YOUR_RESOURCE_GROUP" --name "YOUR_QNA_RESOURCE_NAME" --query key1 -o tsv
Write-Output("##vso[task.setvariable variable=QNA_KEY;]$QNAKEY")
Next you need to create the KB. I have mine seeded with information via a json file in my project repository. You can do this from wherever, or just create a blank KB (I think). Note I'm checking here to see if the KB exists; KB name does not have to be unique, so if you don't do this you will end up creating a lot of duplicate KBs.
$header = #{
"Content-Type"="application/json"
"Ocp-Apim-Subscription-Key"="$(QNA_KEY)"
}
$res = Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/qnamaker/v4.0/knowledgebases" -Method 'Get' -Headers $header
$kb = $res.knowledgebases | Where-Object name -eq "YOUR_KB_NAME"
Write-Host $res.knowledgebases
Write-Host $kb
if (!$kb) {
Write-Host "KB does not exist, so creating new KB"
$body = Get-Content '$(System.DefaultWorkingDirectory)/PATH_TO_MY_MODEL.json' | Out-String
Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/qnamaker/v4.0/knowledgebases/create" -Method 'Post' -Body $body -Headers $header
}
Finally, you will likely want to publish your KB. With LUIS I think you need to train it first (via separate CLI task), but QnA Maker you should be able to publish directly. I do this as a dependent stage to ensure the KB is created before I try to publish.
$QNAKEY= & az cognitiveservices account keys list -g "YOUR_RESOURCE_GROUP" --name "YOUR_QNA_RESOURCE_NAME" --query key1 -o tsv
$header = #{
"Content-Type"="application/json"
"Ocp-Apim-Subscription-Key"="$QNAKEY"
}
$res = Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/qnamaker/v4.0/knowledgebases" -Method 'Get' -Headers $header
$kb = $res.knowledgebases | Where-Object name -eq "YOUR_KB_NAME"
$qnaId = $kb.id
Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/qnamaker/v4.0/knowledgebases/$qnaId" -Method 'Post' -Headers $header
And that's it! At this point your QnA Maker knowledgebase should be created, published, and ready to go.

Azure cli: clone pipeline

Looking at az pipelines documentation it seems it's not possible to clone a pipeline using cli.
I've looked at getting the yaml (az pipelines show -name=x > x_orig.yaml) and then trying to change json and create pipeline from modified yaml, but that feels like a lot of work that could break after next update.
Is there a way to clone a pipline without going the the Web UI?
Currently, there indeed is not available Azure CLI that can clone or export/import a pipeline to create a new pipeline.
I also searched and tried the Azure DevOps REST API for Pipelines, but did not find the available API.
Ideally, the Azure CLI "az pipelines create" can provide an input parameter that allows users specify an existing pipeline as a starting point for the new pipeline.
If your projects really need this feature, I recommend that you can directly report a feature request on the "Azure/azure-cli" repository to ask adding the parameter like as above mentioned. That will allow you directly interact with the appropriate engineering team, and make it more convenient for the engineering team to collect and categorize your suggestions.
As a workaround, we could clone the build definition via power shell script to call REST API.
Note: We need to change the original build definition name.
REST API
Get build definition:
GET https://dev.azure.com/{organization}/{project}/_apis/build/definitions/{definitionId}?api-version=6.0
Create build definition
POST https://dev.azure.com/{organization}/{project}/_apis/build/definitions?api-version=6.0
Power shell script
$connectionToken="{pat}"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($connectionToken)"))
$BuildDefinitionInfoURL = "https://dev.azure.com/{org name}/{project name}/_apis/build/definitions/386"
$BuildDefinitionInfo = Invoke-RestMethod -Uri $BuildDefinitionInfoURL -Headers #{authorization = "Basic $base64AuthInfo"} -Method Get
Write-Host $BuildDefinitionInfo.name
$BuildDefinitionInfo.name = $BuildDefinitionInfo.name +" clone"
Write-Host $BuildDefinitionInfo.name
$body = $BuildDefinitionInfo | ConvertTo-Json -Depth 99
$createBuildDefinitionURL = "https://dev.azure.com/{org name}/{project name}/_apis/build/definitions?api-version=6.0"
$response = Invoke-RestMethod -Uri $createBuildDefinitionURL -ContentType "application/json" -Body $body -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)} -Method POST
Write-Host $response.id
Result:

Store result of Azure CLI task into release variable

I have a release pipeline with multiple jobs.
On the first agent job there is an Azure CLI task that retrieves the keys of a storage account on azure.
The command it executes is :
az storage account keys list --account-name "$(diagnosticsStorageAccountName)" --resource-group "$(resourceGroup)"
What I want to do is store the result of this command and utilize it in a task that is running under a deployment group job.
I've already looked into these resource :
Set Output Variable in Azure CLI task on VSTS
How to modify Azure DevOps release definition variable from a release task?
I've tried the first one but I didn't get it working.
I didn't bothered with the second because it seems way to hacky.
Is there any way do achieve this that isn't hacky ?
The output values didn't get stored properly.
The output of az storage account keys list --account-name "$(diagnosticsStorageAccountName)" --resource-group "$(resourceGroup)"is spread over multiple lines when you use the following syntax:
echo "##vso[task.setvariable variable=testvar;]%myvar%"
So one of the problems was that only the first line of the JSON array was being stored into the variable.
I solved this problem in the following way:
keys=`az storage account keys list --account-name "$(diagnosticsStorageAccountName)" --resource-group "$(resourceGroup)"`
taskvariable="##vso[task.setvariable variable=_[tempVariable];]"
echo $taskvariable$keys
According to documentation echo "##vso[task.setvariable variable=testvar output=true;]%myvar% should make the variable available for the whole release. Unfortunately I had no luck with this.
I overcame this using a Powershell task under the same Agent Job (Ubuntu 16.0) :
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/Release/definitions/$($env:RELEASE_DEFINITIONID)?api-version=5.1"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $(System.AccessToken)"
}
#Parsing JSON string from previous task
$keys = $(#"
$(_tempVariable)
"# | ConvertFrom-Json)
# Assignment of variable value
$pipeline.variables.[variableName].value = $keys[0].value
####****************** update the modified object **************************
$json = #($pipeline) | ConvertTo-Json -Depth 99
$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers #{Authorization = "Bearer $(System.AccessToken)"}
write-host "=========================================================="
Write-host "The value of Variable '[variableName]' is updated to" $updatedef.variables.[variableName].value
write-host "=========================================================="
Please note that in order to get this working there are a couple of things you need to do.
First you need to allow access to the OAuth token on the Agent Job.
On top of that you need to give the "Project Collection Build Service".
Click on security on the release and click on the "Project Collection Build Service" user.
Change the values for the "Edit release" and "Manage release" to allow and save the changes.

Authenticate Azure Pipelines Powershell task against Azure DevOps Rest API

Requirements
My requirement is that for my Azure devops release pipeline I want to tag a specific commit with an annotated tag which contains the build number and the date (which is auto-set on an annotated tag).
Proposed solution
My solution to this is to use a the Azure Powershell pipeline task, the one shown here:
The task (ignore what's in the script box right now) will use the Azure Subscription that I have set in order to authenticate towards the Azure DevOps REST API. I have successfully been able to perform the task I want using a personal access token (PAT) but this is not stable long-term for a whole team and I want to use our Azure Subscription.
The problem
My problem is that I'm not sure how to use the authentication of the Azure Subscription correctly. I seem to get some data using Get-AzureRmContext (see current code below) and then I found a GitHub issue which seems to do sort of the same thing. The code gets some kind of OAuth token but using the code below, Azure still returns to me that I need to sign in, so I assume it's not the correct token. I don't understand how things hatch into each other.
Note that my subscription should have all the permissions it needs to do what I want.
Code so far:
Function Get-AccessToken($tenantId) {
$cache = [Microsoft.IdentityModel.Clients.ActiveDirectory.TokenCache]::DefaultShared
$cacheItem = $cache.ReadItems() | Where-Object { $_.TenantId -eq $tenantId } | Select-Object -First 1
return $cacheItem.AccessToken
}
$context = Get-AzureRmContext
$uri = "https://dev.azure.com/<my_org>/<my_area>/_apis/git/repositories/<project_sha>/annotatedtags?api-version=5.0-preview.1"
$token = Get-AccessToken $context.tenantID
$body = #"
{
"taggedObject": {
"objectId": "$(BUILD.SOURCEVERSION)"
},
"name": "D-$(Build.BuildNumber)",
"message": "dummy"
}
"#
$header = #{"Authorization" = "Bearer" + $token}
Invoke-RestMethod -Uri $uri -Method Post -ContentType "application/json" -Body $body -Headers $header
Any help is greatly appreciated!
There is the example for the build tasks: Use a PowerShell script to customize your build pipeline
You have to enable access to token (option Allow Scripts to Access OAuth Token)
Then use it in your script. Script from the example:
$url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/$($env:SYSTEM_DEFINITIONID)?api-version=5.0"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"

How can one automate promotion of an artifact to a feed view in Azure DevOps?

Our build artifact is an Octopus nuget package. When the build is released it lands into the QA stage where the artifact is deployed through Octopus. This octopus consumes it directly from the Azure Artifacts nuget feed.
If the deployment and the subsequent tests are successful we want to promote the artifact to the Release view of the Azure Artifacts nuget feed, because we think it gives us a different nuget URL that can be used by another Octopus serving the next stage (for historic reasons we have dedicated Octopus per stage - working to change that, but it takes time).
We can promote manually, but we want to do it automatically. How can this be done?
We are testing it on on-premises TFS 2019 RC2.
EDIT 1
The suggested plugin does not seem to install on on-premises TFS 2019 RC2:
Using PowerShell...
$organisationName = '' # Name of organisation
$projectName = '' # Name of project
$feedName = '' # Name of Azure Artifacts feed
$viewName = 'Release' # I believe this can also be Prerelease, but I've not tried it
# List of names of packages within Azure Artifacts feed to be promoted
$packagesToPromote = #('')
# Need a personal access token for this script to work
# PAT token should be assigned to Packaging (Read, Write and Manage) scopes
$azureArtifactsPAT = ''
$AzureArtifactsPAT_Base64 = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($azureArtifactsPAT)"))
$restAPICallHeader = #{ Authorization = "Basic $AzureArtifactsPAT_Base64" }
$feedBaseURL = "https://feeds.dev.azure.com/$organisationName/$projectName/_apis/packaging/feeds"
$packageBaseURL = "https://pkgs.dev.azure.com/$organisationName/$projectName/_apis/packaging/feeds"
$feedIdURL = "$feedBaseURL/$feedName/?api-version=5.1-preview.1"
$feedIdResponse = (Invoke-RestMethod -Method Get -Uri $feedIdUrl -Headers $restAPICallHeader -ContentType 'application/json')
$feedId = $feedIdResponse.id
$viewIdURL = "$feedBaseURL/$feedId/views/$viewName/?api-version=5.1-preview.1"
$viewIdResponse = (Invoke-RestMethod -Method Get -Uri $viewIdUrl -Headers $restAPICallHeader -ContentType 'application/json')
$viewId = $viewIdResponse.id
$restAPICallBodyJson = #{
views = #{
op = 'add'
path = '/views/-'
value = "$viewId"
}
}
$restAPICallBody = (ConvertTo-Json $restAPICallBodyJson)
foreach ($packageName in $packagesToPromote) {
$packageQueryUrl = "$feedBaseURL/$feedId/packages?api-version=5.1-preview.1&packageNameQuery=$packageName"
$packagesResponse = (Invoke-RestMethod -Method Get -Uri $packageQueryUrl -Headers $restAPICallHeader -ContentType 'application/json')
$latestPackageVersion = ($packagesResponse.value.versions | ? { $_.isLatest -eq $True } | Select -ExpandProperty version)
$encodedPackageVersion = [System.Web.HttpUtility]::UrlEncode($latestPackageVersion)
Write-Host "Package Name: $packageName"
Write-Host "Package Version: $latestPackageVersion"
$releaseViewURL = $packageBaseURL `
+ "/$($feedId)" `
+ "/nuget/packages/$packageName" `
+ "/versions/$encodedPackageVersion" `
+ "?api-version=5.1-preview.1"
$response = Invoke-RestMethod -Method Patch -Uri $releaseViewURL -Headers $restAPICallHeader -ContentType 'application/json' -Body $restAPICallBody
Write-Host $response
}
For reference, the script above uses the following API calls:
Feed Management - Get Feed
https://learn.microsoft.com/en-us/rest/api/azure/devops/artifacts/feed%20%20management/get%20feed?view=azure-devops-rest-5.1
Feed Management - Get Feed View
https://learn.microsoft.com/en-us/rest/api/azure/devops/artifacts/feed%20%20management/get%20feed%20view?view=azure-devops-rest-5.1
Artifact Details - Get Packages
https://learn.microsoft.com/en-us/rest/api/azure/devops/artifacts/artifact%20%20details/get%20packages?view=azure-devops-rest-5.1
NuGet - Update Package Version
https://learn.microsoft.com/en-us/rest/api/azure/devops/artifactspackagetypes/nuget/update%20package%20version?view=azure-devops-rest-5.1
As per Azure DevOps documentation the marketplace task Promote package to Release View is the recommended way to accomplish this from a CI/CD pipeline.
The repository can be found on Github.
Edit:
Since you are on-prem with a version that this task doesn't support. I would say that the comments about using the REST api would be the route you need to go in something like a powershell script.
Having never used the REST Api for this task I'm not exactly sure how the body is supposed to look for the request. However, it seems to be documented here.
My understanding of the JSON Patch object is limited, but I would think you might use the replace operation.
{ "op": "replace", "path": "/view", "value": "#Release" }
This article may also be helpful, but I still don't see anything that would relate to the from identifier on the JsonPatchObject definition in the REST Api documentation.
I also recently struggled with trying to implement version using TFS. I've produced some PowerShell scripts (adapting other scripts out on the web) to do package versioning.
https://gist.github.com/oceanexplorer/6a91930419b35c1923974af265777a5f
https://gist.github.com/oceanexplorer/35e0f26962018dc8578c745060365c15
The first step is my build pipeline I use the "Update AssemblyInfo" task to set the build version which then gets embedded into the DLL's.
https://marketplace.visualstudio.com/items?itemName=sebastianlux.UpdateAssemblyInfo
Initially I embedded the above scripts with my project to get things going but eventually in my release pipeline I then have a task that deploys these build scripts via a "NuGet Install" task which effectively pulls them from a feed and unzips them.
In the release pipeline I then have a task "Version Package" which is a custom PowerShell script that calls functions defined in the two gists above, what these do is to unzip the NuGet packages that have been created from the build pipeline and placed in the artifact directory, applies the correct versioning to the package and zips it back up. I have used the following build number format in my build pipeline:
$(version.major).$(version.minor).$(version.patch).$(Date:yyyyMMdd)$(Rev:r)-CI
1.0.0.201902051-CI
This will produce a semantic build number format of:
1.0.0-alpha.201902051
I call the scripts using an inline PowerShell task
##-------------------------------------------
## Import Build Scripts
##-------------------------------------------
gci -Recurse "$(System.DefaultWorkingDirectory)\scripts\*.psm1" | ForEach-Object { Import-Module $_.FullName }
##-------------------------------------------
## Version Files
##-------------------------------------------
Expand-NugetPackages -packagesDirectory "$(artifact.directory)" -Verbose
Add-VersionToAssemblies -suffix "$(Release.EnvironmentName)" -semVer "2.0" -artifactsToApplyTo "nuspec" -isRelease $(isRelease) -Verbose
Compress-NugetPackages -packagesDirectory "$(artifact.directory)" -Verbose
Then a NuGet push task to push the package
Them another inline PowerShell script which sets the release view for the package feed:
##-------------------------------------------
## Import Build Scripts
##-------------------------------------------
gci -Recurse "$(System.DefaultWorkingDirectory)\scripts\*.psm1" | ForEach-Object { Import-Module $_.FullName }
##-------------------------------------------
## Set Package Quality
##-------------------------------------------
Set-PackageQuality -feedName "Libraries" -packageId $(nuget.packageId) -packageVersion $env:semanticVersion -packageQuality $(Release.EnvironmentName)
Something changed recently in the ADO RestApi.
What worked for me was sending a PATCH request like:
curl --location --request PATCH 'https://pkgs.dev.azure.com/YOUR_ORGANIZATION/YOUR_PROJECT/_apis/packaging/feeds/YOUR_FEEDNAME/upack/packages/YOUR_PACKAGENAME/versions/YOUR_PACKAGEVERSION?api-version=5.1-preview.1' \
--header 'Authorization: Basic YOUR_TOKEN' \
--header 'Content-Type: application/json' \
--data-raw '{
"views": {
"op": "add",
"path": "/views/-",
"value": "Release"
}
}'