Azure Functions ARM template redeployment deletes my published functions - azure-devops

I have an Azure Functions (2.0) instance deployed by an ARM Template using Azure DevOps pipelines. I have another pipeline that deploys a functions application to the instance through zip deploy. This almost works perfectly, however, if I deploy the functions Infrastructure as Code, then deploy the app and then redeploy the Infrastructure as Code, my functions app is removed and all the functions disappear. I am using the incremental deployment so I cont see why it does this. Any thoughts on why it is behaving like this or how to troubleshoot?
I have copied my resource group deployment script and ARM template below.
New-AzureRmResourceGroupDeployment -Name ((Get-ChildItem $TemplateFile).BaseName + '-' + ((Get-Date).ToUniversalTime()).ToString('MMdd-HHmm')) `
-ResourceGroupName $ResourceGroupName `
-TemplateFile $TemplateFile `
-TemplateParameterFile $TemplateParametersFile `
-Mode Incremental `
#OptionalParameters `
-Force -Verbose `
-AdminsGroup $AdminsGroup `
-AdminsGroupObjectId $AdminsGroupObjectId `
-ErrorVariable ErrorMessages
ARM template
{
"type": "Microsoft.Web/serverfarms",
"sku": {
"name": "Y1",
"tier": "Dynamic",
"size": "Y1",
"family": "Y",
"capacity": 0
},
"kind": "app",
"apiVersion": "2016-09-01",
"name": "[variables('FunctionPlanNameMyStuff')]",
"location": "[resourceGroup().location]",
"properties": {
"name": "[variables('FunctionPlanNameMyStuff')]",
"perSiteScaling": false,
"reserved": false,
"targetWorkerCount": 0,
"targetWorkerSizeId": 0
}
},
{
"comments": "MyStuff Functions Site",
"type": "Microsoft.Web/sites",
"kind": "functionapp",
"name": "[variables('FunctionSiteNameMyStuff')]",
"apiVersion": "2016-08-01",
"location": "[resourceGroup().location]",
"identity": {
"type": "SystemAssigned"
},
"scale": null,
"properties": {
"enabled": true,
"hostNameSslStates": [
{
"name": "[concat(variables('FunctionSiteNameMyStuff'),'.azurewebsites.net')]",
"sslState": "Disabled",
"virtualIP": null,
"thumbprint": null,
"toUpdate": null,
"hostType": "Standard"
},
{
"name": "[concat(variables('FunctionSiteNameMyStuff'),'.scm.azurewebsites.net')]",
"sslState": "Disabled",
"virtualIP": null,
"thumbprint": null,
"toUpdate": null,
"hostType": "Repository"
}
],
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('FunctionPlanNameMyStuff'))]",
"reserved": false,
"siteConfig": null,
"scmSiteAlsoStopped": false,
"hostingEnvironmentProfile": null,
"clientAffinityEnabled": false,
"clientCertEnabled": false,
"hostNamesDisabled": false,
"containerSize": 1536,
"dailyMemoryTimeQuota": 0,
"cloningInfo": null
},
"resources": [
{
"apiVersion": "2015-08-01",
"name": "appsettings",
"type": "config",
"dependsOn": [
"[resourceId('Microsoft.Web/Sites', variables('FunctionSiteNameMyStuff'))]",
"[resourceId('Microsoft.Storage/storageAccounts', variables('FunctionStorageAccountMyStuff'))]"
],
"properties": {
"AzureWebJobsStorage": "[concat('DefaultEndpointsProtocol=https;AccountName=', variables('FunctionStorageAccountMyStuff'), ';AccountKey=', listKeys(variables('FunctionStorageResourceIdMyStuff'), '2017-10-01').keys[0].value)]",
"AzureWebJobsDashboard": "[concat('DefaultEndpointsProtocol=https;AccountName=', variables('FunctionStorageAccountMyStuff'), ';AccountKey=', listKeys(variables('FunctionStorageResourceIdMyStuff'), '2017-10-01').keys[0].value)]",
"WEBSITE_CONTENTAZUREFILECONNECTIONSTRING": "[concat('DefaultEndpointsProtocol=https;AccountName=', variables('FunctionStorageAccountMyStuff'), ';AccountKey=', listKeys(variables('FunctionStorageResourceIdMyStuff'),'2015-05-01-preview').key1)]",
"WEBSITE_CONTENTSHARE": "[toLower(variables('FunctionSiteNameMyStuff'))]",
"FUNCTIONS_EXTENSION_VERSION": "~2",
"FUNCTIONS_WORKER_RUNTIME": "dotnet",
"FUNCTION_APP_EDIT_MODE": "readwrite",
"KeyVaultUrl": "[concat('https://', variables('KeyVaultName'), '.vault.azure.net/')]",
"DeveloperMode": false
}
}
],
"dependsOn": [
"[resourceId('Microsoft.Web/serverfarms', variables('FunctionPlanNameMyStuff'))]",
"[resourceId('Microsoft.Storage/storageAccounts', variables('FunctionStorageAccountMyStuff'))]"
]
},
{
"comments": "Functions web site config",
"type": "Microsoft.Web/sites/config",
"name": "[concat(variables('FunctionSiteNameMyStuff'), '/', 'web')]",
"apiVersion": "2016-08-01",
"location": "[resourceGroup().location]",
"tags": {
},
"scale": null,
"properties": {
"numberOfWorkers": 1,
"defaultDocuments": [
"Default.htm",
"Default.html",
"Default.asp",
"index.htm",
"index.html",
"iisstart.htm",
"default.aspx",
"index.php",
"hostingstart.html"
],
"netFrameworkVersion": "v4.0",
"phpVersion": "5.6",
"pythonVersion": "",
"nodeVersion": "",
"linuxFxVersion": "",
"requestTracingEnabled": false,
"remoteDebuggingEnabled": false,
"remoteDebuggingVersion": null,
"httpLoggingEnabled": false,
"logsDirectorySizeLimit": 35,
"detailedErrorLoggingEnabled": false,
"publishingUsername": "[concat('$', variables('FunctionSiteNameMyStuff'), 'pubuser')]",
"publishingPassword": "[concat(variables('FnAppPublishingPasswordMyStuff'), uniqueString(resourceGroup().id))]",
"appSettings": null,
"metadata": null,
"connectionStrings": null,
"machineKey": null,
"handlerMappings": null,
"documentRoot": null,
"scmType": "None",
"use32BitWorkerProcess": true,
"webSocketsEnabled": false,
"alwaysOn": false,
"javaVersion": null,
"javaContainer": null,
"javaContainerVersion": null,
"appCommandLine": "",
"managedPipelineMode": "Integrated",
"virtualApplications": [
{
"virtualPath": "/",
"physicalPath": "site\\wwwroot",
"preloadEnabled": false,
"virtualDirectories": null
}
],
"winAuthAdminState": 0,
"winAuthTenantState": 0,
"customAppPoolIdentityAdminState": false,
"customAppPoolIdentityTenantState": false,
"runtimeADUser": null,
"runtimeADUserPassword": null,
"loadBalancing": "LeastRequests",
"routingRules": [],
"experiments": {
"rampUpRules": []
},
"limits": null,
"autoHealEnabled": false,
"autoHealRules": null,
"tracingOptions": null,
"vnetName": "",
"siteAuthEnabled": false,
"siteAuthSettings": {
"enabled": null,
"unauthenticatedClientAction": null,
"tokenStoreEnabled": null,
"allowedExternalRedirectUrls": null,
"defaultProvider": null,
"clientId": null,
"clientSecret": null,
"issuer": null,
"allowedAudiences": null,
"additionalLoginParams": null,
"isAadAutoProvisioned": false,
"googleClientId": null,
"googleClientSecret": null,
"googleOAuthScopes": null,
"facebookAppId": null,
"facebookAppSecret": null,
"facebookOAuthScopes": null,
"twitterConsumerKey": null,
"twitterConsumerSecret": null,
"microsoftAccountClientId": null,
"microsoftAccountClientSecret": null,
"microsoftAccountOAuthScopes": null
},
"cors": null,
"push": null,
"apiDefinition": {
"url": "[concat('https://', variables('FunctionPlanNameMyStuff'), '.azurewebsites.net/swagger/docs/v1')]"
},
"autoSwapSlotName": null,
"localMySqlEnabled": false,
"managedServiceIdentityId": null,
"ipSecurityRestrictions": null,
"http20Enabled": false,
"minTlsVersion": "1.0"
},
"dependsOn": [
"[resourceId('Microsoft.Web/sites', variables('FunctionSiteNameMyStuff'))]"
]
},

Add this app setting to your template:
{ "name": "WEBSITE_RUN_FROM_PACKAGE", "value": "1" }
Functions can only be run from zip file if this setting is present, and redeploying a template without it will remove existing value for this setting (if there was any).

So #curiouscoder's answer does work but it requires more than a one liner really so I'm expanding on it here.
I hit this issue after adding some CORS origins to my ARM template, when I ran the template it removed the functions, to fix it you need to do the following (this assumes a Windows based function app which yours is and so is mine and that you're deploying from Azure DevOps):
Add the WEBSITE_RUN_FROM_PACKAGE setting to your function's app service settings and set it to 1 (can also be set to a URL where the package is but that affects cold start up negatively for function apps)
Now confusingly in your Azure DevOps release pipeline set your deployment method to "zip deploy" and not to "run from package"!
Re-running your ARM template with changes will now not remove the site
The function app will now be running from the zip file rather from a deployment direct to the wwwroot folder, this means the wwwroot folder is made readonly. You'll get errors if you try to edit anything there:
The benefit of this approach is that you can't half deploy your app and having it read only means you can't accidentally mess with it once deployed.
The above does not work as smoothly as a slot change and will require the app to be restarted so it needs to be used in conjunction with a staging app or slots although as of writing the latter are still in preview for function apps.
The original github announcement is worth a read here as they're more detailed than the docs.

Your ARM template is problematic because it sets the App Settings as a separate action from the app creation, which is not supported when using Azure Files.
See this sample for guidance.

Related

Azure DevOps KeyVault Linked Variable group "Value cannot be null. Parameter name: variableGroupParameters" - how do i fix this?

I have a need to automate the creation of KeyVault linked variable groups in ADO as part of a pipeline task. I can actually create the var groups after a bit of experimenting. However, using the az devops invoke method one is unable to specify the Azure Subscription and this has to be done manually, after the event - however when I do this in the web interface and attempt to save I get the error:
Value cannot be null. Parameter name: variableGroupParameters
This means any subsequent editing of the created KeyVault linked var group is pointless as it is unable to save it.
The JSON I am submitting is as follows:
{
"authorized": true,
"description": "$description",
"name": "$name",
"type": "AzureKeyVault",
"variableGroupProjectReferences": [{
"projectReference": {
"id": "$adoProjectID",
"name": "$ProjectName"
},
"name": "$name",
"description": "$description"
}],
"providerData": {
"serviceEndpointId": "$AdoSvcConnId",
"vault": "$KeyVaultNM"
},
"variables": {
"SOMETHINGSECRET": {
"isSecret": true,
"value": null,
"enabled": true,
"contentType": ""
},
"variables": {
"ANOTHERSECRET": {
"isSecret": true,
"value": null,
"enabled": true,
"contentType": ""
}
}
}
}
Where the $tokenized values are replaced during the powershell/az cli task
The command (which works but results in broken var group that can't be edited or saved) is as follows:
az devops invoke --http-method post --area distributedtask --resource variablegroups --in-file "vgroup_azure_rm.json" --encoding utf-8 --route-parameters project=$ProjectName --api-version 5.0-preview
Anyone have an insight in how to fix this please?
I was experiencing the same problem, and the message had nothing to do with the real issue.
I had the following variables:
"variables": {
"SECRET_1": {
"contentType": "",
"isSecret": true,
"value": "",
"expires": null,
"enabled": true
},
"SECRET_2": {
"contentType": "",
"isSecret": true,
"value": "",
"expires": null,
"enabled": true
},
"SECRET_3": {
"contentType": "",
"isSecret": true,
"value": "",
"expires": null,
"enabled": true
}
}
And was receiving the same error as you. I removed the expires property and the issue was solved! So my thinking is that the validation checks for any null value and returns the error you are receiving.
Try changing the property value to an empty string ("") instead of null and see whether that solves the problem.

Whats wrong in my camunda fetchAndLock API request. Getting InvalidRequestException

I have 2 topics to receive data from API, those I can Successfully executed through code. Now I'm trying to execute through rest api using postman tool. now i'm getting InvalidRequestException. Before attempting request I fetched the external tasks using camunda get external-task api and my topics showing there.Later I tried to use /external-task/fetchAndLock API to send input variables.
External tasks response is:
http://localhost:8080/engine-rest/external-task
[
{
"activityId": "Activity_0jokenq",
"activityInstanceId": "Activity_0jokenq:0623e6f2-4837-11ec-8c7e-02426d005d3a",
"errorMessage": null,
"executionId": "0623e6f1-4837-11ec-8c7e-02426d005d3a",
"id": "0623e6f3-4837-11ec-8c7e-02426d005d3a",
"lockExpirationTime": null,
"processDefinitionId": "Process_0qcjqnm:1:da2ae20a-4836-11ec-8c7e-02426d005d3a",
"processDefinitionKey": "Process_0qcjqnm",
"processDefinitionVersionTag": null,
"processInstanceId": "0623bfdb-4837-11ec-8c7e-02426d005d3a",
"retries": null,
"suspended": false,
"workerId": null,
"topicName": "yvalue",
"tenantId": null,
"priority": 0,
"businessKey": null
},
{
"activityId": "Activity_1xxpyet",
"activityInstanceId": "Activity_1xxpyet:0623e6f6-4837-11ec-8c7e-02426d005d3a",
"errorMessage": null,
"executionId": "0623e6f5-4837-11ec-8c7e-02426d005d3a",
"id": "0623e6f7-4837-11ec-8c7e-02426d005d3a",
"lockExpirationTime": null,
"processDefinitionId": "Process_0qcjqnm:1:da2ae20a-4836-11ec-8c7e-02426d005d3a",
"processDefinitionKey": "Process_0qcjqnm",
"processDefinitionVersionTag": null,
"processInstanceId": "0623bfdb-4837-11ec-8c7e-02426d005d3a",
"retries": null,
"suspended": false,
"workerId": null,
"topicName": "testingtopic",
"tenantId": null,
"priority": 0,
"businessKey": null
}
]
my request is:
POST http://localhost:8080/engine-rest/external-task/fetchAndLock
{
"workerId": 1,
"maxTasks": 100,
"topics": [
{
"topicName": "testingtopic",
"lockDuration": 100000,
"variables": {
"a": {
"value": 1,
"type": "long"
},
"b": {
"value": 2,
"type": "long"
},
"id": {
"value": 1,
"type": "long"
}
}
}
],
"asyncResponseTimeout": 5
}
my BPMN diagram is:
Sorry mistake was mine I mentioned wrongly in request body. I mentioned
"variables": {}
But it's a array of json "variables": []
I mentioned here just variable names "variables": ["a","b","id"]
Later I used POST /external-task/{id}/complete request to pass with values to complete the process

Azure DevOps server import pipeline functionality ignores Approvers definition

I have the following definition in my pipeline that I am running on Azure DevOps Server Version Dev17.M153.3
After saving the change I can see the following content has been added to pipeline definition
"approvals": [
{
"rank": 1,
"isAutomated": false,
"isNotificationOn": false,
"approver": {
"displayName": "Aouslender, Alexey",
"url": "http://tdc1tfsapp01:8080/tfs/DefaultCollection/_apis/Identities/2d86d86b-fe02-4e22-aa53-4315cdb3821c",
"_links": {
"avatar": {
"href": "http://tdc1tfsapp01:8080/tfs/DefaultCollection/_apis/GraphProfile/MemberAvatars/win.Uy0xLTUtMjEtMzMwNDk4NzQ2Ni0xODkxMDA3NDIzLTI5MjUxNTc3OTctNDU4NDA1"
}
},
"id": "2d86d86b-fe02-4e22-aa53-4315cdb3821c",
"uniqueName": "DOMAIN\\PXXXXXX",
"imageUrl": "http://tdc1tfsapp01:8080/tfs/DefaultCollection/_apis/GraphProfile/MemberAvatars/win.Uy0xLTUtMjEtMzMwNDk4NzQ2Ni0xODkxMDA3NDIzLTI5MjUxNTc3OTctNDU4NDA1",
"descriptor": "win.Uy0xLTUtMjEtMzMwNDk4NzQ2Ni0xODkxMDA3NDIzLTI5MjUxNTc3OTctNDU4NDA1"
},
"id": 3546
}
]
Now I am exporting pipeline, using export option. Then I am deleting pipeline and importing it using exported json file.
The imported pipeline missing the Approvers definition, nevertheless I can see the definition in exported json.
"preDeployApprovals": {
"approvals": [
{
"rank": 1,
"isAutomated": false,
"isNotificationOn": false,
"approver": {
"displayName": "Aouslender, Alexey",
"url": "http://tdc1tfsapp01:8080/tfs/DefaultCollection/_apis/Identities/2d86d86b-fe02-4e22-aa53-4315cdb3821c",
"_links": {
"avatar": {
"href": "http://tdc1tfsapp01:8080/tfs/DefaultCollection/_apis/GraphProfile/MemberAvatars/win.Uy0xLTUtMjEtMzMwNDk4NzQ2Ni0xODkxMDA3NDIzLTI5MjUxNTc3OTctNDU4NDA1"
}
},
"id": "2d86d86b-fe02-4e22-aa53-4315cdb3821c",
"uniqueName": "DOMAIN\\PXXXXXX",
"imageUrl": "http://tdc1tfsapp01:8080/tfs/DefaultCollection/_apis/GraphProfile/MemberAvatars/win.Uy0xLTUtMjEtMzMwNDk4NzQ2Ni0xODkxMDA3NDIzLTI5MjUxNTc3OTctNDU4NDA1",
"descriptor": "win.Uy0xLTUtMjEtMzMwNDk4NzQ2Ni0xODkxMDA3NDIzLTI5MjUxNTc3OTctNDU4NDA1"
},
"id": 3535
}
],
"approvalOptions": {
"requiredApproverCount": null,
"releaseCreatorCanBeApprover": true,
"autoTriggeredAndPreviousEnvironmentApprovedCanBeSkipped": false,
"enforceIdentityRevalidation": false,
"timeoutInMinutes": 0,
"executionOrder": 1
}
}
Am I missing something here or is it actually a Microsoft bug?
It's by design. Following properties in the release definition are not imported: Agent Queues, Deployment Groups, Deployment Group Tags, Approvals, Variable Groups and values of secret variables.
Generally if in the same team project you can Clone the release definition directly.

Azure DevOps API release definition

I'm getting the following error when trying to copy a release template from a different project
"VS402982: A retention policy is not set for the stage ‘PROJECT-ENV’. Retention policies at the release pipeline level are deprecated.
{
"id": 8,
"name": "PROJECT-ENV",
"rank": 1,
"owner": "#{displayName=Zoe MacKay; _links=; id=; uniqueName=Zoe.MacKay;imageUrl=https://dev.azure.com/incrementalgroup/_apis/GraphProfile/MemberAvatars/}",
"variables": "",
"variableGroups": "",
"preDeployApprovals": "#{approvals=System.Object[]; approvalOptions=}",
"deployStep": "#{id=27}",
"postDeployApprovals": "#{approvals=System.Object[]; approvalOptions=}",
"deployPhases": "",
"environmentOptions": "#{emailNotificationType=OnlyOnFailure; emailRecipients=release.environment.owner;release.creator; skipArtifactsDownload=False; timeoutInMinutes=0; enableAccessToken=False; publishDeploymentStatus=True; badgeEnabled=False; autoLinkWorkItems=False; pullRequestDeploymentEnabled=False}",
"demands": "",
"conditions": "",
"executionPolicy": "#{concurrencyCount=1; queueDepthCount=0}",
"schedules": "",
"currentRelease": "#{id=70; url=https://vsrm.dev.azure.com/; _links=}",
"retentionPolicy": "#{daysToKeep=30; releasesToKeep=3; retainBuild=True}",
"processParameters": "",
"properties": "",
"preDeploymentGates": "#{id=0; gatesOptions=; gates=System.Object[]}",
"postDeploymentGates": "#{id=0; approvals:[]; gatesOptions=; gates=System.Object[]}",
"environmentTriggers": ""
},
copying from current build.
Your JSON is not valid. You'll need to ensure all the nested values are expanded correctly. For example:
"retentionPolicy": "#{daysToKeep=30; releasesToKeep=3; retainBuild=True}",
should be
"retentionPolicy": {
"daysToKeep" : 30,
"releasesToKeep": 3,
"retainBuild": true
},
If you're using PowerShell to create this JSON then ensure to specify the Depth parameter so it expands each value correctly.

TFS 2015 Rest API - Create Build Definition

I'm trying to create a build definition in TFS using his api rest for it.
This is the microsoft docs for TFS Api Rest
I get status code 200 but...
This is what i get when I look into Tfs build definition
Anyone knows why this is happening?
Your second link provided a screenshot that a build definition with no tasks. It seems you had created a build definition successfully, but you didn't add tasks in the build definition.
Check your rest api to see whether you have "build": [] parameter with "task" in it as the following example shows:
"build": [
{
"enabled": true,
"continueOnError": false,
"alwaysRun": false,
"displayName": "Build solution **\\*.sln",
"task": {
"id": "71a9a2d3-a98a-4caa-96ab-affca411ecda",
"versionSpec": "*"
},
"inputs": {
"solution": "**\\*.sln",
"msbuildArgs": "",
"platform": "$(platform)",
"configuration": "$(config)",
"clean": "false",
"restoreNugetPackages": "true",
"vsLocationMethod": "version",
"vsVersion": "latest",
"vsLocation": "",
"msbuildLocationMethod": "version",
"msbuildVersion": "latest",
"msbuildArchitecture": "x86",
"msbuildLocation": "",
"logProjectEvents": "true"
}
},
{
"enabled": true,
"continueOnError": false,
"alwaysRun": false,
"displayName": "Test Assemblies **\\*test*.dll;-:**\\obj\\**",
"task": {
"id": "ef087383-ee5e-42c7-9a53-ab56c98420f9",
"versionSpec": "*"
},
"inputs": {
"testAssembly": "**\\*test*.dll;-:**\\obj\\**",
"testFiltercriteria": "",
"runSettingsFile": "",
"codeCoverageEnabled": "true",
"otherConsoleOptions": "",
"vsTestVersion": "14.0",
"pathtoCustomTestAdapters": ""
}
}
],