Related
I'm setting up an ARM template for my Azure Functions. I have functions_app.json which will function as the "main" template and will be used for other release pipelines as well.
Then there will be project specific parameter templates, like:
counter_function_arm.parameters.test.json
counter_function_arm.parameters.acceptance.json
weather_function_arm.parameters.test.json
etc.
But I'm stuck at this part where it creates the actual Function resource inside the functions_app.json template:
"resources": [
{
"apiVersion": "2018-11-01",
"name": "[variables('resourceName')]",
"type": "Microsoft.Web/sites",
"properties": {
"name": "[variables('resourceName')]",
"siteConfig": {
"appSettings": [
{
"name": "FUNCTIONS_EXTENSION_VERSION",
"value": "~4"
},
{
"name": "FUNCTIONS_WORKER_RUNTIME",
"value": "dotnet"
},
{
"name": "AzureWebJobsStorage",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=',variables('storageAccountName'),';AccountKey=',listKeys(resourceId('xxxx-xxxx','MyResourceGroup','Microsoft.Storage/storageAccounts', variables('storageAccountName')), '2019-06-01').keys[0].value,';EndpointSuffix=','core.windows.net')]"
},
{
"name": "WEBSITE_CONTENTAZUREFILECONNECTIONSTRING",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=',variables('storageAccountName'),';AccountKey=',listKeys(resourceId('xxxx-xxxx','MyResourceGroup','Microsoft.Storage/storageAccounts', variables('storageAccountName')), '2019-06-01').keys[0].value,';EndpointSuffix=','core.windows.net')]"
},
{
"name": "WEBSITE_CONTENTSHARE",
"value": "[toLower(variables('resourceName'))]"
}
]
}
}
}
I left some things out for brevity. The import part is the appSettings section.
My initial idea was to put that array in variables. Those will be the "default" settings. Then from my counter_function_arm.parameters.test.json file I would add another array specific to that application and then union those two together so all AppSettings are configured.
Something like this:
counter_function_arm.parameters.test.json
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"customAppSettings":{
"value": [
{
"name": "CustomProperty",
"value": "some value"
}
]
}
}
}
functions_app.json
"variables": {
"defaultAppSettings": [
{
"name": "FUNCTIONS_EXTENSION_VERSION",
"value": "~4"
},
{
"name": "FUNCTIONS_WORKER_RUNTIME",
"value": "dotnet"
},
{
"name": "AzureWebJobsStorage",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=',variables('storageAccountName'),';AccountKey=',listKeys(resourceId('xxxx-xxxx','MyResourceGroup','Microsoft.Storage/storageAccounts', variables('storageAccountName')), '2019-06-01').keys[0].value,';EndpointSuffix=','core.windows.net')]"
},
{
"name": "WEBSITE_CONTENTAZUREFILECONNECTIONSTRING",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=',variables('storageAccountName'),';AccountKey=',listKeys(resourceId('xxxx-xxxx','MyResourceGroup','Microsoft.Storage/storageAccounts', variables('storageAccountName')), '2019-06-01').keys[0].value,';EndpointSuffix=','core.windows.net')]"
},
{
"name": "WEBSITE_CONTENTSHARE",
"value": "[toLower(variables('resourceName'))]"
}
]
},
"resources": [
{
"apiVersion": "2018-11-01",
"name": "[variables('resourceName')]",
"type": "Microsoft.Web/sites",
"properties": {
"name": "[variables('resourceName')]",
"siteConfig": {
"appSettings": "[union(variables('defaultAppSettings'), parameters('customAppSettings'))]"
}
}
}
So in this case I union the variable variables('defaultAppSettings') and parameters('customAppSettings') together.
But, the problem is that I can't use listKeys in the variables section. I also can't move this to the parameters section and set it as defaultValue because listKeys is also not allowed there.
Adding a hard coded array to union also doesn't work:
"appSettings": "[union([1,2,3], parameters('customAppSettings'))]"
So I am out of ideas.
Is there a way to do this?
I've found a way around this, it is not pretty but it works... Basically I have an array of purely static app settings (in your case FUNCTIONS_EXTENSION_VERSION, FUNCTIONS_WORKER_RUNTIME, WEBSITE_CONTENTSHARE) that you can create in variables, an array of customAppSettings from parameters, and another array that I have to write as raw JSON.
Here's an example :
"resources": [
{
"apiVersion": "2018-11-01",
"name": "[variables('resourceName')]",
"type": "Microsoft.Web/sites",
"properties": {
"name": "[variables('resourceName')]",
"siteConfig": {
"appSettings": "[union(variables('staticAppSettings'), parameters('customAppSettings'), json(concat('[{\"name\": \"AzureWebJobsStorage\",\"value\": \"', concat('DefaultEndpointsProtocol=https;AccountName=', parameters('azureFunctionParameters').functionStorageName,';AccountKey=', listKeys(resourceId(parameters('azureFunctionParameters').mainStorageResourceGroup, 'Microsoft.Storage/storageAccounts', parameters('azureFunctionParameters').functionStorageName), '2021-08-01').keys[1].value, ';EndpointSuffix=core.windows.net'),'\"}]')))]"
}
}
}
]
Sadly this is the best I could come up with, it is very prone to syntax error but if you're using multi-line strings you could format it in a way to avoid errors (but be careful if you're using Azure Devops it is not compatible).
EDIT :
There is another way to accomplish this that may be more suitable for you, which is to use a linkedTemplate :
Main template
"resources": [
{
"type": "Microsoft.Resources/deployments",
"apiVersion": "2021-04-01",
"name": "yourResourceDeployment",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "[concat(parameters('blobBaseUrl'), '/', parameters('resourceGroupName'), '/azurefunctions/baseazurefunction.json')]",
"contentVersion": "1.0.0.0"
},
"parameters": {
"azureFunctionParameters": {
"value": "[parameters('baseAzureFunctionParameters')]"
},
"appSettings": {
"value" : "[union(variables('staticAppSettings'), parameters('customAppSettings'))]"
},
"someMoreAppSettings": {
"value" : [
{
"name": "AzureWebJobsStorage",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=',variables('storageAccountName'),';AccountKey=',listKeys(resourceId('xxxx-xxxx','MyResourceGroup','Microsoft.Storage/storageAccounts', variables('storageAccountName')), '2019-06-01').keys[0].value,';EndpointSuffix=','core.windows.net')]"
},
{
"name": "WEBSITE_CONTENTAZUREFILECONNECTIONSTRING",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=',variables('storageAccountName'),';AccountKey=',listKeys(resourceId('xxxx-xxxx','MyResourceGroup','Microsoft.Storage/storageAccounts', variables('storageAccountName')), '2019-06-01').keys[0].value,';EndpointSuffix=','core.windows.net')]"
}
]
}
}
}
}
]
In the baseazurefunction.json linkedTemplate you can just use
{
"apiVersion": "2018-11-01",
"name": "[variables('resourceName')]",
"type": "Microsoft.Web/sites",
"properties": {
"name": "[variables('resourceName')]",
"siteConfig": {
"appSettings": "[union(parameters('appSettings'), parameters('moreAppSettings'))]"
}
}
}
If you have a security concern over sending connection strings as parameters to other templates, you could wrap your appsettings in a secureobject with a property called array which would contain all your settings and then use it like this instead
"appSettings": {
"type": "secureobject",
"metadata": {
"description": "Secure object with an array property which contains the appsettings"
}
},
"moreAppSettings": {
"type": "secureobject",
"metadata": {
"description": "Secure object with an array property which contains more appsettings"
}
}
.
.
.
"appSettings": "[union(parameters('appSettings').array, parameters('moreAppSettings').array)]"
I'm running into an issue when I attempt to run the 'Azure Resource Group Deploy' release task to create/update a resource group and the resources within it via an ARM Template. In particular, I need to have the Virtual Machine created by the ARM template accessible via WinRM; This needs to be done so that I can copy files (specifically a ZIP file containing the results of a build) to the VM in a later step.
Currently, I have the 'Template' portion of this task set up as follows: https://i.imgur.com/mvZDIMK.jpg (I can't post images since I don't have reputation here yet...)
Unless I've misunderstood (which is definitely possible), the "Configure with WinRM" option should allow the release step to create a WinRM Listener on any Virtual Machines created by this step.
I currently have the following resources in the ARM Template:
{
"type": "Microsoft.Storage/storageAccounts",
"sku": {
"name": "Standard_LRS",
"tier": "Standard"
},
"kind": "Storage",
"name": "[variables('StorageAccountName')]",
"apiVersion": "2018-02-01",
"location": "[parameters('LocationPrimary')]",
"scale": null,
"tags": {},
"properties": {
"networkAcls": {
"bypass": "AzureServices",
"virtualNetworkRules": [],
"ipRules": [],
"defaultAction": "Allow"
},
"supportsHttpsTrafficOnly": false,
"encryption": {
"services": {
"file": {
"enabled": true
},
"blob": {
"enabled": true
}
},
"keySource": "Microsoft.Storage"
}
},
"dependsOn": []
},
{
"name": "[variables('NetworkInterfaceName')]",
"type": "Microsoft.Network/networkInterfaces",
"apiVersion": "2018-04-01",
"location": "[parameters('LocationPrimary')]",
"dependsOn": [
"[concat('Microsoft.Network/networkSecurityGroups/', variables('NetworkSecurityGroupName'))]",
"[concat('Microsoft.Network/virtualNetworks/', variables('VNetName'))]",
"[concat('Microsoft.Network/publicIpAddresses/', variables('PublicIPAddressName'))]"
],
"properties": {
"ipConfigurations": [
{
"name": "ipconfig1",
"properties": {
"subnet": {
"id": "[variables('subnetRef')]"
},
"privateIPAllocationMethod": "Dynamic",
"publicIpAddress": {
"id": "[resourceId(resourceGroup().name, 'Microsoft.Network/publicIpAddresses', variables('PublicIPAddressName'))]"
}
}
}
],
"networkSecurityGroup": {
"id": "[variables('nsgId')]"
}
},
"tags": {}
},
{
"name": "[variables('NetworkSecurityGroupName')]",
"type": "Microsoft.Network/networkSecurityGroups",
"apiVersion": "2018-08-01",
"location": "[parameters('LocationPrimary')]",
"properties": {
"securityRules": [
{
"name": "RDP",
"properties": {
"priority": 300,
"protocol": "TCP",
"access": "Allow",
"direction": "Inbound",
"sourceAddressPrefix": "*",
"sourcePortRange": "*",
"destinationAddressPrefix": "*",
"destinationPortRange": "3389"
}
}
]
},
"tags": {}
},
{
"name": "[variables('VNetName')]",
"type": "Microsoft.Network/virtualNetworks",
"apiVersion": "2018-08-01",
"location": "[parameters('LocationPrimary')]",
"properties": {
"addressSpace": {
"addressPrefixes": [ "10.0.0.0/24" ]
},
"subnets": [
{
"name": "default",
"properties": {
"addressPrefix": "10.0.0.0/24"
}
}
]
},
"tags": {}
},
{
"name": "[variables('PublicIPAddressName')]",
"type": "Microsoft.Network/publicIpAddresses",
"apiVersion": "2018-08-01",
"location": "[parameters('LocationPrimary')]",
"properties": {
"publicIpAllocationMethod": "Dynamic"
},
"sku": {
"name": "Basic"
},
"tags": {}
},
{
"name": "[variables('VMName')]",
"type": "Microsoft.Compute/virtualMachines",
"apiVersion": "2018-06-01",
"location": "[parameters('LocationPrimary')]",
"dependsOn": [
"[concat('Microsoft.Network/networkInterfaces/', variables('NetworkInterfaceName'))]",
"[concat('Microsoft.Storage/storageAccounts/', variables('StorageAccountName'))]"
],
"properties": {
"hardwareProfile": {
"vmSize": "Standard_A7"
},
"storageProfile": {
"osDisk": {
"createOption": "fromImage",
"managedDisk": {
"storageAccountType": "Standard_LRS"
}
},
"imageReference": {
"publisher": "MicrosoftWindowsDesktop",
"offer": "Windows-10",
"sku": "rs4-pro",
"version": "latest"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', variables('NetworkInterfaceName'))]"
}
]
},
"osProfile": {
"computerName": "[variables('VMName')]",
"adminUsername": "[parameters('AdminUsername')]",
"adminPassword": "[parameters('AdminPassword')]",
"windowsConfiguration": {
"enableAutomaticUpdates": true,
"provisionVmAgent": true
}
},
"licenseType": "Windows_Client",
"diagnosticsProfile": {
"bootDiagnostics": {
"enabled": true,
"storageUri": "[concat('https://', variables('StorageAccountName'), '.blob.core.windows.net/')]"
}
}
},
"tags": {}
}
This ARM Template currently works if I do not attempt to configure the VM to have the WinRM Listener.
When I attempt to run the release, I get the following error message:
Error number: -2144108526 0x80338012
The client cannot connect to the destination specified in the request. Verify that the service on the destination is running and is accepting requests. Consult the logs and documentation for the WS-Management service running on the destination, most commonly IIS or WinRM. If the destination is the WinRM service, run the following command on the destination to analyze and configure the WinRM service: "winrm quickconfig".
In all honesty, my problem is likely a lack of understanding, as this is my first time working with VM Setup in any real capacity. Any insight and advice would be greatly appreciated.
you just need to add this to the "windowsConfiguration":
"winRM": {
"listeners": [
{
"protocol": "http"
},
{
"protocol": "https",
"certificateUrl": "<URL for the certificate you got in Step 4>"
}
]
}
you also need to provision certificates
reference: https://learn.microsoft.com/en-us/rest/api/compute/virtualmachines/createorupdate#winrmconfiguration
https://learn.microsoft.com/en-us/azure/virtual-machines/windows/winrm
I have this JSON file which was converted from YAML:
{
"kind": "DeploymentConfig",
"apiVersion": "v1",
"metadata": {
"name": "cdt-cae-deployment"
},
"spec": {
"template": {
"metadata": {
"labels": {
"name": "cdt-cae"
},
"annotations": {
"app_version": "latest"
}
},
"spec": {
"containers": [
{
"name": "cdt-cae",
"image": "containers.nabisco.com/cdt-org/cdt-dev:__IMAGETAG__",
"ports": [
{
"containerPort": 8080,
"protocol": "TCP"
}
],
"env": [
{
"name": "APP_NAME",
"value": "cdt-cae"
},
{
"name": "CISCO_LC",
"value": "dev"
},
{
"name": "OPENSHIFT_MONGODB_DB_USERNAME",
"value": "cdtdev"
},
{
"name": "OPENSHIFT_MONGODB_DB_PORT",
"value": "27058"
},
{
"name": "OPENSHIFT_MONGODB_DB_HOST",
"value": "secret stuff here"
},
{
"name": "OPENSHIFT_MONGODB_DB_PASSWORD",
"valueFrom": {
"secretKeyRef": {
"name": "refapp-secret",
"key": "mongodb-password"
}
}
},
{
"name": "NOTIFICATIONS_CLIENT_SECRET",
"valueFrom": {
"secretKeyRef": {
"name": "refapp-secret",
"key": "notifications-client-secret"
}
}
}
],
"volumeMounts": [
{
"name": "podinfo",
"mountPath": "/etc/metadata",
"readOnly": false
}
],
"imagePullPolicy": "IfNotPresent",
"securityContext": {
"capabilities": {},
"privileged": false
}
}
],
"volumes": [
{
"name": "podinfo",
"downwardAPI": {
"items": [
{
"path": "labels",
"fieldRef": {
"fieldPath": "metadata.labels"
}
},
{
"path": "annotations",
"fieldRef": {
"fieldPath": "metadata.annotations"
}
}
]
}
}
],
"restartPolicy": "Always",
"dnsPolicy": "ClusterFirst"
}
},
"replicas": 3,
"selector": {
"name": "cdt-cae"
},
"triggers": [
{
"type": "ConfigChange"
}
],
"strategy": {
"type": "Rolling",
"rollingParams": {
"updatePeriodSeconds": 1,
"intervalSeconds": 1,
"timeoutSeconds": 120
}
}
}
}
unfortunately this is invalid JSON - I get this message:
Does anyone know what's wrong with the config? It looks like it's actually valid JSON, only that perhaps the schema is wrong..
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details
The
<value> expected, unexpected end of file
means that the thing processing the JSON thinks that the terminating } is missing.
The JSON quoted in the question is fine, but it's been changed from the original because of the secret fields. Maybe the secret has an unescaped quote?
Currently the '=' sign is forbidden in Orion:
http://fiware-orion.readthedocs.io/en/1.5.0/user/forbidden_characters/index.html
But this prevents to make a subscription with a query string:
$ (curl broker.waziup.io/v1/subscribeContext -s -S --header 'Content-Type: application/json' \
--header 'Accept: application/json' --header 'Fiware-Service:waziup' --header 'Fiware-ServicePath:/TEST' -d #- | python -mjson.tool) <<EOF
{
"entities": [
{
"type": "SensingDevice",
"isPattern": "false",
"id": "Sensor1"
}
],
"attributes": [
"temperature"
],
"reference": "http://localhost/v1/sms/send?contact=0039&msg=Sensor1",
"duration": "P1M",
"notifyConditions": [
{
"type": "ONCHANGE",
"condValues": [
"temperature"
]
}
],
"throttling": "PT1S"
}
EOF
Results in:
{
"subscribeError": {
"errorCode": {
"code": "400",
"details": "Illegal value for JSON field",
"reasonPhrase": "Bad Request"
}
}
}
The query string is used to pass parameters to the callback server (I don't see other ways to do it).
Any way around this?
There is a way of setting query parameters in the notification URL, based in custom notifications in NGSIv2. Have a look to "Custom Notifications" section in the NGSIv2 specification.
The subscription you are doing would be something like this:
POST /v2/subscriptions
...
{
"subject": {
"entities": [
{
"id": "Sensor1",
"type": "SensingDevice"
}
],
"condition": {
"attrs": [ "temperature" ]
}
},
"notification": {
"httpCustom": {
"url": "http://localhost/v1/sms/send",
"qs": {
"contact": "0039",
"msg": "Sensor1"
}
},
"attrs": [ "temperature"]
},
"expires": "2016-05-07T18:30:00.00Z",
"throttling": 1
}
Note that you could even generalize the subscriptions for all your sensors using templates, in the following way:
POST /v2/subscriptions
...
{
"subject": {
"entities": [
{
"idPattern": "Sensor.*",
"type": "SensingDevice"
}
],
"condition": {
"attrs": [ "temperature" ]
}
},
"notification": {
"httpCustom": {
"url": "http://localhost/v1/sms/send",
"qs": {
"contact": "0039",
"msg": "${id}"
}
},
"attrs": [ "temperature"]
},
"expires": "2016-05-07T18:30:00.00Z",
"throttling": 1
}
I am trying to create a new vertex in the Bluemix Graph DB service. The schema of my DB is as follows.
{"propertyKeys":[{"name":"name","dataType":"String","cardinality":"SINGLE"},{"name":"languages","dataType":"String","cardinality":"SET"},{"name":"picture","dataType":"String","cardinality":"SINGLE"},{"name":"preferred_language","dataType":"String","cardinality":"SINGLE"},{"name":"bytes","dataType":"Integer","cardinality":"SINGLE"},{"name":"github_id","dataType":"String","cardinality":"SINGLE"},{"name":"twitter_id","dataType":"String","cardinality":"SINGLE"},{"name":"language_percentage","dataType":"Float","cardinality":"SINGLE"}],"vertexLabels":[{"name":"person"},{"name":"language"}],"edgeLabels":[{"name":"codes_in","multiplicity":"MULTI"},{"name":"used_by","multiplicity":"MULTI"}],"vertexIndexes":[{"name":"vByName","propertyKeys":["name"],"composite":true,"unique":false},{"name":"vByPreferredLang","propertyKeys":["preferred_language"],"composite":true,"unique":false},{"name":"vByLanguages","propertyKeys":["languages"],"composite":false,"unique":false}],"edgeIndexes":[{"name":"eByName","propertyKeys":["name"],"composite":true,"unique":false},{"name":"eByLanguagePercentage","propertyKeys":["language_percentage"],"composite":true,"unique":false}]}
I am trying to create the vertex with the following POST body
{"name":"Bob","languages":["Node","Python"],"picture":"https://en.gravatar.com/userimage/12148147/46ccae88e5aae747d53e0b1863f72a4e.jpg?size=200","preferred_language":"Node","github_id":"Bob","twitter_id":"Bob"}
However this results in the following error
{"code":"BadRequestError","message":"Property 'languages' with meta properties need to have a 'val'"}
The languages property has a cardinality of SET, what is the right way to create a property for a SET dataType? I would have assumed it was a JSON array.
Ryan, SET isn't a data type. You could also make languages a string with delimited values.
The only types that are supported in the Beta release are : String,Integer,Boolean,Float
The issue is that you're attempting to create a single vertex property with a data type of List<String>, which is not supported in IBM Graph (only JSON-primitive types are supported). To take advantage of a property with a SET data type you'll need to create multiple vertex properties.
It turns out that the distinction between cardinalities and data types in TinkerPop can be a bit of a confusing. Here's an example that should clarify things:
$ curl https://ibmgraph/11/g/schema -XPOST -Hcontent-type:application/json -d '{"propertyKeys":[{"name":"languages","dataType":"String","cardinality":"SET"}]}' | jq .
{
"requestId": "9e0ea947-f9a1-407b-ab1a-cd9b7fd5d561",
"status": {
"message": "",
"code": 200,
"attributes": {}
},
"result": {
"data": [
{
"propertyKeys": [
{
"name": "languages",
"dataType": "String",
"cardinality": "SET"
}
],
"vertexLabels": [],
"edgeLabels": [],
"vertexIndexes": [],
"edgeIndexes": []
}
],
"meta": {}
}
}
$ curl https://ibmgraph/11/g/vertices -XPOST | jq .
{
"requestId": "2ce85907-2aca-4630-876f-31775e74e1de",
"status": {
"message": "",
"code": 200,
"attributes": {}
},
"result": {
"data": [
{
"id": 4112,
"label": "vertex",
"type": "vertex",
"properties": {}
}
],
"meta": {}
}
}
$ curl https://ibmgraph/11/g/vertices/4112 -XPOST -Hcontent-type:application/json -d '{"languages":"Node"}' | jq .
{
"requestId": "52ad6d49-46c9-41aa-9928-5a567099d773",
"status": {
"message": "",
"code": 200,
"attributes": {}
},
"result": {
"data": [
{
"id": 4112,
"label": "vertex",
"type": "vertex",
"properties": {
"languages": [
{
"id": "si-368-sl",
"value": "Node"
}
]
}
}
],
"meta": {}
}
}
$ curl https://ibmgraph/11/g/vertices/4112 -XPOST -Hcontent-type:application/json -d '{"languages":"Python"}' | jq .
{
"requestId": "19886949-6328-4e19-8cac-8fdab37ef2a5",
"status": {
"message": "",
"code": 200,
"attributes": {}
},
"result": {
"data": [
{
"id": 4112,
"label": "vertex",
"type": "vertex",
"properties": {
"languages": [
{
"id": "si-368-sl",
"value": "Node"
},
{
"id": "16q-368-sl",
"value": "Python"
}
]
}
}
],
"meta": {}
}
}