Invalid JSON file for DeploymentConfig for Kubernetes - kubernetes

I have this JSON file which was converted from YAML:
{
"kind": "DeploymentConfig",
"apiVersion": "v1",
"metadata": {
"name": "cdt-cae-deployment"
},
"spec": {
"template": {
"metadata": {
"labels": {
"name": "cdt-cae"
},
"annotations": {
"app_version": "latest"
}
},
"spec": {
"containers": [
{
"name": "cdt-cae",
"image": "containers.nabisco.com/cdt-org/cdt-dev:__IMAGETAG__",
"ports": [
{
"containerPort": 8080,
"protocol": "TCP"
}
],
"env": [
{
"name": "APP_NAME",
"value": "cdt-cae"
},
{
"name": "CISCO_LC",
"value": "dev"
},
{
"name": "OPENSHIFT_MONGODB_DB_USERNAME",
"value": "cdtdev"
},
{
"name": "OPENSHIFT_MONGODB_DB_PORT",
"value": "27058"
},
{
"name": "OPENSHIFT_MONGODB_DB_HOST",
"value": "secret stuff here"
},
{
"name": "OPENSHIFT_MONGODB_DB_PASSWORD",
"valueFrom": {
"secretKeyRef": {
"name": "refapp-secret",
"key": "mongodb-password"
}
}
},
{
"name": "NOTIFICATIONS_CLIENT_SECRET",
"valueFrom": {
"secretKeyRef": {
"name": "refapp-secret",
"key": "notifications-client-secret"
}
}
}
],
"volumeMounts": [
{
"name": "podinfo",
"mountPath": "/etc/metadata",
"readOnly": false
}
],
"imagePullPolicy": "IfNotPresent",
"securityContext": {
"capabilities": {},
"privileged": false
}
}
],
"volumes": [
{
"name": "podinfo",
"downwardAPI": {
"items": [
{
"path": "labels",
"fieldRef": {
"fieldPath": "metadata.labels"
}
},
{
"path": "annotations",
"fieldRef": {
"fieldPath": "metadata.annotations"
}
}
]
}
}
],
"restartPolicy": "Always",
"dnsPolicy": "ClusterFirst"
}
},
"replicas": 3,
"selector": {
"name": "cdt-cae"
},
"triggers": [
{
"type": "ConfigChange"
}
],
"strategy": {
"type": "Rolling",
"rollingParams": {
"updatePeriodSeconds": 1,
"intervalSeconds": 1,
"timeoutSeconds": 120
}
}
}
}
unfortunately this is invalid JSON - I get this message:
Does anyone know what's wrong with the config? It looks like it's actually valid JSON, only that perhaps the schema is wrong..
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details
Don't mind this just adding more details

The
<value> expected, unexpected end of file
means that the thing processing the JSON thinks that the terminating } is missing.
The JSON quoted in the question is fine, but it's been changed from the original because of the secret fields. Maybe the secret has an unescaped quote?

Related

Azure Front Door - multiple frontends IDs as a parameter (array) to routing rule?

Commonly, when there are multiple frontends IDs assigned to routing rule, they are described like this:
...
"routingRules": [
{
"name": "routingRule1",
"properties": {
"frontendEndpoints": [
{
"id": "[resourceId('Microsoft.Network/frontDoors/frontendEndpoints', parameters('frontDoorName'), 'frontendEndpoint')]"
},
{
"id": "[resourceId('Microsoft.Network/frontDoors/frontendEndpoints', parameters('frontDoorName'), 'frontendEndpoint2')]"
}
],
...
In my case I'd like to do this as a parameter type "array", but I receive error. Please see below my template, parameter file, and error message
this is front door ARM parameters deployment file
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"frontdoor_name": {
"value": "frontdoortest"
},
"frontends_all": {
"value": [
{
"name": "fe1",
"properties": {
"hostName": "frontdoortest.azurefd.net",
"sessionAffinityEnabledState": "Disabled",
"sessionAffinityTtlSeconds": 0,
"resourceState": "Enabled"
}
},
{
"name": "fe1customdns1",
"properties": {
"hostName": "dev.custom1.com",
"sessionAffinityEnabledState": "Enabled",
"sessionAffinityTtlSeconds": 0,
"resourceState": "Enabled"
}
},
{
"name": "fe1customdns2",
"properties": {
"hostName": "dev.custom2.com",
"sessionAffinityEnabledState": "Enabled",
"sessionAffinityTtlSeconds": 0,
"resourceState": "Enabled"
}
}
]
},
"frontends_one": {
"value": [
{
"id": "[concat(resourceId('Microsoft.Network/frontdoors', parameters('frontdoor_name')), '/FrontendEndpoints/fe1customdns1')]",
},
{
"id": "[concat(resourceId('Microsoft.Network/frontdoors', parameters('frontdoor_name')), '/FrontendEndpoints/fe1customdns2')]",
}
]
}
}
this is part of front door ARM resource deployment template
...
"parameters": {
"frontends_all": {
"type": "array"
},
"frontends_one": {
"type": "array"
},
"frontdoor_name": {
"type": "string"
}
}
...
{
"type": "Microsoft.Network/frontDoors",
"apiVersion": "2020-01-01",
"name": "[parameters('frontdoor_name')]",
"location": "Global",
"properties": {
"resourceState": "Enabled",
"frontendEndpoints": "[parameters('frontends_all')]", #### NO ERROR
"routingRules": [
{
"name": "routingRule1",
"properties": {
"frontendEndpoints": "[parameters('frontends_one')]", #### ERROR !!!
"acceptedProtocols": [
"Https"
],
"patternsToMatch": [
"/*",
],
"enabledState": "Enabled",
"resourceState": "Enabled",
"routeConfiguration": {
"#odata.type": "#Microsoft.Azure.FrontDoor.Models.FrontdoorForwardingConfiguration",
"forwardingProtocol": "MatchRequest",
"backendPool": {
"id": "[concat(resourceId('Microsoft.Network/frontdoors', parameters('frontdoor_name')), '/backendpools/test')]"
}
}
}
}
...
[error]BadRequest: Missing Hostname for FrontendEndpoint [resourceId('Microsoft.Network/frontDoors/frontendEndpoints', parameters('frontdoor_name'), 'fe1customdns2')] in RoutingRule routingRule1.,Missing Hostname for FrontendEndpoint [resourceId('Microsoft.Network/frontDoors/frontendEndpoints', parameters('frontdoor_name'), 'fe1customdns')] in RoutingRule routingRule1;
A resource reference was invalid: "[resourceId('Microsoft.Network/frontDoors/frontendEndpoints', parameters('frontdoor_name'), 'fe1customdns2')]";
A resource reference was invalid: "[resourceId('Microsoft.Network/frontDoors/frontendEndpoints', parameters('frontdoor_name'), 'fe1customdns1')]"

Error: Error response from daemon: invalid volume specification - Windows 8.1 Docker Toolbox

Error: Error response from daemon: invalid volume specification: 'C:/Users/Anthony/magento2-devbox:/C:/Users/Anthony/magento2-devbox'
I have had a google around on this but I cant see how this path has been assembled. Most paths exclude the : and i am also not sure why it has assembled this :/C:/ or whether this is just produced for the error message.
This is the YAML for the replica set;
{
"kind": "ReplicaSet",
"apiVersion": "extensions/v1beta1",
"metadata": {
"name": "magento2-monolith-54cdd5b4b7",
"namespace": "default",
"selfLink": "/apis/extensions/v1beta1/namespaces/default/replicasets/magento2-monolith-54cdd5b4b7",
"uid": "e819bfbd-8820-11e9-a613-080027316036",
"resourceVersion": "22855",
"generation": 1,
"creationTimestamp": "2019-06-06T06:04:12Z",
"labels": {
"app.kubernetes.io/instance": "magento2",
"app.kubernetes.io/name": "monolith",
"pod-template-hash": "54cdd5b4b7"
},
"annotations": {
"deployment.kubernetes.io/desired-replicas": "1",
"deployment.kubernetes.io/max-replicas": "1",
"deployment.kubernetes.io/revision": "1"
},
"ownerReferences": [
{
"apiVersion": "apps/v1",
"kind": "Deployment",
"name": "magento2-monolith",
"uid": "9ec9d23e-8691-11e9-a3dd-080027316036",
"controller": true,
"blockOwnerDeletion": true
}
]
},
"spec": {
"replicas": 1,
"selector": {
"matchLabels": {
"app.kubernetes.io/instance": "magento2",
"app.kubernetes.io/name": "monolith",
"pod-template-hash": "54cdd5b4b7"
}
},
"template": {
"metadata": {
"creationTimestamp": null,
"labels": {
"app.kubernetes.io/instance": "magento2",
"app.kubernetes.io/name": "monolith",
"pod-template-hash": "54cdd5b4b7"
}
},
"spec": {
"volumes": [
{
"name": "nginx-config-volume",
"configMap": {
"name": "magento2-monolith-nginx-config",
"defaultMode": 420
}
},
{
"name": "varnish-config-volume",
"configMap": {
"name": "magento2-monolith-varnish-config",
"defaultMode": 420
}
},
{
"name": "code",
"hostPath": {
"path": "C:/Users/Anthony/magento2-devbox",
"type": ""
}
}
],
"containers": [
{
"name": "monolith",
"image": "magento2-monolith:dev",
"ports": [
{
"containerPort": 8050,
"protocol": "TCP"
}
],
"env": [
{
"name": "DEVBOX_ROOT",
"value": "C:/Users/Anthony/magento2-devbox"
},
{
"name": "COMPOSER_HOME",
"value": "C:/Users/Anthony/magento2-devbox/.composer"
},
{
"name": "MAGENTO_ROOT",
"value": "C:/Users/Anthony/magento2-devbox/magento"
},
{
"name": "MAGENTO_ROOT_HOST",
"value": "C:/Users/Anthony/magento2-devbox/magento"
},
{
"name": "DEVBOX_ROOT_HOST",
"value": "C:/Users/Anthony/magento2-devbox"
},
{
"name": "IS_WINDOWS_HOST",
"value": "0"
}
],
"resources": {},
"volumeMounts": [
{
"name": "code",
"mountPath": "C:/Users/Anthony/magento2-devbox"
}
],
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "Never",
"securityContext": {
"privileged": true,
"procMount": "Default"
}
},
{
"name": "monolith-xdebug",
"image": "magento2-monolith:dev-xdebug",
"ports": [
{
"containerPort": 8002,
"protocol": "TCP"
}
],
"env": [
{
"name": "DEVBOX_ROOT",
"value": "C:/Users/Anthony/magento2-devbox"
},
{
"name": "COMPOSER_HOME",
"value": "C:/Users/Anthony/magento2-devbox/.composer"
},
{
"name": "MAGENTO_ROOT",
"value": "C:/Users/Anthony/magento2-devbox/magento"
},
{
"name": "MAGENTO_ROOT_HOST",
"value": "C:/Users/Anthony/magento2-devbox/magento"
},
{
"name": "DEVBOX_ROOT_HOST",
"value": "C:/Users/Anthony/magento2-devbox"
},
{
"name": "IS_WINDOWS_HOST",
"value": "0"
}
],
"resources": {},
"volumeMounts": [
{
"name": "code",
"mountPath": "C:/Users/Anthony/magento2-devbox"
}
],
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "Never",
"securityContext": {
"privileged": true,
"procMount": "Default"
}
},
{
"name": "nginx",
"image": "nginx:1.9",
"resources": {},
"volumeMounts": [
{
"name": "code",
"mountPath": "C:/Users/Anthony/magento2-devbox"
},
{
"name": "nginx-config-volume",
"mountPath": "/etc/nginx/nginx.conf",
"subPath": "nginx.conf"
}
],
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "IfNotPresent",
"securityContext": {
"privileged": true,
"procMount": "Default"
}
},
{
"name": "varnish",
"image": "million12/varnish",
"env": [
{
"name": "VCL_CONFIG",
"value": "/etc/varnish/magento.vcl"
},
{
"name": "VARNISHD_PARAMS",
"value": "-a 0.0.0.0:6081"
}
],
"resources": {},
"volumeMounts": [
{
"name": "varnish-config-volume",
"mountPath": "/etc/varnish/magento.vcl",
"subPath": "varnish.vcl"
}
],
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "Always"
}
],
"restartPolicy": "Always",
"terminationGracePeriodSeconds": 30,
"dnsPolicy": "ClusterFirst",
"securityContext": {},
"schedulerName": "default-scheduler"
}
}
},
"status": {
"replicas": 1,
"fullyLabeledReplicas": 1,
"observedGeneration": 1
}
}
I am new to Docker/Kubernetes after coming over from Vagrant so i do not know where to start. The information i have is from the web browser dashboard
Path is probably not converted to unix style. Since docker 1.9.0, Windows paths are not automatically converted (eg. C:\Users to /c/Users).
So your path should be like :
{
"name": "DEVBOX_ROOT",
"value": "/c/Users/Anthony/magento2-devbox"
}

Running two containers in a single pod via kubectl run does not terminate the pod with two containers

Trying to run a two docker containers in one kubernetes pod. The problem we face is that we have 2 images/containers in a Kubernetes pod whereas we used to have 1 image for this pod previously where it was working perfectly and shutting down the pod gracefully. Now the problem is that with 2 images or containers within the same pod both images are not shutting down at the same time gracefully.
We are doing integration testing of our container against the database which is MongoDB, our container terminates but MongoDB fails to terminate here and continues running
while getopts n:c:l:i:d:t: option
do
case "${option}"
in
n) Company_NAMESPACE=$OPTARG;;
c) Company_CONFIG=$OPTARG;;
l) Company_LICENSE=$OPTARG;;
i) Company_IMAGE=$OPTARG;;
d) Company_CONTAINER_NAME=$OPTARG;;
t) Company_TOKEN=$OPTARG;;
esac
done
kubectl run $Company_CONTAINER_NAME -n $Company_NAMESPACE --restart=Never --overrides='
{
"apiVersion": "v1",
"spec": {
"imagePullSecrets": [
{
"name": "Company-regsecret"
}
],
"initContainers": [
{
"name": "copy-configs",
"image": "busybox",
"command": ["sh", "-c", "cp /tmp/Company-config-volume/server/* /tmp/ng-rt/config/server/ 2>/dev/null || true; cp /tmp/Company-license-volume/licenses/* /tmp/ng-rt/config/licenses 2>/dev/null || true"],
"volumeMounts": [
{
"name": "Company-config-volume",
"mountPath": "mount_path"
},
{
"name": "'$Company_CONFIG'",
"mountPath": "mount_path"
},
{
"name": "Company-license-volume",
"mountPath": "mount_path"
},
{
"name": "'$Company_LICENSE'",
"mountPath": "mount_path"
}
]
}
],
"containers": [
{
"name": "mongodb-test",
"image": "mongo:3.6.8",
"command": [
"numactl",
"--interleave=all",
"mongod",
"--wiredTigerCacheSizeGB",
"0.1",
"--replSet",
"MainRepSet",
"--bind_ip_all"
],
"ports": [{
"containerPort": 27017
}],
"readinessProbe": {
"exec": {
"command": ["mongo", "--eval", "rs.initiate()"]
}
},
"terminationGracePeriodSeconds": 10
},
{
"env": [
{
"name": "AWS_ACCESS_KEY_ID",
"valueFrom": {
"secretKeyRef": {
"key": "AWS_ACCESS_KEY_ID",
"name": "aws-secrets"
}
}
},
{
"name": "AWS_SECRET_ACCESS_KEY",
"valueFrom": {
"secretKeyRef": {
"key": "AWS_SECRET_ACCESS_KEY",
"name": "aws-secrets"
}
}
},
{
"name": "AWS_REGION",
"valueFrom": {
"secretKeyRef": {
"key": "AWS_REGION",
"name": "aws-secrets"
}
}
},
{
"name": "BUILD_ID",
"valueFrom": {
"configMapKeyRef": {
"key": "BUILD_ID",
"name": "config"
}
}
}
],
"command": [
"sh",
"-c",
"mkdir -p mount_path 2\u003e/dev/null && npm test --skipConnectivityTestRethinkDB"
],
"name": "'$Company_CONTAINER_NAME'",
"image": "'$Company_IMAGE'",
"volumeMounts": [
{
"mountPath": "mount_path",
"name": "'$Company_CONFIG'"
},
{
"mountPath": "mount_path",
"name": "'$Company_LICENSE'"
}
]
}
],
"volumes": [
{
"name": "Company-config-volume",
"configMap": {
"name": "'$Company_CONFIG'"
}
},
{
"name": "'$Company_CONFIG'",
"emptyDir": {}
},
{
"name": "Company-license-volume",
"configMap": {
"name": "'$Company_LICENSE'"
}
},
{
"name": "'$Company_LICENSE'",
"emptyDir": {}
}
]
}
}
' --image=$Company_IMAGE -ti --rm --token=$Company_TOKEN
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "test-dev-58663",
"namespace": "d-int-company-dev-v2",
"labels": {
"run": "test-dev-58663"
},
"annotations": {
"kubernetes.io/psp": "nfs-provisioner"
}
},
"spec": {
"volumes": [
{
"name": "company-config-volume",
"configMap": {
"name": "test-core",
"defaultMode": 420
}
},
{
"name": "test-core",
"emptyDir": {}
},
{
"name": "company-license-volume",
"configMap": {
"name": "company-license",
"defaultMode": 420
}
},
{
"name": "company-license",
"emptyDir": {}
},
{
"name": "default-token-wqp5x",
"secret": {
"secretName": "default-token-wqp5x",
"defaultMode": 420
}
}
],
"initContainers": [
{
"name": "copy-configs",
"image": "busybox",
"command": [
"sh",
"-c",
"cp mount_path* mount_path 2>/dev/null || true; cp mount_path* mount_path 2>mount_path|| true"
],
"resources": {},
"volumeMounts": [
{
"name": "company-config-volume",
"mountPath": "mount_path"
},
{
"name": "test-core",
"mountPath": "mount_path"
},
{
"name": "company-license-volume",
"mountPath": "mount_path"
},
{
"name": "company-license",
"mountPath": "mount_path"
},
{
"name": "default-token-wqp5x",
"readOnly": true,
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
}
],
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "Always"
}
],
"containers": [
{
"name": "mongodb-test",
"image": "mongo:3.6.8",
"ports": [
{
"containerPort": 27017,
"protocol": "TCP"
}
],
"resources": {},
"volumeMounts": [
{
"name": "default-token-wqp5x",
"readOnly": true,
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
}
],
"readinessProbe": {
"exec": {
"command": [
"mongo",
"--eval",
"rs.initiate()"
]
},
"timeoutSeconds": 1,
"periodSeconds": 10,
"successThreshold": 1,
"failureThreshold": 3
},
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "IfNotPresent"
},
{
"name": "test-dev-58663",
"image": "image_path",
"command": [
"sh",
"-c",
"mkdir -p mount_path 2>/dev/null && npm test --skipConnectivityTestRethinkDB"
],
"env": [
{
"name": "AWS_ACCESS_KEY_ID",
"valueFrom": {
"secretKeyRef": {
"name": "aws-secrets",
"key": "AWS_ACCESS_KEY_ID"
}
}
},
{
"name": "AWS_SECRET_ACCESS_KEY",
"valueFrom": {
"secretKeyRef": {
"name": "aws-secrets",
"key": "AWS_SECRET_ACCESS_KEY"
}
}
},
{
"name": "AWS_REGION",
"valueFrom": {
"secretKeyRef": {
"name": "aws-secrets",
"key": "AWS_REGION"
}
}
},
{
"name": "BUILD_ID",
"valueFrom": {
"configMapKeyRef": {
"name": "tbsp-config",
"key": "BUILD_ID"
}
}
}
],
"resources": {},
"volumeMounts": [
{
"name": "test-core",
"mountPath": "mount_path"
},
{
"name": "company-license",
"mountPath": "mount_path"
},
{
"name": "default-token-wqp5x",
"readOnly": true,
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
}
],
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "IfNotPresent"
}
],
"restartPolicy": "Never",
"terminationGracePeriodSeconds": 10,
"dnsPolicy": "ClusterFirst",
"serviceAccountName": "default",
"serviceAccount": "default",
"nodeName": "pc212",
"securityContext": {},
"imagePullSecrets": [
{
"name": "company-regsecret"
}
],
"schedulerName": "default-scheduler",
"tolerations": [
{
"key": "node.kubernetes.io/not-ready",
"operator": "Exists",
"effect": "NoExecute",
"tolerationSeconds": 300
},
{
"key": "node.kubernetes.io/unreachable",
"operator": "Exists",
"effect": "NoExecute",
"tolerationSeconds": 300
}
]
},
"status": {
"phase": "Pending",
"conditions": [
{
"type": "Initialized",
"status": "True",
"lastProbeTime": null,
"lastTransitionTime": "2019-05-10T12:50:39Z"
},
{
"type": "Ready",
"status": "False",
"lastProbeTime": null,
"lastTransitionTime": "2019-05-10T12:49:54Z",
"reason": "ContainersNotReady",
"message": "containers with unready status: [mongodb-test test-dev-58663]"
},
{
"type": "PodScheduled",
"status": "True",
"lastProbeTime": null,
"lastTransitionTime": "2019-05-10T12:49:54Z"
}
],
"hostIP": "10.10.2.12",
"podIP": "172.16.4.22",
"startTime": "2019-05-10T12:49:54Z",
"initContainerStatuses": [
{
"name": "copy-configs",
"state": {
"terminated": {
"exitCode": 0,
"reason": "Completed",
"startedAt": "2019-05-10T12:50:39Z",
"finishedAt": "2019-05-10T12:50:39Z",
"containerID": "docker://1bcd12f5848e32e82f7dfde8e245223345e87f70061b789cbbabc0f798436b59"
}
},
"lastState": {},
"ready": true,
"restartCount": 0,
"image": "busybox:latest",
"imageID": "docker-pullable://busybox#sha256:0b184b74edc63924be0d7f67d16f5afbcdbe61caa1aca9312ed3b5c57792f6c1",
"containerID": "docker://1bcd12f5848e32e82f7dfde8e245223345e87f70061b789cbbabc0f798436b59"
}
],
"containerStatuses": [
{
"name": "mongodb-test",
"state": {
"waiting": {
"reason": "PodInitializing"
}
},
"lastState": {},
"ready": false,
"restartCount": 0,
"image": "mongo:3.6.8",
"imageID": ""
},
{
"name": "test-dev-58663",
"state": {
"waiting": {
"reason": "PodInitializing"
}
},
"lastState": {},
"ready": false,
"restartCount": 0,
"image": "image_path",
"imageID": ""
}
],
"qosClass": "BestEffort"
}
}
Both the containers and the hosting pod should terminate gracefully.
You have pod consisting of two containers, one of them is supposed to run indefinitely, another one runs to completion. This is bad practice. You should split your pod into two separate things: Pod with Mongo and Job with your integration script. You have to write logic that watches Job to finish and then terminates both Pod and Job. You can do it like this:
kubectl apply -f integration-test.yaml
kubectl wait --for=condition=Complete --timeout=5m job/test
kubectl delete -f integration-test.yaml

How do I create Virtual Machine with WinRM from an ARM Template?

I'm running into an issue when I attempt to run the 'Azure Resource Group Deploy' release task to create/update a resource group and the resources within it via an ARM Template. In particular, I need to have the Virtual Machine created by the ARM template accessible via WinRM; This needs to be done so that I can copy files (specifically a ZIP file containing the results of a build) to the VM in a later step.
Currently, I have the 'Template' portion of this task set up as follows: https://i.imgur.com/mvZDIMK.jpg (I can't post images since I don't have reputation here yet...)
Unless I've misunderstood (which is definitely possible), the "Configure with WinRM" option should allow the release step to create a WinRM Listener on any Virtual Machines created by this step.
I currently have the following resources in the ARM Template:
{
"type": "Microsoft.Storage/storageAccounts",
"sku": {
"name": "Standard_LRS",
"tier": "Standard"
},
"kind": "Storage",
"name": "[variables('StorageAccountName')]",
"apiVersion": "2018-02-01",
"location": "[parameters('LocationPrimary')]",
"scale": null,
"tags": {},
"properties": {
"networkAcls": {
"bypass": "AzureServices",
"virtualNetworkRules": [],
"ipRules": [],
"defaultAction": "Allow"
},
"supportsHttpsTrafficOnly": false,
"encryption": {
"services": {
"file": {
"enabled": true
},
"blob": {
"enabled": true
}
},
"keySource": "Microsoft.Storage"
}
},
"dependsOn": []
},
{
"name": "[variables('NetworkInterfaceName')]",
"type": "Microsoft.Network/networkInterfaces",
"apiVersion": "2018-04-01",
"location": "[parameters('LocationPrimary')]",
"dependsOn": [
"[concat('Microsoft.Network/networkSecurityGroups/', variables('NetworkSecurityGroupName'))]",
"[concat('Microsoft.Network/virtualNetworks/', variables('VNetName'))]",
"[concat('Microsoft.Network/publicIpAddresses/', variables('PublicIPAddressName'))]"
],
"properties": {
"ipConfigurations": [
{
"name": "ipconfig1",
"properties": {
"subnet": {
"id": "[variables('subnetRef')]"
},
"privateIPAllocationMethod": "Dynamic",
"publicIpAddress": {
"id": "[resourceId(resourceGroup().name, 'Microsoft.Network/publicIpAddresses', variables('PublicIPAddressName'))]"
}
}
}
],
"networkSecurityGroup": {
"id": "[variables('nsgId')]"
}
},
"tags": {}
},
{
"name": "[variables('NetworkSecurityGroupName')]",
"type": "Microsoft.Network/networkSecurityGroups",
"apiVersion": "2018-08-01",
"location": "[parameters('LocationPrimary')]",
"properties": {
"securityRules": [
{
"name": "RDP",
"properties": {
"priority": 300,
"protocol": "TCP",
"access": "Allow",
"direction": "Inbound",
"sourceAddressPrefix": "*",
"sourcePortRange": "*",
"destinationAddressPrefix": "*",
"destinationPortRange": "3389"
}
}
]
},
"tags": {}
},
{
"name": "[variables('VNetName')]",
"type": "Microsoft.Network/virtualNetworks",
"apiVersion": "2018-08-01",
"location": "[parameters('LocationPrimary')]",
"properties": {
"addressSpace": {
"addressPrefixes": [ "10.0.0.0/24" ]
},
"subnets": [
{
"name": "default",
"properties": {
"addressPrefix": "10.0.0.0/24"
}
}
]
},
"tags": {}
},
{
"name": "[variables('PublicIPAddressName')]",
"type": "Microsoft.Network/publicIpAddresses",
"apiVersion": "2018-08-01",
"location": "[parameters('LocationPrimary')]",
"properties": {
"publicIpAllocationMethod": "Dynamic"
},
"sku": {
"name": "Basic"
},
"tags": {}
},
{
"name": "[variables('VMName')]",
"type": "Microsoft.Compute/virtualMachines",
"apiVersion": "2018-06-01",
"location": "[parameters('LocationPrimary')]",
"dependsOn": [
"[concat('Microsoft.Network/networkInterfaces/', variables('NetworkInterfaceName'))]",
"[concat('Microsoft.Storage/storageAccounts/', variables('StorageAccountName'))]"
],
"properties": {
"hardwareProfile": {
"vmSize": "Standard_A7"
},
"storageProfile": {
"osDisk": {
"createOption": "fromImage",
"managedDisk": {
"storageAccountType": "Standard_LRS"
}
},
"imageReference": {
"publisher": "MicrosoftWindowsDesktop",
"offer": "Windows-10",
"sku": "rs4-pro",
"version": "latest"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', variables('NetworkInterfaceName'))]"
}
]
},
"osProfile": {
"computerName": "[variables('VMName')]",
"adminUsername": "[parameters('AdminUsername')]",
"adminPassword": "[parameters('AdminPassword')]",
"windowsConfiguration": {
"enableAutomaticUpdates": true,
"provisionVmAgent": true
}
},
"licenseType": "Windows_Client",
"diagnosticsProfile": {
"bootDiagnostics": {
"enabled": true,
"storageUri": "[concat('https://', variables('StorageAccountName'), '.blob.core.windows.net/')]"
}
}
},
"tags": {}
}
This ARM Template currently works if I do not attempt to configure the VM to have the WinRM Listener.
When I attempt to run the release, I get the following error message:
Error number: -2144108526 0x80338012
The client cannot connect to the destination specified in the request. Verify that the service on the destination is running and is accepting requests. Consult the logs and documentation for the WS-Management service running on the destination, most commonly IIS or WinRM. If the destination is the WinRM service, run the following command on the destination to analyze and configure the WinRM service: "winrm quickconfig".
In all honesty, my problem is likely a lack of understanding, as this is my first time working with VM Setup in any real capacity. Any insight and advice would be greatly appreciated.
you just need to add this to the "windowsConfiguration":
"winRM": {
"listeners": [
{
"protocol": "http"
},
{
"protocol": "https",
"certificateUrl": "<URL for the certificate you got in Step 4>"
}
]
}
you also need to provision certificates
reference: https://learn.microsoft.com/en-us/rest/api/compute/virtualmachines/createorupdate#winrmconfiguration
https://learn.microsoft.com/en-us/azure/virtual-machines/windows/winrm

Service fabric, AD and client certificate security

service fabric cluster communication fail when using my domainname.com certificate. I'm getting this error using the portal and/or using ARM template:
Failed to communicate with the cluster (get cluster health: Client certificate required).
Template (variables were removed):
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"resources": [
{
"apiVersion": "[variables('storageApiVersion')]",
"type": "Microsoft.Storage/storageAccounts",
"name": "[parameters('supportLogStorageAccountName')]",
"location": "[parameters('computeLocation')]",
"dependsOn": [],
"properties": {},
"kind": "Storage",
"sku": {
"name": "[parameters('supportLogStorageAccountType')]"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('storageApiVersion')]",
"type": "Microsoft.Storage/storageAccounts",
"name": "[parameters('applicationDiagnosticsStorageAccountName')]",
"location": "[parameters('computeLocation')]",
"dependsOn": [],
"properties": {},
"kind": "Storage",
"sku": {
"name": "[parameters('applicationDiagnosticsStorageAccountType')]"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('vNetApiVersion')]",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('virtualNetworkName')]",
"location": "[parameters('computeLocation')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('addressPrefix')]"
]
},
"subnets": [
{
"name": "[parameters('subnet0Name')]",
"properties": {
"addressPrefix": "[parameters('subnet0Prefix')]"
}
}
]
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('publicIPApiVersion')]",
"type": "Microsoft.Network/publicIPAddresses",
"name": "[concat(parameters('lbIPName'),'-','0')]",
"location": "[parameters('computeLocation')]",
"properties": {
"dnsSettings": {
"domainNameLabel": "[parameters('dnsName')]"
},
"publicIPAllocationMethod": "Dynamic"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('lbApiVersion')]",
"type": "Microsoft.Network/loadBalancers",
"name": "[concat('LB','-', parameters('clusterName'),'-',parameters('vmNodeType0Name'))]",
"location": "[parameters('computeLocation')]",
"dependsOn": [
"[concat('Microsoft.Network/publicIPAddresses/',concat(parameters('lbIPName'),'-','0'))]"
],
"properties": {
"frontendIPConfigurations": [
{
"name": "LoadBalancerIPConfig",
"properties": {
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(parameters('lbIPName'),'-','0'))]"
}
}
}
],
"backendAddressPools": [
{
"name": "LoadBalancerBEAddressPool",
"properties": {}
}
],
"loadBalancingRules": [
{
"name": "LBRule",
"properties": {
"backendAddressPool": {
"id": "[variables('lbPoolID0')]"
},
"backendPort": "[parameters('nt0fabricTcpGatewayPort')]",
"enableFloatingIP": "false",
"frontendIPConfiguration": {
"id": "[variables('lbIPConfig0')]"
},
"frontendPort": "[parameters('nt0fabricTcpGatewayPort')]",
"idleTimeoutInMinutes": "5",
"probe": {
"id": "[variables('lbProbeID0')]"
},
"protocol": "tcp"
}
},
{
"name": "LBHttpRule",
"properties": {
"backendAddressPool": {
"id": "[variables('lbPoolID0')]"
},
"backendPort": "[parameters('nt0fabricHttpGatewayPort')]",
"enableFloatingIP": "false",
"frontendIPConfiguration": {
"id": "[variables('lbIPConfig0')]"
},
"frontendPort": "[parameters('nt0fabricHttpGatewayPort')]",
"idleTimeoutInMinutes": "5",
"probe": {
"id": "[variables('lbHttpProbeID0')]"
},
"protocol": "tcp"
}
}
],
"probes": [
{
"name": "FabricGatewayProbe",
"properties": {
"intervalInSeconds": 5,
"numberOfProbes": 2,
"port": "[parameters('nt0fabricTcpGatewayPort')]",
"protocol": "tcp"
}
},
{
"name": "FabricHttpGatewayProbe",
"properties": {
"intervalInSeconds": 5,
"numberOfProbes": 2,
"port": "[parameters('nt0fabricHttpGatewayPort')]",
"protocol": "tcp"
}
}
],
"inboundNatPools": [
{
"name": "LoadBalancerBEAddressNatPool",
"properties": {
"backendPort": "3389",
"frontendIPConfiguration": {
"id": "[variables('lbIPConfig0')]"
},
"frontendPortRangeEnd": "4500",
"frontendPortRangeStart": "3389",
"protocol": "tcp"
}
}
]
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('storageApiVersion')]",
"type": "Microsoft.Storage/storageAccounts",
"name": "[variables('uniqueStringArray0')[copyIndex()]]",
"location": "[parameters('computeLocation')]",
"dependsOn": [],
"properties": {},
"copy": {
"name": "storageLoop",
"count": 5
},
"kind": "Storage",
"sku": {
"name": "[parameters('storageAccountType')]"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('vmssApiVersion')]",
"type": "Microsoft.Compute/virtualMachineScaleSets",
"name": "[parameters('vmNodeType0Name')]",
"location": "[parameters('computeLocation')]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', parameters('virtualNetworkName'))]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[0])]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[1])]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[2])]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[3])]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[4])]",
"[concat('Microsoft.Network/loadBalancers/', concat('LB','-', parameters('clusterName'),'-',parameters('vmNodeType0Name')))]",
"[concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName'))]",
"[concat('Microsoft.Storage/storageAccounts/', parameters('applicationDiagnosticsStorageAccountName'))]"
],
"properties": {
"overprovision": "[parameters('overProvision')]",
"upgradePolicy": {
"mode": "Automatic"
},
"virtualMachineProfile": {
"extensionProfile": {
"extensions": [
{
"name": "[concat(parameters('vmNodeType0Name'),'_ServiceFabricNode')]",
"properties": {
"type": "ServiceFabricNode",
"autoUpgradeMinorVersion": false,
"protectedSettings": {
"StorageAccountKey1": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('supportLogStorageAccountName')),'2015-05-01-preview').key1]",
"StorageAccountKey2": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('supportLogStorageAccountName')),'2015-05-01-preview').key2]"
},
"publisher": "Microsoft.Azure.ServiceFabric",
"settings": {
"clusterEndpoint": "[reference(parameters('clusterName')).clusterEndpoint]",
"nodeTypeRef": "[parameters('vmNodeType0Name')]",
"dataPath": "D:\\\\SvcFab",
"durabilityLevel": "Bronze",
"certificate": {
"thumbprint": "[parameters('certificateThumbprint')]",
"x509StoreName": "[parameters('certificateStoreValue')]"
}
},
"typeHandlerVersion": "1.0"
}
},
{
"name": "[concat('VMDiagnosticsVmExt','_vmNodeType0Name')]",
"properties": {
"type": "IaaSDiagnostics",
"autoUpgradeMinorVersion": true,
"protectedSettings": {
"storageAccountName": "[parameters('applicationDiagnosticsStorageAccountName')]",
"storageAccountKey": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('applicationDiagnosticsStorageAccountName')),'2015-05-01-preview').key1]",
"storageAccountEndPoint": "https://core.windows.net/"
},
"publisher": "Microsoft.Azure.Diagnostics",
"settings": {
"WadCfg": {
"DiagnosticMonitorConfiguration": {
"overallQuotaInMB": "50000",
"EtwProviders": {
"EtwEventSourceProviderConfiguration": [
{
"provider": "Microsoft-ServiceFabric-Actors",
"scheduledTransferKeywordFilter": "1",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricReliableActorEventTable"
}
},
{
"provider": "Microsoft-ServiceFabric-Services",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricReliableServiceEventTable"
}
}
],
"EtwManifestProviderConfiguration": [
{
"provider": "cbd93bc2-71e5-4566-b3a7-595d8eeca6e8",
"scheduledTransferLogLevelFilter": "Information",
"scheduledTransferKeywordFilter": "4611686018427387904",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricSystemEventTable"
}
}
]
}
}
},
"StorageAccount": "[parameters('applicationDiagnosticsStorageAccountName')]"
},
"typeHandlerVersion": "1.5"
}
}
]
},
"networkProfile": {
"networkInterfaceConfigurations": [
{
"name": "[concat(parameters('nicName'), '-0')]",
"properties": {
"ipConfigurations": [
{
"name": "[concat(parameters('nicName'),'-',0)]",
"properties": {
"loadBalancerBackendAddressPools": [
{
"id": "[variables('lbPoolID0')]"
}
],
"loadBalancerInboundNatPools": [
{
"id": "[variables('lbNatPoolID0')]"
}
],
"subnet": {
"id": "[variables('subnet0Ref')]"
}
}
}
],
"primary": true
}
}
]
},
"osProfile": {
"adminPassword": "[parameters('adminPassword')]",
"adminUsername": "[parameters('adminUsername')]",
"computernamePrefix": "[parameters('vmNodeType0Name')]",
"secrets": [
{
"sourceVault": {
"id": "[parameters('sourceVaultValue')]"
},
"vaultCertificates": [
{
"certificateStore": "[parameters('certificateStoreValue')]",
"certificateUrl": "[parameters('certificateUrlValue')]"
}
]
}
]
},
"storageProfile": {
"imageReference": {
"publisher": "[parameters('vmImagePublisher')]",
"offer": "[parameters('vmImageOffer')]",
"sku": "[parameters('vmImageSku')]",
"version": "[parameters('vmImageVersion')]"
},
"osDisk": {
"vhdContainers": [
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[0]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]",
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[1]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]",
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[2]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]",
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[3]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]",
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[4]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]"
],
"name": "vmssosdisk",
"caching": "ReadOnly",
"createOption": "FromImage"
}
}
}
},
"sku": {
"name": "[parameters('vmNodeType0Size')]",
"capacity": "[parameters('nt0InstanceCount')]",
"tier": "Standard"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "2016-09-01",
"type": "Microsoft.ServiceFabric/clusters",
"name": "[parameters('clusterName')]",
"location": "[parameters('clusterLocation')]",
"dependsOn": [
"[concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName'))]"
],
"properties": {
"azureActiveDirectory": {
"clientApplication": "55dae335-8436-4900-8f33-abbff16e8036",
"clusterApplication": "d0392358-9fa2-4f9a-aa2a-101859e31c34",
"tenantId": "c8656f45-daf5-42c1-9b29-ac27d3e63bf3"
},
"certificate": {
"thumbprint": "[parameters('certificateThumbprint')]",
"x509StoreName": "[parameters('certificateStoreValue')]"
},
"clientCertificateCommonNames": [],
"clientCertificateThumbprints": [
{
"certificateThumbprint": "C5EFB021F5D8BA8966B43B523B2A6BF8EE8202C5",
"isAdmin": true
},
{
"certificateThumbprint": "C5EFB021F5D8BA8966B43B523B2A6BF8EE8202C5",
"isAdmin": false
}
],
"clusterState": "Default",
"diagnosticsStorageAccountConfig": {
"blobEndpoint": "[reference(concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName')), variables('storageApiVersion')).primaryEndpoints.blob]",
"protectedAccountKeyName": "StorageAccountKey1",
"queueEndpoint": "[reference(concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName')), variables('storageApiVersion')).primaryEndpoints.queue]",
"storageAccountName": "[parameters('supportLogStorageAccountName')]",
"tableEndpoint": "[reference(concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName')), variables('storageApiVersion')).primaryEndpoints.table]"
},
"fabricSettings": [
{
"parameters": [
{
"name": "ClusterProtectionLevel",
"value": "[parameters('clusterProtectionLevel')]"
}
],
"name": "Security"
}
],
"managementEndpoint": "[concat('https://',reference(concat(parameters('lbIPName'),'-','0')).dnsSettings.fqdn,':',parameters('nt0fabricHttpGatewayPort'))]",
"nodeTypes": [
{
"name": "[parameters('vmNodeType0Name')]",
"applicationPorts": {
"endPort": "[parameters('nt0applicationEndPort')]",
"startPort": "[parameters('nt0applicationStartPort')]"
},
"clientConnectionEndpointPort": "[parameters('nt0fabricTcpGatewayPort')]",
"durabilityLevel": "Bronze",
"ephemeralPorts": {
"endPort": "[parameters('nt0ephemeralEndPort')]",
"startPort": "[parameters('nt0ephemeralStartPort')]"
},
"httpGatewayEndpointPort": "[parameters('nt0fabricHttpGatewayPort')]",
"isPrimary": true,
"vmInstanceCount": "[parameters('nt0InstanceCount')]"
}
],
"provisioningState": "Default",
"security": {
"metadata": "The Credential type X509 indicates this is cluster is secured using X509 Certificates.",
"ClusterCredentialType": "X509",
"ServerCredentialType": "X509",
"CertificateInformation": {
"ClusterCertificate": {
"Thumbprint": "[parameters('certificateThumbprint')]",
"X509StoreName": "My"
},
"ServerCertificate": {
"Thumbprint": "[parameters('certificateThumbprint')]",
"X509StoreName": "My"
},
"ClientCertificateThumbprints": [
{
"CertificateThumbprint": "[parameters('certificateThumbprint')]",
"IsAdmin": false
},
{
"CertificateThumbprint": "[parameters('certificateThumbprint')]",
"IsAdmin": true
}
]
}
},
"reliabilityLevel": "Bronze",
"upgradeMode": "Automatic",
"vmImage": "Windows"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
}
],
"outputs": {
"clusterProperties": {
"value": "[reference(parameters('clusterName'))]",
"type": "object"
}
}
}
hint?
It might be as simple as the client certificate is not imported (installed) in the machine from where you are trying to access the explorer. When you open the Service Fabric explorer, it might pop up to select one of the certificates installed in your machine. You have to choose the client certificate at this point.
I compared the ARM template with my (working) template and it looks OK. No obvious issues.
Your ARM template shows you are missing the client certificate within the VMSS OS Profile, which means it will not be installed to each of the nodes during deployment then it is missing when Service Fabric attempts to use it.
Here is an example from one of my ARM templates
"osProfile": {
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]",
"computernamePrefix": "[parameters('vmNodeType0Name')]",
"secrets": [
{
"sourceVault": {
"id": "[parameters('sourceVault')]"
},
"vaultCertificates": [
{
"certificateStore": "My",
"certificateUrl": "[parameters('clusterCertificateUrl')]"
},
{
"certificateStore": "My",
"certificateUrl": "[parameters('adminCertificateUrl')]"
}
]
}
]
},
EDIT:
Where you have two client certificates, I would expect your OS Profile to have three vault certificates for installation on deployment