Azure: Build templates: how to extract defaultStorageAccount - powershell

I am trying to create a custom image in my Azure DevTestLab, from a vhd image stored in the default storage account.
I started modifying the file at the GitHub repo for ARM templates for creating a custom image from a vhd.
I have already created the DevTest lab, and I can run the following Powershell command to get its data:
PS C:\git> $rid = Get-AzureRmResource -ResourceGroupName jgTl9-rg -ResourceType Microsoft.DevTestLab/labs -ResourceName jgTl9 -ApiVersion 2016-05-15
This yields a bunch of information. If I want to extract the default storage account name, I do the following:
PS C:\git> $rid.properties.defaultStorageAccount
So I know it's possible to get it. Now I have modified the variables section of the Azure build template (a JSON file) to look as follows:
"variables": {
"resourceName": "[concat(variables('existingLabName'), '/', variables('imageName'))]",
"resourceType": "Microsoft.DevTestLab/labs/customimages",
"existingLabName": "[replace(resourceGroup().name,'-rg','')]",
"storageAccountID": "/subscriptions/xxxxxxxxxxxx/resourceGroups/jgTl9-rg/providers/Microsoft.Storage/storageAccounts/djgtl91795",
"saidFrags": "[skip(split(variables('storageAccountID'), '/' ), add(length(split(variables('storageAccountID'), '/' )),-1) ) ]",
"storageAccountSuffix": "[take(variables('saidFrags'),1)[0]]",
"existingVhdUri": "[concat('https://',variables('storageAccountSuffix'),'.blob.core.windows.net/uploads/',parameters('imageFileName'))]",
"imageName": "JaysImage"
},
This code actually works when I call the New-AzureRmResourceGroupDeployment command with the azuredeploy.json and azuredeploy.parameters.json.
However, instead of hardcoding the storageAccountID variable, I'd like to extract the defaultStorageAccount from the environment. I tried something like this:
"storageAccountID": "[string( resourceId(resourceGroup().name,'MicrosoftDevTestLab/labs','jgTl9').properties.defaultStorageAccount ))",
I tried a few variations on this but nothing worked. What is the syntax I need to use to extract the same information as I have hardcoded for the storageAccountID?
(I could not find anything in Microsoft's description of template functions).

Couple of issues with your following expression
"storageAccountID": "[string( resourceId(resourceGroup().name,'MicrosoftDevTestLab/labs','jgTl9').properties.defaultStorageAccount ))",
There is syntax issue ie. an extra ) at the end.
The resource provider MicrosoftDevTestlab/labs should be Microsoft.DevTestlab/labs
When resourceId is used it will not give you the entire object. So when you do properties.defaultStorageAccount it will not work. Because it gives you just the string of your dev test lab and nothing else and there is no property on a string of defaultStorageAccount.
Now what you need is reference function like below
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"vDefaultStorageAccount": "[string(resourceId('yourResourceGroupName','Microsoft.DevTestLab/labs','yourTestLabName'))]"
},
"resources": [],
"outputs": {
"defaultStorageAccount":{
"value": "[reference(variables('vDefaultStorageAccount'),'2016-05-15').defaultStorageAccount]",
"type":"string"
}
}
}
In the variable, vDefaultStorageAccount we get the ID of the DevTestLab and then we use reference function to get the object. Then you can do . to get properties. You don't need to do properties.defaultStorageAccount for example.

Related

Save variables with Azure CLI + REST

I'm currently using following 'Script' to obtain Base64EncodedPackage in Powershell 7 (can switch to command prompt also if needed)
$subscriptionId = "0000-0000-0000-0000"
$vm = "MyVmName"
$vmrg = "MyRgName"
$resourceId = "/subscriptions/$subscriptionId/resourceGroups/$vmrg/providers/Microsoft.Compute/virtualMachines/$vm"
#Log in to CLI
az login
#Log in to wanted subscription
az account set --subscription $subscriptionId
#Show Base64EncodedPackages
$package = az rest --method get --uri https://management.azure.com/subscriptions/$subscriptionId/providers/Microsoft.Security/mdeOnboardings?api-version=2021-10-01-preview
$package then will output following
{
"value": [
{
"id": "/subscriptions/0000-0000-0000-0000/providers/Microsoft.Security/mdeOnboardings/default",
"location": "westeurope",
"name": "default",
"properties": {
"onboardingPackageLinux": "Package code",
"onboardingPackageWindows": "Package code that we want to save for further use"
},
"type": "Microsoft.Security/mdeOnboardings"
}
]
}
And the thing that I would want to somehow save is onboardingPackageWindows so I can use it later on without copying manually the package code. I have tried almost everything that I known + google examples but still have issues.
Resolved
I used ConvertFrom-Json to get wanted data.

Azure Function App Push-OutputBinding Adding Subscription Information to JSON Output using Powershell

I have written several Azure Functions over the past year in both powershell and C#. I am currently writing an API that extracts rows from a Storage Account Table and returns that data in a JSON format.
The data pulls fine.
The data converts to JSON just fine.
A JSON formatted response is displayed - which is fine - but the Push-OutputBinding shoves in additional data to my original JSON data - account information, environment information, subscription information, and tenant information.
I've tried a number of different strategies for getting past this. I gave up on using C# to interact with the Tables because the whole Azure.Data.Tables and Cosmos tables packages are a hot mess with breaking changes and package conflicts and .Net 6 requirements for new functions apps. So please don't offer up a C# solution unless you have a working example with specific versions for packages, etc.
Here is the code:
Note that I have verified that $certData and $certJson properly formatted JSON that contain only the data I want to return.
using namespace System.Net
# Input bindings are passed in via param block.
param($Request, $TriggerMetadata)
# Write to the Azure Functions log stream.
Write-Host "PowerShell HTTP trigger function processed a request."
# Interact with query parameters or the body of the request.
$filter = $Request.Query.Filter
if (-not $filter) {
$filter = "ALL"
}
$certData = GetCerts $filter | ConvertTo-Json
#$certJson = $('{ "CertData":"' + $certData + '" }')
$body = "${CertData}"
# Associate values to output bindings by calling 'Push-OutputBinding'.
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = [HttpStatusCode]::OK
ContentType = "application/json"
Body = $body
})
When I call the httpTrigger function, the response looks like this:
{ "CertData":"[
{
"Name": "MySubscriptionName blah blah",
"Account": {
"Id": "my user id",
"Type": "User",
....
},
"Environment": {
"Name": "AzureCloud",
"Type": "Built-in",
...
},
"Subscription": {
"Id": "SubscriptionID",
"Name": "SubscriptionName",
....
},
"Tenant": {
"Id": "TenandID",
"TenantId": "TenantId",
"ExtendedProperties": "System.Collections.Generic.Dictionary`2[System.String,System.String]",
...
},
"TokenCache": null,
"VersionProfile": null,
"ExtendedProperties": {}
},
{
"AlertFlag": 1,
"CertID": "abc123",
"CertName": "A cert Name",
"CertType": "an assigned cert type",
"DaysToExpire": 666,
"Domain": "WWW.MYDOMAIN.COM",
"Expiration": "2033-10-04T21:31:03Z",
"PrimaryDomain": "WWW.MYDOMAIN.COM",
"ResourceGroup": "RANDOM-RESOURCES",
"ResourceName": "SOMERESOURCE",
"Status": "OK",
"Subscription": "MYSUBSCRIPTIONNAME",
"Thumbprint": "ABC123ABC123ABC123ABC123ABC123",
"PartitionKey": "PARKEY1",
"RowKey": "ID666",
"TableTimestamp": "2022-02-03T09:00:28.7516797-05:00",
"Etag": "W/\"datetime'2022-02-03T14%3A00%3A28.7516797Z'\""
},
...
Not only does the returned values add data I don't want exposed, it makes parsing the return data that I do want to get when I make API calls problematic.
How do I get rid of the data added by the Push-OutputBinding?
Was able to resolve issue by modifying run.ps1 as follows:
using namespace System.Net
# Input bindings are passed in via param block.
param($Request, $TriggerMetadata)
# Write to the Azure Functions log stream.
Write-Host "PowerShell HTTP trigger function processed a request."
# Interact with query parameters or the body of the request.
$filter = $Request.Query.Filter
if (-not $filter) {
$filter = "ALL"
}
$certData = ( GetCerts $filter | Select-Object -Skip 1 )
#write-information $certData | Format-List
# Associate values to output bindings by calling 'Push-OutputBinding'.
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = [HttpStatusCode]::OK
Body = $certData
})

Passing environment variables to NOW

I am trying to pass firebase environment variables for deployment with now.
I have encoded these variables manually with base64 and added them to now with the following command:
now secrets add firebase_api_key_dev "mybase64string"
The encoded string was placed within speech marks ""
These are in my CLI tool and I can see them all using the list command:
now secrets ls
> 7 secrets found under project-name [499ms]
name created
firebase_api_key_dev 6d ago
firebase_auth_domain_dev 6d ago
...
In my firebase config, I am using the following code:
const config = {
apiKey: Buffer.from(process.env.FIREBASE_API_KEY, "base64").toString(),
authDomain: Buffer.from(process.env.FIREBASE_AUTH_DOMAIN,"base64").toString(),
...
}
In my now.json file I have the following code:
{
"env": {
"FIREBASE_API_KEY": "#firebase_api_key_dev",
"FIREBASE_AUTH_DOMAIN": "#firebase_auth_domain_dev",
...
}
}
Everything works fine in my local environment (when I run next) as I also have a .env file with these variables, yet when I deploy my code, I get the following error in my now console:
TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be one of type string, Buffer, ArrayBuffer, Array, or Array-like Object. Received type undefined
Does this indicate that my environment variables are not being read? What's the issue here? It looks like they don't exist at all
The solution was to replace my existing now.json with:
{
"build":{
"env": {
"FIREBASE_API_KEY": "#firebase_api_key",
"FIREBASE_AUTH_DOMAIN": "#firebase_auth_domain",
"FIREBASE_DATABASE_URL": "#firebase_database_url",
"FIREBASE_PROJECT_ID": "#firebase_project_id",
"FIREBASE_STORAGE_BUCKET": "#firebase_storage_bucket",
"FIREBASE_MESSAGING_SENDER_ID": "#firebase_messaging_sender_id",
"FIREBASE_APP_ID": "#firebase_app_id",
"FIREBASE_API_KEY_DEV": "#firebase_api_key_dev",
"FIREBASE_AUTH_DOMAIN_DEV": "#firebase_auth_domain_dev",
"FIREBASE_DATABASE_URL_DEV": "#firebase_database_url_dev",
"FIREBASE_PROJECT_ID_DEV": "#firebase_project_id_dev",
"FIREBASE_STORAGE_BUCKET_DEV": "#firebase_storage_bucket_dev",
"FIREBASE_MESSAGING_SENDER_ID_DEV": "#firebase_messaging_sender_id_dev",
"FIREBASE_APP_ID_DEV": "#firebase_app_id_dev"
}
}
}
I was missing the build header.
I had to contact ZEIT support to help me identify this issue.

How to configure the APNS.Certificate in the arm template

I am using the following azuredeploy.json file for setting up the notification hub on the Azure cloud.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"Gcm.GoogleApiKey": {
"type": "string",
"metadata": {
"description": "Google Cloud Messaging API Key"
},
"defaultValue": "AIzaSyAyp9MernKgMS3wFNM3yNWByiP-TaGrqEg"
},
"APNS.Certificate": {
"type": "string",
"metadata": {
"description": "A certificate (in base 64 format) provided by Apple on the iOS Provisioning Portal"
},
"defaultValue": ""
},
"APNS.certificateKey": {
"type": "string",
"metadata": {
"description": "The Certificate Key provided by the iOS Provisioning Portal when registering the application"
},
"defaultValue": "ce469bf21dfa7b9d595d4999bfaca8a94ea47e46"
},
"APNS.endpoint": {
"type": "string",
"metadata": {
"description": "The APNS endpoint to which our service connects. This is one of two values: gateway.sandbox.push.apple.com for the sandbox endpoint or gateway.push.apple.com, for the production endpoint. Any other value is invalid."
},
"allowedValues": [
"gateway.sandbox.push.apple.com",
"gateway.push.apple.com"
],
"defaultValue": "gateway.push.apple.com"
}
},
"variables": {
"hubVersion": "[providers('Microsoft.NotificationHubs', 'namespaces').apiVersions[0]]",
"notificationHubNamespace": "[concat('hubv2', uniqueString(resourceGroup().id))]",
"notificationHubName": "notificationhub"
},
"resources": [
{
"name": "[variables('NotificationHubNamespace')]",
"location": "[resourceGroup().location]",
"type": "Microsoft.NotificationHubs/namespaces",
"apiVersion": "[variables('hubVersion')]",
"comments": "Notification hub namespace",
"properties": {
"namespaceType": "NotificationHub"
},
"resources": [
{
"name": "[concat(variables('NotificationHubNamespace'),'/',variables('NotificationHubName'))]",
"location": "[resourceGroup().location]",
"type": "Microsoft.NotificationHubs/namespaces/notificationHubs",
"apiVersion": "[variables('hubVersion')]",
"properties": {
"GcmCredential": {
"properties": {
"googleApiKey": "[parameters('Gcm.GoogleApiKey')]",
"gcmEndpoint": "https://android.googleapis.com/gcm/send"
}
}
},
"dependsOn": [
"[variables('NotificationHubNamespace')]"
]
}
]
}
],
"outputs": {
}
}
Now I tried to set up the apple push notification service also using the following snippet:
"apnsCredential": {
"properties": {
"apnsCertificate": "[parameters('APNS.Certificate')]",
"certificateKey": "[parameters('APNS.certificateKey')]",
"endpoint": " gateway.sandbox.push.apple.com or gateway.push.apple.com",
}
}
With the above changes, I executed the Deploy-AzureResourceGroup.ps1 using powershell command prompt and on executing it I am getting an error with message 'Bad Request'
Can anyone help me to fix this issue.
Add the proper APNS.Certificate and APNS.certificateKey.
It is failing on trying to verify your details, hence the bad request.
You need a base 64 formatted APNS.Certificate.
APNS.Certificate:
This is the Apple Push Notification certificate in base 64 string-format.
You can use PowerShell to convert the certificate like this (then copie the key from the output file ‘MyPushCert.txt’ and use it.):
$fileContentBytes = get-content ‘Apple_Certificate.p12’ -Encoding Byte
[System.Convert]::ToBase64String($fileContentBytes) | Out-File ‘MyPushCert.txt’
APNS.certificateKey:
This is the password you specified when you exported the certificate.(The password you created on Apple at the time of creating the cert.)
It's impossible to know exactly what caused this without knowing more about your environment/setup. According to this post, one possible issue could be how strong your password is:
After a few hours of pulling my hair out and not getting anything
beyond “Bad Request”, I finally thought to use a password stronger
than “pass#word1”. I’ll be darned, it worked. Not only that, but
provisioning with Azure Resource Manager is asynchronous, so your
scripts finish a heck of a lot sooner than they used to because VMs
provision in parallel.
The post recommends going through Troubleshooting common Azure deployment errors with Azure Resource Manager.
I'm not sure you should be dynamically setting the apiVersion for your templates. They vary depending on what you are deploying.
See Best Practices:
Avoid using a parameter or variable for the API version for a resource type. Resource properties and values can vary by version number. IntelliSense in a code editor cannot determine the correct schema when the API version is set to a parameter or variable. Instead, hard-code the API version in the template.
The correct apiVersion for notification hubs appears to be 2015-04-01: https://github.com/Azure/azure-resource-manager-schemas/blob/master/schemas/2015-04-01/Microsoft.NotificationHubs.json
I had to use PowerShell to resolve this problem.
Partially the idea was taken from here: https://learn.microsoft.com/en-us/azure/notification-hubs/notification-hubs-deploy-and-manage-powershell
Below is the script, which tested locally and is working.
Microsoft.Azure.NotificationHubs -Version 1.0.9 used.
We adopted it for VSTS Release as one of the PowerShell tasks/steps after Notification Hub has been created with ARM template.
Login-AzureRmAccount
Select-AzureRmSubscription -SubscriptionName my-subscription-here
Write-Host "Begin process..."
try
{
# Make sure to reference the latest version of Microsoft.Azure.NotificationHubs.dll
Write-Host "Adding the [Microsoft.Azure.NotificationHubs.dll] assembly to the script...";
$scriptPath = Split-Path (Get-Variable MyInvocation -Scope 0).Value.MyCommand.Path;
$packagesFolder = $scriptPath + "\packs";
Write-Host $packagesFolder;
$assembly = Get-ChildItem $packagesFolder -Include "Microsoft.Azure.NotificationHubs.dll" -Recurse;
write-Host $assembly.FullName;
Add-Type -Path $assembly.FullName;
Write-Host "The [Microsoft.Azure.NotificationHubs.dll] assembly has been successfully added to the script.";
# Create requered variables
$HubNamespace = "hub-namespace";
$HubName = "hub-name";
$ResourceGroup = "resource-group";
$GcmApiKey = "api key here";
# Possible values: gateway.push.apple.com, gateway.sandbox.push.apple.com
$ApnsEndpoint = "gateway.push.apple.com";
# A certificate (in base 64 format) provided by Apple on the iOS Provisioning Portal
$ApnsCertificate = "base 64 certificate here";
$ApnsCertificateKey = "certificate key/password here";
$GcmCredential = New-Object -TypeName Microsoft.Azure.NotificationHubs.GcmCredential -ArgumentList $GcmApiKey;
$ApnsCredential = New-Object -TypeName Microsoft.Azure.NotificationHubs.ApnsCredential;
$ApnsCredential.Endpoint = $ApnsEndpoint;
$ApnsCredential.ApnsCertificate = $ApnsCertificate;
$ApnsCredential.CertificateKey = $ApnsCertificateKey;
# Query the namespace
$FoundNamespaces = Get-AzureRmNotificationHubsNamespace -Namespace $HubNamespace -ResourceGroup $ResourceGroup
# Check if the namespace already exists
if ($FoundNamespaces -and $FoundNamespaces.Length -eq 1)
{
$CurrentNamespace = $FoundNamespaces[0];
Write-Host "The namespace [$HubNamespace] in the [$($CurrentNamespace.Location)] region was found.";
$HubListKeys = Get-AzureRmNotificationHubListKeys -Namespace $HubNamespace -ResourceGroup $ResourceGroup -NotificationHub $HubName -AuthorizationRule DefaultFullSharedAccessSignature;
# Check to see if the Notification Hub exists
if ($HubListKeys)
{
# Create the NamespaceManager object used to update the notification hub
Write-Host "Creating a NamespaceManager object for the [$HubNamespace] namespace...";
$NamespaceManager = [Microsoft.Azure.NotificationHubs.NamespaceManager]::CreateFromConnectionString($HubListKeys.PrimaryConnectionString);
Write-Host "NamespaceManager object for the [$HubNamespace] namespace has been successfully created.";
# Update notification hub with new details
Write-Host "The [$Path] notification hub already exists in the [$HubNamespace] namespace." ;
$NHDescription = $NamespaceManager.GetNotificationHub($HubName);
$NHDescription.GcmCredential = $GcmCredential;
$NHDescription.ApnsCredential = $ApnsCredential;
$NHDescription = $NamespaceManager.UpdateNotificationHub($NHDescription);
Write-Host "The [$HubName] notification hub was updated";
}
else
{
Write-Host "The [$HubName] notification hub does not exist."
}
}
else
{
Write-Host "The [$HubNamespace] namespace does not exist."
}
}
catch [System.Exception]
{
Write-Error($_.Exception.Message)
}
Hope that helps someone.

Powershell - Converting Azure Keyvault Response Data

I am trying to setup a script for setting up my keyvault and deploying my ARM templates. When I create a keyvault I want to take that output and store it into an object, say, $output. The output looks like so
Name : CertificateThumbprint
Value : xxxxx
Name : SourceVault
Value : xxxxxxx
Name : CertificateURL
Value : xxxxxxxxx
I want to convert this to Json (or xml) so that I can access the data and update my template parameters file. However, when I try to ConvertTo-Json or ConvertTo-Xml I get something like
[
{
"pageHeaderEntry": null,
"pageFooterEntry": null,
"autosizeInfo": null,
"shapeInfo": {
"ClassId2e4f51ef21dd47e99d3c952918aff9cd": "..."
},
"groupingEntry": null,
"ClassId2e4f51ef21dd47e99d3c952918aff9cd": "..."
},
{
"shapeInfo": null,
"groupingEntry": null,
"ClassId2e4f51ef21dd47e99d3c952918aff9cd": "..."
},
{
"formatEntryInfo": {
"listViewFieldList": "Microsoft.PowerShell.Commands.Internal.Format.ListViewField Microsoft.PowerShell.Commands.Internal.Format.ListViewField"
...
]
My powershell experience is pretty minimal so I'm not exactly familiar with all the Format options.
$output[2].formatEntryInfo.listViewFieldList...foo...bar
Should allow you to access your properties of the output without having to convert at all