I am trying to setup a script for setting up my keyvault and deploying my ARM templates. When I create a keyvault I want to take that output and store it into an object, say, $output. The output looks like so
Name : CertificateThumbprint
Value : xxxxx
Name : SourceVault
Value : xxxxxxx
Name : CertificateURL
Value : xxxxxxxxx
I want to convert this to Json (or xml) so that I can access the data and update my template parameters file. However, when I try to ConvertTo-Json or ConvertTo-Xml I get something like
[
{
"pageHeaderEntry": null,
"pageFooterEntry": null,
"autosizeInfo": null,
"shapeInfo": {
"ClassId2e4f51ef21dd47e99d3c952918aff9cd": "..."
},
"groupingEntry": null,
"ClassId2e4f51ef21dd47e99d3c952918aff9cd": "..."
},
{
"shapeInfo": null,
"groupingEntry": null,
"ClassId2e4f51ef21dd47e99d3c952918aff9cd": "..."
},
{
"formatEntryInfo": {
"listViewFieldList": "Microsoft.PowerShell.Commands.Internal.Format.ListViewField Microsoft.PowerShell.Commands.Internal.Format.ListViewField"
...
]
My powershell experience is pretty minimal so I'm not exactly familiar with all the Format options.
$output[2].formatEntryInfo.listViewFieldList...foo...bar
Should allow you to access your properties of the output without having to convert at all
Related
I have written several Azure Functions over the past year in both powershell and C#. I am currently writing an API that extracts rows from a Storage Account Table and returns that data in a JSON format.
The data pulls fine.
The data converts to JSON just fine.
A JSON formatted response is displayed - which is fine - but the Push-OutputBinding shoves in additional data to my original JSON data - account information, environment information, subscription information, and tenant information.
I've tried a number of different strategies for getting past this. I gave up on using C# to interact with the Tables because the whole Azure.Data.Tables and Cosmos tables packages are a hot mess with breaking changes and package conflicts and .Net 6 requirements for new functions apps. So please don't offer up a C# solution unless you have a working example with specific versions for packages, etc.
Here is the code:
Note that I have verified that $certData and $certJson properly formatted JSON that contain only the data I want to return.
using namespace System.Net
# Input bindings are passed in via param block.
param($Request, $TriggerMetadata)
# Write to the Azure Functions log stream.
Write-Host "PowerShell HTTP trigger function processed a request."
# Interact with query parameters or the body of the request.
$filter = $Request.Query.Filter
if (-not $filter) {
$filter = "ALL"
}
$certData = GetCerts $filter | ConvertTo-Json
#$certJson = $('{ "CertData":"' + $certData + '" }')
$body = "${CertData}"
# Associate values to output bindings by calling 'Push-OutputBinding'.
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = [HttpStatusCode]::OK
ContentType = "application/json"
Body = $body
})
When I call the httpTrigger function, the response looks like this:
{ "CertData":"[
{
"Name": "MySubscriptionName blah blah",
"Account": {
"Id": "my user id",
"Type": "User",
....
},
"Environment": {
"Name": "AzureCloud",
"Type": "Built-in",
...
},
"Subscription": {
"Id": "SubscriptionID",
"Name": "SubscriptionName",
....
},
"Tenant": {
"Id": "TenandID",
"TenantId": "TenantId",
"ExtendedProperties": "System.Collections.Generic.Dictionary`2[System.String,System.String]",
...
},
"TokenCache": null,
"VersionProfile": null,
"ExtendedProperties": {}
},
{
"AlertFlag": 1,
"CertID": "abc123",
"CertName": "A cert Name",
"CertType": "an assigned cert type",
"DaysToExpire": 666,
"Domain": "WWW.MYDOMAIN.COM",
"Expiration": "2033-10-04T21:31:03Z",
"PrimaryDomain": "WWW.MYDOMAIN.COM",
"ResourceGroup": "RANDOM-RESOURCES",
"ResourceName": "SOMERESOURCE",
"Status": "OK",
"Subscription": "MYSUBSCRIPTIONNAME",
"Thumbprint": "ABC123ABC123ABC123ABC123ABC123",
"PartitionKey": "PARKEY1",
"RowKey": "ID666",
"TableTimestamp": "2022-02-03T09:00:28.7516797-05:00",
"Etag": "W/\"datetime'2022-02-03T14%3A00%3A28.7516797Z'\""
},
...
Not only does the returned values add data I don't want exposed, it makes parsing the return data that I do want to get when I make API calls problematic.
How do I get rid of the data added by the Push-OutputBinding?
Was able to resolve issue by modifying run.ps1 as follows:
using namespace System.Net
# Input bindings are passed in via param block.
param($Request, $TriggerMetadata)
# Write to the Azure Functions log stream.
Write-Host "PowerShell HTTP trigger function processed a request."
# Interact with query parameters or the body of the request.
$filter = $Request.Query.Filter
if (-not $filter) {
$filter = "ALL"
}
$certData = ( GetCerts $filter | Select-Object -Skip 1 )
#write-information $certData | Format-List
# Associate values to output bindings by calling 'Push-OutputBinding'.
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = [HttpStatusCode]::OK
Body = $certData
})
Capture text between two different char using PowerShell. Between the first{ and the last } . Basically there is text with Json in it and I want to capture the json from it. I have looked for examples but so far no luck.
PROJECT Description: Azure Test Project Description
PROJECT ADMINISTRATORS: jjohnson
CONTRIBUTORS: jdoe
BOARD PROCESS: Agile
SPECIAL INSTRUCTIONS:
{
"organization": "https://dev.azure.com/cloudops",
"projectName": "Test Project",
"projectDescription": "Azure Test Project Description",
"projectProcessType": "Agile",
"specialInstructions": "",
"adminMembers": [
{
"userSamAccountName": "jjohnson",
"userEmailAddress": "jjohnson#test.com",
"userPrincipalName": "jjohnson#test.com",
"projectGroupType": "projectAdministrator"
}
],
"contribMembers": [
{
"userSamAccountName": "jdoe",
"userEmailAddress": "jdoe#test.com",
"userPrincipalName": "jdoe#test.com",
"projectGroupType": "projectContributor"
}
]
}
Is this what you were looking for?
[Regex]::Match((Get-Content "sampleinputfile.txt" -Raw),
'^{.+}',
[Text.RegularExpressions.RegexOptions]::Multiline -bor
[Text.RegularExpressions.RegexOptions]::Singleline).Value
Basically this converts the input file to a single (newline-delimited) string (Get-Content -Raw), and then uses the .NET Framework's Match method to do a regular expression match for the lines of text between the { and } characters (inclusive).
Is there any way to approve runs via the CLI or the API (or anything else)? I'm looking for a way to bulk approve multiple runs from different pipelines but it's not available in the UI.
Let's say I have 100 pipelines that have a deployment job to a production environment. I would like to approve all awaiting for approval runs.
Currently, I cannot find something like it in the docs of the Azure DevOps REST API or the CLI.
The feature docs:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/environments
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/approvals
The following question is related but I'm looking for any way of solving it but not just via API:
Approve a yaml pipeline deployment in Azure DevOps using REST api
I was just searching for an answer for this regarding getting the approval id that you would need. In fact there is an undocumented API to approve an approval check.
This is as Merlin explain the following
https://dev.azure.com/{org}/{project}/_apis/pipelines/approvals/{approvalId}
The body has to look like this
[{
"approvalId": "{approvalId}",
"status": {approvalStatus},
"comment": ""
}]
where {approvalStatus} is telling the API if you approved or not. You probly have to try, but I had a 4 as a status. I guess there are only 2 possibilities. Either for "approved" or "denied".
The question is now how you get the approval ID? I found it. You get it by using the timeline API of a classic build. The build API documentation says that you get it by the following
https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}?api-version=5.1
the build timeline you get in the response of the build run, but it has a pattern which is
https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}/Timeline?api-version=5.1
Besides a flat array container a parent / child rleationship from stage, phase, job and tasks, you can find within it something like the following:
{
"records": [
{
"previousAttempts": [
],
"id": "95f5837e-769d-5a92-9ecb-0e7edb3ac322",
"parentId": "9e7965a8-d99d-5b8f-b47b-3ee7c58a5b1c",
"type": "Checkpoint",
"name": "Checkpoint",
"startTime": "2020-08-14T13:44:03.05Z",
"finishTime": null,
"currentOperation": null,
"percentComplete": null,
"state": "inProgress",
"result": null,
"resultCode": null,
"changeId": 73,
"lastModified": "0001-01-01T00:00:00",
"workerName": null,
"details": null,
"errorCount": 0,
"warningCount": 0,
"url": null,
"log": null,
"task": null,
"attempt": 1,
"identifier": "Checkpoint"
},
{
"previousAttempts": [
],
"id": "9e7965a8-d99d-5b8f-b47b-3ee7c58a5b1c",
"parentId": null,
"type": "Stage",
"name": "Power Platform Test (orgf92be262)",
"startTime": null,
"finishTime": null,
"currentOperation": null,
"percentComplete": null,
"state": "pending",
"result": null,
"resultCode": null,
"changeId": 1,
"lastModified": "0001-01-01T00:00:00",
"workerName": null,
"order": 2,
"details": null,
"errorCount": 0,
"warningCount": 0,
"url": null,
"log": null,
"task": null,
"attempt": 1,
"identifier": "Import_Test"
},
{
"previousAttempts": [
],
"id": "e54149c5-b5a7-4b82-8468-56ad493224b5",
"parentId": "95f5837e-769d-5a92-9ecb-0e7edb3ac322",
"type": "Checkpoint.Approval",
"name": "Checkpoint.Approval",
"startTime": "2020-08-14T13:44:03.02Z",
"finishTime": null,
"currentOperation": null,
"percentComplete": null,
"state": "inProgress",
"result": null,
"resultCode": null,
"changeId": 72,
"lastModified": "0001-01-01T00:00:00",
"workerName": null,
"details": null,
"errorCount": 0,
"warningCount": 0,
"url": null,
"log": null,
"task": null,
"attempt": 1,
"identifier": "e54149c5-b5a7-4b82-8468-56ad493224b5"
}
],
"lastChangedBy": "00000002-0000-8888-8000-000000000000",
"lastChangedOn": "2020-08-14T13:44:03.057Z",
"id": "86fb4204-9c5e-4e72-bdb1-eefe230480ec",
"changeId": 73,
"url": "https://dev.azure.com/***"
}
below you can see a step that is called "Checkpoint.Approval". The id of that step IS the approval Id you need to approve everything. If you want to know from which stage the approval is, then you can follow up the parentIds until the parentId property is null.
This will then be the stage.
With this you can successfully get the approval id and use it to approve with the said
What jessehouwing's guess is correct. Now multi-stage still be in preview, and the corresponding SDK/API/extension hasn't been expanded and provided to public.
You may think that what about not using API. I have checked the corresponding code from our backend, all of operations to multi-stage approval contain one required parameter: approvalId. I'm sure you have known that this value is unique and different approval map with different approvalId value. This means, no matter which method you want to try with, approvalId is the big trouble. And based on my known, until now, there's no any api/SDK, third tool or extension can achieve this value directly.
In addition, for multi-stage YAML, its release process logic is not same with the release that defined with UI. So, all of public APIs which can work with release(UI), are not suitable with the release of multi-stage.
We have one undisclosed api, can get Approval message of multi-stage:
https://dev.azure.com/{org}/{project}/_apis/pipelines/approvals/{approvalId}
You can try with listing approval without specifying approvalId: https://dev.azure.com/{org}/{project}/_apis/pipelines/approvals. And its response message: Query for approvals failed. A minimum of one query parameter is required.\r\nParameter name: queryParameters. This represents you must tell system the specified approval(the big trouble I mentioned previously).
In fact, for why approvalId is a necessary part, it is caused from our backend code structure. I'd suggest you raise suggestion on developing API/SDK for multi-stage here.
I can confirm that Sebastian's answer worked for me, even in Azure DevOps 2020 on-prem.
After retrieving the approvalId from either methods used above (I was specifically using a service hook for my integration), I used the following API PATCH call:
https://dev.azure.com/{organization}/{project}/_apis/pipelines/approvals/?api-version=6.0-preview
and in the body:
[
{
"approvalId": "{approvalId}",
"status": {status integer}, (4 - approved; 8 - rejected)
"comment": ""
}
]
The call is passed with the application/json Content-Type, but in some situations it did not like that I was using the [] brackets, so you will need to work around that, only then will the call work.
I was even able to integrate this call into my custom connector in MS Power Automate
I added support to the latest version of the AzurePipelinesPS Powershell module to support bulk pipeline approvals.
Code snippet without using the AzurePipelinesPS sessions
$instance = 'https://dev.azure.com'
$collection = 'your_project'
$project = 'your_project'
$apiVersion = '5.1-preview'
$securePat = 'your_personal_access_token' | ConvertTo-SecureString -Force -AsPlainText
Get-APPipelinePendingApprovalList -Instance $instance -Collection $collection -Project $project -PersonalAccessToken $securePat -ApiVersion $apiVersion | Out-GridView -Passthru | % { Update-APPipelineApproval -Instance $instance -Collection $collection -Project $project -PersonalAccessToken $securePat -ApiVersion $apiVersion -ApprovalId $PSitem.approvalId -status 'approved'}
Code snippet with AzurePipelinesPS sessions
$session = 'your_session'
Get-APPipelinePendingApprovalList $session | Out-GridView -Passthru | % { Update-APPipelineApproval $session -ApprovalId $PSitem.approvalId -status 'approved'}
See the AzurePipelinesPS project page for details on secure session handling.
Function Definitions used in the code above
Get-APPipelinePendingApprovalList
Loops through pipeline build runs with the status of 'notStarted' or 'inProgress' in a project. This build lookup supports filters like pipeline definition ids or a source branch name.
For each build it then looks up the timeline record where the approval id, the stage name and the stage identifier are found.
Optionally with the ExpandApproval switch it can expand each approval with details
The object returned from this function contains the following properties, the values have been mocked
pipelineDefinitionName : MyPipeline
pipelineDefinitionId : 100
pipelineRunId : 2001
pipelineUrl : https://dev.azure.com/your_project/_build/results?
sourceBranch : refs/heads/master
stageName : Prod Deployment
stageIdentifier : Prod_Deployment
approvalId : xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx
Out-GridView
Displays data in a Grid View where the results can be filtered, ordered and selected.
%
The percent sign is shorthand for Foreach-Object
Update-APPipelineApproval
Updates the status of an approval to approved or rejected.
Credit
Thanks to Sebastian Schütze for cracking the timeline part!
The az pipelines extension doesn't suport approvals yet, I suppose due to the fact that multi-stage pipelines are still in preview and the old release hub will eventually be replaced by it.
But there is a REST API you can use to list and update approvals. These can be called from PowerShell with relative ease.
Or use the vsteam powershell module and Get-VSTeamApproval and Set-VSTeamApproval.
I tried to add user to keycloak 3.4.3.Final by admin api with json file like this.
Command
bin\kcadm.bat create users -r master -f add-user.json
add-user.json Content
{
"username" : "myUserName",
"enabled" : true,
"credentials" : [ {
"type" : "password",
"hashedSaltedValue" : "encoded password string",
"salt" : "salt string",
"hashIterations" : 27500,
"algorithm" : "pbkdf2-sha256"
} ],
"realmRoles" : [ "admin" ]
}
Salt value was not registered as it was written in the json file.
For example, when use published test data, salt value "3fBAt5GAGGxFrV9fznpZHQ==" was registed as "ddf040b79180186c45ad5f5fce7a591d" on database.
How can I register my intended salt value?
The above problem was solved. Sorry for lack of verification.
I should write salt byte value base64 encoded.
For those wondering how to get the base64 encoded do the following:
echo -n 'the_value' | openssl base64
you get your salt based64 encoded and you can copy that and put in the JSON.
I am trying to create a custom image in my Azure DevTestLab, from a vhd image stored in the default storage account.
I started modifying the file at the GitHub repo for ARM templates for creating a custom image from a vhd.
I have already created the DevTest lab, and I can run the following Powershell command to get its data:
PS C:\git> $rid = Get-AzureRmResource -ResourceGroupName jgTl9-rg -ResourceType Microsoft.DevTestLab/labs -ResourceName jgTl9 -ApiVersion 2016-05-15
This yields a bunch of information. If I want to extract the default storage account name, I do the following:
PS C:\git> $rid.properties.defaultStorageAccount
So I know it's possible to get it. Now I have modified the variables section of the Azure build template (a JSON file) to look as follows:
"variables": {
"resourceName": "[concat(variables('existingLabName'), '/', variables('imageName'))]",
"resourceType": "Microsoft.DevTestLab/labs/customimages",
"existingLabName": "[replace(resourceGroup().name,'-rg','')]",
"storageAccountID": "/subscriptions/xxxxxxxxxxxx/resourceGroups/jgTl9-rg/providers/Microsoft.Storage/storageAccounts/djgtl91795",
"saidFrags": "[skip(split(variables('storageAccountID'), '/' ), add(length(split(variables('storageAccountID'), '/' )),-1) ) ]",
"storageAccountSuffix": "[take(variables('saidFrags'),1)[0]]",
"existingVhdUri": "[concat('https://',variables('storageAccountSuffix'),'.blob.core.windows.net/uploads/',parameters('imageFileName'))]",
"imageName": "JaysImage"
},
This code actually works when I call the New-AzureRmResourceGroupDeployment command with the azuredeploy.json and azuredeploy.parameters.json.
However, instead of hardcoding the storageAccountID variable, I'd like to extract the defaultStorageAccount from the environment. I tried something like this:
"storageAccountID": "[string( resourceId(resourceGroup().name,'MicrosoftDevTestLab/labs','jgTl9').properties.defaultStorageAccount ))",
I tried a few variations on this but nothing worked. What is the syntax I need to use to extract the same information as I have hardcoded for the storageAccountID?
(I could not find anything in Microsoft's description of template functions).
Couple of issues with your following expression
"storageAccountID": "[string( resourceId(resourceGroup().name,'MicrosoftDevTestLab/labs','jgTl9').properties.defaultStorageAccount ))",
There is syntax issue ie. an extra ) at the end.
The resource provider MicrosoftDevTestlab/labs should be Microsoft.DevTestlab/labs
When resourceId is used it will not give you the entire object. So when you do properties.defaultStorageAccount it will not work. Because it gives you just the string of your dev test lab and nothing else and there is no property on a string of defaultStorageAccount.
Now what you need is reference function like below
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"vDefaultStorageAccount": "[string(resourceId('yourResourceGroupName','Microsoft.DevTestLab/labs','yourTestLabName'))]"
},
"resources": [],
"outputs": {
"defaultStorageAccount":{
"value": "[reference(variables('vDefaultStorageAccount'),'2016-05-15').defaultStorageAccount]",
"type":"string"
}
}
}
In the variable, vDefaultStorageAccount we get the ID of the DevTestLab and then we use reference function to get the object. Then you can do . to get properties. You don't need to do properties.defaultStorageAccount for example.