I am using Invoke-WebRequest to get back an HtmlWebResponseObject object. I want to get the value of the ID field, when the description field matches a particular string (in this case, "server1"). There are hundreds of results returned by Invoke-WebRequest (so the string is super long) Given the sample below, how would I extract the ID?
{
"status": 200,
"data": [
{
"nextRecipient": 0,
"clearSent": true,
"lastSentNotificationOn": 0,
"netscanVersion": "0",
"suppressAlertClear": "false",
"build": "19000",
"lastSentNotificationOnLocal": "",
"id": 6,
"resendIval": 15,
"watchdogUpdatedOn": "2016-04-11 10:28:02 MDT",
"escalatingChainId": 6,
"description": "domain\server1",
"ackComment": "",
"credential2": "",
"updatedOn": 1460392096,
"updatedOnLocal": "2016-04-11 10:28:16 MDT",
"agentConf": "product.code={guid}\r\n# Installer version, Shall not be modified\r\ninstaller.version=0001\r\n\r\n# Generated by Agent Configuration Wizard\r\nserver=url\r\ncompany=company\r\nid=6\r\ncredential==cred\r\n\r\n# logger settings. Set logger.size to 0 to no limitation on logger size, otherwise, size limited to Mbytes specified by that\r\nlogger.output=console\r\nlogger.logfile=\r\nlogger.size=64\r\nlogger.level=info\r\n\r\n#watchdog log level\r\nlogger.watchdog=info\r\n# for each component, add more detailed control here\r\n# e.g. \r\n# logger.level.controller=debug\r\n#\r\n# if not set, it will use default log level, i.e. value of logger.level\r\n#\r\n\r\n#if agent shall watch watchdog, default to false\r\nagentmonitorwatchdog=true\r\n\r\n#whether watchdog upgrades agent, default to true\r\nagent.autoupgrade=true\r\n\r\n#service connection timeouts. Default to 5 seconds for connecting and 30 seconds for sending / reading feeds from server\r\nservice.connect_timeout=5\r\nservice.read_timeout=30\r\n\r\n#SSL & Proxy settings\r\nssl.enable=true\r\nproxy.enable=false\r\nproxy.host= \r\nproxy.port=\r\nproxy.user=\r\nproxy.pass=\r\nproxy.exclude=\r\n\r\n#sbproxy settings\r\nsbproxy.address=127.0.0.1\r\nsbproxy.port=72\r\nsbproxy.logsize=64\r\nsbproxy.restartOn102=false\r\nsbproxy.pdhNoAuthentication=false\r\n\r\n#sbproxy connection pool settings\r\nsbproxy.pool.connections=50\r
Thanks.
Convert the response from a JSON string to an object and expand the relevant properties:
$response | ConvertFrom-Json |
Select-Object -Expand data |
Where-Object { $_.description -match 'server1' } |
Select-Object -Expand id
Related
I'm processing through Powershell script some API result processing.
API data (json) come from this :
$tree = Invoke-WebRequest -Uri "xxxxxxxxmonURLxxxxxxxxxx/130333"
$children = ($tree.Content | ConvertFrom-Json).data.12345.children
Then I loop through $children object using | ForEach
$_ within loop has "147852" as $_.Name, and the following object as $_.Definition
I'd like to parse the object within $_.Definition but cannot figure out how to access it.
The Definition object looks like this:
TypeName : System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
147852 NoteProperty System.Management.Automation.PSCustomObject 147852=#{nodeType=node; name=test1; flag=N0; creationDate=2022-02-17T14:50:16+00:00; hasAlerts=False; children=}
And I wish to access any property within the 147852 key (such as nodeType, name, flag, ..., children).
$_.147852 outputs an error saying 147852 was not found.
Thanks.
API json returned:
{
"data": {
"130333": {
"nodeType": "node",
"name": "Test name",
"flag": "N0",
"children": {
"147852": {
"nodeType": "node",
"name": "test1",
"flag": "N0",
"hasAlerts": false,
"children": {
"147853": {
"nodeType": "node",
"name": "test2",
"flag": "N0",
"children": {
"NP12-N9-S4": {
"nodeType": "agent",
"name": "Win10",
"type": "S"
}
}
}
}
}
}
}
}
Jeroen Mostert provided the crucial pointer in the comments, and Bender the Greatest links to what is effectively a duplicate question, but given that the latter is hashtable-focused, let me recapitulate the problem in the context of custom objects ([pscustomobject]):
Leaving the accidental use of Get-Member out of the picture, your problem ultimately boils down to a parser bug in PowerShell (see GitHub issue #14036):
To avoid it, quote property names that look like numbers - e.g., to access property 147852 on object $obj, use $obj.'147852'
Strictly speaking, the bug only surfaces if you attempt an additional (nested) property access:
# Nested sample custom object.
$obj = [pscustomobject] #{ 147852 = [pscustomobject] #{ name = 'test1' } }
# OK - number-like property is accessed without quoting, but *not nested*.
# However, $obj.'147852' is preferable.
$obj.147852
# *Nested* property access:
# !! BUG triggers error: "Missing property name after reference operator."
$obj.147852.name
# OK: Quoting avoids the problem.
$obj.'147852'.name # -> 'test1'
I have written several Azure Functions over the past year in both powershell and C#. I am currently writing an API that extracts rows from a Storage Account Table and returns that data in a JSON format.
The data pulls fine.
The data converts to JSON just fine.
A JSON formatted response is displayed - which is fine - but the Push-OutputBinding shoves in additional data to my original JSON data - account information, environment information, subscription information, and tenant information.
I've tried a number of different strategies for getting past this. I gave up on using C# to interact with the Tables because the whole Azure.Data.Tables and Cosmos tables packages are a hot mess with breaking changes and package conflicts and .Net 6 requirements for new functions apps. So please don't offer up a C# solution unless you have a working example with specific versions for packages, etc.
Here is the code:
Note that I have verified that $certData and $certJson properly formatted JSON that contain only the data I want to return.
using namespace System.Net
# Input bindings are passed in via param block.
param($Request, $TriggerMetadata)
# Write to the Azure Functions log stream.
Write-Host "PowerShell HTTP trigger function processed a request."
# Interact with query parameters or the body of the request.
$filter = $Request.Query.Filter
if (-not $filter) {
$filter = "ALL"
}
$certData = GetCerts $filter | ConvertTo-Json
#$certJson = $('{ "CertData":"' + $certData + '" }')
$body = "${CertData}"
# Associate values to output bindings by calling 'Push-OutputBinding'.
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = [HttpStatusCode]::OK
ContentType = "application/json"
Body = $body
})
When I call the httpTrigger function, the response looks like this:
{ "CertData":"[
{
"Name": "MySubscriptionName blah blah",
"Account": {
"Id": "my user id",
"Type": "User",
....
},
"Environment": {
"Name": "AzureCloud",
"Type": "Built-in",
...
},
"Subscription": {
"Id": "SubscriptionID",
"Name": "SubscriptionName",
....
},
"Tenant": {
"Id": "TenandID",
"TenantId": "TenantId",
"ExtendedProperties": "System.Collections.Generic.Dictionary`2[System.String,System.String]",
...
},
"TokenCache": null,
"VersionProfile": null,
"ExtendedProperties": {}
},
{
"AlertFlag": 1,
"CertID": "abc123",
"CertName": "A cert Name",
"CertType": "an assigned cert type",
"DaysToExpire": 666,
"Domain": "WWW.MYDOMAIN.COM",
"Expiration": "2033-10-04T21:31:03Z",
"PrimaryDomain": "WWW.MYDOMAIN.COM",
"ResourceGroup": "RANDOM-RESOURCES",
"ResourceName": "SOMERESOURCE",
"Status": "OK",
"Subscription": "MYSUBSCRIPTIONNAME",
"Thumbprint": "ABC123ABC123ABC123ABC123ABC123",
"PartitionKey": "PARKEY1",
"RowKey": "ID666",
"TableTimestamp": "2022-02-03T09:00:28.7516797-05:00",
"Etag": "W/\"datetime'2022-02-03T14%3A00%3A28.7516797Z'\""
},
...
Not only does the returned values add data I don't want exposed, it makes parsing the return data that I do want to get when I make API calls problematic.
How do I get rid of the data added by the Push-OutputBinding?
Was able to resolve issue by modifying run.ps1 as follows:
using namespace System.Net
# Input bindings are passed in via param block.
param($Request, $TriggerMetadata)
# Write to the Azure Functions log stream.
Write-Host "PowerShell HTTP trigger function processed a request."
# Interact with query parameters or the body of the request.
$filter = $Request.Query.Filter
if (-not $filter) {
$filter = "ALL"
}
$certData = ( GetCerts $filter | Select-Object -Skip 1 )
#write-information $certData | Format-List
# Associate values to output bindings by calling 'Push-OutputBinding'.
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = [HttpStatusCode]::OK
Body = $certData
})
When through the documentation of pagerduty was but still not able to understand what parameters to send in the request body and also facing trouble in understanding how to make the api request.If any one can share the sample code on making a pagerduty schedule that would help me alot.
Below is the sample code to create schedules in PagerDuty.
Each list can have multiple items (to add more users / layers)
import requests
url = "https://api.pagerduty.com/schedules?overflow=false"
payload={
"schedule": {
"schedule_layers": [
{
"start": "<dateTime>", # Start Time of layer | "start": "2021-01-01T00:00:00+05:30",
"users": [
{
"user": {
"id": "<string>", # ID of user to add in layer
"summary": "<string>",
"type": "<string>", # "type": "user"
"self": "<url>",
"html_url": "<url>"
}
}
],
"rotation_virtual_start": "<dateTime>", # Start of layer | "rotation_virtual_start": "2021-01-01T00:00:00+05:30",
"rotation_turn_length_seconds": "<integer>", # Layer rotation, for multiple user switching | "rotation_turn_length_seconds": <seconds>,
"id": "<string>", # Auto-generated. Only needed if you want update and existing Schedule Layer
"end": "<dateTime>", # End Time of layer | "end": "2021-01-01T00:00:00+05:30",
"restrictions": [
{
"type": "<string>", # To restrict shift to certain timings Weekly daily etc | "type": "daily_restriction",
"duration_seconds": "<integer>", # Duration of layer | "duration_seconds": "300"
"start_time_of_day": "<partial-time>", #Start time of layer | "start_time_of_day": "00:00:00",
"start_day_of_week": "<integer>"
}
],
"name": "<string>", # Name to give Layer
}
]
"time_zone": "<activesupport-time-zone>", # Timezone to set for layer and its timings | "time_zone": "Asia/Kolkata",
"type": "schedule",
"name": "<string>", # Name to give Schedule
"description": "<string>",# Description to give Schedule
"id": "<string>", # Auto-generated. Only needed if you want update and existing Schedule Layer
}
}
headers = {
'Authorization': 'Token token=<Your token here>',
'Accept': 'application/vnd.pagerduty+json;version=2',
'Content-Type': 'application/json'
}
response = requests.request("POST", url, headers=headers, json=payload)
print(response.text)
Best way to do this is to get the postman collection for PagerDuty and edit the request as per your liking. Once you get a successful response, convert that into code using the inbuilt feature of postman.
Using PagerDuty API for scheduling is not easy. Creating new schedule is okaish, but if you decide to update schedule - it is definitely not trivial. You'll probably occur bunch of limitation: number of restriction per layer, must reuse current layers, etc.
As option you can use a python library pdscheduling https://github.com/skrypka/pdscheduling
Is there any way to approve runs via the CLI or the API (or anything else)? I'm looking for a way to bulk approve multiple runs from different pipelines but it's not available in the UI.
Let's say I have 100 pipelines that have a deployment job to a production environment. I would like to approve all awaiting for approval runs.
Currently, I cannot find something like it in the docs of the Azure DevOps REST API or the CLI.
The feature docs:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/environments
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/approvals
The following question is related but I'm looking for any way of solving it but not just via API:
Approve a yaml pipeline deployment in Azure DevOps using REST api
I was just searching for an answer for this regarding getting the approval id that you would need. In fact there is an undocumented API to approve an approval check.
This is as Merlin explain the following
https://dev.azure.com/{org}/{project}/_apis/pipelines/approvals/{approvalId}
The body has to look like this
[{
"approvalId": "{approvalId}",
"status": {approvalStatus},
"comment": ""
}]
where {approvalStatus} is telling the API if you approved or not. You probly have to try, but I had a 4 as a status. I guess there are only 2 possibilities. Either for "approved" or "denied".
The question is now how you get the approval ID? I found it. You get it by using the timeline API of a classic build. The build API documentation says that you get it by the following
https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}?api-version=5.1
the build timeline you get in the response of the build run, but it has a pattern which is
https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}/Timeline?api-version=5.1
Besides a flat array container a parent / child rleationship from stage, phase, job and tasks, you can find within it something like the following:
{
"records": [
{
"previousAttempts": [
],
"id": "95f5837e-769d-5a92-9ecb-0e7edb3ac322",
"parentId": "9e7965a8-d99d-5b8f-b47b-3ee7c58a5b1c",
"type": "Checkpoint",
"name": "Checkpoint",
"startTime": "2020-08-14T13:44:03.05Z",
"finishTime": null,
"currentOperation": null,
"percentComplete": null,
"state": "inProgress",
"result": null,
"resultCode": null,
"changeId": 73,
"lastModified": "0001-01-01T00:00:00",
"workerName": null,
"details": null,
"errorCount": 0,
"warningCount": 0,
"url": null,
"log": null,
"task": null,
"attempt": 1,
"identifier": "Checkpoint"
},
{
"previousAttempts": [
],
"id": "9e7965a8-d99d-5b8f-b47b-3ee7c58a5b1c",
"parentId": null,
"type": "Stage",
"name": "Power Platform Test (orgf92be262)",
"startTime": null,
"finishTime": null,
"currentOperation": null,
"percentComplete": null,
"state": "pending",
"result": null,
"resultCode": null,
"changeId": 1,
"lastModified": "0001-01-01T00:00:00",
"workerName": null,
"order": 2,
"details": null,
"errorCount": 0,
"warningCount": 0,
"url": null,
"log": null,
"task": null,
"attempt": 1,
"identifier": "Import_Test"
},
{
"previousAttempts": [
],
"id": "e54149c5-b5a7-4b82-8468-56ad493224b5",
"parentId": "95f5837e-769d-5a92-9ecb-0e7edb3ac322",
"type": "Checkpoint.Approval",
"name": "Checkpoint.Approval",
"startTime": "2020-08-14T13:44:03.02Z",
"finishTime": null,
"currentOperation": null,
"percentComplete": null,
"state": "inProgress",
"result": null,
"resultCode": null,
"changeId": 72,
"lastModified": "0001-01-01T00:00:00",
"workerName": null,
"details": null,
"errorCount": 0,
"warningCount": 0,
"url": null,
"log": null,
"task": null,
"attempt": 1,
"identifier": "e54149c5-b5a7-4b82-8468-56ad493224b5"
}
],
"lastChangedBy": "00000002-0000-8888-8000-000000000000",
"lastChangedOn": "2020-08-14T13:44:03.057Z",
"id": "86fb4204-9c5e-4e72-bdb1-eefe230480ec",
"changeId": 73,
"url": "https://dev.azure.com/***"
}
below you can see a step that is called "Checkpoint.Approval". The id of that step IS the approval Id you need to approve everything. If you want to know from which stage the approval is, then you can follow up the parentIds until the parentId property is null.
This will then be the stage.
With this you can successfully get the approval id and use it to approve with the said
What jessehouwing's guess is correct. Now multi-stage still be in preview, and the corresponding SDK/API/extension hasn't been expanded and provided to public.
You may think that what about not using API. I have checked the corresponding code from our backend, all of operations to multi-stage approval contain one required parameter: approvalId. I'm sure you have known that this value is unique and different approval map with different approvalId value. This means, no matter which method you want to try with, approvalId is the big trouble. And based on my known, until now, there's no any api/SDK, third tool or extension can achieve this value directly.
In addition, for multi-stage YAML, its release process logic is not same with the release that defined with UI. So, all of public APIs which can work with release(UI), are not suitable with the release of multi-stage.
We have one undisclosed api, can get Approval message of multi-stage:
https://dev.azure.com/{org}/{project}/_apis/pipelines/approvals/{approvalId}
You can try with listing approval without specifying approvalId: https://dev.azure.com/{org}/{project}/_apis/pipelines/approvals. And its response message: Query for approvals failed. A minimum of one query parameter is required.\r\nParameter name: queryParameters. This represents you must tell system the specified approval(the big trouble I mentioned previously).
In fact, for why approvalId is a necessary part, it is caused from our backend code structure. I'd suggest you raise suggestion on developing API/SDK for multi-stage here.
I can confirm that Sebastian's answer worked for me, even in Azure DevOps 2020 on-prem.
After retrieving the approvalId from either methods used above (I was specifically using a service hook for my integration), I used the following API PATCH call:
https://dev.azure.com/{organization}/{project}/_apis/pipelines/approvals/?api-version=6.0-preview
and in the body:
[
{
"approvalId": "{approvalId}",
"status": {status integer}, (4 - approved; 8 - rejected)
"comment": ""
}
]
The call is passed with the application/json Content-Type, but in some situations it did not like that I was using the [] brackets, so you will need to work around that, only then will the call work.
I was even able to integrate this call into my custom connector in MS Power Automate
I added support to the latest version of the AzurePipelinesPS Powershell module to support bulk pipeline approvals.
Code snippet without using the AzurePipelinesPS sessions
$instance = 'https://dev.azure.com'
$collection = 'your_project'
$project = 'your_project'
$apiVersion = '5.1-preview'
$securePat = 'your_personal_access_token' | ConvertTo-SecureString -Force -AsPlainText
Get-APPipelinePendingApprovalList -Instance $instance -Collection $collection -Project $project -PersonalAccessToken $securePat -ApiVersion $apiVersion | Out-GridView -Passthru | % { Update-APPipelineApproval -Instance $instance -Collection $collection -Project $project -PersonalAccessToken $securePat -ApiVersion $apiVersion -ApprovalId $PSitem.approvalId -status 'approved'}
Code snippet with AzurePipelinesPS sessions
$session = 'your_session'
Get-APPipelinePendingApprovalList $session | Out-GridView -Passthru | % { Update-APPipelineApproval $session -ApprovalId $PSitem.approvalId -status 'approved'}
See the AzurePipelinesPS project page for details on secure session handling.
Function Definitions used in the code above
Get-APPipelinePendingApprovalList
Loops through pipeline build runs with the status of 'notStarted' or 'inProgress' in a project. This build lookup supports filters like pipeline definition ids or a source branch name.
For each build it then looks up the timeline record where the approval id, the stage name and the stage identifier are found.
Optionally with the ExpandApproval switch it can expand each approval with details
The object returned from this function contains the following properties, the values have been mocked
pipelineDefinitionName : MyPipeline
pipelineDefinitionId : 100
pipelineRunId : 2001
pipelineUrl : https://dev.azure.com/your_project/_build/results?
sourceBranch : refs/heads/master
stageName : Prod Deployment
stageIdentifier : Prod_Deployment
approvalId : xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx
Out-GridView
Displays data in a Grid View where the results can be filtered, ordered and selected.
%
The percent sign is shorthand for Foreach-Object
Update-APPipelineApproval
Updates the status of an approval to approved or rejected.
Credit
Thanks to Sebastian Schütze for cracking the timeline part!
The az pipelines extension doesn't suport approvals yet, I suppose due to the fact that multi-stage pipelines are still in preview and the old release hub will eventually be replaced by it.
But there is a REST API you can use to list and update approvals. These can be called from PowerShell with relative ease.
Or use the vsteam powershell module and Get-VSTeamApproval and Set-VSTeamApproval.
I have Azure function written in powershell and processing seems to be stopping for no apparent reason with timeout. Message never gets even to start being processed. This does not seem to be code related since code never even have a change to start since the first line of code is Write-Host "PowerShell queue trigger function processed work item: $QueueItem" which does not appear in log
Host.json
{
"version": "2.0",
"functionTimeout": "00:04:00",
"managedDependency": {
"enabled": true
},
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[1.*, 2.0.0)"
},
"extensions": {
"queues": {
"maxPollingInterval": "00:00:02",
"visibilityTimeout": "00:00:30",
"batchSize": 16,
"maxDequeueCount": 2,
"newBatchThreshold": 8
}
}
}
Function.json
{
"bindings": [
{
"name": "QueueItem",
"type": "queueTrigger",
"direction": "in",
"queueName": "metadataservicequeue",
"connection": "useastbootdiag_STORAGE"
}
]
}
Script
# Input bindings are passed in via param block.
param([string] $QueueItem, $TriggerMetadata)
# Write out the queue message and insertion time to the information log.
Write-Host "PowerShell queue trigger function processed work item: $QueueItem"
Write-Host "Queue item insertion time: $($TriggerMetadata.InsertionTime)"
Write-Host "Starting executing Invoke-AzureRMCommand with parameters $($TriggerMetadata["VMName"]), $($TriggerMetadata["ResourceGroup"])"
$return = Invoke-AzureCommand -vmName $TriggerMetadata["VMName"] -resourceGroup $TriggerMetadata["ResourceGroup"]
Write-Host "Finished executing Invoke-AzureRMCommand with parameters $($TriggerMetadata["VMName"]), $($TriggerMetadata["ResourceGroup"])"
$json = #"
[
{
"Return" : $($return | convertto-json),
"VMName" : "$($TriggerMetadata["VMName"])",
"ResourceGroup" : "$($TriggerMetadata["ResourceGroup"])"
}
]
"#
Write-Host "Outputing following to Log Analytics $json"
Post-LogAnalyticsData -customerId $env:CustomerID -sharedKey $env:SharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($json)) -logType "MetaDataLog"
Powershell Azure Functions by default are not scaling well on consumption plan and poorely scale on dedicated plan due to default settings. If you function is not CPU bound and expected to scale well due to high number of queue items. You'd need to modify FUNCTIONS_WORKER_PROCESS_COUNT (https://learn.microsoft.com/en-us/azure/azure-functions/functions-app-settings) to higher number then default 1 and increase value of PSWorkerInProcConcurrencyUpperBound (https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-powershell#concurrency) to higher number of default 1