How to speed up Function in powershell - powershell

I am getting object ids from function. where $objects has 1000 entries. Below is part of main script. Just to get object ids for 1000 OBJECTS running for hours. If $objects count under 10 it runs quick. Is there way I can speed up below function to process 1000 $objects ???
function get_nodes($obj){
if($obj.protectionSource.name -eq $objectName){
$global:_object_id = $obj.protectionSource.id
$global:object = $obj.protectionSource.name
#break
}
if($obj.name -eq $objectName){
$global:_object_id = $obj.id
$global:object = $obj.name
# break
}
if($obj.PSObject.Properties['nodes']){
foreach($node in $obj.nodes){
if($null -eq $global:_object_id){
get_nodes $node
}
}
}
if($obj.PSObject.Properties['applicationNodes']){
foreach($node in $obj.applicationNodes){
if($null -eq $global:_object_id){
get_nodes $node
}
}
}
}
foreach($source in $sources){
if($null -eq $global:_object_id){
get_nodes $source
}
}
return $global:_object_id
}
foreach($object in $objects){ ###2
# get object ID
$objectId = getObjectId $object
}
$sources may have 1000 or more entries. below is one entry. $objects are literally server names ( server1 , server2 etc).
PS C:\anil\scripts> $sources[0]|ConvertTo-Json
{
"nodes": [
{
"nodes": " ",
"protectionSource": "#{id=2; parentId=1; name=Datacenters; environment=kVMware; vmWareProtectionSource=}",
"protectedSourcesSummary": "",
"unprotectedSourcesSummary": "",
"logicalSize": 0
}
],
"protectionSource": {
"id": 1,
"name": "wpc1uv00vvc.ent.ad.ntrs.com",
"environment": "kVMware",
"vmWareProtectionSource": {
"type": "kVCenter",
"name": "wpc1uv00vvc.ent.ad.ntrs.com",
"id": "#{uuid=683FECB1-A27E-447A-8037-0E1B8DA4BE7A}",
"version": "VMware vCenter Server 6.0.0 build-7462484",
"isVmcEntity": false
}
},
"protectedSourcesSummary": [
{
"environment": "kVMware"
}
],
"unprotectedSourcesSummary": [
{
"environment": "kVMware",
"leavesCount": 24,
"totalLogicalSize": 1224065679360
}
],
"registrationInfo": {
"accessInfo": {
"id": 3953,
"version": 0,
"environment": "kVMware",
"endpoint": "wpc1uv00vvc"
},
"throttlingPolicy": {
"isEnabled": false,
"enforceMaxStreams": false
},
"registrationTimeUsecs": 1585846901734416,
"refreshTimeUsecs": 1670355941177325,
"refreshErrorMessage": "Failed to connect to wpc1uv00vvc. Error: SOAP 1.1 fault: SOAP-ENV:Client[no subcode] \"System error\" Detail: getadd
rinfo failed in tcp_connect(). Connectivity status: Connection to wpc1uv00vvc:443 resulted in : Failed to resolve server name with error: System error",
"authenticationStatus": "kFinished",
"username": "ent\\sys-poc-cohesity"
}
}
PS C:\anil\scripts>

Related

PowerShell FilterScript error with some JSON Files

Thanks to iRon earlier this week with his help via this question, he helped massively on a piece of work I have on the go at the moment.
In summary, we have an Azure CICD pipeline to deploy out Policies. we have a folder with over 200 JSON policy files and the CICD process brings them all together in 1 JSON file and deploy them out. The initial problem we found was the parameters required a leading extra "[" otherwise the process would fail. And this was highlighted in this article (if you search for [[).
Anyway, to the problem. iRons support helped greatly, but out of the 200 files we get a handful where we receive the following message:
Cannot bind argument to parameter 'FilterScript' because it is null.
The code:
Function supplied by iRon:
function Get-Node {
[CmdletBinding()][OutputType([Object[]])] param(
[ScriptBlock]$Where,
[Parameter(ValueFromPipeLine = $True, Mandatory = $True)]$InputObject,
[Int]$Depth = 10
)
process {
if ($_ -isnot [String] -and $Depth -gt 0) {
if ($_ -is [Collections.IDictionary]) {
if (& $Where) {$_}
$_.get_Values() | Get-Node -Where $Where -Depth ($Depth -1)
}
elseif ($_ -is [Collections.IEnumerable]) {
for ($i = 0; $i -lt $_.get_Count(); $i++) { $_[$i] | Get-Node -Where $Where -Depth ($Depth -1) }
}
elseif ($Nodes = $_.PSObject.Properties.Where{$_.MemberType -eq 'NoteProperty'}) {
$Nodes.ForEach{
if (& $Where) { $_ }
$_.Value | Get-Node -Where $Where -Depth ($Depth -1)
}
}
}
}
}
Where I call the above function (this code is in a loop per file):
$policyRule = #("notIn", "in", "value", "equals", "field", "effect", "like", "greaterOrEquals", "name", "resourceGroupName", "resourceGroup", "location", "storageName", "uniqueStorage", "storageContainerPath", "storageAccountAccessKey", "dependsOn", "targetResourceId", "storageId", "enabled")
if ($content.properties -ne $null){
# Loop all the Policy Rule names for fix the values within them
foreach ($rule in $policyRule){
$Node = $content.properties.policyRule | Get-Node -Where {$_.name -eq $rule -and $_.value -Match "^\[\w+" -and $_.value -ne ""}
$Node | ForEach-Object {$_.Value = '[' + $_.Value}
}
}
A JSON file that works:
{
"name": "POLICY-001",
"properties": {
"displayName": "Allowed Locations for Resources",
"policyType": "Custom",
"mode": "Indexed",
"description": "This policy enables you to restrict the locations your organization can specify when deploying resources. Use to enforce your geo-compliance requirements. Excludes resource groups, Microsoft.AzureActiveDirectory/b2cDirectories, and resources that use the 'global' region.",
"metadata": {
"version": "1.0.0",
"category": "General"
},
"parameters": {
"listOfAllowedLocations": {
"type": "Array",
"metadata": {
"description": "The list of locations that can be specified when deploying resources.",
"strongType": "location",
"displayName": "Allowed locations"
},
"allowedValues": [
"uksouth",
"ukwest"
],
"defaultValue": [
"uksouth",
"ukwest"
]
}
},
"policyRule": {
"if": {
"allOf": [
{
"field": "location",
"notIn": "[parameters('listOfAllowedLocations')]"
},
{
"field": "location",
"notEquals": "global"
},
{
"field": "type",
"notEquals": "Microsoft.Resources/subscriptions/resourceGroups"
},
{
"field": "type",
"notEquals": "Microsoft.Resources/b2cDirectories"
}
]
},
"then": {
"effect": "deny"
}
}
},
"excludedScopes": [
],
"nonComplianceMessage": "only allow resources to be deployed from UK South and UK West"
}
One that fails:
{
"Name": "Policy-106",
"Properties": {
"Description": "This policy automatically deploys and enable diagnostic settings to Log Analytics",
"DisplayName": "Apply diagnostic settings for Log Analytics Workspaces",
"Mode": "Indexed",
"policyType": "Custom",
"Metadata": {
"version": "1.0",
"category": "Monitoring"
},
"Parameters": {
"diagnosticsSettingNameToUse": {
"type": "string",
"metadata": {
"displayName": "Apply diagnostic settings for Log Analytics Workspaces - Setting name",
"description": "Name of the policy for the diagnostics settings."
},
"defaultValue": "setViaPolicy"
},
"logAnalytics": {
"type": "string",
"metadata": {
"displayName": "Apply diagnostic settings for Log Analytics Workspaces - Log Analytics workspace",
"description": "Select the Log Analytics workspace from dropdown list",
"strongType": "omsWorkspace",
"assignPermissions": true
},
"defaultValue": "/subscriptions/......"
}
},
"PolicyRule": {
"if": {
"field": "type",
"equals": "Microsoft.OperationalInsights/Workspaces"
},
"then": {
"effect": "deployIfNotExists",
"details": {
"type": "Microsoft.Insights/diagnosticSettings",
"roleDefinitionIds": [
"/providers/Microsoft.Authorization/roleDefinitions/123"
],
"existenceCondition": {
"allOf": [
{
"field": "Microsoft.Insights/diagnosticSettings/logs.enabled",
"equals": "True"
},
{
"field": "Microsoft.Insights/diagnosticSettings/metrics.enabled",
"equals": "True"
},
{
"field": "Microsoft.Insights/diagnosticSettings/workspaceId",
"matchInsensitively": "[parameters('logAnalytics')]"
}
]
},
"deployment": {
"properties": {
"mode": "incremental",
"template": {
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"diagnosticsSettingNameToUse": {
"type": "string"
},
"resourceName": {
"type": "string"
},
"logAnalytics": {
"type": "string"
},
"location": {
"type": "string"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.OperationalInsights/Workspaces/providers/diagnosticSettings",
"apiVersion": "2017-05-01-preview",
"name": "[concat(parameters('resourceName'), '/', 'Microsoft.Insights/', parameters('diagnosticsSettingNameToUse'))]",
"location": "[parameters('location')]",
"dependsOn": [],
"properties": {
"workspaceId": "[parameters('logAnalytics')]",
"metrics": [
{
"category": "AllMetrics",
"timeGrain": null,
"enabled": true,
"retentionPolicy": {
"enabled": false,
"days": 0
}
}
],
"logs": [
{
"category": "Audit",
"enabled": true
}
]
}
}
],
"outputs": {}
},
"parameters": {
"diagnosticsSettingNameToUse": {
"value": "[parameters('diagnosticsSettingNameToUse')]"
},
"logAnalytics": {
"value": "[parameters('logAnalytics')]"
},
"location": {
"value": "[field('location')]"
},
"resourceName": {
"value": "[field('name')]"
}
}
}
}
}
}
}
},
"excludedScopes": [
],
"nonComplianceMessage": ""
}
As mentioned in my original post, my PowerShell is basic at best and any support would be grateful.
The JSON file whose processing fails contains null values, which turn into $null values in PowerShell when converting the JSON file to an object graph via ConvertFrom-Json.
The Get-Node function as shown in your question doesn't support binding $null to its pipeline-binding -InputObject parameter, which results in an error whenever $_.Value happens to be $null in the following statement:
$_.Value | Get-Node -Where $Where -Depth ($Depth - 1)
The solution is simple: Allow the -InputObject parameter to accept $null, by adding the [AllowNull()] attribute to the original function:
function Get-Node {
[CmdletBinding()][OutputType([Object[]])] param(
[ScriptBlock]$Where,
[AllowNull()] # <- now $null may be passed too
[Parameter(ValueFromPipeLine = $True, Mandatory = $True)]$InputObject,
[Int]$Depth = 10
)
process {
if ($_ -isnot [String] -and $Depth -gt 0) {
if ($_ -is [Collections.IDictionary]) {
if (& $Where) { $_ }
$_.get_Values() | Get-Node -Where $Where -Depth ($Depth - 1)
}
elseif ($_ -is [Collections.IEnumerable]) {
for ($i = 0; $i -lt $_.get_Count(); $i++) { $_[$i] | Get-Node -Where $Where -Depth ($Depth - 1) }
}
elseif ($Nodes = $_.PSObject.Properties.Where{ $_.MemberType -eq 'NoteProperty' }) {
$Nodes.ForEach{
if (& $Where) { $_ }
$_.Value | Get-Node -Where $Where -Depth ($Depth - 1)
}
}
}
}
}

PowerShell Replace value in JSON

In our Azure CICD Pipeline, we have an element where we are trying to deploy Policies. We have JSON file per policy in the repo and we bring all these json files together into one file as part of CI which later is deployed via the CD. The PowerShell wasn't written by me, but a Microsoft consultant who was on site a few years back.
The problem is that when all the JSON comes together, we get an illegal syntax e.g.
Altering the code to this works and deploys, but means we have to go through all our files manually replace [ with [[:
In summary the PowerShell bring all of this together, does some manipulation and outputs to a file in the artifacts folder.
This is just a small snippet of the json, but highlights the area and there are many areas like this in the total json that need replacing:
{
"functions": [
],
"variables": {
"location": "UK South"
},
"resources": [{
"properties": {
"displayName": "Allowed Locations for Resources",
"policyType": "Custom",
"mode": "Indexed",
"description": "description.",
"metadata": {
"version": "1.0.0",
"category": "General"
},
"parameters": {
"listOfAllowedLocations": {
"type": "Array",
"metadata": {
"description": "The list of locations that can be specified when deploying resources.",
"strongType": "location",
"displayName": "Allowed locations"
},
"allowedValues": [
"uksouth",
"ukwest"
],
"defaultValue": [
"uksouth",
"ukwest"
]
}
},
"policyRule": {
"if": {
"allOf": [{
"field": "location",
"notIn": "[parameters('listOfAllowedLocations')]"
},
{
"field": "location",
"notEquals": "global"
},
{
"field": "type",
"notEquals": "Microsoft.Resources/subscriptions/resourceGroups"
},
{
"field": "type",
"notEquals": "Microsoft.Resources/b2cDirectories"
}
]
},
"then": {
"effect": "audit"
}
}
},
"name": "Policy1",
"apiVersion": "2019-01-01",
"type": "Microsoft.Authorization/policyDefinitions",
"location": "[variables('location')]"
}]
}
My PowerShell is intro level at best, so I am struggling to get a replace working.
I can obtain the offending area and replace it in a Write-Host, but I don't know how to write the back to the originating object with without making a right mess of things:
if ($content.properties.policyRule.if.allOf -ne $null){
foreach ($param in $content.properties.policyRule.if.allOf){
Write-Host "were here..................."
#$param = ($param | ConvertTo-Json -Depth 100 | % { [System.Text.RegularExpressions.Regex]::Unescape($_) })
if ($param.notIn -ne $null){
$param.notIn.replace('[', '[[')
Write-Host $param.notIn
}
}
Any suggestions would be grateful.
The point is that the allOf node contains an array. Due to the member-access enumeration feature you will be able to conveniently read the notIn property but to write to it, you will need to be specific on the index ([0]) in the allOf node:
$Data = ConvertFrom-Json $Json # $Json contains your $Json snippet
$Data.resources.properties.policyRule.if.allOf[0].notIn = "[[parameters('listOfAllowedLocations')]"
$Data |ConvertTo-Json -Depth 9
In case you want to recursively find your items based on e.g. a specific name and value format from a specific property level, you might use this common reusable function to recursively find (and replace) a node in a complex PowerShell object:
function Get-Node {
[CmdletBinding()][OutputType([Object[]])] param(
[ScriptBlock]$Where,
[AllowNull()][Parameter(ValueFromPipeLine = $True, Mandatory = $True)]$InputObject,
[Int]$Depth = 10
)
process {
if ($_ -isnot [String] -and $Depth -gt 0) {
if ($_ -is [Collections.IDictionary]) {
if (& $Where) { $_ }
$_.get_Values() | Get-Node -Where $Where -Depth ($Depth - 1)
}
elseif ($_ -is [Collections.IEnumerable]) {
for ($i = 0; $i -lt $_.get_Count(); $i++) { $_[$i] | Get-Node -Where $Where -Depth ($Depth - 1) }
}
elseif ($Nodes = $_.PSObject.Properties.Where{ $_.MemberType -eq 'NoteProperty' }) {
$Nodes.ForEach{
if (& $Where) { $_ }
$_.Value | Get-Node -Where $Where -Depth ($Depth - 1)
}
}
}
}
}
Usage
Finding node(s) with a specific name and value (-format):
$Node = $Data.resources.properties.policyRule.if |Get-Node -Where {
$_.name -eq 'notIn' -and $_.value -Match "^\[\w+\('\w+'\)\]$"
}
$Node
Value : [parameters('listOfAllowedLocations')]
MemberType : NoteProperty
IsSettable : True
IsGettable : True
TypeNameOfValue : System.String
Name : notIn
IsInstance : True
Replacing the value of the found node(s):
$Node |ForEach-Object {
$_.Value = '[' + $_.Value
}
$Data |ConvertTo-Json -Depth 9
Results
{
"functions": [],
"variables": {
"location": "UK South"
},
"resources": [
{
"properties": {
"displayName": "Allowed Locations for Resources",
"policyType": "Custom",
"mode": "Indexed",
"description": "description.",
"metadata": {
"version": "1.0.0",
"category": "General"
},
"parameters": {
"listOfAllowedLocations": {
"type": "Array",
"metadata": {
"description": "The list of locations that can be specified when deploying resources.",
"strongType": "location",
"displayName": "Allowed locations"
},
"allowedValues": [
"uksouth",
"ukwest"
],
"defaultValue": [
"uksouth",
"ukwest"
]
}
},
"policyRule": {
"if": {
"allOf": [
{
"field": "location",
"notIn": "[[parameters('listOfAllowedLocations')]"
},
{
"field": "location",
"notEquals": "global"
},
{
"field": "type",
"notEquals": "Microsoft.Resources/subscriptions/resourceGroups"
},
{
"field": "type",
"notEquals": "Microsoft.Resources/b2cDirectories"
}
]
},
"then": {
"effect": "audit"
}
}
},
"name": "Policy1",
"apiVersion": "2019-01-01",
"type": "Microsoft.Authorization/policyDefinitions",
"location": "[variables('location')]"
}
]
}
Update 2022-11-21
Resolved an issue with $Null values in the Get-Node function, see also: PowerShell FilterScript error with some JSON Files (thanks
mklement0).

ConvertFrom-Json : A parameter cannot be found that matches parameter name 'Depth' [duplicate]

can someone please tell me what I am doing wrong here? I am trying to add additional properties to an existing JSON file.
This is my existing JSON File:
{
"filePath": "F:\\Program Files (x86)\\Common Files\\XXXX\\Services\\file.exe",
"Services": {
"BBBBBB": {
"Id": "BBBBBB",
"Installed": true,
"OverrideValues": {}
},
"XXXX1Service": {
"Id": "XXXX1Service",
"Installed": true,
"OverrideValues": {
"HostName": {
"Enabled": true,
"Value": "my.com"
}
}
},
"XXXX2Service": {
"Id": "XXXX2Service",
"Installed": true,
"OverrideValues": {
"Application Tier URL": {
"Value": "my.com"
},
"HostName": {
"Enabled": true,
"Value": "my.com"
}
}
},
"XXXX3Service": {
"Id": "XXXX3Service",
"Installed": true,
"OverrideValues": {
"HostName": {
"Enabled": true,
"Value": "my.com"
}
}
},
"XXXX4Service": {
"Id": "XXXX4Service",
"Installed": true,
"OverrideValues": {
"HostName": {
"Enabled": true,
"Value": "my.com"
}
}
}
}
}
I am trying to insert properties to make it look like this (look under XXX3Service):
{
"filePath": "F:\\Program Files (x86)\\Common Files\\XXXX\\Services\\file.exe",
"Services": {
"BBBBBB": {
"Id": "BBBBBB",
"Installed": true,
"OverrideValues": {}
},
"XXXX1Service": {
"Id": "XXXX1Service",
"Installed": true,
"OverrideValues": {
"HostName": {
"Enabled": true,
"Value": "my.com"
}
}
},
"XXXX2Service": {
"Id": "XXXX2Service",
"Installed": true,
"OverrideValues": {
"Application Tier URL": {
"Value": "my.com"
},
"HostName": {
"Enabled": true,
"Value": "my.com"
}
}
},
"XXXX3Service": {
"Id": "XXXX3Service",
"Installed": true,
"OverrideValues": {
"HostName": {
"Enabled": true,
"Value": "my.com"
},
"XXX 3 Service": {
"Enabled": true
},
"My Token": {
"Value": "1234567890"
},
"Authority": {
"Value": "authority.com"
},
"ThisSecret": {
"Value": "1234-1234-1234-1234"
}
}
},
"XXXX4Service": {
"Id": "XXXX4Service",
"Installed": true,
"OverrideValues": {
"HostName": {
"Enabled": true,
"Value": "my.com"
}
}
}
}
}
My Code looks like this:
# Get json content from a file
$json = Get-Content .\source.json
# convert the json to be a powershell object
$psObject = $json | ConvertFrom-Json
# useful for referencing to see all of the properties
# $psObject.Services | Select-Object *
# $psObject.Services.XXXX3Service
# $psObject.Services.XXXX3Service.OverrideValues
#* ConvertTo-Json the max allowed depth is 100
$newProperty = #"
{
"OverrideValues": {
"HostName": {
"Enabled": true,
"Value": "my.com"
},
"XXX 3 Service": {
"Enabled": true
},
"My Token": {
"Value": "1234567890"
},
"Authority": {
"Value": "authority.com"
},
"ThisSecret": {
"Value": "1234-1234-1234-1234"
}
}
}
"# | ConvertFrom-Json -Depth 100
# get the name to overwrite the existing property so it's not nested
$propName = $newProperty.PSObject.Properties | Select-Object -Expand Name
# here we supply the $propname value in json as well as use the value(s) under the $newProperty to overwrite the existing property
$psObject.Services.XXXX3Service | Add-Member -MemberType NoteProperty -Name $propName -Value $newProperty.$propName -Force
# lastly convert and export the json
$psObject | ConvertTo-Json -Depth 100 | Out-File "C:\temp\good.json"
however I am getting the following errors:
ConvertFrom-Json : A parameter cannot be found that matches parameter name 'Depth'.
At C:\Users\user\Downloads\json.ps1:38 char:23
+ "# | ConvertFrom-Json -Depth 100
+ ~~~~~~
+ CategoryInfo : InvalidArgument: (:) [ConvertFrom-Json], ParameterBindingException
+ FullyQualifiedErrorId : NamedParameterNotFound,Microsoft.PowerShell.Commands.ConvertFromJsonCommand
Add-Member : Cannot bind argument to parameter 'Name' because it is null.
At C:\Users\user\Downloads\json.ps1:44 char:77
+ ... Service | Add-Member -MemberType NoteProperty -Name $propName -Value ...
+ ~~~~~~~~~
+ CategoryInfo : InvalidData: (:) [Add-Member], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.AddMemberCo
mmand
tl;dr
Remove -Depth 100 from your ConvertFrom-Json call.
With ConvertFrom-Json, use of -Depth is generally not needed, and in Windows PowerShell that parameter doesn't even exist (whereas it does for the complementary cmdlet, ConvertTo-Json, where it is often needed).
As Cpt.Whale points out, in Windows PowerShell the ConvertFrom-Json - note the From - does not have a -Depth parameter. It exists in PowerShell (Core) 7+, where, however, it is rarely needed.
Despite the parameter having the same name as ConvertTo-Json's (see below), it functions very differently: It is used to enforce that the input JSON isn't more deeply nested than the given -Depth value, and reports an error if it is.
As zett42 points out, the default value is 1024, so only more deeply nested JSON would cause an error - which seems unlikely. However, you can use -Depth to increase that limit, up to 2,147,483,647 ([int]::MaxValue), at least formally.
By contrast, ConvertTo-Json - note the To - does have this parameter in Windows PowerShell too, and, unfortunately, it is frequently needed:
By default, PowerShell limits the serialization depth to two levels of recursion, necessitating an explicit -Depth argument to properly serialize more deeply nested object graphs - see this post. Absent that, if the default serialization depth is exceeded, quiet truncation occurs in Windows PowerShell, whereas in PowerShell (Core) you now at least get a warning.

Powershell Iterate through multidimensional array of hashtables to find a match and combine values from both arrays

I need to combine values from 2 JSONs:
If there is a match in alerts IDs, I need to create structure, that will take data from both jsons
Result for a match should look like:
$array = #()
$hashtable = #{}
$hashtable.AlertID (does not matter what JSON is it from)
$hashtable.Tags (from JSON 1)
$hashtable.IncidentName (from JSON2)
$hashtable.IncidentID (from JSON2)
$array += $hashtable
I would prefer if this would be done with c style powershell loop.
c style for loop = for ($x = 0; $x -array.count; $x++)
JSON 1:
[
{
"Status": "Active",
"IncidentId": "3",
"tags": "SINC0008009",
"AlertId": [
"da637563185629568182_-638872186",
"da637563185631732095_1120592736",
"da637563185706412029_-614525914",
"da637563185760439486_-276692370",
"da637563185856325888_-1949235651",
"da637563186785996176_2128073884",
"da637563186789897000_1239551047",
"da637563186806513555_1512241399",
"da637563193194338043_-244132089"
],
"severity": "Medium"
},
{
"Status": "Active",
"IncidentId": "4",
"tags": "SINC0008008",
"AlertId": [
"da637643650725801726_1735022501",
"da637643650741237104_1473290917",
"da637643650748739479_-40211355",
"da637643652767933265_-1887823168",
"da637643670830160376_-443360743"
],
"severity": "Medium"
},
{
"Status": "Active",
"IncidentId": "2",
"tags": null,
"AlertId": [
"caD76232A5-F386-3C5D-94CD-7C82A7F778DC"
],
"severity": "Medium"
},
{
"Status": "Active",
"IncidentId": "1",
"tags": null,
"AlertId": [
"ca6534FF45-D62A-3FB7-BD6B-FF5029C553DB"
],
"severity": "Medium"
}
]
JSON 2:
{
"value": [
{
"incidentId": 3,
"incidentName": "Multi-stage incident involving Initial access & Discovery on one endpoint",
"status": "Active",
"severity": "Medium",
"tags": ["SINC0000001"],
"comments": [],
"alerts": [
{
"alertId": "da637563185629568182_-638872186",
"incidentId": 3,
"description": "A suspicious PowerShell activity was observed on the machine. ",
"status": "New",
"severity": "Medium",
"devices": [
{
"deviceDnsName": "xxxxx"
}
],
"entities": [
{
"entityType": "User",
"accountName": "xxxxxx",
"userPrincipalName": "xxx#xx.xx"
},
{
"entityType": "Process"
},
{
"entityType": "Process",
"verdict": "Suspicious"
},
{
"entityType": "File"
}
]
},
{
"alertId": "da637563185631732095_1120592736",
"incidentId": 3,
"devices": [
{
"osPlatform": "Windows10",
"version": "1909"
}
],
"entities": [
{
"entityType": "User",
"remediationStatus": "None"
}
]
}
]
},
{
"incidentId": 4,
"incidentName": "Multi-stage incident involving Initial access & Discovery on one endpoint",
"status": "Active",
"severity": "Medium",
"tags": ["SINC0000002"],
"comments": [],
"alerts": [
{
"alertId": "da637563185629568182_-638872186",
"incidentId": 3,
"description": "A suspicious PowerShell activity was observed on the machine. ",
"status": "New",
"severity": "Medium",
"devices": [
{
"deviceDnsName": "xxxxx"
}
],
"entities": [
{
"entityType": "User",
"accountName": "xxxxxx",
"userPrincipalName": "xxx#xx.xx"
},
{
"entityType": "Process"
},
{
"entityType": "Process",
"verdict": "Suspicious"
},
{
"entityType": "File"
}
]
},
{
"alertId": "da637563185631732095_1120592736",
"incidentId": 3,
"devices": [
{
"osPlatform": "Windows10",
"version": "1909"
}
],
"entities": [
{
"entityType": "User",
"remediationStatus": "None"
}
]
}
]
}
]
}
Till now, I was looking into using nested foreach loop to address it but it does not behave like I want. I am looking for for loop as I could use the indexes.
Instead of creating an array of Hashtables, I think it's better to create an array of PsCustomObjects, because outputting the result to console/file/json would be a lot easier then.
$json1 = Get-Content -Path 'X:\json1.json' -Raw | ConvertFrom-Json
$json2 = Get-Content -Path 'X:\json2.json' -Raw | ConvertFrom-Json
$result = foreach ($incident in $json1) {
foreach ($alertId in $incident.AlertId) {
$json2.value | Where-Object { $_.alerts.alertId -eq $alertId } | ForEach-Object {
# output an object with the wanted properties
[PsCustomObject]#{
AlertID = $alertId # from json1
Tags = $incident.Tags # from json1
IncidentName = $_.incidentName # from json2
IncidentID = $_.incidentId # from json2
}
}
}
}
# output on screen
$result | Format-Table -AutoSize # or use Out-GridView
# output to new JSON
$result | ConvertTo-Json
# output to CSV file
$result | Export-Csv -Path 'X:\incidents.csv' -NoTypeInformation
Using your examples, the output to console window is:
AlertID Tags IncidentName IncidentID
------- ---- ------------ ----------
da637563185629568182_-638872186 SINC0008009 Multi-stage incident involving Initial access & Discovery on one endpoint 3
da637563185629568182_-638872186 SINC0008009 Multi-stage incident involving Initial access & Discovery on one endpoint 4
da637563185631732095_1120592736 SINC0008009 Multi-stage incident involving Initial access & Discovery on one endpoint 3
da637563185631732095_1120592736 SINC0008009 Multi-stage incident involving Initial access & Discovery on one endpoint 4

delete a object from the json file using a powershell

In this Question i want to accomplish is that i am trying to delete a specific object in the json file.
But while doing so i am experiencing some difficulties i tried to refer article Iterate over JSON and remove JSON element in PowerShell
and implement the same but however it is deleting the entire element but i want to delete a specific object in the element not the entire element following are the required things
1. json file
{
"name": "JourneyPack",
"description": "Details of the journey across india",
"author": "Sachin",
"version": "1.0.0",
"main": "main.js",
"build": {
"applicationID": "desktop",
"Necessaryfiles": [
"main.js",
"package.json",
],
"Storage": {
"output": "./reunited"
},
"DeatilsOfJourney": [
{
"from": "../Pune",
"to": "../travel/Pune",
"filter": [
"**/*
]
},
{
"from": "../Delhi",
"to": "../travel/Delhi",
"filter": [
"**/*"
]
},
{
"from": "../Jharkhand",
"to": "../travel/Jharkhand",
"filter": [
"**/*"
]
},
],
"IOS": {
"category": "desktop"
},
"Windows": {
"icon": "images/desktopicons/icons.ico",
"target": [
"divfrieght"
],
"publisherName": [
"Sachin"
]
},
"divfrieght": {
"PointClick": true,
"standaloneMachine": true,
"allowrise": true,
"allowinstdir": true,
"menu": "JourneyPack"
}
},
"private": true,
}
following is the tried code again this is that i have referred from Iterate over JSON and remove JSON element in PowerShell
2. tried code
$inputFile = '<THE FULL PATH AND FILENAME TO YOUR JSON FILE>'
$outputFile = '<THE FULL PATH AND FILENAME FOR THE OUTPUT JSON FILE>'
$apijson = Get-Content -Path $inputFile -Raw | ConvertFrom-Json
# for safety, first make a copy of the original .paths object
$newPaths = $apijson.paths
foreach ($element in $newPaths.PSObject.Properties) {
$objName = $element.Name
$objValue = $element.Value
$objProperties = $objValue.PSObject.Properties
foreach ($prop in $objProperties) {
if ($prop.Value.'from' -eq 'Jharkhand') {
$propName = $prop.Name
$objProperties.Remove($propName)
Write-Host "Removed object $objName -- $propName"
}
}
}
# now overwrite the $apijson.paths with this cleaned up version
$apijson.paths = $newPaths
# I assume you want to convert it back to a .JSON file??
$apijson | ConvertTo-Json -Depth 100 | Set-Content -Path $outputFile -Force
i want to delete the object where "from" is equal to "../Jharkhand/"
Desired Output
{
"name": "JourneyPack",
"description": "Details of the journey across india",
"author": "Sachin",
"version": "1.0.0",
"main": "main.js",
"build": {
"applicationID": "desktop",
"Necessaryfiles": [
"main.js",
"package.json",
],
"Storage": {
"output": "./reunited"
},
"DeatilsOfJourney": [
{
"from": "../Pune",
"to": "../travel/Pune",
"filter": [
"**/*
]
},
{
"from": "../Delhi",
"to": "../travel/Delhi",
"filter": [
"**/*"
]
},
],
"IOS": {
"category": "desktop"
},
"Windows": {
"icon": "images/desktopicons/icons.ico",
"target": [
"divfrieght"
],
"publisherName": [
"Sachin"
]
},
"divfrieght": {
"PointClick": true,
"standaloneMachine": true,
"allowrise": true,
"allowinstdir": true,
"menu": "JourneyPack"
}
},
"private": true,
}
if anyone could help that would be really helpful
".paths" property does not belong to you json file, so removed this part of your script.
# for safety, first make a copy of the original .paths object
$newPaths = $apijson.paths
Try this code:
$inputFile = 'input.json'
$outputFile = 'output.json'
$apijson = Get-Content -Path $inputFile -Raw | ConvertFrom-Json
foreach ($element in $apijson.PSObject.Properties) {
$objName = $element.Name
$objValue = $element.Value
$objProperties = $objValue.PSObject.Properties
foreach ($prop in $objProperties) {
# Your object lies in this array
if ($prop.Name -eq 'DeatilsOfJourney') {
[System.Collections.ArrayList]$arr = $prop.Value
#Iterate over your array and find that object which you want to remove
for ($i = 0; $i -lt $arr.count; $i++) {
if ($arr[$i].'from' -eq '../Jharkhand')
{
$arr.RemoveAt($i)
$i--
}
}
$prop.Value = $arr
}
}
}
$apijson | ConvertTo-Json -Depth 100 | Set-Content -Path $outputFile -Force