PowerShell - Rename duplicate filenames in an object array - powershell

If I have some JSON like this:
$inputJson = #"
{
"attachments" :
[
{
"name": "attachment.eml",
"attachment_url": "https://www.attachment1.com"
},
{
"name": "attachment.eml",
"attachment_url": "https://www.attachment2.com"
},
{
"name": "attachment.eml",
"attachment_url": "https://www.attachment3.com"
}
]
}
where there is an attachments array and each element in that array has the same name but different URLs, how can I change all the names so that it outputs like this:
$outputJson = #"
{
"attachments" :
[
{
"name": "(attachment.eml)[1].eml",
"attachment_url": "https://www.attachment1.com"
},
{
"name": "(attachment.eml)[2].eml",
"attachment_url": "https://www.attachment2.com"
},
{
"name": "(attachment.eml)[3].eml",
"attachment_url": "https://www.attachment3.com"
}
]
}
"#
which renames each attachment to prevent duplicate names, but it preserves the URLs.
Ideally, the code would also make sure none of the new names already exist in the array as well. So if there is already an attachment named (attachment.eml)[1].eml in the array, it would handle that as well.
I've thought about somehow using Group-Object but I haven't quite figured out how to make that work.

Requesting for code is considered off-topic here. However, for self-training purposes, the following code snippet could do the job. Partially commented and itemized to (auxiliary) intermediate variables:
$inputJson = #"
{
"attachments" :
[
{
"name": "(attachment.eml)[2].eml",
"attachment_url": "https://www.attachment222.com"
},
{
"name": "attachmentOne.eml",
"attachment_url": "https://www.attachmentOne.com"
},
{
"name": "attachment.eml",
"attachment_url": "https://www.attachment1.com"
},
{
"name": "attachment.eml",
"attachment_url": "https://www.attachment2.com"
},
{
"name": "attachment.eml",
"attachment_url": "https://www.attachment3.com"
},
{
"name": "attachmnt.eml",
"attachment_url": "https://www.attachmnt1.com"
},
{
"name": "attachmnt.eml",
"attachment_url": "https://www.attachmnt2.com"
}
]
}
"#
$objectJson = $inputJson | ConvertFrom-Json
$AttachsGrouped = $objectJson.attachments | Group-Object -Property Name
$newAttachments = $AttachsGrouped | ForEach-Object {
# debug output
write-host $_.Count, $_.Group.GetType().Name, $_.name -ForegroundColor Cyan
if ( $_.Count -gt 1 ) {
$addCnt = 1
$( for ( $i = 0; $i -lt $_.Count; $i++ )
{
# make sure none of the new names already exist in the array
While ( "($($_.name))[$($i+$addCnt)].eml" -in
$objectJson.attachments.name ) { $addCnt++ }
[PSCustomObject]#{
'name' = "($($_.name))[$($i+$addCnt)].eml"
'attachment_url' = $_.Group[$i].attachment_url
}
}
)
} else {
# retain original definition
$_.Group[0]
}
}
$outputJson = [PSCustomObject]#{
# for better output readability
'attachments' = $newAttachments | Sort-Object -Property Name
}
# result
$outputJson # | ConvertTo-Json -Depth 2
Result:
$outputJson
attachments
-----------
{#{name=(attachment.eml)[1].eml; attachment_url=https://www.attachment1.com},...
$outputJson.Attachments
name attachment_url
---- --------------
(attachment.eml)[1].eml https://www.attachment1.com
(attachment.eml)[2].eml https://www.attachment222.com
(attachment.eml)[3].eml https://www.attachment2.com
(attachment.eml)[4].eml https://www.attachment3.com
(attachmnt.eml)[1].eml https://www.attachmnt1.com
(attachmnt.eml)[2].eml https://www.attachmnt2.com
attachmentOne.eml https://www.attachmentOne.com

I think that the following should work:
uniques = []
new_attachments = []
attachments.forEach( a=>
{ var u = uniques.find( u => u.name == a.name);
if (u == undefined) {
uniques.push({name: a.name, count: 1});
u = uniques.find(u => u.name == a.name)
}else{
u.count++
}
new_attachments.push( "(" + a.name.slice(0,-4) + ")[" + u.count + "].eml" )
} )
You would have to change the last line from:
new_attachments.push( "(" + a.name.slice(0,-4) + ")[" + u.count + "].eml" )
to:
a.name = "(" + a.name.slice(0,-4) + ")[" + u.count + "].eml"

Related

Converting Json Table

I am trying to convert below Json table to Json array object, below json failed because of there is single nested json object, but its working when there are multiple objects in nested json
[
{
"name": "PrimaryResult",
"columns": [
{
"name": "Computer",
"type": "string"
},
{
"name": "LastHeartbeat",
"type": "datetime"
}
],
"rows": [
[
"xxxxxxx.dev.org",
"2022-01-19T04:49:48.937Z"
]
]
}
]
Expected output is as below
[
{
"Computer": xxxxxxx.dev.org",
"LastHeartbeat": "2022-01-19T04:49:48.937Z"
}
]
I have tried with the below Json script it works when there are multiple objects in array
[
{
"name": "PrimaryResult",
"columns": [
{
"name": "Computer",
"type": "string"
},
{
"name": "LastHeartbeat",
"type": "datetime"
}
],
"rows": [
[
"zzzzz.test.org",
"2022-01-04T09:06:45.44Z"
],
[
"yyyy.dev.org",
"2022-01-04T09:06:30.233Z"
],
[
"xxxx.prod.org",
"2022-01-04T09:06:08.893Z"
],
[
"xxxx.dev.org",
"2022-01-04T09:06:04.667Z"
]
]
}
]
I have tried with below powershell script
=============================================
$TriggerMetadata = Get-Content .\test4.json | ConvertFrom-Json
$affected_resources = $TriggerMetadata.tables
$resources =
ForEach($Row in $affected_resources.rows)
{
$TmpHash = [Ordered]#{}
For($i = 0; $i -lt $Row.Length; $i++ )
{
$TmpHash.Add($affected_resources.columns.name[$i], $Row[$i] )
}
[PSCustomObject]$TmpHash
}
$body = $resources | ConvertTo-Json -Depth 100
$Body
Getting below error
Exception calling "Add" with "2" argument(s): "Key cannot be null.
Parameter name: key"
At line:22 char:13
+ $TmpHash.Add($affected_resources.columns.name[$i], $Row[$ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : ArgumentNullException
PowerShell 'unravels' arrays when there is only one element in it.
In this case that produces the error you encounter you can fix by using the , unary comma operator which will wrap the single element of the input array into another single element array.
When PowerShell unwraps that, the result is the original array of row values from the json file
Assuming you have the JSON converted in a variable $json:
$headers = $json.columns.name
# if there is only one 'rows 'element, wrap it in an array by prefixing with comma
if ($json.rows[0] -is [array]) { $rows = $json.rows } else { $rows = ,$json.rows }
$resources = $rows | ForEach-Object {
$hash = [ordered]#{}
for ($i = 0; $i -lt $headers.Count; $i++) {
$hash[$headers[$i]] = $_[$i]
}
[PsCustomObject]$hash
}
Output:
{
"Computer": "zzzzz.test.org",
"LastHeartbeat": "2022-01-04T09:06:45.44Z"
}
If you really do need the square brackets around that, you can do the test as before and enclose the result in '[..]'
if ($json.rows[0] -is [array]) {
$body = $resources | ConvertTo-Json -Depth 100
}
else {
$body = "[ {0} ]" -f ($resources | ConvertTo-Json -Depth 100)
}
Output:
[ {
"Computer": "zzzzz.test.org",
"LastHeartbeat": "2022-01-04T09:06:45.44Z"
} ]

how to validation JSON as key from String param in power shell

I want to validate from PowerShell using this JSON
{
"Customer": {
"OnSubscription": true,
"Name": "John",
"Surname": "Smith"
},
"Data": {
"Basket": [
"apples",
"pears",
"oranges",
"strawberries"
]
}
}
here is my script
$json = Get-Content .\basket.json | ConvertFrom-Json
param(
[string[]]$param = "Customer"
)
Write-Host $param
if ($param -eq $json) {
Write-Host "Valid Customer Key"}
else
{
$message = "Invalid Customer Key"
Write-Host $message
}
$output = ($json.$param.PSObject.Properties.Name | ForEach-Object {
"$_={0}" -f $json.$param.$_
}) -join "`n"
$Path2 = ".\output\env"
$output | Out-File $Path2
The parameter create for checking the JSON, is there a valid customer key or not?
For example, input param ABC, but in JSON there is no ABC. So its showing message like "Invalid Customer Key". When condition is valid customer key, Its showing message like "Valid Customer Key"
but the result I always got "Invalid Customer Key"
Is something wrong with if condition?
You might want to clean your code up a bit and dont think with such complexity. Your Task is quite easy, I have mocked something up for you.
loadMe.Json
{
"content":[
{
"Customer":{
"OnSubscription":true,
"Name":"John",
"Surname":"Smith"
},
"Data":{
"Basket":[
"apples",
"pears",
"oranges",
"strawberries"
]
}
},
{
"Customer":{
"OnSubscription":true,
"Name":"John2",
"Surname":"Smith"
},
"Data":{
"Basket":[
"apples",
"pears",
"oranges",
"strawberries"
]
}
},
{
"Customer":{
"OnSubscription":true,
"Name":"John3",
"Surname":"Smith"
},
"Data":{
"Basket":[
"apples",
"pears",
"oranges",
"strawberries"
]
}
}
]
}
Step One - Get Content of Json into an Object
$myJson = Get-Content -Path (Join-Path $PSScriptRoot '\loadMe.json') | ConvertFrom-json
Step Two - Loop through content and match names
foreach($item in $myJson.content) {
if($item.Customer.Name -eq $myOtherVariable) {
...
}
}
The following code gives the wanted output, task accomplished
$myJson = Get-Content -Path (Join-Path $PSScriptRoot '\loadMe.json') | ConvertFrom-json
foreach($item in $myJson.content) {
Write-Host $item.Customer.Name
}
Output -->
John
John2
John3

How to Iterate through HTTP JSON Response when structure is not known using PowerShell?

I am trying to iterate through some JSON when the exact structure is unknown.
This JSON looks like this:
"object" : {
"Item1" : {
"property1" : "a"
"property2" : "b"
}
}
"Item2" : {
"property1" : "c"
"property2" : "d"
}
}
}
The problem is, I don't know what the actual name of Item1 or Item2 is going to be. It is a string of alphanumeric characters that is different on each call.
I have tried
$json_response = $response.object
foreach($item in $json_response) {
$id = $item.property1
Write-Host $id
}
However the $id value never gets set to the value of proprty1. The Write-Host always prints out an empty string.
If I simply do a
Write-Host $json_response
I get something like
#{Item1=; item2=}
I thought this may have been a hash table that would allow me to iterate through it using keys, but there is no Keys property.
Can anyone assist?
Update: Lee_Dailey's response has gotten me further, but still cannot access the properties. With Lee_Dailey's help, I have come up with the following:
foreach ($item in $response_json.PSobject.Properties) {
$json2 = $item | ConvertTo-Json
Write-Host $json2
}
This creates the following JSON
{
"Value": {
"property1": "a",
"property2": "b"
}
}
However I still cannot access property1. Doing
$id = json2.Value.property1
Write-Host $id
results in an empty value. How do I access the properties in the JSON in the $json2 variable?
Update 2. I think I got it working, but I don't know if this is correct or a hack. Updating the code from above to
$json2 = $item | ConvertTo-Json | ConvertFrom-Json
Seems to allow me to do
$id = $json2.Value.property1
I am not sure where "Value" came from. It is not part of the HTTP response. It also seems odd to convert to JSON and then convert it back, however that seems to expose the properties.
I'm guessing the json is supposed to be this. These badly structured objects are common. You can loop through properties of "object" using either .psobjects.properties or get-member. Also, in newer versions of powershell, you can convert json to hashtable and use the keys property.
{
"object": {
"Item1": {
"property1": "a",
"property2": "b"
}
},
"Item2": {
"property1": "c",
"property2": "d"
}
}
See also: Iterating through a JSON file PowerShell
In an ideal world, it would look like this. But it comes out more verbose.
{
"object": [
{
"name": "Item1",
"value": [
{
"name": "property1",
"value": "a"
},
{
"name": "property2",
"value": "b"
}
]
},
{
"name": "Item2",
"value": [
{
"name": "property1",
"value": "c"
},
{
"name": "property2",
"value": "d"
}
]
}
]
}
$a = cat file.json | convertfrom-json
$a.object
name value
---- -----
Item1 {#{name=property1; value=a}, #{name=property2; value=b}}
Item2 {#{name=property1; value=c}, #{name=property2; value=d}}
$a.object.value
name value
---- -----
property1 a
property2 b
property1 c
property2 d

Filter list for objects missing a property

I have a generic list of custom objects that look like this:
id : 1
displayName : server1.domain.tdl
autoProperties : { #{name = auto.bios_version; value = 6.00 }, #{name = auto.specialDevice; value = True} }
id : 2
displayName : server2.domain.tdl
autoProperties : { #{name = auto.bios_version; value = 6.00 } }
Some of them have the "auto.SpecialDevice" property and some do not. I am trying to filter out those that do NOT have "auto.SpecialDevice".
As a sample, this code gets to what I have:
$string = ' [
{
"id": 1,
"displayName": "server1.domain.tdl",
"autoProperties": [
{
"name": "auto.bios_version",
"value": "6.00"
},
{
"name": "auto.specialDevice",
"value": "True"
}
]
},
{
"id": 2,
"displayName": "server2.domain.tdl",
"autoProperties": [
{
"name": "auto.bios_version",
"value": "6.00"
}
]
}
]
'
$list = [System.Collections.Generic.List[PSObject]]::New()
$list.Add(($string | ConvertFrom-Json))
So, the objects are in a variable called, $list, then I have tried the following, which returns both devices:
$list | Where-Object { -Not $_.autoProperties['auto.specialDevice'] }
What is the right way to do this?
(adding list population code suggested by #ansgar wiechers and #john rees)
You can populate the list from that json like this:
$list = $string | ConvertFrom-Json
Once you have the list populated with these objects, the following will work:
$list | where {$_.Autoproperties.name -notcontains 'auto.specialDevice'}
That is because $_.AutoProperties.name is a list of all of the names in the autoproperties collection.

PowerShell hash table from multiple Excel columns and use further

I am reading few columns of an Excel file and storing the values in a hash. My objective it to use this hash further like
Hostname: $computer['server']['hostname'] # Hostname: host1
IP: $computer['server']['ip'] # IP: x.x.x.x
Environment: $computer['server']['Environment'] # Environment: production
Code snippet:
$computers = #{}
$computers['Server'] = #{}
$computers['Server']['Hostname'] = #()
$computers['Server']['Environment'] = #()
$computers['Server']['ip'] = #()
for ($startRow=2; $startRow -le $rowCount; $startRow++) {
$hostname = $workSheet.Cells.Item($startRow,2).Value()
$environment = $workSheet.Cells.Item($startRow,1).Value()
$pip = $workSheet.Cells.Item($startRow,4).Value()
$sip = $workSheet.Cells.Item($startRow,5).Value()
$computers['Server']['Hostname'] += $hostname
$computers['Server']['Environment'] += $environment
$computers['Server']['ip'] += $ip
}
foreach ($computer in $computers) {
foreach ($server in $computer['Server']) {
$myhost = $computer['Server']['Hostname']
$environ = $computers['Server']['Environment']
Write-Host "$myhost : $environ `n"
}
}
Actual output:
host1 host2 host3 host4 : prod dev prod stag
Expected output:
host1: prod
host2: dev
host3: prod
host4: stag
EDIT NOTE: I can always call and display the variables in first for loop itself while reading the Excel files but I also want to store them in a hash table for later usage.
You're getting that result because the data structure you created looks like this (using JSON notation):
{
"Server": {
"Hostname": [ "host1", "host2", "host3", "host4" ],
"Environment": [ "prod", "dev", "prod", "stag" ],
"IP": [ ... ]
}
}
when you actually want something like this:
{
"Server": [
{
"Hostname": "host1",
"Environment": "prod",
"IP": ...
},
{
"Hostname": "host2",
"Environment": "dev",
"IP": ...
},
{
"Hostname": "host3",
"Environment": "prod",
"IP": ...
},
{
"Hostname": "host4",
"Environment": "stag",
"IP": ...
}
]
}
To get the desired result you need to create an array of hashtables and assign that to the key "Server", or just make $computers an array if "Server" is your only key anyway:
$computers = #(for ($startRow=2; $startRow -le $rowCount; $startRow++) {
...
#{
'Hostname' = $hostname
'Environment' = $environment
'IP' = $ip
}
})
You can then enumerate the computers like this:
foreach ($computer in $computers) {
'{0}: {1}' -f $computer['Hostname', 'Environment']
}
Alternatively you could make $computers a hash of hashes
$computers = #{}
for ($startRow=2; $startRow -le $rowCount; $startRow++) {
...
$computers[$hostname] = #{
'Environment' = $environment
'IP' = $ip
}
})
and enumerate the hosts like this:
foreach ($computer in $computers.GetEnumerator()) {
'{0}: {1}' -f $computer.Key, $computer.Value['Environment']
}