JSON Nested Array to CSV - powershell

I have a Json file that I need to convert to CSV file using Powershell.
Can somebody please help me out?
The input file containing the Json data is called "data.json" and I would like the CSV output file to be called "dataout.csv"
Thanks for any help you can provide.
Cheers

As per comment:
If I understand your question correctly, you want to $usagetype for each $productName.lines and you want to do that for each $cloudaccount.lines...
This means that the foreach ( $usagetype in $productName.lines ) { ... loop should go inside the foreach ( $productName in $cloudaccount.lines ) { ... loop and the output [pscustomobject] #{ should go inside that inner loop: foreach ( $productName in $cloudaccount.lines ) { ...
In other words, if you need to read a JSON Nested Array, you will need to nest your loops. Thus:
$Data = $json | ConvertFrom-Json
$output = foreach ( $customer in $data ) {
$customerName = "$($customer.entity.company) ($($customer.entity.name))"
foreach ( $cloudAccount in $customer.lines ) {
$cloudAccountNumber = $cloudAccount.entity.id
# Continue to nest down to get out all colums data
foreach ( $productName in $cloudaccount.lines ) {
$cloudproductname = $productName.entity.id
foreach ( $usagetype in $productName.lines ) {
$cloudusagetype = $usagetype.entity.id
$cloudprice = $usagetype.data.price
$cloudcost = $usagetype.data.cost
$cloudusage = $usagetype.data.usage
# output the result
[pscustomobject] #{
"Customer Name" = $customerName
"Cloud Account Number" = $cloudAccountNumber
"Product Name" = $cloudproductname
"Usage Type" = $cloudusagetype
"Price" = $cloudprice
"Cost" = $cloudcost
"Usage" = $cloudusage
}
}
}
}
}
# Convert to csv
$output | Export-Csv -Path myfil.csv

Related

Adding a column to a datatable in powershell

I am trying to add a column to data I have imported (and will export) as a CSV.
I am importing a CSV:
What I want to do add another column, perhaps "10/15/22" when the process runs, and then update the values under that date.
In effect, the document will grow to the right, adding a column each time it is run.
I have an object called $test. It will have values like:
$test.users = "7"
$test.SomeBSValue = "22"
$test.Computers = "52"
When all is said and done, my output should look like:
Adding to the list any values I have that are not part of the list already, but recording the values I have under the heading for the date.
So, if the script is run and collects 100 data points, those data point would all be in the CSV under the date.
I would have thought this would be easy, but now I am drawing a complete blank.
I've considered (but have not coded) even trying to put into a GUI grid view and then reading the data back and writing the CSV (but there should be an easier way, right?)
Since you don't actually use it as a CSV we can treat it like regular content.
Say we have a file in C:\test called test.csv that looks as follows:
"Settings","08/15/22","09/15/22"
"Users",0,0
"Computers",0,1
"SomeValue1",0,2
"SomeValue2",0,2
"SomeValue3",0,2
"Stat1",0,10
"Stat2",7,0
"Stat3",0,0
"SomeBSValue",1,2
We can import it, add the row from the object to each corresponding row and right the file to test2.csv.
$test = #{
Settings = "10/15/22"
users = "7"
Computers = "52"
SomeValue1 = "22"
SomeValue2 = "24"
SomeValue3 = "25"
Stat1 = "4"
Stat2 = "3"
Stat3 = "2"
SomeBSValue = "1"
}
$content = Get-Content "C:\test\test.csv"
$newContent = #()
foreach($row in $content){
foreach($key in $test.Keys){
if($row -like "*$key*"){
$row = $row + "," + $test."$key"
$newContent += $row
}
}
}
$newContent | Out-File "C:\test\test2.csv"
After running the script it will have added the values from the object:
"Settings","08/15/22","09/15/22",10/15/22
"Users",0,0,7
"Computers",0,1,52
"SomeValue1",0,2,22
"SomeValue2",0,2,22
"SomeValue3",0,2,22
"Stat1",0,10,4
"Stat2",7,0,4
"Stat3",0,0,4
Edit:
If you want the date between quotes, replace $row = $row + "," + $test."$key" with this:
if($key -eq "Settings"){
$row = $row + "," + '"' + $test."$key" + '"'
}else{
$row = $row + "," + $test."$key"
}
This idea is pretty terrible idea, as you stated, "grow to the right" is definitely not a good approach and you should consider a better way of doing it, data should always expand vertically.
As for the solution, you can create new columns easily with Select-Object and dynamically generated calculated properties.
Note, this should definitely not be considered an efficient approach. This will be slow because Select-Object is slow.
function Add-Column {
param(
[Parameter(ValueFromPipeline, DontShow, Mandatory)]
[object] $InputObject,
[Parameter(Mandatory)]
[string] $ColumnName,
[Parameter(Mandatory)]
[string] $ReferenceProperty,
[Parameter(Mandatory)]
[hashtable] $Values
)
begin {
$calculatedProp = #{ N = $ColumnName }
}
process {
$calculatedProp['E'] = { 0 }
if($value = $InputObject.$ReferenceProperty) {
if($Values.ContainsKey($value)) {
$calculatedProp['E'] = { $Values[$value] }
}
}
$InputObject | Select-Object *, $calculatedProp
}
}
Usage
Import-Csv path\to\csv | Add-Column -ColumnName '09/15/22' -ReferenceProperty Settings -Values #{
users = "7"
SomeBSValue = "22"
Computers = "52"
}
Result
Settings 08/15/22 09/15/22
-------- -------- --------
Users 0 7
Computers 0 52
SomeValue1 0 0
SomeValue2 0 0
SomeValue3 0 0
Stat1 0 0
Stat2 7 0
Stat3 0 0
SomeBSValue 1 22
This function allows then pipe into Export-Csv at ease:
Import-Csv path\to\csv | Add-Column ... | Export-Csv path\to\newCsv

Want to combine data from multiple text files with some logic per line, then output to a csv file

Similar to what I would do with XLS VLOOKUP's, I want to use data from a first imported (text or csv) file, lookup additional data into another imported object, and append data from 2nd object to the available fields in 1st imported file.
Then write it all to a .CSV file.
I'm new to powershell, so any help is welcome.
I started already with this code
cls
#Date
$date = Get-Date -Format yyyyMMddHHmm #ss
#files
$ProductionORderFile = "C:\Mydir\prod-orders.txt"
$UOMFile = "C:\Mydir\uom.txt"
#file_Import
$ProductionOrders = get-content -Path $ProductionOrderFile
$UOMs = import-csv -Path $UOMFile -Delimiter '|' -header UOM_SKU, UOM_UOM, UOM_UOMT, UOM_Qty, UOM_StdUOM, UOM_StdUOMT, UOM_EAN, UOM_kg, UOM_kgT
#main_loop
foreach ($line in $ProductionOrders)
{
$PO_ProdOrder = $line.SubString(0,10)
$PO_ProdOrderSKU = $line.SubString(11,10)
$PO_ProdOrderCust = $line.SubString(21,8)
$UOM_Found = "false"
$DLUOMQTY = 0
foreach ($UOM in $UOMs)
{
If ($PO_ProdOrderSKU = $UOM.UOM_SKU -And $UOM_UOM = "KAR")
{
$UOM_Found = "found"
break
}
if ($UOM_FOUND <> "found")
If ($PO_ProdOrderSKU = $UOM.UOM_SKU -And $UOM_UOM = "LTP")
{
$UOM_Found = "found"
break
}
if ($UOM_FOUND <> "found")
If ($PO_ProdOrderSKU = $UOM.UOM_SKU -And $UOM_UOM = "DIS")
{
$UOM_Found = "found"
break
}
}
if ($UOM_FOUND = "found)
{
$DLUOMQTY = $UOM.UOM_qty
}
$SavePath = "C:\Other\Temp\2207\rapid-sap-uom2" + "_" + $date + ".csv"
So, after this step, I want to save my line with all data from 1st file (prod-orders), preferably by column, and then add some columns from 2nd file uom as per above logic .
How to proceed ?
many thanks in advance for guiding a new user.

PowerShell-How to get values from this JSON file

This is the partial data after removed the confidential information.
{
"WebhookName":"Azure-CustomAlert-Webhook",
"RequestBody":"{\"schemaId\":\"azureMonitorCommonAlertSchema\",\"data\":{\"essentials\":{\"alertId\":\"/subscriptions/XXXXXXXXX/providers/Microsoft.AlertsManagement/alerts/XXXXXXX\",\"alertRule\":\"Low Memory\",\"severity\":\"Sev3\",\"signalType\":\"Log\",\"monitorCondition\":\"Fired\",\"monitoringService\":\"Log Analytics\",\"alertTargetIDs\":[\"/subscriptions/XXXXX/resourcegroups/XXXX-RG/providers/microsoft.operationalinsights/workspaces/workspacename\"],\"configurationItems\":[\"USE2V5TMP9001\"],\"originAlertId\":\"XXXXX\",\"firedDateTime\":\"2022-03-09T17:49:41.4631455Z\",\"description\":\"Triggers an alert for a low memory condition\",\"essentialsVersion\":\"1.0\",\"alertContextVersion\":\"1.1\"},\"alertContext\":{\"SearchQuery\":\"Perf | where ( CounterName == \\\"% Used Memory\\\" or CounterName == \\\"% Committed Bytes In Use\\\" ) | where Computer contains (\\\"TMP\\\") | summarize AggregatedValue = avg(CounterValue) by Computer, bin(TimeGenerated, 5m)\",\"SearchIntervalStartTimeUtc\":\"2022-03-09T17:42:10Z\",\"SearchIntervalEndtimeUtc\":\"2022-03-09T17:47:10Z\",\"ResultCount\":1,\"SeverityDescription\":\"Informational\",\"WorkspaceId\":\"XXXXX\",\"SearchIntervalDurationMin\":\"5\",\"AffectedConfigurationItems\":[\"USE2V5TMP9001\"],\"AlertType\":\"Metric measurement\",\"IncludeSearchResults\":true,\"Dimensions\":[{\"Name\":\"Computer\",\"Value\":\"USE2V5TMP9001\"}],\"SearchIntervalInMinutes\":\"5\",\"SearchResults\":{\"tables\":[{\"name\":\"PrimaryResult\",\"columns\":[{\"name\":\"Computer\",\"type\":\"string\"},{\"name\":\"TimeGenerated\",\"type\":\"datetime\"},{\"name\":\"AggregatedValue\",\"type\":\"real\"}],\"rows\":[[\"USE2V5TMP9001\",\"2022-03-09T17:42:10Z\",38.267662048339851]]}],\"dataSources\":[{\"resourceId\":\"/subscriptions/XXXXX/resourcegroups/XXXX/providers/microsoft.operationalinsights/workspaces/XXXX\",\"region\":\"eastus2\",\"tables\":[\"Perf\"]}]},\"Threshold\":9,\"Operator\":\"Greater Than\",\"IncludedSearchResults\":\"True\"},\"customProperties\":null}}",
"RequestHeader":{
"Connection":"Keep-Alive",
"Expect":"100-continue",
"Host":"xxxx.webhook.eus2.azure-automation.net",
"User-Agent":"IcMBroadcaster/1.0",
"X-CorrelationContext":"RkkKACgAAAACAAAAEADvqM+sXFG+SYkp7Tcy2IZaAQAQAMflO8/GhoFLrHCgd8ILz2o=",
"x-ms-request-id":"8fdd10d2-4a36-43a5-8e65-4eb20f3b9865"
}
}
The above json i got it from the Azure Log Search alert and trying to customize it.
From the above json can i get the values if i refer the column section keys?.
Ex: If i mention computer then i should be able to get value USE2V5TMP9001 and AggregatedValue is 38.267662048339851.
Use the columns array to map the individual row values to the correct type and property name:
$json = #'
<json goes here>
'#
$data = $json |ConvertFrom-Json
$columnDefinitions = $data.columns
$rows = foreach($row in $data.rows){
# prepare dictionary to hold the individual column values
$properties = [ordered]#{}
for($i = 0; $i -lt $row.Length; $i++){
# extract value and column metadata
$value = $row[$i]
$name = $columnDefinitions[$i].name
$type = $columnDefinitions[$i].type
# make sure to translate any type names if necessary
# eg. translate `real` -> `decimal`
if($type -eq 'real'){ $type = 'decimal' }
# convert value to correct type and store in property dictionary
$properties[$name] = $value -as $type
}
# create new object based on the row values
[pscustomobject]$properties
}
$rows will now contain 1 or more objects with the expected values so you can now do:
$rows |ForEach-Object {
# this now resolves `USE2V5TMP9001`
$_.computer
}

How can I run compare content in a Variable against a hashtable on PowerShell

I have a hashtable as below:
$ProjectType = #{
CSharp = 'FAE04EC0-301F-11D3-BF4B-00C04F79EFBC'
Web_Application = '349C5851-65DF-11DA-9384-00065B846F21'
Windows_Communication_Foundation = '3D9AD99F-2412-4246-B90B-4EAA41C64699'
Windows_Presentation_Foundation = '60DC8134-EBA5-43B8-BCC9-BB4BC16C2548'
Test = '3AC096D0-A1C2-E12C-1390-A8335801FDAB'
Silverlight = 'A1591282-1198-4647-A2B1-27E5FF5F6F3B'
}
I want to run the hashtable against contents in a variable $file, and get a return of the projecttype name from the hashtable if the value (guid) is found in $file.
You most likely want to reverse the order of your Keys and Values on your Hash Table:
$ProjectType = #{
'FAE04EC0-301F-11D3-BF4B-00C04F79EFBC' = 'CSharp'
'349C5851-65DF-11DA-9384-00065B846F21' = 'Web_Application'
'3D9AD99F-2412-4246-B90B-4EAA41C64699' = 'Windows_Communication_Foundation'
'60DC8134-EBA5-43B8-BCC9-BB4BC16C2548' = 'Windows_Presentation_Foundation'
'3AC096D0-A1C2-E12C-1390-A8335801FDAB' = 'Test'
'A1591282-1198-4647-A2B1-27E5FF5F6F3B' = 'Silverlight'
}
# using this as example
$exampleFile = #'
349C5851-65DF-11DA-9384-00065B846F21
60DC8134-EBA5-43B8-BCC9-BB4BC16C2548
A1591282-1198-4647-A2B1-27E5FF5F6F3B
00000000-1198-4647-A2B1-27E5FF5F6F3B
'# -split '\r?\n'
foreach($line in $exampleFile)
{
if($val = $ProjectType[$line])
{
"$line => $val"
continue
}
"$line => could not be found on reference table."
}
Flip the keys and values around so that the reference GUIDs are the keys:
$ProjectType = #{
'FAE04EC0-301F-11D3-BF4B-00C04F79EFBC' = 'CSharp'
'349C5851-65DF-11DA-9384-00065B846F21' = 'Web_Application'
'3D9AD99F-2412-4246-B90B-4EAA41C64699' = 'Windows_Communication_Foundation'
'60DC8134-EBA5-43B8-BCC9-BB4BC16C2548' = 'Windows_Presentation_Foundation'
'3AC096D0-A1C2-E12C-1390-A8335801FDAB' = 'Test'
'A1591282-1198-4647-A2B1-27E5FF5F6F3B' = 'Silverlight'
}
Now you can easily look up any value:
$file = '3AC096D0-A1C2-E12C-1390-A8335801FDAB'
if($ProjectType.ContainsKey($file)){
Write-Host "Found project type guid for '$($ProjectType[$file])'"
}

PowerShell nested property iteration

Looking for some input on the scripts below. Essentially I'm retrieving some json data with Invoke-WebRequest. I needed to export many of the properties to CSV. The data returned from Invoke-WebRequest is contained in an array: $devicedetails. This script works for outputting to CSV.
& {
Foreach ($PC in $DeviceDetails)
{
Foreach ($AppName in $PC.categories)
{
ForEach ($App in $AppName.apps)
{
ForEach($Status in $App.health_status)
{
[pscustomobject] #{
"Device ID" = $PC.device_id
"Device Type" = $PC.device_type
"Device Name" = $PC.device_name
Nickname = $PC.nick_name
Last_Seen = $PC.last_seen
Compliance_Category_Status = $Appname.issue
Compliance_Category = $Appname.category_id
Product_Name = $App.name
Product_Vendor = $App.vendor
Product_Version = $App.version
Product_Health_Item = $Status.status
Product_Health_Status = $Status.issue
}
}
}
}
}
} | Export-CSV -PAth $Outfile -NoTypeInformation
Curious if this is the best way to output properties to CSV. Additionally, I now have the need to do some additional processes on the custom object I'm creating but if I assign a variable to that custom object as shown below, it takes several minutes to complete whereas just exporting to CSV takes 12-13 seconds. Why is the performance so bad?
$DeviceOutput= #()
Foreach ($PC in $DeviceDetails)
{
Foreach ($AppName in $PC.categories)
{
ForEach ($App in $AppName.apps)
{
ForEach($Status in $App.health_status)
{
$DeviceOutput += [pscustomobject] #{
"Device ID" = $PC.device_id
"Device Type" = $PC.device_type
"Device Name" = $PC.device_name
"Nickname" = $PC.nick_name
Compliance_Category_Status = $Appname.issue
Compliance_Category = $Appname.category_id
Product_Name = $App.name
Product_Vendor = $App.vendor
Product_Version = $App.version
Product_Health_Item = $Status.status
Product_Health_Status = $Status.issue
}
}
}
}
}