Understanding Powershell: example - Convert JSON to CSV - powershell

I've read several posts (like Convert JSON to CSV using PowerShell) regarding using PowerShell to CSV. I have also read that it is relatively poor form to use the pipe syntax in scripts -- that it's really meant for command line and can create a hassle for developers to maintain over time.
Using this sample JSON file...
[
{
"a": "Value 1",
"b": 20,
"g": "Arizona"
},
{
"a": "Value 2",
"b": 40
},
{
"a": "Value 3"
},
{
"a": "Value 4",
"b": 60
}
]
...this code...
((Get-Content -Path $pathToInputFile -Raw) | ConvertFrom-Json) | Export-CSV $pathToOutputFile -NoTypeInformation
...creates a file containing CSV as expected.
"a","b","g"
"Value 1","20","Arizona"
"Value 2","40",
"Value 3",,
"Value 4","60",
This code...
$content = Get-Content -Path $pathToInputFile -Raw
$psObj = ConvertFrom-Json -InputObject $content
Export-Csv -InputObject $psObj -LiteralPath $pathToOutputFile -NoTypeInformation
...creates a file containing nonsense:
"Count","Length","LongLength","Rank","SyncRoot","IsReadOnly","IsFixedSize","IsSynchronized"
"4","4","4","1","System.Object[]","False","True","False"
It looks like maybe an object definition(?).
What is the difference? What PowerShell nuance did I miss when converting the code?
The answer to Powershell - Export a List of Objects to CSV says the problem is from the -InputObject option causing the object, not it's contents, to be sent to Export-Csv, but doesn't state how to remedy the problem without using the pipe syntax. I'm thinking something like -InputObject $psObj.contents. I realize that's not a real thing, but I Get-Members doesn't show me anything that looks like it will solve this.

This is not meant as an answer but just to give you a vague representation of what ConvertTo-Csv and Export-Csv are doing and to help you understand why -InputObject is meant to be bound from the pipeline and should not be used manually.
function ConvertTo-Csv2 {
param(
[parameter(ValueFromPipeline)]
[Object] $InputObject
)
begin {
$isFirstObject = $true
filter Normalize {
if($_ -match '"') { return $_.Replace('"','""') }
$_
}
}
process {
if($isFirstObject) {
$headers = $InputObject.PSObject.Properties.Name | Normalize
$isFirstObject = $false
[string]::Format('"{0}"', [string]::Join('","', $headers))
}
$values = foreach($value in $InputObject.PSObject.Properties.Value) {
$value | Normalize
}
[string]::Format('"{0}"', [string]::Join('","', $values))
}
}
As we can observe, there is no loop enumerating the $InputObject in the process block of this function, yet, because of how this block works, each object coming from the pipeline is processed and converted to a Csv string representation of the object.
Within a pipeline, the Process block executes once for each input object that reaches the function.
If instead, we attempt to use the InputObject parameter from the function, the object being passed as argument will be processed only once.
Calling the function at the beginning, or outside of a pipeline, executes the Process block once.

Get-Members doesn't show me anything that looks like it will solve this
Get-Member
It's because how you pass values has different behavior.
The pipeline enumerates values, it's almost like a foreach($item in $pipeline). Passing by Parameter skips that
Here I have an array of 3 letters.
$Letters = 'a'..'c'
I'm getting different types
Get-Member -InputObject $Letters
# [Object[]]
# [char]
$letters | Get-Member
Processed for each item
$letters | ForEach-Object {
"iteration: $_"
}
iteration: a
iteration: b
iteration: c
Compare to
ForEach-Object -InputObject $Letters {
"iteration: $_"
}
iteration: a b c
Detecting types
Here's a few ways to inspect objects.
using ClassExplorer
PS> ($Letters).GetType().FullName
PS> ($Letters[0]).GetType().FullName # first child
System.Object[]
System.Char
PS> $Letters.count
PS> $Letters[0].Count
3
1
$Letters.pstypenames -join ', '
$Letters[0].pstypenames -join ', '
System.Object[], System.Array, System.Object
System.Char, System.ValueType, System.Object
Tip: $null.count always returns 0. It does not throw an error.
if($neverExisted.count -gt 1) { ... }
Misc
I have also read that it is relatively poor form to use the pipe syntax in scripts
This is not true, Powershell is designed around piping objects.
Tip: $null.count always returns 0. It does not throw an error.
Maybe They were talking about
Example2: slow operations
Some cases when you need something fast, the overhead to Foreach-Object over a foreach can be an issue. It makes it so you have to use some extra syntax.
If you really need speed, you should probably be calling dotnet methods anyway.
Example1: Piping when you could use a parameter
I'm guessing they meant piping a variable in cases where you can pass parameters?
$text = "hi-world"
# then
$text | Write-Host
# vs
Write-Host -InputObject $Text

Related

Powershell Array Output to html

Apologies if this is irrelevant but I'm new to powershell and I've been scratching my head on this for a few days on and off now. I'm trying to write a script that will output two columns of data to a html document. I've achieved most of it by learning through forums and testing different combinations.
The problem is although it gives me the result I need within powershell itself; it will not properly display the second column results for Net Log Level.
So the script looks at some folders and pulls the * value which is always three digits (this is the Site array). It then looks within each of these folders to the Output folder and grabs a Net Log Level node from a file inside there. The script is correctly listing the Sites but is only showing the last value for Net Log Level which is 2. You can see this in the screenshot above. I need this to take every value for each Site and display as appropriate. The image of the incorrect result is below. I need the result to be 1,4,2,2,2. Any help would be greatly appreciated!
function getSite {
Get-ChildItem C:\Scripts\ServiceInstalls\*\Output\'Config.exe.config' | foreach {
$Site = $_.fullname.substring(27, 3)
[xml]$xmlRead = Get-Content $_
$NetLogLevel = $xmlRead.SelectSingleNode("//add[#key='Net Log Level']")
$NetLogLevel = $NetLogLevel.value
New-Object -TypeName System.Collections.ArrayList
$List1 += #([System.Collections.ArrayList]#($Site))
New-Object -TypeName System.Collections.ArrayList
$List2 += #([System.Collections.ArrayList]#($NetLogLevel))
}
$Results = #()
ForEach($Site in $List1){
$Results += [pscustomobject]#{
"Site ID" = $Site
"Net Log Level" = $NetLogLevel
}
}
$Results | ConvertTo-HTML -Property 'Site','Net Log Level' | Set-Content Output.html
Invoke-Item "Output.html"
}
getSite
Restructure your code as follows:
Get-ChildItem 'C:\Scripts\ServiceInstalls\*\Output\Config.exe.config' |
ForEach-Object {
$site = $_.fullname.substring(27, 3)
[xml]$xmlRead = Get-Content -Raw $_.FullName
$netLogLevel = $xmlRead.SelectSingleNode("//add[#key='Net Log Level']").InnerText
# Construct *and output* a custom object for the file at hand.
[pscustomobject] #{
'Site ID' = $site
'Net Log Level' = $netLogLevel
}
} | # Pipe the stream of custom objects directly to ConvertTo-Html
ConvertTo-Html | # No need to specify -Property if you want to use all properties.
Set-Content Output.html
As for what you tried:
New-Object -TypeName System.Collections.ArrayList in effect does nothing: it creates an array-list instance but doesn't save it in a variable, causing it to be enumerated to the pipeline, and since there is nothing to enumerate, nothing happens.
There is no point in wrapping a [System.Collections.ArrayList] instance in #(...): its elements are enumerated and then collected in a regular [object[]] array - just use #(...) by itself.
Using += to "grow" an array is quite inefficient, because a new array must be allocated behind the scenes every time; often there is no need to explicitly create an array - e.g. if you can simply stream objects to another command via the pipeline, as shown above, or you can let PowerShell itself implicitly create an array for you by assigning the result of a pipeline or foreach loop as a whole to a variable - see this answer.
Also note that when you use +=, the result is invariably a regular [object[] array, even if the RHS is a different collection type such as ArrayList.
There are still cases where iteratively creating an array-like collection is necessary, but you then need to use the .Add() method of such a collection type in order to grow the collection efficiently - see this answer.
Instead of populating two separate lists, simply create the resulting objects in the first loop:
function getSite {
$Results = Get-ChildItem C:\Scripts\ServiceInstalls\*\Output\'Config.exe.config' | ForEach-Object {
$Site = $_.fullname.substring(27, 3)
[xml]$xmlRead = Get-Content $_
$NetLogLevel = $xmlRead.SelectSingleNode("//add[#key='Net Log Level']")
$NetLogLevel = $NetLogLevel.value
[pscustomobject]#{
"Site ID" = $Site
"Net Log Level" = $NetLogLevel
}
}
$Results | ConvertTo-HTML -Property 'Site', 'Net Log Level' | Set-Content Output.html
Invoke-Item "Output.html"
}
getSite

Powershell script to retrieve JSON content that has square bracket in the beginning and the end

I have a Json file, input.json, with a content of
[
{
"id": "1234abc",
"createdDate":"2021-10-04"
}
]
However, when I use the Powershell script below to retrieve its content, I get nothing in return.
$inputread=Get-Content -raw "D:\sourcefolder\input.json" | ConvertFrom-Json
Write-Output "Input is $inputread
I got only
Input is
Please advise me how to get the content of the Json file.
the Square brackets indicate that the Json String is an array of objects.
So your variable $Inputread will be an array of objects.
if you use .GetType() you can see the BaseType is a Object.Array
$inputread.gettype()
When you try to Write-Output, you are specifying the whole Array, but not indicating what property you want to output from the Objects within the Array.
at its ugliest you could just do a
Write-Output "Input is $($inputread.Id)"
but you may find it better to use one of these as they will be more readable.
#Alt 1 - ForEach Loop
Foreach($Item in $inputread){
Write-Output "Input is $($Item.Id)"
}
#Alt 2 - Foreach-Object
$inputread|%{Write-Output "Input is $($_.Id)"}
#Alt 3 - For Loop with $I as Index number
$ObjCount = ($inputread|measure).count
for ($I=0;$I -lt $ObjCount;$I++){
$Obj = $inputread[$I]
Write-Output "Input is $($Obj.Id)"
}

Converting string with object layout to object

Some objects has been saved to a txt.file
looking like this:
#{flightNumber=01; flightDate=2010-01-10; flightIdentification=201001}
#{flightNumber=01; flightDate=2010-01-10; flightIdentification=201002}
and I'm trying to read them in another program and convert them back into objects. What bothers me is that it understands each of the "objects" as a string and I have been unable to cast it into an object.
$list = Get-Content -Path 'C:\Users\XXXXX\Downloads\TemplateObject.txt'
foreach (#object in $list) {
Write-Host $object.flightNumber
}
From what I've shown, I would expect to see 2 different objects with the variables flightNumber, flightDate and flightIdentification
I've tried piping it by using ConvertFrom-StringData
I've tried casting to an object
I expect 2 separate objects containing 3 variables in each.
Don't pipe objects directly to files!
As has been pointed out, take advantage of built-in options for serialization to disk, like ConvertTo-Csv/Export-Csv for flat objects, ConvertTo-Json or Export-Clixml for more complex objects.
As a one-off thing, if you need to recover and re-encode this data, you could use the regex -replace operator to add quotes around the values, at which point the parser should accept them as hashtable entries and you can cast it to an object:
$string = '#{flightNumber=01; flightDate=2010-01-10; flightIdentification=201001}'
# Place double-quotes around anything found between a `=` and `;` or `}`
$quotedString = $string -replace '(?<=\=)([^=;}]+)(?=\s*(?:;|}))', '"$1"'
# Parse the resulting string as if it was PowerShell code
$errors = #()
$objectAST = [System.Management.Automation.Language.Parser]::ParseInput($quotedString, [ref]$null,[ref]$errors)
$objects = if(-not $errors){
# This is pretty dangerous, you should NEVER do this in a production script
$objectAST.GetScriptBlock.Invoke() |ForEach-Object {
[pscustomobject]$_
}
}
# This variable now contains the re-animated objects
$objects
You can convert a string to a hashtable using convertfrom-stringdata after some manipulation:
$a = '#{flightNumber=01; flightDate=2010-01-10; flightIdentification=201001}'
$a = $a -replace '#{' -replace '}' -replace ';',"`n" | ConvertFrom-StringData
[pscustomobject]$a
flightNumber flightIdentification flightDate
------------ -------------------- ----------
01 201001 2010-01-10

Powershell - Iterate through variables dynamically

I am importing a CSV file with two records per line, "Name" and "Path".
$softwareList = Import-Csv C:\Scripts\NEW_INSTALLER\softwareList.csv
$count = 0..($softwareList.count -1)
foreach($i in $count){
Write-Host $softwareList[$i].Name,$softwareList[$i].Path
}
What I am trying to do is dynamically assign the Name and Path of each record to a WPFCheckbox variable based on the $i variable. The names for these checkboxes are named something such as WPFCheckbox0, WPFCheckbox1, WPFCheckbox2 and so on. These objects have two properties I planned on using, "Command" to store the $SoftwareList[$i].path and "Content" to store the $SoftwareList[$i].Name
I cannot think of a way to properly loop through these variables and assign the properties from the CSV to the properties on their respective WPFCheckboxes.
Any suggestions would be very appreciated.
Invoke-Expression is one way, though note Mathias' commented concerns on the overall approach.
Within your foreach loop, you can do something like:
invoke-expression "`$WPFCheckbox$i`.Command = $($SoftwareList[$i].Path)"
invoke-expression "`$WPFCheckbox$i`.Content= $($SoftwareList[$i].Name)"
The back-tick ` just before the $WPFCheckBox prevents what would be an undefined variable from being immediately evaluated (before the expression is invoked), but the $I is. This gives you a string with your $WPFCheckbox1, to which you then append the property names and values. The $SoftwareList values are immediately processed into the raw string.
The Invoke-Expression then evaluates and executes the entire string as if it were a regular statement.
Here's a stand-alone code snippet to play with:
1..3 |% {
invoke-expression "`$MyVariable$_` = New-Object PSObject"
invoke-expression "`$MyVariable$_` | add-member -NotePropertyName Command -NotePropertyValue [String]::Empty"
invoke-expression "`$MyVariable$_`.Command = 'Path #$_'"
}
$MyVariable1 | Out-String
$MyVariable2 | Out-String
$MyVariable3 | Out-String
As a side note (since I can't comment yet on your original question,) creating an array just to act as iterator through the lines of the file is really inefficient. There are definitely better ways to do that.

Writing $null to Powershell Output Stream

There are powershell cmdlets in our project for finding data in a database. If no data is found, the cmdlets write out a $null to the output stream as follows:
Write-Output $null
Or, more accurately since the cmdlets are implemented in C#:
WriteOutput(null)
I have found that this causes some behavior that is very counter to the conventions employed elsewhere, including in the built-in cmdlets.
Are there any guidelines/rules, especially from Microsoft, that talk about this? I need help better explaining why this is a bad idea, or to be convinced that writing $null to the output stream is an okay practice. Here is some detail about the resulting behaviors that I see:
If the results are piped into another cmdlet, that cmdlet executes despite no results being found and the pipeline variable ($_) is $null. This means that I have to add checks for $null.
Find-DbRecord -Id 3 | For-Each { if ($_ -ne $null) { <do something with $_> }}
Similarly, If I want to get the array of records found, ensuring that it is an array, I might do the following:
$recsFound = #(Find-DbRecord -Category XYZ)
foreach ($record in $recsFound)
{
$record.Name = "Something New"
$record.Update()
}
The convention I have seen, this should work without issue. If no records are found, the foreach loop wouldn't execute. Since the Find cmdlet is writing null to the output, the $recsFound variable is set to an array with one item that is $null. Now I would need to check each item in the array for $null which clutters my code.
$null is not void. If you don't want null values in your pipeline, either don't write null values to the pipeline in the first place, or remove them from the pipeline with a filter like this:
... | Where-Object { $_ -ne $null } | ...
Depending on what you want to allow through the filter you could simplify it to this:
... | Where-Object { $_ } | ...
or (using the ? alias for Where-Object) to this:
... | ? { $_ } | ...
which would remove all values that PowerShell interprets as $false ($null, 0, empty string, empty array, etc.).