Import-Csv data parse to array - powershell

I have a CSV file which contains multiline in some cells. I will use this data to compare the value got it from powershell.
This returns differences between the object, however the value is the same.
Expected Results should return nothing because both values are the same.
CSV content:
Value
System\CurrentControlSet\Control\ProductOptions
System\CurrentControlSet\Control\Server Applications
Software\Microsoft\Windows NT\CurrentVersion
Code:
PS> $data = Import-Csv .\tr.csv
PS> $data.Value
System\CurrentControlSet\Control\ProductOptions
System\CurrentControlSet\Control\Server Applications
Software\Microsoft\Windows NT\CurrentVersion
PS> $regval = ((Get-ItemProperty HKLM:\SYSTEM\CurrentControlSet\Control\SecurePipeServers\winreg\AllowedExactPaths).machine | Out-String).Trim()
PS> $regval
System\CurrentControlSet\Control\ProductOptions
System\CurrentControlSet\Control\Server Applications
Software\Microsoft\Windows NT\CurrentVersion
PS> Compare-Object $data.Value $regval
System\CurrentControlSet\Control\ProductOptions... =>
System\CurrentControlSet\Control\ProductOptions... <=
PS> $Tostring = ($data.Value | out-string).Trim()
PS> Compare-Object $Tostring $regval
InputObject SideIndicator
----------- -------------
System\CurrentControlSet\Control\ProductOptions... =>
System\CurrentControlSet\Control\ProductOptions... <=
PS> $Tostring.Length
145
PS> $regval.Length
147

This post no longer answers the OP's question directly but provides background information that is helpful for similar situations. This particular issue is solved by handling CR and LF characters before comparing the data. See Marked Answer for details.
Since $data in this case is an object with a property called value that holds your data, you need to compare what is stored in the value property to your $regval:
Compare-Object $data.value $regval
$regval is an array of strings before you pipe it to Out-String. After the pipe, it then becomes a string object. See below for type information before piping to Out-String.
$regval.gettype().fullname
System.String[]
$data is an array of objects (PSCustomObjects), which each have a property called Value that needs to be referenced directly if you want its data:
$data.gettype().fullname
System.Object[]
$data | Get-Member
TypeName: System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
Value NoteProperty string Value=System\CurrentControlSet\Control\ProductOptions
($regval | Get-member).where({$_.MemberType -eq "Property"})
TypeName: System.String
Name MemberType Definition
---- ---------- ----------
Length Property int Length {get;}
In order to compare the data of two objects using Compare-Object, best results seem to come when the objects are collections of the same type. PowerShell will automatically do conversions in the background in some cases like Compare-Object "1" 1. Maybe that has something to do with value types as I am not entirely sure. I would do the comparison before converting any of your data to different types. Then if you reference the Value property of $data, this condition becomes true:
$data.value | Get-member -type Property
TypeName: System.String
Name MemberType Definition
---- ---------- ----------
Length Property int Length {get;}
You can reference MKlement0's Explanation for more information about how PowerShell handles the array type.

The likeliest explanation is that:
the multi-line value from your CSV (obtained from a single field) contains LF-only (Unix-style) newlines,
whereas the string derived form the registry values has CRLF (Windows-style) newlines, due to applying Out-String to an array of strings.
The most direct fix is to remove the CR chars. from $regval (you can use "`r" in PowerShell to generate a CR char):
# ...
# Remove all CRs from $regval.
# Note that not providing a replacement string (missing 2nd RHS operand)
# default to the empty string, which effectively *removes* what was matched.
$regval = $regval -replace "`r"
# Should now work as expected.
Compare-Object $data.Value $regval
That said:
Since you're comparing just two objects that are strings, you can avoid the overhead of Compare-Object and simply use -eq:
$data.Value -eq $regVal
Alternatively, you can split the multi-line values into arrays of lines and compare them individually; note that if you use regex "`r?`n" or ('\r?\n') to match newlines to split by - which matches both LF-only and CRLF newlines - you needn't remove CR chars. beforehand or even apply Out-String to the array output from the Get-ItemProperty HKLM:\... call to begin with; however, with the variable values from your question, you'd use:
# Newline-style-agnostic
Compare-Object ($data.Value -split "`r?`n") ($regval -split "`r?`n")
# Or, knowing that $data.Value has LF, and $regval CRLF
Compare-Object ($data.Value -split "`n") ($regval -split "`r`n")
# Or, by using the [string[]] array of registry values directly:
$regvals = (Get-ItemProperty HKLM:\SYSTEM\CurrentControlSet\Control\SecurePipeServers\winreg\AllowedExactPaths).machine
Compare-Object ($data.Value -split "`n") $regvals
As for what you tried:
$Tostring = ($data.Value | out-string).Trim()
If $data.Value is a single string that doesn't have a trailing newline - whether or not it has embedded newlines - the above is an effective no-op:
An input object that is already a string is passed through as-is by Out-String.
While Out-String does append a trailing CRLF newline (on Windows), the subsequent .Trim() call removes it again.

Related

Powershell eq operator saying hashes are different, while Write-Host is showing the opposite

I have a script that periodically generates a list of all files in a directory, and then writes a text file of the results to a different directory.
I'd like to change this so it checks the newest text file in the output directory, and only makes a new one if there's differences. It seemed simple enough.
Here's what I tried:
First I get the most recent file in the directory, grab the hash, and write my variable values to the console:
$lastFile = gci C:\ReportOutputDir | sort LastWriteTime | select -last 1 | Select-Object -ExpandProperty FullName
$oldHash = Get-FileHash $lastFile | Select-Object Hash
Write-Host 'lastFile = '$lastFile
Write-Host 'oldHash = '$oldHash
Output:
lastFile = C:\ReportOutputDir\test1.txt
oldHash = #{Hash=E7787C54F5BAE236100A24A6F453A5FDF6E6C7333B60ED8624610EAFADF45521}
Then I do the exact same gci on the FileList dir, and create a new file (new_test.txt), then grab the hash of this file:
gci -Path C:\FileLists -File -Recurse -Name -Depth 2 | Sort-Object | out-file C:\ReportOutputDir\new_test.txt
$newFile = gci C:\ReportOutputDir | sort LastWriteTime | select -last 1 | Select-Object -ExpandProperty FullName
$newHash = Get-FileHash $newFile | Select-Object Hash
Write-Host 'newFile = '$newFile
Write-Host 'newHash = '$newHash
Output:
newFile = C:\ReportOutputDir\new_test.txt
newHash = #{Hash=E7787C54F5BAE236100A24A6F453A5FDF6E6C7333B60ED8624610EAFADF45521}
Finally, I attempt my -eq operator where I'd usually simply remove the newFile if it's equal. For now, I'm just doing a simple :
if ($newHash -eq $oldHash){
'files are equal'
}
else {'files are not equal'}
And somehow, I'm getting
files are not equal
What gives? Also, for the record I was originally trying to save the gci output to a variable and comparing the contents of the last file to the gci output, but was also having trouble with the -eq operator. Fairly new to powershell stuff so I'm sure I'm doing something wrong here.
Select-Object Hash creates an object with a .Hash property and it is that property that contains the hash string.
The object returned is of type [pscustomobject], and two instances of this type never compare as equal - even if all their property names and values are equal:
The reason is that reference equality is tested, because [pscustomobject] is a .NET reference type that doesn't define custom equality-testing logic.
Testing reference equality means that only two references to the very same instance compare as equal.
A quick example:
PS> [pscustomobject] #{ foo = 1 } -eq [pscustomobject] #{ foo = 1 }
False # !! Two distinct instances aren't equal, no matter what they contain.
You have two options:
Compare the .Hash property values, not the objects as a whole:
if ($newHash.Hash -eq $oldHash.Hash) { # ...
If you don't need a [pscustomobject] wrapper for the hash strings, use Select-Object's -ExpandProperty parameter instead of the (possibly positionally implied) -Property parameter:
Select-Object -ExpandProperty Hash
As for why the Write-Host output matched:
When you force objects to be converted to string representations - essentially, Write-Host calls .ToString() on its arguments - the string representations of distinct [pscustomobject] instances that have the same properties and values will be the same:
PS> "$([pscustomobject] #{ foo = 1 })" -eq "$([pscustomobject] #{ foo = 1 })"
True # Same as: '#{foo=1}' -eq '#{foo=1}'
However, you should not rely on these hashtable-like string representations to determine equality of [pscustomobject]s as a whole, because of the inherent limitations of these representations, which can easily yield false positives.
This answer shows how to compare [pscustomobject] instances as a whole, by comparing all of their property values, by passing all property names to Compare-Object -Property - but note that this assumes that all property values are either strings or instances of .NET value types or corresponding properties must again either reference the very same instance of a .NET reference type or be of a type that implements custom equality-comparison logic.

PS .Contains() method does not work on Get-Content return object

I tried to use the .Contains() method to check if a file contains a certain string:
# $file content: 123
$content = Get-Content -Path $file
$content.Contains("1") # Is always false, why?
"$content".Contains("1") # Works
Why are the " required so that the .Contains method works?
$content | Get-Member
TypeName: System.String
Name MemberType Definition
---- ---------- ----------
(...)
Contains Method bool Contains(string value)
As commented, using Get-Content returns an array of lines.
By quoting that result "$content" PowerShell merges these single lines together into a single string and then the String's method .Contains() works as expected.
If you add switch -Raw to the Get-Content statement, the result is a single multiline string on which the .Contains() method works.
If you don't pass -raw parameter to Get-Content cmdlet, the content is returned as an array of newline-delimited strings.

Using Hashtables With Powershell

I am using hashtables in powershell for multiple entries my hastable is working fine, but for the single entry its throwing the error by not picking the value it meant to be. Suppose I am having a Line
Aman;Indore;789858585;23
Raman;Delhi;785458545;35
for this two entry, my hashtable is working fine when i am giving command for example userinput.name[0] so it will pick Aman as the value and user.input.name[1] then it picks Raman as the value.
But the same code with single entry of
Aman;Indore;789858585;23
when I am giving userinput.name[0] so it is picking up A. The first letter of Aman it's picking instead of the complete name.
You might not even realise it, but you're using a PowerShell feature called Member Enumeration which was introduced in PowerShell 3.0.
Basically, if you attempt to access a named property on an array, if the property doesn't exist on the array object itself then PowerShell will look for that property on all the items in the array and return those values in a new array instead.
For example:
PS> $csv = #"
Aman;Indore;789858585;23
Raman;Delhi;785458545;35
"#
PS> $data = $csv | ConvertFrom-Csv -Delimiter ";" -Header #("Name", "Location", "Mobile", "Age");
PS> $data
Name Location Mobile Age
---- -------- ------ ---
Aman Indore 789858585 23
Raman Delhi 785458545 35
PS> $data.Name
Aman
Raman
# note this returns an array of objects thanks to member enumeration
PS> $data.Name.GetType().FullName
System.Object[]
PS> $data.Name[0]
Aman
In your case, $data.Name returns a new array containing the value of the Name property on all the items in $data - effectively #("Aman", "Raman"). So, when you use $data.Name[0], you're retrieving the first item in the array created by member enumeration - i.e. Aman - and all's well in the world...
Now, the catch is that if there's only one item in the array created by member enumeration it gets "unrolled" to be the value of the first item:
PS> $csv = #"
Aman;Indore;789858585;23
"#
PS> $data = $csv | ConvertFrom-Csv -Delimiter ";" -Header #("Name", "Location", "Mobile", "Age");
PS> $data
# Name Location Mobile Age
# ---- -------- ------ ---
# Aman Indore 789858585 23
# note this returns a single string because the "member enumeration" array
# gets unrolled if there's only one item
PS> $data.Name.GetType().FullName
System.String
PS> $data.Name
# Aman
PS> $data.Name[0]
# A
And in your case the $data.Name[0] is equivalent to "Aman"[0] which returns A.
To fix this, rather than inadvertently use member enumeration by doing $data.Name[0], where the result can vary based on the number of items in the array, you can use one of the following:
PS> $data[0].Name
Aman
PS> #($data.Name)[0]
Aman
The first option is probably more performant in the general case, but the second is a useful workaround in some cases where the first won't work (e.g. when dealing with return values from functions / where-object, etc).

how to remove curly braces and get proper output in csv file

I m trying to get the list of vm from nutanix with name, ipaddress,
The output I am recieving includes ipaddress with curly braces which gives output as System.String[]
I have taken all the values in a array by using a for loop, than have exported the values to csv
Script which i have written is as follows-
foreach ($vmachine in $vm){
$obj = "" | Select "vmName", "ipAddresses", "description", "protectionDomainName", "powerState"
$obj.vmName = $vmachine.vmName
$obj.ipAddresses = $vmachine.ipAddresses
$obj.description = $vmachine.description
$obj.protectionDomainName = $vmachine.protectionDomainName
$obj.powerState = $vmachine.powerState
$outArrayVM += $obj
$obj =$null
}
$outArrayVM | Export-Csv d:\z.csv
Expected output should be some ipaddress like 10.x.x.x, but m getting #{ipAddresses=System.String[]}
This happens because $vmachine.ipAddresses is a string array object. You want a string representation of that with controlled formatting. There are many ways to accomplish this. Here is one that will join multiple IPs (if they exist) using a ;. If there is only one IP, it will appear with no semi-colon:
$obj.ipAddresses = $vmachine.ipAddresses -join ";"
Here's an example of your scenario:
$ip = #("10.1.23.45")
$ip.gettype()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Object[] System.Array
$obj.name = "test"
$obj.ip = $ip
$obj
name ip
---- --
test {10.1.23.45}
$obj | convertto-csv
#TYPE Selected.System.String
"name","ip"
"test","System.Object[]"
Converting the ip property of $obj to string forces PowerShell to interpret the property as a string rather than a collection. Thus, the braces notation ({}) goes away.
$obj.ip = $ip -join ";"
$obj | convertto-csv
#TYPE Selected.System.String
"name","ip"
"test","10.1.23.45"
Here are some other alternatives to set the ip property value as a string:
$obj.ip = -join $ip # No join character here. Works best with only one IP.
$obj.ip = $ip[0] # Accesses first element of array $ip, which will be a string. Only works with one IP.
$obj.ip = [string]$ip # Uses string type accelerator to cast $ip as string. This will join multiple IPs with a space between each IP.
Explanation:
When a ConvertTo-Csv or Export-Csv is run, the input object property is converted using the ToString() method. If the reference type of that object property (System.Array in this case) does not have an override for the ToString() method, then that method will return the fully qualified type name of the property. In this instance, that FQTN is System.Object[]. This is predictable with a little digging.
Testing with [Int32], you would expect the string conversion to provide a string representation of the integer data because it does have an override:
$int = 1
$int.gettype().fullname
System.Int32
($int | Get-Member).where{$_.Name -eq "ToString"}
TypeName: System.Int32
Name MemberType Definition
---- ---------- ----------
ToString Method string ToString(), string ToString(string format), string ToString(System.IFormatProvider provid...
$int.ToString()
1
$int.ToString().gettype().fullname
System.String
Testing with [Array], you would not expect the string conversion to provide a string representation of the array data because it does not have an override:
$arr = [array]1
$arr.gettype().fullname
System.Object[]
([System.Object[]] | Get-Member -Static).where{$_.name -eq "ToString"}
$arr.toString()
System.Object[]
See Export-Csv and Object.ToString Method for supplemental explanations and examples.

Convert Invoke-RestMethod output to object

I'm trying to use PowerShell Invoke-RestMethod on a ticketing system's API and then convert the output into a PowerShell object.
As an example when I use Invoke-RestMethod to get the properties of a ticket I get this.
$object = Invoke-RestMethod '[URI here]'
$object.GetType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True String System.Object
$object
EWREST_supported_user0='Yes';
EWREST_updater_team='Support Team';
EWREST_eng_id='CLT';
EWREST_testlabelsub='Test Label';
EWREST_time_created='17:21:03';
EWREST_cr_conversion_related_to='Support Case';
EWREST__1901_full_name='secuser Testuser1';
EWREST_summary='Ticket Title';
EWREST_i_would_like_to_reopen_my_ticket='No';
EWREST_assigned_team_leader='Agiloft Admin';
EWREST_id='183255';
EWREST_severity='Sev 4';
EWREST_problem_description='<div>This is an example of a ticket note that takes up multiple lines when read via API<\/div><div> <\/div><div>Example note info here<\/div><div> <\/div>
<div>Additional example note info here<\/div><div> <\/div><div>Even more note info here<\/div>';
EWREST_demo_data='No';
What I would like to be able to do is manipulate $object as an object by doing things like $object.EWREST_category and get "Networking". So I have been trying to figure out how to maniuplate $object which is just a string of attributes to a traditional PowerShell object with properties.
Can someone offer some pointers on how to go about that?
Since you already have a string with key/value pairs I'd just do a little cleanup (remove the single quotes and the semicolons), convert the string to a hashtable, then build a custom object from that:
$response = Invoke-RestMethod '[URI here]'
$props = $response -replace "'" -replace ';' | ConvertFrom-StringData
$object = New-Object -Type PSObject -Property $props
Edit: To mangle the multiline value into one line you can use another replacement with a negative lookbehind assertion ((?<!...)) that removes newlines only if they're not preceded by a single quote followed by a semicolon. However, since that same property contains other semicolons you also need to modify the semicolon replacement, so that it only removes semicolons if they're followed by a newline or the end of the string (using a positive lookahead assertion, (?=...)).
$props = $response -replace "(?<!';)`n" -replace "'" -replace ";(?=`n|`$)" |
ConvertFrom-StringData
Maybe next naive script could suffice?
$ob= "EWREST_supported_user0='Yes';
EWREST_category='Networking';
EWREST_updater_team='Admin Team';
EWREST_time_created='12:56:53';
EWREST_cr_conversion_related_to='Support Case';
" # this is that string
# transform string to an array:
$oba = $ob.Split("`r`n", [System.StringSplitOptions]::RemoveEmptyEntries)
$obah=#{} # create empty hash table
# and fill it from array then:
$oba | ForEach-Object {
$aux=$_.split('=;') # key / value pair
$obah[$aux[0]] = $aux[1] } # hash[key] = value
$obah.Keys # display hash table keys (only debug)
$obah.EWREST_time_created # hash table item use (example: property)
$obah['EWREST_category'] # hash table item use (another approach: index)