Given the following input XML sample -- assume multiple LogMessage entries.
<LogMessages>
<LogMessage time="2017-12-08 11:44:05.202" messageID="A10">
<![CDATA[Long non-xml string here containing <TS "2017120811431218"> somewhere in the body"]>
</LogMessage>
</LogMessages>
I am using the following code to capture the values of attributes time, messageID, and capture a group in the CDATA.
[xml]$xml = Get-Content input.xml
$xml.LogMessages.LogMessage | Where-Object {$_.messageID -eq "A10"} | Select-Object -Property time,messageID,#{Name="A10Timestamp"; Expression=$_."#cdata-section" -match '<TS "(?<group>[0-9]{16})">' | Select-Object $Matches['group'] }} `
| Export-Csv output.csv -NoTypeInformation
Output looks like:
time messageID Group
---- --------- ---------------
2017-12-08 11:43:12.183 S6F1 #{2017120811431218=}
The #{ and } wrapping the captured group value is undesired. I am concerned about this particular use of the $Matches variable...I think what gets printed is a Match object and not the group string that it matched on... or something like this.
What is going on and how do I get the entries in the Group column appear as 2017120811431218 and not #{2017120811431218=}?
The Match operator returns a boolean and populates the $Matches variable with the matching results.
In other words, you should void what is returned by the -Match operator (and not pipe it) and than simply return the $Matches['group'] to the Expression:
Expression={$Void = $_."#cdata-section" -match '<TS "(?<group>[0-9]{16})">'; $Matches['group']}
Related
I am new to PowerShell and trying to get a list of VM names and their associated IP Addresses from within Hyper-V.
I am getting the information fine on the screen but when I try to export to csv all I get for the IP Addresses is System.Collections.Generic.List`1[System.String] on each line.
There are suggestions about "joins" or "ConvertTo-CSV" but I don't understand the syntax for these.
Can anyone help?
This is the syntax I am using...
Get-VM | Select -ExpandProperty VirtualNetworkAdapters | select name, IPV4Addresses | Export-Csv -Path "c:\Temp\VMIPs.csv"
If an object you export as CSV with Export-Csv or ConvertTo-Csv has property values that contain a collection (array) of values, these values are stringified via their .ToString() method, which results in an unhelpful representation, as in the case of your array-valued .IPV4Addresses property.
To demonstrate this with the ConvertTo-Csv cmdlet (which works analogously to Export-Csv, but returns the CSV data instead of saving it to a file):
PS> [pscustomobject] #{ col1 = 1; col2 = 2, 3 } | ConvertTo-Csv
"col1","col2"
"1","System.Object[]"
That is, the array 2, 3 stored in the .col2 property was unhelpfully stringified as System.Object[], which is what you get when you call .ToString() on a regular PowerShell array; other .NET collection types - such as [System.Collections.Generic.List[string]] in your case - stringify analogously; that is, by their type name.
Assuming you want to represent all values of an array-valued property in a single CSV column, to fix this problem you must decide on a meaningful string representation for the collection as a whole and implement it using Select-Object with a calculated property:
E.g., you can use the -join operator to create a space-separated list of the elements:
PS> [pscustomobject] #{ col1 = 1; col2 = 2, 3 } |
Select-Object col1, #{ n='col2'; e={ $_.col2 -join ' ' } } |
ConvertTo-Csv
"col1","col2"
"1","2 3"
Note how array 2, 3 was turned into string '2 3'.
OtherObjectPipedStuff | Select-object name,IPV4Addresses | export-csv PP.csv -NoTypeinformation
I am new to PowerShell and trying to get a list of VM names and their associated IP Addresses from within Hyper-V.
I am getting the information fine on the screen but when I try to export to csv all I get for the IP Addresses is System.Collections.Generic.List`1[System.String] on each line.
There are suggestions about "joins" or "ConvertTo-CSV" but I don't understand the syntax for these.
Can anyone help?
This is the syntax I am using...
Get-VM | Select -ExpandProperty VirtualNetworkAdapters | select name, IPV4Addresses | Export-Csv -Path "c:\Temp\VMIPs.csv"
If an object you export as CSV with Export-Csv or ConvertTo-Csv has property values that contain a collection (array) of values, these values are stringified via their .ToString() method, which results in an unhelpful representation, as in the case of your array-valued .IPV4Addresses property.
To demonstrate this with the ConvertTo-Csv cmdlet (which works analogously to Export-Csv, but returns the CSV data instead of saving it to a file):
PS> [pscustomobject] #{ col1 = 1; col2 = 2, 3 } | ConvertTo-Csv
"col1","col2"
"1","System.Object[]"
That is, the array 2, 3 stored in the .col2 property was unhelpfully stringified as System.Object[], which is what you get when you call .ToString() on a regular PowerShell array; other .NET collection types - such as [System.Collections.Generic.List[string]] in your case - stringify analogously; that is, by their type name.
Assuming you want to represent all values of an array-valued property in a single CSV column, to fix this problem you must decide on a meaningful string representation for the collection as a whole and implement it using Select-Object with a calculated property:
E.g., you can use the -join operator to create a space-separated list of the elements:
PS> [pscustomobject] #{ col1 = 1; col2 = 2, 3 } |
Select-Object col1, #{ n='col2'; e={ $_.col2 -join ' ' } } |
ConvertTo-Csv
"col1","col2"
"1","2 3"
Note how array 2, 3 was turned into string '2 3'.
OtherObjectPipedStuff | Select-object name,IPV4Addresses | export-csv PP.csv -NoTypeinformation
I have a csv file, that contains the next data:
Pages,Pages BN,Pages Color,Customer
145,117,28,Report_Alexis
46,31,15,Report_Alexis
75,27,48,Report_Alexis
145,117,28,Report_Jack
46,31,15,Report_Jack
75,27,48,Report_Jack
145,117,28,Report_Amy
46,31,15,Report_Amy
75,27,48,Report_Amy
So what i need to do , is sum each column based on the report name and the export to another csv file like this
Pages,Pages BN,Pages Color,Customer
266,175,91,Report_Alexis
266,175,91,Report_Jack
266,175,91,Report_Amy
How can i do this?
I tried with this:
$coutnpages = Import-Csv "C:\temp\testcount\final file2.csv" |where {$_.Filename -eq 'Report_Jack'} | Measure-Object -Property Pages -Sum
then
$Countpages.Sum | Set-Content -Path "C:\temp\testcount\final file3.csv"
But this is just one, and then i dont know how to follow.
Can you please help me?
Working code
$IdentityColumns = #('Customer')
$ColumnsToSum = #('Pages', 'Pages BN', 'Pages Color')
$CSVFileInput = 'S:\SCRIPTS\1.csv'
Import-Csv -Path $CSVFileInput |
Group-Object -Property $IdentityColumns |
ForEach-Object {
$resultHT = #{ Customer = $_.Name } # This is result HashTable (Key-Value collection). We add here sum's next line.
#($_.Group | Measure-Object -Property $ColumnsToSum -Sum ) | # Run calculating of sum for all $ColumnsToSum`s in one line
ForEach-Object { $resultHT[$_.Property] = $_.Sum } # For each calculated property we set property in result HashTable
return [PSCustomObject]$resultHT # Convert HashTable to PSCustomObject. This better.
} | # End of ForEach-Object by groups
Select #($ColumnsToSum + $IdentityColumns) | # This sets order of columns. It may be important.
Out-GridView # Or replace with Export-Csv
#Export-Csv ...
Explanation:
Use Group-Object to make collection of groups. Groups have 4 properties:
Name - Name of group, equals to stingified values of property(-ies) you're grouping by
Values - Collection of values of properties you're grouping by (not stringified)
Count - Count of elements grouped into this group
Group - Values of elements grouped into this group
For grouping by single string properties (in this case it is ok), you can easily use Name of group, otherwise, always use Values.
So after Group-Object, you iterate not on collection-of-rows of CSV, but on collection-of-collections-of-rows grouped by some condition.
Measure-Object can process more than one propertiy for single pass (not mixing between values from different properties), we use this actively. This results in array of objects with attribute Property equal to passed to Measure-Object and value (Sum in our case). We move those Property=Sum pairs to hashtable.
[PSCustomObject] converts hashtable to object. Objects are always better for output.
How to find the count of unique numbers in a CSV file? When I use the following command in PowerShell ISE
1,2,3,4,2 | Sort-Object | Get-Unique
I can get the unique numbers but I'm not able to get this to work with CSV files. If for example I use
$A = Import-Csv C:\test.csv | Sort-Object | Get-Unique
$A.Count
it returns 0. I would like to count unique numbers for all the files in a given folder.
My data looks similar to this:
Col1,Col2,Col3,Col4
5,,7,4
0,,9,
3,,5,4
And the result should be 6 unique values (preferably written inside the same CSV file).
Or would it be easier to do it with Notepad++? So far I have found examples only on how to count the unique rows.
You can try the following (PSv3+):
PS> (Import-CSV C:\test.csv |
ForEach-Object { $_.psobject.properties.value -ne '' } |
Sort-Object -Unique).Count
6
The key is to extract all property (column) values from each input object (CSV row), which is what $_.psobject.properties.value does;
-ne '' filters out empty values.
Note that, given that Sort-Object has a -Unique switch, you don't need Get-Unique (you need Get-Unique only if your input already is sorted).
That said, if your CSV file is structured as simply as yours, you can speed up processing by reading it as a text file (PSv2+):
PS> (Get-Content C:\test.csv | Select-Object -Skip 1 |
ForEach-Object { $_ -split ',' -ne '' } |
Sort-Object -Unique).Count
6
Get-Content reads the CSV file as a line of strings.
Select-Object -Skip 1 skips the header line.
$_ -split ',' -ne '' splits each line into values by commas and weeds out empty values.
As for what you tried:
Import-CSV C:\test.csv | Sort-Object | Get-Unique:
Fundamentally, Sort-Object emits the input objects as a whole (just in sorted order), it doesn't extract property values, yet that is what you need.
Because no -Property argument is passed to Sort-Object to base the sorting on, it compares the custom objects that Import-Csv emits as a whole, by their .ToString() values, which happen to be empty[1]
, so they all compare the same, and in effect no sorting happens.
Similarly, Get-Unique also determines uniqueness by .ToString() here, so that, again, all objects are considered the same and only the very first one is output.
[1] This may be surprising, given that using a custom object in an expandable string does yield a value: compare $obj = [pscustomobject] #{ foo ='bar' }; $obj.ToString(); '---'; "$obj". This inconsistency is discussed in this GitHub issue.
In a PowerShell script, I have some objects that I pass to the Format-Table CmdLet.
The output of my script looks like this:
Something...
Operation AttributeName AttributeValue
--------- ------------- --------------
Delete Member John Doe
Something else...
Since the meaning of the fields is pretty self-explanatory, I would like to remove the headers, the '---' separators and the blank lines at the beginning and at the end from the output of Format-Table.
I don't think that the CmdLet supports this (or at least if there's a parameter to do this I couldn't find it).
What would the best way to leave only the lines with the actual values from the output of Format-Table?
Try the -HideTableHeaders parameter to Format-Table:
gci | ft -HideTableHeaders
(I'm using PowerShell v2. I don't know if this was in v1.)
Try -ExpandProperty. For example, I use this for sending the clean variable to Out-Gridview -PassThru , otherwise the variable has the header info stored. Note that these aren't great if you want to return more than one property.
An example:
Get-ADUser -filter * | select name -expandproperty name
Alternatively, you could do this:
(Get-ADUser -filter * ).name
The -HideTableHeaders parameter unfortunately still causes the empty lines to be printed (and table headers appearently are still considered for column width). The only way I know that could reliably work here would be to format the output yourself:
| % { '{0,10} {1,20} {2,20}' -f $_.Operation,$_.AttributeName,$_.AttributeValue }
Here is how I solve this. I just pipe the output to Out-String and then pass that output to the .NET Trim function:
(gci | ft -HideTableHeaders | Out-String).Trim()
This will strip out the line breaks before and after the table.
You can also use TrimStart to just take care of the header's line break if you still want the trailing line breaks.
(gci | ft -HideTableHeaders | Out-String).TrimStart()
Another approach is to use ForEach-Object to project individual items to a string and then use the Out-String CmdLet to project the final results to a string or string array:
gci Microsoft.PowerShell.Core\Registry::HKEY_CLASSES_ROOT\CID | foreach { "CID Key {0}" -f $_.Name } | Out-String
#Result: One multi-line string equal to:
#"
CID Key HKEY_CLASSES_ROOT\CID\2a621c8a-7d4b-4d7b-ad60-a957fd70b0d0
CID Key HKEY_CLASSES_ROOT\CID\2ec6f5b2-8cdc-461e-9157-ffa84c11ba7d
CID Key HKEY_CLASSES_ROOT\CID\5da2ceaf-bc35-46e0-aabd-bd826023359b
CID Key HKEY_CLASSES_ROOT\CID\d13ad82e-d4fb-495f-9b78-01d2946e6426
"#
gci Microsoft.PowerShell.Core\Registry::HKEY_CLASSES_ROOT\CID | foreach { "CID Key {0}" -f $_.Name } | Out-String -Stream
#Result: An array of single line strings equal to:
#(
"CID Key HKEY_CLASSES_ROOT\CID\2a621c8a-7d4b-4d7b-ad60-a957fd70b0d0",
"CID Key HKEY_CLASSES_ROOT\CID\2ec6f5b2-8cdc-461e-9157-ffa84c11ba7d",
"CID Key HKEY_CLASSES_ROOT\CID\5da2ceaf-bc35-46e0-aabd-bd826023359b",
"CID Key HKEY_CLASSES_ROOT\CID\d13ad82e-d4fb-495f-9b78-01d2946e6426")
The benefit of this approach is that you can store the result to a variable and it will NOT have any empty lines.
I know it's 2 years late, but these answers helped me to formulate a filter function to output objects and trim the resulting strings. Since I have to format everything into a string in my final solution I went about things a little differently.
Long-hand, my problem is very similar, and looks a bit like this
$verbosepreference="Continue"
write-verbose (ls | ft | out-string) # this generated too many blank lines
Here is my example:
ls | Out-Verbose # out-verbose formats the (pipelined) object(s) and then trims blanks
My Out-Verbose function looks like this:
filter Out-Verbose{
Param([parameter(valuefrompipeline=$true)][PSObject[]]$InputObject,
[scriptblock]$script={write-verbose "$_"})
Begin {
$val=#()
}
Process {
$val += $inputobject
}
End {
$val | ft -autosize -wrap|out-string |%{$_.split("`r`n")} |?{$_.length} |%{$script.Invoke()}
}
}
Note1: This solution will not scale to like millions of objects(it does not handle the pipeline serially)
Note2: You can still add a -noheaddings option.
If you are wondering why I used a scriptblock here, that's to allow overloading like to send to disk-file or other output streams.