I have a variable results ($result) of several rows of data or object like this:
PS> $result | ft -auto;
name value
---- -----
a 1
a 2
b 30
b 20
....
what I need to get all the rows of name and max(value) like this filtered output:
PS> $result | ? |ft -auto
name value
---- -----
a 2
b 30
....
Not sure what command or filters available (as ? in above) so that I can get each name and only the max value for the name out?
$result | group name | select name,#{n='value';e={ ($_.group | measure value -max).maximum}}
This should do the trick:
PS> $result | Foreach {$ht=#{}} `
{if ($_.Value -gt $ht[$_.name].Value) {$ht[$_.Name]=$_}} `
{$ht.Values}
This is essentially using the Begin/Process/End scriptblock parameters of the Foreach-Object cmdlet to stash input objects with a max value based on a key into a hashtable.
Note: watch out for extra spaces after the line continuation character (`) - there shouldn't be any.
Related
I'm confused how the $_ variable works in certain contexts of piping. In this example for backing up a Bitlocker key:
Get-BitlockerVolume | % {$_.KeyProtector | ? RecoveryPassword | Backup-BitlockerKeyProtector -MountPoint $_.MountPoint}
This is how I read it in English:
Get all BitLockerVolume objects
For each BitLockerVolume object, pipe the KeyProtector fields forwards
Pipe KeyProtector objects forwards further for those with a RecoverPassword
Run the Backup-BitlockerKeyProtector, and supply the MountPoint
However, MountPoint is a field of the BitLockerVolume object, as shown here:
PS C:\Windows\system32> Get-BitLockerVolume | Get-Member | Where-Object {$_.Name -eq "MountPoint"}
TypeName: Microsoft.BitLocker.Structures.BitLockerVolume
Name MemberType Definition
---- ---------- ----------
MountPoint Property string MountPoint {get;}
So, for the entire block wrapped in brakcets { }, will the $_ variable ALWAYS be the same through any amount of piping? For example, the object we are piping forwards is changing. It's no longer a BitLockerVolume Object, but instead a KeyProtector object. So will the $_ always refer to the BitLockerVolume object in this case, or will it change further down the pipeline depending on different types of objects piped further through the chain?
So $_ is the info from the current pipe.
1,2 | %{
$_
}
response
1
2
while
1,2 | %{
"a","b" | %{
$_
}
}
response
a
b
a
b
We can see in the first that the output from %_ is from the last info given which is 1,2. While the next example still loops 1,2 but the output is from the pipe inside a,b.
There are ways around this by storing the first pipe information into a variable in the second pipe
1,2 | %{
$Num = $_
"a","b" | %{
$Num
}
}
which case the output is
1
1
2
2
In the example you gave lets look at it formated
Get-BitlockerVolume | % {
$_.KeyProtector | ? RecoveryPassword | Backup-BitlockerKeyProtector -MountPoint $_.MountPoint
}
You have 2 different pipes. The First is getting 'BitlockerVolumevolume'.
The second starts with you sending the BitlockerVolume's KeyProtector.
Its like saying
For each Bitlocker volume, Get KeyProtector.
For each KeyProtector, Get me ones that have the member RecoveryPassword
Foreach KeyProtector with member RecoveryPassword, Backup Bitlocker Key Protector Using KeyProtector's Mountpoints
So on one final note I would also assume the example you gave wouldnt work.
What you might be looking for is this...
Get-BitlockerVolume | % {
$MountPoint = $_.MountPoint
$_.KeyProtector | ? RecoveryPassword | Backup-BitlockerKeyProtector -MountPoint $MountPoint -KeyProtectorId $_.KeyProtectorId
}
Let's expand the aliases and fill in the implied parameters. $_ can only be used inside script blocks '{ }' that are options to cmdlets. Just because you're in a pipe, doesn't mean you can use $_ . The $_ here belongs to Foreach-Object. Where-Object is using a comparison statement.
Get-BitlockerVolume | Foreach-Object -Process {
$_.KeyProtector | Where-Object -Property RecoveryPassword |
Backup-BitlockerKeyProtector -MountPoint $_.MountPoint
}
I know there are already good answers here, but I feel one important question was not addressed. The question of what happens to $_ throughout the Foreach-Object {} block when there is nesting. I am going use ArcSet's example since that was the answer selected.
1,2 | % {
"$_ before second foreach"
'a','b' | % {
"$_ inside second foreach"
}
"$_ after second foreach"
}
1 before second foreach
a inside second foreach
b inside second foreach
1 after second foreach
2 before second foreach
a inside second foreach
b inside second foreach
2 after second foreach
Notice that $_ becomes the current object being processed by the code within the Foreach-Object {} blocks. When entering the second Foreach-Object block, $_ changes. When exiting the second Foreach-Object block, $_ changes back to the object that will be continued to be processed by the remainder of the first block. So $_ neither remains the same nor is lost during the block processing. You will need to either assign $_ as another variable or in applicable situations use the -PipelineVariable switch to access those objects within different blocks.
Id' like to build on ArcSet's answer just a little bit. Since I finally understood the value of the $PSItem is changing when the type changes down the pipeline, I ran this code to do a little check.
Get-BitLockerVolume | % {$_.GetType()}
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True False BitLockerVolume System.Object
True False BitLockerVolume System.Object
True False BitLockerVolume System.Object
Here we can see some objects returned by the pipeline are of the BitLockerVolume type.
Now, based on my original question/example, if we pipe it further based on KeyProtector the object type will change for the $PSItem variable.
Get-BitLockerVolume | % { $_.KeyProtector | ? RecoveryPassword | % {$_.GetType()}}
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True False BitLockerVolumeKeyProtector System.Object
True False BitLockerVolumeKeyProtector System.Object
So at this point, at the end of the pipeline, we execute some other cmdlet like Backup-BitlockerKeyProtector and reference the $PSItem variable, aka $_, then it will refer to the object types last passed through the pipeline, in this case, the objects would be of the BitLockerVolumeKeyProtector type.
I have two CSV Files which look like this:
test.csv:
"Col1","Col2"
"1111","1"
"1122","2"
"1111","3"
"1121","2"
"1121","2"
"1133","2"
"1133","2"
The second looks like this:
test2.csv:
"Number","signs"
"1111","ABC"
"1122","DEF"
"1111","ABC"
"1121","ABC"
"1133","GHI"
Now the goal is to get a summary of all points from test.csv assigned to the "signs" of test2.csv. Reference are the numbers, as you may see.
Should be something like this:
ABC = 8
DEF = 2
GHI = 4
I have tried to test this out but cannot get the goal. What I have so far is:
$var = "C:\PathToCSV"
$csv1 = Import-Csv "$var\test.csv"
$csv2 = Import-Csv "$var\test2.csv"
# Process: group by 'Item' then sum 'Average' for each group
# and create output objects on the fly
$test1 = $csv1 | Group-Object Col1 | ForEach-Object {
New-Object psobject -Property #{
Col1 = $_.Name
Sum = ($_.Group | Measure-Object Col2 -Sum).Sum
}
}
But this gives me back the following output:
Ps> $test1
Sum Col1
--- ----
4 1111
2 1122
4 1121
4 1133
I am not able to get the summary and the mapping of the signs.
Not sure if I understand your question correctly, but I'm going to assume that for each value from the column "signs" you want to lookup the values from the column "Number" in the second CSV and then calculate the sum of the column "Col2" for all matches.
For that I'd build a hashtable with the pre-calculated sums for the unique values from "Col1":
$h1 = #{}
$csv1 | ForEach-Object {
$h1[$_.Col1] += [int]$_.Col2
}
and then build a second hashtable to sum up the lookup results for the values from the second CSV:
$h2 = #{}
$csv2 | ForEach-Object {
$h2[$_.signs] += $h1[$_.Number]
}
However, that produced a different value for "ABC" than what you stated as the desired result in your question when I processed your sample data:
Name Value
---- -----
ABC 12
GHI 4
DEF 2
Or did you mean you want to sum up the corresponding values for the unique numbers for each sign? For that you'd change the second code snippet to something like this:
$h2 = #{}
$csv2 | Group-Object signs | ForEach-Object {
$name = $_.Name
$_.Group | Select-Object -Unique -Expand Number | ForEach-Object {
$h2[$name] += $h1[$_]
}
}
That would produce the desired result from your question:
Name Value
---- -----
ABC 8
GHI 4
DEF 2
I have seen powershell script which also I have in mind. What I would like to add though is another column which would show the side indicator comparators ("==", "<=", "=>") and be named them as MATCH(if "==") and MISMATCH(if "<=" and "=>").
Any advise on how I would do this?
Here is the link of the script (Credits to Florent Courtay)
How can i reorganise powershell's compare-object output?
$a = Compare-Object (Import-Csv 'C:\temp\f1.csv') (Import-Csv 'C:\temp\f2.csv') -property Header,Value
$a | Group-Object -Property Header | % { New-Object -TypeName psobject -Property #{Header=$_.name;newValue=$_.group[0].Value;oldValue=$_.group[1].Value}}
========================================================================
The output I have in mind:
Header1 Old Value New Value STATUS
------ --------- --------- -----------
String1 Value 1 Value 2 MATCH
String2 Value 3 Value 4 MATCH
String3 NA Value 5 MISMATCH
String4 Value 6 NA MISMATCH
Here's a self-contained solution; simply replace the ConvertFrom-Csv calls with your Import-Csv calls:
# Sample CSV input.
$csv1 = #'
Header,Value
a,1
b,2
c,3
'#
$csv2 = #'
Header,Value
a,1a
b,2
d,4
'#
Compare-Object (ConvertFrom-Csv $csv1) (ConvertFrom-Csv $csv2) -Property Header, Value |
Group-Object Header | Sort-Object Name | ForEach-Object {
$newValIndex, $oldValIndex = ((1, 0), (0, 1))[$_.Group[0].SideIndicator -eq '=>']
[pscustomobject] #{
Header = $_.Name
OldValue = ('NA', $_.Group[$oldValIndex].Value)[$null -ne $_.Group[$oldValIndex].Value]
NewValue = ('NA', $_.Group[$newValIndex].Value)[$null -ne $_.Group[$newValIndex].Value]
Status = ('MISMATCH', 'MATCH')[$_.Group.Count -gt 1]
}
}
The above yields:
Header OldValue NewValue Status
------ -------- -------- ------
a 1 1a MATCH
c 3 NA MISMATCH
d NA 4 MISMATCH
Note:
The assumption is that a given Header column value appears at most once in each input file.
The Sort-Object Name call is needed to sort the output by Header valuesThanks, LotPings.
, because, due to how Compare-Object orders its output (right-side-only items first), the order of groups created by Group-Object would not automatically reflect the 1st CSV's order of header values (d would appear before c).
The powershell script:
$test = import-csv “C:\CSVFiles\test.csv”
ForEach ($item in $test)
{
$Name = $item.(“Name”)
$property = $item.("property")
$location = $item.(“location”)
Write-Output "$Name=$Name"
Write-Output "Property=$property"
Write-Output "Location=$location"
}
This script shows all the data for name,property and location for each row. I want the results to only show the data of one row;for example the row: n1;13;computer
The Cvs file =
Name;property;location
n1;13;computer
n2;65;computer
n3;12;tablet
n4;234;phone
n5;123;phone
n6;125;phone
What the current script spits out:
Name=n1
Property=13
Location=computer
Name=n2
Property=65
Location=computer
Name= n3
Property=12
Location=tablet
Name=n4
Property=234
Location=phone
Name=n5
Property=123
Location=phone
Name=n6
Property=125
Location=phone
There are many ways to select a Row of a csv and to present the data
For demonstration I use an inline csv with a here string.
$Test = #"
Name;property;location
n1;13;computer
n2;65;computer
n3;12;tablet
n4;234;phone
n5;123;phone
n6;125;phone
"# | ConvertFrom-Csv -delimiter ';'
> $test[0]
Name property location
---- -------- --------
n1 13 computer
> $test | where-object Name -eq 'n1'
Name property location
---- -------- --------
n1 13 computer
> $test | where-object Name -eq 'n1' | Select-Object Name,property
Name property
---- --------
n1 13
> $test | where-object Name -eq 'n1' | ForEach-Object {"Name:{0} has property: {1}" -f $_.Name,$_.property}
Name:n1 has property: 13
Once imported the csv rows contents are converted to objects
If you want to get the original row of the csv matching a criteria don't import but:
> Get-Content "C:\CSVFiles\test.csv" | Select-String '^n1'
n1;13;computer
^n1 is a regular expression anchoring the pattern at line begin.
Select-String -Path "C:\CSVFiles\test.csv" -Pattern '^n1'
Is the same without a pipe
So your current output is having 3 objects that are having 3 headers majorly.
One is Name; Second one is Property and the third one is Location.
As part of solution, you can either pull the records by specifying the index value or you can use the .Headername to pull all the sets of same object. Like:
Avoiding Foreach and accessing with $test[0] or $test[1]
Or you can use like: $test.Name directly to have all of the names from the csv
I'm using the following script to get the comma counts.
Get-Content .\myFile |
% { ($_ | Select-String `, -all).matches | measure | select count } |
group -Property count
It returns,
Count Name Group
----- ---- -----
131 85 {#{Count=85}, #{Count=85}, #{Count=85}, #{Count=85}...}
3 86 {#{Count=86}, #{Count=86}, #{Count=86}}
Can I show the line number in the Group column instead of #{Count=86}, ...?
The files will have a lot of lines and majority of the lines have the same comma. I want to group them so the output lines will be smaller
Can you use something like this?
$s = #"
this,is,a
test,,
with,
multiple, commas, to, count,
"#
#convert to string-array(like you normally have with multiline strings)
$s = $s -split "`n"
$s | Select-String `, -AllMatches | Select-Object LineNumber, #{n="Count"; e={$_.Matches.Count}} | Group-Object Count
Count Name Group
----- ---- -----
2 2 {#{LineNumber=1; Count=2}, #{LineNumber=2; Count=2}}
1 1 {#{LineNumber=3; Count=1}}
1 4 {#{LineNumber=4; Count=4}}
If you don't want the "count" property multiple times in the group, you need custom objects. Like this:
$s | Select-String `, -AllMatches | Select-Object LineNumber, #{n="Count"; e={$_.Matches.Count}} | Group-Object Count | % {
New-Object psobject -Property #{
"Count" = $_.Name
"LineNumbers" = ($_.Group | Select-Object -ExpandProperty LineNumber)
}
}
Output:
Count LineNumbers
----- -----------
2 {1, 2}
1 3
4 4