Pivot an array of array? - powershell

The following code defined a matrix.
$a = #('a','b','x',10),
#('a','b','y',20),
#('c','e','x',50),
#('c','e','y',30)
$a | % { "[$_]"}
I want to pivot the array by x and y. The expected result array should be
[a b 10 20]
[c e 50 30]
- - -- --
x y
I think it needs group-object and then mapping. How to use group-object on an array?
(BTW, why the question has been down voted twice?)

You can't use Group-Object with an array (at least not the way you want) since Group-Object works on object properties. A workaround is to organize your rows into a label that you want to group on, followed by the values to assign to the group. Then you can group on the label:
$a | %{
new-object PsObject -prop #{"label" = "$($_[0]),$($_[1])"; value=#{ $_[2]=$_[3]}}
} | Group-Object label
So, then you have a group with your entries stroed as an array of hashtables within each group:
Count Name Group
----- ---- -----
2 a,b {#{value=System.Collections.Hashtable; label=a,b}, #{value=System.Collections.Hashtable; label=a,b}}
2 c,e {#{value=System.Collections.Hashtable; label=c,e}, #{value=System.Collections.Hashtable; label=c,e}}
You can then expand out each row to get the info you desire:
$a | %{
new-object PsObject -prop #{"label" = "$($_[0]),$($_[1])"; value=#{ $_[2]=$_[3]}}
} |
group label | % {
"[$(#($_.Name -split ",") + #($_.Group.value.values))]"
}
which gives:
[a b 10 20]
[c e 50 30]
To answer your second comment, no the above won;t guarantee the order. To guarantee it, you'll have to be explicit:
$a | %{
new-object PsObject -prop #{"label" = "$($_[0]),$($_[1])"; value=#{ $_[2]=$_[3]}}
} |
group label | % {
"[$(#($_.Name -split ",") + #($_.Group.value.x, $_.Group.value.y))]"
}

Related

Convert multiple txt file to one single csv file [duplicate]

When we're trying to export data to other functions via the pipeline, we observe some strange behavior in PowerShell.
Example code:
$Array = #()
$Obj1 = [PSCustomObject]#{
Member1 = 'First'
Member2 = 'Second'
}
$Obj2 = [PSCustomObject]#{
Member1 = 'First'
Member2 = 'Second'
Member3 = 'Third'
}
$Array = $Obj1, $Obj2
$Array | Out-GridView -Title 'Not showing Member3'
$Array = $Obj2, $Obj1
$Array | Out-GridView -Title 'All members correctly displayed'
In the example above you can see that when the first object only contains 2 properties, the Out-GridView CmdLet (and others) only show 2 properties, even though the second object has 3 properties. However, when the first object in the array has 3 properties it does display them all correctly.
Is there a way around this? Because it's not possible to predict up front how many properties on an object there will be and if the object with the most properties will be the first one in the array.
I had the same experience once and created the following reusable 'Union' function:
# 2021-08-25 Removed Union function
Usage:
$Obj1, $Obj2 | Union | Out-GridView -Title 'Showing all members'
It is also supposed to work with complex objects. Some standard cmdlets output multiple object types at once and if you view them (e.g. Out-GridView) or dump them in a file (e.g. Export-Csv) you might miss a lot of properties. Take as another example:
Get-WmiObject -Namespace root/hp/instrumentedBIOS -Class hp_biosSetting | Union | Export-Csv ".\HPBIOS.csv"
Added 2014-09-19:
Maybe this is already between the lines in the comments $Array | Select * | … will not resolve the issue but specifically selecting the properties $Array | Select Member1, Member2, Member3 | … does.
Besides, although in most cases the Union function will work, there are some exceptions to that as it will only align the first object with the rest.
Consider the following object:
$List = #(
New-Object PSObject -Property #{Id = 2}
New-Object PSObject -Property #{Id = 1}
New-Object PSObject -Property #{Id = 3; Name = "Test"}
)
If you Union this object everything appears to be fine and if you e.g. ExportTo-CSV and work with the export .csv file from then on you will never have any issue.
$List | Union
Id Name
-- ----
2
1
3 Test
Still there is a catch as only the first object is aligned. If you e.g. sort the result on Id (Sort Id) or take just the last 2 (Select -Last 2) entries, the Name is not listed because the second object doesn’t contain the Name property:
$List | Union | Sort Id
Id
--
1
2
3
Therefor I have rewritten the Union-Object (Alias Union) function`):
Union-Object
# 2021-08-25 Removed Union-Object function
Syntax:
$Array | Union | Out-GridView -Title 'All members correctly displayed'
Update 2021-08-25
Based on az1d helpful feedback on an error caused by equal property names with different casing, I have created a new UnifyProperties function.
(I will no longer use the name UnionObject for his)
function UnifyProperties {
$Names = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase)
$InputCollected = #($Input)
$InputCollected.ForEach({
foreach ($Name in $_.psobject.Properties.Name) { $Null = $Names.Add($Name) }
})
$inputCollected | Select-Object #($Names)
}
Usage:
[pscustomobject] #{ one = 1; two = 2; three = 3 },
[pscustomobject] #{ ONE = 10; THREE = 30; FOUR = 4 } |
UnifyProperties
one two three FOUR
--- --- ----- ----
1 2 3
10 30 4
See also: #13906 Add -UnifyProperties parameter to Select-Object

Powershell - counting the number of times words from a list are in a csv column - zero value important

I am trying to count the number of times a list of words are in a csv column where a zero value is important.
This code only returns a value if the word is in the csv column and I also want a 0 if the word is not in the column
Import-Csv C:\Users\Work_PC\Documents\TAT\July\Stock.csv -Header Animal | where {$_.Animal -in $searchTerms} | Group-Object Animal -NoElement
Examples stock.csv
Animal,Someothervalue,anothervalue,
Cow,1,2,
Sheep,1,3
Pig,1,4
Cow,1,2,
Sheep,1,3
Pig,1,4
Cow,1,2,
Cow,1,2,
Sheep,1,3
Pig,1,4
Cow,1,2,
example $searchterms
Cow
Sheep
Pig
Horse
Donkey
Using the above code returns this value
Count Name
----- ----
5 Cow
3 Sheep
3 Pig
I would like it to return
Count Name
----- ----
5 Cow
3 Sheep
3 Pig
0 Horse
0 Donkey
Thanks
Group-Object will not list items it is unaware of.
Personally I would use the list of search terms to create objects and assign values, this also allows for additional data to be considered if required:
$SearchTerms = "Cow","Sheep","Pig","Horse","Donkey"
$GrpObjResults = Import-Csv C:\Users\Work_PC\Documents\TAT\July\Stock.csv -Header Animal | where {$_.Animal -in $searchTerms} | Group-Object Animal -NoElement
$ObjProps = #{
Name = [String]
Count = [int]
}
$Results = #()
Foreach($Term in $SearchTerms){
$ListEntry = New-Object -TypeName PSObject -Property $ObjProps
$ListEntry.Name = $Term
$ListEntry.Count = ($GrpObjResults | Where {$_.Name -eq $Term}).Count
$Results += $ListEntry
}
$Results
This will give the desired output of:
Count Name
----- ----
5 Cow
3 Sheep
3 Pig
0 Horse
0 Donkey
It looks from your example, the CSV file already has headers, so you could simply iterate the search terms and do a Select-Object to get what you seek:
$searchTerms = 'Cow','Sheep','Pig','Horse','Donkey'
$csv = Import-Csv 'C:\Users\Work_PC\Documents\TAT\July\Stock.csv'
foreach ($term in $searchTerms) {
'' | Select-Object #{Name = 'Count'; Expression = {($csv | Where-Object {$_.Animal -eq $term}).Count}},
#{Name = 'Name'; Expression = {$term}}
}
Output:
Count Name
----- ----
5 Cow
3 Sheep
3 Pig
0 Horse
0 Donkey

Add up the data if the reference from another file is correct

I have two CSV Files which look like this:
test.csv:
"Col1","Col2"
"1111","1"
"1122","2"
"1111","3"
"1121","2"
"1121","2"
"1133","2"
"1133","2"
The second looks like this:
test2.csv:
"Number","signs"
"1111","ABC"
"1122","DEF"
"1111","ABC"
"1121","ABC"
"1133","GHI"
Now the goal is to get a summary of all points from test.csv assigned to the "signs" of test2.csv. Reference are the numbers, as you may see.
Should be something like this:
ABC = 8
DEF = 2
GHI = 4
I have tried to test this out but cannot get the goal. What I have so far is:
$var = "C:\PathToCSV"
$csv1 = Import-Csv "$var\test.csv"
$csv2 = Import-Csv "$var\test2.csv"
# Process: group by 'Item' then sum 'Average' for each group
# and create output objects on the fly
$test1 = $csv1 | Group-Object Col1 | ForEach-Object {
New-Object psobject -Property #{
Col1 = $_.Name
Sum = ($_.Group | Measure-Object Col2 -Sum).Sum
}
}
But this gives me back the following output:
Ps> $test1
Sum Col1
--- ----
4 1111
2 1122
4 1121
4 1133
I am not able to get the summary and the mapping of the signs.
Not sure if I understand your question correctly, but I'm going to assume that for each value from the column "signs" you want to lookup the values from the column "Number" in the second CSV and then calculate the sum of the column "Col2" for all matches.
For that I'd build a hashtable with the pre-calculated sums for the unique values from "Col1":
$h1 = #{}
$csv1 | ForEach-Object {
$h1[$_.Col1] += [int]$_.Col2
}
and then build a second hashtable to sum up the lookup results for the values from the second CSV:
$h2 = #{}
$csv2 | ForEach-Object {
$h2[$_.signs] += $h1[$_.Number]
}
However, that produced a different value for "ABC" than what you stated as the desired result in your question when I processed your sample data:
Name Value
---- -----
ABC 12
GHI 4
DEF 2
Or did you mean you want to sum up the corresponding values for the unique numbers for each sign? For that you'd change the second code snippet to something like this:
$h2 = #{}
$csv2 | Group-Object signs | ForEach-Object {
$name = $_.Name
$_.Group | Select-Object -Unique -Expand Number | ForEach-Object {
$h2[$name] += $h1[$_]
}
}
That would produce the desired result from your question:
Name Value
---- -----
ABC 8
GHI 4
DEF 2

powershell compare two files and list their columns with side indicator as match/mismatch

I have seen powershell script which also I have in mind. What I would like to add though is another column which would show the side indicator comparators ("==", "<=", "=>") and be named them as MATCH(if "==") and MISMATCH(if "<=" and "=>").
Any advise on how I would do this?
Here is the link of the script (Credits to Florent Courtay)
How can i reorganise powershell's compare-object output?
$a = Compare-Object (Import-Csv 'C:\temp\f1.csv') (Import-Csv 'C:\temp\f2.csv') -property Header,Value
$a | Group-Object -Property Header | % { New-Object -TypeName psobject -Property #{Header=$_.name;newValue=$_.group[0].Value;oldValue=$_.group[1].Value}}
========================================================================
The output I have in mind:
Header1 Old Value New Value STATUS
------ --------- --------- -----------
String1 Value 1 Value 2 MATCH
String2 Value 3 Value 4 MATCH
String3 NA Value 5 MISMATCH
String4 Value 6 NA MISMATCH
Here's a self-contained solution; simply replace the ConvertFrom-Csv calls with your Import-Csv calls:
# Sample CSV input.
$csv1 = #'
Header,Value
a,1
b,2
c,3
'#
$csv2 = #'
Header,Value
a,1a
b,2
d,4
'#
Compare-Object (ConvertFrom-Csv $csv1) (ConvertFrom-Csv $csv2) -Property Header, Value |
Group-Object Header | Sort-Object Name | ForEach-Object {
$newValIndex, $oldValIndex = ((1, 0), (0, 1))[$_.Group[0].SideIndicator -eq '=>']
[pscustomobject] #{
Header = $_.Name
OldValue = ('NA', $_.Group[$oldValIndex].Value)[$null -ne $_.Group[$oldValIndex].Value]
NewValue = ('NA', $_.Group[$newValIndex].Value)[$null -ne $_.Group[$newValIndex].Value]
Status = ('MISMATCH', 'MATCH')[$_.Group.Count -gt 1]
}
}
The above yields:
Header OldValue NewValue Status
------ -------- -------- ------
a 1 1a MATCH
c 3 NA MISMATCH
d NA 4 MISMATCH
Note:
The assumption is that a given Header column value appears at most once in each input file.
The Sort-Object Name call is needed to sort the output by Header valuesThanks, LotPings.
, because, due to how Compare-Object orders its output (right-side-only items first), the order of groups created by Group-Object would not automatically reflect the 1st CSV's order of header values (d would appear before c).

Powershell: Combine single arrays into columns

Given:
$column1 = #(1,2,3)
$column2 = #(4,5,6)
How can I combine them into an object $matrix which gets displayed as a matrix with the single arrays as columns:
column1 column2
------- -------
1 4
2 5
3 6
It seems that all of my solutions today requires calculated properties. Try:
$column1 = #(1,2,3)
$column2 = #(4,5,6)
0..($column1.Length-1) | Select-Object #{n="Id";e={$_}}, #{n="Column1";e={$column1[$_]}}, #{n="Column2";e={$column2[$_]}}
Id Column1 Column2
-- ------- -------
0 1 4
1 2 5
2 3 6
If the lengths of the arrays are not equal, you could use:
$column1 = #(1,2,3)
$column2 = #(4,5,6,1)
$max = ($column1, $column2 | Measure-Object -Maximum -Property Count).Maximum
0..$max | Select-Object #{n="Column1";e={$column1[$_]}}, #{n="Column2";e={$column2[$_]}}
I wasn't sure if you needed the Id, so I included it in the first sample to show how to include it.
Little better, maybe:
$column1 = #(1,2,3)
$column2 = #(4,5,6,7)
$i=0
($column1,$column2 | sort length)[1] |
foreach {
new-object psobject -property #{
loess = $Column1[$i]
lowess = $column2[$i++]
}
} | ft -auto
loess lowess
----- ------
1 4
2 5
3 6
7
Here's something I created today. It takes a range of 0 to one of the column lengths, then maps it to a list of hashes. Use the select to turn it into a proper table.
$table = 0..$ColA.Length | % { #{
ColA = $ColA[$_]
ColB = $ColB[$_]
}} | Select ColA, ColB
Using the following variables:
$ColA = #(1, 2, 3)
$ColB = #(4, 5, 6)
Results in
ColB ColA
---- ----
1 4
2 5
3 6
I came up with this.. but it seems too verbose. Anything shorter?
&{
for ($i=0; $i -lt $y.Length; $i++) {
New-Object PSObject -Property #{
y = $y[$i]
loess = $smooth_loess[$i]
lowess = $smooth_lowess[$i]
}
}
} | Format-Table -AutoSize
Here is a combination of mjolinor and Frode F. solutions. I ran into some problems using Frode's object construction trick using select-object. For some reason it would output hash values likely representing object references. I only code in PowerShell a few times a year, so I am just providing this in case anyone else finds it useful (perhaps even my future self).
$column1 = #(1,2,3)
$column2 = #(4,5,6,7)
$column3 = #(2,5,5,2,1,3);
$max = (
$column1,
$column2,
$column3 |
Measure-Object -Maximum -Property Count).Maximum;
$i=0
0..$max |
foreach {
new-object psobject -property #{
col1 = $Column1[$i]
col3 = $column3[$i]
col2 = $column2[$i++]
}
} | ft -auto