I have data in this shape:
externalName,day,workingHours,hoursAndMinutes
PRJF,1,11,11:00
PRJF,2,11,11:00
PRJF,3,0,0:00
PRJF,4,0,0:00
CFAW,1,11,11:00
CFAW,2,11,11:00
CFAW,3,11,11:00
CFAW,4,11,11:00
CFAW,5,0,0:00
CFAW,6,0,0:00
and so far code is
$gdata = Import-csv $filepath\$filename | Group-Object -Property Externalname;
$test = #()
$test += foreach($rostername in $gdata) {
$rostername.Group | Select -Unique externalName,
#{Name = 'AllDays';Expression = {(($rostername.Group) | measure -Property day).count}},
}
$test;
What I can't work out is how to do a conditional count of the lines where day is non-zero.
The aim is to produce two lines:
PRJF, 4, 2, 11
CFAW, 6, 4, 11
i.e. Roster name, roster length, days on, average hours worked per day on.
You need a where-object to filter for non zero workinghours
I'd use a [PSCustomObject] to generate a new table
EDIT a bit more efficient with only one Measure-Object
## Q:\Test\2018\08\06\SO_51700660.ps1
$filepath = 'Q:\Test\2018\08\06'
$filename = 'SO_S1700660.csv'
$gdata = Import-Csv (Join-Path $filepath $filename) | Group-Object -Property Externalname
$test = ForEach($Roster in $gdata) {
$WH = ($Roster.Group.Workinghours|Where-Object {$_ -ne 0}|Measure-Object -Ave -Sum)
[PSCustomObject]#{
RosterName = $Roster.Name
RosterLength = $Roster.Count
DaysOn = $WH.count
AvgHours = $WH.Average
TotalHours = $WH.Sum
}
}
$test | Format-Table
Sample output:
> .\SO_51700660.ps1
RosterName RosterLength DaysOn AvgHours TotalHours
---------- ------------ ------ -------- ----------
PRJF 4 2 11 22
CFAW 6 4 11 44
Related
Got a .ps where im getting alarmgroups from several files. Im trying to add them to an Object but the Problem is every new file hes adding another header into the Object. Is there a possibility, adding the header only 1 time. Append my data to the Object and when hes finished sorting the hole Object?
My Code.
$rootPath = $PSScriptRoot
if ($rootPath -eq "") {
$rootPath = Split-Path -Parent -Path $MyInvocation.MyCommand.Definition
}
$alarmPath = "$rootPath\Alarmgroups"
$mdi_alarms_template = "$rootPath\tmpl\mdi-alarms.tmpl.html"
$mdi_alarms = "$rootPath\mdi-alarms.html"
$fileNames = Get-ChildItem -Path $alarmPath -Filter *.algrp
$AlarmgroupIndexString = $LanguageIDString = $tmpString = $ID_string = $html_output= ""
$MachineCode = "Out_2.Alarm.Current"
$BitNo = $Element = $Format2Element = $Format3BitID =0
$BitID = $Format3TextID = 1
$list = New-Object System.Collections.ArrayList
Clear-Content "$rootPath\test.txt"
Clear-Content "$rootPath\list.txt"
Clear-Content "$rootPath\output.txt"
# Parse each alarm group file in the project
foreach ($file in $fileNames) {
$Content = [xml](Get-Content -Path $file.FullName)
$ns = New-Object System.Xml.XmlNamespaceManager($Content.NameTable)
$ns=#{asdf="http://asdf-automation.co.at/AS/VC/Project"}
$AlarmgroupIndex = Select-Xml -Xml $Content -XPath "//asdf[contains(#Name,'Index')]" -namespace $ns | select -ExpandProperty node
$AlarmgroupIndexString = $AlarmgroupIndex.Value
$AlarmgroupLanguageText = Select-Xml -Xml $Content -XPath "//asdf:TextLayer" -namespace $ns | select -ExpandProperty node
$AlarmgroupIndexMap = Select-Xml -Xml $Content -XPath "//asdf:Index" -namespace $ns | select -ExpandProperty node
$LUT =#{}
$AlarmgroupIndexMap | foreach{
$LUT.($_.ID) = $_.Value
}
$tmpArray =#()
$list = $AlarmgroupLanguageText | foreach{
$LanguageIDString = $_.LanguageId
$AlarmgroupTextLayer = Select-Xml -Xml $Content -XPath "//asdf:TextLayer[#LanguageId='$LanguageIDString']/asdf:Text" -namespace $ns | select -ExpandProperty node
$AlarmgroupTextLayer | foreach{
if($LUT.ContainsKey($_.ID))
{
$ID_string = $LUT[$_.ID]
}
[pscustomobject]#{
Language = $LanguageIDString
GroupID = $AlarmgroupIndexString
TextID = $ID_string #-as [int]
Text = $_.Value
}
$ID_string =""
}
$LanguageIDString=""
}
$list = $list |Sort-Object -Property Language, GroupID, {$_.TextID -as [int]}
# $list = $list |Sort-Object -Property #{Expression={$_.Language}}, #{Expression={$_.TextId}} , #{Expression={$_.TextID -as [int]}}
$list | Out-File "$rootPath\list.txt" -Append -Encoding utf8
Output:
GroupID Language TextID Text
------- -------- ------ ----
24 aa Group
24 aa 0
24 aa 1
24 aa 2
24 aa 3
24 aa 4
24 aa 5
24 aa 6
24 aa 7
24 aa 8
24 aa 9
24 aa 10
GroupID Language TextID Text
------- -------- ------ ----
24 ar Group
24 ar 0
24 ar 1
24 ar 2
24 ar 3
24 ar 4
24 ar 5
24 ar 6
24 ar 7
So i have several headers in my outputfile. Is it possible to erase them or add elements to the Object without the header. Tried several solution nothing worked.
If i understand it correctly im generating an Object and add it to an object with all values incl. header.
[pscustomobject]#{
Language = $LanguageIDString
GroupID = $AlarmgroupIndexString
TextID = $ID_string #-as [int]
Text = $_.Value
You can assign all the output from the foreach loop to a single variable and then move the file-write logic to the end of the script where you can output it all at once:
# Assign results of entire `foreach(){}` statement to `$combinedLists`
$combinedLists = foreach ($file in $fileNames) {
# XML navigation + object creation + assignment to `$list` still goes here
# We sort and then instead of assigning the output to a variable directly,
# we just let it "bubble up" from here to the `$combinedList` assignment
$list |Sort-Object -Property Language, GroupID, {$_.TextID -as [int]}
}
# Now we can write everything to file at once (overwrites existing contents)
$combinedLists |Out-File "$rootPath\list.txt" -Force -Encoding utf8
Wondering if someone would be able to help me. Problem is that I'm trying to Import , Group, Sum and the Export a CSV. The problem is that my CSV has a unknown number of columns of the following format.
GroupA,GroupB,GroupC,ValueA,ValueB,ValueC,ValueD...
GroupA, B and C are constant and the fields I want to group by - I know the names of these fields in advance. The problem is there are an unknown number of Value columns - all of which I want to Sum (and don't know the names of in advance.)
I'm comfortable getting this code working if I know the name of the Value fields and have a fixed number of Value Fields. But I'm struggling to get code for unknown names and number of columns.
$csvImport = import-csv 'C:\input.csv'
$csvGrouped = $csvImport | Group-Object -property GroupA,GroupB,GroupC
$csvGroupedFinal = $csvGrouped | Select-Object #{Name = 'GroupA';Expression={$_.Values[0]}},
#{Name = 'GroupB';Expression={$_.Values[1]}},
#{Name = 'GroupC';Expression={$_.Values[2]}},
#{Name = 'ValueA' ;Expression={
($_.Group|Measure-Object 'ValueA' -Sum).Sum
}}
$csvGroupedFinal | Export-Csv 'C:\output.csv' -NoTypeInformation
Example Input Data -
GroupA, GroupB, Value A
Sam, Apple, 10
Sam, Apple, 20
Sam, Orange, 50
Ian, Apple, 15
Output Data -
GroupA, GroupB, Value A
Sam, Apple, 30
Sam, Orange, 50
Ian, Apple, 15
The following script should work. Pay your attention to the $FixedNames variable:
$csvImport = #"
Group A,Group B,Value A
sam,apple,10
sam,apple,20
sam,orange,50
ian,apple,15
"# | ConvertFrom-Csv
$FixedNames = #('Group A', 'Group B', 'Group C')
# $aux = ($csvImport|Get-Member -MemberType NoteProperty).Name ### sorted (wrong)
$aux = ($csvImport[0].psobject.Properties).Name ### not sorted
$auxGrpNames = #( $aux | Where-Object {$_ -in $FixedNames})
$auxValNames = #( $aux | Where-Object {$_ -notin $FixedNames})
$csvGrouped = $csvImport | Group-Object -property $auxGrpNames
$csvGroupedFinal = $csvGrouped |
ForEach-Object {
($_.Name.Replace(', ',','), (($_.Group |
Measure-Object -Property $auxValNames -Sum
).Sum -join ',')) -join ','
} | ConvertFrom-Csv -Header $aux
$csvGroupedFinal
Tested likewise for
$csvImport = #"
Group A,Group B,Value A,Value B
sam,apple,10,1
sam,apple,20,
sam,orange,50,5
ian,apple,15,51
"# | ConvertFrom-Csv
as well as for more complex data of Group A,Group B,Group C,Value A,Value B header.
Edit updated according to the beneficial LotPings' comment.
After importing this script splits the properties (columns) into Groups / Values
It groups dynamically and sums on only value fields independent of the number
The input ordering is maintained with a final Select-Object
## Q:\Test\2019\01\17\SO_54237887.ps1
$csvImport = Import-Csv '.\input.csv'
$Cols = ($csvImport[0].psobject.Properties).Name
# get list of group columns by name and wildcard
$GroupCols = $Cols | Where-Object {$_ -like 'Group*'}
# a different approach would be to select a number of leading columns
# $GroupCols = $Cols[0..1]
$ValueCols = $Cols | Where-Object {$_ -notin $GroupCols}
$OutCols = ,'Groups' + $ValueCols
$csvGrouped = $csvImport | Group-Object $GroupCols | ForEach-Object{
$Props = #{Groups=$_.Name}
ForEach ($ValCol in $ValueCols){
$Props.Add($ValCol,($_.Group|Measure-Object $ValCol -Sum).Sum)
}
[PSCustomObject]$Props
}
$csvGrouped | Select-Object $OutCols
With this sample input file
GroupA GroupB ValueA ValueB
------ ------ ------ ------
Sam Apple 10 15
Sam Apple 20 25
Sam Orange 50 75
Ian Apple 15 20
Sample output for any number of Groups and values
Groups ValueA ValueB
------ ------ ------
Sam, Apple 30 40
Sam, Orange 50 75
Ian, Apple 15 20
Without any change in code it does process data from Hassans answer too:
Groups ValueA ValueB ValueC
------ ------ ------ ------
Sam, Apple 30 4 20
Sam, Orange 50 4 5
Ian, Apple 15 3 3
script1.ps1
Import-Csv 'input.csv' | `
Group-Object -Property GroupA,GroupB | `
% {$b=$_.name -split ', ';$c=($_.group | `
Measure-Object -Property Value* -Sum).Sum;
[PScustomobject]#{GroupA=$b[0];
GroupB=$b[1];
Sum=($c | Measure-Object -Sum).Sum }}
input.csv
GroupA, GroupB, ValueA, ValueB, ValueC
Sam, Apple, 10, 1, 10
Sam, Apple, 20, 3, 10
Sam, Orange, 50, 4, 5
Ian, Apple, 15, 3, 3
OUTPUT
PS D:\coding> .\script1.ps1
GroupA GroupB Sum
------ ------ ---
Sam Apple 54
Sam Orange 59
Ian Apple 21
Given the following array of objects:
Email Domain Tally
----- ----- -----
email1#domainA.com domainA.com 4
email1#domainB.com domainB.com 1
email2#domainC.com domainC.com 6
email4#domainA.com domainA.com 1
I'd like to "group by" Domain and add up Tally as I go. The end result would like this:
Domain Tally
------ -----
domainA.com 5
domainB.com 1
domainC.com 6
I have something that works but I feel like it's overly complicated.
$AllTheAddresses = Get-AllTheAddresses
$DomainTally = #()
foreach ($Addy in $AllTheAddresses)
{
if ($DomainTally | Where-Object {$_.RecipientDomain -eq $Addy.RecipientDomain})
{
$DomainTally |
Where-Object {$_.RecipientDomain -eq $Addy.RecipientDomain} |
ForEach-Object {$_.Tally += $Addy.Tally }
}
else
{
$props = #{
RecipientDomain = $Addy.RecipientDomain
Tally = $Addy.Tally
}
$DomainTally += New-Object -TypeName PSObject -Property $props
}
}
In my example, I'm creating the addresses as hashtables, but PowerShell will let you refer to the keys by .Property similar to an object.
If you're truly just summing by the Domain, then it seems like you don't need anything more complicated than a HashTable to create your running total.
The basic summation:
$Tally = #{}
$AllTheAddresses | ForEach-Object {
$Tally[$_.Domain] += $_.Tally
}
Using this sample data...
$AllTheAddresses = #(
#{ Email = "email1#domainA.com"; Domain = "domainA.com"; Tally = 4 };
#{ Email = "email1#domainB.com"; Domain = "domainB.com"; Tally = 1 };
#{ Email = "email1#domainC.com"; Domain = "domainC.com"; Tally = 6 };
#{ Email = "email1#domainA.com"; Domain = "domainA.com"; Tally = 1 }
)
And you get this output:
PS> $tally
Name Value
---- -----
domainC.com 6
domainB.com 1
domainA.com 5
Here is a "PowerShellic" version, notice the piping and flow of the data.
You could of course write this as a one liner (I did originally before I posted the answer here). The 'better' part of this is using the Group-Object and Measure-Object cmdlets. Notice there are no conditionals, again because the example uses the pipeline.
$AllTheAddresses |
Group-Object -Property Domain |
ForEach-Object {
$_ |
Tee-Object -Variable Domain |
Select-Object -Expand Group |
Measure-Object -Sum Tally |
Select-Object -Expand Sum |
ForEach-Object {
New-Object -TypeName PSObject -Property #{
'Domain' = $Domain.Name
'Tally' = $_
}
} |
Select-Object Domain, Tally
}
A more terse version
$AllTheAddresses |
Group Domain |
% {
$_ |
Tee-Object -Variable Domain |
Select -Expand Group |
Measure -Sum Tally |
Select -Expand Sum |
% {
New-Object PSObject -Property #{
'Domain' = $Domain.Name
'Tally' = $_
}
} |
Select Domain, Tally
}
Group-Object is definitely the way to go.
In the interest of terseness:
Get-AllTheAddresses |Group-Object Domain |Select-Object #{N='Domain';E={$_.Name}},#{N='Tally';E={($_.Group.Tally |Measure-Object).Sum}}
Given:
$column1 = #(1,2,3)
$column2 = #(4,5,6)
How can I combine them into an object $matrix which gets displayed as a matrix with the single arrays as columns:
column1 column2
------- -------
1 4
2 5
3 6
It seems that all of my solutions today requires calculated properties. Try:
$column1 = #(1,2,3)
$column2 = #(4,5,6)
0..($column1.Length-1) | Select-Object #{n="Id";e={$_}}, #{n="Column1";e={$column1[$_]}}, #{n="Column2";e={$column2[$_]}}
Id Column1 Column2
-- ------- -------
0 1 4
1 2 5
2 3 6
If the lengths of the arrays are not equal, you could use:
$column1 = #(1,2,3)
$column2 = #(4,5,6,1)
$max = ($column1, $column2 | Measure-Object -Maximum -Property Count).Maximum
0..$max | Select-Object #{n="Column1";e={$column1[$_]}}, #{n="Column2";e={$column2[$_]}}
I wasn't sure if you needed the Id, so I included it in the first sample to show how to include it.
Little better, maybe:
$column1 = #(1,2,3)
$column2 = #(4,5,6,7)
$i=0
($column1,$column2 | sort length)[1] |
foreach {
new-object psobject -property #{
loess = $Column1[$i]
lowess = $column2[$i++]
}
} | ft -auto
loess lowess
----- ------
1 4
2 5
3 6
7
Here's something I created today. It takes a range of 0 to one of the column lengths, then maps it to a list of hashes. Use the select to turn it into a proper table.
$table = 0..$ColA.Length | % { #{
ColA = $ColA[$_]
ColB = $ColB[$_]
}} | Select ColA, ColB
Using the following variables:
$ColA = #(1, 2, 3)
$ColB = #(4, 5, 6)
Results in
ColB ColA
---- ----
1 4
2 5
3 6
I came up with this.. but it seems too verbose. Anything shorter?
&{
for ($i=0; $i -lt $y.Length; $i++) {
New-Object PSObject -Property #{
y = $y[$i]
loess = $smooth_loess[$i]
lowess = $smooth_lowess[$i]
}
}
} | Format-Table -AutoSize
Here is a combination of mjolinor and Frode F. solutions. I ran into some problems using Frode's object construction trick using select-object. For some reason it would output hash values likely representing object references. I only code in PowerShell a few times a year, so I am just providing this in case anyone else finds it useful (perhaps even my future self).
$column1 = #(1,2,3)
$column2 = #(4,5,6,7)
$column3 = #(2,5,5,2,1,3);
$max = (
$column1,
$column2,
$column3 |
Measure-Object -Maximum -Property Count).Maximum;
$i=0
0..$max |
foreach {
new-object psobject -property #{
col1 = $Column1[$i]
col3 = $column3[$i]
col2 = $column2[$i++]
}
} | ft -auto
I have:
$report.gettype().name
Object[]
echo $report
Item Average
-- -------
orange 0.294117647058824
orange -0.901960784313726
orange -0.901960784313726
grape 9.91335740072202
grape 0
pear 3.48736462093863
pear -0.0324909747292419
pear -0.0324909747292419
apple 12.1261261261261
apple -0.0045045045045045
I want to create a variable, $total, (such as a hash table) which contains the sum of the 'Average' column for each item, for example,
echo $total
orange -1.5097
grape 9.913
pear 3.423
apple 12.116
Right now I'm thinking of looping through the $report, but it's hell ugly, and I am looking for something more elegant than the following starting point (incomplete):
$tmpPrev = ""
foreach($r in $report){
$tmp = $r.item
$subtotal = 0
if($tmp <> $tmpPrev){
$subtotal += $r.average
}
How could I do this?
Cmdlets Group-Object and Measure-Object help to solve the task in a PowerShell-ish way:
Code:
# Demo input
$report = #(
New-Object psobject -Property #{ Item = 'orange'; Average = 1 }
New-Object psobject -Property #{ Item = 'orange'; Average = 2 }
New-Object psobject -Property #{ Item = 'grape'; Average = 3 }
New-Object psobject -Property #{ Item = 'grape'; Average = 4 }
)
# Process: group by 'Item' then sum 'Average' for each group
# and create output objects on the fly
$report | Group-Object Item | %{
New-Object psobject -Property #{
Item = $_.Name
Sum = ($_.Group | Measure-Object Average -Sum).Sum
}
}
Output:
Sum Item
--- ----
3 orange
7 grape
I've got a more command-line solution.
Given $report
$groupreport = $report | Group-Object -Property item -AsHashTable
is
Name Value
---- -----
grape {#{Item=grape; Average=9.91335740072202}, #{Item=grape; Average=0}}
orange {#{Item=orange; Average=0.294117647058824}, #{Item=orange; Average=-0.901960784313726...
apple {#{Item=apple; Average=12.1261261261261}, #{Item=apple; Average=-0.0045045045045045}}
pear {#{Item=pear; Average=3.48736462093863}, #{Item=pear; Average=-0.0324909747292419}, #...
then
$tab=#{}
$groupreport.keys | % {$tab += #{$_ = ($groupreport[$_] | measure-object -Property average -sum)}}
gives
PS> $tab["grape"]
Count : 2
Average :
Sum : 9,91335740072202
Maximum :
Minimum :
Property : Average
PS> $tab["grape"].sum
9,91335740072202
It seems short and usable.
Summary
$groupreport = $report | Group-Object -Property item -AsHashTable
$tab = #{}
$groupreport.keys | % {$tab += #{$_ = ($groupreport[$_] | measure-object -Property average -sum)}}
$tab.keys | % {write-host $_ `t $tab[$_].sum}
I don't know if you can get rid of looping. What about:
$report | % {$averages = #{}} {
if ($averages[$_.item]) {
$averages[$_.item] += $_.average
}
else {
$averages[$_.item] = $_.average
}
} {$averages}