SQL GROUP BY with SUM in PowerShell - powershell

What is the simplest way to write this in PowerShell:
SELECT col1, SUM(col3) AS SumOfValues
FROM dbo.Table
GROUP BY col1
How can I combine Group-Object and Measure-Object?
I have found this answer link, but there must be a simpler way.

Like this? With a calculated property at the end.
'id,amount
1,4
1,5
2,4
2,6
3,3
3,5' | convertfrom-csv | group id | select name,
#{n='Sum';e={$_.group.amount | measure -sum | % sum}}
Name Sum
---- ---
1 9
2 10
3 8

You need to use the property GROUP. The script outputs "$group | Format-Table" which shows what the real issue is.
This is what each row of the Group Looks like :
Count Name Group
----- ---- -----
3 a {#{col1=a; col2=x; col3=1}, #{col1=a; col2=x; col3=2}, #{col1=a; col2=x; col3=3}}
See code below :
$input = #"
col1,col2,col3
a,x,1
a,x,2
a,x,3
b,x,4
b,x,5
b,x,6
c,x,7
c,x,8
c,x,9
"#
$table = $input | ConvertFrom-Csv
$table | Format-Table
$groups = $table | Group-Object {$_.col1}
$outputTable = [System.Collections.ArrayList]::new()
foreach($group in $groups)
{
$group | Format-Table
$newRow = New-Object -TypeName psobject
$newRow | Add-Member -NotePropertyName col1 -NotePropertyValue $group.Name
$sum = 0
$group.Group | foreach{$sum += $_.col3}
Write-Host "sum = " $sum
$newRow | Add-Member -NotePropertyName SumOfValues -NotePropertyValue $sum
$outputTable.Add($newRow) | Out-Null
}
$outputTable | Format-Table

Related

Read Excel values and create variables from column names in PowerShell

Say I have an Excel file:
I need to create n variables called $col1 up to $coln and read in their respective values.
What is the best approch? I have been trying with a hashtable but I need to be able loop through the columns. This is my code. I have not wraped it around a loop yet. I can create the column names manually but I need to be able to index the columns in the Excel file.
$ExcelRowToRead = 2;
$ListOfColumns = #{"Job_name" = 1 ; "Run_time_start" = 2}
$excel = New-Object -ComObject excel.application;
$workbook = $excel.Workbooks.Open("pathtofile.xlsx");
$workbook.sheets.item(1).activate()
$WorkbookTotal=$workbook.Worksheets.item(1)
$ListOfColumns["Job_name"] = $WorkbookTotal.Cells.Item($ExcelRowToRead, 1) # This needs to be an index
$ListOfColumns["Job_name"].Text
Taking Lee_Dailey's advice here and saving first to a CSV:
$excel = New-Object -ComObject excel.application
$workbook = $excel.Workbooks.Open("pathtofile.xlsx")
$sheet = $workbook.Sheets |
Where-Object Name -eq "SheetName" |
Select-Object -First 1
$tempCsv = New-TemporaryFile
$sheet.SaveAs($tempCsv, 6)
$sheetData = Get-Content $tempCsv | ConvertFrom-Csv
At this point you have an object where you can extract its properties via $sheetData.col1, $sheetData.col2, etc. but if you really need them as separate variables, perhaps:
$sheetData |
Get-Member |
Where-Object MemberType -Eq NoteProperty |
ForEach-Object {
New-Variable -Name $_.Name -Value $sheetData.$($_.Name)
}
In case you don't have / want Excel:
via GitHub - dfinke/ImportExcel
> # Install-Package ImportExcel
> $sheetData = Import-Excel '.\Excel 001.xlsx'
> $sheetData[0]
col1 col2 col3
---- ---- ----
value1 value2 value3
> $sheetData[0] |
Get-Member |
Where-Object { $_.MemberType -eq 'NoteProperty' } |
Select-Object -Property Name
Name
----
col1
col2
col3
> $sheetData |
Get-Member |
Where-Object MemberType -Eq NoteProperty |
ForEach-Object {
New-Variable -Name $_.Name -Value $sheetData.$($_.Name)
}
> Get-Variable col*
Name Value
---- -----
col1 value1
col2 value2
col3 value3

If a user exists in both objects, return their value from object 1 [Powershell]

I'm trying to compare two different objects and return the ID of the user from Object 1 if their email address exists in object 2.
I.e. Object 1
| user | id | emailaddress |
+-----------+------------+--------------------+
| test user | asfasfasdf | test.user#test.com |
| ima test | bsvxcffasd | ima.test#test.com |
+-----------+------------+--------------------+
Object 2
+--------------------+
| email |
+--------------------+
| test.user#test.com |
| ima.test#test.com |
+--------------------+
Consider the 2 objects above, my goal objective is to check if a user exists in Object 2 and Object 1. If they exist in Object 2 then I want to return their ID value.
This code is where i'm up to, this will return the users who email address exists in both objects but not their ID:
$x = $object1 | Select-Object -ExpandProperty emailaddress
$y = $object2 | Select-Object -ExpandProperty email
$z = Compare-Object $x $y -IncludeEqual -ExcludeDifferent
$userids = #()
foreach($a in $z.inputobject){
if($object2.email -contains $a){
$userids += $a
}
}
Attempt 2 based on Olaf's reply:
$object1 = New-Object -Typename psobject
$object1 | Add-Member -MemberType NoteProperty -Name email -Value $otherobject.members.email
$object1 | Add-Member -MemberType NoteProperty -Name id -Value $otherobject.members.id
$object2 = New-Object -Typename psobject
$object2 | Add-Member -MemberType NoteProperty -Name email -Value $otherobject2.emailaddress
$ComparedUsers = Compare-Object -ReferenceObject $object1 -DifferenceObject $object2 -IncludeEqual -ExcludeDifferent -PassThru
You should not use -ExpandProperty when you want to use other properties of the object as well. And I'd recommend to use the same property names for both objects.
Something like this should push you to the right direction:
$object1 =
#'
user,id,email
testuser,asfasfasdf,test.user#test.com
imatest,bsvxcffasd,ima.test#test.com
other,lkjshfdlakjs,other.test#test.com
'# |
ConvertFrom-Csv
$object2 =
#'
email
test.user#test.com
ima.test#test.com
any.test#test.com
'# |
ConvertFrom-Csv
Compare-Object -ReferenceObject $object1 -DifferenceObject $object2 -Property 'email' -IncludeEqual -ExcludeDifferent -PassThru
The output of that would be ...
user id email SideIndicator
---- -- ----- -------------
testuser asfasfasdf test.user#test.com ==
imatest bsvxcffasd ima.test#test.com ==

Change arraylist rows to column using Powershell

I am trying to change the format of an arraylist after I have used
group-object
to count all the entries in the list.
this is a sample
$list = [System.Collections.ArrayList]#()
$list = "letter","solo","nap","nap","nap","sharp","ignite","tap","tap","tap","tap","evoke"
$list | Group-Object | select name,count
This is the sample output
Name Count
---- -----
letter 1
solo 1
nap 3
sharp 1
ignite 1
tap 4
evoke 1
What I would like is
letter solo nap sharp ignite tap evoke
-------- ----- ---- ---- ----- ------ ----
1 1 3 4 1 4 1
Then when exporting to excel it would format like this
Everything I have tried doesn't seem to pay off, or even get close to what I am trying to do and I think I am missing something obvious or have hit my PowerShell skill limitations. Could someone please help. Thank you
You could create a PSObject, add the properties to it with Add-Member, then format the output to a table with Format-Table:
$list = "letter","solo","nap","nap","nap","sharp","ignite","tap","tap","tap","tap","evoke"
$groups = $list | Group-Object | Select-Object Name, Count
$psObject = New-Object -TypeName psobject
foreach ($group in $groups) {
$psObject | Add-Member -NotePropertyName $group.Name -NotePropertyValue $group.Count
}
$psObject | Format-Table
Output:
evoke ignite letter nap sharp solo tap
----- ------ ------ --- ----- ---- ---
1 1 1 3 1 1 4
Skip Group-Object altogether - instead, use a dictionary to keep track of the count, then cast the whole dictionary to a custom object:
$properties = [ordered]#{}
$list |ForEach-Object {
$properties[$_]++
}
$counts = [pscustomobject]$properties
$counts will now hold an object like what you describe, formatting as a table gives you:
PS C:\> $counts |Format-Table
letter solo nap sharp ignite tap evoke
------ ---- --- ----- ------ --- -----
1 1 3 1 1 4 1
You may try something like:
$list = [System.Collections.ArrayList]#()
$list = "letter","solo","nap","nap","nap","sharp","ignite","tap","tap","tap","tap","evoke"
$group = $list | Group-Object | select name,count
$a = [PSCustomObject]#{}
foreach ($item in $group) {
$a | Add-Member -NotePropertyName $item.name -NotePropertyValue $item.count
}
$a | ft
One solution would be to put it into an PsObject and then export that object into a CSV:
$list = [System.Collections.ArrayList]#()
$list = "letter","solo","nap","nap","nap","sharp","ignite","tap","tap","tap","tap","evoke"
$hash = $list | Group-Object | select name,count
$object = New-Object psobject
foreach( $item in $hash ) {
$column_name = $item.Name
$row_value = $item.Count
$object | Add-Member -MemberType NoteProperty -Name $column_name -Value $row_value
}
$object | Export-csv 'Path to your CSV' -NoTypeInformation

Reading txt-file, change rows to columns, save txt file

I have a txt files (semicolon separated) containing over 3 million records where columns 1 to 4 have some general information. Columns 5 and 6 have detailed information. There can be up to 4 different detailed information for the same general information in columns 1 to 4.
My sample input:
Server;Owner;Company;Username;Property;Value
Srv1;Dave;Sandbox;kwus91;Memory;4GB
Srv1;Dave;Sandbox;kwus91;Processes;135
Srv1;Dave;Sandbox;kwus91;Storage;120GB
Srv1;Dave;Sandbox;kwus91;Variant;16
Srv2;Pete;GWZ;aiwq71;Memory;8GB
Srv2;Pete;GWZ;aiwq71;Processes;234
Srv3;Micael;P12;mxuq01;Memory;16GB
Srv3;Micael;P12;mxuq01;Processes;239
Srv3;Micael;P12;mxuq01;Storage;160GB
Srv4;Stefan;MTC;spq61ep;Storage;120GB
Desired output:
Server;Owner;Company;Username;Memory;Processes;Storage;Variant
Srv1;Dave;Sandbox;kwus91;4GB;135;120GB;16
Srv2;Pete;GWZ;aiwq71;8GB;234;;
Srv3;Micael;P12;mxuq01;16GB;239;160GB;
Srv4;Stefan;MTC;spq61ep;;;120GB;
If a values doesn't exist for general information (Columns 1-4) it has to stay blank.
My current code:
$a = Import-csv .\Input.txt -Delimiter ";"
$a | FT -AutoSize
$b = #()
foreach ($Server in $a.Server | Select -Unique) {
$Props = [ordered]#{ Server = $Server }
$Owner = ($a.where({ $_.Server -eq $Server})).Owner | Select -Unique
$Company = ($a.where({ $_.Server -eq $Server})).Company | Select -Unique
$Username = ($a.where({ $_.Server -eq $Server})).Username | Select -Unique
$Props += #{Owner = $Owner}
$Props += #{Company = $Company}
$Props += #{Username = $Username}
foreach ($Property in $a.Property | Select -Unique){
$Value = ($a.where({ $_.Server -eq $Server -and
$_.Property -eq $Property})).Value
$Props += #{ $Property = $Value }
}
$b += New-Object -TypeName PSObject -Property $Props
}
$b | FT -AutoSize
$b | Export-Csv .\Output.txt -NoTypeInformation -Delimiter ";"
After a lot of trying and getting errors: My script works.
But it takes a lot of time.
Is there a possibility to make performance better for around 3 Million lines in txt file? I'm calculating with more or less 2.5 Million unique values for $Server.
I'm running Windows 7 64bit with PowerShell 4.0.
try Something like this:
#Import Data and create empty columns
$List=import-csv "C:\temp\file.csv" -Delimiter ";"
#get all properties name with value not empty
$ListProperty=($List | where Value -ne '' | select property -Unique).Property
#group by server
$Groups=$List | group Server
#loop every rows and store data by group and Property Name
$List | %{
$Current=$_
#Take value not empty and group by Property Name
$Group=($Groups | where Name -eq $Current.Server).Group | where Value -ne '' | group Property
#Add all property and first value not empty
$ListProperty | %{
$PropertyName=$_
$PropertyValue=($Group | where Name -eq $PropertyName | select -first 1).Group.Value
$Current | Add-Member -Name $PropertyName -MemberType NoteProperty -Value $PropertyValue
}
$Current
} | select * -ExcludeProperty Property, Value -unique | export-csv "c:\temp\result.csv" -notype -Delimiter ";"

How to Group data from two arrays and get the count

if I have two arrays and region column has no same values
like
$data1=
Region Type
------ -----------
EuropeWest Operational
EuropeWest Operational
EuropeWest Operational
EuropeNorth Operational
USCentral Operational
USCentral Operational
AsiaEast Operational
AsiaEast Operational
AsiaEast Operational
$data2=
Region Type
------ -----------
EuropeWest MigrateSource
EuropeWest MigrateSource
EuropeNorth MigrateSource
USCentral MigrateSource
USEast MigrateSource
output should be as:
Region Operational MigrateSource
------ ----------- -----------
EuropeWest 4 2
EuropeNorth 1 1
USCentral 2 0
AsiaEast 3 1
Useast 0 1
Any help much appreciated?
I was able to group it but did'nt get any clue how to use foreach loop here:
$data1 | group -Property region | select name,#{n='Operationaclcount';e={$_.count}}
$data2 | group -Property region | select name,#{n='Migratesourcecountt';e={$_.count}}
Since you have two objects with a property name, if we combine these together we have a full list of names. With a ForEach loop, we'll loop over these names and use a Where-Object to filter each of the two objects you created for the count. We'll then create a new object with [pscustomobject]. Finally a quick test if a name is missing from a group that means the count was zero.
$OpCount = $data1 |
Group-Object -Property region |
Select-Object name,#{n='Operationalcount';e={$_.count}}
$MigCount = $data2 |
Group-Object -Property region |
Select-Object name,#{n='Migratesourcecount';e={$_.count}}
$CombinedNames = $OpCount.name + $MigCount.name
Foreach ($Name in $CombinedNames) {
$entry = [pscustomobject]#{
Operational = $OpCount |
Where-Object {$_.name -eq $Name} |
Select-Object -Expand Count
MigrateSource = $MigCount |
Where-Object {$_.name -eq $Name} |
Select-Object -Expand Count
}
if ($entry.Operational -eq $null) { $entry.Operational = 0 }
if ($entry.MigrateSource -eq $null) { $entry.MigrateSource = 0 }
$entry
}
If you want to combine two arrays just use a plus. Something like this:
$data1 + $data2 | Group-Object region | % {
New-Object psobject -property #{
Name = $_.Name
Operational = $_.Count
MigrateSource = #($_.Group | Select Type -Unique).Count
}
}