Group by and add or subtract depending on the indicator - powershell

I have a file which has transaction_date, transaction_amount and debit_credit_indicator. I want to write a program which shows for each date total count and total amount.
Total amount is calculated as follows -
if debit_credit_indicator is 'C' add else if 'D' subtract.
I got till grouping by indicators but don't know how to proceed after wards.
My ouput looks like this
TRANSACTION_DATE DEBIT_CREDIT_INDICA TotalAmount Count
TOR
---------------- ------------------- ----------- -----
2019-02-26 C 1478
2019-02-25 D 100
2019-02-26 D 200
param([string]$inputFileName=30)
(Get-Content $inputFileName) -replace '\|', ',' | Set-Content c:\learnpowershell\test.csv
$transactionData = Import-csv c:\learnpowershell\test.csv | Group-Object -Property TRANSACTION_DATE, DEBIT_CREDIT_INDICATOR
[Array] $newsbData += foreach($gitem in $transactionData)
{
$gitem.group | Select -Unique TRANSACTION_DATE, DEBIT_CREDIT_INDICATOR, `
#{Name = ‘TotalAmount’;Expression = {(($gitem.group) | measure -Property TRANSACTION_AMOUNT -sum).sum}},
#{Name = ‘Count’;Expression = {(($gitem.group) | Measure-Object -count).count}}
};
write-output $newsbData

I suppose you want replace '|' by ',' because you dont know -delimiter option otherwise keep you code for replace. now i propose my code for your problem:
#import en group by date
import-csv "c:\learnpowershell\test.csv" -Delimiter '|' | group TRANSACTION_DATE | %{
$TotalCredit=0
$TotalDebit=0
$CountRowCredit=0
$CountRowDebit=0
$HasProblem=$false
#calculation by date for every group
$_.Group | %{
if ($_.DEBIT_CREDIT_INDICATOR -EQ 'C')
{
$TotalCredit+=$_.transaction_amount
$CountRowCredit++
}
elseif ($_.DEBIT_CREDIT_INDICATOR -EQ 'D')
{
$TotalDebit+=$_.transaction_amount
$CountRowDebit++
}
else
{
$HasProblem=$true
}
}
#output result
[pscustomobject]#{
TRANSACTION_DATE=$_.Name
CountRow=$_.Count
Credit_Total=$TotalCredit
Credit_CountRow=$CountRowCredit
Debit_Total=-$TotalDebit
Debit_CountRow=$CountRowDebit
Total_DebitCredit=$TotalCredit - $TotalDebit
HasProblem=$HasProblem
}
}
You can add ' | Format-Table ' if you want print result formated in table

Related

Powershell - Group and count from CSV file

In my CSV file, I have two columns with header Start_date and Status. I am trying to find out the success percentage for each Start_date
Start_date Status
------------------------------------------
02-03-2022 Completed
02-03-2022 Completed
03-03-2022 Failed
03-03-2022 Completed
I am looking for a final output like below which I export CSV
Start_date Total Completed Failed Success %
02-03-2022 2 2 0 100
03-03-2022 2 1 1 50
As a first step, I am trying to get the count of each day job using below code.
$data = Import-Csv "C:\file.csv"
$data | group {$_.Start_date} | Sort-Object {$_.Start_date} | Select-Object {$_.Status}, Count
Above code will give me output like
$_.Status Count
--------- -----
1
1
it is not showing the date value. what will be the correct approach for this issue ?
You can use Group-Object to group the objects by the date column, then it's just math:
$csv = #'
Start_date,Status
02-03-2022,Completed
02-03-2022,Completed
03-03-2022,Failed
03-03-2022,Completed
'# | ConvertFrom-Csv
# This from your side should be:
# Import-Csv path/to/csv.csv | Group-Object ....
$csv | Group-Object Start_date | ForEach-Object {
$completed, $failed = $_.Group.Status.where({ $_ -eq 'Completed' }, 'Split')
$totalc = $_.Group.Count
$complc = $completed.Count
$failc = $failed.Count
$success = $complc / $totalc
[pscustomobject]#{
Start_Date = $_.Name
Total = $totalc
Completed = $complc
Failed = $failc
Success = $success.ToString('P0')
}
}
Here's another one:
$csv = Import-Csv C:\Temp\tmp.csv
$Results = #()
foreach ($group in $csv | Group Start_date)
{
$Completed = ($group.Group | group status | ? Name -eq Completed).Count
$Failed = ($group.Group | group status | ? Name -eq Failed).Count
$row = "" | Select Start_date,Total,Completed,Failed,"Success %"
$row.Start_date = $group.Name
$row.Total = $group.Count
$row.Completed = $Completed
$row.Failed = $Failed
$row."Success %" = $Completed / 2 * 100
$Results += $row
}
$results
Start_date Total Completed Failed Success %
---------- ----- --------- ------ ---------
02-03-2022 2 2 0 100
03-03-2022 2 1 1 50

How to export two variables into same CSV as joined via PowerShell?

I have a PowerShell script employing poshwsus module like below:
$FileOutput = "C:\WSUSReport\WSUSReport.csv"
$ProcessLog = "C:\WSUSReport\QueryLog2.txt"
$WSUSServers = "C:\WSUSReport\Computers.txt"
$WSUSPort = "8530"
import-module poshwsus
ForEach ($Server in Get-Content $WSUSServers)
{
& connect-poshwsusserver $Server -port $WSUSPort | out-file $ProcessLog -append
$r1 = & Get-PoshWSUSClient | select #{name="Computer";expression={$_.FullDomainName}},#{name="LastUpdated";expression={if ([datetime]$_.LastReportedStatusTime -gt [datetime]"1/1/0001 12:00:00 AM") {$_.LastReportedStatusTime} else {$_.LastSyncTime}}}
$r2 = & Get-PoshWSUSUpdateSummaryPerClient -UpdateScope (new-poshwsusupdatescope) -ComputerScope (new-poshwsuscomputerscope) | Select Computer,NeededCount,DownloadedCount,NotApplicableCount,NotInstalledCount,InstalledCount,FailedCount
}
What I need to do is to export CSV outpout including the results with the columns (like "inner join"):
Computer, NeededCount, DownloadedCount, NotApplicableCount, NotINstalledCount, InstalledCount, FailedCount, LastUpdated
I have tried to use the line below in foreach, but it didn't work as I expected.
$r1 + $r2 | export-csv -NoTypeInformation -append $FileOutput
I appreciate if you may help or advise.
EDIT --> The output I've got:
ComputerName LastUpdate
X A
Y B
X
Y
So no error, first two rows from $r2, last two rows from $r1, it is not joining the tables as I expected.
Thanks!
I've found my guidance in this post: Inner Join in PowerShell (without SQL)
Modified my query accordingly like below, works like a charm.
$FileOutput = "C:\WSUSReport\WSUSReport.csv"
$ProcessLog = "C:\WSUSReport\QueryLog.txt"
$WSUSServers = "C:\WSUSReport\Computers.txt"
$WSUSPort = "8530"
import-module poshwsus
function Join-Records($tab1, $tab2){
$prop1 = $tab1 | select -First 1 | % {$_.PSObject.Properties.Name} #properties from t1
$prop2 = $tab2 | select -First 1 | % {$_.PSObject.Properties.Name} #properties from t2
$join = $prop1 | ? {$prop2 -Contains $_}
$unique1 = $prop1 | ?{ $join -notcontains $_}
$unique2 = $prop2 | ?{ $join -notcontains $_}
if ($join) {
$tab1 | % {
$t1 = $_
$tab2 | % {
$t2 = $_
foreach ($prop in $join) {
if (!$t1.$prop.Equals($t2.$prop)) { return; }
}
$result = #{}
$join | % { $result.Add($_,$t1.$_) }
$unique1 | % { $result.Add($_,$t1.$_) }
$unique2 | % { $result.Add($_,$t2.$_) }
[PSCustomObject]$result
}
}
}
}
ForEach ($Server in Get-Content $WSUSServers)
{
& connect-poshwsusserver $Server -port $WSUSPort | out-file $ProcessLog -append
$r1 = & Get-PoshWSUSClient | select #{name="Computer";expression={$_.FullDomainName}},#{name="LastUpdated";expression={if ([datetime]$_.LastReportedStatusTime -gt [datetime]"1/1/0001 12:00:00 AM") {$_.LastReportedStatusTime} else {$_.LastSyncTime}}}
$r2 = & Get-PoshWSUSUpdateSummaryPerClient -UpdateScope (new-poshwsusupdatescope) -ComputerScope (new-poshwsuscomputerscope) | Select Computer,NeededCount,DownloadedCount,NotApplicableCount,NotInstalledCount,InstalledCount,FailedCount
Join-Records $r1 $r2 | Select Computer,NeededCount,DownloadedCount,NotApplicableCount,NotInstalledCount,InstalledCount,FailedCount, LastUpdated | export-csv -NoTypeInformation -append $FileOutput
}
I think this could be made simpler. Since Select-Object's -Property parameter accepts an array of values, you can create an array of the properties you want to display. The array can be constructed by comparing your two objects' properties and outputting a unique list of those properties.
$selectProperties = $r1.psobject.properties.name | Compare-Object $r2.psobject.properties.name -IncludeEqual -PassThru
$r1,$r2 | Select-Object -Property $selectProperties
Compare-Object by default will output only differences between a reference object and a difference object. Adding the -IncludeEqual switch displays different and equal comparisons. Adding the -PassThru parameter outputs the actual objects that are compared rather than the default PSCustomObject output.

Multiple Criteria Matching in PowerShell

Hello PowerShell Scriptwriters,
I got an objective to count rows, based on the multiple criteria matching. My PowerShell script can able to fetch me the end result, but it consumes too much time[when the rows are more, the time it consumes becomes even more]. Is there a way to optimism my existing code? I've shared my code for your reference.
$csvfile = Import-csv "D:\file\filename.csv"
$name_unique = $csvfile | ForEach-Object {$_.Name} | Select-Object -Unique
$region_unique = $csvfile | ForEach-Object {$_."Region Location"} | Select-Object -Unique
$cost_unique = $csvfile | ForEach-Object {$_."Product Cost"} | Select-Object -Unique
Write-host "Save Time on Report" $csvfile.Length
foreach($nu in $name_unique)
{
$inc = 1
foreach($au in $region_unique)
{
foreach($tu in $cost_unique)
{
foreach ($mainfile in $csvfile)
{
if (($mainfile."Region Location" -eq $au) -and ($mainfile.'Product Cost' -eq $tu) -and ($mainfile.Name -eq $nu))
{
$inc++ #Matching Counter
}
}
}
}
$inc #expected to display Row values with the total count.And export the result as csv
}
You can do this quite simply using the Group option on a Powershell object.
$csvfile = Import-csv "D:\file\filename.csv"
$csvfile | Group Name,"Region Location","Product Cost" | Select Name, Count
This gives output something like the below
Name Count
---- ------
f1, syd, 10 2
f2, syd, 10 1
f3, syd, 20 1
f4, melb, 10 2
f2, syd, 40 1
P.S. the code you provided above is not matching all of the fields, it is simply checking the Name parameter (looping through the other parameters needlessly).

Check if Two Values in Array / Data Table match

I put this question on here before, but I missed an important detail which causes huge issue. There will be duplicate account numbers. So I'm doing it by current_read_date now to avoid duplicates in account number. To ensure values being added to $accounts are new from the CSV.
I am trying to get all the accounts from $f which do not match the accounts in $table4 into $accounts. But I need to also check if the current_read_date matches or not.
CSV into Array $f:
Account_no |occupant_code|current_read_date
-----------|-------------|-----------------
12345 | 1 | 7/17/2017 15:32:00 AM
67890 | 2 | 7/17/2017 12:00:00 AM
45678 | 3 | 7/17/2017 12:00:00 AM
DataTable $table4
Account_no |occupant_code|current_read_date
-----------|-------------|-----------------
12345 | 1 | 7/17/2017 12:00:00 AM
12345 | 1 | 7/17/2017 15:32:00 AM
67890 | 1 | 7/17/2017 13:00:00 AM
67890 | 1 | 7/17/2017 22:00:00 AM
45678 | 3 | 7/17/2017 12:00:00 AM
Desired result:
$accounts =
67890 | 2 | 7/17/2017 12:00:00 AM
Current code:
$accounts = Import-Csv $f |
select account_no, current_read_date |
where { $table4.account_no -notcontains $_.account_no }
What this needs to do is to check that current_read_date doesn't match, i.e.:
12345: account and date from $f and $table4 match; so it's ignored
67890: account matches $table4, but the current_read_date does not match, so it is a new value, thus it is added to $accounts.
I believe I need to use Group-Object, but I do not know how to use that correctly.
I tried:
Import-Csv $f |
select account_no, occupant_code |
Group-Object account_no |
Where-Object { $_.Group.current_read_date -notcontains $table4.current_read_date }
This is the previous question:
How to use Group-Object on this?
All the answers here failed because I forgot to provide the information that account_no is not unique; there will be frequent duplicates.
All assistance would be greatly appreciated, I've been stuck on this for awhile.
I've also tried this
$testList = #()
$testList = Import-Csv $f | select account_no, occupant_code, current_read_date, current_reading
$accounts = new-object System.Collections.ArrayList
$testSet = $table4
foreach($myThing in $testList)
{
if($myThing.account_no -in $testSet.account_no )
{
foreach($ts in $testSet)
{
if ($myThing.account_no -match $ts.account_no -and $myThing.occupant_code -match $ts.occupant_code)
{
$ts.account_no
$ts.occupant_code
}
else {
$accounts.add($myThing) | out-null
write-host $mything
}
}
}
This fails because it goes through each number, therefore, 12345 will be checked against 67890, and will added 12345 to the $accounts list, even though it already exists, because I cannot compare each individually at a time with table4.
Thanks
$accounts = $f | Where {
$Record = $_
$AccNo = $table4 | Where {$_.Account_no -eq $Record.Account_no}
!($AccNo | Where {$_.current_read_date -eq $Record.current_read_date})
}
$accounts | Format-Table
Result:
Account_no occupant_code current_read_date
---------- ------------- -----------------
67890 2 7/17/2017 12:00:00 AM
Build a reference list from the records in $table4
$ref = $table4 | ForEach-Object {
$_.Account_no, $_.occupant_code, $_.current_read_date -join ','
}
then filter the records from $f by that reference list:
$accounts = Import-Csv $f | Where-Object {
$ref -notcontains ($_.Account_no, $_.occupant_code, $_.current_read_date -join ',')
} | Select-Object -Expand Account_no -Unique

if loop not seems to be working

I have a code as given below:
$datearray = #()
$temp = Get-Content "C:\temp.txt"
$temp1 = Get-Content "C:\temp1.txt"
foreach ($te in $temp) {
$t = $te -split '-'
$da = $t[1]
$mo = $t[2]
$yea = $t[3]
$fulldate = "$da-$mo-$yea"
if ($temp1 -match $fulldate) {
if ($fulldate -match $te) {
$datearray += $_
$fmt = 'dd-MM-yy-HH-mm'
$culture = [Globalization.CultureInfo]::InvariantCulture
*!* $datearray | sort { [DateTime]::ParseExact(($_ -split '-', 2)[1], $fmt, $culture) } | select -Last 1 | Add-Content "c:\temp4.txt"
} else {
#some operation
}
} else {
#some operation
}
}
For your understanding, I will show you how temp1.txt looks like:
17-07-15
18-07-15
19-07-15
20-07-15
21-07-15
22-07-15
23-07-15
temp.txt is:
testdatabase-17-07-15-22-00
testdatabase-17-07-15-23-00
testdatabase-21-07-15-10-00
testdatabase-21-07-15-23-00
What I am trying to do is that whenever it reaches the code marked with *!*, it goes back to foreach loop in the top every time. That marked code is not getting executed.
Can someone please tell me the solution?
Use the Group-Object cmdlet to group the databases by date, then select the most recent database name from each group:
$fmt = 'dd-MM-yy-HH-mm'
$culture = [Globalization.CultureInfo]::InvariantCulture
Get-Content 'C:\temp.txt' |
select #{n='Timestamp';e={[DateTime]::ParseExact(($_ -split '-', 2)[1], $fmt, $culture)}},
#{n='Database';e={$_}} |
group { $_.Timestamp.Date } |
% { $_.Group | sort Timestamp | select -Last 1 -Expand Database }
The code uses a select statement to transform the list of lines into a list of custom objects with a Timestamp and a Database property in order to simplify grouping and sorting the database names by date.
Inspecting the output after each step of the pipeline should help you understand the logic behind this. Get-Content produces a list of strings with the lines from the file:
PS C:\> Get-Content 'C:\temp.txt'
testdatabase-17-07-15-22-00
testdatabase-17-07-15-23-00
testdatabase-21-07-15-10-00
testdatabase-21-07-15-23-00
By using Select-Object with calculated properties the list of strings is transformed into a list of custom objects with 2 properties, the database name and the timestamp (as a DateTime object):
PS C:\> Get-Content 'C:\temp.txt' |
>> select #{n='Timestamp';e={[DateTime]::ParseExact(($_ -split '-', 2)[1], $fmt, $culture)}},
>> #{n='Database';e={$_}}
>>
Timestamp Database
--------- --------
17.07.2015 22:00:00 testdatabase-17-07-15-22-00
17.07.2015 23:00:00 testdatabase-17-07-15-23-00
21.07.2015 10:00:00 testdatabase-21-07-15-10-00
21.07.2015 23:00:00 testdatabase-21-07-15-23-00
Grouping these objects by the date portion of the timestamp gets you a list of GroupInfo objects whose Group property contains a list of the database names for a given date:
PS C:\> Get-Content 'C:\temp.txt' |
>> select #{n='Timestamp';e={[DateTime]::ParseExact(($_ -split '-', 2)[1], $fmt, $culture)}},
>> #{n='Database';e={$_}} |
>> group { $_.Timestamp.Date }
>>
Count Name Group
----- ---- -----
2 17.07.2015 00:00:00 {#{Timestamp=17.07.2015 22:00:00; Database=testdatabase-17-07-15-22-00}, #{Timestamp...
2 21.07.2015 00:00:00 {#{Timestamp=21.07.2015 10:00:00; Database=testdatabase-21-07-15-10-00}, #{Timestamp...
The ForEach-Object loop then sorts the elements of each group by timestamp and selects the last (most recent) database name from each group:
PS C:\> Get-Content 'C:\temp.txt' |
>> select #{n='Timestamp';e={[DateTime]::ParseExact(($_ -split '-', 2)[1], $fmt, $culture)}},
>> #{n='Database';e={$_}} |
>> group { $_.Timestamp.Date } |
>> % { $_.Group | sort Timestamp | select -Last 1 -Expand Database }
>>
testdatabase-17-07-15-23-00
testdatabase-21-07-15-23-00