Powershell Compare-object IF different then ONLY list items from one file, not both - powershell

I have deleted my original question because I believe I have a more efficient way to run my script, thus I'm changing my question.
$scrubFileOneDelim = "|"
$scrubFileTwoDelim = "|"
$scrubFileOneBal = 2
$scrubFileTwoBal = 56
$scrubFileOneAcctNum = 0
$scrubFileTwoAcctNum = 0
$ColumnsF1 = Get-Content $scrubFileOne | ForEach-Object{($_.split($scrubFileOneDelim)).Count} | Measure-Object -Maximum | Select-Object -ExpandProperty Maximum
$ColumnsF2 = Get-Content $scrubFileTwo | ForEach-Object{($_.split($scrubFileTwoDelim)).Count} | Measure-Object -Maximum | Select-Object -ExpandProperty Maximum
$useColumnsF1 = $ColumnsF1-1;
$useColumnsF2 = $ColumnsF2-1;
$fileOne = import-csv "$scrubFileOne" -Delimiter "$scrubFileOneDelim" -Header (0..$useColumnsF1) | select -Property #{label="BALANCE";expression={$($_.$scrubFileOneBal)}},#{label="ACCTNUM";expression={$($_.$scrubFileOneAcctNum)}}
$fileTwo = import-csv "$scrubFileTwo" -Delimiter "$scrubFileTwoDelim" -Header (0..$useColumnsF2) | select -Property #{label="BALANCE";expression={$($_.$scrubFileTwoBal)}},#{label="ACCTNUM";expression={$($_.$scrubFileTwoAcctNum)}}
$hash = #{}
$hashTwo = #{}
$fileOne | foreach { $hash.add($_.ACCTNUM, $_.BALANCE) }
$fileTwo | foreach { $hashTwo.add($_.ACCTNUM, $_.BALANCE) }
In this script I'm doing the following, counting header's to return the count and use it in a range operator in order to dynamically insert headers for later manipulation. Then I'm importing 2 CSV files. I'm taking those CSV files and pushing them into their own hashtable.
Just for an idea of what I'm trying to do from here...
CSV1 (as a hashtable) looks like this:
Name Value
---- -----
000000000001 000000285+
000000000002 000031000+
000000000003 000004685+
000000000004 000025877+
000000000005 000000001+
000000000006 000031000+
000000000007 000018137+
000000000008 000000000+
CSV2 (as a hashtable) looks like this:
Name Value
---- -----
000000000001 000008411+
000000000003 000018137+
000000000007 000042865+
000000000008 000009761+
I would like to create a third hash table. It will have all the "NAME" items from CSV2, but I don't want the "VALUE" from CSV2, I want it to have the "VALUE"s that CSV1 has. So in the end result would look like this.
Name Value
---- -----
000000000001 000000285+
000000000003 000004685+
000000000007 000018137+
000000000008 000000000+
Ultimately I want this to be exported as a csv.
I have tried this with just doing a compare-object, not doing the hashtables with the following code, but I abandoned trying to do it this way because file 1 may have 100,000 "accounts" where file 2 only has 200, and the result I was getting listed close to the 100,000 accounts that I didn't want to be in the result. They had the right balances but I want a file that only has those balances for the accounts listed in file 2. This code below isn't really a part of my question, just showing something I've tried. I just think this is much easier and faster with a hash table now so I would like to go that route.
#Find and Rename the BALANCE and ACCOUNT NUMBER columns in both files.
$fileOne = import-csv "$scrubFileOne" -Delimiter "$scrubFileOneDelim" -Header (0..$useColumnsF1) | select -Property #{label="BALANCE";expression={$($_.$scrubFileOneBal)}},#{label="ACCT-NUM";expression={$($_.$scrubFileOneAcctNum)}}
$fileTwo = import-csv "$scrubFileTwo" -Delimiter "$scrubFileTwoDelim" -Header (0..$useColumnsF2) | select -Property #{label="BALANCE";expression={$($_.$scrubFileTwoBal)}},#{label="ACCT-NUM";expression={$($_.$scrubFileTwoAcctNum)}}
Compare-Object $fileOne $fileTwo -Property 'BALANCE','ACCTNUM' -IncludeEqual -PassThru | Where-Object{$_.sideIndicator -eq "<="} | select * -Exclude SideIndicator | export-csv -notype "C:\test\f1.txt"

What you are after is filtering the Compare-Object function. This will show only one side of the result. YOu will need to place this before you exclude that property for it to work.
| Where-Object{$_.sideIndicator -eq "<="} |

Assuming that you have the following hash tables:
$hash = #{
'000000000001' = '000000285+';
'000000000002' = '000031000+';
'000000000003' = '000004685+';
'000000000004' = '000025877+';
'000000000005' = '000000001+';
'000000000006' = '000031000+';
'000000000007' = '000018137+';
'000000000008' = '000000000+';
}
$hashTwo = #{
'000000000001' = '000008411+';
'000000000003' = '000018137+';
'000000000007' = '000042865+';
'000000000008' = '000009761+';
}
you can create the third hash table by iterating over the keys from the second hash table and then assigning those keys to the value from the first hash table.
$hashThree = #{}
ForEach ($key In $hashTwo.Keys) {
$hashThree["$key"] = $hash["$key"]
}
$hashThree
The output of $hashThree is:
Name Value
---- -----
000000000007 000018137+
000000000001 000000285+
000000000008 000000000+
000000000003 000004685+
If you want the order of the data maintained (and you are using PowerShell 6 Core), you can use [ordered]#{} when creating the hash tables.

Related

Group csv column data and display count using powershell

I am having below data in my csv
"Path_Name","Lun_Number","status"
"vmhba0:C2:T0:L1","1","active"
"vmhba0:C1:T0:L1","1","active"
"vmhba1:C0:T7:L230","230","active"
"vmhba1:C0:T7:L231","231","active"
"vmhba1:C0:T7:L232","230","active"
"vmhba1:C0:T7:L235","231","active"
"vmhba1:C0:T7:L236","230","active"
I need to group the data based on Lun_Number and create a column to get the count of those Lun_Number
expected output
"Path_Name","Lun_Number","status","Count"
"vmhba0:C2:T0:L1","1","active", 2
"vmhba0:C1:T0:L1","1","active",
"vmhba1:C0:T7:L230","230","active",3
"vmhba1:C0:T7:L231","230","active",
"vmhba1:C0:T7:L232","230","active",
"vmhba1:C0:T7:L235","231","active",2
"vmhba1:C0:T7:L236","231","active",
Please let me know how can I do that. I tried group-object, sort-object but it doesn't seems to be working
Below is the code which is generating the above csv
$status_csv = Import-Csv -Path E:\pathstate.csv
$path_csv = Import-Csv -Path E:\PathInfo.csv
foreach($row in $path_csv)
{
$path_1 = $row.Path_Name
$path_2 = $status_csv | where{$_.Name -match "^$path_1$" }
[PsCustomObject]#{
Path_Name = $path_1
Lun_Number = $row.Lun_Number
status = $path_2.PathState
} | Export-Csv -Path E:\FinalReport.csv -NoTypeInformation -Append | Group-Object Lun_Number
}
I can see the approach you're trying to take, and I think something like the following could be useful:
$path_csv = Import-Csv 'sample.csv'
$Unique_Counts = $path_csv.Lun_Number | Group-Object | Select-Object Name,Count
This will help give you the output that you can use as part of your mapping for later, where you can make it dynamic to match with the row you're checking. Which you can use to pull out through a loop, such as $Unique_Counts.
Meaning that it you do something like $Unique_Counts[0].Count, will be able to grab the Lun_Number associated to it (listed as Name in the array).
Name Count
---- -----
1 2
230 3
231 2
If you're okay with having the count for each row, you can then use something like what you have:
foreach($row in $path_csv)
{
$path_1 = $row.Path_Name
[PsCustomObject]#{
Path_Name = $path_1
Lun_Number = $row.Lun_Number
status = $row.status
count = $Unique_Counts | Where-Object {$_.name -eq $row.Lun_Number} | Select-Object -ExpandProperty Count
} | Export-Csv $finalReport -NoTypeInformation -Append
}
This then provides me with the following outcome:
"Path_Name","Lun_Number","status","count"
"vmhba0:C2:T0:L1","1","active","2"
"vmhba0:C1:T0:L1","1","active","2"
"vmhba1:C0:T7:L230","230","active","3"
"vmhba1:C0:T7:L231","231","active","2"
"vmhba1:C0:T7:L232","230","active","3"
"vmhba1:C0:T7:L235","231","active","2"
"vmhba1:C0:T7:L236","230","active","3"
Hope this helps, it may be useful to understand more with the use case, but at least you can grab the unique count for all Lun_numbers. Just has to put it in all rows too.

Powershell - Using ConvertFrom-csv

I'm brand new to Powershell. I have a variable that contains comma separated values. What I want to do is read each entry in the csv string variable, and assign it to a variable. I am using ConvertFrom-csv to separate the data with headers.
How can I assign each value to a variable, or even better, use ConvertTo-csv to create a new csv string which only has, for example, columns 2/3/6/7 in it?
I would ultimately want to write that data out to a new csv file.
Here is my test code:
#Setup the variable
$Data = "test1,test2,test3,1234,5678,1/1/2021,12/31/2021"
$Data | ConvertFrom-csv -Header Header1,Header2, Header3, Header4, Header5, Header6, Header7
# Verify that an object has been created.
$Data |
ConvertFrom-csv -Header Header1,Header2, Header3, Header4, Header5, Header6, Header7 |
Get-Member
#Show header1
Write-Host "--------Value from $Data----------------------------------------"
$Data[0] #doesn't work, only displays the first character of the string
Write-Host "-----------------------------------------------------------------"
Let me suggest a different approach. If you use ConvertFrom-Csv and assign the result of a variable ($data), this will be an array of Custom Objects. You can run this through a loop that steps through the elements of the array , one at a time, and then through an inner loop that steps through the properties of each object one at a time, setting a variable with the same name as the field header and the same value as the current record's value.
I don't have code that does exactly what you want. But I'm including code that I wrote a few years back that does something similar only using Import-Csv instead of ConverFrom-Csv.
Import-Csv $driver | % {
$_.psobject.properties | % {Set-variable -name $_.name -value $_.value}
Get-Content $template | % {$ExecutionContext.InvokeCommand.ExpandString($_)}
}
Focus on the first inner loop. Each property of the current object will have a name that came from the header and a value that came from the current record of the Csv file. You can ignore the line that says ExpandString. That's just what I choose to do with the variables once they have been defined.
How can I assign each value to a variable, or even better, use ConvertTo-Csv to create a new csv string which only has, for example, columns 2/3/6/7 in it?
This is one way of automating this:
# Define the CSV without headers
$Data = "test1,test2,test3,1234,5678,1/1/2021,12/31/2021"
# Set the number of headers needed
$headers = $Data.Split(',') | ForEach-Object -Begin { $i = 1 } -Process {
"Header$i"; $i++
}
# Set the desired columns we want
$desiredColumns = 2,3,6,7 | ForEach-Object { $_ - 1 } | ForEach-Object {
$headers[$_]
}
# Convert to CSV and filter by Desired Columns
$Data | ConvertFrom-Csv -Header $headers | Select-Object $desiredColumns
Result
Header2 Header3 Header6 Header7
------- ------- ------- -------
test2 test3 1/1/2021 12/31/2021
Result as CSV
$Data | ConvertFrom-Csv -Header $headers |
Select-Object $desiredColumns | ConvertTo-Csv -NoTypeInformation
"Header2","Header3","Header6","Header7"
"test2","test3","1/1/2021","12/31/2021"

Get results of For-Each arrays and display in a table with column headers one line per results

I am trying to get a list of files and a count of the number of rows in each file displayed in a table consisting of two columns, Name and Lines.
I have tried using format table but I don't think the problem is with the format of the table and more to do with my results being separate results. See below
#Get a list of files in the filepath location
$files = Get-ChildItem $filepath
$files | ForEach-Object { $_ ; $_ | Get-Content | Measure-Object -Line} | Format-Table Name,Lines
Expected results
Name Lines
File A
9
File B
89
Actual Results
Name Lines
File A
9
File B
89
Another approach how to make a custom object like this: Using PowerShell's Calculated Properties:
$files | Select-Object -Property #{ N = 'Name' ; E = { $_.Name} },
#{ N = 'Lines'; E = { ($_ | Get-Content | Measure-Object -Line).Lines } }
Name Lines
---- -----
dotNetEnumClass.ps1 232
DotNetVersions.ps1 9
dotNETversionTable.ps1 64
Typically you would make a custom object like this, instead of outputting two different kinds of objects.
$files | ForEach-Object {
$lines = $_ | Get-Content | Measure-Object -Line
[pscustomobject]#{name = $_.name
lines = $lines.lines}
}
name lines
---- -----
rof.ps1 11
rof.ps1~ 7
wai.ps1 2
wai.ps1~ 1

Split array column and result into extra Colums

I have a table (CSV) which shows all the users that have ever logged on to a bunch of computers. Users can have 2 accounts, one username starts with "a" the other with "b", like a100 and b100 (the user behind is the same person).
Now I need to get the computers that have more then 2 accounts logged on which do not belong the same users. So A64 and B64 are not reported as separate users.
Here is the base list I have:
PC1,A64,B52,B64,A41
PC2,A51,B42,B51,A23
PC3,A42,B51
PC4,A5,B5
PC5,A1,B1,A14,A6
My plan was to split the "User"-column into more columns, so the table would look like this:
Computername,user1,user2,user3,user4,UserX
After this was done, I could iterate through the table and remove the leading letter in the Username, then I would try to get rid of doubles.
Do you think that makes sense?
Now I got stuck in the first task already. I know how to iterate though the second Column but how do I managed to get the result into another array so the output would be like:
Computername,user1,user2,user3,user4,UserX
Can you help me split?
$UserComputers = import-csv -Delimiter ";" "input.csv" -Header
'Computername','user1','user2','user3','user4'
$UserComputers | Select-Object *,
#{n='User1';e={$_.User1.Split(',')[0]}},
#{n='User2';e={$_.User1.Split(',')[1]}}
I get the error: Select-Object : The property cannot be processed because the property "User1" already exists.
It is useful to make "user" an array.
Get-Content "input.csv" | foreach {
$name, $users = $_.Split(",")
[pscustomobject]#{ Name = $name; Users = $users }
} | Where-Object { ($_.Users.Substring(1) | Select-Object -Unique).Count -gt 2 }
The output is below.
Name Users
---- -----
PC1 {A64, B52, B64, A41}
PC2 {A51, B42, B51, A23}
PC3 {A42, B51}
PC5 {A1, B1, A14, A6}
Input File (input.csv)
PC1,A64,B52,B64,A41
PC2,A51,B42,B51,A23
PC3,A42,B51
PC4,A5,B5
PC5,A1,B1,A14,A6
Powershell Script
Get-Content -Path .\input.csv |
Select-Object #{ Name = "Computer"; Expression = { $_.Split(',')[0] } },
#{ Name="Users"; Expression = { $_.Split(',')[1..($_.Split(',').Length-1)] |
Foreach-Object { $_.Substring(1) } | Select-Object -Unique } } |
Where-Object { $_.Users.Count -gt 2 }
Result:
Computer Users
------------- -----
PC1 {64, 52, 41}
PC2 {51, 42, 23}
PC5 {1, 14, 6}
P.S. Bonus: If you want to see more than 4 elements of the array on the screen change the variable
$FormatEnumerationLimit = 20
Explanation of the variable meaning
If your file is like your base list, you can do the following to build a new file with all the columns you need:
$maxColCount = 0
$data = get-content input.csv
foreach ($line in $data) {
$MaxColCount = [math]::Max($maxcolcount,($line -split ",").count)
}
$headers = #("ComputerName")
$MaxUserCount = $MaxColCount - 1
Foreach ($c in (1..$MaxUserCount)) {
$Headers += "User$c"
}
$Headers = $Headers -join ","
$Headers,$data | Set-Content "output.csv"
The code above assumes input.csv has the following format and each column after the first is a user:
PC1,A64,B52,B64,A41
PC2,A51,B42,B51,A23
PC3,A42,B51
PC4,A5,B5
PC5,A1,B1,A14,A6

Compare-Object Output Format

I have two CSV files I need to compare. Both of which may (or may not) contain an additional delimited field (delimited via a "|" character), like so:
(new.csv)
Title,Key,Value
Jason,Son,Hair=Red|Eyes=Blue
James,Son,Hair=Brown|Eyes=Green
Ron,Father,Hair=Black
Susan,Mother,Hair=Black|Eyes=Brown|Dress=Green
(old.csv)
Title,Key,Value
Jason,Son,Hair=Purple|Eyes=Blue
James,Son,Hair=Brown|Eyes=Green
Ron,Father,Hair=Purple
Susan,Mother,Hair=Black|Eyes=Brown|Dress=Blue
My problem comes in when I attempt to compare the two files...
$fileNew = "new.csv"
$fileOld = "old.csv"
$fileDiffOutputFile = "diff.txt"
$csvNewLog = (Import-CSV ($fileNew))
$csvOldLog = (Import-CSV ($fileOld))
$varDifferences = Compare-Object $csvOldLog $csvNewLog -property Title,Value
$varDifferences | Group-Object -Property Title | % {New-Object -TypeName PSObject -Property #{ NewValue=($_.group[0].Value); Title=$_.name; OldValue=($_.group[1].Value) } } | Out-File $fileDiffOutputFile -Append
Resulting in this output:
(diff.txt)
OldValue Title NewValue
-------- ----- --------
Hair=Purple|Eyes=Blue Jason Hair=Red|Eyes=Blue
Hair=Purple Ron Hair=Black
Hair=Black|Eyes=Brown|D... Susan Hair=Black|Eyes=Brown|...
Some of the values are inevitably going to extend out past the max length of the column, as it does with Susan above.
So, my question could have a couple of solutions that I can think of:
Is there an easier way to isolate the values so that I only pull out the changed values, and not the entire string of delimited values?
If not, is it possible to get the format to show the entire string (including the unchanged values part of the delimited string) instead?
If you include a format-table -wrap in your last line, like so?
$fileNew = "new.csv"
$fileOld = "old.csv"
$fileDiffOutputFile = "diff.txt"
$csvNewLog = (Import-CSV ($fileNew))
$csvOldLog = (Import-CSV ($fileOld))
$varDifferences = Compare-Object $csvOldLog $csvNewLog -property Title,Value
$varDifferences | Group-Object -Property Title | % {New-Object -TypeName PSObject -Property #{ NewValue=($_.group[0].Value); Title=$_.name; OldValue=($_.group[1].Value) } } | Format-Table -wrap | Out-File $fileDiffOutputFile -Append