please, could you help me find a solution to handle csv file with multiple field column
File1.csv
Teams,Category,Members
Team1,A,Smith;Johnson
Team1,C,Jones;Miller;Garcia
Team3,E,Wilson;Martinez
Team4,A,Martin;Jackson;White;Williams
File2.csv
Teams,Category,Members
Team1,A,Smith;Johnson
Team2,C,Jones;Miller;Garcia
Team3,E,Wilson;Martinez;Gonzalez;Hall
Team4,A,Martin;Jackson;Williams
Diff :
Add Gonzalez and Hall on teams 3
Remove White on Team-4
$1 = Import-Csv -Path ".\File1.csv" -Delimiter ','
$2 = Import-Csv -Path ".\File2.csv" -Delimiter ','
Compare-Object $1 $2 -Property Members -PassThru
Result :
Teams Category Members SideIndicator
Team3 E Wilson;Martinez;Gonzalez;Hall =>
Team4 A Martin;Jackson;Williams =>
Team3 E Wilson;Martinez <=
Team4 A Martin;Jackson;White;Williams <=
what is expected :
Teams Category Members SideIndicator
Team3 E Gonzalez and Hall =>
Team4 A White <=
I'd compare objects first to find differencies (notice that I compare two properties: Teams and Members to avoid missing entries in case the membership of different teams matches) and then compare the arrays created from matching objects:
$1 = Import-Csv -Path ".\File1.csv" -Delimiter ','
$2 = Import-Csv -Path ".\File2.csv" -Delimiter ','
$comparisonRes = Compare-Object $1 $2 -Property Teams,Members -PassThru
foreach ($obj in $comparisonRes | Where-Object SideIndicator -eq "=>") {
# $obj = ($comparisonRes | Where-Object SideIndicator -eq "=>")[0]
$matchingEntry = $1 | Where-Object {$_.Teams -eq $obj.Teams}
$matchingEntryMembers = $matchingEntry.Members -split ";"
$currentEntryMembers = $obj.Members -split ";"
$diffMembers = Compare-Object $matchingEntryMembers $currentEntryMembers
# Uncomment to log
# $diffMembers
# Do something with $diffMembers here
}
You might want to use json instead of csv which supports arrays and numbers. Otherwise the teams look like two semicolon separated strings.
file1.json
[
{"Teams":"Team1","Category":"A","Members":["Smith","Johnson"]},
{"Teams":"Team1","Category":"C","Members":["Jones","Miller","Garcia"]},
{"Teams":"Team3","Category":"E","Members":["Wilson","Martinez"]},
{"Teams":"Team4","Category":"A","Members":["Martin","Jackson","White","Williams"]}
]
file2.json
[
{"Teams":"Team1","Category":"A","Members":["Smith","Johnson"]},
{"Teams":"Team2","Category":"C","Members":["Jones","Miller","Garcia"]},
{"Teams":"Team3","Category":"E","Members":["Wilson","Martinez","Gonzalez","Hall"]},
{"Teams":"Team4","Category":"A","Members":["Martin","Jackson","Williams"]}
]
$1 = cat file1.json | convertfrom-json
$2 = cat file2.json | convertfrom-json
Compare-Object $1 $2 -Property Members -PassThru
Teams Category Members SideIndicator
----- -------- ------- -------------
Team3 E {Wilson, Martinez, Gonzalez, Hall} =>
Team4 A {Martin, Jackson, Williams} =>
Team3 E {Wilson, Martinez} <=
Team4 A {Martin, Jackson, White, Williams} <=
Here's a closer answer. Run compare-object on members only one line at a time, then add teams and category to it.
$1 = cat file1.json | convertfrom-json
$2 = cat file2.json | convertfrom-json
for($i = 0; $i -lt $1.length; $i++) {
compare-object $1[$i].members $2[$i].members |
select #{n='Teams'; e={$1[$i].teams}},
#{n='Category'; e={$1[$i].Category}},
#{n='Members'; e={$_.inputobject}},
sideindicator
}
Teams Category Members SideIndicator
----- -------- ------- -------------
Team3 E Gonzalez =>
Team3 E Hall =>
Team4 A White <=
Here's another way using a zip function PowerShell/CLI: "Foreach" loop with multiple arrays on both lists of objects.
$1 = cat file1.json | convertfrom-json
$2 = cat file2.json | convertfrom-json
function Zip($a1, $a2) { # function allows it to stream
while ($a1) {
$x, $a1 = $a1 # $a1 gets the tail of the list
$y, $a2 = $a2
[tuple]::Create($x, $y)
}
}
zip $1 $2 | % {
$whole = $_ # will lose this $_ in the select
compare-object $whole.item1.members $whole.item2.members |
select #{n='Teams'; e={$whole.item1.teams}},
#{n='Category'; e={$whole.item1.Category}},
inputobject,sideindicator
}
Teams Category InputObject SideIndicator
----- -------- ----------- -------------
Team3 E Gonzalez =>
Team3 E Hall =>
Team4 A White <=
For example, replace LINE2 1243 with LINE2 1 because it is on line 1 of test.txt.
# Find the line number:
$lines = sls "LINE2" test.txt | Select-Object -ExpandProperty LineNumber
test.txt:
abc LINE2 1243
lmn LINE2 1250
xyz LINE2 1255
Using:
gc test.txt | % { $_ -replace "LINE2.*", "LINE2 $lines" }
I get:
abc LINE2 1 2 3
lmn LINE2 1 2 3
xyz LINE2 1 2 3
How do I supply index[0], and only index[0], to the first instance of the string, index[1] to the second instance and so on till finished.
Doing it another way:
foreach ($line in $lines){
gc test.txt | % { $_ -replace "LINE2.*", "LINE2 $line" }
}
I get:
abc LINE2 1
lmn LINE2 1
xyz LINE2 1
abc LINE2 2
lmn LINE2 2
xyz LINE2 2
abc LINE2 3
lmn LINE2 3
xyz LINE2 3
How do I get index[0] to only the first instance of the string and so on.
You could use a for loop with an index to achieve this (If I got you right) ;-)
$lines = Select-String "LINE2" -Path C:\sample\test.txt | Select-Object -ExpandProperty LineNumber
Get-Content -Path C:\sample\test.txt -OutVariable Content
for ($index = 0; $index -lt $lines.count; $index++) {
$Content[$index] -replace "LINE2.*", "LINE2 $($lines[$index])"
}
Output:
abc LINE2 1
lmn LINE2 2
xyz LINE2 3
this is a somewhat different way to do things. [grin] what is does ...
reads in the file
i faked this with a here-string, but use Get-Content when doing this for real.
gets the matching lines
it uses the way that -match works against a collection to pull the lines that match the target.
splits on the spaces
selects the 1st two items from that array
adds a $Counter to the collection
joins the three items with a space delimiter
sends the resulting line to the $Results collection
shows that collection on screen
saves it to a text file
here's the code ...
# fake reading in a text file
# in real life, use Get-Content
$InStuff = #'
cba line1 1234
abc LINE2 1243
mnl line4 1244
lmn LINE2 1250
zyx line9 1251
xyz LINE2 1255
qwe line9 1266
'# -split [environment]::NewLine
$Target = 'Line2'
$Counter = 1
$Results = foreach ($IS_Item in ($InStuff -match $Target))
{
$IS_Item.Split(' ')[0..-1] + $Counter -join ' '
$Counter ++
}
# on screen
$Results
# to a file
$Results |
Set-Content -LiteralPath "$env:TEMP\somebadhat.txt"
on screen ...
abc 1243 1
lmn 1250 2
xyz 1255 3
in the text file ...
abc 1243 1
lmn 1250 2
xyz 1255 3
I have written a PowerShell script in which I am reading file (not necessary that it will is CSV). In this I have two columns, case number and cin number. I want to remove all records if this pair is getting repeated
Input File:
abc 1234 class1
def 1234 class2
abc 5678 class3
def 1234 class4
ghi 1001 class5
Desired Output is:
abc 1234 class1
def 1234 class2
abc 5678 class3
ghi 1001 class5
For this I have wrriten following script (input file will have more columns):
$inputFile ="D:\powershell\test files\01PRD_MAGI_12_03.txt";
$outputFile = "D:\powershell\test files\output.txt" ;
$stream = [System.IO.StreamWriter]"$outputFile"
$seenPair = #{}
$lines = Get-Content $inputFile
$count = 0
foreach($line in $lines){
$count = $count + 1
#write all header rows and last line as it is in output file
if($count -eq 1 -or $count -eq 2 -or $count -eq 3 -or $line.startsWith('*') -or $line.startsWith('-')){
$stream.WriteLine($line);
}
else{
$ldrCaseNum = $line.Substring(3,7)
$cin = $line.Substring(70,9)
if($seenPair.Contains($ldrCaseNum)){
$cinVal = $seenPair.Get_Item($ldrCaseNum)
if($cin -eq $cinVal){
continue;
}
else{
$stream.WriteLine($line)
}
}
else{
$seenPair.Add($ldrCaseNum,$cin)
$stream.WriteLine($line)
}
}
}
$stream.close()
But in this case, If a same pair is getting repeated which is stored in $seenPair variable then that record will get removed. But for same key, different value pair will get repeated then it won't remove it, because it holds already a value for that key.
Example:
abc 1234 class1
def 1234 class2
abc 5678 class3
abc 5678 class3
def 1234 class4
ghi 1001 class5
output will be:
abc 1234 class1
def 1234 class2
abc 5678 class3
abc 5678 class3
ghi 1001 class5
Desired output is:
abc 1234 class1
def 1234 class2
abc 5678 class3
ghi 1001 class5
Is there any other solution for this?
I'm going to leave parsing the file into objects as an exercise for the reader, and focus on removing duplicate pairs.
# First we have an array of objects
$Objects = #'
abc 1234 class1
def 1234 class2
abc 5678 class3
def 1234 class4
ghi 1001 class5
'# |ConvertFrom-Csv -Delimiter " " -Header Case,Cin,Class
Now, all we need in order to remove objects where the "Case/Cin" combination already exists is Sort-Object -Unique:
PS C:\> $Objects |Sort-Object -Property Case,Cin -Unique
case cin class
---- --- -----
abc 1234 class1
abc 5678 class3
def 1234 class4
ghi 1001 class5
This is my $array:
Name GUID
---- ----
PC001 AAAA
PC001 BBBB
PC001 CCCC
PC002 AAAA
PC002 DDDD
PC003 AAAA
PC003 CCCC
Here's my script:
$Guid = "CCCC"
$workingName = $array | where Guid -eq $Guid | select name
$array.remove($workingName) #broke!
What I'm trying to achieve is that if $Guid = "CCCC" it would remove all instances of the Name and CCCC from the array, where CCCC exists, so in this instance five lines (three lines for PC001, two lines for PC0003), if it was BBBB it would only remove the three lines for PC001.
FYI: this is part of a much larger script
Well, you could create a new variable or just update the exiting one using a where statement.
$Array | ? {$_.GUID -ne 'CCCC'}
This will return the array back without the entries
Alternatively you can use methods, like you are trying to do there but you need to build the array a bit differently...here is an example.
$Proc = Get-Process | Select -first 2
$a = New-Object System.Collections.ArrayList
$a.Add($Proc[0])
$a.Add($Proc[1])
$a
Write-Warning 'removing index 1'
$a.remove($Proc[1])
$a
$Guid = "CCCC"
$workingName = $array | where Guid -eq $Guid | select -ExpandProperty name
$array = $array | where { $_.name -notin $workingName }
I have a file like this:
CONTOSO-A\AAA
CONTOSO-B\BBB
CONTOSO-B\CCC
CONTOSO-A\AAA
....
....
How can count each line to get:
CONTOSO-A\AAA - 2
CONTOSO-B\BBB - 1
CONTOSO-B\CCC - 1
Get-Content .\file.txt | Group-Object | Select-Object name, count
I'd use a hash table:
$counts = #{}
Get-Content c:\somedir\somefile.txt |
foreach { $counts[$_]++ }
$counts
Name Value
---- -----
CONTOSO-A\AAA 2
CONTOSO-B\CCC 1
CONTOSO-B\BBB 1
The simplest way to do this is probably:
PS C:\temp> #"
CONTOSO-A\AAA
CONTOSO-B\BBB
CONTOSO-B\CCC
CONTOSO-A\AAA
"# | set-content test.txt
get-content test.txt | group -NoElement
Count Name
----- ----
2 CONTOSO-A\AAA
1 CONTOSO-B\BBB
1 CONTOSO-B\CCC
Using the -NoElement option to group or Group-Object means you don't have to do a separate select to extract just name and count.
To get the exact format you asked for:
PS C:\temp> get-content test.txt | group -NoElement | % { $_.Name +" - "+$_.Count }
CONTOSO-A\AAA - 2
CONTOSO-B\BBB - 1
CONTOSO-B\CCC - 1
$stat = #{};
cat file.txt | % { $stat["$_"] = $stat["$_"] + 1; }
$stat;