I have a CSV file with Name and Status columns. We need to read the Name from the CSV file and perform a set of action on that, once we finish the action on one Name we need to update the status of that Name as 'Completed' on the same .csv file.
Name Status
poc_dev_admin
poc_dev_qa
poc_dev_qa1
poc_dev_qa2
poc_dev_qa3
poc_dev_qa7
poc_dev_qa8
poc_dev_ro
poc_dev_support
poc_dev_test1
poc_dev_test14
poc_dev_test15
poc_dev_test16
After execution of poc_dev_admin the CSV file should be like the following:
Name Status
poc_dev_admin Completed
poc_dev_qa
poc_dev_qa1
poc_dev_qa2
poc_dev_qa3
poc_dev_qa7
poc_dev_qa8
poc_dev_ro
poc_dev_support
poc_dev_test1
poc_dev_test14
poc_dev_test15
poc_dev_test16
I have tried with this logic and it updates Status column for all groups rather than updating it after processing the respective group. May I what change should we make to work this as expected?
$P = Import-Csv -Path c:\test\abintm.csv
foreach ($groupname in $P) {
### Process $groupname####
$newprocessstatus = 'Completed'
$P | Select-Object *, #{n='Status';e={$newprocessstatus}} |
Export-Csv -NoTypeInformation -Path C:\test\abintm.csv
}
Something like this should work for what you described:
$file = 'c:\test\abintm.csv'
$csv = Import-Csv $file
foreach ($row in $csv) {
$groupname = $row.Name
# ...
# process $groupname
# ...
$row.Status = 'Completed'
}
$csv | Export-Csv $file -NoType
If you need to update the CSV before processing the next record you can put the CSV export at the end of the loop (right after setting $row.Status to "Completed").
Modified your code for the use case you've specified,
Changed current item variable in foreach loop to $row instead of $groupname
However, defined the $groupname variable inside the foreach block
$List = Import-Csv -Path "c:\test\abintm.csv"
foreach ($row in $List) {
$groupName = $row.Name
$status = $row.Status
if ($groupName -eq "poc_dev_admin") {
$status = "Completed"
($List | ? { $_.Name -eq $groupName }).Status = $status
}
}
if (($List | ? { $_.Status -eq "Completed" }) -ne $null) {
# $List has been modifed and hence write back the updated $List to the csv file
$List | Export-Csv -Notypeinformation -Path "C:\test\abintm.csv"
}
# else nothing is written
Related
UPDATE (06/21/22): See my updated script below, which utilizes some of the answer.
I am building a script to search for $name through a large batch of CSV files. These files can be as big as 67,000 KB. This is my script that I use to search the files:
Powershell Script
Essentially, I use Import-Csv. I change a few things depending on the file name, however. For example, some files don't have headers, or they may use a different delimiter. Then I store all the matches in $results and then return that variable. This is all put in a function called CSVSearch for ease of running.
#create function called CSV Search
function CSVSearch{
#prompt
$name = Read-Host -Prompt 'Input name'
#set path to root folder
$path = 'Path\to\root\folder\'
#get the file path for each CSV file in root folder
$files = Get-ChildItem $path -Filter *.csv | Select-Object -ExpandProperty FullName
#count files in $files
$filesCount = $files.Count
#create empty array, $results
$results= #()
#count for write-progress
$i = 0
foreach($file in $files){
Write-Progress -Activity "Searching files: $i out of $filesCount searched. $resultsCount match(es) found" -PercentComplete (($i/$files.Count)*100)
#import method changes depending on CSV file name found in $file (headers, delimiters).
if($file -match 'File1*'){$results += Import-Csv $file -Header A, Name, C, D -Delimiter '|' | Select-Object *,#{Name='FileName';Expression={$file}} | Where-Object { $_.'Name' -match $name}}
if($file -match 'File2*'){$results += Import-Csv $file -Header A, B, Name -Delimiter '|' | Select-Object *,#{Name='FileName';Expression={$file}} | Where-Object { $_.'Name' -match $name}}
if($file -match 'File3*'){$results += Import-Csv $file | Select-Object *,#{Name='FileName';Expression={$file}} | Where-Object { $_.'Name' -match $name}}
if($file -match 'File4*'){$results += Import-Csv $file | Select-Object *,#{Name='FileName';Expression={$file}} | Where-Object { $_.'Name' -match $name}}
$i++
$resultsCount = $results.Count
}
#if the loop ends and $results array is empty, return "No matches."
if(!$results){Write-Host 'No matches found.' -ForegroundColor Yellow}
#return results stored in $results variable
else{$results
Write-Host $resultsCount 'matches found.' -ForegroundColor Green
Write-Progress -Activity "Completed" -Completed}
}
CSVSearch
Below are what the CSV files look like. Obviously, the amount of the data below is not going to equate to the actual size of the files. But below is the basic structure:
CSV files
File1.csv
1|Moonknight|QWEPP|L
2|Star Wars|QWEPP|T
3|Toy Story|QWEPP|U
File2.csv
JKLH|1|Moonknight
ASDF|2|Star Wars
QWER|3|Toy Story
File3.csv
1,Moonknight,AA,DDD
2,Star Wars,BB,CCC
3,Toy Story,CC,EEE
File4.csv
1,Moonknight,QWE
2,Star Wars,QWE
3,Toy Story,QWE
The script works great. Here is an example of the output I would receive if $name = Moonknight:
Example of results
A : 1
Name : Moonknight
C: QWE
FileName: Path\to\root\folder\File4.csv
A : 1
Name : Moonknight
B : AA
C : DDD
FileName: Path\to\root\folder\File3.csv
A : JKLH
B : 1
Name : Moonknight
FileName: Path\to\root\folder\File2.csv
A : 1
Name : Moonknight
C : QWEPP
D : L
FileName: Path\to\root\folder\File1.csv
4 matches found.
However, it is very slow, and I have a lot of files to search through. Any ideas on how to speed my script up?
Edit: I must mention. I tried importing the data into a hash table and then searching the hash table, but that was much slower.
UPDATED SCRIPT - My Solution (06/21/22):
This update utilizes some of Santiago's script below. I was having a hard time decoding everything he did, as I am new to PowerShell. So I sort of jerry rigged my own solution, that used a lot of his script/ideas.
The one thing that made a huge difference was outputting $results[$i] which returns the most recent match as the script is running. Probably not the most efficient way to do it, but it works for what I'm trying to do. Thanks!
function CSVSearch{
[cmdletbinding()]
param(
[Parameter(Mandatory)]
[string] $Name
)
$files = Get-ChildItem 'Path\to\root\folder\' -Filter *.csv -Recurse | %{$_.FullName}
$results = #()
$i = 0
foreach($file in $files){
if($file -like '*File1*'){$results += Import-Csv $file -Header A, Name, C, D -Delimiter '|' | Where-Object { $_.'Name' -match $Name} | Select-Object *,#{Name='FileName';Expression={$file}}}
if($file -like' *File2*'){$results += Import-Csv $file -Header A, B, Name -Delimiter '|' | Where-Object { $_.'Name' -match $Name} | Select-Object *,#{Name='FileName';Expression={$file}}}
if($file -like '*File3*'){$results += Import-Csv $file | Where-Object { $_.'Name' -match $Name} | Select-Object *,#{Name='FileName';Expression={$file}}}
if($file -like '*File4*'){$results += Import-Csv $file | Where-Object { $_.'Name' -match $Name} | Select-Object *,#{Name='FileName';Expression={$file}}}
$results[$i]
$i++
}
if(-not $results) {
Write-Host 'No matches found.' -ForegroundColor Yellow
return
}
Write-Host "$($results.Count) matches found." -ForegroundColor Green
}
Give this one a try, it should be a bit faster. Select-Object has to reconstruct your object, if you use it before filtering, you're actually recreating your entire CSV, you want to filter first (Where-Object / .Where) before reconstructing it.
.Where should be a faster than Where-Object here, the caveat is that the intrinsic method requires that the collections already exists in memory, there is no pipeline processing and no streaming.
Write-Progress will only slow down your script, better remove it.
Lastly, you can use splatting to avoid having multiple if conditions.
function CSVSearch {
[cmdletbinding()]
param(
[Parameter(Mandatory)]
[string] $Name,
[Parameter()]
[string] $Path = 'Path\to\root\folder\'
)
$param = #{
File1 = #{ Header = 'A', 'Name', 'C', 'D'; Delimiter = '|' }
File2 = #{ Header = 'A', 'B', 'Name' ; Delimiter = '|' }
File3 = #{}; File4 = #{} # File3 & 4 should have headers ?
}
$results = foreach($file in Get-ChildItem . -Filter file*.csv) {
$thisparam = $param[$file.BaseName]
$thisparam['LiteralPath'] = $file.FullName
(Import-Csv #thisparam).where{ $_.Name -match $name } |
Select-Object *, #{Name='FileName';Expression={$file}}
}
if(-not $results) {
Write-Host 'No matches found.' -ForegroundColor Yellow
return
}
Write-Host "$($results.Count) matches found." -ForegroundColor Green
$results
}
CSVSearch -Name Moonknight
If you want the function to stream results as they're found, you can use a Filter, this is a very efficient filtering technique, certainly faster than Where-Object:
function CSVSearch {
[cmdletbinding()]
param(
[Parameter(Mandatory)]
[string] $Name,
[Parameter()]
[string] $Path = 'Path\to\root\folder\'
)
begin {
$param = #{
File1 = #{ Header = 'A', 'Name', 'C', 'D'; Delimiter = '|' }
File2 = #{ Header = 'A', 'B', 'Name' ; Delimiter = '|' }
File3 = #{}; File4 = #{} # File3 & 4 should have headers ?
}
$counter = [ref] 0
filter myFilter {
if($_.Name -match $name) {
$counter.Value++
$_ | Select-Object *, #{N='FileName';E={$file}}
}
}
}
process {
foreach($file in Get-ChildItem $path -Filter *.csv) {
$thisparam = $param[$file.BaseName]
$thisparam['LiteralPath'] = $file.FullName
Import-Csv #thisparam | myFilter
}
}
end {
if(-not $counter.Value) {
Write-Host 'No matches found.' -ForegroundColor Yellow
return
}
Write-Host "$($counter.Value) matches found." -ForegroundColor Green
}
}
Each user has a domain1.com and domain2.com separate email address. I would like to compare email addresses within CSV1 & CSV2 and if the prefix name on their emails match, pull the value into CSV1. This will later allow me to run other powershell commands on a source / target kind of scenario.
I have it somewhat, working below although the domain2_email column isn't matching the user row. It looks to be trying to import all of them into an array.
I have used Joe.Bloggs here to test if the IF is working, but ideally - I would like to have it search each entry in CSV1.
End Goal:
Search CSV2 with the name part of the primarySMTPAddress value and if match, put them into another column within CSV1.
CSV1 will have a user with domain1.com and domain2.com values within the row.
# Pull in mailbox data
Get-EXOMailbox -Filter {EmailAddresses -like *#domain1.com -and RecipientTypeDetails -notlike "SharedMailbox"} | Export-Csv -Path './domain1export.csv'
Get-EXOMailbox -Filter {EmailAddresses -like *#domain2.com -and RecipientTypeDetails -notlike "SharedMailbox"} | Export-Csv -Path './domain2xexport.csv'
$outfile = './outfile.csv'
$finalOutfile = './finalOutfile.csv'
# Check if outfile.csv exists, if it does - delete.
if (test-path $outfile) {
Remove-Item $outfile
Write-Host "$outfile has been deleted"
}
$CSV1 = Import-Csv -Path './domain1export.csv'
$CSV2 = Import-Csv -Path './domain2export.csv'
$CVS1 | foreach {
# Splits the name prefix from email
$_ | Select-Object *,#{Name='Name_Prefix';Expression={$_.PrimarySmtpAddress.Split("#")[0] }} | Export-Csv -Path $outfile -Append -NoTypeInformation
}
$outfile2 = Import-Csv -Path $outfile
foreach ($item in $outfile2) {
if ($outfile2.Name_Prefix -match "Joe.Bloggs") {
$item | Select-Object *,#{Name='Domain2_email';Expression={$CSV2.PrimarySmtpAddress}} | Export-Csv -Path $finalOutfile -Append -NoTypeInformation
}
}
Data
CSV1
UserPrincipalName,Alias,DisplayName,EmailAddresses,PrimarySmtpAddress,RecipientType,RecipientTypeDetails
joe.bloggs#domain1.com,d1bloggsj,Domain1 Joe Bloggs,SIP:joe.blogggs#domain1.com SMTP:joe.blogggs#domain1.com,Joe.Bloggs#domain1.com,UserMailbox,UserMailbox
foo.bar#domain1.com,d1barf,Domain1 Foo Bar,SIP:foo.bar#domain1.com SMTP:foo.bar#domain1.com ,foo.bar#domain1.com,UserMailbox,UserMailbox
CSV2
UserPrincipalName,Alias,DisplayName,EmailAddresses,PrimarySmtpAddress,RecipientType,RecipientTypeDetails
joe.bloggs#domain2.com,d2bloggsj,Domain2 Joe Bloggs,SIP:joe.blogggs#domain2.com SMTP:joe.blogggs#domain2.com,Joe.Bloggs#domain2.com,UserMailbox,UserMailbox
foo.bar#domain2.com,d1barf,Domain2 Foo Bar,SIP:foo.bar#domain2.com SMTP:foo.bar#domain2.com ,foo.bar#domain2.com,UserMailbox,UserMailbox
So both CSV1 and CSV2 would contain a column called PrimarySmtpAddress and you need to match those, correct?
Try
$CSV1 = Import-Csv -Path './domain1export.csv'
$CSV2 = Import-Csv -Path './domain2export.csv'
$result = foreach ($item in $csv1) {
$prefix = $item.PrimarySmtpAddress.Split("#")[0]
# try and find a match
$matching = $csv2 | Where-Object { $_.PrimarySmtpAddress.Split("#")[0] -eq $prefix }
$item | Select-Object *, #{Name = 'Name_Prefix'; Expression = {$prefix }},
#{Name = 'Domain2_email';Expression = {$matching.PrimarySmtpAddress }}
}
$finalOutfile = './finalOutfile.csv'
$result | Export-Csv -Path $finalOutfile -NoTypeInformation
BTW. According to the docs, the -Filter parameter is a string, not a scriptblock, so you should use
Get-EXOMailbox -Filter "EmailAddresses -like '*#domain1.com' -and RecipientTypeDetails -ne 'SharedMailbox'"
I would do it like this, first create a hash table using the Csv1 where the Keys are the mail prefix and the Values are the mail suffix and the, using a calculated property with Select-Object you can search for each line on Csv2 if the mail's prefix exists and obtain the corresponding Values (mail suffix):
Note, below code assumes that there are unique mail prefixes, if this was not the case, an if condition would need to be added to the first loop to avoid Key collision. I would also recommend you to use the MailAddress Class to parse your emails.
$map = #{}
foreach($line in $csv1) {
$mail = [mailaddress]$line.PrimarySmtpAddress
$map[$mail.User] = $mail
}
$csv2 | Select-Object *, #{
Name = 'SecondaryDomain'
Expression = { $map[([mailaddress]$_.PrimarySmtpAddress).User].Host }
}, #{
Name = 'SecondarySmtpAddress'
Expression = { $map[([mailaddress]$_.PrimarySmtpAddress).User].Address }
}
$result | Format-Table
I have two csv file where I contain data, I need to check if value from CSV 1 exist in CSV 2 and if so then replace this value in file2 with data from file1, if no just skip to another row,
File1.csv
NO;Description
L001;DREAM
L002;CAR
L003;PHONE
L004;HOUSE
L005;PLANE
File2.csv
ID;Name;Status*;Scheduled Start Date;Actual Start Date;Actual End Date;Scheduled End Date;SLA
144862;DREAM;Scheduled;1524031200;;;1524033000;
149137;CAR;Implementation In Progress;1528588800;;;1548968400;
150564;PHONE;Scheduled;1569456000;;;1569542400;
150564;HOUSE;Scheduled;1569456000;;;1569542400;
150564;PLANE;;;;;;
I tried something like that but it is not working for me:
$file1 = Import-Csv "C:\Users\file1.csv" |Select-Object -ExpandProperty Description
$file2 = Import-Csv "C:\Users\file1.csv" |Select-Object -ExpandProperty NO
Import-Csv "C:\Users\file3.csv" |Where-Object {$file1 -like $_.Name} |ForEach-Object {
$_.Name = $file2($_.NO)
} |Out-File "C:\Users\File4.csv"
File4.csv should like that:
ID;Name;Status*;Scheduled Start Date;Actual Start Date;Actual End Date;Scheduled End Date;SLA
144862;L001;Scheduled;1524031200;;;1524033000;
149137;L002;Implementation In Progress;1528588800;;;1548968400;
150564;L003;Scheduled;1569456000;;;1569542400;
150564;L004;Scheduled;1569456000;;;1569542400;
150564;L005;;;;;;
Maybe there is another way to achive my goal! Thank you
Here's one approach you can take:
Import both CSV files with Import-Csv
Create a lookup hash table from the first CSV file, where the Description you want to replace are the keys, and NO are the values.
Go through the second CSV file, and replace any values from the Name column from the hash table, if the key exists. We can use System.Collections.Hashtable.ContainsKey to check if the key exists. This is a constant time O(1) operation, so lookups are fast.
Then we can export the final CSV with Export-Csv. I used -UseQuotes Never to put no " quotes in your output file. This feature is only available in PowerShell 7. For lower PowerShell versions, you can have a look at How to remove all quotations mark in the csv file using powershell script? for other alternatives to removing quotes from a CSV file.
Demo:
$csvFile1 = Import-Csv -Path .\File1.csv -Delimiter ";"
$csvFile2 = Import-Csv -Path .\File2.csv -Delimiter ";"
$ht = #{}
foreach ($item in $csvFile1) {
if (-not [string]::IsNullOrEmpty($item.Description)) {
$ht[$item.Description] = $item.NO
}
}
& {
foreach ($line in $csvFile2) {
if ($ht.ContainsKey($line.Name)) {
$line.Name = $ht[$line.Name]
}
$line
}
} | Export-Csv -Path File4.csv -Delimiter ";" -NoTypeInformation -UseQuotes Never
Or instead of wrapping the foreach loop inside a script block using the Call Operator &, we can use Foreach-Object. You can have a look at about_script_blocks for more information about script blocks.
$csvFile2 | ForEach-Object {
if ($ht.ContainsKey($_.Name)) {
$_.Name = $ht[$_.Name]
}
$_
} | Export-Csv -Path File4.csv -Delimiter ";" -NoTypeInformation -UseQuotes Never
File4.csv
ID;Name;Status*;Scheduled Start Date;Actual Start Date;Actual End Date;Scheduled End Date;SLA
144862;L001;Scheduled;1524031200;;;1524033000;
149137;L002;Implementation In Progress;1528588800;;;1548968400;
150564;L003;Scheduled;1569456000;;;1569542400;
150564;L004;Scheduled;1569456000;;;1569542400;
150564;L005;;;;;;
Update
For handling multiple values with the same Name, we can transform the above to use a hash table of System.Management.Automation.PSCustomObject, where we have two properties Count to keep track of the current item we're seeing and NO which is an array of numbers:
$csvFile1 = Import-Csv -Path .\File1.csv -Delimiter ";"
$csvFile2 = Import-Csv -Path .\File2.csv -Delimiter ";"
$ht = #{}
foreach ($row in $csvFile1) {
if (-not $ht.ContainsKey($row.Description) -and
-not [string]::IsNullOrEmpty($item.Description)) {
$ht[$row.Description] = [PSCustomObject]#{
Count = 0
NO = #()
}
}
$ht[$row.Description].NO += $row.NO
}
& {
foreach ($line in $csvFile2) {
if ($ht.ContainsKey($line.Name)) {
$name = $line.Name
$pos = $ht[$name].Count
$line.Name = $ht[$name].NO[$pos]
$ht[$name].Count += 1
}
$line
}
} | Export-Csv -Path File4.csv -Delimiter ";" -NoTypeInformation -UseQuotes Never
If your files aren't too big, you could do this with a simple ForEach-Object loop:
$csv1 = Import-Csv -Path 'D:\Test\File1.csv' -Delimiter ';'
$result = Import-Csv -Path 'D:\Test\File2.csv' -Delimiter ';' |
ForEach-Object {
$name = $_.Name
$item = $csv1 | Where-Object { $_.Description -eq $name } | Select-Object -First 1
# update the Name property and output the item
if ($item) {
$_.Name = $item.NO
# if you output the row here, the result wil NOT contain rows that did not match
# $_
}
# if on the other hand, you would like to retain the items that didn't match unaltered,
# then output the current row here
$_
}
# output on screen
$result | Format-Table -AutoSize
#output to new CSV file
$result | Export-Csv -Path 'D:\Test\File4.csv' -Delimiter ';' -NoTypeInformation
Result on screen:
ID Name Status* Scheduled Start Date Actual Start Date Actual End Date Scheduled End Date SLA
-- ---- ------- -------------------- ----------------- --------------- ------------------ ---
144862 L001 Scheduled 1524031200 1524033000
149137 L002 Implementation In Progress 1528588800 1548968400
150564 L003 Scheduled 1569456000 1569542400
150564 L004 Scheduled 1569456000 1569542400
150564 L005
I am fairly new in powershell scripting and need help on the following output in a csv format. I am trying to select a column e.g. ACCOUNT.UPLOAD and make and if/else statement to output it in another csv file. May someone help please.
Output csv should look like below:
$results = Import-Csv 'C:\Users\test\Desktop\Test\customer.csv' |
Select-Object "ACCOUNT.UPLOAD"
ForEach ($row in $results)
{
If ($row.Type0 -ne 'CP000101', 'CP000102')
{
$row."ACCOUNT.UPLOAD" = "$($row.ACCOUNT.UPLOAD)"
Write-Host $row."ACCOUNT.UPLOAD"
}
}
$results | Export-Csv C:\Users\test\Desktop\Test\test.csv -NoTypeInformation
Thank you
This will get you what you need. Added comments to explain what I have done.
$results = Import-Csv "C:\Users\test\Desktop\Test\customer.csv" | Select-Object "ACCOUNT.UPLOAD"
# Created array to be able to add individual results from foreach
$TheCSV = #()
ForEach ($row in $results) {
# You can use a -ne in the right hand side, if done like this.
If (($row.'ACCOUNT.UPLOAD' -ne 'CP000101') -and $row.'ACCOUNT.UPLOAD' -ne 'CP000102') {
# Adds the ROW column to the entry and finds the index that it was in from $results.
# Did a +2 as it does not include the header and it starts at value 0. So to match it up with the actual excel row numbers, add 2.
$row | Add-Member -Name 'ROW' -type NoteProperty -Value "$([array]::IndexOf($results.'ACCOUNT.UPLOAD',$row.'ACCOUNT.UPLOAD')+2)"
$TheCSV += $row
}
}
$TheCSV | Export-Csv "C:\Users\test\Desktop\Test\test.csv" -NoTypeInformation
Do it PowerShell way:
param(
[Parameter(Position=0)]
$InputFile = 'D:\\Temp\\Data.csv',
[Parameter(Position=1)]
$OutputFile = 'D:\\Temp\\Output.csv'
)
Import-Csv $InputFile |
Select-Object "ACCOUNT.UPLOAD" |
%{
$lineno++
if ($_.'ACCOUNT.UPLOAD' -notin #('CP000101', 'CP000102')) {
$_ | Add-Member -Name 'ROW' -type NoteProperty -Value $lineno
$_ # Output to pipeline
}
} -Begin { $lineno = 1 } |
Export-Csv $OutputFile -NoTypeInformation
Using:
.\Script.ps1
.\Script.ps1 inputfilename.csv outputfilefname.csv
$csv = Import-Csv "C:\csv.csv"
Foreach ($line in $csv) {
Get-CMBoundary -BoundaryName $line.Name |
Where-Object { $_.Value -eq $line.Subnet } |
Remove-CMBoundary -Verbose
}
The above code works perfectly to clear subnets from an sccm site. What i want to do though is, only remove 1 Site or Line in the CSV at a time - using a variable already entered.
So something like:
$csv = Import-Csv "C:\csv.csv"
$site = "London"
Foreach ($line in $csv) {
Get-CMBoundary -BoundaryName $line.$site |
Where-Object { $_.Value -eq $line.Subnet } |
Remove-CMBoundary -Verbose
}