I need to do below using PowerShell scripting.
I have the below data in CSV file. I need to append another prefix to the name according to the subject supplied
Name,Subject,Roll,Div
Mark,Agri,3,Div1,
Tom,CS,6,Div3,
Alex,Aero,9,Div6
Suppose i am supplied the subject as Agri, the name should have prefix as P-. So the same CSV file will be saved and have below content
Name,Subject,Roll,Div
P-Mark,Agri,3,Div1,
Tom,CS,6,Div3,
Alex,Aero,9,Div6
Here's what I have so far:
(Import-Csv E:\Test.csv -Delimiter ',') | ForEach-Object{
if($_.Subject -match "Agri" )
{
$_ = "P-" + "$_";
}
else
{
$_ = "F-" + "$_";
}
} | Export-Csv E:\Test.csv -Delimiter ','
I wasn't able to pipe this together - might just be ignorance on my part about powershell syntax. I was able to do it with the following using two lines:
($csv = Import-Csv E:\Test.csv -Delimiter ',') | ForEach {
if ($_.Subject -match "Agri") {
$_.Name = 'P-' + $_.Name
} else {
$_.Name = 'F-' + $_.Name
}
}
$csv | Export-Csv E:\Test.csv -Delimiter ',' -NoType
Notice:
$_ = "P-" + $_;
Becomes
$_.Name = "P-" + $_.Name;
It seems to me the use of paren's in the following clause.
($csv = Import-Csv E:\Test.csv -Delimiter ',')
will assign to $csv the output of Import-Csv PRIOR to the ForEach processing and therefore the ForEach effects are completely lost. Not sure
what you purpose is in using those paren's.
Related
We are supposed to edit a CSV-file in PowerShell and export the file afterwards
The CSV-file contains:
"ID";"date";"number"
"YYY-12345";"24.01.2023";"123456910"
Now we should add "-001" after the ID-block..but for the whole column
thats the code for now:
$folder_csv = 'C:\Abschluss'
$folder_fileout = 'C:\Abschluss\Ausgabe'
$files = Get-ChildItem $folder_csv -File -Filter *.csv
foreach ($file in $files) {
$data = Import-csv $($file.Fullname) -Delimiter ";"
foreach($dataset in $data) {
$data.ID + "-001"
$data | export-csv "$folder_fileout\test.txt" -Delimiter ";" -NoType -Encoding UTF8 -append
}
}
it shows the results in the console of PowerShell but not in the created .txt-file.
You're pretty close with your code, the main issue is that you're currently doing $data.ID + "-001" and $data is actually the complete array of objects, you want to refer to $dataset instead (the object being enumerated). Aside from that, it seems you're looking to merge all Csvs into one, hence you could leverage the PowerShell pipeline using an outer ForEach-Object loop instead of foreach:
$folder_csv = 'C:\Abschluss'
$folder_fileout = 'C:\Abschluss\Ausgabe'
Get-ChildItem $folder_csv -File -Filter *.csv | ForEach-Object {
foreach($line in $_ | Import-csv -Delimiter ';') {
# update the Id property of this object
$line.Id = $line.Id + '-001'
# output the updated object
$line
}
} | Export-Csv "$folder_fileout\test.txt" -Delimiter ";" -NoTypeInformation -Encoding UTF8
I am trying to Remove unnecessary commas in a column in the CSV file. For now, I know a few issues and hard-coded it, But I wanted the code to be dynamic. Any suggestions are greatly appreciated.
$FilePath = "C:\Test\"
Get-ChildItem $FilePath -Filter .csv | ForEach-Object {
(Get-Content $_.FullName -Raw) | Foreach-Object {
$_ -replace ',"Frederick, Fred",' , ',"Frederick Fred",' `
-replace ',"Brian, Josiah",' , ',"Brian Josiah",' `
-replace ',"Lisinopril ,Tablet / 20MG",' , ',"Lisinopril Tablet / 20MG",'
} | Set-Content $_.FullName
}
Try this, also note that I worked with the csv sample that you gave here.It might not work with other csv files.
also make sure that you change the path of %YOURCSVFILE% to the real path of your file
#import the csv
$csv = Import-Csv -Path %YOURCSVFILE% -Delimiter ','
#going each row and replacing commas
foreach ($desc in $csv){
$desc.Desc = $desc.Desc -replace ',',''
}
#exporting the csv
$csv | Export-csv -NoTypeInformation "noCommas.csv"
Here's a few more alteratives for you:
Method 1. Loop through the rows with foreach(..) and capture the output:
$result = foreach ($row in (Import-Csv -Path 'D:\Test\FileWithCommasInDescription.csv')) {
$row.Desc = $row.Desc -replace ','
$row # output the updated item
}
$result | Export-Csv -Path 'D:\Test\FileWithoutCommasInDescription.csv' -NoTypeInformation
Method 2. Use ForEach-Object and the automatic variable $_. Pipe the results through:
Import-Csv -Path 'D:\Test\FileWithCommasInDescription.csv' | ForEach-Object {
$_.Desc = $_.Desc -replace ','
$_ # output the updated item
} | Export-Csv -Path 'D:\Test\FileWithoutCommasInDescription.csv' -NoTypeInformation
Method 3. Use a calculated property:
Import-Csv -Path 'D:\Test\FileWithCommasInDescription.csv' |
Select-Object ID, #{Name = 'Desc'; Expression = {$_.Desc -replace ','}}, Nbr -ExcludeProperty Desc |
Export-Csv -Path 'D:\Test\FileWithoutCommasInDescription.csv' -NoTypeInformation
All will result in a new CSV file
"ID","Desc","Nbr"
"12","Frederick Fred","11"
"21","Brian Josiah","31"
"13","Lisinopril Tablet / 20MG","17"
I have two csv file where I contain data, I need to check if value from CSV 1 exist in CSV 2 and if so then replace this value in file2 with data from file1, if no just skip to another row,
File1.csv
NO;Description
L001;DREAM
L002;CAR
L003;PHONE
L004;HOUSE
L005;PLANE
File2.csv
ID;Name;Status*;Scheduled Start Date;Actual Start Date;Actual End Date;Scheduled End Date;SLA
144862;DREAM;Scheduled;1524031200;;;1524033000;
149137;CAR;Implementation In Progress;1528588800;;;1548968400;
150564;PHONE;Scheduled;1569456000;;;1569542400;
150564;HOUSE;Scheduled;1569456000;;;1569542400;
150564;PLANE;;;;;;
I tried something like that but it is not working for me:
$file1 = Import-Csv "C:\Users\file1.csv" |Select-Object -ExpandProperty Description
$file2 = Import-Csv "C:\Users\file1.csv" |Select-Object -ExpandProperty NO
Import-Csv "C:\Users\file3.csv" |Where-Object {$file1 -like $_.Name} |ForEach-Object {
$_.Name = $file2($_.NO)
} |Out-File "C:\Users\File4.csv"
File4.csv should like that:
ID;Name;Status*;Scheduled Start Date;Actual Start Date;Actual End Date;Scheduled End Date;SLA
144862;L001;Scheduled;1524031200;;;1524033000;
149137;L002;Implementation In Progress;1528588800;;;1548968400;
150564;L003;Scheduled;1569456000;;;1569542400;
150564;L004;Scheduled;1569456000;;;1569542400;
150564;L005;;;;;;
Maybe there is another way to achive my goal! Thank you
Here's one approach you can take:
Import both CSV files with Import-Csv
Create a lookup hash table from the first CSV file, where the Description you want to replace are the keys, and NO are the values.
Go through the second CSV file, and replace any values from the Name column from the hash table, if the key exists. We can use System.Collections.Hashtable.ContainsKey to check if the key exists. This is a constant time O(1) operation, so lookups are fast.
Then we can export the final CSV with Export-Csv. I used -UseQuotes Never to put no " quotes in your output file. This feature is only available in PowerShell 7. For lower PowerShell versions, you can have a look at How to remove all quotations mark in the csv file using powershell script? for other alternatives to removing quotes from a CSV file.
Demo:
$csvFile1 = Import-Csv -Path .\File1.csv -Delimiter ";"
$csvFile2 = Import-Csv -Path .\File2.csv -Delimiter ";"
$ht = #{}
foreach ($item in $csvFile1) {
if (-not [string]::IsNullOrEmpty($item.Description)) {
$ht[$item.Description] = $item.NO
}
}
& {
foreach ($line in $csvFile2) {
if ($ht.ContainsKey($line.Name)) {
$line.Name = $ht[$line.Name]
}
$line
}
} | Export-Csv -Path File4.csv -Delimiter ";" -NoTypeInformation -UseQuotes Never
Or instead of wrapping the foreach loop inside a script block using the Call Operator &, we can use Foreach-Object. You can have a look at about_script_blocks for more information about script blocks.
$csvFile2 | ForEach-Object {
if ($ht.ContainsKey($_.Name)) {
$_.Name = $ht[$_.Name]
}
$_
} | Export-Csv -Path File4.csv -Delimiter ";" -NoTypeInformation -UseQuotes Never
File4.csv
ID;Name;Status*;Scheduled Start Date;Actual Start Date;Actual End Date;Scheduled End Date;SLA
144862;L001;Scheduled;1524031200;;;1524033000;
149137;L002;Implementation In Progress;1528588800;;;1548968400;
150564;L003;Scheduled;1569456000;;;1569542400;
150564;L004;Scheduled;1569456000;;;1569542400;
150564;L005;;;;;;
Update
For handling multiple values with the same Name, we can transform the above to use a hash table of System.Management.Automation.PSCustomObject, where we have two properties Count to keep track of the current item we're seeing and NO which is an array of numbers:
$csvFile1 = Import-Csv -Path .\File1.csv -Delimiter ";"
$csvFile2 = Import-Csv -Path .\File2.csv -Delimiter ";"
$ht = #{}
foreach ($row in $csvFile1) {
if (-not $ht.ContainsKey($row.Description) -and
-not [string]::IsNullOrEmpty($item.Description)) {
$ht[$row.Description] = [PSCustomObject]#{
Count = 0
NO = #()
}
}
$ht[$row.Description].NO += $row.NO
}
& {
foreach ($line in $csvFile2) {
if ($ht.ContainsKey($line.Name)) {
$name = $line.Name
$pos = $ht[$name].Count
$line.Name = $ht[$name].NO[$pos]
$ht[$name].Count += 1
}
$line
}
} | Export-Csv -Path File4.csv -Delimiter ";" -NoTypeInformation -UseQuotes Never
If your files aren't too big, you could do this with a simple ForEach-Object loop:
$csv1 = Import-Csv -Path 'D:\Test\File1.csv' -Delimiter ';'
$result = Import-Csv -Path 'D:\Test\File2.csv' -Delimiter ';' |
ForEach-Object {
$name = $_.Name
$item = $csv1 | Where-Object { $_.Description -eq $name } | Select-Object -First 1
# update the Name property and output the item
if ($item) {
$_.Name = $item.NO
# if you output the row here, the result wil NOT contain rows that did not match
# $_
}
# if on the other hand, you would like to retain the items that didn't match unaltered,
# then output the current row here
$_
}
# output on screen
$result | Format-Table -AutoSize
#output to new CSV file
$result | Export-Csv -Path 'D:\Test\File4.csv' -Delimiter ';' -NoTypeInformation
Result on screen:
ID Name Status* Scheduled Start Date Actual Start Date Actual End Date Scheduled End Date SLA
-- ---- ------- -------------------- ----------------- --------------- ------------------ ---
144862 L001 Scheduled 1524031200 1524033000
149137 L002 Implementation In Progress 1528588800 1548968400
150564 L003 Scheduled 1569456000 1569542400
150564 L004 Scheduled 1569456000 1569542400
150564 L005
I am trying to convert two TXT files into one CSV file using powershell script. When files have same structure, and same number of rows then case looks be easy. But in my case txt files have diffrent structure.
Pipe sign in both txt files is not a delimiter should be treat as normal character and it is a string.
File URL.txt
L5020|http://linktosite.de|URL
L100|http://sitelink.de|URL
L50|http://abcde.de|URL
L511|http://bbcccddeee.de|URL
L300|http://link123456.de|URL
L5450|http://randomlink.de|URL_DE
L5460|http://randomwebsitelink.de|URL_DE
File URL1.txt
L5020|http://linktosite.de|URL|P555
L100|http://sitelink.de|URL|P523
L50|http://abcde.de|URL|P53
L511|http://bbcccddeee.de|URL|P540
CSV which I expect should look like as below and delimiter is ";"
HEADER1;HEADER2
L5020|http://linktosite.de|URL;L5020|http://linktosite.de|URL|P555
L100|http://sitelink.de|URL;L100|http://sitelink.de|URL|P523
L50|http://abcde.de|URL;L50|http://abcde.de|URL|P53
L511|http://bbcccddeee.de|URL;L511|http://bbcccddeee.de|URL|P540
L300|http://link123456.de|URL;
L5450|http://randomlink.de|URL_DE;
L5460|http://randomwebsitelink.de|URL_DE;
I tried something like that
$URL = "C:\Users\XXX\Desktop\URL.txt"
$URLcontent = Get-Content $URL
$URL1 = "C:\Users\XXX\Desktop\URL1.txt"
$URLcontent1 = Get-Content $URL1
$results = #() # Empty array to store new created rows in
$csv = Import-CSV "C:\Users\XXX\Desktop\map.csv" -Delimiter ';'
foreach ($row in $csv) {
$properties = [ordered]#{
HEADER1 = $URLcontent
HEADER2 = $URLcontent1
}
# insert the new row as an object into the results-array
$results += New-Object psobject -Property $properties
}
# foreach-loop filled the results-array - export it as a CSV-file
$results | Export-Csv "C:\Users\XXXX\Desktop\map_final.csv" -NoTypeInformation
And something like that:
import-csv URL.txt -Header 'HEADER1' | Export-CSV "C:\Users\xxx\Desktop\URL.csv" -Delimiter ';' -NoTypeInformation
import-csv URL1.txt -Header 'HEADER2' | Export-CSV "C:\Users\xxx\Desktop\URL1.csv" -Delimiter ';' -NoTypeInformation
Get-ChildItem "C:\Users\xx\Desktop" -Filter "URL*.csv" | Select-Object -ExpandProperty FullName | Import-Csv | Export-Csv .\combinedcsvs.csv -NoTypeInformation -Append
Without any succes...
BR
Based on the updates in your question, if you want to build something yourself, you probably want to do something like this:
$Url1 = #(Get-Content .\URL1.txt)
$i = 0
Get-Content .\URL.txt | Foreach-Object {
[pscustomobject]#{
HEADER1 = $_
HEADER2 = If ($i -lt $URL1.Count) { $URL1[$i++] }
}
} | Export-Csv .\combinedcsvs.csv -Delimiter ';' -NoTypeInformation -Append
In case you do not want to go through the hassle of reinventing the wheel (with all pitfalls including performance tuning). Using the Join-Object I mentioned in the comment:
Import-Csv .\URL.txt -Header HEADER1 |
LeftJoin (Import-Csv .\URL1.txt -Header HEADER2) |
Export-Csv .\combinedcsvs.csv -Delimiter ';' -NoTypeInformation -Append
Note1: I am not sure why you trying to import anything like map.csv, I think that is required.
Note2: If you still want to go your own way, try to avoid using the increase assignment operator (+=) to create a collection it is a very expensive operator.
Note3: it is generally not a good idea to join lines on their line index as the list might not be sorted or have duplicates, therefore it is better to join lists on a specific property, like the the Url:
Import-Csv .\URL.txt -Delimiter '|' -Header Lid,Url,Type |
LeftJoin (Import-Csv .\URL1.txt -Delimiter '|' -Header Lid2,Url,Type2,Pid) -On Url |
Format-Table # or: Export-Csv .\combinedcsvs.csv -Delimiter ';' -NoTypeInformation
Lid Url Type Lid2 Type2 Pid
--- --- ---- ---- ----- ---
L5020 http://linktosite.de URL L5020 URL P555
L100 http://sitelink.de URL L100 URL P523
L50 http://abcde.de URL L50 URL P53
L511 http://bbcccddeee.de URL L511 URL P540
L300 http://link123456.de URL
L5450 http://randomlink.de URL_DE
L5460 http://randomwebsitelink.de URL_DE
Or on all three (Lid, Url and Type) properties:
Import-Csv .\URL.txt -Delimiter '|' -Header Lid,Url,Type |
LeftJoin (Import-Csv .\URL1.txt -Delimiter '|' -Header Lid,Url,Type,Pid) -On Lid,Url,Type |
Format-Table # or: Export-Csv .\combinedcsvs.csv -Delimiter ';' -NoTypeInformation
Lid Url Type Pid
--- --- ---- ---
L5020 http://linktosite.de URL P555
L100 http://sitelink.de URL P523
L50 http://abcde.de URL P53
L511 http://bbcccddeee.de URL P540
L300 http://link123456.de URL
L5450 http://randomlink.de URL_DE
L5460 http://randomwebsitelink.de URL_DE
If you only want to combine lines where both files contain data, you can do the following:
$f1 = Get-Content file1.txt
$f2 = Get-Content file2.txt
$output = for ($i = 0; $i -lt [math]::Min($f1.count,$f2.count); $i++) {
$f2[$i],$f1[$i] -join '|'
}
$output | Set-Content newfile.txt
If you want to combine all coinciding lines plus add extra lines from one of the files, you can do the following:
$output = for ($i = 0; $i -lt [math]::Max($f1.count,$f2.count); $i++) {
if ($f1[$i] -and $f2[$i]) {
$f2[$i],$f1[$i] -join '|'
}
else {
$f2[$i],$f1[$i] | Where {$_}
}
}
$output | Set-Content newfile.txt
I based my own solution on How to separate one row into multiple rows based on a single cell containing multiple values in the same row in a CSV file using PowerShell
Trying to iterate over one row, retrieve the zips, split it by ' ' and create a new PSCustomObject for each using the zipcodes and place the zipcode in the column with new rows in the same csv.
Sample (yes, hood = no value)
City,State,hood,zip,lat,lng
Carmel,IN,,"46290 46032 46033 46280 46082",39.9783711,-86.1180435
My non working solution is a modification of the solution in the link sited
$x = Get-Content 'D:\Carmel-IN.csv'
$y = $x | ConvertFrom-Csv -Delimiter ' '
$y | Foreach {
$current = $_
$current.zip -split ' ' | foreach {
[PSCUstomObject]#{
zip = $_
}
}
} | ConvertTo-Csv -NoTypeInformation -Delimiter ' ' | % {
$_ -replace '"',''
}
The output is the same on ISE and console ... the one word zip and nothing is written to file.
I am very new to PS, using 5.1. I have a feeling the correction is very simple but I cant see it. Looking for a result file as ...
City,State,hood,zip,lat,lng
Carmel,IN,,"46290 46032 46033 46280 46082",39.9783711,-86.1180435
,,,46290,,
,,,46032,,
,,,46033,,
,,,46280,,
,,,46082,,
UPDATE:
I managed to rework the script to get result I can use
$x = Get-Content 'D:\temp\Carmel-IN.csv'
$exportlocation = 'D:\temp\Carmel-IN.csv'
$y = $x | ConvertFrom-Csv -Delimiter ','
$y | Foreach {
$current = $_
$current.zip -split ' ' | foreach {
[PSCUstomObject]#{
City = $null
State = $null
hood = $null
zip = $_
lat = $null
lng = $null
}
}
} | Export-CSV -Append -NoTypeInformation -Delimiter ',' -Force $exportlocation | % { $_ -replace '"',''}
and the result is
City,State,hood,zip,lat,lng
Carmel,IN,,"46290 46032 46033 46280 46082",39.9783711,-86.1180435
,,,"46290",,
,,,"46032",,
,,,"46033",,
,,,"46280",,
,,,"46082",,
I am going to care of the extra "'s in another script
The | % { $_ -replace '"',''} doesn't seem to be removing them
Thanks for all suggestions
Use Select-Object:
$x = Get-Content 'D:\Carmel-IN.csv'
$y = $x | ConvertFrom-Csv -Delimiter ' '
foreach($row in $y){
foreach($zip in -split $row.zip){
$row |select *,#{Name='zip';Expression={$zip}} -ExcludeProperty zip
}
}
Alternatively, if you want to preserve the order of columns, set up the property list of properties to select prior to the loop:
$columns = $y[0].psobject.Properties.Name |ForEach-Object {
if($_ -eq 'zip'){
$_ = #{Name = 'zip';Expression = { -split $zip }}
}
$_
}
foreach($row in $y){
foreach($zip in -split $row.Zip){
$row |select $columns
}
}