I have been battling this for way too long and I hope you can be of assistance.
I have a CSV for which I need to add some values, preferrably with Powershell. I need to add header row and one column with a fixed text value.
My CSV, before anything done to it, looks like this:
contact-email;contact-sms;contact-firstname
test#example.com;+3580000000;Mike
And I need it to look like this:
contact-email;contact-sms;contact-firstname;order-site
test#example.com;+3580000000;Mike;Finland
So the last column "order-site" needs to be added and every line in that column should have a value of "Finland".
So far I have written this Powershell script I got off a tutorial:
$file = Import-Csv E:\Raportit\SALES\SALES_TD_S01.csv -Delimiter "`t" -Encoding Default -Header "contact-email;contact-sms;contact-firstname"
foreach($c in $file) {
Add-Member -Input $c -MemberType NoteProperty -Name "order-site" -Value "Finland"
}
$file | Export-Csv -Path "C:\Temp\test.csv" -Encoding Default -NoTypeInformation
But unfortunately, this makes the CSV look like this:
"contact-email;contact-sms;contact-firstname","order-site"
"test#example.com;+3580000000;Mike","Finland"
For the use case I have for this file, it need to look like the first should-look-like example, without double quotes and columns separated by semicolon (;). The double quotes are OK as long as the output looks like this:
"contact-email;contact-sms;contact-firstname;order-site"
"test#example.com;+3580000000;Mike;Finland"
I thank you guys so much in advance, I know this is probably a super simple task but I just cannot wrap my head around it to save my life.
if the file HAS headers:
Import-Csv -Path 'E:\Raportit\SALES\SALES_TD_S01.csv' -Delimiter ';' |
Select-Object *, #{Name = 'order-site'; Expression = {'Finland'}} |
Export-Csv -Path "C:\Temp\test.csv" -Delimiter ';' -NoTypeInformation
if the file DOES NOT HAVE headers:
$headers = 'contact-email','contact-sms','contact-firstname'
Import-Csv -Path 'E:\Raportit\SALES\SALES_TD_S01.csv' -Delimiter ';' -Header $headers |
Select-Object *, #{Name = 'order-site'; Expression = {'Finland'}} |
Export-Csv -Path "C:\Temp\test.csv" -Delimiter ';' -NoTypeInformation
CSV Output:
"contact-email";"contact-sms";"contact-firstname";"order-site"
"test#example.com";"+3580000000";"Mike";"Finland"
Related
I am trying to Remove unnecessary commas in a column in the CSV file. For now, I know a few issues and hard-coded it, But I wanted the code to be dynamic. Any suggestions are greatly appreciated.
$FilePath = "C:\Test\"
Get-ChildItem $FilePath -Filter .csv | ForEach-Object {
(Get-Content $_.FullName -Raw) | Foreach-Object {
$_ -replace ',"Frederick, Fred",' , ',"Frederick Fred",' `
-replace ',"Brian, Josiah",' , ',"Brian Josiah",' `
-replace ',"Lisinopril ,Tablet / 20MG",' , ',"Lisinopril Tablet / 20MG",'
} | Set-Content $_.FullName
}
Try this, also note that I worked with the csv sample that you gave here.It might not work with other csv files.
also make sure that you change the path of %YOURCSVFILE% to the real path of your file
#import the csv
$csv = Import-Csv -Path %YOURCSVFILE% -Delimiter ','
#going each row and replacing commas
foreach ($desc in $csv){
$desc.Desc = $desc.Desc -replace ',',''
}
#exporting the csv
$csv | Export-csv -NoTypeInformation "noCommas.csv"
Here's a few more alteratives for you:
Method 1. Loop through the rows with foreach(..) and capture the output:
$result = foreach ($row in (Import-Csv -Path 'D:\Test\FileWithCommasInDescription.csv')) {
$row.Desc = $row.Desc -replace ','
$row # output the updated item
}
$result | Export-Csv -Path 'D:\Test\FileWithoutCommasInDescription.csv' -NoTypeInformation
Method 2. Use ForEach-Object and the automatic variable $_. Pipe the results through:
Import-Csv -Path 'D:\Test\FileWithCommasInDescription.csv' | ForEach-Object {
$_.Desc = $_.Desc -replace ','
$_ # output the updated item
} | Export-Csv -Path 'D:\Test\FileWithoutCommasInDescription.csv' -NoTypeInformation
Method 3. Use a calculated property:
Import-Csv -Path 'D:\Test\FileWithCommasInDescription.csv' |
Select-Object ID, #{Name = 'Desc'; Expression = {$_.Desc -replace ','}}, Nbr -ExcludeProperty Desc |
Export-Csv -Path 'D:\Test\FileWithoutCommasInDescription.csv' -NoTypeInformation
All will result in a new CSV file
"ID","Desc","Nbr"
"12","Frederick Fred","11"
"21","Brian Josiah","31"
"13","Lisinopril Tablet / 20MG","17"
Below is one of the file data I have in text file
B97SW | CHANGED | rc=0 >>
Server Name";"SystemFolderPath";"IdenityReference";"FileSystemRights";"Vulnerable
B97SW;C:\Windows\system32;CREATOR OWNER;268435456;No
B97SW;C:\Windows\system32;NT AUTHORITY\SYSTEM;268435456;No
B97SW;C:\Windows\system32;NT AUTHORITY\SYSTEM;Modify, Synchronize;No
........
I am trying to replace ";" with "," and write to csv.
Below is the code I wrote but it is not writing the data in csv.
$FileList = Get-ChildItem -Path "C:\Files"
$props=[ordered]#{
ServerName=''
SystemFolderPath=''
IdenityReference=''
FileSystemRights=''
Vulnerable=''
}
New-Object PsObject -Property $props |
Export-Csv C:\2021.csv -NoTypeInformation
$FinalData = #()
foreach($n_file in $FileList)
{
$FileName = $n_file.FullName
$FileContent = Get-Content -Path $FileName | Select-Object -Skip 2
foreach($line in $FileContent)
{
$line = $line -replace(";",",")
$line | Export-Csv -Path C:\2021.csv -Append -NoTypeInformation -Force
}
}
output I am getting
"ServerName","SystemFolderPath","IdenityReference","FileSystemRights","Vulnerable"
"","","","",""
,,,,
,,,,
Please let me know what is wrong I am doing here.
$line | Export-Csv -Path C:\2021.csv -Append -NoTypeInformation -Force
This doesn't work because Export-Csv expects object(s) with properties, but $line is just a string. You need to parse it into an object first, using ConvertFrom-Csv.
Try this:
$FileList = Get-ChildItem -Path "C:\Files"
foreach($n_file in $FileList)
{
$FileName = $n_file.FullName
Get-Content -Path $FileName |
Select-Object -Skip 2 |
ConvertFrom-Csv -Delimiter ';' -Header ServerName, SystemFolderPath, IdenityReference, FileSystemRights, Vulnerable |
Export-Csv -Path C:\2021.csv -Append -NoTypeInformation -Force
}
As we have skipped the original headers, we have to supply these through the -Header parameter of ConvertFrom-Csv.
Your CSV file is goofed up in two ways. First, there is a line of garbage before the header line. Second, in the header line the semi-colons are surrounded by double quotes. The correct form would be to surround the header names with quotes instead.
Once these format errors are fixed, you can read the csv file with this:
Import-Csv myfile.csv -delimiter ";"
Or if you want to produce a comma delimited csv file, try this:
Import-Csv myfile.csv -delimiter ";" | Export-Csv newfile.csv
The result will be correct but it will have a lot of unnecessary double quotes.
I am trying to convert two TXT files into one CSV file using powershell script. When files have same structure, and same number of rows then case looks be easy. But in my case txt files have diffrent structure.
Pipe sign in both txt files is not a delimiter should be treat as normal character and it is a string.
File URL.txt
L5020|http://linktosite.de|URL
L100|http://sitelink.de|URL
L50|http://abcde.de|URL
L511|http://bbcccddeee.de|URL
L300|http://link123456.de|URL
L5450|http://randomlink.de|URL_DE
L5460|http://randomwebsitelink.de|URL_DE
File URL1.txt
L5020|http://linktosite.de|URL|P555
L100|http://sitelink.de|URL|P523
L50|http://abcde.de|URL|P53
L511|http://bbcccddeee.de|URL|P540
CSV which I expect should look like as below and delimiter is ";"
HEADER1;HEADER2
L5020|http://linktosite.de|URL;L5020|http://linktosite.de|URL|P555
L100|http://sitelink.de|URL;L100|http://sitelink.de|URL|P523
L50|http://abcde.de|URL;L50|http://abcde.de|URL|P53
L511|http://bbcccddeee.de|URL;L511|http://bbcccddeee.de|URL|P540
L300|http://link123456.de|URL;
L5450|http://randomlink.de|URL_DE;
L5460|http://randomwebsitelink.de|URL_DE;
I tried something like that
$URL = "C:\Users\XXX\Desktop\URL.txt"
$URLcontent = Get-Content $URL
$URL1 = "C:\Users\XXX\Desktop\URL1.txt"
$URLcontent1 = Get-Content $URL1
$results = #() # Empty array to store new created rows in
$csv = Import-CSV "C:\Users\XXX\Desktop\map.csv" -Delimiter ';'
foreach ($row in $csv) {
$properties = [ordered]#{
HEADER1 = $URLcontent
HEADER2 = $URLcontent1
}
# insert the new row as an object into the results-array
$results += New-Object psobject -Property $properties
}
# foreach-loop filled the results-array - export it as a CSV-file
$results | Export-Csv "C:\Users\XXXX\Desktop\map_final.csv" -NoTypeInformation
And something like that:
import-csv URL.txt -Header 'HEADER1' | Export-CSV "C:\Users\xxx\Desktop\URL.csv" -Delimiter ';' -NoTypeInformation
import-csv URL1.txt -Header 'HEADER2' | Export-CSV "C:\Users\xxx\Desktop\URL1.csv" -Delimiter ';' -NoTypeInformation
Get-ChildItem "C:\Users\xx\Desktop" -Filter "URL*.csv" | Select-Object -ExpandProperty FullName | Import-Csv | Export-Csv .\combinedcsvs.csv -NoTypeInformation -Append
Without any succes...
BR
Based on the updates in your question, if you want to build something yourself, you probably want to do something like this:
$Url1 = #(Get-Content .\URL1.txt)
$i = 0
Get-Content .\URL.txt | Foreach-Object {
[pscustomobject]#{
HEADER1 = $_
HEADER2 = If ($i -lt $URL1.Count) { $URL1[$i++] }
}
} | Export-Csv .\combinedcsvs.csv -Delimiter ';' -NoTypeInformation -Append
In case you do not want to go through the hassle of reinventing the wheel (with all pitfalls including performance tuning). Using the Join-Object I mentioned in the comment:
Import-Csv .\URL.txt -Header HEADER1 |
LeftJoin (Import-Csv .\URL1.txt -Header HEADER2) |
Export-Csv .\combinedcsvs.csv -Delimiter ';' -NoTypeInformation -Append
Note1: I am not sure why you trying to import anything like map.csv, I think that is required.
Note2: If you still want to go your own way, try to avoid using the increase assignment operator (+=) to create a collection it is a very expensive operator.
Note3: it is generally not a good idea to join lines on their line index as the list might not be sorted or have duplicates, therefore it is better to join lists on a specific property, like the the Url:
Import-Csv .\URL.txt -Delimiter '|' -Header Lid,Url,Type |
LeftJoin (Import-Csv .\URL1.txt -Delimiter '|' -Header Lid2,Url,Type2,Pid) -On Url |
Format-Table # or: Export-Csv .\combinedcsvs.csv -Delimiter ';' -NoTypeInformation
Lid Url Type Lid2 Type2 Pid
--- --- ---- ---- ----- ---
L5020 http://linktosite.de URL L5020 URL P555
L100 http://sitelink.de URL L100 URL P523
L50 http://abcde.de URL L50 URL P53
L511 http://bbcccddeee.de URL L511 URL P540
L300 http://link123456.de URL
L5450 http://randomlink.de URL_DE
L5460 http://randomwebsitelink.de URL_DE
Or on all three (Lid, Url and Type) properties:
Import-Csv .\URL.txt -Delimiter '|' -Header Lid,Url,Type |
LeftJoin (Import-Csv .\URL1.txt -Delimiter '|' -Header Lid,Url,Type,Pid) -On Lid,Url,Type |
Format-Table # or: Export-Csv .\combinedcsvs.csv -Delimiter ';' -NoTypeInformation
Lid Url Type Pid
--- --- ---- ---
L5020 http://linktosite.de URL P555
L100 http://sitelink.de URL P523
L50 http://abcde.de URL P53
L511 http://bbcccddeee.de URL P540
L300 http://link123456.de URL
L5450 http://randomlink.de URL_DE
L5460 http://randomwebsitelink.de URL_DE
If you only want to combine lines where both files contain data, you can do the following:
$f1 = Get-Content file1.txt
$f2 = Get-Content file2.txt
$output = for ($i = 0; $i -lt [math]::Min($f1.count,$f2.count); $i++) {
$f2[$i],$f1[$i] -join '|'
}
$output | Set-Content newfile.txt
If you want to combine all coinciding lines plus add extra lines from one of the files, you can do the following:
$output = for ($i = 0; $i -lt [math]::Max($f1.count,$f2.count); $i++) {
if ($f1[$i] -and $f2[$i]) {
$f2[$i],$f1[$i] -join '|'
}
else {
$f2[$i],$f1[$i] | Where {$_}
}
}
$output | Set-Content newfile.txt
I'm sure this is ridiculously easy, but I'm a noob and trying to learn PowerShell.
I want to write an integer to each line of a tab delimited file, i.e. each line has 20 tabs; put a 1 after the nth tab.
No need to overwrite what's already there because in the current scenario there isn't anything.
Thanks!
If there is a header line then just import the file as a CSV, run it through a ForEach-Object loop and set that column to the integer that you want, then export the CSV again.
Import-CSV $File -Delimiter "`t" | ForEach{$_.ColumnName = $Integer} | Export-CSV $File -Delimiter "`t" -NoTypeInfo
If there is no header you could do the same thing and define your own headers. Except you would use ConvertTo-CSV instead of Export-CSV and then use Select to skip the header row, and use Set-Content to write the file. For my example I set the 7th column to $Integer.
$Headers = 1..20|ForEach{"Col$_"}
Import-CSV $File -Delimiter "`t" -Header $Headers | ForEach{$_.Col7 = $Integer} | ConvertTo-CSV -Delimiter "`t" -NoTypeInfo | Select -Skip 1 | Set-Content $File
So I am looking at breaking up a CSV using Powershell. The CSV is delmited by | which isn't a problem, and I am looking to break it up into multiple smaller csvs while retaining the original. The breaks would occur based off of the value in a single column containing one of a list of values.
What I have done so far is to import the csv (delimited by |) and then
foreach($line in $csv) {
if($columnValue -like $target1) {
export-csv filename1.csv -Delimiter `| $line -append)}
elseif($columnValue -like $target2) {
export-csv filename2.csv -Delimiter `| $line -append)}
etc.
However I do not think it is exporting correctly, and I do not want there to be the quotes (and yes I know this is standard but I do not want them) Also I want the header from the original csv to be applied to the child csvs and its not being applied.
sorry if theres a better way to format the code still new here
Here is where I suggest the awesomeness of the Switch cmdlet. It compares something against multiple potential matches, and executes those matches where appropriate.
Switch($csv){
{$_.column -match $target1} {$_ | Export-CSV filename1.csv -append -delimiter '|'}
{$_.column -match $target2} {$_ | Export-CSV filename2.csv -append -delimiter '|'}
{$_.column -match $target3} {$_ | Export-CSV filename3.csv -append -delimiter '|'}
}
$data = import-csv $csvfile
$data | ?{$_.val -eq $criteria1} | export-csv -path "File1.csv"
$data | ?{$_.val -eq $criteria2} | export-csv -path "File2.csv"