Powershell script to Find and replace Particular value in CSV file - powershell

I have a csv file in path and i need to fetch the file and do Find & Replace $(dollar) value to ,(comma) using PowerShell script. please help me on the script .
CSV file
id|name|place
1|adam$|USA
2|john|USA
3|Jack$|England
Expected output : Need to do Find & Replace $(dollar) value to ,(comma) under name column
id|name|place
1|adam,|USA
2|john|USA
3|Jack,|England
Command Used
Import-csv D:\CSV\Customer.csv
ForEach-Object { $_.name -replace '$',',' }
i didn't got any error nor the expected changes. i need to update the values in existing CSV file
Below Answer by Zett42 works but
(Import-Csv D:\CSV\Customer.csv -Delimiter '|') |
Select-Object 'id',
#{ n = 'name'; e = { $_.name.Replace('$', ',') } },
'place' |
Export-Csv D:\CSV\Customer.csv -Delimiter '|'
but the tricky part is we are replacing the ,(Comma) value and rewriting the CSV files again, so CSV file creation part considers the value after ,(comma) as new column and move the value to column B instead of Column A.
Before :
After :

This should do the trick:
Import-Csv D:\CSV\Customer.csv -Delimiter '|' |
ForEach-Object {
$_.name = $_.name.Replace('$', ',')
$_ # Implicit output - gets passed to Export-Csv
} |
Export-Csv D:\CSV\Customer_new.csv -Delimiter '|' -NoTypeInformation
In your code, there is a pipe symbol | missing at the end of the Import-Csv line.
As the CSV uses non-standard delimiters, you have to specify them using -Delimiter.
Also, the -replace operator doesn't modify its arguments, so you are actually just outputting the replaced value. Also, '$' has special meaning in regular expression patterns, which -replace uses, so you have to backslash-escape it or use the String.Replace() method as I did in the code above. The String.Replace() method doesn't use RegEx.
In the ForEach-Object script block, the $_ at the end is necessary to output the current object again, so it gets send down the pipeline to Export-Csv. Otherwise Export-Csv wouldn't have anything to output.
Alternative code using Select-Object:
Import-Csv D:\CSV\Customer.csv -Delimiter '|' |
Select-Object 'id',
#{ n = 'name'; e = { $_.name.Replace('$', ',') } },
'place' |
Export-Csv D:\CSV\Customer_new.csv -Delimiter '|' -NoTypeInformation
To overwrite the existing file instead of creating a new file use the group operator () to collect all data in memory and close the old file before overwriting the file:
(Import-Csv D:\CSV\Customer.csv -Delimiter '|') |
Select-Object 'id',
#{ n = 'name'; e = { $_.name.Replace('$', ',') } },
'place' |
Export-Csv D:\CSV\Customer.csv -Delimiter '|' -NoTypeInformation
IMO this is somewhat cleaner as you don't need to modify the current pipeline object and don't need to explicitly forward it to the next command.
It uses a calculated property to generate the new value for the name column. In this case we don't need to assign the result back to $_.name because calculated properties automatically assign the output from the script block to the property.

Related

Trouble importing CSV and then exporting with some manipulation

I have the following file:
2018|CIB_C21_A_1_1_FromRangeAmount|.0000
As you can see, it has no headers. The first column can be ignored. I need to take the 2nd column and then export but I also need to add (2) additional columns in the export.
The export should look like this:
Account,Parent,Data Storage
CIB_C21_A_1_1|CIB_C21_A|Never Share
As you can see, I need to take the first column and also use the first 9 chracters as a value for another column. Then I need to add another column with the same value at all times.
I have been trying to play around with this some of this logic but can't quote any anything to work...
Import-Csv -Path "C:\TEMPCT\Revenue_RebatesDataForPlanning.csv" -Header zz,Account -Delim "|" |
sort "Account" –Unique |
Select "Account", "Parent" |
Export-Csv -Path "C:\TEMPCT\Revenue_RebatesDataForPlanningzz.csv" -Force -NoTypeInformation
Could anyone recommend some suggestions?
You can use calculated properties with Select-Object to create the desired new columns. To get the 9 first characters of the value from "Account" column you can use the String.Substring Method.
# This would be:
# Import-Csv path/to/csv.csv -Header Ignore, Account -Delimiter '|' | ...
#'
2018|CIB_C21_A_1_1_FromRangeAmount|.0000
2018|CIB_C22_A_1_1_FromRangeAmount|.0000
2018|CIB_C23_A_1_1_FromRangeAmount|.0000
2018|CIB_C24_A_1_1_FromRangeAmount|.0000
2018|CIB_C25_A_1_1_FromRangeAmount|.0000
'# | ConvertFrom-Csv -Header Ignore, Account -Delimiter '|' |
Select-Object Account, #{
Name = 'Parent'
Expression = { $_.Account.Substring(0,9) }
}, #{
Name = 'Data Storage'
Expression = { 'Never Share' }
} | ConvertTo-Csv -NoTypeInformation
How your code should look:
Import-Csv path/to/csv.csv -Header Ignore, Account -Delimiter '|' |
Select-Object Account, #{
Name = 'Parent'
Expression = { $_.Account.Substring(0,9) }
}, #{
Name = 'Data Storage'
Expression = { 'Never Share' }
} | Export-Csv path/to/newCsv.csv -NoTypeInformation

Import 1 column of csv, replace string, then join each row into one row with values surrounded in quotes

Im using powershell and I have a csv file that has a ton of data but the only field I am concerned with is the first one.
I need to:
Extract the first column of data with the header: Ticket_ID
Remove a trailing string from the values in this column (site:)
Join all of the values in the column into a single, space delimited row, but ensure each value retains the double quotes wrapping each of them.
Prepend this row with a long string and write it to a file. (I havent tried to tackle this part yet because I cant get 1-3 to work fully.)
I have tried several permutations of this code:
$TT = (Import-Csv '.\sourcefile.csv').______Ticket_ID______ -join (" ")
$TT = $TT -replace 'site:',''
$TT | Set-Content -Path .\JulyTickets.csv
Example of my csv:
"______Ticket_ID______","____________Title____________","___________CreateDate___________"
"site:TICKET1","SOMETITLE","01/01/1901"
"site:TICKET2","SOMETITLE","01/01/1901"
"site:TICKET3","SOMETITLE","01/01/1901"
"site:TICKET4","SOMETITLE","01/01/1901"
"site:TICKET5","SOMETITLE","01/01/1901"
My frustrations:
If I use Export-CSV the resulting csv only contains the character length of the values.
If I use Set-Content, I can successfully extract the first column, and remove the leading string (site:), join all of the rows into a single row, and write this out to a file but it does not retain the double quotes around each of the values in the csv ("site:TICKET1"). And the output looks like this:
TICKET1 TICKET2 TICKET3 TICKET4 TICKET5
I'm not sure if I'm doing things out of order here, or if I'm missing something but I'm unable to get both the values inline, and individually surrounded in quotes.
Required Output:
this is a long string that needs to be prepended to the data "TICKET1" "TICKET2" "TICKET3" "TICKET4" "TICKET5"
This should do it. The need for pipes and PowerShell's Calculated Properties expression in the end is to overcome powershell removing double quotes
$string = 'this is a long string that needs to be prepended to the data ' + (Get-Content -Path \.sourcefile.csv | Convertfrom-Csv -Delimiter "," | select ______Ticket_ID______ -ExpandProperty ______Ticket_ID______ | ForEach-Object {$_ -replace 'site:', '';} | select #{ Name = 'Ticket'; Expression = {$([char]34) + $_ + $([char]34)}} | Select Ticket -ExpandProperty Ticket);
write-host $string
returns
this is a long string that needs to be prepended to the data "TICKET1" "TICKET2" "TICKET3" "TICKET4" "TICKET5"
Try this out, I'm using the -Header argument since the only needed column is the first one.
$csv = #'
"______Ticket_ID______","____________Title____________","___________CreateDate___________"
"site:TICKET1","SOMETITLE","01/01/1901"
"site:TICKET2","SOMETITLE","01/01/1901"
"site:TICKET3","SOMETITLE","01/01/1901"
"site:TICKET4","SOMETITLE","01/01/1901"
"site:TICKET5","SOMETITLE","01/01/1901"
'# |
ConvertFrom-Csv -Header ID |
Select-Object -Skip 1 |
ForEach-Object {
'"{0}"' -f $_.ID.TrimStart('site:')
}
$toPrepend = 'this is a long string that needs to be prepended to the data'
"$toPrepend $($csv -join ' ')"
# Above yields => this is a long string that needs to be prepended to the data "TICKET1" "TICKET2" "TICKET3" "TICKET4" "TICKET5"
Note, I'm only using ConvertFrom-Csv as an example here but in your case, it should look like this:
$TT = Import-Csv '.\sourcefile.csv' -Header ID | Select-Object -Skip 1 | ForEach-Object {
'"{0}"' -f $_.ID.TrimStart('site:')
}
$toPrepend = 'this is a long string that needs to be prepended to the data'
"$toPrepend $($TT -join ' ')" | Out-File JulyTickets.csv # => Not sure why CSV here, since this is no longer a CSV

I use -NoTypeInformation so why do I get header back when using Out-File?

I filtered by date this file data1.csv
2017.11.1,09:55,1.1,1.2,1.3,1.4,1
2017.11.2,09:55,1.5,1.6,1.7,1.8,2
I don't get a header with -NoTypeInformation:
$CutOff = (Get-Date).AddDays(-2)
$filePath = "data1.csv"
$Data = Import-Csv $filePath -Header Date,Time,A,B,C,D,E
$Data2 = $Data | Where-Object {$_.Date -as [datetime] -gt $Cutoff} | convertto-csv -NoTypeInformation -Delimiter "," | % {$_ -replace '"',''}
But when rewriting with Out-File
$Data2 | Out-File "data2.csv" -Encoding utf8 -Force
I get header back as data2.csv contains:
Date,Time,A,B,C,D,E
2017.11.2,09:55,1.5,1.6,1.7,1.8,2
Why do I have Date,Time,A,B,C,D,E ?
-NoTypeInformation is not about the header but the data type of the rows in the file. Remove it to see what shows up. From Microsoft
Omits the type information header from the output. By default, the string in the output contains #TYPE followed by the fully-qualified name of the object type.
Emphasis mine.
CSVs need headers. That is why it is making one. If you don't want to see the header in the output use Select-Object -Skip 1 to remove it.
$Data |
Where-Object {$_.Date -as [datetime] -gt $Cutoff} |
ConvertTo-CSV -NoTypeInformation -Delimiter "," |
Select-Object -Skip 1 |
% {$_ -replace '"'}
I would not pipe Out-File to itself. You could pipe to Set-Content here just as well.
I am guessing this whole process is to keep the source file in the same state just with some lines filtered out based on date. You could skip most of this just by parsing the date out in each line.
$threshold = (Get-Date).AddDays(-2)
$filePath = "c:\temp\bagel.txt"
(Get-Content $filePath) | Where-Object{
$date,$null=$_.Split(",",2)
[datetime]$date -gt $threshold
} | Set-Content $filePath
Now you don't have to worry about PowerShell CSV object structure or output since we act on the raw data of the file itself.
That will take each line of the input file and filter it out if the parsed date does not match the threshold. Change encoding on the input output cmdlets as you see necessary. What $date,$null=$_.Split(",",2) is doing is splitting the line
on the comma into 2 parts. First of which becomes $date and since this is just a filtering condition we dump the rest of the line into $null.
Properly-formed CSV files must have column headers. Your use of -NoTypeInformation in generating the CSV does not affect column headers; instead, it affects whether the PowerShell object type information is included. If you Export-CSV without -NoTypeInformation, the first line of your CSV file will have a line that looks like #TYPE System.PSCustomObject, which you don't want if you're going to open the CSV in a spreadsheet program.
If you subsequently Import-CSV, the headers (Date, Time, A, B, C) are used to create the fields of a PSObject, so that you can refer to them using the standard dot notation (e.g., $CSV[$line].Date).
The ability to specify -Header on Import-CSV is essentially a "hack" to allow the cmdlet to handle files that are comma-separated, but which did not include column headers.

Powershell how to add data in the csv file to one column

I have a csv file with a few columns. For example
col1;col2;col3;col4
text1;text2;text3;text4
text5;text6;text7;text8
text9;text10;text11;text12
I want to add text to column col3, like this:
col1;col2;col3;col4
text1;text2;append\text3;text4
text5;text6;append\text7;text8
text9;text10;append\text11;text12
So question is:
How to do this?
(I'm stuck whit that how I add data to each column in col3.)
Give this a whirl:
Import-Csv -Delim ';' cols.csv |
ForEach-Object {$_.col3 = "prepend\$($_.col3)";$_} |
Export-Csv cols2.csv -Delim ';' -NoTypeInformation
Use the -NoTypeInformation parameter to avoid this comment getting put at the top of your CSV:
#TYPE System.Management.Automation.PSCustomObject
However, if you don't mind the comment then you can leave off the -NoTypeInformation parameter.
You probably know how to read/write data. To change the data, send them to Foreach-Object, alter the data and pass the object further to Export-Csv.
Import-Csv d:\temp\so\csv1.txt -Delimiter ';' |
ForEach-Object { $_.col3 = 'append\' +$_.col3; $_ } |
Export-Csv d:\temp\so\csv2.txt -Delimiter ';'

Powershell import-csv with empty headers

I'm using PowerShell To import a TAB separated file with headers. The generated file has a few empty strings "" on the end of first line of headers. PowerShell fails with an error:
"Cannot process argument because the
value of argument "name" is invalid.
Change the value of the "name"
argument and run the operation again"
because the header's require a name.
I'm wondering if anyone has any ideas on how to manipulate the file to either remove the double quotes or enumerate them with a "1" "2" "3" ... "10" etc.
Ideally I would like to not modify my original file. I was thinking something like this
$fileContents = Get-Content -Path = $tsvFileName
$firstLine = $fileContents[0].ToString().Replace('`t""',"")
$fileContents[0] = $firstLine
Import-Csv $fileContents -Delimiter "`t"
But Import-Csv is expecting $fileContents to be a path. Can I get it to use Content as a source?
You can either provide your own headers and ignore the first line of the csv, or you can use convertfrom-csv on the end like Keith says.
ps> import-csv -Header a,b,c,d,e foo.csv
Now the invalid headers in the file is just a row that you can skip.
-Oisin
If you want to work with strings instead use ConvertFrom-Csv e.g.:
PS> 'FName,LName','John,Doe','Jane,Doe' | ConvertFrom-Csv | Format-Table -Auto
FName LName
----- -----
John Doe
Jane Doe
I ended up needing to handle multiple instances of this issue. Rather than use the -Header and manually setting up each import instance I wrote a more generic method to apply to all of them. I cull out all of the `t"" instances of the first line and save the file to open as a $filename + _bak and import that one.
$fileContents = Get-Content -Path $tsvFileName
if( ([string]$fileContents[0]).ToString().Contains('""') )
{
[string]$fixedFirstLine = $fileContents[0].ToString().Replace('`t""',"")
$fileContents[0] = $fixedFirstLine
$tsvFileName = [string]::Format("{0}_bak",$tsvFileName
$fileContents | Out-File -FilePath $tsvFileName
}
Import-Csv $tsvFileName -Delimiter "`t"
My Solution if you have much columns :
$index=0
$ColumnsName=(Get-Content "C:\temp\yourCSCFile.csv" | select -First 1) -split ";" | %{
$index++
"Header_{0:d5}" -f $index
}
import-csv "C:\temp\yourCSCFile.csvv" -Header $ColumnsName