I have a large csv file with 41000 rows of data. Within this I have to go through and inspect each row for a particular string PRCXX and if the row contains that in column 6 then I take the value of column 10 and replace the value in column 9 with it.
I have code that works however it takes a very long time to parse through line by line. I am looking for some help to try and optimize it. I have tried switching to a ForEach loop however I am not sure how exactly to get it to work with what I am trying to accomplish and haven't been able to find any examples to work from.
Here is the code that I have that currently is working just takes along time to complete.
Import-Csv $TransFile -Header 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19 | ForEach-Object {
if ($_.6 -match "PRCXX") {
$_.9 = $_.10
}
$_ | Export-Csv test2.csv -NoTypeInformation -Append -Delimiter ","
}
Thanks for any help that you can provide.
Thanks to JosefZ this is fixed. Moved the Export to occur outside the loop and that cleared it up.
Import-Csv $TransFile -Header 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19 | ForEach-Object {
if ($_.6 -match "PRCXX") {
$_.9 = $_.10
}
$_
} | Export-Csv test2.csv -NoTypeInformation -Append -Delimiter ","
Related
I have a csv file in which i need to add a new column. I do that with import-csv and then adding a property to the objects before writing it back to csv with export-csv.
Somehow its not working as expected. I took all real functionality out and reduced it to the following snipped:
$findReplaceList = Import-Csv -Path $mappingListPath -Delimiter ';'
Import-Csv $sourceFile.FullName |
Select-Object *, #{Name = 'column3'; Expression =
{ $newKey = $_."Issue key" ; $newKey }
} |
Export-Csv $outputFullFileName -NoTypeInformation -Delimiter ';'
please note that in this code the real calculation of the Expression is replaced with something very simple.
here, with
$newKey = $_."Issue key"
I just try to take the value in column "issue key" and use that for the newly added column.
Somehow the $newKey is always NULL - although $_."Issue key" is not?
What am i doing wrong?
The real code for $newKey will be:
{$newKey = $_.$FieldsToSelectFromTargetFilesIntoMappingFile[0]; (#($findReplaceList).where( { $_.$FieldsToSelectFromTargetFilesIntoMappingFile[0] -eq $newKey })).$FieldsToSelectFromTargetFilesIntoMappingFile[3] }
The mapping File (reference by $mappingListPath) is a simple csv like that:
image of a simple sample mapping table
Problem solved.
Thanks to all the comments and the working code sample from #Bender the greatest i was able to identify the issue with my code: The csv mapping file has ';' as delimiters... With the code i wrote the different "columns" in the csv where not identified as such, but each line was just written into one single property.
By changing
Import-Csv $sourceFile.FullName |
to
Import-Csv $sourceFile.FullName -delimiter ';' |
the problem was solved.
I have a csv file with two columns and multiple rows, which has the information of files with folder location and its corresponding size, like below
"Folder_Path","Size"
"C:\MSSQL\DATA\UsersData\FTP.txt","21345"
"C:\MSSQL\DATA\UsersData\Norman\abc.csv","78956"
"C:\MSSQL\DATA\UsersData\Market_Database\123.bak","1234456"
What i want do is remove the "C:\MSSQL\DATA\" part from every row in the csv and keep the rest of the folder path after starting from UsersData and all other data intact as this info is repetitive. So my csv should like this below.
"Folder_Path","Size"
"UsersData\FTP.txt","21345"
"UsersData\Norman\abc.csv","78956"
"UsersData\Market_Database\123.bak","1234456"
What i am running is as below
Import-Csv ".\abc.csv" |
Select-Object -Property #{n='Folder_Path';e={$_.'Folder_Path'.Split('C:\MSSQL\DATA\*')[0]}}, * |
Export-Csv '.\output.csv' -NTI
Any help is appreciated!
Seems like a job for a simple string replace:
Get-Content "abc.csv" | foreach { $_.replace("C:\MSSQL\DATA\", "") | Set-Content "output.csv"
or:
[System.IO.File]::WriteAllText("output.csv", [System.IO.File]::ReadAllText("abc.csv" ).Replace("C:\MSSQL\DATA\", ""))
This should work:
Import-Csv ".\abc.csv" |
Select-Object -Property #{n='Folder_Path';e={$_.'Folder_Path' -replace '^.*\\(.*\\.*)$', '$1'}}, Size |
Export-Csv '.\output.csv' -NoTypeInformation
In PowerShell I'm importing a CSV SAMTemp2 which will have a field called SO. Sometimes SO will be populated with "NW" and in these cases I just want to pull the field called ProdProj from the same line and replace the data in SO with the data in ProdProj then export it the data in that condition.
$RepNW = Import-Csv $SAMTemp2
foreach($d in $data){
If($d.SO -eq "NW"){($d.SO).Replace($d.ProdProj)}}
$RepNW | Export-Csv $SAMTemp -NoTypeInformation
I don't get an error, but this doesn't seem to do anything, either. Can anyone assist me, please?
Update
Per Matt below, I tried:
$RepNW = Import-Csv $SAMTemp2
foreach($d in $RepNW){
If($d.SO -eq "NW"){$d.SO = ($d.SO).Replace($d.ProdProj)}}
$RepNW | Export-Csv $SAMTemp -NoTypeInformation
But I'm not seeing any change. Any assistance is appreciated.
As LotPings pointed out in this line foreach($d in $data){, you haven't defined $data and it seems that you mean it to be foreach($d in $RepNW){
Secondly, rather than using Replace() you can just set one property to be equal to the other.
Last, this probably easiest to do all in the pipeline with ForEach-Object
Import-Csv $SAMTemp2 | ForEach-Object {
If($_.SO -eq "NW"){
$_.SO = $_.ProdProj
}
$_
} | Export-Csv $SAMTemp -NoTypeInformation
I have a csv file with five columns. The last column being the students grade level. What I want to do is change the value of that column based on the gradelevel. For example if the gradelevel is 12 I want to change that to 2016, 11 to 2017, so on and so forth.
update: I did get it to semi work using the below:
Get-Content users.csv | ForEach-Object -Process {$_ -replace '12','2016'} | Set-Content users1.csv
What happens is if the student id has a 12 in it that gets changed as well to 2016. Example would be 120045 gets change to 20160045
You can import the csv, loop it in a foreach and use the $_ (this) operator and then export ist to csv
Somesthing like:
# import CSV
$list = import-csv P:\ath\to\file.csv -Delimiter ";" -Encoding Standard
# % means foreach
$list | % {
# The grades is the key for each line
# the first line of your csv represent the keys you can use
# e.g. first colum: Name | Grade | Class
# $_.Name | $_.Grade | $_.Class are your keys for every entry (2nd line and above)
# here you can work with your grades an do what you want
if($_.Grade -eq 11){
# set a new value to the grade in the actual line
$_.Grade = 2016
}
}
# now export the new list into csv
export-csv P:\ath\to\new.csv -Delimiter ";" -NoTypeInformation
That should be a basic to work with.
Greetz Eldo.O
Eldo you code worked great. I did have to change the last line from:
export-csv P:\ath\to\new.csv -Delimiter ";" -NoTypeInformation
to
$list | Export-Csv P:\ath\to\new.csv -Delimiter "," -NoTypeInformation
I was also able to add more if statements to accomplish exactly what I needed.
I have a csv file with a few columns. For example
col1;col2;col3;col4
text1;text2;text3;text4
text5;text6;text7;text8
text9;text10;text11;text12
I want to add text to column col3, like this:
col1;col2;col3;col4
text1;text2;append\text3;text4
text5;text6;append\text7;text8
text9;text10;append\text11;text12
So question is:
How to do this?
(I'm stuck whit that how I add data to each column in col3.)
Give this a whirl:
Import-Csv -Delim ';' cols.csv |
ForEach-Object {$_.col3 = "prepend\$($_.col3)";$_} |
Export-Csv cols2.csv -Delim ';' -NoTypeInformation
Use the -NoTypeInformation parameter to avoid this comment getting put at the top of your CSV:
#TYPE System.Management.Automation.PSCustomObject
However, if you don't mind the comment then you can leave off the -NoTypeInformation parameter.
You probably know how to read/write data. To change the data, send them to Foreach-Object, alter the data and pass the object further to Export-Csv.
Import-Csv d:\temp\so\csv1.txt -Delimiter ';' |
ForEach-Object { $_.col3 = 'append\' +$_.col3; $_ } |
Export-Csv d:\temp\so\csv2.txt -Delimiter ';'