Use a specific Line of a csv File - powershell

I hope that someone can give me a little help.
I want to search a csv file for an entry in column F. The entries in this line should be able to then be processed further.
How can this be implemented?
Thanks in advance
Daniel


The main cmdlet for importing CSV data is Import-Csv (how surprising). From there you can filter accordingly:
Import-Csv foo.csv | Where-Object { $_.F -eq 'bar' }
or select just the data from a single column:
Import-Csv foo.csv | Select-Object -Expand F
or do both (with aliases now):
Import-Csv foo.csv | ? { $_.F -eq 'bar' } | select -exp F
Further processing can then be done with ForEach-Object or % (its alias):
... | % {
<# do stuff with $_ here #>
}


Just prototype, you need to replace <anchors> with what you need:
Import-Csv YourFile.csv |
where { $_.F <operator> <value> } |
foreach {
<process $_ with column names attached as it's properties>
}

Related

Compare 2 .csv files

I have two .csv files with many information in it. If at the end of the sentence is a "M", I have to look if this row is in the other file. When it's there I have to look if the code at the beggining of the row is the same, when not then I have to do nothing, but when it's the same I have to make a new file.
This is the information I have to look if it's in the other file:
You can see that the information is here:
I also have rows with a "B" at the end but this is unimportant:
Now, when the information is here, I have to export all rows that are same in both files.
I have to export the rows in a new file which have the same code at the beginning which is circeld in red:
I have tried different solutions that I looked up in the Internet, but nothing really works.
Perhaps something like this?
$datenbank = Import-Csv "C:\Users\information1.csv"
$zentral = Import-Csv "C:\Users\information2.csv"
$new = ""
foreach ($line in $datenbank) {
$Spalte = $line.Split(",")
foreach ($z in $Zentral) {
$found = $false
foreach ($d in $Datenbanktyp) {
if ($d.$Spalte[1] -eq $z.$Spalte[1]) {
$found = $true
}
}
if ($found -eq $true) {
$new += $z
}
}
}
Or can it work with a if..elseif..else loop?

Let's see if I got this right. You have one file where the second-last column contains a letter. If that letter is "M" you want to check if the value of the column before that (partially) matches a column from a second file. If it does, you then want to export all rows from the second file that have the same value in the first column as the matched row to a new file.
Since you didn't reveal the column names I'm going to dub the third- and second-last columns from the first file "Erin" and "Marty", the match column from the second file "Pat", and the first column from the second file "Gene".
$datenbank | Where-Object {
$_.Marty -ceq 'M'
} | Select-Object -Expand Erin -Unique | ForEach-Object {
$outfile = "export_${_}.csv" # adjust output filename as you see fit
$firstcol = $zentral |
Where { $_.Pat -like "*${_}*" } |
Select-Object -Expand Gene
$zentral | Where-Object {
$_.Gene -eq $firstcol
} | Export-Csv $outfile
}
Another approach would be to group your second file by the first column and then check if the groups contain a matching value.
$groups = $zentral | Group-Object Gene
$datenbank | Where-Object {
$_.Marty -ceq 'M'
} | Select-Object -Expand Erin -Unique | ForEach-Object {
$outfile = "export_${_}.csv" # adjust output filename as you see fit
$groups | Where-Object {
$_.Group.Pat -like "*${_}*"
} | Select-Object -Expand Group | Export-Csv $outfile
}
Replace "Erin", "Marty", "Pat" and "Gene" with the actual column titles from your CSV files. Should your files not contain column titles you need to specify them via the -Header parameter of Import-Csv, otherwise the cmdlet will interpret the first data row as the headers.

Parse line of text and match with parse of CSV

As a continuation of a script I'm running, working on the following.
I have a CSV file that has formatted information, example as follows:
File named Import.csv:
Name,email,x,y,z
\I\RS\T\Name1\c\x,email#jksjks,d,f
\I\RS\T\Name2\d\f,email#jsshjs,d,f
...
This file is large.
I also have another file called Note.txt.
Name1
Name2
Name3
...
I'm trying to get the content of Import.csv and for each line in Note.txt if the line in Note.txt matches any line in Import.csv, then copy that line into a CSV with append. Continue adding every other line that is matched. Then this loops on each line of the CSV.
I need to find the best way to do it without having it import the CSV multiple times, since it is large.
What I got does the opposite though, I think:
$Dir = PathToFile
$import = Import-Csv $Dir\import.csv
$NoteFile = "$Dir\Note.txt"
$Note = GC $NoteFile
$Name = (($Import.Name).Split("\"))[4]
foreach ($j in $import) {
foreach ($i in $Note) {
$j | where {$Name -eq "$i"} | Export-Csv "$Dir\Result.csv" -NoTypeInfo -Append
}
}
This takes too long and I'm not getting the extraction I need.

This takes too long and I'm not getting the extraction I need.
That's because you only assign $name once, outside of the outer foreach loop, so you're basically performing the same X comparisons for each line in the CSV.
I would rewrite the nested loops as a single Where-Object filter, using the -contains operator:
$Import |Where-Object {$Note -contains $_.Name.Split('\')[4]} |Export-Csv "$Dir\Result.csv" -NoTypeInformation -Append

Group the imported data by your distinguishing feature, filter the groups by name, then expand the remaining groups and write the data to the output file:
Import-Csv "$Dir\import.csv" |
Group-Object { $_.Name.Split('\')[4] } |
Where-Object { $Note -contains $_.Name } |
Select-Object -Expand Group |
Export-Csv "$Dir\Result.csv" -NoType

Append existing column in csv by matching values with array and with condition

I will do my best to break this down as simply as I can.
what I have so far that is working:
Currently I have two csv files...
test1.csv
test1ColumnN,test1ColumnI,test1ColumnD,selectDomainOne,selectDomainTwo,selectDomainThree
asdf,asdf,asdf,,,
nValue1,iValue1,dValue1,sValue1,,
qwer,asdf,zxcv,,,
nValue2,iValue2,dValue2,,,
qwer,zxcv,asdf,lkjh,,
nValue3,iValue3,dValue3,sValue3,,
zxcv,qwer,asdf,,poiu,
nValue1,iValue1,dValue1,,sValue1,
nValue4,iValue4,dValue4,,sValue4,
asdf,qwer,zxcv,fghj,mnbv,
nValue5,iValue5,dValue5,,,
asdf,cvbn,erty,,,uytr
nValue7,iValue7,dValue7,,,sValue7
nValue8,iValue8,dValue8,,,sValue8
nValue9,iValue9,dValue9,,,sValue9
qwer,asdf,zxcv,poiu,lkjh,mnbv
test2.csv
DomainCatagories,test2ColumnS,test2ColumnA,test2ColumnN,test2ColumnI,test2ColumnD
DomainOne,sValue1,aValue1,nValue1,,dValueN
DomainOne,sValue2,aValue2,,iValue2,dValue2
DomainOne,sValue3,aValue2,nValue3,iValue3,dValue3
DomainTwo,sValue1,aValue2,,iValue1,dValueN
DomainTwo,sValue4,aValue1,nValue4,,dValueN
DomainTwo,sValue5,aValue1,nValue5,iValue5,dValue5
DomainThree,sValue7,aValue2,nValue7,iValue7,dValue7
DomainThree,sValue8,aValue1,nValue8,iValue8,dValue8
DomainThree,sValue9,aValue2,nValue9,iValue9,dValue9
Now I want to add a column (inside test2.csv) to match the sValue# from both test1.csv and test2.csv with the condition of ($_.DomainCatagories='DomainOne' from test2.csv) and ($_.selectDomainOne from test1.csv)
To do this, I am using the following code...
#Create Column
$domainNameOne = #{}
$domainNameOne = Import-Csv 'C:\Scripts\Tests\test1.csv' | Where-Object {$_.selectDomainOne} | Select-Object -Expand 'selectDomainOne'
(Import-Csv 'C:\Scripts\Tests\test2.csv') |
Select-Object -Property *, #{n='Test1sValues';e={
if($_.DomainCatagories -eq 'DomainOne'){
if(($domainNameOne -contains $_.test2ColumnS) -and ($_.test2ColumnS)){
$_.test2ColumnS
} Else {
'Not found in test1'
}}}} | Export-Csv "C:\Scripts\Tests\test2-Temp" -NoType
Move-Item "C:\Scripts\Tests\test2-Temp" 'C:\Scripts\Tests\test2.csv' -Force
After the code is run, I get the following test2.csv (isCorrect)...
"DomainCatagories","test2ColumnS","test2ColumnA","test2ColumnN","test2ColumnI","test2ColumnD","Test1sValues"
"DomainOne","sValue1","aValue1","nValue1","","dValueN","sValue1"
"DomainOne","sValue2","aValue2","","iValue2","dValue2","Not found in test1"
"DomainOne","sValue3","aValue2","nValue3","iValue3","dValue3","sValue3"
"DomainTwo","sValue1","aValue2","","iValue1","dValueN",""
"DomainTwo","sValue4","aValue1","nValue4","","dValueN",""
"DomainTwo","sValue5","aValue1","nValue5","iValue5","dValue5",""
"DomainThree","sValue7","aValue2","nValue7","iValue7","dValue7",""
"DomainThree","sValue8","aValue1","nValue8","iValue8","dValue8",""
"DomainThree","sValue9","aValue2","nValue9","iValue9","dValue9",""
What I have that is not working:
Next I run the following code...
#Append Column
$domainNameThree = #{}
$domainNameThree = Import-Csv 'C:\Scripts\Tests\test1.csv' | Where-Object {$_.selectDomainThree} | Select-Object -Expand 'selectDomainThree'
(Import-Csv 'C:\Scripts\Tests\test2.csv') | % {
if($_.DomainCatagories -eq 'DomainThree'){
if(($domainNameThree -contains $_.test2ColumnS) -and ($_.test2ColumnS)){
$_.Test1sValues = $_.test2ColumnS
} Else {
$_.Test1sValues = 'Not found in test1'
}}} | Export-Csv "C:\Scripts\Tests\test2-Temp" -NoType
Move-Item "C:\Scripts\Tests\test2-Temp" 'C:\Scripts\Tests\test2.csv' -Force
Instead of adding the values in the correct rows, it completely blanks out the whole file and saves it as an empty file.
End Goal
What I want the code to produce, is this (notice values filled in on last 3 rows in the last column)...
"DomainCatagories","test2ColumnS","test2ColumnA","test2ColumnN","test2ColumnI","test2ColumnD","Test1sValues"
"DomainOne","sValue1","aValue1","nValue1","","dValueN","sValue1"
"DomainOne","sValue2","aValue2","","iValue2","dValue2","Not found in test1"
"DomainOne","sValue3","aValue2","nValue3","iValue3","dValue3","sValue3"
"DomainTwo","sValue1","aValue2","","iValue1","dValueN",""
"DomainTwo","sValue4","aValue1","nValue4","","dValueN",""
"DomainTwo","sValue5","aValue1","nValue5","iValue5","dValue5",""
"DomainThree","sValue7","aValue2","nValue7","iValue7","dValue7","sValue7"
"DomainThree","sValue8","aValue1","nValue8","iValue8","dValue8","sValue8"
"DomainThree","sValue9","aValue2","nValue9","iValue9","dValue9","sValue9"
What am I doing wrong in that 2nd code snippet?

The example you show from What I have that is not working: is missing a key portion. Export-Csv will take everything piped into it to populate the CSV but you are not providing any.
Problem is that you are not passing anything through the pipe. Merely just updating one property. The simplest thing to do is add $_ after the if statement. Or you could just use a calculated property which you have done before in another one of your questions. The example below from Compare dates with different formats in csv file even uses an if statement.
Import-Csv $csvFile | Select-Object *, #{n='MatchDates';e={ if((([datetime]$_.Date1).Date -eq $_.Date3) -and (([datetime]$_.Date2).Date -eq $_.Date3) -and (([datetime]$_.Date1).Date -eq $_.Date2)){ 'Match Found' }Else{ 'No Match Found' }}} |
Export-Csv "$csvFile-results.csv" -NoTypeInformation -Force

How to change column position in powershell?

Is there any easy way how to change column position? I'm looking for a way how to move column 1 from the beginning to the and of each row and also I would like to add zero column as a second last column. Please see txt file example below.
Thank you for any suggestions.
File sample
TEXT1,02/10/2015,55.930,57.005,55.600,56.890,1890
TEXT2,02/10/2015,51.060,52.620,50.850,52.510,4935
TEXT3,02/10/2015,50.014,50.74,55.55,52.55,5551
Output:
02/10/2015,55.930,57.005,55.600,56.890,1890,0,TEXT1
02/10/2015,51.060,52.620,50.850,52.510,4935,0,TEXT2
02/10/2015,50.014,50.74,55.55,52.55,5551,0,TEXT3

Another option:
#Prepare test file
(#'
TEXT1,02/10/2015,55.930,57.005,55.600,56.890,1890
TEXT2,02/10/2015,51.060,52.620,50.850,52.510,4935
TEXT3,02/10/2015,50.014,50.74,55.55,52.55,5551
'#).split("`n") |
foreach {$_.trim()} |
sc testfile.txt
#Script starts here
$file = 'testfile.txt'
(get-content $file -ReadCount 0) |
foreach {
'{1},{2},{3},{4},{5},{6},0,{0}' -f $_.split(',')
} | Set-Content $file
#End of script
#show results
get-content $file
02/10/2015,55.930,57.005,55.600,56.890,1890,0,TEXT1
02/10/2015,51.060,52.620,50.850,52.510,4935,0,TEXT2
02/10/2015,50.014,50.74,55.55,52.55,5551,0,TEXT3

Sure, split on commas, spit the results back minus the first result joined by commas, add a 0, and then add the first result to the end and join the whole thing with commas. Something like:
$Input = #"
TEXT1,02/10/2015,55.930,57.005,55.600,56.890,1890
TEXT2,02/10/2015,51.060,52.620,50.850,52.510,4935
TEXT3,02/10/2015,50.014,50.74,55.55,52.55,5551
"# -split "`n"|ForEach{$_.trim()}
$Input|ForEach{
$split = $_.split(',')
($Split[1..($split.count-1)]-join ','),0,$split[0] -join ','
}

I created file test.txt to contain your sample data. I Assigned each field a name, "one","two","three" etc so that i could select them by name, then just selected and exported back to csv in the order you wanted.
First, add the zero to the end, it will end up as second last.
gc .\test.txt | %{ "$_,0" } | Out-File test1.txt
Then, rearrange order.
Import-Csv .\test.txt -Header "one","two","three","four","five","six","seven","eight" | Select-Object -Property two,three,four,five,six,seven,eight,one | Export-Csv test2.txt -NoTypeInformation
This will take the output file and get rid of quotes and header line if you would rather not have them.
gc .\test2.txt | %{ $_.replace('"','')} | Select-Object -Skip 1 | out-file test3.txt

trim object contents in csv import

I need to run a trim method on each value extracted from the csv import object. haven't tried something like below but for me I don't want to have to define a trim command at the end of each one of my variables being passed to functions.
$csvobj = "c:\somestuff.csv"
foreach ($csvitem in $csvobj) {
$csvitem.value1.trim()
$csvitem.value2.trim()
}
Thanks in advance, SS

This will trim all values in the csv file and assign the result to $csv.
$csv = Import-Csv c:\somestuff.csv | Foreach-Object {
$_.PSObject.Properties | Foreach-Object {$_.Value = $_.Value.Trim()}
}

You can try this :
$a = import-csv "c:\somestuff.csv"
$a | % {$b=$_;$_.psobject.Properties | % {$c=$_.name ;$b."$c"=($b."$c").trim()}}
First : I import all the lines into $a.
Second : In a first loop I read each object (each CSV line), then for each object in a second loop I read each property, then I can trim each var.

Here's a slight modification of the above accepted answer. This strips all leading / tailing spaces from all .csv files in the current folder. Hope it helps!
gci -filter *.csv | foreach {
echo $_.NAME
$csv = Import-Csv $_.NAME
$csv | Foreach-Object {
$_.PSObject.Properties | Foreach-Object { $_.Value = $_.Value.Trim() }
}
$csv | Export-Csv $_.NAME -NoTypeInformation
}