I have two files/outputs.
$file1 contains:
Name,Address,Number
$file2 contains:
Name
I'm looking to match those two files on Name and output the entire line from $file1
Assuming that your files are CSV files:
Combine Import-Csv with Where-Object to filter those lines, with the help of the -inoperator:
Import-Csv $file1 | # Parse the CSV file into objects
Where-Object Name -in (Import-Csv $file2).Name # Filter
Note:
This is a conceptually simple, but potentially slow solution, depending on the size of the input files. More efficient - but more elaborate - solutions are possible.
The filtered CSV rows are output as objects (of type [pscustomobject]), as parsed by Import-Csv; you could convert them back to CSV via ConvertTo-Csv (in memory) or save them back to a file with Export-Csv.
Related
I have a csv with columns A, B, C. I would like to import that to powershell and only on column C, remove any rows that have the word "Unknown" listed. If Column A or B has "Unknown", they stay, however, if Column C has it, the entire row gets deleted. Per the picture below, Row 4 would be deleted.
Can someone please provide a sample script to do this?
Thanks!
So, you have 3 problems you need to solve:
Import the data from the CSV file
Filter it based on the value of column C
Export the filtered data to a file again
To import, use the aptly named Import-Csv cmdlet:
$data = Import-Csv .\path\to\file.csv
Import-Csv will parse the CSV file and for each row it reads, it will output 1 object with properties corresponding to the column names in the header row.
To filter these objects based on the value of their C property, use the Where-Object cmdlet:
$filteredData = $data |Where-Object C -ne 'Unknown'
Where-Object will test whether the C property on each object does not have the value 'Unknown' (-ne = not equals), and discard any object for which that's not the case.
To re-export the filtered data, use the Export-Csv cmdlet:
$filteredData |Export-Csv .\path\to\output.csv -NoTypeInformation
You can also combine all three statements into a single pipeline expression:
Import-Csv .\path\to\file.csv |Where-Object C -ne 'Unknown' |Export-Csv .\path\to\output.csv -NoTypeInformation
This "one-liner" approach might be preferable if you're working on large CSV files (> hundreds of thousands of records), as it doesn't require reading the entire CSV file into memory at once.
$Data = Get-Content "C:\file.csv" | ConvertFrom-Csv
$Data | Where-Object {$_.C-ne 'Unknown'} | Export-Csv "C:\file_New.csv"
I have a need to generate two header rows to an existing csv file because the system where the csv will be uploaded needs the two header rows. The csv file will contain data that I want to keep.
I have been testing a powershell script to do this, and I can write a single row of headers, but am struggling to write two rows.
Below is the powershell script I am currently trying to build out.
$file = "C:\Users\_svcamerarcgis\Desktop\Test.csv"
$filedata = import-csv $file -Header WorkorderETL 'n ICFAORNonICFA, WONUmber, Origin
$filedata | export-csv $file -NoTypeInformation
The end result I'm looking for should be as follows:
WorkorderETL
ICFAORNonICFA, WONUmber, Origin
xxx,yyy,zzz
The sole purpose of Import-Csv's -Header parameter is to provide an array of column names to serve as the property names of the custom objects that the CSV rows are parsed into - you cannot repurpose that for special output formatting for later exporting.
You can use the following approach instead, bypassing the need for Import-Csv and Export-Csv altogether (PSv5+):
$file = 'C:\Users\User\OneDrive\Scripts\StackTesting\Test.csv'
# Prepend the 2-line header to the existing file content
# and save it back to the same file
# Adjust the encoding as needed.
#'
WorkorderETL
ICFAORNonICFA,WONUmber,Origin
'# + (Get-Content -Raw $file) | Set-Content $file -NoNewline -Encoding utf8
To be safe, be sure to create a backup of the original file first.
Since the file is being read (in full) and rewritten in the same pipeline, there's a hypothetical chance of data loss if writing back to the input file get interrupted.
You may be better trying to handle this as a text file, considering you are just trying to add a single line at the top of the CSV:
$file = "C:\Users\User\OneDrive\Scripts\StackTesting\Test.csv"
$CSV = "c1r1, c2r1, c3r1 `nc1r2, c2r2, c3r2"
$filedata = Get-Content $file
$filedata = "WorkorderETL`n" + $CSV
$filedata | Out-File $file
This will resul in the CSV file holding:
WorkorderETL
c1r1, c2r1, c3r1
c1r2, c2r2, c3r2
Which looks to be what you want.
I have a tab delimited txt file and i need to switch first and second column names (without switching columns data). In other words I need to rename A(Id) to B(ExternalId) and B(ExternalId) to A(Id). Other columns in the file (other data) should stay unchanged. I'm very new in PowerShell, please advice. As I understand I need to use import/export csv cmdlet.
I tryed this, but it's not working the right way...
Import-Csv 'C:\original_users.txt' |
Select-Object Id, #{Name="ExternalId";Expression={$_."Id"}}; Select-Object ExternalId, #{Name="Id";Expression={$_."ExternalId"}} |
Export-Csv 'C:\changed_users.txt'
The Import-CSV and Export-CSV cmdlets have their strengths but this might not be one of them. The latter cmdlet would introduce quoting that might not be in your original file and that might not be desired.
Either way why not just do some text manipulation on the first line! Lets read in the file and and output the first lined, edited, and the remainder of the file. This sample uses a new location but you could easily write it back to the same file.
# Get the full file into a variable
$fullFile = Get-Content "c:\temp\mockdata.csv"
# Parse the first line into a column array
$columns = $fullFile[0].Split("`t")
# Rebuild the header by switching the columns order as desired.
$newHeader = ($columns[1],$columns[0] + ($columns | Select-Object -Skip 2)) -join "`t"
# Write the header back to file then the rest of the data.
$outputPath = "C:\somepath.txt"
$newHeader | Set-Content $outputPath
$fullFile | Select-Object -Skip 1 | Add-Content $outputPath
This also preserves the presence of other columns and their data.
I am having problem with filtering a specific value from my csv file. The csv file looks as follows:
"1","19/Oct/2016","15:03:58","19/Oct/2014","15:03:58","0:00:00","---","---","nice_meme#help.com","---","sip","1232Kbps","---","Out","1140","1","---","---","---","user:---","---","---","---","---","---","---","---","Failed Attempt; ""Your call could not be completedOver.""","3","---","---","---","---","---","---","---","---","---","---","---","---","---"
As you can see there are multiple values with '---', I tried a lot of ways to remove these three dashes. I do not know how to filter them out using PowerShell. I want to get the rows that are not equal to the three dashes.
Something like this:
$a = Import-CSV -Path "C:\Transformed\test.csv" | Where-Object {$_.Header -neq "---"}
I am a powershell newbie and I need a script that does a write output to a delineated text file in powershell.
In sequence, this is what I would like to do in powershell:
read data from excel file and store to variables
read data from table in mssql
write output the each row from (2) to a text file (pipe delineated) appending the value from (1)
I was able to figure out sequence 1 and 2 however I am stumped on (3).
Here is a snippet of what i am trying to do:
# Iterate through the dataset
foreach ($row in $ds.tables["location"].rows)
{
Out-file ?
}
Please help!
$myData | Export-CSV -delimiter '|' -Path $MyFileName
Note, the Export-CSV cmdlet will handle looping over the data, so no need for your own loop.