This question already has answers here:
delete empty rows at end of the csv file using Powershell
(2 answers)
Closed 1 year ago.
To delete empty rows in multiple CSV files by using Powershell. this is the code I am trying with for each loop. But it is writing empty files. The resulting file should open in excel and notepad++ with proper format.
Get-ChildItem $paramDest -Filter *Test*.csv | ForEach-Object {
$content = Get-Content $_.FullName | Where { $_.Replace("","","").trim() -ne "" }
Out-File -InputObject $content -FilePath $_.FullName
}
I have some other set of files, the code should work for both these kinds of files. I am okay to have 2 separate codes for these 2 separate files.
Here another file sample format
Try with this
Input
Command
Import-Csv -Path "C:\sample.csv" | Where-Object { $_.PSObject.Properties.Value -ne '' } | Export-Csv -Path "C:\Sample_clean.csv" -NoTypeInformation
Output*
Edit: To update Multiple files, thanks to #Santiago Squarzon
Get-ChildItem $paramDest -Filter *Test*.csv |
Foreach-Object {
# get the content of file
$content = Get-Content $_.FullName
# replace all "," with empty string and exclude those lines and save the output to original file
$content | where { $_.Replace('","','').trim('"') -ne '' } | Set-Content $_.FullName
}
Related
First of all, I'm a complete newbie at Powershell.
I've basically compiled a script from a number google search results and it works to a certain degree, so be gentle :)
I have a number of large plain text files that need scanning, junk data needs removing, and characters need renaming. Then create a new file in the same directory
Here is the script I have for individual files, I have replaced actual keywords for something unrelated, but for testing purposes you should see what I am trying to achieve:
Get-Content C:\Temp\Tomatoes-2022-09-27.txt |
Where-Object { - $_.Contains('red') } | # Keeping only lines containing "red"
Foreach {$_ -replace "[/()]",":"}| # replacing specific characters to a colon
Where-Object { -not $_.Contains('too red') } | # removing lines containing "too red"
Set-Content C:\Temp\Tomatoes-2022-09-27Ripe.txt # saving as a new file *Ripe.txt
This works for individual files just fine but what I need to do is the same process for any file within the Temp directory.
They all have similar names other than the date.
Here's what I have compiled for all files, but it overwrites existing files rather than creating a new one and I don't know how to get it to write to new files ie Tomotoes*Ripe.txt: *being the unique date
Get-ChildItem C:\Temp\*.* -Recurse | ForEach-Object {
(Get-Content $_) |
Where-Object { - $_.Contains('red') } |
ForEach-Object { $_ -replace "[/()]", ":" } |
Where-Object { -not $_.Contains('too red') } |
Set-Content $_
}
Or will it be better to create a copy first using New-Item then process the other jobs?
It's going to be something very simple I know! And will most definitely kick myself once corrected.
Thanks in advance
Looks like what you want is something like this:
Get-ChildItem -Path 'C:\Temp' -File -Recurse | ForEach-Object {
$newFile = Join-Path -Path $_.DirectoryName -ChildPath ('{0}Ripe{1}' -f $_.BaseName, $_.Extension)
$newContent = Get-Content $_.FullName |
Where-Object { $_ -like '*red*' -and $_ -notlike '*too red*' } |
ForEach-Object { $_ -replace "[/()]", ":" }
$newContent | Set-Content -Path $newFile
}
To complement Theo's helpful answer - which is probably the most straightforward in your case - with a streaming, single-pipeline solution that showcases two advanced techniques:
the common -PipelineVariable (-pv) parameter, which allows you to store a cmdlet's current pipeline output object in a self-chosen variable that can be referenced later in a script block in a later pipeline segment.
delay-bind script blocks, which allow you to use a script block to dynamically determine a parameter value, typically based on the pipeline input object at hand; in this case, the pipeline variable is used.
# Create sample files
'red1' > t1.txt
'red2' > t2.txt
Get-ChildItem -PipelineVariable file t?.txt | # note `-PipelineVariable file`
Get-Content | # read file line by line
Where-Object { $_.Contains('red') } | # sample filter
ForEach-Object { $_ -replace 'e', '3' } | # sample transformation
Set-Content -LiteralPath { # delay-bind script block
# Determine the output file name based on pipelinve variable $file
'{0}Ripe{1}' -f $file.BaseName, $file.Extension
}
This results in files t1Ripe.txt and t2Ripe.txt, containing r3d1 and r3d2, respectively.
I need to merge a bunch of CSV files into one big CSV and selecting only for the rows that contain value "XX" in column 2
I created the script to merge the CSV (that also removes the first 8 lines of the header) but I don't know how to filter the rows to be copied into the merged file with the condition above.
$InFiles = Get-ChildItem -LiteralPath '.\' -File | Where-Object { $_.Extension -eq '.csv' }
ForEach ($File In $InFiles)
{
$File | Get-Content | Select-Object -Skip 8 | Out-File -LiteralPath '.\out\OutFile.csv' -Append -Encoding ascii
}
This question already has an answer here:
How to replace the content of every file in the directory?
(1 answer)
Closed 5 years ago.
I am trying to read multiple CSV files inside multiple sub directories and doing some row deletion based on filter using PowerShell scripting.
Get-ChildItem -Path J:\new -Recurse -Filter daq*.csv | ForEach-Object {
Get-Content $_.FullName | Where {
$_ -notmatch "serverTime"
} | Out-File $_.FullName
}
But I want the output to be in same directory structure as the source file.
The directory structure is as below:
root/
sub_dir1/
1.csv
2.csv
sub_dir2/
1.csv
2.csv
Is there a way I can do it?
Probably not optimal, or even idiomatic, but the following worked in a quick test:
Get-ChildItem -Path J:\new -Recurse -Filter daq*.csv |
ForEach-Object {
$origFile = $_
$filteredContent = Get-Content $origFile.FullName | Where{$_ -notmatch "serverTime"}
$filteredContent | Out-File $origFile.FullName
}
All we're doing, is loading the content of each file and filtering it to $filteredContent, then writing it back out to the original file.
Another option (courtesy of Replacing Contents of a Text File Using Powershell) is to wrap the Get-Content command in parens, which forces the full content to be loaded and then passed down the pipeline. This will give shorter code, but arguably it's less understandable e.g.
Get-ChildItem -Path J:\new -Recurse -Filter daq*.csv |
ForEach-Object {
$origFile = $_
( Get-Content $origFile.FullName ) |
Where{$_ -notmatch "serverTime"} |
Out-File $origFile.FullName
}
I have a text file that looks like this:
Data I'm NOT looking for
More data that doesn't matter
Even more data that I don't
&Start/Finally the data I'm looking for
&Data/More data that I need
&Stop/I need this too
&Start/Second batch of data I need
&Data/I need this too
&Stop/Okay now I'm done
Ending that I don't need
Here is what the output needs to be:
File1.txt
&Start/Finally the data I'm looking for
&Data/More data that I need
&Stop/I need this too
File2.txt
&Start/Second batch of data I need
&Data/I need this too
&Stop/Okay now I'm done
I need to do this for every file in a folder (sometimes there will be multiple files that will need to be filtered.) The files names can be incrementing: ex. File1.txt, File2.txt, File3.txt.
This is what I have tried with no luck:
ForEach-Object{
$text -join "`n" -split '(?ms)(?=^&START)' -match '^&START' |
Out-File B:\PowerShell\$filename}
Thanks!
Looks like you were pretty close: your code correctly extracted the paragraphs of interest, but intra-paragraph out-filtering of non-&-starting lines was missing, and you needed to write to paragraph-specific output files:
$text -join "`n" -split '(?m)(?=^&Start)' -match '^&Start' |
ForEach-Object { $ndx=0 } { $_ -split '\n' -match '^&' | Out-File "File$((++$ndx)).txt" }
This creates sequentially numbered files starting with File1.txt for every paragraph of interest.
To do it for every file in a folder, with output filenames using fixed naming scheme File<n> across all input files (and thus cumulative numbering):
Get-ChildItem -File . | ForEach-Object -Begin { $ndx=0 } -Process {
(Get-Content -Raw $_) -split '(?m)(?=^&Start)' -match '^&Start' |
ForEach-Object { $_ -split '\n' -match '^&' | Out-File "File$((++$ndx)).txt" }
}
To do it for every file in a folder, with output filenames based on the input filenames and numbering per input file (PSv4+, due to use of -PipelineVariable):
Get-ChildItem -File . -PipelineVariable File | ForEach-Object {
(Get-Content -Raw $_) -split '(?m)(?=^&Start)' -match '^&Start' |
ForEach-Object {$ndx=0} { $_ -split '\n' -match '^&' | Out-File "$($File.Name)$((++$ndx)).txt" }
}
You post a second question (against the rules) and it was deleted but here is my quick answer for it. I hope it will help you and give you more sense how PS works:
$InputFile = "C:\temp\test\New folder (3)\File1.txt"
# get file content
$a=Get-Content $InputFile
# loop for every line in range 2 to last but one
for ($i=1; $i -lt ($a.count-1); $i++)
{
#geting string part between & and / , and construct output file name
$OutFile = "$(Split-Path $InputFile)\$(($a[$i] -split '/')[0] -replace '&','').txt"
$a[0]| Out-File $OutFile #creating output file and write first line in it
$a[$i]| Out-File $OutFile -Append #write info line
$a[-1]| Out-File $OutFile -Append #write last line
}
Something like this?
$i=0
gci -path "C:\temp\ExplodeDir" -file | %{ (get-content -path $_.FullName -Raw).Replace("`r`n`r`n", ";").Replace("`r`n", "~").Split(";") | %{if ($_ -like "*Start*") {$i++; ($_ -split "~") | out-file "C:\temp\ResultFile\File$i.txt" }} }
I am trying to do a simple script that pulls in the name of the file and the contents of said text file into a CSV file. I am able to pull in all of the information well enough but it's not splitting up into different columns in the CSV file. When I open up the CSV file in excel everything is in the first column, and I need the two bits of information separated into separate columns. So far my working code is as follows:
$Data = Get-ChildItem -Path c:path -Recurse -Filter *.txt |
where {$_.lastwritetime -gt(Get-Date).addDays`enter code here`(-25)}
$outfile = "c:path\test.csv"
rm $outfile
foreach ($info in $Data) {
$content = Get-Content $info.FullName
echo "$($info.BaseName) , $content" >> $outfile
}
I figured out how to seperate the information by rows but I need it by columns. I'm new to powershell and can't seem to get past this little speed bump. Any input would be greatly appreciated. Thanks in advance!
Output:
Itm# , TextContent
Itm2 , NextTextContent
What I need:
Itm# | Text Content |
Itm2 | NextTextContent |
Except for a few syntactical errors your code appears to be working as expected. I worry if you are having issues in Excel with you text import. I touched up your code a bit but it is functionally the same as what you had.
$Data = Get-ChildItem -Path "C:\temp" -Recurse -Filter *.txt |
Where-Object {$_.LastWriteTime -gt (Get-Date).addDays(-25)}
$outfile = "C:\temp\test.csv"
If(Test-Path $outfile){Remove-Item $outfile -Force}
foreach ($info in $Data) {
$content = Get-Content $info.FullName
"$($info.BaseName) , $content" | Add-Content $outfile
}
I don't know what version of Excel you have but look for the text import wizard.
Do you mean something like this?
Get-ChildItem -Path C:\path -Recurse -Filter *.txt |
Where-Object { $_.LastWriteTime -gt (Get-Date).AddDays(-25) } | ForEach-Object {
New-Object PSObject -Property #{
"Itm#" = $_.FullName
"TextContent" = Get-Content $_.FullName
} | Select-Object Itm#,TextContent
} | Export-Csv List.csv -NoTypeInformation
Excel will treat the data in csv files which are delimited bij the ; as a single columns.
I always use the -delimiter switch on export-csv or convertto-csv to set this as a delimiter.