I have the following code in powershell:
foreach ($file in $Files)
{
(Get-Content $file.PSPath) |
Foreach-Object {$_ -replace [regex]::escape($find), $repl} |
Set-Content $file.PSPath
}
It searches a bunch of files and makes replacements and works fine. I need to make a copy of the file before modifying it by copying it somewhere. What I don't know is how to know if a replacement was made.
My question is how do you know if it made a replacement? Do I have to do a search separately (like substring)? Obviously, I only want to make 1 backup regardless of how many lines are changed in the file. I'm not using wildcards/regex expressions if that matters.
Use the -match operator to perform a match test. Then if the result is true, perform the operations you want, such as backing up the file e.g.:
$esc_find = [regex]::escape($find)
$content = $file | Get-Content
if ($content -match $esc_find) {
Copy-Item -Path $file.FullName -Destination ([IO.Path]::ChangeExtension($file, 'bak')
$content -replace $esc_find | Set-Content -Path $file.FullName
}
Related
First of all, I'm a complete newbie at Powershell.
I've basically compiled a script from a number google search results and it works to a certain degree, so be gentle :)
I have a number of large plain text files that need scanning, junk data needs removing, and characters need renaming. Then create a new file in the same directory
Here is the script I have for individual files, I have replaced actual keywords for something unrelated, but for testing purposes you should see what I am trying to achieve:
Get-Content C:\Temp\Tomatoes-2022-09-27.txt |
Where-Object { - $_.Contains('red') } | # Keeping only lines containing "red"
Foreach {$_ -replace "[/()]",":"}| # replacing specific characters to a colon
Where-Object { -not $_.Contains('too red') } | # removing lines containing "too red"
Set-Content C:\Temp\Tomatoes-2022-09-27Ripe.txt # saving as a new file *Ripe.txt
This works for individual files just fine but what I need to do is the same process for any file within the Temp directory.
They all have similar names other than the date.
Here's what I have compiled for all files, but it overwrites existing files rather than creating a new one and I don't know how to get it to write to new files ie Tomotoes*Ripe.txt: *being the unique date
Get-ChildItem C:\Temp\*.* -Recurse | ForEach-Object {
(Get-Content $_) |
Where-Object { - $_.Contains('red') } |
ForEach-Object { $_ -replace "[/()]", ":" } |
Where-Object { -not $_.Contains('too red') } |
Set-Content $_
}
Or will it be better to create a copy first using New-Item then process the other jobs?
It's going to be something very simple I know! And will most definitely kick myself once corrected.
Thanks in advance
Looks like what you want is something like this:
Get-ChildItem -Path 'C:\Temp' -File -Recurse | ForEach-Object {
$newFile = Join-Path -Path $_.DirectoryName -ChildPath ('{0}Ripe{1}' -f $_.BaseName, $_.Extension)
$newContent = Get-Content $_.FullName |
Where-Object { $_ -like '*red*' -and $_ -notlike '*too red*' } |
ForEach-Object { $_ -replace "[/()]", ":" }
$newContent | Set-Content -Path $newFile
}
To complement Theo's helpful answer - which is probably the most straightforward in your case - with a streaming, single-pipeline solution that showcases two advanced techniques:
the common -PipelineVariable (-pv) parameter, which allows you to store a cmdlet's current pipeline output object in a self-chosen variable that can be referenced later in a script block in a later pipeline segment.
delay-bind script blocks, which allow you to use a script block to dynamically determine a parameter value, typically based on the pipeline input object at hand; in this case, the pipeline variable is used.
# Create sample files
'red1' > t1.txt
'red2' > t2.txt
Get-ChildItem -PipelineVariable file t?.txt | # note `-PipelineVariable file`
Get-Content | # read file line by line
Where-Object { $_.Contains('red') } | # sample filter
ForEach-Object { $_ -replace 'e', '3' } | # sample transformation
Set-Content -LiteralPath { # delay-bind script block
# Determine the output file name based on pipelinve variable $file
'{0}Ripe{1}' -f $file.BaseName, $file.Extension
}
This results in files t1Ripe.txt and t2Ripe.txt, containing r3d1 and r3d2, respectively.
Data mapping project, in house system to new vendor system. First step is find all the occurrences of current database field names (or column names to be precise) in the C# .cs source files. Trying to use Powershell. Have recently created PS searches with Get-ChildItem and Select-String that work well but the search string array was small and easily hard coded inline. But the application being ported has a couple hundred column names and significant amounts of code. So armed with a text file of all the column names Pipleline would seem like a god tool to create a the basic cross ref for further analysis. However, I was not able to get the Pipeline to work with an external variable anyplace other than first step. Trying using -PipelineVariable, $_. and global variable. Did not find anything specific after lots of searching. P.S. This is my first question to StackoOverflow, be kind please.
Here is what I hoped would work but do dice so far.
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr = [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force |
Select-String $s | Select-Object Path, LineNumber, line | Export-csv $outputfile
}
Did find that this will print the list one time but not twice. In fact it seems using the variable in this way results in processing simply skipping any further pipeline steps.
foreach ($s in $arr) {Write-Host $s | Write $s}
If it isn't possible to do this in Powershell easily my fallback is to do with C# although would much rather get the level up with PowerShell if anyone can point me to the correct understanding of how to do things in the Pipepline, or alternatively construct an equivalent function. Seems like such a natural fit for Powershell.
Thanks.
You're calling Export-csv $outputfile in a loop, which rewrites the whole file in every iteration, so that only the last iteration's output will end up in the file.
While you could use -Append to iteratively append to the output file, it is worth aking a step back: Select-String can accept an array of patterns, causing a line that matches any of them to be considered a match.
Therefore, your code can be simplified as follows:
$inputFile = 'C:\DataColumnsNames.txt'
$outputFile = 'C:\DataColumnsUsages.txt'
Get-ChildItem C:\ProjectFolder -Filter *.cs -Recurse -Force -ea SilentlyContinue |
Select-String -Pattern (Get-Content $inputFile) |
Select-Object Path, LineNumber, line |
Export-csv $outputfile
-Pattern (Get-Content $inputFile) passes the lines of input file $inputFile as an array of patterns to match.
By default, these lines are interpreted as regexes (regular expressions); to ensure that they're treated as literals, add -SimpleMatch to the Select-String call.
This answer to a follow-up question shows how to include the specific pattern among the multiple ones passed to -Pattern that matched on each line in the output.
I think you want to append each occurrence to the csv file. And you need to get the content of the file. Try this:
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force | Foreach {
Get-Content "$_.Fullname" | Select-String $s | Select-Object Path, LineNumber, line | Export-csv -Append -Path "$outputfile"
}
}
-Append was not introduced before powershell v3.0 (Windows 8) then try this:
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force | Foreach {
Get-Content "$_.Fullname" | Select-String $s | Select-Object Path, LineNumber, line | ConvertTo-CSV -NoTypeInformation | Select-Object -Skip 1 | Out-File -Append -Path "$outputfile"
}
}
How can i read every csv file the specific folder? When script below is executed, it only will remove quote character of one csv file.
$file="C:\test\IV-1-2020-04-02.csv"
(GC $file) | % {$_ -replace '"', ''} > $file
Get-ChildItem -Path C:\test\ -Filter '*.csv'
The output only will remove the quote character of "IV-1-2020-04-02.csv". What if i have different filename ?
You can iterate each .csv file from Get-ChildItem and replace the quotes " with '' using Set-Content.
$files = Get-ChildItem -Path "YOUR_FOLDER_PATH" -Filter *.csv
foreach ($file in $files)
{
Set-Content -Path $file.FullName -Value ((Get-Content -Path $file.FullName -Raw) -replace '"', '')
}
Make sure to pass your folder path to -Path, which tells Get-ChildItem to fetch every file from this folder
Its also faster to use the -Raw switch for Get-Content, since it reads the file into one string and preserves newlines. If you omit this switch, Get-Content will by default split the lines by newlines into an array of strings
If you want to read files in deeper sub directories as well, then add the -Recurse switch to Get-ChildItem:
$files = Get-ChildItem -Path "YOUR_FOLDER_PATH" -Filter *.csv -Recurse
Addtionally, you could also use Foreach-Object here:
Get-ChildItem -Path "YOUR_FOLDER_PATH" -Filter *.csv -Recurse | ForEach-Object {
Set-Content -Path $_.FullName -Value ((Get-Content -Path $_.FullName -Raw) -replace '"', '')
}
Furthermore, you could replace Foreach-Object with its alias %. However, If your using VSCode and have PSScriptAnalyzer enabled, you may get this warning:
'%' is an alias of 'ForEach-Object'. Alias can introduce possible problems and make scripts hard to maintain. Please consider changing alias to its full content.
Which warns against using aliases for maintainability. Its much safer and more portable to use the full version. I only use the aliases for quick command line usage, but when writing scripts I use the full versions.
Note: The above solutions could potentially corrupt the CSV if some lines need quoting. This solution simply goes through the whole file and replaces every quote with ''. PowerShell 7 offers a -UseQuotes AsNeeded option for Export-Csv, so you may look into that instead.
Don't just replace all the " unless you are very certain that it's a good idea; otherwise replace the " when it shouldn't matter because the field doesn't contain text with a comma, double quote, nor line break. (see RFC-4180 section 2, #6 and #7)
As with any script that overwrites its working files, make sure you have backups of those files should you want an undo option later on...
$tog = $true
$sep = ':_:'
$header=#()
filter asString{
$obj=$_
if($tog){
$header=(gm -InputObject $obj -Type NoteProperty).Name
$hc = $header.Count-1
$tog=$false
$str = $header -join $sep
$str = "$sep$str" -replace '"','""'
$str = $str -replace "$sep(((?!$sep)[\s\S])*(,|""|\n)((?!$sep)[\s\S])*)",($sep+'"$1"')
($str -replace $sep,',').Substring(1)
}
$str = (0..$hc | %{$obj.($header[$_])}) -join $sep
$str = "$sep$str" -replace '"','""'
$str = $str -replace "$sep(((?!$sep)[\s\S])*(,|""|\n)((?!$sep)[\s\S])*)",($sep+'"$1"')
($str -replace $sep,',').Substring(1)
}
ls *.csv | %{$tog=$true;import-csv $_ | asString | sc "$_.new";$_.FullName} | %{if(test-path "$_.new"){mv "$_.new" $_ -force}}
Note: the CSV files are expected to contain their own headers. You could work around that if you needed to with the use of the -Header option of Import-Csv
How delete a line having a word quickly in multiple large files using PowerShell
i am using the below code but it take long time
$files = Get-ChildItem "D:\mjautomation\v19.0\filesdd\"
foreach ($file in $files) {
$c = Get-Content $file.fullname | where { $_ -notmatch "deletethisline" }
$c | Set-Content $file.fullname
The following should be reasonably fast due to use of switch -File, but note that it requires reading each file into memory as a whole (minus the excluded lines):
foreach ($file in Get-ChildItem -File D:\mjautomation\v19.0\filesdd) {
Set-Content $file.FullName -Value $(
switch -Regex -File $file.FullName {
'deletethisline' {} # ignore
default { $_ } # pass line through
}
)
}
If you don't want to read each file into memory in (almost) full, use a [System.IO.StreamWriter] instance, as shown in this answer instead of Set-Content to write to a temporary file, and then replace the original file.
Doing so has the added advantage of avoiding the small risk of data loss that writing back to the original file via in-memory operations bears.
If you want to make do with the - slower - Get-Content cmdlet, use the following; the same caveats as above apply:
foreach ($file in Get-ChildItem -File D:\mjautomation\v19.0\filesdd) {
Set-Content $file.FullName -Value (
#(Get-Content $file.FullName) -notmatch 'deletethisline'
)
}
Note that as an alternative to the foreach loop you can use a single pipeline with the ForEach-Object cmdlet - Get-ChildItem ... | ForEach-Object { <# work with $_ #> } - but doing so is slower (though in many cases that won't matter).
I have a text (.txt) file with following content:
Car1
Car2
Car3
Car4
Car5
For changing Car1 for random text I used this script:
Get-ChildItem "C:\Users\boris.magdic\Desktop\q" -Filter *.TXT |
Foreach-Object{
$content = Get-Content $_.FullName
$content | ForEach-Object { $_ -replace "Car1", "random_text" } | Set-Content $_.FullName
}
This is working ok, but now I want to add one text line under Car2 in my text file.
How can I do that?
Just chain another -replace and use a new line!
Get-ChildItem "C:\Users\boris.magdic\Desktop\q" -Filter *.TXT |
Foreach-Object{
$file = $_.FullName
$content = Get-Content $file
$content | ForEach-Object { $_ -replace "Car1", "random_text" -replace "(Car2)","`$1`r`nOtherText" } | Set-Content $file
}
First thing is that | Set-Content $_.FullName would not work since the file object does not exist in that pipe. So one simple this to do it save the variable for use later in the pipe. You can also use the ForEach($file in (Get-ChildItem....)) construct.
The specific change to get what you want is the second -replace. We place what you want to match in brackets to that we can reference it in the replacement string with $1. We use a backtick to ensure PowerShell does not treat it as a variable.
We can remove some redundancy as well since -replace will work against the strings of file as a whole
Get-ChildItem "c:\temp" -Filter *.TXT |
Foreach-Object{
$file = $_.FullName
(Get-Content $file) -replace "Car1", "random_text" -replace "(Car2)","`$1`r`nOtherText" | Set-Content $file
}
While this does work with your sample text I want to point out that more complicated strings might require more finesse to ensure you make the correct changed and that the replacements we are using are regex based and do not need to be for this specific example.
.Replace()
So if you were just doing simple replacements then we can update your original logic.
Foreach-Object{
$file = $_.FullName
$content = Get-Content $_.FullName
$content | ForEach-Object { $_.replace("Car1", "random_text").replace("Car2","Car2`r`nOtherText")} | Set-Content $file
}
So that is just simple text replacement chained using the string method .Replace()