I am trying to do something very simple in PowerShell.
Reading the contents of a file
Manipulation some string
Saving the modified test back to the file
function Replace {
$file = Get-Content C:\Path\File.cs
$file | foreach {$_ -replace "document.getElementById", "$"} |out-file -filepath C:\Path\File.cs
}
I have tried Set-Content as well.
I always get unauthorized exception. I can see the $file has the file content, error is coming while writing the file.
How can I fix this?
This is likely caused by the Get-Content cmdlet that gets a lock for reading and Out-File that tries to get its lock for writing. Similar question is here: Powershell: how do you read & write I/O within one pipeline?
So the solution would be:
${C:\Path\File.cs} = ${C:\Path\File.cs} | foreach {$_ -replace "document.getElementById", '$'}
${C:\Path\File.cs} = Get-Content C:\Path\File.cs | foreach {$_ -replace "document.getElementById", '$'}
$content = Get-Content C:\Path\File.cs | foreach {$_ -replace "document.getElementById", '$'}
$content | Set-Content C:\Path\File.cs
Basically you need to buffer the content of the file so that the file can be closed (Get-Content for reading) and after that the buffer should be flushed to the file (Set-Content, during that write lock will be required).
The accepted answer worked for me if I had a single file operation, but when I did multiple Set-Content or Add-Content operations on the same file, I still got the "is being used by another process" error.
In the end I had to write to a temp file, then copy the temp file to the original file:
(Get-Content C:\Path\File.cs) | foreach {$_ -replace "document.getElementById", '$'} | Set-Content C:\Path\File.cs.temp
Copy-Item C:\Path\File.cs.temp C:\Path\File.cs
Personal experience: I had the "locked file syndrome" in one of my procedures. I found it was caused by a New-Object assignment on the file. I realised that I had not issued a "Dispose()" call on the object. I rewrote the offending code to dispose of the 'New-Object' as soon as convenient and the "locked file" syndrome was resolved.
A learning event for me = always dispose of each New-Object!
Related
i'm really new to Powershell (2 days) and i am not good yet. :(
My Question ist:
How to change a character in multiple .txt-files and save/ overwrite the existing file in Powershell
My goal is to Copy multiple RAW-Files in a new folder, change the file-name from .tsv to .txt and at least change one character in these files from % to percent.
What i've got so far:
The first two steps are working, but i'm losing my mind with the third step (the replacement).
Copy-Item -Path "C:\Users\user\Desktop\RAW\*.tsv" -Destination "C:\Users\user\Desktop\TXT" -Recurse
Set-Location "C:\Users\User\Desktop\TXT"
Get-ChildItem *.tsv | Rename-Item -NewName { $_.Name -replace '.tsv','.txt' }
This works fine for me and now i am not able to get further ...
I am able to replace the "%" in one specific file, and save it in a new file, but this doesn't work for a batch processing with changing file-names.
$file = "A.txt"
Get-Content $file | Foreach {$_ -replace "%", "percent"} | Set-Content A_1.txt
It would be perfect, if "$file = "A.txt"" would be "all the files in this path with .txt" and
"Set-Content A_1.txt" would be "overwrite the existing file".
I hope someone will help me, thank you! <3 <3 <3
You already have some of the solution in your first code snippet, you need to iterate over the files again to perform the replace and save.
$txtFiles = Get-ChildItem -Name *.txt
ForEach ($file in $txtFiles) {
(Get-Content $file) | ForEach-Object {
$_ -replace '%','percent'
} | Set-Content $file
}
The first line adds all the text files to an array, the foreach loop iterates over the files of the array and grabs the content of the file and unloads it - that's the reason for the parenthesis, the Foreach-Object then iterates over the content of the file and saves it to the same file name as before.
If you skip the parentheses around Get-Content $file the file would still be loaded into memory and you would get an error message about not being able to save the file.
I am trying to run this script on a 50GB file in Windows 2012 R2 and I would like to hopefully get the three replace statements into one pass rather than three. Also, it is important that the replaces occur in that order. Any suggestions to simplify this and make it run efficiently would be greatly appreciated!
$filePath = "D:\FileLocation\file_name.csv"
(Get-Content $filePath | out-string).Replace('"', '""') | Set-Content $filePath
(Get-Content $filePath | out-string).Replace('|~|', '"') | Set-Content $filePath
(Get-Content $filePath | out-string).Replace('|#|', ',') | Set-Content $filePath
With such a large file, I suggest you process the file line by line (or in batches) which should speed up the entire process.
You can copy the Script mentioned by True here http://community.idera.com/powershell/ask_the_experts/f/learn_powershell-12/18821/how-to-remove-specific-rows-from-csv-files-in-powershell
but instead of writing $Line straight away, performing you replaces
$sw.WriteLine($line.replace().replace().replace())
Be careful with get-content since that will try to load the entire file and becomes very slow once you are out of memory.
Also be careful if you don't have much disk space. The linked solution will make a copy of the file (with the changes) before replacing it.
You can use -replace operator
$filepath="c:\temp\text.txt"
(Get-Content $filepath) -replace 'test','1' -replace 'text','2' -replace '123','3' |Set-Content $filepath
You can combine the .replace() in the same line.
$filepath="/Users/me/Desktop/text.txt"
'test text 123' |Out-File -Path $filepath
(Get-Content $filepath|Out-String).Replace('test','1').Replace('text','2').Replace('123','3')|Set-Content $filepath
Get-Content $filepath
1 2 3
I have a working powershell script to find and and replace a few different strings with a new string in thousands of files, without changing the modified date on the files. In any given file there could be hundreds of instances of said strings to replace. The files themselves aren't very large and probably range from 1-50MB (a quick glance at the directory I am testing with shows the largest as ~33MB).
I'm running the script inside a Server 2012 R2 VM with 4 vCPUs and 4GB of RAM. I have set the MaxMemoryPerShellMB value for Powershell to 3GB. As mentioned previously, the script works, but after 2-4 hours powershell will start throwing OutOfMemoryExceptions and crash. The script is 'V2 friendly' and I haven't adopted it to V3+ but I doubt that matters too much.
My question is whether or not the script can be improved to prevent/eliminate the memory exceptions I am running into at the moment. I don't mind if it runs slower, as long as it can get the job done without having to check back every couple of hours and restart it.
$i=0
$all = Get-ChildItem -Recurse -Include *.txt
$scriptfiles = Select-String -Pattern string1,string2,string3 $all
$output = "C:\Temp\scriptoutput.txt"
foreach ($file in $scriptFiles)
{
$filecreate=(Get-ChildItem $file.Path).creationtime
$fileaccess=(Get-ChildItem $file.Path).lastaccesstime
$filewrite=(Get-ChildItem $file.Path).lastwritetime
"$file.Path,Created: $filecreate,Accessed: $fileaccess,Modified: $filewrite" | out-file -FilePath $output -Append
(Get-Content $file.Path) | ForEach-Object {$_ -replace "string1", "newstring" `
-replace "string2", "newstring" `
-replace "string3", "newstring"
} | Set-Content $file.Path
(Get-ChildItem $file.Path).creationtime=$filecreate
(Get-ChildItem $file.Path).lastaccesstime=$fileaccess
(Get-ChildItem $file.Path).lastwritetime=$filewrite
$filecreate=(Get-ChildItem $file.Path).creationtime
$fileaccess=(Get-ChildItem $file.Path).lastaccesstime
$filewrite=(Get-ChildItem $file.Path).lastwritetime
"$file.Path,UPDATED Created: $filecreate,UPDATED Accessed: $fileaccess,UPDATED Modified: $filewrite" | out-file -FilePath $output -Append
$i++}
Any comments, criticisms, and suggestions welcomed.
Thanks
Biggest issue I can see is that you are repeatedly getting the file for every property you are querying. Replace that with one call per loop pass and save it to be used during the pass. Also Out-File is one of the slower methods of outputting data to file.
$output = "C:\Temp\scriptoutput.txt"
$scriptfiles = Get-ChildItem -Recurse -Include *.txt |
Select-String -Pattern string1,string2,string3 |
Select-Object -ExpandProperty Path
$scriptfiles | ForEach-Object{
$file = Get-Item $_
# Save currrent file times
$filecreate=$file.creationtime
$fileaccess=$file.lastaccesstime
$filewrite=$file.lastwritetime
"$file,Created: $filecreate,Accessed: $fileaccess,Modified: $filewrite"
# Update content.
(Get-Content $file) -replace "string1", "newstring" `
-replace "string2", "newstring" `
-replace "string3", "newstring" | Set-Content $file
# Write all the original times back.
$file.creationtime=$filecreate
$file.lastaccesstime=$fileaccess
$file.lastwritetime=$filewrite
# Verify the changes... Should not be required but it is what you were doing.
$filecreate=$file.creationtime
$fileaccess=$file.lastaccesstime
$filewrite=$file.lastwritetime
"$file,UPDATED Created: $filecreate,UPDATED Accessed: $fileaccess,UPDATED Modified: $filewrite"
} | Set-Content $output
Not tested but should be fine.
Depending on what you replacements are actually like you could probably save some time there as well. Test first before running in production obviously.
I remove the counter you had since it appeared nowhere in the code.
Your logging could easily be csv based since you have all the object ready to go but I just want to be sure we are one the right track before we go to far.
My Old Bat file
Copy F:\File.hdr+F:*.csv F:\FinalOutput.csv
the HDR file is a single entry file that has only header data for the CSV files
Is there a way to perform this in PowerShell (to combine all the CSV files into a single file)?
Here is my powershell script that doesn't work
$CSVFolder = 'F:\Input\';
$OutputFile = 'F:\Output\NewOutput.csv';
$CSV= #();
Get-ChildItem -Path $CSVFolder -Filter *.inv | ForEach-Object {
$CSV += #(Import-Csv -Path $CSVFolder\$_)
}
$CSVHeader = Import-Csv 'F:\Input\Headings.hdr'
$CSV = $CSVHeader + $CSV
$CSV | Export-Csv -Path $OutputFile -NoTypeInformation -Force;
I get the list of FileNames that are exported and not the content of the Files.
The script is also modifying the date/time stamp on my INV files. It shouldn't be doing that.
You can skip the whole CSV bit if you just append the files as you would before.
Something like this should work:
# First we create the new file and add the header.
get-content $headerfile | set-content $outputfile
# Then we get the input files, read them out with get-content
# and append them to the output file (add-content).
get-childitem -path $csvfolder *.inv | get-content | add-content $outputfile
The CSV commandlets are handy if you want to be processing the CSV data in your script, but in your case simply appending the files will do the trick. Not bothering with the CSV conversion will be a lot faster as Powershell doesn't have to parse the CSV lines and create PS-objects. It's really fast with pure text though.
Another trick here is how the get-content and add-content are used in the pipeline. Since they are aware of the pipeline you can pass in file objects without having to use a foreach loop. This makes your statements a lot shorter.
How about:
get-childitem *.inv | foreach-object {
import-csv $_ -header (get-content Headings.hdr)
} | export-csv NewOutput.csv -notypeinformation
I have five .sql files and know the name of each file. For this example, call them one.sql, two.sql, three.sql, four.sql and five.sql. I want to append the text of all files and create one file called master.sql. How do I do this in PowerShell? Feel free to post multiple answers to this problem because I am sure there are several ways to do this.
My attempt does not work and creates a file with several hundred thousand lines.
PS C:\sql> get-content '.\one.sql' | get-content '.\two.sql' | get-content '.\three.sql' | get-content '.\four.sql' | get-content '.\five.sql' | out-file -encoding UNICODE master.sql
Get-Content one.sql,two.sql,three.sql,four.sql,five.sql > master.sql
Note that > is equivalent to Out-File -Encoding Unicode. I only tend to use Out-File when I need to specify a different encoding.
There are some good answers here but if you have a whole lot of files and maybe you don't know all of the names this is what I came up with:
$vara = get-childitem -name "path"
$varb = foreach ($a in $vara) {gc "path\$a"}
example
$vara = get-childitem -name "c:\users\test"
$varb = foreach ($a in $vara) {gc "c:\users\test\$a"}
You can obviously pipe this directly into | add-content or whatever but I like to capture in variables so I can manipulate later on.
See if this works better
get-childitem "one.sql","two.sql","three.sql","four.sql","five.sql" | get-content | out-file -encoding UNICODE master.sql
I needed something similar, Chris Berry's post helped, but I think this is more efficient:
gci -name "*PathToFiles*" | gc > master.sql
The first part gci -name "*PathToFiles*" gets you your file list. This can be done with wildcards to just get your .sql files i.e. gci -name "\\share\folder\*.sql"
Then pipes to Get-Content and redirects the output to your master.sql file. As noted by Kieth Hill, you can use Out-File in place of > to better control your output if needed.
I think logical way of solving this is to use Add-Content
$files = Get-ChildItem '.\one.sql', '.\two.sql', '.\three.sql', '.\four.sql', '.\five.sql'
$files | foreach { Get-Content $_ | Add-Content '.\master.sql' -encoding UNICODE }
hovewer Get-Content is usually very slow when reading multiple very large files. If its your case this article could help: http://keithhill.spaces.live.com/blog/cns!5A8D2641E0963A97!756.entry
What about:
Get-Content .\one.sql,.\two.sql,.\three.sql,.\four.sql,.\five.sql | Set-Content .\master.sql
Here is how I do concatenate sql files from the Sql folder:
# Set the current location of the script to use relative path
Set-Location $PSScriptRoot
# Concatenate all the sql files
$concatSql = Get-Content -Path .\Sql\*.sql
# Write/overwrite sql to single file
Add-Content -Path concatFile.sql -Value $concatSql