Use .NET for fast read/write of large files - powershell

Question

i am trying to search through a number of large files and replace parts of the text, but i keep running into errors.
i tried this, but sometimes i'll get an 'out of memory' error in powershell
#region The Setup
$file = "C:\temp\168MBfile.txt"
$hash = #{
ham = 'bacon'
toast = 'pancakes'
}
#endregion The Setup
$obj = [System.IO.StreamReader]$file
$contents = $obj.ReadToEnd()
$obj.Close()
foreach ($key in $hash.Keys) {
$contents = $contents -replace [regex]::Escape($key), $hash[$key]
}
try {
$obj = [System.IO.StreamWriter]$file
$obj.Write($contents)
} finally {
if ($obj -ne $null) {
$obj.Close()
}
}
then i tried this (in the ISE), but it crashes with a popup message (sorry, don't have the error on hand) and tries to restart the ISE
$arraylist = New-Object System.Collections.ArrayList
$obj = [System.IO.StreamReader]$file
while (!$obj.EndOfStream) {
$line = $obj.ReadLine()
foreach ($key in $hash.Keys) {
$line = $line -replace [regex]::Escape($key), $hash[$key]
}
[void]$arraylist.Add($line)
}
$obj.Close()
$arraylist
and finally, i came across something like this, but i'm not sure how to use it properly, and i am not even sure if i am going about this the right way.
$sourcestream = [System.IO.File]::Open($file)
$newstream = [System.IO.File]::Create($file)
$sourcestream.Stream.CopyTo($newstream)
$sourcestream.Close()
any advice would be greatly appreciated.

Answer 1

You can start with a readcount of 1000 and tweak it based on the performance you get:
get-content textfile -Readcount 1000 |
foreach-object {do something} |
set-content textfile
or
(get-content textfile -Readcount 1000) -replace 'something','withsomething' |
set-content textfile