Read Invoke-WebRequest line by line - powershell

Question

I'm trying to keep a central list of log file locations where my log file cleanup script can grab the most up to date list.
$logpaths = (Invoke-WebRequest -UseBasicParsing -Uri 'http://10.7.58.99/logpaths.txt').Content
foreach($logpath in $logpaths)
{
"line"
$logpath
}
My script was sort of working but I was seeing some strange behavior so when I broke it down I found that within the foreach loop it just loops once and dumps the entire contents.
If I download the file the a text file on the local machine I can then use [System.IO.File]::ReadLines and it steps through perfectly. However, I don't want to download the file each time I run it or store it on the local server at all for that matter. How can I step through the content of Invoke-WebRequest line by line?

Answer 1

Based on this example from the .NET docs, you could read a response stream line-by-line like this, which should have better performance.
$url = 'http://10.7.58.99/logpaths.txt'
& {
$myHttpWebRequest = [System.Net.WebRequest]::Create($url)
$myHttpWebResponse = $myHttpWebRequest.GetResponse()
$receiveStream = $myHttpWebResponse.GetResponseStream()
$encode = [System.Text.Encoding]::GetEncoding("utf-8")
$readStream = [System.IO.StreamReader]::new($receiveStream, $encode)
while (-not $readStream.EndOfStream) {
$readStream.ReadLine()
}
$myHttpWebResponse.Close()
$readStream.Close()
} | foreach {
$logPath = $_
}
You might want to turn this into a nice little function. Let me know if you need help.