I have a script that is using Add-Content to log progress. The log files are written to a network share, and on some networks I have gotten stream not readable errors. Rare, but often enough to want to address the issue.
I found [this thread][1] that seems to offer an answer. And I initially implemented this loop
$isWritten = $false
do {
try {
Add-Content -Path $csv_file -Value $newline -ErrorAction Stop
$isWritten = $true
}
catch {
}
} until ( $isWritten )
but I added a 1 second wait between tries, and limited myself to 20 tries. I figured no network could be such crap that it would timeout for longer than that. But on one network I still have problems, so I bumped the count to 60, and STILL have failures to write to the log. I tried [System.IO.File]::AppendAllText($file, $line) and that seems to solve all the timeouts, at least in 20 some odd tries it hasn't failed, where before I would get 1 or two failures in 10 tries. But the formatting is off, likely I need to set the encoding.
But more importantly, I wonder what is actually the SOURCE of the issue in Add-Content, and why does [System.IO.File]::AppendAllText() not have the issue, and is this a sign of potentially other problems with the network, or with the machines at this one location? Or just a bug in PowerShell that I need to work around. FWIW, it's PS 5.1 on Windows 10 21H2.
Also, FWIW, the logs can get to a few hundred lines long, but I often see the error in the first 10 lines.
[1]: add-content produces stream not readable
Related
I have been trying to come up with a better solution to error management than what I do now. Current practice is to test as much as possible to avoid errors, then try/catch and log exceptions. The problem is that that only works when I KNOW there's a possibility of an exception. What I have been trying to find is a way to have some sort of generic exception trap, that logs those exceptions with enough detail to actually troubleshoot.
What I have come up with is this
$errorLog = "$env:temp\errors.log"
# main
&{
# productive code here
} 2>> $errorLog
trap {
Out-File -InputObject $psItem.Exception.Message -FilePath $errorLog -Append
Out-File -InputObject $psItem.ScriptStackTrace -FilePath $errorLog -Append
Out-File -InputObject '' -FilePath $errorLog -Append
}
if (([System.IO.FileInfo]::new($errorLog)).Length -eq 0) {
Remove-Item $pxToolsErrorLog
} else {
Write-Host 'Errors'
}
This
&{
}
defines the main code block, and 2>> $pxToolErrorLog redirects standard errors in that code block to the log file.
The Trap outputs unhandled exceptions to the same log.
And the final conditional looks to see if I logged any errors, deletes the log if not, and let's me know if I did.
I have only played with this in a test scenario, but I am looking to implement it with a utility I am working on now, and I wonder if there is something I am missing that argues against this approach? Mostly because I am self taught, and regularly do silly things that I later discover are a bad idea, and I have never seen this approach or anything like it mentioned in the MANY blog posts and SO threads and such that I have read on the topic. It seems unlikely to me that I really did blunder in to something both new and better. So, does anyone see anything in this simple example that screams "No!!!!" ? Or something that makes the Spidey sense tingle? Or even just a bit of a code smell I am missing? Other than "You really SHOULD know what might cause an exception!" because I know that, and I am trying to learn that, but I want something a bit more robust to help in the meantime. I should mention that the ultimate goal is to use it in a program that will have MANY thousands of lines of code, and also use classes over functions to the extent possible, in no small part because I am sick and tired of chasing pipeline pollution issues and would prefer the rigor of types. So, any issues related to classes and scaling are especially pertinent.
Also, to clarify, I would STILL do things like test to see if a file exists before doing anything with that file, and also try/catch wherever I know there's potential for exceptions. I am just looking to get a log, with line numbers and other details, for anything I miss. Then I can address those issues with tests or try/catch. But that does raise a question in my mind, could this be a viable option for development, but SHOULD be removed for production code? Or is it safe/performant enough to keep in production code, with something more elegant than just dumping Errors to the console if errors occurred?
I have many scheduled jobs running on a server in our environment, but this morning a monitoring team contacted me about a service account profile eating up all the disk space. I dug around and found logs in C:\Users\service_account\AppData\Local\Microsoft\Office365\Powershell directory, but 3 of them on different dates were over 10 GB in size.
I narrowed it down to one script we have the checks the AD dir sync every hour to make sure it doesn't take longer than an hour, but I'm not sure what the best practice is for this? Also, most logs in this location are only 4 kb in size and indicate typical tracking like, connection times, cmdlets initializing, etc.
I'm not sure how to open such a huge file to see what may be the problem. Has anyone else run into this kind of thing? I can't find much on line. Also, I do run Remove-pssession * to close my sessions. Thx in advance...
You have a couple of options.
Read your file from disk one line or groups of lines at a time, work with each line, and then write each line back out to disk., if needed.
Instead of caching the entire file in RAM by using plain Get-Content, you’re reading it off disk a bit at a time. The below represents a very simplistic example of how to do this.
$file = New-Object System.IO.StreamReader -Arg "test.txt"
while ($line = $file.ReadLine()) {
# $line has your line
}
$file.close()
Or using ...
Get-Content -Readcount 100
... to process chunks of lines at a time, the above will give you arrays of 100 lines each.
There are recent posts on this forum regarding folks working with large files and PoSH and having to resort to using th e System.IO.StreamReader / System.IO.StreamWriter .Net aproach to handle the use case.
For example:
Unable to Find and Replace a string in 2GB XML file using Powershell
But there are lots more on this forum, just search for them using [powershell] large file
I am recovering files from a hard drive wherein some number of the files are unreadable. I'm unable to change the hardware level timeout / ERC, and it's extremely difficult to work around when I have several hundred thousand files, any tens of thousands of which might be unreadable.
The data issues were the result of a controller failure. Buying a matching drive (all the way down), I've been able to access the drive, and can copy huge swaths of it without issues. However, there are unreadable files dotted throughout the drive that, when accessed, will cause the SATA bus to hang. I've used various resumable file copy applications like robocopy, RichCopy, and a dozen others, but they all have the same issue. They have a RETRY count that is based on actually getting an error reported from the drive. The issue is that the drive is taking an extremely long time to report the error, and this means that a single file may take up to an hour to fail officially. I know how fast each file SHOULD be, so I'd like to build a powershell CMDLET or similar that will allow me to pass in a source and destination file name, and have it try to copy the file. If, after 5 seconds, the file hasn't copied (or if it has - this can be a dumb process), I'd like it to quit. I'll write a script that fires off each copy process individually, waiting for the process before it to finish, but I'm so far unable to find a good way of putting a time limit on the process.
Any suggestions you might have would be greatly appreciated!
Edit: I would be happy with spawning a Copy-Item in a new thread, with a new PID, then counting down, then killing that PID. I'm just a novice at PowerShell, and have seen so many conflicting methods for imposing timers that I'm lost on what the best practices way would be.
Edit 2: Please note that applications like robocopy will utterly hang when encountering the bad regions of the disk. These are not simple hangs, but bus hangs that windows will try to preserve in order to not lose data. In these instances task manager is unable to kill the process, but Process Explorer IS. I'm not sure what the difference in methodology is, but regardless, it seems relevant.
I'd say the canonical way of doing things like this in PowerShell are background jobs.
$timeout = 300 # seconds
$job = Start-Job -ScriptBlock { Copy-Item ... }
Wait-Job -Job $job -Timeout $timeout
Stop-Job -Job $job
Receive-Job -Job $job
Remove-Job -Job $job
Replace Copy-Item inside the scriptblock with whatever command you want to run. Beware though, that all variables you want to use inside the scriptblock must be either defined inside the scriptblock, passed in via the -ArgumentList parameter, or prefixed with the using: scope qualifier.
An alternative to Wait-Job would be a loop that waits until the job is completed or the timeout is reached:
$timeout = (Get-Date).AddMinutes(5)
do {
Start-Sleep -Milliseconds 100
} while ($job.State -eq 'Running' -and (Get-Date) -lt $timeout)
I have a script I'm using to loop through a bunch of domains and get dates from whois.exe. This works line-by-line, but when run as a script, it'll freeze. Here is where it gets stuck:
ForEach ($domain in $domains)
{
$domainname = $domain.Name
Write-Host "Processing $domainname..."
# WhoIsCL responds with different information depending on if it's a .org or something else.
if($domainname -like "*.org" -and $domainname)
{
$date = .\WhoIs.exe -v "$domainname" | Select-String -Pattern "Registry Expiry Date: " -AllMatches
Write-Host "Domain is a .org" -ForegroundColor "Yellow"
When I CTRL+C to cancel the command, I can verify that $domain is the correct variable. I can then write this:
if($domainname -like "*.org" -and $domainname)
{
"Test"
}
... and "Test" appears in the command line. I then run:
$date = .\WhoIs.exe -v "$domainname" | Select-String -Pattern "Registry Expiry Date: " -AllMatches
Upon checking the date, it comes out right and I get the appropriate date. Given it freezes right as it says "Processing $domainname..." and right before "Domain is a .org", I can only assume WhoIs.exe is freezing. So, why does this happen as the script is being run, but not directly from the Powershell window?
Lastly, I did a final test by simply copying and pasting the entire script into a Powershell window (which is just silly, but it appears to function) and get the same result. It freezes at whois.exe.
My best guess is that whois.exe needs to be run differently to be reliable in Powershell in my for-loop. However, I don't seem to have a way to test using it in a Start-Process and get string output.
Anyways, advise would be great. I've definitely hit a wall.
Thanks!
If your script is running through lots of domains, it could be that you're being throttled. Here is a quote from the Nominet AUP:
The maximum query rate is 5 queries per second with a maximum of 1,000
queries per rolling 24 hours. If you exceed the query limits a block
will be imposed. For further details regarding blocks please see the
detailed instructions for use page. These limits are not per IP
address, they are per user.
http://registrars.nominet.org.uk/registration-and-domain-management/acceptable-use-policy
Different registrars may behave differently, but I'd expect some sort of rate limit. This would explain why a script (with high volume) behaves differently to ad-hoc manual lookups.
Proposed solution from the comments below is to add Start-Sleep -Seconds 1 to the loop between each Whois lookup.
I want to monitor a log file which is constantly being added to (every few seconds) over the course of a 2 hour period. I am currently using Get-Content file.txt –Wait which displays the content to the screen and allows me to see what’s being added to the file but I need to take this a step further and actually watch for specific messages and if something I’m looking for in the log file appears, then do something. Initially I used a .net file reader with a for loop as shown below
try {
for(;;) {
$line = $log_reader.ReadLine()
if ($line -match "something")
{
Write-Host "we have a match"
Break
}
The issue with this however is that it was causing the process that is generating the log file to fall over - it throws an error (because another process is using the log file it’s creating – I thought this was odd because I assumed the .net stream reader would just be ‘reading’ the file). I don’t have any control over the process which is generating the log file so I don’t know what it’s doing exactly (I’m guessing it has the file in read/write mode with some kind of lock which gets upset when I try to read the file using a .net stream reader). Doing a Get-Content on the file doesn’t seem to cause this issue however.
The question is, how can I use something like Get-Content (or another process) to monitor the log file but move onto another part of the script if a message I’m looking for in the log appears?
If you constantly want to monitor the log file and catch the desired pattern as soon as it appears:
while(!(Select-String -path 'c:\log.txt' -pattern 'something' -quiet)){};
## when finds the pattern, comes out of the loop and proceed to next part of the script
Write-host 'we have a match'
You may want to change the path of the file to yours.
Though this would work, there will be a lot of processing that your computer have to do, since the while loop is a constant loop. If you can afford to introduce some delay, like if it is OK to find the error 30 sec or whatever is your threshold, after it appeared, then you can consider introducing sleep:
while(!(Select-String -path 'c:\log.txt' -pattern 'something' -quiet)){start-sleep -seconds 30};
## when finds the pattern, comes out of the loop and proceed to next part of the script
Write-host 'we have a match'
You can write a small logic to terminate the script after two hours, otherwise it would become an infinite loop, if 'something' doesn't gets written at all in the log file.
Edit 1:
If you want to print the new lines at the console, you can try manipulating a bit like:
while(!(Select-String -path 'c:\log.txt' -pattern 'something' -quiet))
{
$a = get-content 'c:\log.txt'
if(($a.count) -gt $b )
{
$a[$b..($a.count)]
}
$b = ($a.count)
start-sleep -Seconds 30
}
## To print the line containing the pattern when while loop exited
(Get-content 'c:\log.txt')[-1]