How to pull a range of failed services from a remote server after a reboot - powershell

Caveat: Without spiking the cpu while a Get-WmiObject call parses the whole event log to match my specified filter.
Situation: I am working on a script that remotely runs some checks, then reboots a pc. I want it to check the health once the server reboots (after sleeping for some time) to make sure services that were supposed to start did. I've been running into "Automatic" services that start and then shut down (as intended) but then my current version picks them up as failed if they've already run. It was suggested that I check the event log for "Service Control Manager" errors, and report on those, the only problem now is that with the below script, we have servers who's event log can range anywhere from 20K to several hundred thousand events, and on a 2k server with 20K, this takes roughly 20 seconds to complete, and the cpu pegs near 100% while it's running.
I'm still learning powershell/wmi, so any advice would be appreciated.
function Check_Startup_Events {
BEGIN {
$time = [System.Management.ManagementDateTimeConverter]::ToDmtfDateTime((Get-Date).AddMinutes(-15))
}
PROCESS {
$results = Get-WmiObject Win32_NTLogEvent -computername $_ -Filter "LogFile='System' and SourceName='Service Control Manager' and TimeGenerated>='$time' and EventType=1" |
Format-Table -Autosize EventCode, Message
$results
}
}

$results = Get-EventLog -ComputerName w2kserver -LogName System -After $time
foreach ($result in $results){
if ($result.Source -eq "Service Control Manager" -and $result.EntryType -eq "Error"){
Write-Host $_.Description}}
I ran this against a 60k big event log on a W2K server in our environment. It takes a while to run but runs locally and does not tax the server. Not sure how you would want to output the data but I think Get-EventLog will do what you want.

Related

Powershell Script cycles through machines but hangs if one loses network temporarily

I have a powershell script that parses a txt file which is full of machine names, then one by one, it creates a session to the system, runs a few commands, and moves to the next system. The script usually take about 10-30 seconds to run on each system depending on the case encountered in the script.
Once in a while the system that is currently being checked will lose the network connection for some various reason. When this happens the console starts writing yellow warning messages about attempting to reconnect for 4 minutes and then disconnects the session when it cannot reconnect.
Even if it establishes the connection again within the 4 minutes, it doesn't do anything after that, it's like the script just freezes. It won't move on to the next system and it doesn't stop the script, I have to manually stop it, or if i manually run the script, i can hit control+c to break out of the current loop, and it then moves on to the next machine in the list.
Is there any way to break out of the current loop if a warning is encountered so it can move on to the next machine? That would be my ideal solution. thanks!
Script is simple..
foreach($server in Get-Content .\machines.txt) {
if($server -match $regex){
invoke-command $server -ErrorAction SilentlyContinue -ScriptBlock{
command1
command2
command3
}
}
this is what happens
PS C:\temp> .\script.ps1
machine1
machine2
machine3
machine4
machine5
WARNING: The network connection to machine5 has been interrupted. Attempting to reconnect for up to 4 minutes...
WARNING: Attempting to reconnect to machine5 ...
WARNING: Attempting to reconnect to machine5 ...
WARNING: Attempting to reconnect to machine5 ...
WARNING: The network connection to machine5 has been restored.
But it never goes on to machine6
When i work remotely with multiple machines i usually start the processes on the machines in parallel. So i have less impact when single machines are timing out. I use powershell 7 ForEach-Object -Parallel Feature for this https://devblogs.microsoft.com/powershell/powershell-foreach-object-parallel-
feature/
Try something like this:
$Credential=Get-Credential
#all Necessary parameters must be in the Object i give to ForEach Object
$myHosts = #(
#Hosts i want to connect to with values i want to use in the loop
#{Name="probook";cred=$Credential;param1="one_1";param2="two_1"}
#{Name="probook";cred=$Credential;param1="one_2";param2="two_2"}
)
$var1="one"
$var2="two"
$myHosts | ForEach-Object -Parallel {
#Variables outside of this "Parallel" Loop are not available. Because this is startet as separate SubProcess
#All Values come from the Object i piped in the ForEach-Object
$myHost=$_
#This is written to your local Shell
Write-Host ("Computer: "+ $env:Computername)
Write-Host $myHost.param1
Write-Host $myHost.param2
Write-Host $myHost.cred.UserName
Invoke-Command -ComputerName $myHost.Name -Credential $myHost.cred -ArgumentList #($myHost.param1,$myHost.param2) -ScriptBlock {
#Variables outside of of this Invoke Command Script Block are not available because this is a new Remote-Shell on the remote Host
#Parameters in Ordner of -Argument List
param($param1,$param2)
#Do your things on the Remote-Host here
#This is not Visbible -> it is only written on the "remote Shell"
Write-Host $env:Computername
#Here you get Back Values from the remote Shell
$env:Computername
$param1
$param2
}
} -ThrottleLimit 5
Hmm his is indeed a Problem.
You could experiment with:
Start-Job
(https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/start-job?view=powershell-7.1)
Get-Job (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/get-job?view=powershell-7.1)
Receive-Job (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/receive-job?view=powershell-7.1)
So you have more control what the processes do.
You start Background Jobs with Start-Job. Start-Job deliveres Job Objects Back -> save them in a array or variables
With Get-Job you see all Jobs currently Running
with Receive-Job you get back the output from a Job so far. You can use receive-Job to get back all PSObjects a Background Job has written.
Cannot explain in Detail, but this woul be another try i would do for this job.

Determining when machine is in good state for Powershell Remoting?

Update - the original question claimed that I was able to successfully perform an Invoke-Command and then shortly after was unable to; I thought it was due to processes going on during login after a windows upgrade.
It turns out the PC was actually starting, running a quick batch/cmd file, and then restarting. This is what was leading to being able to do PS Remoting and then suddenly not. The restart was quick enough after first boot that I didn't realize it was happening. Sorry for the bad question.
For the curious, the machine was restarting because of a remnant of the Microsoft Deployment Toolkit in-place upgrade process. The way MDT completes its task-sequence post-upgrade is problematic for many reasons, and now I've got another to count.
Old details (no longer relevant, with incorrect assumption that machine was not restarting after first successful Invoke-Command):
I'm automating various things with VMs in Hyper-V using powershell and powershell remoting. I'll start up a VM and then want to run some commands on it via powershell.
I'm struggling with determining when I can safely start running the remote commands via things like Invoke-Command. I can't start immediately as I need to let the machine start up.
Right now I poll the VM with a one second sleep between calls until the following function returns $true:
function VMIsReady {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$True)][object]$VM
)
$heartbeat = $vm.Heartbeat
Write-Host "vm heartbeat is $heartbeat"
if (($heartbeat -eq 'OkApplicationsHealthy') -or ($heartbeat -eq 'OkApplicationsUnknown'))
{
try
{
Invoke-Command -VMName $vm.Name -Credential $(GetVMCredentials) {$env:computername} | out-null
}
catch [System.Management.Automation.RuntimeException]
{
Write-Host 'Caught expected automation runtime exception'
return $false
}
Write-Host 'remoting ready'
return $true
}
}
This usually works well; however, after a windows upgrade has happened, there are issues. I'll get Hyper-V remoting errors of various sorts even after VMIsReady returns $true.
These errors are happening while the VM is in the process of first user login after upgrade (Windows going through "Hi;We've got some updates for your PC;This might take several minutes-Don't turn off your PC). VMIsReady returns true right as this sequence starts - I imagine I probably should be waiting until the sequence is done, but I've no idea how to know when that is.
Is there a better way of determining when the machine is in a state where I can expect remoting to work without issue? Perhaps a way to tell when a user is fully logged on?
You can use Test-WSMan.
Of run a script on the invoke that will receive a response from the server.
[bool]$Response | Out-Null
try{
$Response = Invoke-Command -ComputerName Test-Computer -ScriptBlock {return $true}
}catch{
return $false
}
if ($Response -ne $true){
return $false
}else{
return $true
}

powershell takes a long time to close connection?

I'm trying to run this powershell script on several remote network servers, but it takes a long time to complete the task and move to the next one.
Here's a basic script that will query a single remote server and return the data I want in a fast manner, but takes a long time to complete the entire script.
Get-EventLog -ComputerName WebServer1 -LogName System -EntryType Error -After ((Get-Date).Date.AddDays(-1))
I was running it through the Powershell ISE and from the command line in Powershell and both have the same behavior.
Any help is appreciated!
Thanks
As for the reason: mjolinor outlined that - Get-EventLog parses the whole logfile, and then filters the output to meet your parameters. What we want to do right now is to parse the event log one by one from the newest entries while the condition is true.
$i = 0
$EventParser = do
{
Get-EventLog -ComputerName Localhost -LogName System -EntryType Error -Newest $i; $i++
}
while
(
(Get-EventLog -ComputerName Localhost -LogName System -EntryType Error -Newest $i | Select -Last 1).TimeWritten -ge ((Get-Date).Date.AddHours(-24))
)
$EventParser|Where-Object { $_.TimeWritten -ge ((Get-Date).Date.AddHours(-24)) } | Sort-Object -Property Index -Descending | Get-Unique -OnType
I have measured your command and the loop I have written - here is the output for my machine
For $EventParser:
TotalMilliseconds : 634,6669
For your command (I changed webserver1 to localhost)
TotalSeconds : 14,3049668
I hope the script above will do the trick to speed up log parsing.
PS.
I have ran into one thing that I couldn't figure out. As you see, I am actually filtering the output twice for, I believe, the same condition (!). If I remove the last-line Where-Object statement, I receive 35 entries for my machine and it lists output even from 4 days ago ?? (Normally I receive 19 entries both from mine and original command). I have had to, to my surprise, re-apply the filter to receive the desired scope. Can you assist with that in a comment or should I post a question on that?
EDIT:
Now the problem doesn't occur. Edit: I have tweaked the code a little bit, I noticed that in some cases it may return duplicated entries.
Best regards,
AlexP

PowerShell: get-eventlog takes too long to complete

I am trying to script the review of the past 30 days of the Application and System Event Logs on remote servers, looking for only Warnings, Errors, or Critical entries.
Borrowing from what I have found here and in other forums, I have come up with:
$Date = Get-Date
$Range = $Date.AddDays(-30)
$Range = $range.ToShortDateString();
$LogName = Read-Host "Which Log? (Application, System)"
$Server = Read-Host "Please Enter Server Name"
get-eventlog $LogName -ComputerName $Server -After $range | where {$_.EntryType -eq "Error" -or $_.EntryType -eq "Warning" -or $_.EntryType -eq "Critical"}
This seems to run fairly quickly, but then hangs several (5-10+) minutes before returning to a prompt, if it does....
Note: if I remove the code:
-After $range
I can simply break the output with ctrl-c and get on with my day, but I'd rather it run as intended and then stop...
So: Any ideas on how to eliminate this hang?
I am also open to ideas on how to make the code more elegant (and faster)!
And I wouldn't mind the script checking both the Application and System logs without having to run twice....
Using the -EntryType string-array parameter on Get-EventLog is much faster than retrieving the entire event log and then filtering with Where-Object
Try get-eventlog -Logname System -EntryType ("Error", "Warning")
However... if I put "Critical" in the -EntryType array, I get: The argument "Critical" does not belong to the set "Error,Information,FailureAudit,SuccessAudit,Warning" specified by the ValidateSet attribute. which makes me wonder whether you should be heeding the advice as listed in Get-Help Get-EventLog:
The cmdlets that contain the EventLog noun (the EventLog cmdlets) work
only on classic event logs. To get events from logs that use the
Windows Event Log technology in Windows Vista and later versions of
Windows, use Get-WinEvent.
Using Get-WinEvent instead, I think this is what you want:
Get-Winevent -FilterHashtable #{LogName="System","Application"; Level=1,2,3; startTime=$range}
That will check for events of level 1, 2, or 3 (Critical, Error, Warning, respectively), and search both the Application and System logs in the same invocation.
I found that for remote systems, I could query several systems at once faster than I could individually, using the same command, if I wrapped it into an Invoke-Command. This is my solution. The more systems, the more time it saved. YMMV
$command = {Get-EventLog -LogName Application -After (Get-Date).AddHours("-24")}
Invoke-Command -ComputerName "foo1","foo2","foo3","foo4" -ScriptBlock $command

Measure 'Idle' time between CTRL-ALT-DEL and user typing in password and loging on -Windows 7

Windows 7 has the built in 'Boot Performance Diagnostics' and judging by the numerous reboots i've done, it does generate every now and then a detailed log on the user's login process and possible slowness.
That is not good enough for what I'm after though.
I want to measure EVERY Boot on a given machine.
There is little information however available on how to force it, except fiddling with registry keys that are System Protected so you don't tamper with them.
Some of the information can be found in the eventlogs so i switched to tracing the eventid 12
$yesterday = (get-date) - (New-TimeSpan -day 2)
$startuplog= Get-WinEvent -FilterHashTable #{LogName='System'; ID=12;
StartTime=$yesterday} -ErrorAction SilentlyContinue
But does anyone know how one can measure when the system was ready (ctrl-alt-del) and when the user hit the enter button after typing in the password? Is there a flag that can be set to raise such an event in a (diagnostics) event log?
You can compare the power state timestamp to the "Last Interactive Logon" feature of AD DS. That feature requires a domain functional level (DFF) of Windows Server 2008 r2 to work and workstation infrastructure of windows vista or later. The "msDS-LastSuccessfulInteractiveLogonTime" attribute is what you want. It's the time stamp of the last successful interactive logon (ctrl+alt+del).
To enable Last Interactive Logon on your domain:
http://technet.microsoft.com/en-us/library/dd446680(v=ws.10).aspx
Command to query attribute:
$Computer = 'hostname'
Get-ADComputer -Filter "name -eq $Computer" -Properties * | Select msDS-LastSuccessfulInteractiveLogonTime
P.S. Try to get away from using "-ErrorAction". In it's place, use Try/Catch/Finally code blocks.
http://blogs.technet.com/b/heyscriptingguy/archive/2010/03/11/hey-scripting-guy-march-11-2010.aspx