Issue creating a script with dual logic in deeming a website online - powershell

Question

I am currently trying to create a script that allows me to check multiple web url's in order to see if they are online and active. My company has multiple servers with different environments active (Production, Staging, Development etc.) I need a script that can check all the environments URL's and tell me whether or not they are online each and every morning so I can be ahead of the game in addressing any servers or websites being down.
My issue however is I can't solely base the logic strictly on an HTTP code to deem the site online or not, some of our websites may be online from an HTTP standpoint but have components or webparts of the site that is down displaying an error message on the page.
I am having trouble coming up with a script that can not only check the HTTP status as well as scan the page and parse out any error messages and then write to host based on both pieces of logic whether or not the site is "Online" or "Down"
Here is what I have so far, you will notice it does not include anything regarding parse for key words as I don't know how to implement...
#Lower Environments Checklist Automated Script
Write-Host Report generated at (Get-date)
write-host("Lower Environments Status Check");
$msg = ""
$array = get-content C:\LowerEnvChecklist\appurls.txt
$log = "C:\LowerEnvChecklist\lowerenvironmentslog.txt"
write-host("Checking appurls.txt...One moment please.");
("`n--------------------------------------------------------------------------- ") | out-file $log -Append
Get-Date | Out-File $log -Append
("`n***Checking Links***") | out-file $log -Append
("`n") | out-file $log -Append
for ($i=0; $i -lt $array.length; $i++) {
$HTTP_Status = -1
$HTTP_Request = [System.Net.WebRequest]::Create($array[$i])
$HTTP_Request.Timeout =60000
$HTTP_Response = $HTTP_Request.GetResponse()
$HTTP_Status = [int]$HTTP_Response.StatusCode
If ($HTTP_Status -eq 200) {
$msg = $array[$i] + " is ONLINE!"
}
Else {
$msg = $array[$i] + " may be DOWN, please check!"
}
$HTTP_Response.Close()
$msg | Out-File $log -Append -width 120
write-host $msg
}
("`n") | out-file $log -Append
("`n***Lower Environments Checklist Completed***") | out-file $log -Append
write-host("Lower Environments Checklist Completed");
appurls.txt just contains the internal URLs I need checked FYI.
Any help would be much appreciated! Thanks.

Answer 1

Here is something to at least give you an idea what to do. Need to capture the website data in order to parse it. Then we run a regex query against that which is built from an array of strings. Those strings are texts that might be seen on a page that is not working.
# build a regex query of error strings to match against.
$errorTexts = "error has occurred","Oops","Unable to display widget data","unexpected error occurred","temporarily unavailable"
$regex = ($errorTexts | ForEach-Object{[regex]::Escape($_)}) -join "|"
# Other preproccessing would go here
# Loop through each element of the array
ForEach($target in $array){
# Erase results for the next pass in case of error.
$result, $response, $stream, $page = $null
# Navigate to the website.
$result = [System.Net.WebRequest]::Create($target)
$response = $result.GetResponse()
$stream = [System.IO.StreamReader]$response.GetResponseStream()
$page = $stream.ReadToEnd()
# Determine if the page is truly up based on the information above.
If($response.StatusCode -eq 200){
# While the page might have rendered need to determine there are no errors present
if($page -notmatch $regex){
$msg = "$target is online!"
} else {
$msg = "$target may be DOWN, please check!"
}
} else {
$msg = "$target may be DOWN, please check!"
}
# Log Result
$msg | Out-File $log -Append -width 120
# Close the connection
$response.Close()
}
# Other postproccessing would go here
I wanted to show what a here-string looked like to replace some of your out-file repetition. Your log file header used to be several lines of this. I have reduced it to one.
#"
---------------------------------------------------------------------------
$(Get-Date)
***Checking Links***
"# | Out-File $log -Append
Also consider CodeReview.SE for critiquing working code. There are other areas which could in theory be improved but are out of scope for this question.