Trying to Use Power shell Script to test availability of website - powershell

I am trying to use a powershell script to test the availability of certain websites. I have a script here that writes "Site is OK" if the site returns a http 200 code. It should return "The Site may be down, please check!" If it returns any other code. I put in 'https://www.google.com/cas76' which should obviously return a 404 error however the script returns "Site is ok" How should I go about fixing my code so it returns "The Site may be down, please check!"
Tried putting in websites that would obviously not return a 200 code.
# First we create the request.
$HTTP_Request = [System.Net.WebRequest]::Create('https://www.google.com/cas76')
# We then get a response from the site.
$HTTP_Response = $HTTP_Request.GetResponse()
# We then get the HTTP code as an integer.
$HTTP_Status = [int]$HTTP_Response.StatusCode
If ($HTTP_Status -eq 200) {
Write-Host "Site is OK!"
}
Else {
Write-Host "The Site may be down, please check!"
}
# Finally, we clean up the http request by closing it.
$HTTP_Response.Close()
Code acknowledges that there is a 404 error
Exception calling "GetResponse" with "0" argument(s): "The remote server returned an error: (404) Not Found."
At C:\Users\TX394UT\Desktop\Web_Bot_Project\WebsiteMonitoring.ps1:6 char:1
+ $HTTP_Response = $HTTP_Request.GetResponse()
However, Site is OK! Prints on the console.

the way that the webclient handles returned errors is ... odd. [grin]
that message is treated as a non-terminating error and needs to be handled as such. the most obvious way is with Try/Catch/Finally. when capturing the exception message, the 404 StatusCode is converted to the text value for it - NotFound.
$TestUrl = 'https://www.google.com/cas76'
try
{
$Response = (Invoke-WebRequest -Uri $TestUrl -ErrorAction Stop).StatusCode
}
catch
{
$Response = $_.Exception.Response.StatusCode
}
$Response
output for the bad url = NotFound
output for a good url = 200

Related

Powershell script to determine if web page exists

I need a way to determine from a PS script if any web page is up or down, regardless of whether it first prompts for credentials. Even if the page requires that java is installedd or whatever other reason. The goal here is to determine that the page is there and it shouldn't matter whether it works properly or if it can be displayed. After all is said and done it should just tell me that site/page is UP or DOWN after executing the script with .\sitecheck.ps1 'https://trac.edgewall.org/login'
It'd also be nice if we could print why the page is down (like when you get a 401 error) and print the error message and status code (integer).
I'm trying to work off of this script which obviously doesn't work properly because I'm trying to find a solution:
# First we create the request.
$url = $args[0]
$HTTP_Request = [System.Net.WebRequest]::Create($url)
# We then get a response from the site.
$HTTP_Response = $HTTP_Request.GetResponse()
# We then get the HTTP code as an integer.
$HTTP_Status = [int]$HTTP_Response.StatusCode
If ($HTTP_Status -eq 200) {
Write-Host "Site is OK!"
}
Else {
Write-Host "The Site may be down, please check!"
}
# Finally, we clean up the http request by closing it.
If ($HTTP_Response -eq $null) { } Else { $HTTP_Response.Close()}
Someone responded with this answer to a similar question on this site:
"If the URL needs credentials, you need to add $HTTP_Request.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials. You need a Try..Catch around the $HTTP_Response = $HTTP_Request.GetResponse() line, and if that fails, $HTTP_Response will be null and so can't be closed because it's already null - like when you get a (404) Not Found, you will have no response and error will be You cannot call a method on a null-valued expression if you try to do .Close() on it."
Unfortunately I don't exactly know how to do that. Currently I'm getting the error below. Most of the actual error message is accurate since I haven't entered the correct credentials hence a 401 error code:
Exception calling "GetResponse" with "0" argument(s): "The remote
server returned an error: (401) Unauthorized." At
C:\Users\test\sitecheck.ps1:11 char:1
+ $HTTP_Response = $HTTP_Request.GetResponse()
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : WebException
Don't expect to receive a 200 because you haven't accessed the page yet. Look, I can even click on the hyperlink you posted here on StackOverflow: before accessing the page the banner ask for login (I haven't accessed the page yet)
Then, because I don't have the credentials what I receive is a 401 Unauthorized.
So what I suggest you to do is to check if Apache Subversion is up and running instead:
# First we create the request.
$url = $args[0]
$HTTP_Request = [System.Net.WebRequest]::Create('https://svn.edgewall.org')
# We then get a response from the site.
$HTTP_Response = $HTTP_Request.GetResponse()
# We then get the HTTP code as an integer.
$HTTP_Status = [int]$HTTP_Response.StatusCode
If ($HTTP_Status -eq 200) {
Write-Host "Site is OK!"
}
Else {
Write-Host "The Site may be down, please check!"
}
# Finally, we clean up the http request by closing it.
If ($HTTP_Response -eq $null) { } Else { $HTTP_Response.Close()}
**
EDIT:
**
After your comment I've found a solution for you here:
Paste this code in a .ps1 file and execute it like in picture:
$url = $args[0]
try {
$HttpWebResponse = $null;
$HttpWebRequest = [System.Net.HttpWebRequest]::Create($url);
$HttpWebResponse = $HttpWebRequest.GetResponse();
if ($HttpWebResponse) {
Write-Host -Object $HttpWebResponse.StatusCode.value__;
Write-Host -Object $HttpWebResponse.GetResponseHeader("X-Detailed-Error");
}
}
catch {
$ErrorMessage = $Error[0].Exception.ErrorRecord.Exception.Message;
$Matched = ($ErrorMessage -match '[0-9]{3}')
if ($Matched) {
Write-Host -Object ('HTTP status code was {0} ({1})' -f $HttpStatusCode, $matches.0);
}
else {
Write-Host -Object $ErrorMessage;
}
$HttpWebResponse = $Error[0].Exception.InnerException.Response;
$HttpWebResponse.GetResponseHeader("X-Detailed-Error");
}
This script will always print you the status code of the page. So now when you target https://trac.edgewall.org/login it will return you 401 which is the right status code.
You can see a list of all the error codes here: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes

Can I use If/Else statement in Pester?

I want to test a function when it succeeds but also whether it throws the correct error when it fails.
I wrote a function Test-URLconnection which should test whether a URL is accessible, otherwise it should throw an error.
Describe 'Test-URLconnection'{
$Error.Clear()
$countCases = #(
#{'url' = 'www.google.com'}
#{'url' = 'www.facebook.com'}
#{'url' = 'www.bbc.com'}
)
It "The URL status should be confirmed." -TestCases $countCases {
param($url)
if ([bool](test-URLconnection -URL $url -ErrorAction SilentlyContinue)) {
test-URLconnection -URL $url | Should -Be "$url = OK"
}
else {
$Error[0].Exception.Message | Should -Be "$url cannot be accessed."
}
}
}
I expected the two tests to pass because, even though Facebook cannot be accessed via Invoke-WebRequest (the command which I use in Test-URLconnection) it should have been caught with the else-statement.
This is the console output:
Describing Test-URLconnection
[+] The URL status should be confirmed. 319ms
[-] The URL status should be confirmed. 278ms
HttpException: www.facebook.com cannot be accessed.
at test-URLconnection<Begin>, <No file>: line 51
at <ScriptBlock>, PATH: line 15
[+] The URL status should be confirmed. 556ms
Tests completed in 1.54s
Tests Passed: 2, Failed: 1, Skipped: 0, Pending: 0, Inconclusive: 0
Can one use an if/else-statement with Pester?
I followed the advise from Mark Wragg and created a separate Context for testing errors with the following code.
Context "Test cases which fail" {
$Error.Clear()
$countCases = #(
#{'url' = 'www.asdfasf.com'}
#{'url' = 'www.facebook.com'}
)
It "The correct Error message should be thrown" -TestCases $countCases{
param($url)
{ test-URLconnection -URL $url } | Should -Throw -ExceptionType ([System.Web.HttpException])
}
}
A small hint, I was quite struggling with the ExceptionType catch. Keep in mind to wrap the function in curly braces. Otherwise, it was failing for me.

Get HTTP response code when response code does not equal 200

Im trying to write a script which loops trough a CSV File with a bunch of links in it. Then i want to check with Invoke-WebRequest if the page is reachable. This works fine if the HTTP response code is "200". But I also want the script to write out the response code when it is somewhat like 4xx, 3xx, 5xx etc. So far the response code variable is just empty.
The problem is that when ''Invoke-WebRequest'' returns not 200(thats kinda the point of the script) it generates an error instead of giving me a clean output with the HTTP Response Code.
My script so far:
$Links = Import-Csv -Path 'Path\to\file.csv' -Delimiter ";"
$user = Get-Credential
function Check-Link{
$name_de = $Link.title_de
$link_de = $Link.url_de
Write-Host $link_de
$HTTP_Response = (Invoke-WebRequest $Link.url_de -Credential $user -Method Head).StatusCode
if($HTTP_Response -ne 200){
Write-Host "The page $name_de is not reachable! Response Code: $HTTP_Response"
}
else{
Write-Host "The page $name_de is reachable!"
}
}
$DeadLinks
foreach($Link in $Links){
Check-Link
Start-Sleep 10
}
Now when the Response code does NOT equal 200 the output ist:
www.example.com
the page example.com is not reachable! Response Code:
The expected output would be:
www.example.com
the page example.com is not reachable! Response Code: 401

Powershell WebScraping and Terminating Exception Handling

I'm using powershell to check several different sites for data. If the site has no data for today, it will throw a NullReferenceException. I'd like for the script to output a message that there is no data, then continue on to the other sites without halting.
In Java, I can simply just try/catch/finally, but Powershell isn't acting as nicely.
try {
$webRequest = Invoke-WebRequest -URI "http://###.##.###.##:####/abcd.aspx?"
} catch [System.NullReferenceException]{
Write-Host "There is no data"
}
The full error displays in console, and the Write-Host never actually appears.
try {
$webRequest = Invoke-WebRequest -URI "http://host/link" -erroraction stop
}
catch [System.NullReferenceException]{
Write-Host "There is no data"
}
Powershell distinguishes between terminating and non terminating errors, for catch to work, you need the error to be terminating. https://blogs.technet.microsoft.com/heyscriptingguy/2015/09/16/understanding-non-terminating-errors-in-powershell/
UPD: to get the type of exception, after you get the error just do:
$Error[0].Exception.GetType().FullName
and you use that to catch that specific error after
to continue on specific error with invoke-webrequest you can do something like this:
try { Invoke-WebRequest "url" }
catch { $req = $_.Exception.Response.StatusCode.Value__}
if ($req -neq 404) { do stuff }
That's probably because the exception may not be a Null Reference exception. I Initially thought it to be a Non-Terminating error, but invoke-webrequest throws a terminating error.
In this case, you may simply try (without catching a specific exception type)
--Edited Per OP Comments--
try
{
Invoke-WebRequest -URI "http://doc/abcd.aspx?" -ErrorAction Stop
}
catch
{
if($_.Exception.GetType().FullName -eq "YouranticipatedException")
{
Write-Host ("Exception occured in Invoke-WebRequest.")
# You can also get the response code thru "$_.Exception.Response.StatusCode.Value__" if there is response to your webrequest
}

" The remote server returned an error: (401) Unauthorized "

I am trying to verify if my URLs get a response.
in other words, i am trying to check if the authentication has succeeded approaching the site.
I used:
$HTTP_Request = [System.Net.WebRequest]::Create('http://example.com')
$HTTP_Response = $HTTP_Request.GetResponse()
$HTTP_Status = [int]$HTTP_Response.StatusCode
If ($HTTP_Status -eq 200) {
Write-Host "Site is OK!"
} Else {
Write-Host "The Site may be down, please check!"
}
$HTTP_Response.Close()
and I got the response:
The remote server returned an error: (401) Unauthorized.
but after that i got:
site is ok
Does that mean it's ok? If not, what is wrong?
You are getting OK because you rerun your command and $HTTP_Response contains an object from a previous, successful run. Use try/catch combined with a regex to extract correct status code(and clean up your variables):
$HTTP_Response = $null
$HTTP_Request = [System.Net.WebRequest]::Create('http://example.com/')
try{
$HTTP_Response = $HTTP_Request.GetResponse()
$HTTP_Status = [int]$HTTP_Response.StatusCode
If ($HTTP_Status -eq 200) {
Write-Host "Site is OK!"
}
else{
Write-Host ("Site might be OK, status code:" + $HTTP_Status)
}
$HTTP_Response.Close()
}
catch{
$HTTP_Status = [regex]::matches($_.exception.message, "(?<=\()[\d]{3}").Value
Write-Host ("There was an error, status code:" + $HTTP_Status)
}
.Net HttpWebRequest.GetResponse() throws exceptions on non-OK status codes, you can read more about it here: .Net HttpWebRequest.GetResponse() raises exception when http status code 400 (bad request) is returned
I think the following happens:
You are trying to query a web page using GetResponse(). This fails, so the following Statements run with the values you set in a previous run. This leads to the Output you described.
I personally tend to use the invoke-webrequest in my scripts. This makes it easier to handle Errors, because it supports all Common Parameters like Erroraction and so on.