Powershell script to determine if web page exists - powershell

I need a way to determine from a PS script if any web page is up or down, regardless of whether it first prompts for credentials. Even if the page requires that java is installedd or whatever other reason. The goal here is to determine that the page is there and it shouldn't matter whether it works properly or if it can be displayed. After all is said and done it should just tell me that site/page is UP or DOWN after executing the script with .\sitecheck.ps1 'https://trac.edgewall.org/login'
It'd also be nice if we could print why the page is down (like when you get a 401 error) and print the error message and status code (integer).
I'm trying to work off of this script which obviously doesn't work properly because I'm trying to find a solution:
# First we create the request.
$url = $args[0]
$HTTP_Request = [System.Net.WebRequest]::Create($url)
# We then get a response from the site.
$HTTP_Response = $HTTP_Request.GetResponse()
# We then get the HTTP code as an integer.
$HTTP_Status = [int]$HTTP_Response.StatusCode
If ($HTTP_Status -eq 200) {
Write-Host "Site is OK!"
}
Else {
Write-Host "The Site may be down, please check!"
}
# Finally, we clean up the http request by closing it.
If ($HTTP_Response -eq $null) { } Else { $HTTP_Response.Close()}
Someone responded with this answer to a similar question on this site:
"If the URL needs credentials, you need to add $HTTP_Request.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials. You need a Try..Catch around the $HTTP_Response = $HTTP_Request.GetResponse() line, and if that fails, $HTTP_Response will be null and so can't be closed because it's already null - like when you get a (404) Not Found, you will have no response and error will be You cannot call a method on a null-valued expression if you try to do .Close() on it."
Unfortunately I don't exactly know how to do that. Currently I'm getting the error below. Most of the actual error message is accurate since I haven't entered the correct credentials hence a 401 error code:
Exception calling "GetResponse" with "0" argument(s): "The remote
server returned an error: (401) Unauthorized." At
C:\Users\test\sitecheck.ps1:11 char:1
+ $HTTP_Response = $HTTP_Request.GetResponse()
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : WebException

Don't expect to receive a 200 because you haven't accessed the page yet. Look, I can even click on the hyperlink you posted here on StackOverflow: before accessing the page the banner ask for login (I haven't accessed the page yet)
Then, because I don't have the credentials what I receive is a 401 Unauthorized.
So what I suggest you to do is to check if Apache Subversion is up and running instead:
# First we create the request.
$url = $args[0]
$HTTP_Request = [System.Net.WebRequest]::Create('https://svn.edgewall.org')
# We then get a response from the site.
$HTTP_Response = $HTTP_Request.GetResponse()
# We then get the HTTP code as an integer.
$HTTP_Status = [int]$HTTP_Response.StatusCode
If ($HTTP_Status -eq 200) {
Write-Host "Site is OK!"
}
Else {
Write-Host "The Site may be down, please check!"
}
# Finally, we clean up the http request by closing it.
If ($HTTP_Response -eq $null) { } Else { $HTTP_Response.Close()}
**
EDIT:
**
After your comment I've found a solution for you here:
Paste this code in a .ps1 file and execute it like in picture:
$url = $args[0]
try {
$HttpWebResponse = $null;
$HttpWebRequest = [System.Net.HttpWebRequest]::Create($url);
$HttpWebResponse = $HttpWebRequest.GetResponse();
if ($HttpWebResponse) {
Write-Host -Object $HttpWebResponse.StatusCode.value__;
Write-Host -Object $HttpWebResponse.GetResponseHeader("X-Detailed-Error");
}
}
catch {
$ErrorMessage = $Error[0].Exception.ErrorRecord.Exception.Message;
$Matched = ($ErrorMessage -match '[0-9]{3}')
if ($Matched) {
Write-Host -Object ('HTTP status code was {0} ({1})' -f $HttpStatusCode, $matches.0);
}
else {
Write-Host -Object $ErrorMessage;
}
$HttpWebResponse = $Error[0].Exception.InnerException.Response;
$HttpWebResponse.GetResponseHeader("X-Detailed-Error");
}
This script will always print you the status code of the page. So now when you target https://trac.edgewall.org/login it will return you 401 which is the right status code.
You can see a list of all the error codes here: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes

Related

Powershell capturing an error (ExtendedTypeSystemException)

I am trying to catch an error nicely, as I would do in Java. The program is similar to this:
try
{
New-Object System.DirectoryServices.DirectoryEntry($SearchString, $username, $pass)
Write-Ouput "Continue program"
}
catch
{
Write-Output "Some error"
}
The exception is never captured. In fact, the catch method executes when everything goes well.
From the different errors I can get, it doesn't look like an exception is raised:
format-default : The following exception occurred while retrieving member "distinguishedName": "The server is not operational.
"
+ CategoryInfo : NotSpecified: (:) [format-default], ExtendedTypeSystemException
+ FullyQualifiedErrorId : CatchFromBaseGetMember,Microsoft.PowerShell.Commands.FormatDefaultCommand
format-default : The following exception occurred while retrieving member "distinguishedName": "The user name or password is
incorrect.
"
+ CategoryInfo : NotSpecified: (:) [format-default], ExtendedTypeSystemException
+ FullyQualifiedErrorId : CatchFromBaseGetMember,Microsoft.PowerShell.Commands.FormatDefaultCommand
How can I capture these errors in a more user-friendly way?
Maybe in this case it is not possible.
After some testing, I think the problem is that no exception is raised for some reason, despite of what the error message says. So that is why I can't capture it. I have also tried to use getType, close and other methods but I get the same error:
The following exception occurred while retrieving member "GetType": "The user name or password is incorrect.
"
...
This solution does not seem to work either, same behaviour.
Question related.
need add 'erroraction'
try
{
New-Object System.DirectoryServices.DirectoryEntry($SearchString, $username, $pass) -ErrorAction Stop
Write-Ouput "Continue program"
}
catch
{
$Err = $error[0]
Write-Output $Err.Exception
}
You could also just default back to this...
If ($(New-Object System.DirectoryServices.DirectoryEntry($SearchString, $username, $pass)) -eq $false)
{Write-Verbose 'Continue program' -Verbose}
Else {Write-Warning -Message 'Something went wrong with your directory command'}
# Results
<#
WARNING: Something went wrong with your directory command
#>
... though like you try/catch is my preference. I am not in an ADDS environment to test the try/catch one at this time.
Know that you are not the only one stressed out by this. See:
Powershell DirectoryService object error neither caught nor trapped
Yet, after a bit of digging at my notes on ADDS (ADSI stuff from back in the day) in the past, here's the real deal on this namespace. The result is not really $true or $false, unlike System.DirectoryServices.AccountManagement; with what you are using, is provided with incorrect credentials you get the error. If the credentials are correct the returned object will contain a valid result, the result is Boolean output.
So, if you really want to stick with Try/Catch, do this instead:
try
{
$AuthResult = [bool](New-Object System.DirectoryServices.DirectoryEntry($SearchString, $username, $pass)).distinguishedName -ne 'False'
'Continue program'
}
catch {Write-Warning -Message 'Bad credentials passed'}
# Resuls
<#
WARNING: Bad credentials passed
#>
You of course, if so inclined you can trap error data and output as needed.
Yet, that is not much different than what the check provided in the If/Then. So, your choice.
I managed to do it with this solution:
$o = New-Object DirectoryServices.DirectoryEntry ($path, $user, $pass)
if ($path -eq 'LDAP://') {
'Computer is not member of a domain.' } elseif ($o.DistinguishedName) {
'Invalid user credentials.' } But if for some reason you must retrieve the actual error message, you can do it like this:
$o = New-Object DirectoryServices.DirectoryEntry ($path, $user, $pass)
try {
$o | Out-Default } catch {
$_.Exception.InnerException.Message }
This worked for me. The object will instantiate even though there might be an error that won't show up until you try to use it later. Using this, you can set a boolean to whether the distinguishedName was set or not:
$Entry = New-Object System.DirectoryServices.DirectoryEntry($SearchString, $username, $pass)
$succeeded = [bool]$Entry.distinguishedName
if ($succeeded)
{
Write-Ouput "Continue program"
}
else
{
Write-Output "Some error"
}

Trying to Use Power shell Script to test availability of website

I am trying to use a powershell script to test the availability of certain websites. I have a script here that writes "Site is OK" if the site returns a http 200 code. It should return "The Site may be down, please check!" If it returns any other code. I put in 'https://www.google.com/cas76' which should obviously return a 404 error however the script returns "Site is ok" How should I go about fixing my code so it returns "The Site may be down, please check!"
Tried putting in websites that would obviously not return a 200 code.
# First we create the request.
$HTTP_Request = [System.Net.WebRequest]::Create('https://www.google.com/cas76')
# We then get a response from the site.
$HTTP_Response = $HTTP_Request.GetResponse()
# We then get the HTTP code as an integer.
$HTTP_Status = [int]$HTTP_Response.StatusCode
If ($HTTP_Status -eq 200) {
Write-Host "Site is OK!"
}
Else {
Write-Host "The Site may be down, please check!"
}
# Finally, we clean up the http request by closing it.
$HTTP_Response.Close()
Code acknowledges that there is a 404 error
Exception calling "GetResponse" with "0" argument(s): "The remote server returned an error: (404) Not Found."
At C:\Users\TX394UT\Desktop\Web_Bot_Project\WebsiteMonitoring.ps1:6 char:1
+ $HTTP_Response = $HTTP_Request.GetResponse()
However, Site is OK! Prints on the console.
the way that the webclient handles returned errors is ... odd. [grin]
that message is treated as a non-terminating error and needs to be handled as such. the most obvious way is with Try/Catch/Finally. when capturing the exception message, the 404 StatusCode is converted to the text value for it - NotFound.
$TestUrl = 'https://www.google.com/cas76'
try
{
$Response = (Invoke-WebRequest -Uri $TestUrl -ErrorAction Stop).StatusCode
}
catch
{
$Response = $_.Exception.Response.StatusCode
}
$Response
output for the bad url = NotFound
output for a good url = 200

Powershell -UseDefaultCredentials to Authenticate URL

New to the forum and new to Powershell. Hopefully in the right place. I'm trying to authenticate a web url and use the -UseDefaultCredentials within Powershell. I found this code on Stackoverflow - 404 Response:
$URI='http://www.example.com'
try {
$HttpWebResponse = $null;
$HttpWebRequest = [System.Net.HttpWebRequest]::Create("$URI");
$HttpWebResponse = $HttpWebRequest.GetResponse();
if ($HttpWebResponse) {
Write-Host -Object $HttpWebResponse.StatusCode.value__;
Write-Host -Object $HttpWebResponse.GetResponseHeader("X-Detailed-Error");
}
}
catch {
$ErrorMessage = $Error[0].Exception.ErrorRecord.Exception.Message;
$Matched = ($ErrorMessage -match '[0-9]{3}')
if ($Matched) {
Write-Host -Object ('HTTP status code was {0} ({1})' -f $HttpStatusCode, $matches.0);
}
else {
Write-Host -Object $ErrorMessage;
}
$HttpWebResponse = $Error[0].Exception.InnerException.Response;
$HttpWebResponse.GetResponseHeader("X-Detailed-Error");
}
The code above is doing what I want with returning the HTTP code. However, the question is, where do I insert the -UseDefaultCredentials within this block of code? I've tried adding it in to within and outside the try statement but get the error message:
Error Message
I originally used this piece of code:
Invoke-WebRequest -Uri http://www.example.com -UseDefaultCredentials
but my problem I experienced was then trying to write out the HTTP return code compared to the code that is listed above.
Any help would be appreciated.

How do you get the response from a 404 page requested from powershell

I have to call an API exposed by TeamCity that will tell me whether a user exists. The API url is this: http://myteamcityserver.com:8080/httpAuth/app/rest/users/monkey
When called from the browser (or fiddler), I get the following back:
Error has occurred during request processing (Not Found).
Error: jetbrains.buildServer.server.rest.errors.NotFoundException: No user can be found by username 'monkey'.
Could not find the entity requested. Check the reference is correct and the user has permissions to access the entity.
I have to call the API using powershell. When I do it I get an exception and I don't see the text above. This is the powershell I use:
try{
$client = New-Object System.Net.WebClient
$client.Credentials = New-Object System.Net.NetworkCredential $TeamCityAgentUserName, $TeamCityAgentPassword
$teamCityUser = $client.DownloadString($url)
return $teamCityUser
}
catch
{
$exceptionDetails = $_.Exception
Write-Host "$exceptionDetails" -foregroundcolor "red"
}
The exception:
System.Management.Automation.MethodInvocationException: Exception calling "DownloadString" with "1" argument(s): "The remote server returned an error: (404) Not Found." ---> System.Net.WebException: The remote server returned an error: (404) Not Found.
at System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request)
at System.Net.WebClient.DownloadString(Uri address)
at CallSite.Target(Closure , CallSite , Object , Object )
--- End of inner exception stack trace ---
at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception)
at System.Management.Automation.Interpreter.ActionCallInstruction`2.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
I need to be able to check that the page is returned contains the text described above. This way I know whether I should create a new user automatically or not.
I could just check for 404, but my fear is that if the API is changed and the call really returns a 404, then I would be none the wiser.
Change your catch clause to catch the more specific WebException, then you can use the Response property on it to get the status code:
{
#...
}
catch [System.Net.WebException]
{
$statusCode = [int]$_.Exception.Response.StatusCode
$html = $_.Exception.Response.StatusDescription
}
BrokenGlass gave the answer, but this might help:
try
{
$URI='http://8bit-museum.de/notfound.htm'
$HTTP_Request = [System.Net.WebRequest]::Create($URI)
"check: $URI"
$HTTP_Response = $HTTP_Request.GetResponse()
# We then get the HTTP code as an integer.
$HTTP_Status = [int]$HTTP_Response.StatusCode
}
catch [System.Net.WebException]
{
$statusCode = [int]$_.Exception.Response.StatusCode
$statusCode
$html = $_.Exception.Response.StatusDescription
$html
}
$HTTP_Response.Close()
Response:
check: http://8bit-museum.de/notfound.htm
404
Not Found
another approach:
$URI='http://8bit-museum.de/notfound.htm'
try {
$HttpWebResponse = $null;
$HttpWebRequest = [System.Net.HttpWebRequest]::Create("$URI");
$HttpWebResponse = $HttpWebRequest.GetResponse();
if ($HttpWebResponse) {
Write-Host -Object $HttpWebResponse.StatusCode.value__;
Write-Host -Object $HttpWebResponse.GetResponseHeader("X-Detailed-Error");
}
}
catch {
$ErrorMessage = $Error[0].Exception.ErrorRecord.Exception.Message;
$Matched = ($ErrorMessage -match '[0-9]{3}')
if ($Matched) {
Write-Host -Object ('HTTP status code was {0} ({1})' -f $HttpStatusCode, $matches.0);
}
else {
Write-Host -Object $ErrorMessage;
}
$HttpWebResponse = $Error[0].Exception.InnerException.Response;
$HttpWebResponse.GetResponseHeader("X-Detailed-Error");
}
if i understand the question then $ErrorMessage = $Error[0].Exception.ErrorRecord.Exception.Message contains the errormessage you are looking for.
(source: Error Handling in System.Net.HttpWebRequest::GetResponse() )
Another simple example, hope this helps:
BEGIN
{
# set an object to store results
$queries = New-Object System.Collections.ArrayList
Function Test-Website($Site)
{
try
{
# check the Site param passed in
$request = Invoke-WebRequest -Uri $Site
}
catch [System.Net.WebException] # web exception
{
# if a 404
if([int]$_.Exception.Response.StatusCode -eq 404)
{
$request = [PSCustomObject]#{Site=$site;ReturnCode=[int]$_.Exception.Response.StatusCode}
}
else
{
# set a variable to set a value available to automate with later
$request = [PSCustomObject]#{Site=$site;ReturnCode='another_thing'}
}
}
catch
{
# available to automate with later
$request = [PSCustomObject]#{Site=$site;ReturnCode='request_failure'}
}
# if successful as an invocation and has
# a StatusCode property
if($request.StatusCode)
{
$siteURI = $Site
$response = $request.StatusCode
}
else
{
$response = $request.ReturnCode
}
# return the data
return [PSCustomObject]#{Site=$Site;Response=$response}
}
}
PROCESS
{
# test all the things
$nullTest = Test-Website -Site 'http://www.Idontexist.meh'
$nonNullTest = Test-Website -Site 'https://www.stackoverflow.com'
$404Test = Test-Website -Site 'https://www.stackoverflow.com/thispagedoesnotexist'
# add all the things to results
$queries.Add($nullTest) | Out-Null
$queries.Add($nonNullTest) | Out-Null
$queries.Add($404Test) | Out-Null
# show the info
$queries | Format-Table
}
END{}
Output:
Site Response
---- --------
http://www.Idontexist.meh another_thing
https://www.stackoverflow.com 200
https://www.stackoverflow.com/thispagedoesnotexist 404
You could try using the Internet Explorer COM object instead. It allows you to check the browser return codes and navigate the HTML object model.
Note: I've found that you need to run this from an elevated PowerShell prompt in order to maintain the COM object definition.
$url = "http://myteamcityserver.com:8080/httpAuth/app/rest/users/monkey"
$ie = New-Object -ComObject InternetExplorer.Application
Add this to See the browser
$ie.visibility = $true
Navigate to the site
$ie.navigate($url)
This will pause the script until the page fully loads
do { start-sleep -Milliseconds 250 } until ($ie.ReadyState -eq 4)
Then verify your URL to make sure it's not an error page
if ($ie.document.url -ne $url) {
Write-Host "Site Failed to Load" -ForegroundColor "RED"
} else {
[Retrieve and Return Data]
}
You can navigate HTML Object model via $ie.document. Using Get-Member and HTML methods such as GetElementsByTagName() or GetElementById().
If credentials are an issue, build this into a function then use Invoke-Command with the -Credentials parameter to define your logon information.

" The remote server returned an error: (401) Unauthorized "

I am trying to verify if my URLs get a response.
in other words, i am trying to check if the authentication has succeeded approaching the site.
I used:
$HTTP_Request = [System.Net.WebRequest]::Create('http://example.com')
$HTTP_Response = $HTTP_Request.GetResponse()
$HTTP_Status = [int]$HTTP_Response.StatusCode
If ($HTTP_Status -eq 200) {
Write-Host "Site is OK!"
} Else {
Write-Host "The Site may be down, please check!"
}
$HTTP_Response.Close()
and I got the response:
The remote server returned an error: (401) Unauthorized.
but after that i got:
site is ok
Does that mean it's ok? If not, what is wrong?
You are getting OK because you rerun your command and $HTTP_Response contains an object from a previous, successful run. Use try/catch combined with a regex to extract correct status code(and clean up your variables):
$HTTP_Response = $null
$HTTP_Request = [System.Net.WebRequest]::Create('http://example.com/')
try{
$HTTP_Response = $HTTP_Request.GetResponse()
$HTTP_Status = [int]$HTTP_Response.StatusCode
If ($HTTP_Status -eq 200) {
Write-Host "Site is OK!"
}
else{
Write-Host ("Site might be OK, status code:" + $HTTP_Status)
}
$HTTP_Response.Close()
}
catch{
$HTTP_Status = [regex]::matches($_.exception.message, "(?<=\()[\d]{3}").Value
Write-Host ("There was an error, status code:" + $HTTP_Status)
}
.Net HttpWebRequest.GetResponse() throws exceptions on non-OK status codes, you can read more about it here: .Net HttpWebRequest.GetResponse() raises exception when http status code 400 (bad request) is returned
I think the following happens:
You are trying to query a web page using GetResponse(). This fails, so the following Statements run with the values you set in a previous run. This leads to the Output you described.
I personally tend to use the invoke-webrequest in my scripts. This makes it easier to handle Errors, because it supports all Common Parameters like Erroraction and so on.