Get HTTP request and tolerate 500 server error in powershell - powershell

In Powershell v3.0 I would like to return the response code from an HTTP GET, such as 200 OK or 500 Internal Server Error. (This is for a hack-deploy to do a quick warmup of a deployed site and see if it works, a sort of a mini acceptance test. The status code is truly all I want.)
Against my wishes, HttpWebRequest.GetResponse throws an error when it receives a 500 Internal Server Error. This is annoying because it isn't really an error to me in my use case. Anyway, I figured I could catch the exception and still peel out the underlying response code, but I'm having trouble with that.
Here's some almost-working code:
function WebResponseStatusCode (
[Parameter(Mandatory=$true)][string] $url
) {
$req = [system.Net.HttpWebRequest]::Create($url)
try {
$res = $req.GetResponse();
$statuscode = $res.statuscode;
}
catch [System.Net.WebException] {
#the outer error is a System.Management.Automation.ErrorRecord
Write-Host "error!"
return = $_.Response.statuscode; #nope
}
finally {
if (!($res -eq $null)) {
$res.Close();
}
}
return $statuscode;
}
The problem is of course that $_ has no Response property. Neither does $_.InnerException, even when cast:
return [System.Net.WebException]($_.InnerException)
I've played around with $_ | Get-Member and exploring all its properties. I thought $_.TargetObject had some promise but it doesn't appear to.
(Update) I also think I tried variations on $_.Exception.Response though may have gotten it wrong.
Getting just a response code seems like such a simple thing to do.

Here's an example, though it does a couple more things to allow you to test redirections as well as expected exceptions.
function GetWebSiteStatusCode {
param (
[string] $testUri,
$maximumRedirection = 5
)
$request = $null
try {
$request = Invoke-WebRequest -Uri $testUri -MaximumRedirection $maximumRedirection -ErrorAction SilentlyContinue
}
catch [System.Net.WebException] {
$request = $_.Exception.Response
}
catch {
Write-Error $_.Exception
return $null
}
$request.StatusCode
}
GetWebSiteStatusCode -testUri "https://www.google.com/"
GetWebSiteStatusCode -testUri "https://www.google.com/foobar"
GetWebSiteStatusCode -testUri "http://google.com/" -maximumRedirection 0
GetWebSiteStatusCode -testUri "https://accounts.google.com" -maximumRedirection 0
GetWebSiteStatusCode -testUri "https://www.googleapis.com/coordinate/v1/teams/1/custom_fields?fields=1111&key="
GetWebSiteStatusCode -testUri "https://www.googleapis.com/shopping/search/v1/test/products/sasdf/asdf/asdf?key="
#Next test would be for an expected 500 page.
#GetWebSiteStatusCode -testUri "https://www.somesite.com/someurlthatreturns500"

Related

Invoke-Webrequest 400 bad request detailed message

I am running Invoke-Webrequest to upload a file to Nexus server the first time when I run it, works fine but when I re-run it, fails because the file already exists and Nexus is set to not allowing overwriting existing files. What I need is that when I run Invoke-Webrequest when the file I'm trying to upload already exists, it just simply returns 400 bad request. How can I get more detailed error message so that I know what the problem is with the request?
try {
// invoke-webrequest
}
catch {
throw $_.Exception.Response
}
i guess this will help you quite a lot, i find it in this Answer:
function ParseErrorForResponseBody($Error) {
if ($PSVersionTable.PSVersion.Major -lt 6) {
if ($Error.Exception.Response) {
$Reader = New-Object System.IO.StreamReader($Error.Exception.Response.GetResponseStream())
$Reader.BaseStream.Position = 0
$Reader.DiscardBufferedData()
$ResponseBody = $Reader.ReadToEnd()
if ($ResponseBody.StartsWith('{')) {
$ResponseBody = $ResponseBody | ConvertFrom-Json
}
return $ResponseBody
}
}
else {
return $Error.ErrorDetails.Message
}
}
try {
$result = Invoke-WebRequest ...
}
catch {
ParseErrorForResponseBody($_)
}

HTTP Status codes via Powershell

Somehow the below codes works fine for the first few test URLs within C:\testurl.txt then it hung up forever when it is processing the 4th URL from the C:\testurl.txt , no idea why it hangs up?
It is already working fine for up to 3 URLs but stuck up on 4th onward
CLS
$urllist = Get-Content "C:\testurl.txt" # URLs to test one in each line
foreach ($url in $urllist) {
Write-Host $url
$req = [System.Net.WebRequest]::Create($url)
try {
$res = $req.GetResponse()
} catch [System.Net.WebException] {
$res = $_.Exception.Response
}
$res.StatusCode
#Print OK or whatever
[int]$res.StatusCode
#Print 200 or whatever
}
It is working fine for up to 3 URLs but hangs the script on 4th URL without any output or error message. Here is the example of c:\testurl.txt
http://www.google.com
http://www.google.com
http://www.google.com
http://www.google.com
http://www.hotmail.com
http://www.gmail.com
http://www.yahoo.com
http://www.msn.com
Please note each URL will be in a new line, you will see that script will stop at (the 4th one) you may try with your own URLs, etc too
then it hung up forever
No - it's hung until the underlying TCP connections of the previous requests time out.
The .NET CLR will internally pool all WebRequest dispatches so that only a finite number of external requests will be initiated concurrently, and as long as you have a number of un-closed WebResponse objects in memory, your requests will start queuing up.
You can avoid this by closing them (as you should):
foreach ($url in $urllist) {
Write-Host $url
$req = [System.Net.WebRequest]::Create($url)
try {
$res = $req.GetResponse()
}
catch [System.Net.WebException] {
$res = $_.Exception.Response
}
finally {
$res.StatusCode
#Print OK or whatever
[int]$res.StatusCode
#Print 200 or whatever
$res.Dispose()
# close connection, dispose of response stream
}
}

Restart application pool based on http response code

I am trying to write a PowerShell script that will restart an application pool in IIS if a 503 response code is received.
So far I have managed to retrieve the response code for every crm application under the default website in IIS. However I am unsure how I would go about finding the application pool name. I've tried the below, but it returns the same application pool for each site. Can anyone help?
$getSite = (Get-WebApplication -Site 'Default Web Site')
$SiteURL = ForEach ($site in $getSite.path) {("http://localhost")+$site}
ForEach ($crm in $SiteURL){
$req = [system.Net.WebRequest]::Create($crm)
try {
$res = $req.GetResponse()
} catch [System.Net.WebException] {
$res = $_.Exception.Response
}
$ApplicationPool = ForEach ($app in $getSite.applicationpool) {$app}
if([int]$res.StatusCode -eq 503) {write-host ($crm + ' ' + [int]$res.StatusCode) + $app}
}
I think you need to access $_.Exception.InnerException for the the Response property.
Your $ApplicationPool assignment doesn't make much sense, as you would only need one applicationPool name per $crm app you test:
foreach($App in #(Get-WebApplication -Site 'Default Web Site')){
# Uri for the application
$TestUri = 'http://localhost{0}' -f $App.path
# Create WebRequest
$Request = [system.Net.WebRequest]::Create($TestUri)
try {
# Get the response
$Response = $Request.GetResponse()
} catch [System.Net.WebException] {
# If it fails, get Response from the Exception
$Response = $_.Exception.InnerException.Response
}
# The numerical value of the StatusCode value is the HTTP status code, ie. 503
if(503 -eq ($Response.StatusCode -as [int])){
# Restart the app pool
Restart-WebAppPool -Name $App.applicationPool
}
}

How do you get the response from a 404 page requested from powershell

I have to call an API exposed by TeamCity that will tell me whether a user exists. The API url is this: http://myteamcityserver.com:8080/httpAuth/app/rest/users/monkey
When called from the browser (or fiddler), I get the following back:
Error has occurred during request processing (Not Found).
Error: jetbrains.buildServer.server.rest.errors.NotFoundException: No user can be found by username 'monkey'.
Could not find the entity requested. Check the reference is correct and the user has permissions to access the entity.
I have to call the API using powershell. When I do it I get an exception and I don't see the text above. This is the powershell I use:
try{
$client = New-Object System.Net.WebClient
$client.Credentials = New-Object System.Net.NetworkCredential $TeamCityAgentUserName, $TeamCityAgentPassword
$teamCityUser = $client.DownloadString($url)
return $teamCityUser
}
catch
{
$exceptionDetails = $_.Exception
Write-Host "$exceptionDetails" -foregroundcolor "red"
}
The exception:
System.Management.Automation.MethodInvocationException: Exception calling "DownloadString" with "1" argument(s): "The remote server returned an error: (404) Not Found." ---> System.Net.WebException: The remote server returned an error: (404) Not Found.
at System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request)
at System.Net.WebClient.DownloadString(Uri address)
at CallSite.Target(Closure , CallSite , Object , Object )
--- End of inner exception stack trace ---
at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception)
at System.Management.Automation.Interpreter.ActionCallInstruction`2.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
I need to be able to check that the page is returned contains the text described above. This way I know whether I should create a new user automatically or not.
I could just check for 404, but my fear is that if the API is changed and the call really returns a 404, then I would be none the wiser.
Change your catch clause to catch the more specific WebException, then you can use the Response property on it to get the status code:
{
#...
}
catch [System.Net.WebException]
{
$statusCode = [int]$_.Exception.Response.StatusCode
$html = $_.Exception.Response.StatusDescription
}
BrokenGlass gave the answer, but this might help:
try
{
$URI='http://8bit-museum.de/notfound.htm'
$HTTP_Request = [System.Net.WebRequest]::Create($URI)
"check: $URI"
$HTTP_Response = $HTTP_Request.GetResponse()
# We then get the HTTP code as an integer.
$HTTP_Status = [int]$HTTP_Response.StatusCode
}
catch [System.Net.WebException]
{
$statusCode = [int]$_.Exception.Response.StatusCode
$statusCode
$html = $_.Exception.Response.StatusDescription
$html
}
$HTTP_Response.Close()
Response:
check: http://8bit-museum.de/notfound.htm
404
Not Found
another approach:
$URI='http://8bit-museum.de/notfound.htm'
try {
$HttpWebResponse = $null;
$HttpWebRequest = [System.Net.HttpWebRequest]::Create("$URI");
$HttpWebResponse = $HttpWebRequest.GetResponse();
if ($HttpWebResponse) {
Write-Host -Object $HttpWebResponse.StatusCode.value__;
Write-Host -Object $HttpWebResponse.GetResponseHeader("X-Detailed-Error");
}
}
catch {
$ErrorMessage = $Error[0].Exception.ErrorRecord.Exception.Message;
$Matched = ($ErrorMessage -match '[0-9]{3}')
if ($Matched) {
Write-Host -Object ('HTTP status code was {0} ({1})' -f $HttpStatusCode, $matches.0);
}
else {
Write-Host -Object $ErrorMessage;
}
$HttpWebResponse = $Error[0].Exception.InnerException.Response;
$HttpWebResponse.GetResponseHeader("X-Detailed-Error");
}
if i understand the question then $ErrorMessage = $Error[0].Exception.ErrorRecord.Exception.Message contains the errormessage you are looking for.
(source: Error Handling in System.Net.HttpWebRequest::GetResponse() )
Another simple example, hope this helps:
BEGIN
{
# set an object to store results
$queries = New-Object System.Collections.ArrayList
Function Test-Website($Site)
{
try
{
# check the Site param passed in
$request = Invoke-WebRequest -Uri $Site
}
catch [System.Net.WebException] # web exception
{
# if a 404
if([int]$_.Exception.Response.StatusCode -eq 404)
{
$request = [PSCustomObject]#{Site=$site;ReturnCode=[int]$_.Exception.Response.StatusCode}
}
else
{
# set a variable to set a value available to automate with later
$request = [PSCustomObject]#{Site=$site;ReturnCode='another_thing'}
}
}
catch
{
# available to automate with later
$request = [PSCustomObject]#{Site=$site;ReturnCode='request_failure'}
}
# if successful as an invocation and has
# a StatusCode property
if($request.StatusCode)
{
$siteURI = $Site
$response = $request.StatusCode
}
else
{
$response = $request.ReturnCode
}
# return the data
return [PSCustomObject]#{Site=$Site;Response=$response}
}
}
PROCESS
{
# test all the things
$nullTest = Test-Website -Site 'http://www.Idontexist.meh'
$nonNullTest = Test-Website -Site 'https://www.stackoverflow.com'
$404Test = Test-Website -Site 'https://www.stackoverflow.com/thispagedoesnotexist'
# add all the things to results
$queries.Add($nullTest) | Out-Null
$queries.Add($nonNullTest) | Out-Null
$queries.Add($404Test) | Out-Null
# show the info
$queries | Format-Table
}
END{}
Output:
Site Response
---- --------
http://www.Idontexist.meh another_thing
https://www.stackoverflow.com 200
https://www.stackoverflow.com/thispagedoesnotexist 404
You could try using the Internet Explorer COM object instead. It allows you to check the browser return codes and navigate the HTML object model.
Note: I've found that you need to run this from an elevated PowerShell prompt in order to maintain the COM object definition.
$url = "http://myteamcityserver.com:8080/httpAuth/app/rest/users/monkey"
$ie = New-Object -ComObject InternetExplorer.Application
Add this to See the browser
$ie.visibility = $true
Navigate to the site
$ie.navigate($url)
This will pause the script until the page fully loads
do { start-sleep -Milliseconds 250 } until ($ie.ReadyState -eq 4)
Then verify your URL to make sure it's not an error page
if ($ie.document.url -ne $url) {
Write-Host "Site Failed to Load" -ForegroundColor "RED"
} else {
[Retrieve and Return Data]
}
You can navigate HTML Object model via $ie.document. Using Get-Member and HTML methods such as GetElementsByTagName() or GetElementById().
If credentials are an issue, build this into a function then use Invoke-Command with the -Credentials parameter to define your logon information.

Powershell Command: HttpWebRequest stucks after fetching two requests

I have a list of URL in a text file and I want to test whether all of them are reachable or not. I fired the following command in windows powershell but somehow after displaying the status of first two requests, the command stucks somewhere and never returns. Am I missing something?
cat .\Test.txt | % { [system.Net.WebRequest]::Create("$_").GetResponse().StatusCode }
Text File
http://www.google.com
http://www.yahoo.com
http://www.bing.com
Output:
OK
OK
----> after that it stucks.
use Invoke-WebRequest instead:
$sites = 'http://www.google.com','http://www.yahoo.com','http://www.bing.com'
foreach ($site in $sites) {
Invoke-WebRequest $site
$site
}
From memory: You have to explicitly close the Response stream:
$req = [System.Net.HttpWebRequest]::Create($aRequestUrl);
$response = $null
try
{
$response = $req.GetResponse()
# do something with the response
}
finally
{
# Clear the response, otherwise the next HttpWebRequest may fail... (don't know why)
if ($response -ne $null) { $response.Close() }
}