Measure response time using Invoke-WebRequest similar to curl - powershell

I have a curl command which response time by breaking it by each action in invoking a service.
curl -w "#sample.txt" -o /dev/null someservice-call
I want to measure the response time in a similar way using PowerShell's built-in Invoke-WebRequest call. So far I am able to get total response time using Measure-Command. Can someone please help me with this?
Content of sample.txt used in curl:
time_namelookup: %{time_namelookup}\n
time_connect: %{time_connect}\n
time_appconnect: %{time_appconnect}\n
time_pretransfer: %{time_pretransfer}\n
time_redirect: %{time_redirect}\n
time_starttransfer: %{time_starttransfer}\n
----------\n
time_total: %{time_total}\n

time in milliseconds:
$url = "google.com"
(Measure-Command -Expression { $site = Invoke-WebRequest -Uri $url -UseBasicParsing }).Milliseconds

This seems to do it without any noticable overhead:
$StartTime = $(get-date)
Invoke-WebRequest -Uri "google.com" -UseBasicParsing
Write-Output ("{0}" -f ($(get-date)-$StartTime))

As the other solutions point out, there is a performance catch when using powershell only.
The most efficient solution would probably be to write some c# with the measurements built in. But when it's not properly compiled beforehand, the loading-time will increase dramatically when the C# needs to be compiled.
But there is another way.
Since you can use almost all dotnet constructs within powershell, you can just write the same request and measurement logic within powershell itself.
I have written a small method which should do the trick:
function Measure-PostRequest {
param(
[string] $Url,
[byte[]] $Bytes,
[switch] $Block
)
$content = [Net.Http.ByteArrayContent]::new($bytes);
$client = [Net.Http.HttpClient]::new();
$stopwatch = [Diagnostics.Stopwatch]::new()
$result = $null;
if ($block) {
# will block and thus not allow ctrl+c to kill the process
$stopwatch.Start()
$result = $client.PostAsync($url, $content).GetAwaiter().GetResult()
$stopwatch.Stop()
} else {
$stopwatch.Start()
$task = $client.PostAsync($url, $content)
while (-not $task.AsyncWaitHandle.WaitOne(200)) { }
$result = $task.GetAwaiter().GetResult()
$stopwatch.Stop()
}
[PSCustomObject]#{
Response = $result
Milliseconds = $stopwatch.ElapsedMilliseconds
}
}

Related

PowerShell Pester Mock Rest API Calls

Is there any simple approach on how to mock a Rest API Calls in Pester.
Here is my code, I just need to mock those Rest API Calls in Pester and test them, could someone help me here.
Describe 'Test worldclockapi.com' {
BeforeAll {
$response = Invoke-WebRequest -Method 'GET' -Uri 'http://worldclockapi.com/api/json/utc/now'
$responseContent = $response.Content | ConvertFrom-Json
Write-Output $responseContent
}
It 'It should respond with 200' {
$response.StatusCode | Should -Be 200
}
It 'It should have a null service response' {
$responseContent.serviceResponse | Should -BeNullOrEmpty
}
It 'It should be the right day of the week' {
$dayOfWeek = (Get-Date).DayOfWeek
$responseContent.dayOfTheWeek | Should -Be $dayOfWeek
}
It 'It should be the right year' {
$year = Get-Date -Format 'yyyy'
$responseContent.currentDateTime | Should -BeLike "*$year*"
}
It 'It should be the right month' {
$month = Get-Date -Format 'MM'
$responseContent.currentDateTime | Should -BeLike "*$month*"
}
# These two tests assume you are running this outside daylight savings (during the winter) .. hacky but good way to showcase the syntax ;)
It 'It should not be daylight savings time' {
$responseContent.isDayLightSavingsTime | Should -Not -Be $true
}
It 'It should not be daylight savings time another way' {
$responseContent.isDayLightSavingsTime | Should -BeFalse
}
}
It's probably easiest to use a real response as a template for your mock output.
Invoke-WebRequest -Method 'GET' -Uri 'http://worldclockapi.com/api/json/utc/now' |
Export-Clixml .\response.xml
The above command will serialize a real response from the API to a file.
Now we can import the file to use with our mock. All it takes is to use the Mock command to define our mock.
Mock
-CommandName : command we are mocking
-ParameterFilter: An optional filter to limit mocking behavior only to usages of CommandName where the values of the parameters passed to the command pass the filter. This ScriptBlock must return a boolean value.
-MockWith: A ScriptBlock specifying the behavior that will be used to mock CommandName. The default is an empty ScriptBlock. It is here that we import the file to be our output.
Mock -CommandName Invoke-WebRequest -ParameterFilter { $Method -eq 'GET' } -MockWith { Import-Clixml .\response.xml }
Now when Invoke-WebRequest is called with -Method 'Get' our mock will be called instead and will return our object which we import using Import-CliXml (or other method - json, xml, etc. Note: the mock command must come before any calls to the command we are mocking. Also note that the mock will be called even inside other functions that use the command.
BeforeAll {
Mock -CommandName Invoke-WebRequest -ParameterFilter { $Method -eq 'GET' } -MockWith { Import-Clixml .\response.xml }
# on this next line our mock will be called instead and will return our prepared object
$response = Invoke-WebRequest -Method 'GET' -Uri 'http://worldclockapi.com/api/json/utc/now'
$responseContent = $response.Content | ConvertFrom-Json
Write-Output $responseContent
}
I think Daniel's answer is great, but if you are working on a large or shared repository then you just need to be careful about managing those XML files too. Another option, which I use, is to have one large Json file for all your returned objects using real responses. It can be imported in either BeforeAll or BeforeDiscovery depending on how your tests are structured.
The reason for my supplementary answer is really just to cover error responses too, because it is important to have test cases that show how you deal with a REST call failing. Wrapping Invoke-WebRequest in your own function might be useful for returning personalised errors, handling header responses, and having constants for a site name or an allowed set of API paths. Depending on the version of PowerShell, this is how I might handle a 404, for example.
Context " When a path does not exist in the API" {
BeforeAll {
Mock Invoke-WebRequest {
# Use the actual types returned by your function or directly from Invoke-WebRequest.
if ($PSVersionTable.PSEdition -eq "Desktop") {
$WR = New-MockObject -Type 'System.Net.HttpWebResponse'
$Code = [System.Net.HttpStatusCode]::NotFound
# Use Add-Member because StatusCode is a read-only field on HttpWebResponse
$WR | Add-Member -MemberType NoteProperty -Name StatusCode -Value $Code -Force
$Status = [System.Net.WebExceptionStatus]::ProtocolError
$Ex = [System.Net.WebException]::new("404", $null, $Status, $WR)
}
else {
$Message = [System.Net.Http.HttpResponseMessage]::new()
$Message.StatusCode = [System.Net.HttpStatusCode]::NotFound
$Ex = [Microsoft.PowerShell.Commands.HttpResponseException]::new("404", $Message)
}
throw $Ex
} -ParameterFilter {
$Uri -eq "http://worldclockapi.com/api/json/utc/NEVER" -and
$Method -eq "Get"
}
$GetRestTimeParams = #{
Uri = "http://worldclockapi.com/api/json/utc/NEVER"
Method = "Get"
}
}
It " Get-RestTime should not run successfully" {
{ Get-RestTime #GetRestTimeParams } | Should -Throw
}
It " Get-RestTime should throw a 404 error" {
$ShouldParams = #{
# Use the actual types returned by your function or directly from Invoke-WebRequest.
ExceptionType = [System.Net.WebException]
ExpectedMessage = "404: NotFound"
}
{
Get-RestTime #GetRestTimeParams
} | Should -Throw #ShouldParams
}
}

PowerShell Start-Job not running a Scriptblock

I have been struggling for a few days now with running Start-Job with a -Scriptblock.
If I run this, I get today's date back in Receive-Job:
$sb = {Get-Date}
Start-Job -ScriptBlock $sb
Now, I have a script which does some reporting via 65+ API calls for 8 different statuses. In total, it does 500+ API calls and takes almost 20mins to complete (2-5seconds per API call, x 500), so I am looking to run each foreach block in parallel.
The script works well as a script but not as a Start-Job.
The Scriptblock will get a session token, gets data from API server in a foreach loop and then populates the results into a $Global: variable as it goes.
When I run the job, it "Completes" instantly and Has More Data is False (so I can't see any error messages):
67 Job67 BackgroundJob Completed False localhost foreach ($item....
Any suggestions as to where I am going wrong?
My sample code looks like this:
$sb = {# Get count of Status1
foreach ($item in $Global:GetItems) {
$item1 = $item.id
If ($null -eq $TokenExpiresIn) { .\Get-Token.ps1 }
$Global:TokenExpiresIn = New-TimeSpan -Start $TokenExpiresNow -End $Script:TokenExpiresTime
$Method = "GET"
$Fromitems = $item1
$Status = "foo"
$Limit = "1"
$Fields = "blah"
$URL = "https://$APIServer/apis/v1.1/status?customerId=1&itemId=$Fromitems&status=$Status&fields=$Fields&limit=$Limit"
$Header = #{"Accept" = "*/*"; "Authorization" = "Bearer $SessionToken" }
$itemStatsResponse = Invoke-WebRequest -Method $Method -Uri $URL -Header $Header
$itemStatsJSON = ConvertFrom-Json -InputObject $itemStatsResponse
$itemCount = $itemStatsJSON.metadata.count
$Global:GetItems | Where-Object -Property id -EQ $item1 | Add-Member -MemberType NoteProperty "foo" -Value $itemCount
}
}
I am running the Scriptblock as follows:
Start-Job -Name "foo" -ScriptBlock $sb
I am using PowerShell 7.1. The script as a whole runs successfully through all 500+ API calls, however it's a bit slow, so I am also looking at how I can "milti-thread" my API calls moving forward for better performance.
Thanks in advance for your support/input. The other posts on StackOverflow relating to PowerShell and Scriptblocks have not assisted me thus far.

URL health-check PowerShell script correctly gets HTTP 200 on most sites, but incorrect '0' status code on some...API timeout issue?

I have a URL health-checking PowerShell script which correctly gets an HTTP 200 status code on most of my intranet sites, but a '0' status code is returned on a small minority of them. The '0' code is an API return rather than from the web site itself, according to my research of questions from others who have written similar URL-checking PowerShell scripts. Thinking this must be a timeout issue, where API returns '0' before the slowly-responding web site returns its 200, I've researched yet more questions about this subject area on SO and implemented a suggestion from someone to insert a timeout in the script. The timeout setting though, no matter how high I set the timeout value, doesn't help. I still get the same '0' "response" code from the same web sites even though those web sites are up and running as checked from any regular web browser. Any thoughts on how I could tweak the timeout setting in the script below in order to get the correct 200 response code?
The Script:
$URLListFile = "C:\Users\Admin1\Documents\Scripts\URL Check\URL_Check.txt"
$URLList = Get-Content $URLListFile -ErrorAction SilentlyContinue
#if((test-path $reportpath) -like $false)
#{
#new-item $reportpath -type file
#}
#For every URL in the list
$result = foreach($Uri in $URLList) {
try{
#For proxy systems
[System.Net.WebRequest]::DefaultWebProxy = [System.Net.WebRequest]::GetSystemWebProxy()
[System.Net.WebRequest]::DefaultWebProxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
#Web request
$req = [system.Net.WebRequest]::Create($uri)
$req.Timeout=5000
$res = $req.GetResponse()
}
catch {
#Err handling
$res = $_.Exception.Response
}
$req = $null
#Getting HTTP status code
$int = [int]$res.StatusCode
# output a formatted string to capture in variable $result
"$int - $uri"
#Disposing response if available
if($res){
$res.Dispose()
}
}
# output on screen
$result
#output to log file
$result | Set-Content -Path "C:\Users\Admin1\Documents\Scripts\z_Logs\URL_Check\URL_Check_log.txt" -Force
Current output:
200 - http://192.168.1.1/
200 - http://192.168.1.2/
200 - http://192.168.1.250/config/authentication_page.htm
0 - https://192.168.1.50/
200 - http://app1-vip-http.dev.local/
0 - https://CA/certsrv/Default.asp
Perhaps using PowerShell cmdlet Invoke-WebRequest works better for you. It has many more parameters and switches to play around with like ProxyUseDefaultCredentials and DisableKeepAlive
$pathIn = "C:\Users\Admin1\Documents\Scripts\URL Check\URL_Check.txt"
$pathOut = "C:\Users\Admin1\Documents\Scripts\z_Logs\URL_Check\URL_Check_log.txt"
$URLList = Get-Content -Path $pathIn
$result = foreach ($uri in $URLList) {
try{
$res = Invoke-WebRequest -Uri $uri -UseDefaultCredentials -UseBasicParsing -Method Head -TimeoutSec 5 -ErrorAction Stop
$status = [int]$res.StatusCode
}
catch {
$status = [int]$_.Exception.Response.StatusCode.value__
}
# output a formatted string to capture in variable $result
"$status - $uri"
}
# output on screen
$result
#output to log file
$result | Set-Content -Path $pathOut -Force

How to download a zip file with PowerShell from request stream

Currently, I make a POST request to a external website then I am supposed to get a zip file in return. I can get the zip file, but it comes in an xml with just the name.zip and nothing is downloaded. I have no idea why it is not downloading. My code is below on the piece where I make the actual request. I am not sure if I am over engineering this or what else I would have to do to get the actual file to download.
$url = "https://thewebsite.net/v6_1?id=$messageID"
Write-Output($url)
$Body = [byte[]][char[]]$xmlMessage
Write-Output($Body)
$Request = [System.Net.HttpWebRequest]::CreateHttp($url);
$Request.Method="POST"
$Request.ContentType = 'text/xml;charset=utf-8'
$Request.ContentLength = $Body.Length
$Request.ClientCertificates.Add($Certificate)
Write-Output($Request.ClientCertificates)
$Stream = $Request.GetRequestStream();
$Stream.Write($Body, 0, $Body.Length);
$Response = $Request.GetResponse()
$totalLength = [System.Math]::Floor($Response.get_ContentLength()/1024)
$responseStream = $Response.GetResponseStream()
$targetStream = New-Object -TypeName System.IO.FileStream -ArgumentList "D:\path\to\save\test.txt", Create
$buffer = new-object byte[] 1GB
$count = $responseStream.Read($buffer,0,$buffer.length)
$downloadedBytes = $count
while ($count -gt 0)
{
[System.Console]::CursorLeft = 0
[System.Console]::Write("Downloaded {0}K of {1}K", [System.Math]::Floor($downloadedBytes/1024), $totalLength)
$targetStream.Write($buffer, 0, $count)
$count = $responseStream.Read($buffer,0,$buffer.length)
$downloadedBytes = $downloadedBytes + $count
Write-Output($count)
}
$targetStream.Flush()
$targetStream.Close()
$targetStream.Dispose()
$responseStream.Dispose()
Unfortunately without certain download URI it's hard to clarify either you case is nontrivial or you just select non optimal way to get remote file. Routine way to get ".zip" (or any other 'octet/stream' file) with Power-Shell is execute the following command
Invoke-WebRequest -uri "https://thewebsite.net/v6_1?id=$messageID" -Method "GET" -Outfile (-join($messageID,".zip"))
then $messageID.zip file would be created in directory from which you execute Power-Shell
Progress would be shown in console window automatically. I test this example just before write the answer and it works independently on method "POST"/"GET" when remote host actually return "octet/stream" in the response. Maybe in you case file is not directly returned after requesting
thewebsite.net/v6_1?id=$messageID
but it is not a point of you original question.
Have you tried using Invoke-WebRequest?
$path = [Environment]::GetFolderPath("MyDocuments")
Invoke-WebRequest "example.com" -OutFile "$path\ZippedFile.zip"
A variable does not have to be used, as the path can be completely defined in the Invoke-WebRequest line if desired.

Powershell Command: HttpWebRequest stucks after fetching two requests

I have a list of URL in a text file and I want to test whether all of them are reachable or not. I fired the following command in windows powershell but somehow after displaying the status of first two requests, the command stucks somewhere and never returns. Am I missing something?
cat .\Test.txt | % { [system.Net.WebRequest]::Create("$_").GetResponse().StatusCode }
Text File
http://www.google.com
http://www.yahoo.com
http://www.bing.com
Output:
OK
OK
----> after that it stucks.
use Invoke-WebRequest instead:
$sites = 'http://www.google.com','http://www.yahoo.com','http://www.bing.com'
foreach ($site in $sites) {
Invoke-WebRequest $site
$site
}
From memory: You have to explicitly close the Response stream:
$req = [System.Net.HttpWebRequest]::Create($aRequestUrl);
$response = $null
try
{
$response = $req.GetResponse()
# do something with the response
}
finally
{
# Clear the response, otherwise the next HttpWebRequest may fail... (don't know why)
if ($response -ne $null) { $response.Close() }
}