I'm attempting to test this section of a PowerShell function:
# post
$Response = Invoke-WebRequest -Method POST -Uri $Uri -Body $Body -ContentType 'application/xml'
# parse Response.Content; return as System.Xml.XmlDocument
[xml]$Response.Content
by mocking the BasicHtmlWebResponseObject that is returned by Invoke-WebRequest:
Mock Invoke-WebRequest {
$WebResponse = [System.Net.HttpWebResponse]::new()
[System.Net.HttpWebResponse].GetField('m_StatusCode', 'NonPublic, Instance').SetValue(
$WebResponse,
200,
'NonPublic,SetField',
$null,
(Get-Culture)
)
$Content = '<?xml version="1.0" encoding="UTF-8"?><response><control>failure<status></status></control><operation><result><status>failure</status></result></operation></response>'
$Response = [Microsoft.PowerShell.Commands.BasicHtmlWebResponseObject]::new($WebResponse,$Content)
return $Response
}
This assertion fails because I'm not creating the HttpWebResponse or BasicHtmlWebResponseObject correctly:
It "returns the response's Content object" {
# act
$Content = Send-Request -Session $Session
# assert
Assert-MockCalled Invoke-WebRequest
$Content | Should -BeOfType [xml]
$Content.response.control.status | Should -Be 'success'
$Content.response.operation.result.status | Should -Be 'success'
}
** edit **
I thought about using New-MockObject:
Mock Invoke-WebRequest {
$Response = New‐MockObject -Type Microsoft.PowerShell.Commands.BasicHtmlWebResponseObject
$Response.Content = '<?xml version="1.0" encoding="...'
}
but, the Content property is read-only.
** /edit **
What am I missing?
A slightly simpler alternative might be to wrap your invoke-webrequest in a function and just mock that instead. E.g.
function Get-XmlFromUri
{
param( $Uri, $Method, $Body )
$Response = Invoke-WebRequest -Method $Method -Uri $Uri -Body $Body -ContentType 'application/xml’
[xml]$Response.Content
}
Now you can mock Get-XmlFromUri and just return a System.Xml.XmlDocument object from hard-coded xml, which is much easier to create than a BasicHtmlWebResponseObject that needs reflection calls spin up.
Mock Get-XmlFromUri {
[xml] '<?xml version="1.0" encoding="UTF-8"?>
<response>
<control><status>success</status></control>
<operation><result><status>success</status></result></operation>
</response>'
}
Or, depending on how much like BasicHtmlWebResponseObject your code needs it to be, you can just return a hashtable from your invoke-webrequest mock that has the properties you need:
Mock Invoke-WebRequest {
new-object pscustomobject -property #{
Content = '<?xml version="1.0" encoding="UTF-8"?>
<response>
<control><status>success</status></control>
<operation><result><status>success</status></result></operation>
</response>’
}
}
(apologies for code formatting - currently typing one handed on an iPhone at 4 AM holding a not-very-sleepy baby :-S)
On PowerShell Core this doesn't work for me:
[System.Net.HttpWebResponse].GetField('m_StatusCode', 'NonPublic, Instance')
And thats why your Mock isn't returning what you expect. That line does work on Windows PowerShell however. Not sure what the right equivalent is on PSCore, needs research but thought i'd get you this far in the meantime.
This works:
Mock Invoke-WebRequest {
$Response = New-MockObject -Type Microsoft.PowerShell.Commands.BasicHtmlWebResponseObject
$Content = `
'<?xml version="1.0" encoding="UTF-8"?>
<response>
<control><status>success</status></control>
<operation><result><status>success</status></result></operation>
</response>'
$Response | Add-Member -NotePropertyName Content -NotePropertyValue $Content -Force
$Response
}
Related
I want to POST an associative array to a web-service.
Therefore I create the array in PowerShell this way:
$Data = #{
"MyProperty" = "bla bla";
"MyFolder" = #{
"MySubFolder1" = #{
"MySubProperty" = "bla bla";
}
"MySubFolder2" = #{
"MySubProperty" = "bla bla";
}
}
}
I use the following line to POST the data to the web-service:
$Response = Invoke-RestMethod -Method Post -Uri $URL -Body $Data
The point is, that not the whole array is reaching the web-service.
Only the top-level data is received correctly. The nested arrays are not accessible. (System.Collections.Hashtable)
It looks like this:
Array
(
[MyProperty] => bla bla
[MyFolder] => System.Collections.Hashtable
)
I did also try Invoke-WebRequest instead of Invoke-RestMethod and did play with ConvertTo-JSON, but I was not able to find a clean solution to transfer the data nicely to the web-service.
How can I solve this problem?
Thank you!
$Data has to be converted to JSON using ConvertTo-JSON.
Using the following code-line did the trick:
$Response = Invoke-RestMethod -Method Post -Uri $URL -Body ( $Data | ConvertTo-JSON -Compress ) -ContentType "application/json; charset=utf-8"
I wrote the below Powershell script to get the JSON data from an API endpoint (https://data.melbourne.vic.gov.au/resource/vh2v-4nfs) and then write this data in JSON format to Azure Event hub. I am able to successfully get the data from the endpoint however the data is not getting ingested into Azure Event Hub.
Can anyone please let me know what's wrong with the below code:
$url = "https://data.melbourne.vic.gov.au/resource/vh2v-4nfs"
$apptoken = "k7lQcUCVFoROv7rQh9fSSXMkZ"
# Set header to accept JSON
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Accept","application/json")
$headers.Add("X-App-Token",$apptoken)
$results = Invoke-RestMethod -Uri $url -Method get -Headers $headers
$results
$method = "POST"
$URI = "https://YOURNS.servicebus.windows.net/eh-streetparking/messages"
$signature = "SharedAccessSignature sr=YOURNS.servicebus.windows.net%2feh-streetparking&sig=K6bfL1VjW9FUcL0B5xaI%3d&se=16722&skn=eh-sap-streetparking"
#$authInfo = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("$signature"))
# API headers
$headers = #{
"Authorization"=$signature;
# "Content-Type"="application/json;type=entry;charset=utf-8";
"Content-Type"="application/json";
}
# execute the Azure REST API
foreach ( $result in $results)
{
Invoke-RestMethod -Uri $URI -Method $method -Headers $headers -Body $result
}
The value you have presented as the return result from your Invoke-RestMethod is actually a deserialized PowerShell object, not JSON. It appears to be having its quotes removed at some point too.
PSObject ($results) looks like this: $x = #{account_id="12345"; username="12345"; is_locked="False"; employee_id="12345"; first_name="John"; middle_initial="Roger"; last_name="Doe"; full_name="John Roger Doe"}
You can do this to access individual values:
$x.full_name
Finally, follow this syntax to send POST request:
$Cred = Get-Credential
$Url = "https://server.contoso.com:8089/services/search/jobs/export"
$Body = #{
search = "search index=_internal | reverse | table index,host,source,sourcetype,_raw"
output_mode = "csv"
earliest_time = "-2d#d"
latest_time = "-1d#d"
}
Invoke-RestMethod -Method 'Post' -Uri $url -Credential $Cred -Body $body -OutFile output.csv
I'm trying to send a file content to server:
$uri = ...
$headers = #{
...
"Content-Type" = "application/json"
}
[string] $content = Get-Content .\filename -Encoding utf8 -Raw
$body = #{
...
"content" = $content
} | ConvertTo-Json
$response = Invoke-WebRequest $uri -Method 'PUT' -Headers $headers -Body $body
But all of non-ascii symbols are changed to another similar symbols or question marks
How can I escape them?
I've read documentation and I know about parameter -EscapeHandling of cmdlet ConvertTo-Json, but it's available from PowerShell 6.2, I have only 5.1
As a result, I wrote a simple function:
function EscapeNonAscii([string] $s)
{
$sb = New-Object System.Text.StringBuilder;
for ([int] $i = 0; $i -lt $s.Length; $i++)
{
$c = $s[$i];
if ($c -gt 127)
{
$sb = $sb.Append("\u").Append(([int] $c).ToString("X").PadLeft(4, "0"));
}
else
{
$sb = $sb.Append($c);
}
}
return $sb.ToString();
}
And used it like this:
$updateFileResponse = Invoke-WebRequest $updateFileUri -Method 'PUT' -Headers $headers -Body (EscapeNonAscii $body)
It helped. For everybody who will google it in the future, it's a request to GitLab API Update existing file in repository
PS: I use PS as C# because I know it badly. If somebody knows how to rewrite this fragment better please let me know.
PPS: And also I know that StringBuilder.Append changes an existing object, but I add here assigning ($sb = $sb.Append($c) instead of simple $sb.Append($c)) because it prints every action to console. If you know how to fix it please let me know.
As a follow-up to this question, instead of using a long array in the script I wanted to draw from a text file. So I replaced this:
$URLs = 'http://websiteone.com','http://websitetwo.com','http://websitethree.com'
with this
$URLs = Get-Content ./urlfile.txt
or (functionally the same as far I know) this
$URLs = #(Get-Content ./urlfile.txt)
But I end up with Invoke-RestMethod : The remote server returned an error: (400) Bad Request.
Incorporating the great response form my last question, my foreach loop looks like this:
foreach($URL in $URLs) {
$BODY = #([pscustomobject]#{"client" = #{"clientId" = "company"; "clientVersion" = "1.0"}; "threatInfo" = #{"threatTypes" = "MALWARE","SOCIAL_ENGINEERING","THREAT_TYPE_UNSPECIFIED","UNWANTED_SOFTWARE","POTENTIALLY_HARMFUL_APPLICATION"; "platformTypes" = "ANY_PLATFORM"; "threatEntryTypes" = "URL","EXECUTABLE","THREAT_ENTRY_TYPE_UNSPECIFIED"; "threatEntries" = #{"url" = $URL}}})
$JSONBODY = $BODY | ConvertTo-Json
$Result = Invoke-RestMethod -Method 'POST' -Uri $Uri -Body $JSONBODY -Headers $HEADERS
if ( ([string]::IsNullOrEmpty($Result)) ) {} else {write-host $URL "ALERT: Safe browsing match!"}
}
... but this doesn't work if I create the array with the Get-Content cmdlet. If I run the script either way, then type $URLs, I get the exact same data returned. What am I doing wrong with get-content?
The Invoke-RestMethod cmdlet is there to make one Rest request at a time and can't take an array.
You will need to add a forEach loop to step through your $urls one at a time, something like this:
foreach($url in $urls){
$result = Invoke-RestMethod -Uri $url
#do something with $result
}
So to integrate into your sample from the previous question, you should have a urls.txt file which looks like this:
http://google.com
http://small.com
https://fast.com/
And then your code would look like this:
$URLs = get-content .\urls.txt
$HEADERS = #{ 'Content-Type' = "application/json" }
$GOOGLE_API_KEY='[API Key]'
$Uri = 'https://safebrowsing.googleapis.com/v4/threatMatches:find?key='+ $GOOGLE_API_KEY
foreach($URL in $URLs) {
$BODY = #([pscustomobject]#{"client" = #{"clientId" = "company"; "clientVersion" = "1.0"}; "threatInfo" = #{"threatTypes" = "MALWARE","SOCIAL_ENGINEERING","THREAT_TYPE_UNSPECIFIED","UNWANTED_SOFTWARE","POTENTIALLY_HARMFUL_APPLICATION"; "platformTypes" = "ANY_PLATFORM"; "threatEntryTypes" = "URL"; "threatEntries" = #{"url" = $URL}}})
$JSONBODY = $BODY | ConvertTo-Json
$result = Invoke-RestMethod -Method 'POST' -Uri $Uri -Body $JSONBODY -Headers $HEADERS
[pscustomObject]#{SiteName=$url;ThreatInfo=$result.Matches}
}
This would load up the list of $urls from your text file, then run a Rest Request on each, storing the result in $result. Finally, it will make a new PowerShell Object with the site name and show you if there are any matches from the Google SafeBrowsing API.
You'll need to run the command interactively and see which properties from $result are meaningful to you, but you can see all of the expected properties in the Google API Docs.
Edit
Found the bug. It turns out when we use Get-Content the object returned back retains some of the document formatting information from the original file! We can see this by inspecting $JSONBODY. We also see that the conversion to Json from [PSCustomObject is leaving a lot of cruft behind too.
To fix this, we should cast $URL into a string using the ToString() method and also ditch casting to [psCustomObject] too as shown below.
$BODY = #{
"client" = #{
"clientId" = "company"; "clientVersion" = "1.0"
};
"threatInfo" = #{
"threatTypes" = "MALWARE",
"SOCIAL_ENGINEERING",
"THREAT_TYPE_UNSPECIFIED",
"UNWANTED_SOFTWARE",
"POTENTIALLY_HARMFUL_APPLICATION"; "platformTypes" = "ANY_PLATFORM"; "threatEntryTypes" = "URL"; "threatEntries" = #{
"url" = $URL.ToString()
}
}
}
$JSONBODY = $BODY | ConvertTo-Json
I'm trying to send a file via Invoke-RestMethod in a similar context as curl with the -F switch.
Curl Example
curl -F FileName=#"/path-to-file.name" "https://uri-to-post"
In powershell, I've tried something like this:
$uri = "https://uri-to-post"
$contentType = "multipart/form-data"
$body = #{
"FileName" = Get-Content($filePath) -Raw
}
Invoke-WebRequest -Uri $uri -Method Post -ContentType $contentType -Body $body
}
If I check fiddler I see that the body contains the raw binary data, but I get a 200 response back showing no payload has been sent.
I've also tried to use the -InFile parameter with no luck.
I've seen a number of examples using a .net class, but was trying to keep this simple with the newer Powershell 3 commands.
Does anyone have any guidance or experience making this work?
The accepted answer won't do a multipart/form-data request, but rather a application/x-www-form-urlencoded request forcing the Content-Type header to a value that the body does not contain.
One way to send a multipart/form-data formatted request with PowerShell is:
$ErrorActionPreference = 'Stop'
$fieldName = 'file'
$filePath = 'C:\Temp\test.pdf'
$url = 'http://posttestserver.com/post.php'
Try {
Add-Type -AssemblyName 'System.Net.Http'
$client = New-Object System.Net.Http.HttpClient
$content = New-Object System.Net.Http.MultipartFormDataContent
$fileStream = [System.IO.File]::OpenRead($filePath)
$fileName = [System.IO.Path]::GetFileName($filePath)
$fileContent = New-Object System.Net.Http.StreamContent($fileStream)
$content.Add($fileContent, $fieldName, $fileName)
$result = $client.PostAsync($url, $content).Result
$result.EnsureSuccessStatusCode()
}
Catch {
Write-Error $_
exit 1
}
Finally {
if ($client -ne $null) { $client.Dispose() }
if ($content -ne $null) { $content.Dispose() }
if ($fileStream -ne $null) { $fileStream.Dispose() }
if ($fileContent -ne $null) { $fileContent.Dispose() }
}
The problem here was what the API required some additional parameters. Initial request required some parameters to accept raw content and specify filename/size. After setting that and getting back proper link to submit, I was able to use:
Invoke-RestMethod -Uri $uri -Method Post -InFile $filePath -ContentType "multipart/form-data"
I found this post and changed it a bit
$fileName = "..."
$uri = "..."
$currentPath = Convert-Path .
$filePath="$currentPath\$fileName"
$fileBin = [System.IO.File]::ReadAlltext($filePath)
$boundary = [System.Guid]::NewGuid().ToString()
$LF = "`r`n"
$bodyLines = (
"--$boundary",
"Content-Disposition: form-data; name=`"file`"; filename=`"$fileName`"",
"Content-Type: application/octet-stream$LF",
$fileBin,
"--$boundary--$LF"
) -join $LF
Invoke-RestMethod -Uri $uri -Method Post -ContentType "multipart/form-data; boundary=`"$boundary`"" -Body $bodyLines
For anyone wondering (like Jelphy) whether David's answer can be used with cookies/credentials, the answer is yes.
First set the session with Invoke-WebRequest:
Invoke-WebRequest -Uri "$LoginUri" -Method Get -SessionVariable 'Session'
Then POST to the Login URL, which stores the authentication cookie in $Session:
$Response = Invoke-WebRequest -Uri "$Uri" -Method Post -Body $Body -WebSession $Session
The steps above are the standard way to deal with session in Powershell. But here is the important part. Before creating the HttpClient, create an HttpClientHandler and set it's CookieContainer property with the cookies from the session:
$ClientMessageHandler = New-Object System.Net.Http.HttpClientHandler
$ClientMessageHandler.CookieContainer = $Session.Cookies
Then pass this object to the HttpClient constructor
$Client = [System.Net.Http.HttpClient]::new($ClientMessageHandler)
Voila, you now have an HttpClient with session cookies set automatically via Invoke-WebRequest. The rest of David's example should work (copied here for completeness):
$MultipartFormData = New-Object System.Net.Http.MultipartFormDataContent
$FileStream = [System.IO.File]::OpenRead($FilePath)
$FileName = [System.IO.Path]::GetFileName($FilePath)
$FileContent = New-Object System.Net.Http.StreamContent($FileStream)
$MultipartFormData.Add($FileContent, $FieldName, $FileName)
$Result = $Client.PostAsync($url, $content).Result
$Result.EnsureSuccessStatusCode()
I had many files to upload with each request, so I factored out this last bit into a lambda function:
function Add-FormFile {
param ([string]$Path, [string]$Name)
if ($Path -ne "")
{
$FileStream = [System.IO.File]::OpenRead($Path)
$FileName = [System.IO.Path]::GetFileName($Path)
$FileContent = [System.Net.Http.StreamContent]::new($FileStream)
$MultipartFormData.Add($FileContent, $Name, $FileName)
}
}