Convert cURL to PowerShell Invoke WebRequest - powershell

I do not know much about APIs, but nothing I have seen seems to get me what I need.
This works in the cmd line:
curl -X POST --user user:password --data-raw "Request=GET%2C%2892837F755%29%2C%28" http://url.com
I cannot seem to convert this to anything working in PowerShell. I have tried this, and this, and this, and nothing seems to work.
Anyone got any suggestions?
Thanks to a comment, I tried this:
Invoke-WebRequest -Method Post -UseBasicParsing -Uri "http://example.com/foo?some=param&Request=GET%%2C%%2892837F‌​755%%29%%2C%%28" -Headers #{Authorization=[Convert]::ToBase64String([Text.Encoding]::A‌​SCII.GetBytes('usern‌​ame:password'))}
But it returned this:
Invoke-WebRequest :
401 Authorization Required
Authorization Required
This server could not verify that you
are authorized to access the document
requested. Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.
Apache/2.2.3 (Red Hat) Server at rplus.intdata.com Port 80
At line:1 char:1
+ Invoke-WebRequest -Method Post -UseBasicParsing -Uri ""http://example ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
EDIT2:
I figured out how to see the outgoing request from curl, and it looks like this:
C:\windows\system32>curl -X POST --user user:password --data-raw "Request=GET%2C%2892837F755%29%2C%28" http://url.com
* Trying 100.100.100.100...
* TCP_NODELAY set
* Connected to url.com (100.100.100.100) port 80 (#0)
* Server auth using Basic with user 'user'
> POST /url HTTP/1.1
> Host: url.com
> Authorization: Basic *****
> User-Agent: curl/7.52.1
> Accept: */*
> Content-Length: 87
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 87 out of 87 bytes
* HTTP 1.0, assume close after body
< HTTP/1.0 200 Script results follow
< Content-type: text/plain
<
"theResponseIExpected"
* Curl_http_done: called premature == 0
* Closing connection 0
(I changed a bunch of the actual strings in the text above...).
SO, it looks like there's this basic authentication in the header, followed by some 20-digit alphanumeric thing (***** in the text above). What is that? How does curl generate that? If it is some sort of encryption or translation, it looks as if it is only changing the password, and not the user. Not sure if that is the same as what this does - ToBase64String([Text.Encoding]::ASCII.GetBytes('user:password'))
Edit 3:
This seems to work, and seems to get a response, but PowerShell doesnt seem to like it:
$user = "user"
$pass = "password"
$pair = "${user}:${pass}"
$bytes = [System.Text.Encoding]::ASCII.GetBytes($pair)
$base64 = [System.Convert]::ToBase64String($bytes)
$basicAuthValue = "Basic $base64"
$headers = #{ Authorization = $basicAuthValue }
Invoke-WebRequest -Uri "url.com?Request=..." -Headers $headers
I am now getting the following:
Invoke-WebRequest : The server committed a protocol violation. Section=ResponseStatusLine
At line:13 char:1
I have found a few posts related to this, but they all want me to change a config file. I have tried that, but it does not seem to work. It seems the response is in HTTP1.0, and InvokeWebRequest wants 1.1. Is there a way within this script to let it allow 1.0 responses??

Related

Issue in inoke-webrequest to update my chromecast firmware

I need to update firmware of my chromecast but I have restriction on my internet and then I cannot update it automatically. I need to update it by pc while I am using vpn. Therefore, I am using following script in powershell. However, the code run successfully but there is no result.
Invoke-WebRequest -Method Post -ContentType "application/json" -Body '{"params": "ota foreground"}' -
Uri "http://<my chrom IP>:8008/setup/reboot" -Verbose -UserAgent "curl"
Here is result of running that script:
VERBOSE: POST http://<my chrom IP>:8008/setup/reboot with -1-byte payload
VERBOSE: received 0-byte response of content type
StatusCode : 200
StatusDescription : OK
Content : {}
RawContent : HTTP/1.1 200 OK
Access-Control-Allow-Headers: Content-Type
Content-Length: 0
Cache-Control: no-cache
Headers : {[Access-Control-Allow-Headers, Content-Type], [Content-Length, 0], [Cache-Control, no-cache]}
RawContentLength : 0

Invoke-Restmethod returning text/html when expecting json with Powershell

Command being used:
$Tenant = "tentant_id"
$ClientID = "application_id"
$ClientSecret = "application_password"
$Url = "http://login.microsoftonline.com/$Tenant/oauth2/v2.0/token"
$Body = #{
'client_id' = $ClientID
'scope' = "https://graph.microsoft.com/.default"
'client_secret' = $ClientSecret
'grant_type' = 'client_credentials'
}
Invoke-RestMethod -Method Post -Uri $Url -Body $Body -Verbose
Actual Result:
VERBOSE: POST http://login.microsoftonline.com/<tenant_ID>/oauth2/v2.0/token with -1-byte payload
VERBOSE: received 17293-byte response of content type text/html; charset=utf-8
html
----
html
Expected Result:
{
"token_type": "Bearer",
"expires_in": 3599,
"ext_expires_in": 0,
"access_token": "eyJ0eXAiOiJKV1Q......."
}
Is there something that I'm missing? For some reason when using the above code in Powershell it's returning some sort of webpage, when it should give the token. Using the Chrome extension Postman I was able to construct a POST request using the same url, and client id, scope, client secret and grant type in the body which gives the correct response. Using the access token from that, I can then use the following to get a proper response back:
Invoke-RestMethod -Method Get -Headers #{Authorization = "Bearer $AccessToken"} -Uri https://graph.microsoft.com/v1.0/users/jsmith#example.com
Here's a link to pastebin for the output of $HtmlObject | format-list *
https://pastebin.com/KMMKDGut
As it's too large to format as code here.
Usually, when you get back an HTML response to a JSON or SOAP call, it means that IIS was really upset and is returning back an IIS/ASP.net error page. In that case, you typically need to look at the event logs on the server itself to figure out why IIS/ASP.Net was upset.
Thanks to the suggestion by Marc LaFleur, I took a look at the request with Fiddler. It showed the following:
HTTP/1.1 302 Found
Location: https://login.microsoftonline.com:443/clientID/oauth2/v2.0/token
Server: Microsoft-IIS/8.5
client-request-id: 1d918059-ab39-4d41-8191-b0416d4f03f4
X-Powered-By: ASP.NET
Date: Wed, 10 Jan 2018 21:52:51 GMT
Content-Length: 209
<html><head><title>Object moved</title></head><body>
<h2>Object moved to here.</h2>
</body></html>
Apparently all I had to do was change the original URL to https so that it doesn't have to redirect. Not sure what was happening in the Html object returned though and why I wasn't seeing that in Fiddler.

Using the Alexa voice service RestAPI with cURL

I'd like to the the Alexa voice API (https://developer.amazon.com/public/solutions/alexa/alexa-voice-service/rest/speechrecognizer-requests) with curl. The voicerecogniser API call is more complex than I'm used to using and needs an MP3 file attaching that includes the voice sample. Can anyone advise on how the following would be structured with curl? (There's more info at the given link)
POST /v1/avs/speechrecognizer/xxxxxxxxxxxx HTTP/1.1
Host: access-alexa-na.amazon.com
Authorization: Bearer xxxxxxxxxxxx
Content-Type: multipart/form-data; boundary=boundary_term
Transfer-Encoding: chunked
--boundary_term
Content-Disposition: form-data; name="request"
Content-Type: application/json; charset=UTF-8
{
"messageHeader": {
"deviceContext": [
{
"name": "playbackState",
"namespace": "AudioPlayer"
"payload": {
"streamId": "xxxxxxxxxxxx",
"offsetInMilliseconds": "xxxxxxxxxxxx",
"playerActivity": "xxxxxxxxxxxx"
}
},
{
...
},
...
]
},
"messageBody": {
"profile": "alexa-close-talk",
"locale": "en-us",
"format": "audio/L16; rate=16000; channels=1"
}
}
--boundary_term
Content-Disposition: form-data; name="audio"
Content-Type: audio/L16; rate=16000; channels=1
...encoded_audio_data...
--boundary_term--
I'm no bash expert but this how I was able to interact with AVS using cURL. I generate a file containing the multipart body content which includes the binary audio data and pass that along to cURL.
############################################################
# First we creat a bunch of variables to hold data.
############################################################
# Auth token
TOKEN="Atza|IQEBLjAsAhR..."
# Boundary
BOUNDARY="BOUNDARY1234"
BOUNDARY_DASHES="--"
# Newline characters
NEWLINE='\r\n';
# Metadata headers
METADATA_CONTENT_DISPOSITION="Content-Disposition: form-data; name=\"metadata\"";
METADATA_CONTENT_TYPE="Content-Type: application/json; charset=UTF-8";
# Audio headers
AUDIO_CONTENT_TYPE="Content-Type: audio/L16; rate=16000; channels=1";
AUDIO_CONTENT_DISPOSITION="Content-Disposition: form-data; name=\"audio\"";
# Metadata JSON body
METADATA="{\
\"messageHeader\": {},\
\"messageBody\": {\
\"profile\": \"alexa-close-talk\",\
\"locale\": \"en-us\",\
\"format\": \"audio/L16; rate=16000; channels=1\"\
}\
}"
############################################################
# Then we start composing the body using the variables.
############################################################
# Compose the start of the request body
POST_DATA_START="
${BOUNDARY_DASHES}${BOUNDARY}${NEWLINE}${METADATA_CONTENT_DISPOSITION}${NEWLINE}\
${METADATA_CONTENT_TYPE}\
${NEWLINE}${NEWLINE}${METADATA}${NEWLINE}${NEWLINE}${BOUNDARY_DASHES}${BOUNDARY}${NEWLINE}\
${AUDIO_CONTENT_DISPOSITION}${NEWLINE}${AUDIO_CONTENT_TYPE}${NEWLINE}"
# Compose the end of the request body
POST_DATA_END="${NEWLINE}${NEWLINE}${BOUNDARY_DASHES}${BOUNDARY}${BOUNDARY_DASHES}${NEWLINE}"
# Now we create a request body file to hold everything including the binary audio data.
# Write metadata to body file
echo -e $POST_DATA_START > multipart_body.txt
# Append binary audio data to body file
cat hello.wav >> multipart_body.txt
# Append closing boundary to body file
echo -e $POST_DATA_END >> multipart_body.txt
############################################################
# Finally we get to compose the cURL request command
# passing it the generated request body file as the multipart body.
############################################################
# Compose cURL command and write to output file
curl -X POST \
-H "Authorization: Bearer ${TOKEN}"\
-H "Content-Type: multipart/form-data; boundary=${BOUNDARY}"\
--data-binary #foo.txt\
https://access-alexa-na.amazon.com/v1/avs/speechrecognizer/recognize\
> response.txt
The audio MUST be mono channel, sampled at 16k Hz, and signed 16 bit PCM. Otherwise AVS sends nothing back.
For more information check out my Alexa Voice Service (AVS) with cURL blog post.

curl command line equivalent to this perl code

I want to write a curl command for a POST request equivalent to this Perl code:
use strict;
use warnings;
use LWP::UserAgent;
my $base = 'http://www.uniprot.org/mapping/';
my $params = {
from => 'ACC',
to => 'P_REFSEQ_AC',
format => 'tab',
query => 'P13368'
};
my $agent = LWP::UserAgent->new();
push #{$agent->requests_redirectable}, 'POST';
my $response = $agent->post($base, $params);
$response->is_success ?
print $response->content :
die 'Failed, got ' . $response->status_line .
' for ' . $response->request->uri . "\n";
I tried with (and many other variants) :
curl -X POST -H "Expect:" --form "from=ACC;to=P_REFSEQ_AC;format=tab; query=P13368" http://www.uniprot.org/mapping/ -o out.tab
The Perl code retrieves the expected result, but the curl command line does not. It retrieves the web page from "http://www.uniprot.org/mapping/" but does not make the POST request.
I looked for an error in the response header, but didn't find anything suspicious.
> POST http://www.uniprot.org/mapping/ HTTP/1.1
> User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
> Host: www.uniprot.org
> Accept: */*
> Proxy-Connection: Keep-Alive
> Content-Length: 178
> Content-Type: multipart/form-data; boundary=----------------------------164471d8347f
>
} [data not shown]
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Server: Apache-Coyote/1.1
< Vary: User-Agent
< Vary: Accept-Encoding
< X-Hosted-By: European Bioinformatics Institute
< Content-Type: text/html;charset=UTF-8
< Date: Wed, 05 Aug 2015 20:32:00 GMT
< X-UniProt-Release: 2015_08
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Headers: origin, x-requested-with, content-type
< X-Cache: MISS from localhost
< X-Cache-Lookup: MISS from localhost:3128
< Via: 1.0 localhost (squid/3.1.20)
< Connection: close
<
I spent almost three days looking for a solution in the web, but nothing is working for me.
It looks like the server expects the data as application/x-www-form-urlencoded and not as multipart/form-data as you do with the --form argument. The following should work:
curl -v -L --data \
"from=ACC&to=P_REFSEQ_AC&format=tab&query=P13368" \
http://www.uniprot.org/mapping/ -o out.tab
With --data you get the expected content-type header, but you must do the encoding yourself. With -L curl follows a redirect which is needed here to get the resulting data.
The -X POST option is not needed since POST is the default method when sending data. And -H "Expect:" is not needed either.

REST interface using nginx httpluamodule

I'm just getting started with nginx and the HttpLuaModule. I've created a test configuration file to familiarize myself with how things work.
now I'm trying to write some logic to accept GET, POST and DELETE requests for a specific type of resource.
I would like to create a "location" entry that would match the following URI / accept the following curl calls:
curl -i -X GET http://localhost/widgets/widget?name=testname&loc=20000 -H "Accept:application/json"
This is what my current nginx.conf looks like:
server {
listen 80;
server_name nsps2;
root /var/www/;
index index.html index.htm;
#charset koi8-r;
#access_log logs/host.access.log main;
#curl http://localhost/hello?name=johndoe
location /hello {
default_type "text/plain";
content_by_lua '
local rquri = ngx.var.request_uri;
ngx.say("the uri is ", rquri ,".")
local name = ngx.var.arg_name or "Anonymous"
ngx.say("Hello, ", name, "!")
';
}
location / {
root /var/www/;
index index.html index.htm;
}
#curl -i -X GET http://localhost/widgets/widget?name=testname&loc=20000 -H "Accept:application/json"
location /widgets/widget {
root /var/www/widgets;
default_type "text/pain";
content_by_lua '
local arga,argb = ngx.arg[1], ngx.arg[2] ;
ngx.say("the arga is ", arga ,".")
ngx.say("the argb is ", argb, ".")
';
}
Using the last "location" entry, I'm trying to
1. prove that the system is getting the GET request
2. prove that I understand how to access the parameters passed in with the GET request.
I'm getting an error right now that looks like this:
2015/02/24 20:18:19 [error] 2354#0: *1 lua entry thread aborted: runtime error: content_by_lua:2: API disabled in the context of content_by_lua*
stack traceback:
coroutine 0:
[C]: in function '__index'
content_by_lua:2: in function <content_by_lua:1>, client: 127.0.0.1, server: nsps2, request: "GET /widgets/widget?name=testname?loc=20000 HTTP/1.1", host: "localhost"
I'm not too sure about what this error means / is trying to tell me.
Any tips would be appreciated.
Thank you.
EDIT 1
I think the problem was something syntactically wrong. I was reading the manual and found an example to try. I've changed the code to look like this:
location /widgets/widget {
default_type "text/pain";
content_by_lua '
local args = ngx.req.get_uri_args()
for key, val in pairs(args) do
if type(val) == "table" then
ngx.say(key, ": ", table.concat(val, ", "))
else
ngx.say(key, ": ", val)
end
end
';
}
Now when I call the app like this:
mytestdevbox2:/var/www/nsps2# curl -i -X GET http://localhost/widgets/widget?name=testname&loc=20000 -H "Accept:application/json"
-ash: -H: not found
mytestdevbox2:/var/www/nsps2# HTTP/1.1 200 OK
Server: nginx/1.6.2
Date: Tue, 24 Feb 2015 21:32:44 GMT
Content-Type: text/pain
Transfer-Encoding: chunked
Connection: keep-alive
name: testname
[1]+ Done curl -i -X GET http://localhost/widgets/widget?name=testname
After the system displays the "name: testname" stuff, it just sits there until I hit "enter". After I do that, then it proceeds to display the 1+ stuff.
I'm not too sure what it's doing.
EDIT 2:
Adding quotes to the curl call did fix the problem:
mytestdevbox2:/var/www/nsps2# curl -i -X GET 'http://localhost/widgets/widget?name=testname&loc=20000'
HTTP/1.1 200 OK
Server: nginx/1.6.2
Date: Wed, 25 Feb 2015 12:59:29 GMT
Content-Type: text/pain
Transfer-Encoding: chunked
Connection: keep-alive
loc: 20000
name: testname
mytestdevbox2:/var/www/nsps2#
The problem described in EDIT 1 doesn't have anything to do with your question about nginx and is caused by not quoting the parameters for curl when you execute the command:
curl -i -X GET http://localhost/widgets/widget?name=testname&loc=20000
This is executed as two commands separated by '&': curl -i -X GET http://localhost/widgets/widget?name=testname and loc=20000; that's why you see the output after you already got the prompt back as the first command is now executed in the background. "1+ Done" message is a confirmation that the background process is terminated; it's just shown after you press Enter.
Wrap the URL with the query string in quotes and you should see the expected behavior.