Error in POST request generated by perl - perl

I am getting an error trying to send a POST request from a perl script:
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->credentials($netloc,$realm,$username,$password);
use HTTP::Request::Common;
my $req = HTTP::Request::Common::POST($url,'content' => $conf);
$req->as_string() is
POST http:.....
Content-Length: 31003
Content-Type: application/x-www-form-urlencoded
.....&createTime=1361370652541&parameters=HASH(0x28fd658)&parameters=HASH(0x28fd670)&parameters=HASH(0x28fd6e8)&parameters=HASH(0x28fd760)&nodes=32&.....&alerts=HASH(0x632d00)&alerts=HASH(0x245abd0)&.....
the error I get is
Unexpected character ('H' (code 72)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
which makes me suspect that the problem is with the repeated
parameters=HASH(...) and alerts=HASH(...) elements;
instead I want to see something like
alerts=%5B%7B%22email%22%3A%22foo%40bar.com%22%2C%22type%22%3A%221%22%2C%22timeout%22%3A%22%22%7D%5D
$conf is a hash reference, and $conf->{"parameters"} and
$conf->{"alerts"}` are array references (whose elements are hash
references).
what am I doing wrong?

You can't post references; you presumably need to serialize them in some way; what is the server expecting?
From the error it looks like either the entire array or possibly each of the hashes in it should be serialized in JSON:
use JSON; # preferably have JSON::XS installed
my %prepared_conf = %$conf;
for my $field ( 'parameters', 'alerts' ) {
$prepared_conf{$field} =
JSON::to_json( $prepared_conf->{$field}, { 'ascii' => 1 } );
}
my $req = HTTP::Request::Common::POST($url,'content' => \%prepared_conf);

Related

POST API in Perl using LWP::UserAgent with authentication

I am trying to use POST method in perl to send information to an API.
I would like to call the below api which requires following inputs:
URI: https://www.cryptopia.co.nz/api/SubmitTrade
Input Parameters are:-
Market: The market symbol of the trade e.g. 'DOT/BTC' (not required if 'TradePairId' supplied)
TradePairId: The Cryptopia tradepair identifier of trade e.g. '100' (not required if 'Market' supplied)
Type: the type of trade e.g. 'Buy' or 'Sell'
Rate: the rate or price to pay for the coins e.g. 0.00000034
Amount: the amount of coins to buy e.g. 123.00000000
Please can you tell me how I can call this api from perl ?
Request Structure:
REQUEST_SIGNATURE: API_KEY + "POST" + URI + NONCE + HASHED_POST_PARAMS
API_KEY: Your Cryptopia api key
URI: the request uri. e.g. https://www.cryptopia.co.nz/Api/SubmitTrade
HASHED_POST_PARAMS: Base64 encoded MD5 hash of the post parameters
NONCE: unique indicator for each request.
So question is how do I join this api https://www.cryptopia.co.nz/api/SubmitTrade and pass the arguments to it with authentication and check if returned result is success?
Result Example:
{
"Success":true,
"Error":null,
"Data":
{
"OrderId": 23467,
"FilledOrders": [44310,44311]
}
}
I don't have a Cryptopia account to test this, but this might at least get you closer to a working prototype.
Setup
Load the required modules and create a LWP::UserAgent object.
use strict;
use warnings;
use LWP::UserAgent;
use JSON;
use Digest::MD5 qw(md5);
use Digest::SHA qw(hmac_sha256);
use MIME::Base64;
use URI::Encode qw(uri_encode);
my $api_key = 'PUBLIC KEY';
my $api_private_key = 'PRIVATE KEY';
my $ua = LWP::UserAgent->new;
Generating the request
The tricky part is generating the authorization header. The API keys for your account will be base64-encoded - hence the call to decode_base64() when using it to sign the request.
my $url = "https://www.cryptopia.co.nz/api/GetTradeHistory";
my %req = (
Market => 'DOT/BTC',
);
my $nonce = int(rand(1000000));
my $post_data = encode_json(\%req);
my $post_data_enc = encode_base64(md5($post_data), "");
my $encoded_url = lc(uri_encode($url, encode_reserved => 1));
my $req_signature = sprintf("%sPOST%s%s%s", $api_key, $encoded_url, $nonce, $post_data_enc);
# Sign request signature with private key.
my $req_signature_hmac = encode_base64(hmac_sha256($req_signature, decode_base64($api_private_key)));
# Generate value for 'Authorization' header field.
my $auth_header_value = sprintf("amx %s:%s:%s", $api_key, $req_signature_hmac, $nonce);
Notes ...
An empty string is passed as the second argument to encode_base64() - to prevent it from appending a new-line to the output.
uri_encode() from the URI::Encode module is used the encode the URL.
You might want to use an alternative source or random generator for the nonce.
Sending the request
You could create a HTTP::Request object explicitly and pass that to LWP::UserAgent's request() method, but there is a post() convenience method which does all this for you.
The example here uses the Content parameter to set the content of the post body, and sets the Authorization and Content-Type headers at the same time.
my $response = $ua->post($url,
Content => $post_data,
'Content-Type' => 'application/json',
Authorization => $auth_header_value
);
die "Request failed: ", $response->content unless $response->is_success();
print $response->content, $/;
Checking the response
It may be enough to check the LWP Response object as above - by invoking the $response->is_success() method. Nonetheless, if you want to explicitly check for success, just decode the JSON response - using $resp = decode_json($response->decoded_content). Then examine the relevant keys in the resulting hash.

Using variable for HTTP request headers with Perl

I am trying to write a function to create HTTP requests (POST and GET mostly) in Perl. I am keeping everything generic by using variables so that I don't have to worry about the type of request, the payload, headers, etc, however HTTP::Request->header() doesn't seem to like my variable:
my($req_type, $headers, $endpoint, $args, $request, $jsonfile) = #_;
my $ua = LWP::UserAgent->new;
my $req = HTTP::Request->new($req_type => $endpoint);
$req->content_type('application/json');
foreach (#$headers) {
$req->push_header($_);
}
$req->content($args);
$req->content($request);
print "request : ".$req->as_string;
I tried a few different approches, and using push_header got me the closest, but I realize it may not be the best solution. I think it might have something to do with single quotes getting passed in:
#headers = "'x-auth-token' => '$_token'";
I can post more of the code if it is helpful. I'm hoping some Perl guru will know exactly what I'm doing wrong. I'm sure it's something to do with the format of the string I'm passing in.
#headers = "'x-auth-token' => '$_token'";
The header function expects to be passed two arguments. The header name and the header value.
You are passing it one argument: a string containing a fragment of Perl code.
You need to format your data more sensibly.
my %headers = (
"x-auth-token" => $_token;
);
and
foreach my $header_name (keys %headers) {
$req->push_header($header_name => $headers{$header_name});
}

How may I bypass LWP's URL encoding for a GET request?

I'm talking to what seems to be a broken HTTP daemon and I need to make a GET request that includes a pipe | character in the URL.
LWP::UserAgent escapes the pipe character before the request is sent.
For example, a URL passed in as:
https://hostname/url/doSomethingScript?ss=1234&activities=Lec1|01
is passed to the HTTP daemon as
https://hostname/url/doSomethingScript?ss=1234&activities=Lec1%7C01
This is correct, but doesn't work with this broken server.
How can I override or bypass the encoding that LWP and its friends are doing?
Note
I've seen and tried other answers here on StackOverflow addressing similar problems. The difference here seems to be that those answers are dealing with POST requests where the formfield parts of the URL can be passed as an array of key/value pairs or as a 'Content' => $content parameter. Those approaches aren't working for me with an LWP request.
I've also tried constructing an HTTP::Request object and passing that to LWP, and passing the full URL direct to LWP->get(). No dice with either approach.
In response to Borodin's request, this is a sanitised version of the code I'm using
#!/usr/local/bin/perl -w
use HTTP::Cookies;
use LWP;
my $debug = 1;
# make a 'browser' object
my $browser = LWP::UserAgent->new();
# cookie handling...
$browser->cookie_jar(HTTP::Cookies->new(
'file' => '.cookie_jar.txt',
'autosave' => 1,
'ignore_discard' => 1,
));
# proxy, so we can watch...
if ($debug == 1) {
$browser->proxy(['http', 'ftp', 'https'], 'http://localhost:8080/');
}
# user agent string (pretend to be Firefox)
$agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.12) Gecko/20050919 Firefox/1.0.7';
# set the user agent
$browser->agent($agent);
# do some things here to log in to the web site, accept session cookies, etc.
# These are basic POSTs of filled forms. Works fine.
# [...]
my $baseURL = 'https://hostname/url/doSomethingScript?ss=1234&activities=VALUEA|VALUEB';
#values = ['Lec1', '01', 'Lec1', '02'];
while (1) {
if (scalar(#values) < 2) { last; }
my $vala = shift(#values);
my $valb = shift(#values);
my $url = $basEURL;
$url =~ s/VALUEA/$vala/g;
$url =~ s/VALUEB/$valb/g;
# simplified. Would usually check request for '200' response, etc...
$content = $browser->get($url)->content();
# do something here with the content
# [...]
# fails because the '|' character in the url is escaped after it's handed
# to LWP
}
# end
As #bchgys mentions in his comment, this is (almost) answered in the linked thread. Here are two solutions:
The first and arguably cleanest one is to locally override the escape map in URI::Escape to not modify the pipe character:
use URI;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new();
my $res;
{
# Violate RFC 2396 by forcing broken query string
# local makes the override take effect only in the current code block
local $URI::Escape::escapes{'|'} = '|';
$res = $ua->get('http://server/script?q=a|b');
}
print $res->request->as_string, "\n";
Alternatively, you can simply undo the escaping by modifying the URI directly in the request after the request has been created:
use HTTP::Request;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new();
my $req = HTTP::Request->new(GET => 'http://server/script?q=a|b');
# Violate RFC 2396 by forcing broken query string
${$req->uri} =~ s/%7C/|/;
my $res = $ua->request($req);
print $res->request->as_string, "\n";
The first solution is almost certainly preferable because it at least relies on the %URI::Escape::escapes package variable which is exported and documented, so that's probably as close as you're gonna get to doing this with a supported API.
Note that in either case you are in violation of RFC 2396 but as mentioned you may have no choice when talking to a broken server that you have no control over.

How to post array form using LWP

I am having problems creating an array that I can pass as a form using LWP. Basic code is
my $ua = LWP::UserAgent->new();
my %form = { };
$form->{'Submit'} = '1';
$form->{'Action'} = 'check';
for (my $i=0; $i<1; $i++) {
$form->{'file_'.($i+1)} = [ './test.txt' ];
$form->{'desc_'.($i+1)} = '';
}
$resp = $ua->post('http://someurl/test.php', 'Content_Type' => 'multipart/form-data'
, 'Content => [ \%form ]');
if ($resp->is_success()) {
print "OK: ", $resp->content;
}
} else {
print $claimid->as_string;
}
I guess I am not creating the form array correctly or using the wrong type as when I check the _POST variables in test.php nothing has been set :(
The problem is that for some reason you've enclosed your form values in single quotes. You want to send the data structure. E.g.:
$resp = $ua->post('http://someurl/test.php',
'Content_Type' => 'multipart/form-data',
'Content' => \%form);
You want to either send the hash reference of %form, not the has reference contained within an array reference as you had ([ \%form ]). If you had wanted to send the data as an array reference, then you'd just use[ %form ]` which populates the array with the key/value pairs from the hash.
I'd suggest that you read the documentation for HTTP::Request::Common, the POST section in particular for a cleaner way of doing this.

Can I pass GET string in UserAgent post method

I call in this mode:
my $ua = new LWP::UserAgent;
my $response= $ua->post('www.example.com', {param1=>'val1',param2=>'val2'...} );
Can I call the above in the same way passing the values in GET form?:
my $response= $ua->post('www.example.com?param=val1&param2=val2' );
It is because I'm using Firebug and when I go to Net tab under the "POST" tab it shows individual parameters as well as a GET string for POST submitted requests.
So I was wondering if I use GET string in this function call.
Parametersapplication/x-www-form-urlencoded
Itemid 4 option com_search
searchword dsd task search Source
Content-Type:
application/x-www-form-urlencoded
Content-Length: 53
searchword=dsd&task=search&option=com_search&Itemid=4
In short you can pass GET strings yes, but if your end code does not accept GET METHOD it will fail.
Also you might still need to specify some parameters since the post method asks for post(url,array_with_parameters).
sub post {
require HTTP::Request::Common;
my($self, #parameters) = #_;
my #suff = $self->_process_colonic_headers(\#parameters, (ref($parameters[1]) ? 2 : 1));
return $self->request( HTTP::Request::Common::POST( #parameters ), #suff );
}
Using along with HTTP::Request you can specify it at the content in the way you prefer:
# Create a user agent object
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->agent("MyApp/0.1 ");
# Create a request
my $req = HTTP::Request->new(POST => 'http://www.example.com');
$req->content_type('application/x-www-form-urlencoded');
$req->content('searchword=dsd&task=search&option=com_search&Itemid=4');
# Pass request to the user agent and get a response back
my $res = $ua->request($req);
# Check the outcome of the response
if ($res->is_success) {
print $res->content;
} else {
print $res->status_line, "\n";
}
Read more...