Perl: LWP::UserAgent Regarding rules for passing parameters - perl

e.g.
I have a command
curl -H "Content-Type: application/json" -H "Authorization: Bearer ACCESS_TOKEN" -X GET https://api.xxx/v1/datapackages/yyy
(1) I am confused when to use [ ] while passing the parameters. which one below is correct and why?
without [ ]
my $r2 = $ua -> get("https://api.xxx/v1/datapackages/yyy",
'Content-Type' => 'application/json',
Authorization => 'Bearer '.$token,
);
or with [ ]
my $r2 = $ua -> get("https://api.xxx/v1/datapackages/yyy",
[
'Content-Type' => 'application/json',
Authorization => 'Bearer '.$token,
]
);
(2) I see sometimes the parameter is quoted and sometimes not. Should "Authorization" be quoted and write as
with ' '
'Authorization' => 'Bearer '.$token,
or without ' '
Authorization => 'Bearer '.$token,
Thanks a lot!

The documentation for get() says this:
get
my $res = $ua->get( $url );
my $res = $ua->get( $url , $field_name => $value, ... );
This method will dispatch a GET request on the given URL. Further arguments can be given to initialize the headers of the request. These are given as separate name/value pairs. The return value is a response object. See HTTP::Response for a description of the interface it provides.
So, that just talks about passing arguments as key/value pairs. It says nothing about using an array reference. So, if I were you, I'd stick with the first approach.
The "fat comma" (=>) is documented in perldoc perlop (because it's an operator) which says:
The => operator (sometimes pronounced "fat comma") is a synonym for the comma except that it causes a word on its left to be interpreted as a string if it begins with a letter or underscore and is composed only of letters, digits and underscores. This includes operands that might otherwise be interpreted as operators, constants, single number v-strings or function calls. If in doubt about this behavior, the left operand can be quoted explicitly.
Otherwise, the => operator behaves exactly as the comma operator or list argument separator, according to context.
Basically, if the thing on its left is a single word (by which I mean a sequence of alphanumeric characters and underscores), then you can omit the quote marks around it.

Related

POST API in Perl using LWP::UserAgent with authentication

I am trying to use POST method in perl to send information to an API.
I would like to call the below api which requires following inputs:
URI: https://www.cryptopia.co.nz/api/SubmitTrade
Input Parameters are:-
Market: The market symbol of the trade e.g. 'DOT/BTC' (not required if 'TradePairId' supplied)
TradePairId: The Cryptopia tradepair identifier of trade e.g. '100' (not required if 'Market' supplied)
Type: the type of trade e.g. 'Buy' or 'Sell'
Rate: the rate or price to pay for the coins e.g. 0.00000034
Amount: the amount of coins to buy e.g. 123.00000000
Please can you tell me how I can call this api from perl ?
Request Structure:
REQUEST_SIGNATURE: API_KEY + "POST" + URI + NONCE + HASHED_POST_PARAMS
API_KEY: Your Cryptopia api key
URI: the request uri. e.g. https://www.cryptopia.co.nz/Api/SubmitTrade
HASHED_POST_PARAMS: Base64 encoded MD5 hash of the post parameters
NONCE: unique indicator for each request.
So question is how do I join this api https://www.cryptopia.co.nz/api/SubmitTrade and pass the arguments to it with authentication and check if returned result is success?
Result Example:
{
"Success":true,
"Error":null,
"Data":
{
"OrderId": 23467,
"FilledOrders": [44310,44311]
}
}
I don't have a Cryptopia account to test this, but this might at least get you closer to a working prototype.
Setup
Load the required modules and create a LWP::UserAgent object.
use strict;
use warnings;
use LWP::UserAgent;
use JSON;
use Digest::MD5 qw(md5);
use Digest::SHA qw(hmac_sha256);
use MIME::Base64;
use URI::Encode qw(uri_encode);
my $api_key = 'PUBLIC KEY';
my $api_private_key = 'PRIVATE KEY';
my $ua = LWP::UserAgent->new;
Generating the request
The tricky part is generating the authorization header. The API keys for your account will be base64-encoded - hence the call to decode_base64() when using it to sign the request.
my $url = "https://www.cryptopia.co.nz/api/GetTradeHistory";
my %req = (
Market => 'DOT/BTC',
);
my $nonce = int(rand(1000000));
my $post_data = encode_json(\%req);
my $post_data_enc = encode_base64(md5($post_data), "");
my $encoded_url = lc(uri_encode($url, encode_reserved => 1));
my $req_signature = sprintf("%sPOST%s%s%s", $api_key, $encoded_url, $nonce, $post_data_enc);
# Sign request signature with private key.
my $req_signature_hmac = encode_base64(hmac_sha256($req_signature, decode_base64($api_private_key)));
# Generate value for 'Authorization' header field.
my $auth_header_value = sprintf("amx %s:%s:%s", $api_key, $req_signature_hmac, $nonce);
Notes ...
An empty string is passed as the second argument to encode_base64() - to prevent it from appending a new-line to the output.
uri_encode() from the URI::Encode module is used the encode the URL.
You might want to use an alternative source or random generator for the nonce.
Sending the request
You could create a HTTP::Request object explicitly and pass that to LWP::UserAgent's request() method, but there is a post() convenience method which does all this for you.
The example here uses the Content parameter to set the content of the post body, and sets the Authorization and Content-Type headers at the same time.
my $response = $ua->post($url,
Content => $post_data,
'Content-Type' => 'application/json',
Authorization => $auth_header_value
);
die "Request failed: ", $response->content unless $response->is_success();
print $response->content, $/;
Checking the response
It may be enough to check the LWP Response object as above - by invoking the $response->is_success() method. Nonetheless, if you want to explicitly check for success, just decode the JSON response - using $resp = decode_json($response->decoded_content). Then examine the relevant keys in the resulting hash.

Using variable for HTTP request headers with Perl

I am trying to write a function to create HTTP requests (POST and GET mostly) in Perl. I am keeping everything generic by using variables so that I don't have to worry about the type of request, the payload, headers, etc, however HTTP::Request->header() doesn't seem to like my variable:
my($req_type, $headers, $endpoint, $args, $request, $jsonfile) = #_;
my $ua = LWP::UserAgent->new;
my $req = HTTP::Request->new($req_type => $endpoint);
$req->content_type('application/json');
foreach (#$headers) {
$req->push_header($_);
}
$req->content($args);
$req->content($request);
print "request : ".$req->as_string;
I tried a few different approches, and using push_header got me the closest, but I realize it may not be the best solution. I think it might have something to do with single quotes getting passed in:
#headers = "'x-auth-token' => '$_token'";
I can post more of the code if it is helpful. I'm hoping some Perl guru will know exactly what I'm doing wrong. I'm sure it's something to do with the format of the string I'm passing in.
#headers = "'x-auth-token' => '$_token'";
The header function expects to be passed two arguments. The header name and the header value.
You are passing it one argument: a string containing a fragment of Perl code.
You need to format your data more sensibly.
my %headers = (
"x-auth-token" => $_token;
);
and
foreach my $header_name (keys %headers) {
$req->push_header($header_name => $headers{$header_name});
}

How may I bypass LWP's URL encoding for a GET request?

I'm talking to what seems to be a broken HTTP daemon and I need to make a GET request that includes a pipe | character in the URL.
LWP::UserAgent escapes the pipe character before the request is sent.
For example, a URL passed in as:
https://hostname/url/doSomethingScript?ss=1234&activities=Lec1|01
is passed to the HTTP daemon as
https://hostname/url/doSomethingScript?ss=1234&activities=Lec1%7C01
This is correct, but doesn't work with this broken server.
How can I override or bypass the encoding that LWP and its friends are doing?
Note
I've seen and tried other answers here on StackOverflow addressing similar problems. The difference here seems to be that those answers are dealing with POST requests where the formfield parts of the URL can be passed as an array of key/value pairs or as a 'Content' => $content parameter. Those approaches aren't working for me with an LWP request.
I've also tried constructing an HTTP::Request object and passing that to LWP, and passing the full URL direct to LWP->get(). No dice with either approach.
In response to Borodin's request, this is a sanitised version of the code I'm using
#!/usr/local/bin/perl -w
use HTTP::Cookies;
use LWP;
my $debug = 1;
# make a 'browser' object
my $browser = LWP::UserAgent->new();
# cookie handling...
$browser->cookie_jar(HTTP::Cookies->new(
'file' => '.cookie_jar.txt',
'autosave' => 1,
'ignore_discard' => 1,
));
# proxy, so we can watch...
if ($debug == 1) {
$browser->proxy(['http', 'ftp', 'https'], 'http://localhost:8080/');
}
# user agent string (pretend to be Firefox)
$agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.12) Gecko/20050919 Firefox/1.0.7';
# set the user agent
$browser->agent($agent);
# do some things here to log in to the web site, accept session cookies, etc.
# These are basic POSTs of filled forms. Works fine.
# [...]
my $baseURL = 'https://hostname/url/doSomethingScript?ss=1234&activities=VALUEA|VALUEB';
#values = ['Lec1', '01', 'Lec1', '02'];
while (1) {
if (scalar(#values) < 2) { last; }
my $vala = shift(#values);
my $valb = shift(#values);
my $url = $basEURL;
$url =~ s/VALUEA/$vala/g;
$url =~ s/VALUEB/$valb/g;
# simplified. Would usually check request for '200' response, etc...
$content = $browser->get($url)->content();
# do something here with the content
# [...]
# fails because the '|' character in the url is escaped after it's handed
# to LWP
}
# end
As #bchgys mentions in his comment, this is (almost) answered in the linked thread. Here are two solutions:
The first and arguably cleanest one is to locally override the escape map in URI::Escape to not modify the pipe character:
use URI;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new();
my $res;
{
# Violate RFC 2396 by forcing broken query string
# local makes the override take effect only in the current code block
local $URI::Escape::escapes{'|'} = '|';
$res = $ua->get('http://server/script?q=a|b');
}
print $res->request->as_string, "\n";
Alternatively, you can simply undo the escaping by modifying the URI directly in the request after the request has been created:
use HTTP::Request;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new();
my $req = HTTP::Request->new(GET => 'http://server/script?q=a|b');
# Violate RFC 2396 by forcing broken query string
${$req->uri} =~ s/%7C/|/;
my $res = $ua->request($req);
print $res->request->as_string, "\n";
The first solution is almost certainly preferable because it at least relies on the %URI::Escape::escapes package variable which is exported and documented, so that's probably as close as you're gonna get to doing this with a supported API.
Note that in either case you are in violation of RFC 2396 but as mentioned you may have no choice when talking to a broken server that you have no control over.

Error in POST request generated by perl

I am getting an error trying to send a POST request from a perl script:
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->credentials($netloc,$realm,$username,$password);
use HTTP::Request::Common;
my $req = HTTP::Request::Common::POST($url,'content' => $conf);
$req->as_string() is
POST http:.....
Content-Length: 31003
Content-Type: application/x-www-form-urlencoded
.....&createTime=1361370652541&parameters=HASH(0x28fd658)&parameters=HASH(0x28fd670)&parameters=HASH(0x28fd6e8)&parameters=HASH(0x28fd760)&nodes=32&.....&alerts=HASH(0x632d00)&alerts=HASH(0x245abd0)&.....
the error I get is
Unexpected character ('H' (code 72)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
which makes me suspect that the problem is with the repeated
parameters=HASH(...) and alerts=HASH(...) elements;
instead I want to see something like
alerts=%5B%7B%22email%22%3A%22foo%40bar.com%22%2C%22type%22%3A%221%22%2C%22timeout%22%3A%22%22%7D%5D
$conf is a hash reference, and $conf->{"parameters"} and
$conf->{"alerts"}` are array references (whose elements are hash
references).
what am I doing wrong?
You can't post references; you presumably need to serialize them in some way; what is the server expecting?
From the error it looks like either the entire array or possibly each of the hashes in it should be serialized in JSON:
use JSON; # preferably have JSON::XS installed
my %prepared_conf = %$conf;
for my $field ( 'parameters', 'alerts' ) {
$prepared_conf{$field} =
JSON::to_json( $prepared_conf->{$field}, { 'ascii' => 1 } );
}
my $req = HTTP::Request::Common::POST($url,'content' => \%prepared_conf);

Having problem setting parameters for LWP::UserAgent

my %parameters = (
key => 'value'
);
my $response = $ua->get('http://example.com/i', %parameters);
I'm trying to get content of http://example.com/i?key=value,but after debugging I found the %parameters are stored in http headers instead of url parameters.
What's wrong in my code?
Though perldoc tells me that :
$ua->get( $url , $field_name => $value, ... )
But it should also work if I put those parameters in a %parameters,right?
The additional parameters to get are HTTP headers. For GET requests, arguments are included in the URL itself, URLencoded. You can use the URI module to create the appropriate URLs including GET variables, or construct them yourself (probably using URI::Escape to urlencode the values).
e.g.:
my %parameters = (
key => 'value'
);
my $url = URI->new("http://example.com/i");
$url->query_form(%parameters);
my $response = $ua->get($url);
From the fine manual:
$ua->get( $url )
$ua->get( $url , $field_name => $value, ... )
This method will dispatch a GET request on the given $url. Further arguments can be given to initialize the headers of the request.
Emphasis mine. You're misreading the documentation, the extra parameters for get() are HTTP header fields, not CGI parameters. If you want to include some CGI parameters then you'll have to add them to the URI yourself (preferably with URI).