Pass perl file arguments to LWP HTTP request - perl

Here is my issue with handling argument Perl. I need to pass Perl argument argument to a http request (Webservice) whatever Argument given to the perl file.
perl wsgrep.pl -name=john -weight -employeeid -cardtype=physical
In the wsgrep.pl file, i need to pass the above arguments to http post params.
like below,
http://example.com/query?name=john&weight&employeeid&cardtype=physical.
I am using LWP Package for this url to get response.
Is there any good approach to do this?
Updated:
Inside the wsgrep.pl
my ( %args, %config );
my $ws_url =
"http://example.com/query";
my $browser = LWP::UserAgent->new;
# Currently i have hard-coded the post param arguments. But it should dynamic based on the file arguments.
my $response = $browser->post(
$ws_url,
[
'name' => 'john',
'cardtype' => 'physical'
],
);
if ( $response->is_success ) {
print $response->content;
}
else {
print "Failed to query webservice";
return 0;
}
I need to construct post parameter part from the given arguments.
[
'name' => 'john',
'cardtype' => 'physical'
],

Normally, to url-encode params, I'd use the following:
use URI;
my $url = URI->new('http://example.com/query');
$url->query_form(%params);
say $url;
Your needs are more elaborate.
use URI qw( );
use URI::Escape qw( uri_escape );
my $url = URI->new('http://example.com/query');
my #escaped_args;
for (#ARGV) {
my ($arg) = /^-(.*)/s
or die("usage");
push #escaped_args,
join '=',
map uri_escape($_),
split /=/, $arg, 2;
}
$url->query(#escaped_args ? join('&', #escaped_args) : undef);
say $url;

Related

Reuse LWP Useragent object with HTTP POST query in a for/while loop

I am using LWP Useragent to make multiple POST calls with basic Authorization, wherein POST URL parameters are read from a CSV file. Here is my code:
use strict;
use warnings;
use LWP::UserAgent;
use JSON 'from_json';
use MIME::Base64 'encode_base64';
use Data::Dumper;
my #assets;
my %data;
my $response;
my $csvfile = 'ScrappedData_Coins.csv';
my $dir = "CurrencyImages";
open (my $csv, '<', "$dir/$csvfile") || die "cant open";
foreach (<$csv>) {
chomp;
my #currencyfields = split(/\,/);
push(#assets, \#currencyfields);
}
close $csv;
my $url = 'https://example.org/objects?';
my %options = (
"username" => 'API KEY',
"password" => '' ); # Password field is left blank
my $ua = LWP::UserAgent->new(keep_alive=>1);
$ua->agent("MyApp/0.1");
$ua->default_header(
Authorization => 'Basic '. encode_base64( $options{username} . ':' . $options{password} )
);
my $count =0;
foreach my $row (#cryptoassets) {
$response = $ua->post(
$url,
Content_Type => 'multipart/form-data',
Content => {
'name'=>${$row}[1],
'lang' => 'en',
'description' => ${$row}[6],
'parents[0][Objects][id]' => 42100,
'Objects[imageFiles][0]' =>[${$row}[4]],
}
);
if ( $response->is_success ) {
my $json = eval { from_json( $response->decoded_content ) };
print Dumper $json;
}
else {
$response->status_line;
print $response;
}
}
sleep(2);
}
Basically, I want to reuse the LWP object. For this, I am creating the LWP object, its headers, and response objects once with the option of keep_alive true, so that connection is kept open between server and client. However, the response from the server is not what I want to achieve. One parameter value ('parents[0][Objects][id]' => 42100) seems to not get passed to the server in HTTP POST calls. In fact, its behavior is random, sometimes the parentID object value is passed, and sometimes not, while all other param values are passing correctly. Is this a problem due to the reusing of the LWP agent object or is there some other problem? Because when I make a single HTTP POST call, all the param values are passed correctly, which is not the case when doing it in a loop. I want to make 50+ POST calls.
Reusing the user-agent object would not be my first suspicion.
Mojo::UserAgent returns a complete transaction object when you make a request. It's easy for me to inspect the request even after I've sent it. It's one of the huge benefits that always annoyed my about LWP. You can do it, but you have to break down the work to form the request first.
In this case, create the query hash first, then look at it before you send it off. Does it have everything that you expect?
Then, look at the request. Does the request match the hash you just gave it?
Also, when does it go wrong? Is the first request okay but the second fails, or several are okay then one fails?
Instead of testing against your live system, you might try httpbin.org. You can send it requests in various ways
use Mojo::UserAgent;
use Mojo::Util qw(dumper);
my $hash = { ... };
say dumper( $hash );
my $ua = Mojo::UserAgent->new;
$ua->on( prepare => sub { ... } ); # add default headers, etc
my $tx = $ua->post( $url, form => $hash );
say "Request: " . $tx->req->to_string;
I found the solution myself. I was passing form parameter data (key/value pairs) using hashref to POST method. I changed it to arrayref and the problem was solved. I read how to pass data to POST method on CPAN page. Thus, reusing LWP object is not an issue as pointed out by #brian d foy.
CPAN HTTP LWP::UserAgent API
CPAN HTTP Request Common API

Running perl code using `atom` throws me Undefined subroutine &main::send_request

I'm writing my first perl script for the requirement
generate HTTP request against a particular web uri in succession using different URL scheme patterns
use HTTP::Request::Generator 'generate_requests';
use URI;
use HTTP::Request::Common;
use strict; # safety net
use warnings; # safety ne
use Test::LWP::UserAgent 'send_request';
use LWP::UserAgent 'send_request';
use Test::More;
use URI;
use HTTP::Request::Common;
use LWP::UserAgent;
my $g = generate_requests(
method => 'POST',
host => ['example.com','www.example.com'],
pattern => 'https://example.com/{bar,foo,gallery}/[00..99].html',
wrap => sub {
my( $req ) = #_;
# Fix up some values
$req->{headers}->{'Content-Length'} = 666;
},
);
while( my $r = $g->()) {
send_request( $r );
};
I'm using atom editor and activeperl on windows 10, I get following error from running above code.
Undefined subroutine &main::send_request called at C:\Users\ADMINI~1\AppData\Local\Temp\atom_script_tempfiles\0ac821e0-0886-11eb-9588-291dbc37d883 line 57.
I have already installed all necessary modules and lib but i think its unable to refer the method send_request. Pls assist.
NOTE
I have replaced real values in variable for privacy reasons.
UPDATE
I plan to use following module
pattern => 'https://example.{com,org,net}/page_[00..99].html', from
https://metacpan.org/pod/HTTP::Request::Generator.
LWP::UserAgent is an object-oriented module. It doesn't export functions. You want to call send_request like this:
my $ua = 'LWP::UserAgent'->new;
while ( my $r = $g->() ) {
$ua->send_request( $r );
}
That said, send_request is an undocumented internal method. I think it is probably more intended for people who are subclassing LWP::UserAgent. You probably want the request method instead.
my $ua = 'LWP::UserAgent'->new;
while ( my $r = $g->() ) {
my $response = $ua->request( $r );
}
Full code:
use strict;
use warnings;
use HTTP::Request::Generator 'generate_requests';
use LWP::UserAgent;
my $ua = 'LWP::UserAgent'->new;
my $gen = generate_requests(
method => 'POST',
host => [ 'example.com', 'www.example.com' ],
pattern => 'https://example.com/{bar,foo,gallery}/[00..99].html',
wrap => sub {
my ( $req ) = #_;
# Fix up some values
$req->{'headers'}{'Content-Length'} = 666;
},
);
while ( my $req = $gen->() ) {
my $response = $ua->request( $req );
# Do something with $response here?
}

how to pass variable to http post API in Perl?

My snippet as below, I googled and cannot find a solution to pass variable to
the post request in my quoted string.
Most of the google result are just pass plain Json key value string pairs to the
post content. But I need to pass the parameter to the inner Json value part and call the related REST api. any suggestions? Thanks!
#!/usr/bin/perl -w
use strict;
use warnings;
# Create a user agent object
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->agent("MyApp/0.1");
# Create a request
my $req = HTTP::Request->new(POST => 'https://oapi.dingtalk.com/robot/send?access_token=foofb73f');
$req->content_type('application/json');
my $var1="value from var";
# works fine
my $message = '{"msgtype": "text","text":{"content":"plain string ok"}}';
# failed to compile
# my $message = '{"msgtype": "text","text":{"content":$var1}}';
$req->content($message);
# Pass request to the user agent and get a response back
my $res = $ua->request($req);
# Check the outcome of the response
if ($res->is_success) {
print $res->content;
} else {
print $res->status_line, "n";
}
You didn't properly convert the value into a JSON string.
use Cpanel::JSON::XS qw( encode_json );
my $message = encode_json({ msgtype => "text", text => { content => $var1 } });

Perl JIRA POST error "headers must be presented as a hashref"

I am writing a Perl script to POST an attachment to JIRA using
REST::Client to access the API
but I am getting an error.
use REST::Client;
use warnings;
use strict;
use File::Slurp;
use MIME::Base64;
my $user = 'user';
my $pass = 'pass';
my $url = "http://******/rest/api/2/issue/BugID/attachments";
my $client = REST::Client->new();
$client->addHeader( 'Authorization', 'Basic' . encode_base64( $user . ':' . $pass ) );
$client->addHeader( 'X-Atlassian-Token', 'no-check' );
$client->setHost( $url );
# my %header = ('Authorization' => 'Basic'. encode_base64($user . ':' . $pass),'X-Atlassian-Token' => 'no-check');
my $attachment = "C:\\Folder\\Test.txt";
$client->POST(
$url,
'Content_Type' => 'form-data',
'Content' => [ 'file' => [$attachment] ]
);
if ( $client->responseCode() eq '200' ) {
print "Updated\n";
}
# print the result
print $client->responseContent() . "\n";
The error I get is
REST::Client exception: headers must be presented as a hashref at C:\Users\a\filename.pl line 24.
As shown in the code, I have tried setting headers in different ways but I still get same error.
Please suggest if there is any other method.
I have tried using JIRA module but it gives error too.
According to the documentation, the POST method:
Takes an optional body content and hashref of custom request headers.
You need to put your headers in a hashref, e.g.:
$client->POST($url, $content, {
foo => 'bar',
baz => 'qux'
});
But...it looks like you're expecting REST::Client to use HTTP::Request::Common to construct a multipart/form-data request. Unfortunately, that's not the case, so you'll have to build the content by hand.
You could use HTTP::Request::Common directly like this:
use strict;
use warnings 'all';
use 5.010;
use HTTP::Request::Common;
use REST::Client;
my $client = REST::Client->new;
my $url = 'http://www.example.com';
my $req = POST($url,
Content_Type => 'form-data',
Content => [ file => [ 'foo.txt' ] ]
);
$client->POST($url, $req->content(), {
$req->headers->flatten()
});
But this is a bit convoluted; I would recommend dropping REST::Client and using LWP::UserAgent instead. REST::Client is just a thin wrapper for LWP::UserAgent with a few convenience features, like prepending a default host to all requests. In this case, it's just getting in the way and I don't think the conveniences are worth the trouble.
From the documentation:
POST ( $url, [$body_content, %$headers] )
And you're doing:
$client->POST(
$url,
'Content_Type' => 'form-data',
'Content' => [ 'file' => [$attachment] ]
);
So - passing a list of scalars, with an arrayref at the end.
Perhaps you want something like:
$client->POST(
$url,
$attachment,
{ 'Content-Type' => 'form-data' }
);
Note the {} to construct an anonymous hash for the headers.
Although you probably want to open and include the 'attachment', because there's nothing in REST::Client about opening files and sending them automagically.

Clone request headers in Vanilla Perl CGI to LWP UserAgent

I have a perl CGI application that I want to take the users request headers, and turn those around into an LWP::UserAgent get request. Basically the goal is to replicate the incoming users headers and use those to make a separate request.
I've tried to create the headers myself but when I attempt to display the CGI headers and then my clone UserAgent headers, they aren't exactly the same. Here's what I got:
my $cgi = new CGI;
my %headers = map { $_ => $cgi->http($_) } $cgi->http;
my $req_headers = HTTP::Headers->new( %headers );
my $ua = LWP::UserAgent->new( default_headers => $req_headers );
print Dumper $ua->default_headers;
Basically, %headers and $ua->default_headers are not identical. $ua->default_headers has an agent that identifies itself as a perl script. I can manually set $ua->agent("") but there are other imperfections and the headers still aren't identical.
What's the best way to do what I want? There's got to be an easier solution...
It looks like the problem has to do with naming of incoming http headers compared to what HTTP::Headers uses.
The incoming parameters all have HTTP_ prefix in them where HTTP::Headers doesn't use that naming convention (which makes sense). Plus it looks like (a quick read in the code) that HTTP::Headers does the right thing in converting '-' into '_' for its own use.
I would recommended changing your map to following that removes the prefix:
# remove leading HTTP_ from keys, note: this assumes all keys have pattern
# HTTP_*
%headers = map { ( /^HTTP_(.*?)$/ ) => $cgi->http($_) } $cgi->http;
Here is the debugging script I used:
my $cgi = CGI->new;
my %headers = map { $_ => $cgi->http($_) } $cgi->http;
my $req_headers = HTTP::Headers->new( %headers );
my $ua = LWP::UserAgent->new( default_headers => $req_headers );
print "Content-type: text/plain\n\n";
print Dumper($ua->default_headers);
print Dumper( \%headers );
# remove HTTP_ from $_
%headers = map { ( /^HTTP_(.*?)$/ ) => $cgi->http($_) } $cgi->http;
$req_headers = HTTP::Headers->new( %headers );
$ua = LWP::UserAgent->new( default_headers => $req_headers );
print "headers part deux:\n";
print Dumper( $ua );
Hope that Helps