I can connect to a remote url using the proxy from tsocks in the following way:
tsocks telnet host port
How can I do the same using the Perl's LWP::UserAgent module?
I've been so far trying it like this, but it doesn't work:
use strict;
use warnings;
use v5.16;
use LWP::UserAgent;
use HTTP::Request::Common;
use Data::Dumper;
#my $ua = LWP::UserAgent->new();
my $ua = LWP::UserAgent->new(timeout => 10,
ssl_opts => {
#verify_hostname => 0,
verify_hostname => 0,
SSL_verify_mode => '0x01',
SSL_version => 'SSLv23:!SSLv3:!SSLv2',
}
);
$ua->proxy(['http', 'https' ], 'https://proxy_host:proxy_port' );
my $request = GET ( 'https://remote_url', Accept => 'application/json' );
$request->authorization_basic( 'username', 'password' );
say $request->as_string();
my $response = $ua->request( $request );
say $response->as_string();
By the way, I don't have socks installed on this server. So I'll need to do it without them.
Thanks!
LWP::UserAgent and SOCKS
You may use LWP::Protocol::socks perl package make LWP::UserAgent SOCKS capable.
IMHO it is much better way for your own perl scripts.
tsocks may be better for legacy perl scripts you do not want to modify.
I found a solution which might be very specific for my case:
Before running the script, I just call tsocks like this:
tsocks script_name.pl
Related
I am writing a Perl script to POST an attachment to JIRA using
REST::Client to access the API
but I am getting an error.
use REST::Client;
use warnings;
use strict;
use File::Slurp;
use MIME::Base64;
my $user = 'user';
my $pass = 'pass';
my $url = "http://******/rest/api/2/issue/BugID/attachments";
my $client = REST::Client->new();
$client->addHeader( 'Authorization', 'Basic' . encode_base64( $user . ':' . $pass ) );
$client->addHeader( 'X-Atlassian-Token', 'no-check' );
$client->setHost( $url );
# my %header = ('Authorization' => 'Basic'. encode_base64($user . ':' . $pass),'X-Atlassian-Token' => 'no-check');
my $attachment = "C:\\Folder\\Test.txt";
$client->POST(
$url,
'Content_Type' => 'form-data',
'Content' => [ 'file' => [$attachment] ]
);
if ( $client->responseCode() eq '200' ) {
print "Updated\n";
}
# print the result
print $client->responseContent() . "\n";
The error I get is
REST::Client exception: headers must be presented as a hashref at C:\Users\a\filename.pl line 24.
As shown in the code, I have tried setting headers in different ways but I still get same error.
Please suggest if there is any other method.
I have tried using JIRA module but it gives error too.
According to the documentation, the POST method:
Takes an optional body content and hashref of custom request headers.
You need to put your headers in a hashref, e.g.:
$client->POST($url, $content, {
foo => 'bar',
baz => 'qux'
});
But...it looks like you're expecting REST::Client to use HTTP::Request::Common to construct a multipart/form-data request. Unfortunately, that's not the case, so you'll have to build the content by hand.
You could use HTTP::Request::Common directly like this:
use strict;
use warnings 'all';
use 5.010;
use HTTP::Request::Common;
use REST::Client;
my $client = REST::Client->new;
my $url = 'http://www.example.com';
my $req = POST($url,
Content_Type => 'form-data',
Content => [ file => [ 'foo.txt' ] ]
);
$client->POST($url, $req->content(), {
$req->headers->flatten()
});
But this is a bit convoluted; I would recommend dropping REST::Client and using LWP::UserAgent instead. REST::Client is just a thin wrapper for LWP::UserAgent with a few convenience features, like prepending a default host to all requests. In this case, it's just getting in the way and I don't think the conveniences are worth the trouble.
From the documentation:
POST ( $url, [$body_content, %$headers] )
And you're doing:
$client->POST(
$url,
'Content_Type' => 'form-data',
'Content' => [ 'file' => [$attachment] ]
);
So - passing a list of scalars, with an arrayref at the end.
Perhaps you want something like:
$client->POST(
$url,
$attachment,
{ 'Content-Type' => 'form-data' }
);
Note the {} to construct an anonymous hash for the headers.
Although you probably want to open and include the 'attachment', because there's nothing in REST::Client about opening files and sending them automagically.
I have a script that needs to send logfiles to a web server.
NOTE: Server side uploading is already done. This is the client that needs to upload to the server.
On the server side, I have the typical cgi script that accepts any .zip file I send it under the name "file" (Example input type="file" name="file" )
So my question is: Is there an easy way to upload a .zip file to a web server via perl?
Something simple would be ideal, like:
upload_file('http://wherever.com/upload.cgi', 'somefile.zip');
You can use HTTP::Request::Common and LWP::UserAgent like this:
use strict;
use warnings;
use HTTP::Request::Common;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new(
env_proxy => 1,
);
my $req = POST 'http://someserver.com/upload.cgi',
Content_Type => 'form-data',
Content => [ pageAction => 'upload', file => ['myfile.zip'] ];
$ua->request($req);
This can also be written as
use strict;
use warnings;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new(
env_proxy => 1,
);
$ua->request('http://someserver.com/upload.cgi',
Content_Type => 'form-data',
Content => [ pageAction => 'upload', file => ['myfile.zip'] ]
);
This is because $ua->post(...) is the same as $ua->request(POST ...) except the special :name headers are handled first.
Getting 500 handshaker error:443 over https. The host service I am sending XML to does not support TLS 1.2, they do support 1.0 and 1.1. Currently using LWP 6.03 on CentOS 6. Using the code below they claim I am still sending using TLS1.2
use LWP::UserAgent;
$ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 0,SSL_version => 'SSLv23:!TLSv12' });
$req = HTTP::Request->new(GET => 'https://secure-host-server');
$res = $ua->request($req);
if ($res->is_success) {
print $res->content;
} else {
print "Error: " . $res->status_line . "\n";
}
Is it possible to print the TLS version as it is sent to the host? Anything I can do to verify I am using TLS1.1?
Setting SSL_version via LWP::UserAgent would not work. I tried countless methods to try to get my code to send XML via TLSv1 without luck, The following code did the trick.
use Net::SSLGlue::LWP;
use IO::Socket::SSL;
my $context = new IO::Socket::SSL::SSL_Context(
SSL_version => 'tlsv1',
SSL_verify_mode => Net::SSLeay::VERIFY_NONE(),
);
IO::Socket::SSL::set_default_context($context);
use LWP::UserAgent;
use HTTP::Request::Common;
I have a problem. I asked this question yesterday but not the way I really wanted it to be done, so my apologies. What I want to do is post to a page email and password but that page also requires specific headers to be sent. (I believe they are called request headers). I tried LWP::UserAgent but I could only post data, not the headers with it. How would I do this? I am new to Perl. Thanks for your help.
You previously mentioned the user Agent and cookies. While you could add those to the request, that's usually something you want over multiple requests and thus can be set in the UA itself.
use HTTP::Cookies qw( );
use HTTP::Request::Common qw( POST );
use LWP::UserAgent qw( );
my $cookies = HTTP::Cookies->new();
$cookies->set_cookie(...);
my $ua = LWP::UserAgent->new(
agent => ...,
cookie_jar => $cookies,
);
my $request = POST(
'http://www.example.com/',
Referer => '...',
Content => [
username => ...,
...
],
);
my $response = $ua->request($request);
use strict;
use warnings;
use LWP::UserAgent qw( );
my $ua = LWP::UserAgent->new();
$ua->default_header('Cookie' => '...');
$ua->default_header('Referer' => '...');
my $response = $ua->post('url_goes_here', {username=>'test', password=>'pass'});
if($response->is_success)
{
print $response->as_string;
}
this worked for me. Hopefully someone can learn a thing or two from this answer. The trick is to use default_header
I have the Perl & LWP book, but how do I set the user-agent string?
This is what I've got:
use LWP::UserAgent;
use LWP::Simple; # Used to download files
my $u = URI->new($url);
my $response_u = LWP::UserAgent->new->get($u);
die "Error: ", $response_u->status_line unless $response_u->is_success;
Any suggestions, if I want to use LWP::UserAgent like I do here?
From the LWP cookbook:
use LWP::UserAgent;
$ua = new LWP::UserAgent;
$ua->agent("$0/0.1 " . $ua->agent);
# $ua->agent("Mozilla/8.0") # pretend we are very capable browser
$req = new HTTP::Request 'GET' => 'http://www.sn.no/libwww-perl';
$req->header('Accept' => 'text/html');
# send request
$res = $ua->request($req);
I do appreciate the LWP cookbook solution which mentions the subclassing solution with a passing reference to lwp-request.
a wise perl monk once said: the ole subclassing LWP::UserAgent trick
package AgentP;
use base 'LWP::UserAgent';
sub _agent { "Mozilla/8.0" }
sub get_basic_credentials {
return 'admin', 'password';
}
package main;
use AgentP;
my $agent = AgentP->new;
my $response = $agent->get( 'http://127.0.0.1/hideout.html' );
print $agent->agent();
the entry has been revised with some poor humor, use statement, _agent override, and updated agent print line.
Bonus material for the interested: HTTP basic auth provided with get_basic_credentials override, which is how most people come to find the subclassing solution. _methods are sacred or something; but it does scratch an itch doesn't it?