Using a proxy with WWW::Mechanize - perl

Im trying to perform tests on my servers upload function using several proxies. By the time I get my ip through api.ipify.org. The console outputs my true ip, not the proxies ip.
use strict; use warnings;
use WWW::Mechanize;
my $file = "proxies.txt";
open (FH, "< $file") or die "Can't open $file for read: $!";
my #lines = <FH>;
close FH or die "Cannot close $file: $!";
print "Loaded proxy list";
my $m = WWW::Mechanize->new(
autocheck => 1,
agent_alias => 'Mozilla',
cookie_jar => {},
ssl_opts => {verify_hostname => 0},
quiet => 0,
);
my $httpl = "http://";
$m->no_proxy('localhost');
my $ua = LWP::UserAgent->new;
for(my $i=0; $i < 47; $i++)
{
$m->proxy('http', $httpl . '' . $lines[$i]);
print "Connecting to proxy " . $lines[$i];
$m->get("https://api.ipify.org?format=json");
print $m->content;
for(my $j = 0; $j <= 10; $j++){
$m->get("http://example.com");
system("node genran.js");
$m->post('http://example.com/upload.php',
Content_Type => "form-data",
Content => [
'password' => '',
'public' => 'yes',
'uploadContent' => [ 'spam.txt', 'Love pecons', 'Content_$
file => [ 'x86.png', 'image_name', 'Content-Type' => 'ima$
]
);
print $m->content;
}}

$m->proxy('http', $httpl . '' . $lines[$i]);
print "Connecting to proxy " . $lines[$i];
$m->get("https://api.ipify.org?format=json");
You set only a proxy for http but make a https request. You need to set the proxy for https too like this:
$m->proxy('https', ... put your https proxy here ...);
Or to use the same proxy for multiple protocols:
$m->proxy(['http','https'], ... );
Also, make sure that you use at least version 6.06 of LWP::UserAgent and LWP::Protocol::https for proper support of proxies with https, i.e.
use LWP::UserAgent;
print LWP::UserAgent->VERSION,"\n";
use LWP::Protocol::https;
print LWP::Protocol::https->VERSION,"\n";

Related

Perl script as a router between icecast2 and VLC client

I wrote a Perl script to act as router between an icecast2 server and a VLC client. I start the script via
plackup --listen :8000 testStreamingServer.pl --debug
The problem is, I always get to the print "Starting server\n"; line but neither VLC, Firefox or curl can connect to the audio stream. There is no further info on the server side. No debug message, nothing.
Here is the code:
#!/usr/bin/env perl
use strict;
use warnings;
use Data::Dumper;
use LWP::UserAgent;
use Plack::Runner;
use File::Temp qw(tempfile);
my $source_url = 'http://server1.example.com/stream.mp3';
my $redirect_url = 'http://server2.example.com/stream.mp3';
my $ua = LWP::UserAgent->new();
print "Starting server\n";
my $app = sub {
print "Request received\n";
my $response = $ua->get($source_url);
if ($response->is_success) {
print "Source server is online, stream is playable\n";
my ($fh, $filename) = tempfile();
my $content_response = $ua->get($source_url, ':content_file' => $filename);
my $headers = $content_response->headers();
print Dumper $headers;
my $size = -s $filename;
open(my $file, "<", $filename);
my $file_content = do { local $/; <$file> };
close $file;
return [
200,
[ 'Content-Type' => 'audio/mpeg',
'Content-Length' => $size,
'icy-metaint' => 8192,
'icy-name' => 'My Stream',
'icy-description' => 'My stream description',
'icy-genre' => 'variety',
'icy-pub' => 1,
'icy-br' => 128,
'Connection' => 'close' ],
[ $file_content ],
];
} else {
print "Source server is offline, redirecting to backup stream\n";
return [
302,
[ 'Location' => $redirect_url, 'Connection' => 'close' ],
[],
];
}
};
my $runner = Plack::Runner->new;
$runner->parse_options(qw(--listen :8000 --server HTTP::Server::PSGI));
$runner->run($app);
print "Server started\n";
I tried print debugging, line by line. I am clueless right now.

How to write a client program for Net::WebSocket::Server program?

I have a server program that listens on 9000 port. But I can't find a way to write a client program for that server that connects server at 9000 port. Here is the main part of server program:
use strict;
use warnings;
use Net::WebSocket::Server;
my $port = "9000";
my $msg_count = 0;
print "starting server on $port \n\n";
my $count = 2400;
Net::WebSocket::Server->new(
listen => $port,
silence_max => 5,
tick_period => 300,
on_tick => sub {
my ($serv) = #_;
print "connections >> " . $serv->connections . "\n";
print $_->ip() for( $serv->connections() ); print "\n";
print $_->port() for( $serv->connections() ); print "\n\n";
$count++;
},
on_connect => sub {
my ($serv, $conn) = #_;
$conn->on(
handshake => sub {
my ($conn, $handshake) = #_;
my $tmp = $handshake->req->origin;
print "here ... $tmp \n\n";
},
utf8 => sub {
my ($conn, $msg) = #_;
my $IP = $conn->ip();
my $PORT = $conn->port();
my $SERVER = $conn->server();
my $SOCKET = $conn->socket();
my $str = Dumper $SOCKET;
I searched internet and what that sounds understandable to me is the following client program:
use strict;
use warnings;
use IO::Socket::SSL;
my $cl=IO::Socket::SSL->new("http://localhost:9000") or die "error=$!, ssl_error=$SSL_ERROR";
if($cl) {
$cl->connect_SSL or die $#;
# Something about certificates?
$cl->syswrite("Command");
close($cl);
}
But its not working. The error client program generates is as follows:
Expected 'PeerService' at client2.pl line 5.
I am newbie in Socket programming and currently understanding websockets programming in Perl.
Note: I am on windows platform.
I ran the example code suggested https://stackoverflow.com/questions/37318581/simple-perl-websocket-client. It gives error "Can't use an undefined value as a subroutine reference at C:/Strawberry/perl/site/lib/Protocol/WebSocket/Client.pm line 103.":
use strict;
use warnings;
use Protocol::WebSocket::Client;
my $client = Protocol::WebSocket::Client->new(url => 'ws://localhost:9000') or die "$!";
my $reply = "Free\n";
# Sends a correct handshake header
$client->connect or die "$!";
# Register on connect handler
$client->on(
connect => sub {
$client->write('hi there');
}
) or die "$!";
# Parses incoming data and on every frame calls on_read
$client->read($reply);
print "$reply\n";
# Sends correct close header
$client->disconnect;
Please investigate following demo code snippets for WebSocket Server and Client.
Note: please do not forget to alter code to match your server origin (ip address and port)
use strict;
use warnings;
use feature 'say';
use Net::WebSocket::Server;
my $origin = 'http://192.168.1.160:8080'; # server origin
my $port = 8080;
$| = 1;
say "Starting server on $port";
Net::WebSocket::Server->new(
listen => $port,
tick_period => 60,
on_tick => sub {
my ($serv) = #_;
my $stamp = 'Server time: ' . scalar localtime;
$_->send_utf8($stamp) for $serv->connections;
},
on_connect => sub {
my ($serv, $conn) = #_;
$conn->on(
handshake => sub {
my ($conn, $handshake) = #_;
$conn->disconnect() unless $handshake->req->origin eq $origin;
},
ready => sub {
my ($conn) = #_;
say "Client: connect IP $conn->{ip} PORT $conn->{port}";
my $msg = 'Connected server time is ' . scalar localtime . "\n";
$_->send_utf8($msg) for $conn->server->connections;
},
utf8 => sub {
my ($conn, $msg) = #_;
say "Client message: $conn->{ip} $msg";
$_->send_utf8('Server reply: ' . $msg)
for $conn->server->connections;
$conn->disconnect() if $msg eq 'exit';
},
binary => sub {
my ($conn, $msg) = #_;
$_->send_binary($msg) for $conn->server->connections;
},
pong => sub {
my ($conn, $msg) = #_;
$_->send_utf8($msg) for $conn->server->connections;
},
disconnect => sub {
my ($conn, $code, $reason) = #_;
say "Client: disconnect IP $conn->{ip} PORT $conn->{port}";
},
);
},
)->start;
Client
use strict;
use warnings;
use feature 'say';
use IO::Async::Loop;
use Net::Async::WebSocket::Client;
my $HOST = '192.168.1.160';
my $PORT = 8080;
my $loop = IO::Async::Loop->new;
my $client = Net::Async::WebSocket::Client->new(
on_text_frame => sub {
my ( $self, $frame ) = #_;
say $frame;
},
);
my $input = IO::Async::Stream->new_for_stdin(
on_read => sub {
my ( $self, $buffref, $eof ) = #_;
my $msg;
$msg = $1 while $$buffref =~ s/^(.*)\n//;
$client->send_text_frame( $msg );
$loop->loop_stop if $msg eq 'exit';
return 0;
},
);
$loop->add( $client );
$loop->add( $input );
$client->connect(
url => "ws://$HOST:$PORT/"
)->then( sub {
say 'Successfully connected to server';
$client->send_text_frame( scalar localtime );
})->get;
$loop->run;
say 'Bye, until next time';
exit 0;
References:
Net::WebSocket::Server
Net::Async::WebSocket::Client
IO::Async::Loop

Perl IO::Async parallel tcp connections

I've got a problem that I can't easily find the solution to for some reason.
I try to build multiple parallel TCP connections to a server via IO::Async.
My goal is to run TCP connections in parallel. The connections do not need to communicate between themselves but I need to catch and save output of them in a hash.
The following code is an example with a single connection.
#!/usr/bin/perl
use strict;
use warnings;
use IO::Async::Loop;
use IO::Async::Stream;
my $CRLF = "\x0d\x0a"; # because \r\n is not portable
my $HOST = shift #ARGV or die "Need HOST";
my $PORT = shift #ARGV or die "Need PORT";
my $loop = IO::Async::Loop->new;
my $socket;
$loop->connect(
host => $HOST,
service => $PORT,
socktype => 'stream',
on_connected => sub { $socket = shift },
on_resolve_error => sub { die "Cannot resolve - $_[0]\n" },
on_connect_error => sub { die "Cannot connect\n" },
);
$loop->loop_once until defined $socket;
# $socket is just an IO::Socket reference
my $peeraddr = $socket->peerhost . ":" . $socket->peerport;
print STDERR "Connected to $peeraddr\n";
# We need to create a cross-connected pair of Streams. Can't do that
# easily without a temporary variable
my ( $socketstream, $stdiostream );
$socketstream = IO::Async::Stream->new(
handle => $socket,
on_read => sub {
my ( undef, $buffref, $eof ) = #_;
while( $$buffref =~ s/^(.*)$CRLF// ) {
$stdiostream->write( $1 . "\n" );
}
return 0;
},
on_closed => sub {
print STDERR "Closed connection to $peeraddr\n";
$stdiostream->close_when_empty;
},
);
$loop->add( $socketstream );
$stdiostream = IO::Async::Stream->new_for_stdio(
on_read => sub {
my ( undef, $buffref, $eof ) = #_;
while( $$buffref =~ s/^(.*)\n// ) {
$socketstream->write( $1 . $CRLF );
}
return 0;
},
on_closed => sub {
$socketstream->close_when_empty;
},
);
$loop->add( $stdiostream );
$loop->await_all( $socketstream->new_close_future, $stdiostream->new_close_future );
How could I modify this code to handle an IP list as asynchronous connections and store output in a dedicated hash?
Finally maybe to limit max parallel connection to 100.
Any ideas?

Web Server with HTTP::Daemon, HTML not rendering

I figured out a way to create a quick web server in Perl:
#!/usr/bin/env perl -s -wl
use strict;
use HTTP::Daemon;
use HTTP::Headers;
use HTTP::Response;
sub help {
print "$0 -port=<port-number>";
}
our $port;
our $addr = "localhost";
$port = 9000 unless defined $port;
my $server = HTTP::Daemon->new(
LocalAddr => $addr,
LocalPort => $port,
Listen => 1,
Reuse => 1,
);
die "$0: Could not setup server" unless $server;
print "$0: http://$addr:$port Accepting clients";
while (my $client = $server->accept()) {
print "$0: Client received";
$client->autoflush(1);
my $request = $client->get_request;
print "$0: Client's Request Received";
print "$0: Request: " . $request->method;
if ($request->method eq 'GET') {
my $header = HTTP::Headers->new;
$header->date( time );
$header->server("$0");
$header->content_type('text/html');
my $content = "<!doctype html><html><head><title>Hello World</title></head><body><h1>Hello World!</h1></body></html>";
my $response = HTTP::Response->new(200);
$response->content($content);
$response->header("Content-Type" => "text/html");
$client->send_response($response);
}
print "$0: Closed";
$client->close;
undef($client);
}
But for some reason, every time I access localhost:9000 it displays part of the HTTP Header - date, server, content-length and content-type - and the content. It doesn't render it as an HTML page. Is there something I'm missing?
This is caused by the -l switch:
#!/usr/bin/env perl -s -wl
^
It sets the output record separator to the value of the input record separator (a newline), which results in additional newlines being added to HTTP server output, and a broken HTTP response.

Use proxy with perl script

I want to use a proxy with this perl script but I'm not sure how to make it use a proxy.
#!/usr/bin/perl
use IO::Socket;
$remote = IO::Socket::INET->new(
Proto => "tcp",
PeerAddr => "localhost",
PeerPort => "8080",
)
or die "cannot connect";
print $remote "GET / HTTP/1.0\n\n";
while ( <$remote> ) { print }
Use the LWP::UserAgent module, which has built-in proxy support.
use LWP::UserAgent;
$ua = LWP::UserAgent->new;
$ENV{HTTP_proxy} = "http://ip:port";
$ua->env_proxy; # initialize from environment variables
my $req = HTTP::Request->new(GET => 'http://google.com/');
print $ua->request($req)->as_string;
delete $ENV{HTTP_PROXY};
Straight from one of my scripts:
use LWP::UserAgent;
my($ua) = LWP::UserAgent->new;
if ($opts->{'proxy'}) {
my($ip) = Sys::HostIP->hostip;
if (($ip =~ m{^16\.143\.}) ||
($ip =~ m{^161\.}) ||
($ip =~ m{^164\.})) {
$ua->proxy(http => 'http://localhost:8080');
}
else {
$ua->proxy(http => "");
}
}
else {
$ua->env_proxy;
}
#***** get current entry *****
my($req) = HTTP::Request->new(GET => "http://stackoverflow.com/questions/1746614/use-proxy-with-perl-script");
my($raw) = $ua->request($req)->content;