How to handle errors in plack delayed response - perl

Tried to handle errors in delayed response.
Every time i send [200, [ 'Content-Type', 'application/json' ]
and got error before flushing the other things like that
$w->write("MyData");
$w->close();
i've got a warning in stdout and error in stderr, but page keeps loading.
it'll be loading until i stop app or stop page loading by hand.
how i can stop loading page in code or how to correctly handle errors in such apps where i use delayed response?
Perl version 5.24
Kelp version 1.02
Running Plack with Corona.
We're handling error throwing Exception::Class.
Catching errors with Try::Tiny.
Also tried eval and others things, but it doesn't work.
But changed Try::Tiny -> TryCatc and return if got any error, but
i need write return for every catch block, it looks very bad
#!/usr/bin/perl
use strict;
use warnings;
use Kelp::Less;
get '/hello' => sub {
return sub {
my $res = shift;
my $w = $res->([200, [ 'Content-Type', 'application/json' ]]);
my $data = 10 / 0;
$w->write("MyData");
$w->close();
}
};
run;
I'm looking for correct error handling,
do i need try{} catch{}; on every code that might fail?
Thanks #ikegami for answer, but page still loading after tries with Object::Destoyer and Sub::ScopeFinalizer. As i understand $w(writer) doesn't cause page loading. After exiting scope, $w comes undef then there's nothing to close, here is code.
#!/usr/bin/perl
use strict;
use warnings;
use Object::Destroyer;
use Kelp::Less;
get '/hello' => sub {
return sub {
my $res = shift;
my $w = $res->([200, [ 'Content-Type', 'application/json' ]]);
my $g = Object::Destroyer->new( sub { $w->close if $w } );
my $zzz = 1 / 0;
$w->write("DATA");
$w->close();
}
};
run;
so i've come up with that solution, what do you think?
#!/usr/bin/perl
use strict;
use warnings;
use Try::Tiny;
use Object::Destroyer;
use Kelp::Less;
get '/hello' => sub {
return sub {
my $res = shift;
my $w = $res->([200, [ 'Content-Type', 'application/json' ]]);
my $g = Object::Destroyer->new( sub { $w->close if $w; } );
my $ans = try {
my $zzz = 1 / 0;
}
catch {
print $_;
return;
};
return unless $ans;
$w->write("DATA");
$w->close();
}
};
run;

Solve this problem with wrapping app with
Plack::Middleware::HTTPExceptions

Related

Perl: An asynchronous http proxy via mojolicious

I made a simple http proxy, it's work fine, but not fast, because in the function handle_request, I use
my $tx = $ua->start( Mojo::Transaction::HTTP->new(req=>$request) );
to do the request, it's blocking.
I try to use a callback like:
$ua->start( Mojo::Transaction::HTTP->new(req=>$request) )=>sub{ ... }
to make it's non-blocking, and then, got a mistake:
'error' => { 'message' => 'Premature connection close'}
I guess that's because function handle_request return immediately, it does not wait the callback to be finished. If I use semaphore to wait the callback, that's mean it's blocking again.
#!/usr/bin/perl
use strict;
use warnings;
use utf8;
use Mojo::IOLoop::Server;
use Mojo::UserAgent;
use Mojo::Message::Response;
use Mojo::Message::Request;
use Mojo::Transaction::HTTP;
use Data::Dumper;
binmode STDOUT, ":encoding(UTF-8)";
my %buffer;
Mojo::IOLoop->server( {port => 3128} => sub {
my ($loop, $stream, $client) = #_;
$stream->on(
read => sub {
my ($stream, $chunk) = #_;
my $buffer = $buffer{$client}{read_buffer} .= $chunk;
if ($buffer =~ /^GET\s+|POST\s+|HEAD\s+(.*)\r\n\r\n$/i) {
$buffer{$client}{read_buffer} = '';
&handle_request($client,$stream,$buffer);
}
elsif ($buffer =~ /^CONNECT\s+(.*)\r\n\r\n$/i) {
$buffer{$client}{read_buffer} = '';
&handle_connect($stream,$buffer);
}
elsif($buffer{$client}{connection})
{
$buffer{$client}{read_buffer} = '';
Mojo::IOLoop->stream($buffer{$client}{connection})->write($chunk);
}
if(length($buffer)>= 20 *1024 * 1024) {
delete $buffer{$client};
Mojo::IOLoop->remove($client);
return;
}
});
});
sub handle_request{
my($client,$stream,$chunk) = #_;
my $request = Mojo::Message::Request->new;
$request = $request->parse($chunk);
my $ua = Mojo::UserAgent->new;
my $tx = $ua->start( Mojo::Transaction::HTTP->new(req=>$request) );
$stream->write( $tx->res->to_string );
}
sub handle_connect{
my ($stream, $chunk) = #_;
my $request = Mojo::Message::Request->new;
my $ua = Mojo::UserAgent->new;
$request = $request->parse($chunk);
print Dumper($request);
}
Mojo::IOLoop->start;
Hope to get some suggestions .
You have 2 problem:
You try to call nonblocking variant of $ua->start when your code have blocking style. Function handle_request must have callback as parameter.
If you have chain of callback then the best way to implement it is to use Mojo::IOLoop::Delay.
When you create variable $ua in non-blocking style in sub handle_request then your variable is destoyed by garbage collector because first execute exit of sub handle_request and $ua destroyed, because it is local variable and then get answer from $ua. So you get Premature connection close. You need to save instance of $ua elsewhere to prevent such error.
Upd.
I write bad variant of http/https proxy which work only via CONNECT method and have bug with not full first http message.
Upd.
I add another example of http/https proxy which correctly read first http message and work not only via CONNECT method.
Upd.
Oh, author of the Mojo wrote example of https proxy

Skip if a request takes too much time

I have the following code to request a header from an URL:
#!/usr/bin/env perl
use strict;
use warnings;
use LWP;
use Data::Dumper;
my $request = HTTP::Request -> new ( HEAD => 'http://www.vliruos.be/media/6352100/nss2015_annex_3_budget.xlsx' );
my $agent = LWP::UserAgent -> new;
my $response = $agent -> request ( $request );
print $response -> header ( 'Content-Length');
...
I don't know the reason, but the request seems very slow, it takes more than 10 seconds for me. I just want to implement a rule: if it does not return anything in 10 seconds, it should give up and resume the commands after the print.
Does anyone know how to implement this?
You could use SIGALRM.
$SIG{ALRM} = sub { die "timeout" };
eval {
alarm(10);
# long-time operations here
alarm(0);
};
if ($#) {
if ($# =~ /timeout/) {
# timed out; do what you will here
} else {
alarm(0); # clear the still-pending alarm
die; # propagate unexpected exception
}
}

perl html treebuilder how to handle error condition

The task is quite simple: access a url and parse it based on the result. In case there is an error (404, 500 etc etc), take appropriate action. The last piece is the one that I am having issue with.
I have listed both the pieces of code that I currently use. The longer one (LWP+TreeBuilder) works for both conditions ; the shorter one (TreeBuilder) works for the first condition but does not work for the error condition. If I use TreeBuilder and the site returns a 404 or some other error, the script simply exits ! Any ideas ?
Longer code that works
use LWP::Simple;
use LWP::UserAgent;
use HTML::TreeBuilder;
$url="http://some_url.com/blahblah" ;
$response = LWP::UserAgent->new->request( HTTP::Request->new( GET => $url ));
if ($response->is_success) {
$p = HTML::TreeBuilder->new();
$p->parse($response->content);
} else {
warn "Couldn't get $url: ", $response->status_line, "\n";
}
Shorter one that does not
use HTML::TreeBuilder;
$url="http://some_url.com/blahblah" ;
$tree = HTML::TreeBuilder->new_from_url($url) ;
To quote the docs:
If LWP is unable to fetch the URL, or the response is not HTML (as determined by content_is_html in HTTP::Headers), then new_from_url dies, and the HTTP::Response object is found in $HTML::TreeBuilder::lwp_response.
Try this:
use strict;
use warnings;
use HTML::TreeBuilder 5; # need new_from_url
use Try::Tiny;
my $url="http://some_url.com/blahblah" ;
my $p = try { HTML::TreeBuilder->new_from_url($url) };
unless ($p) {
my $response = $HTML::TreeBuilder::lwp_response;
if ($response->is_success) {
warn "Content of $url is not HTML, it's " . $response->content_type . "\n";
} else {
warn "Couldn't get $url: ", $response->status_line, "\n";
}
}
the script simply exits
No, it throws an exception. You could always catch the exception with eval BLOCK if you so desired.
my $tree = eval { HTML::TreeBuilder->new_from_url($url) }
or warn($#);

How to clean-up HTTP::Async if still in use

I am using Perl library HTTP::Async as follows:
use strict;
use warnings;
use HTTP::Async;
use Time::HiRes;
...
my $async = HTTP::Async->new( ... );
my $request = HTTP::Request->new( GET => $url );
my $start = [Time::HiRes::gettimeofday()];
my $id = $async->add($request);
my $response = undef;
while (!$response) {
$response = $async->wait_for_next_response(1);
last if Time::HiRes::tv_interval($start) > TIME_OUT;
}
...
When while loop timeout and script ends, I experience the the following error message:
HTTP::Async object destroyed but still in use at script.pl line 0
HTTP::Async INTERNAL ERROR: 'id_opts' not empty at script.pl line 0
What are my options? How can I "clean-up" HTTP::Async object if still in use, but not needed anymore?
I would suggest that you remove incomplete requests, but the module does not provide any interface to do so.
Option 1: Add removal functionality.
Add the following to your script:
BEGIN {
require HTTP::Async;
package HTTP::Async;
if (!defined(&remove)) {
*remove = sub {
my ($self, $id) = #_;
my $hashref = $self->{in_progress}{$id}
or return undef;
my $s = $hashref->{handle};
$self->_io_select->remove($s);
delete $self->{fileno_to_id}{ $s->fileno };
delete $self->{in_progress}{$id};
delete $self->{id_opts}{$id};
return $hashref->{request};
};
}
if (!defined(&remove_all)) {
*remove_all = sub {
my ($self) = #_;
return map $self->remove($_), keys %{ $self->{in_progress} };
};
}
}
You should contact the author and see if he can add this feature. $id is the value returned by add.
Option 2: Silence all warnings from the destructor.
If you're ok with not servicing all the requests, there's no harm in silencing the warnings. You can do so as follows:
use Sub::ScopeFinalizer qw( scope_finalizer );
my $async = ...;
my $anchor = scope_finalizer {
local $SIG{__WARN__} = sub { };
$async = undef;
};
...
Note that this will silence all warnings that occur during the object's destruction, so I don't like this as much.
It's not too hard to subclass HTTP::Async for a more general solution. I use this to be able to abort all pending requests:
package HTTP::Async::WithFlush;
use strict;
use warnings;
use base 'HTTP::Async';
use Time::HiRes qw(time);
sub _flush_to_send {
my $self = shift;
for my $request (#{ $self->{to_send} }) {
delete $self->{id_opts}->{$request->[1]};
}
$self->{to_send} = [];
}
sub _flush_in_progress {
my $self = shift;
# cause all transfers to time out
for my $id (keys %{ $self->{in_progress} }) {
$self->{in_progress}->{$id}->{finish_by} = time - 1;
}
$self->_process_in_progress;
}
sub _flush_to_return {
my $self = shift;
while($self->_next_response(-1)) { }
}
sub flush_pending_requests {
my $self = shift;
$self->_flush_to_send;
$self->_flush_in_progress;
$self->_flush_to_return;
return;
}
1;
This is (maybe) easier on using the module internals than the code by #ikegami.

How do I make parallel HTTP requests in Perl, and receive them back in order?

Using Perl, I'm looking for a simple way to perform a handful of HTTP requests in parallel, where I get responses back in the same order I sent them after they complete, e.g.:
my ($google, $perl) = foobar(GET => 'http://www.google.com/',
GET => 'http://www.perl.org/');
Is there a module I should be looking at?
I know I can do the bookkeeping by hand, but I feel spoiled after being able to do this using jQuery's when method, and I'd love to have as simple a solution using Perl.
Thanks for your help.
use threads;
use LWP::UserAgent qw( );
my $ua = LWP::UserAgent->new();
my #threads;
for my $url ('http://www.google.com/', 'http://www.perl.org/') {
push #threads, async { $ua->get($url) };
}
for my $thread (#threads) {
my $response = $thread->join;
...
}
The best part is that the parent doesn't wait for all requests to be completed. As soon as the right request is completed, the parent will unblock to process it.
If you used Parallel::ForkManager or something else where you can't wait for a specific child, you can use the following code to order the results:
for my $id (0..$#urls) {
create_task($id, $urls[$id]);
}
my %responses;
for my $id (0..$#urls) {
if (!exists($responses{$id})) {
my ($id, $response) = wait_for_a_child_to_complete();
$responses{$id} = $response;
redo;
}
my $response = delete($responses{$id});
...
}
I am a fan of Mojo!
From the Mojo::UserAgent documentation:
use Mojo;
use Mojo::UserAgent;
# Parallel requests
my $ua = Mojo::UserAgent->new;
$ua->max_redirects(5);
my $delay = Mojo::IOLoop->delay;
for my $url ('http://www.google.com/', 'http://www.perl.org/') {
$delay->begin;
$ua->get($url => sub {
my ($ua, $tx) = #_;
$delay->end($tx->res->dom);
});
}
my #responses = $delay->wait;
print join "\n", #responses
Enjoy!
EDIT
Btw. you do not have to process the responses at the end, you may do it in between:
# ...
$ua->get($url => sub {
my ($ua, $tx) = #_;
$delay->end(1);
# process $tx->res here
});
# ...
$delay->wait;