Why does Mechanize onerror callback set content to error message? - perl

Question

use WWW::Mechanize;
$mech = new WWW::Mechanize( onerror => \&mecherror );
$mech->get("http://stackoverflow.comxxxx");
print $mech->content;
sub mecherror {
$mech->get("http://stackoverflow.com");
}
The output on line 4 is an error string relating to the first failed get and not the content of the get executed in sub mecherror. Why?

Answer 1

The onerror callback of WWW::Mechanize is meant to supply a response to error
onerror => \&func
Reference to a die-compatible function, such as Carp::croak, that is called when there's a fatal error.
It is clearly not intended for recovery or any use of the object that is being constructed.
That said, your call in onerror works, but the object doesn't get to know about it.
use warnings;
use strict;
use feature 'say';
use WWW::Mechanize;
my $mech = new WWW::Mechanize( onerror => \&mecherror );
$mech->get("http://stackoverflow.comxxxx");
say $mech->content;
sub mecherror {
my $response = $mech->get("http://stackoverflow.com");
# say $mech->content;
say "response is " . ref($response);
say $response->decoded_content;
}
This shows that we duly got an HTTP::Response object, and prints out the page. Then we may hope to pass a reference to the callback to connect it to the calling code. However, a mechanism for this is not provided -- this is not supported. We are warned against messing with internals, though.
As for why the object isn't updated, it depends on the callback implementation. From the source code we see that the code reference goes into object's data and is run when needed via the wrapper
sub die {
my $self = shift;
return unless my $handler = $self->{onerror};
return $handler->(#_);
}
A lot of other code is involved when this triggers, while nothing is done here to change the object's state. That is just unsupported and may result in undefined behavior.
Note that the callback here knows what $mech is because it is global, so it has the right object.
To summarize discussions in comments, it is plausible that the page retrieved by the callback gets overwritten by the error message. We get to see this when invoking content, and it appears to be due to this part of the method (see source)
$content = $self->response()->decoded_content(charset => 'none');
The decoded_content method is from HTTP::Response, inherited via LWP::UserAgent, and the error message indeed seems to have come from that class. (Neither W::M nor LWP::UA have a method named "decode_content.") This is summarized in W::M::content page
$mech->content(...)
Returns the content that the mech uses internally for the last page fetched. Ordinarily this is the same as $mech->response()->decoded_content(), [ ... ]
However, we anyway couldn't rely on the object being in a consistent state, as discussed.