I'm using WWW::Mechanize::Firefox to crawl pages that load some JavaScript after they have been loaded.
My code regarding this problem:
my ($firemech) = WWW::Mechanize::Firefox->new(tab => 'current', );
$firemech->get($url);
die "Cannot connect to $url\n" if !$firemech->success();
print "I'm connected!\n";
my ($retries) = 10;
while ($retries-- and ! $firemech->is_visible( xpath => '//*[#class="areaMapC"]' )) {
sleep 1;
}
die "Timeout" unless $retries;
my ($content) = $firemech->content();
Everything goes fine for the first page, it loads the page and recognizes the class that loads later. Then, it changes to the second page correctly (I see Firefox changing it), but it never gets to the "I'm connected!" print.
I checked the URL and that the wanted class is there, the code it's the same for both the pages (except some IDs which I want to extract). I really have no clue what's going wrong.
A page of this kind is HERE.
I solved this one by simply using
my ($firemech) = WWW::Mechanize::Firefox->new();
Instead of what I used before. It still keeps the content in one tab so it works perfectly with my needs.
Related
I have a strange problem with the Perl Debugger in EPIC. When I started using it, all worked fine as expected. I would set breakpoints, run a program in debug mode, step through it ... Then, something must have changed but I have no idea what. Now, the debugger or EPIC or Eclipse, I don't know who, sends tons of output to the console window that have nothing to do with my program but with the - seemingly to me - internals of EPIC or the Perl debugger. Here's a little cutout from this output, there is an endless amount of this and the things my program wants to output just get lost.
What is the problem here? I did not find any setting I could change in the preferences, I don't know what to search for, I couldn't find anything in the EPIC docs.
Edit: one thing I remember is I used "use diagnostics;" in another program that belongs to the project, but that is not run or used by the program that is producing this debug output. Maybe it has something to do with that? Can it "get stuck" and be remembered by the debugger later? I know it sounds silly, but that's all I got.
DB<396> ;{
do 'dumpvar_epic.pm' unless defined &dumpvar_epic::dump_lexical_vars;
my $offset = 0;
my $savout = CORE::select($DB::OUT);
dumpvar_epic::dump_lexical_vars($offset);
CORE::select($savout);
};
12|$forceingest|1|1|17|SCALAR(0x40fb5a4)|3|'0'|1|3
4|$lll|1|2|14|REF(0x2739ff4)|37|Log::Log4perl::Logger=HASH(0x41bbe64)|3|...|1|3
9|$reingest|1|1|17|SCALAR(0x40fbf14)|3|'1'|1|3
13|$secondsPause|1|1|17|SCALAR(0x40fb774)|5|'300'|1|5
4|$tdh|1|2|14|REF(0x40ff0dc)|15|GLOB(0x42e7484)|3|...|1|3
12|$transferDir|1|1|17|SCALAR(0x40f9f24)|31|'c:/temp/xml/transfer/probleme'|2|31
8|$usedata|1|1|17|SCALAR(0x40fbf84)|3|'1'|1|3
6|%datah|1|1|15|HASH(0x40f2f8c)|3|...|1|3
5|#data|1|1|16|ARRAY(0x41002ec)|3|...|1|3
13|#transactions|1|1|16|ARRAY(0x40ff07c)|3|...|1|3
14|#transferFiles|1|1|16|ARRAY(0x40ff04c)|3|...|1|3
7|#tx_ref|1|1|16|ARRAY(0x40feffc)|3|...|1|3
DB<397> ;{
do 'dumpvar_epic.pm' unless defined &dumpvar_epic::dump_lexical_vars;
my $offset = 0;
my $varexpr = <<'EOT';
${$h->{'$lll'}}
EOT
my $subref = \&dumpvar_epic::dump_hash_expr;
my $savout = CORE::select($DB::OUT);
my $savbuf = $|;
$| = 0;
$subref->($offset, $varexpr);
$| = $savbuf;
print "";
CORE::select($savout);
};
3|ALL|1|2|14|REF(0x41d9f64)|15|CODE(0x41dbee4)|3|...|1|3
5|DEBUG|1|2|14|REF(0x41d9ba4)|15|CODE(0x41dbee4)|3|...|1|3
5|ERROR|1|2|14|REF(0x41d737c)|15|CODE(0x41d9b34)|3|...|1|3
5|FATAL|1|2|14|REF(0x41d735c)|15|CODE(0x41d9b34)|3|...|1|3
4|INFO|1|2|14|REF(0x41d748c)|15|CODE(0x41d9b34)|3|...|1|3
3|OFF|1|2|14|REF(0x41d9fe4)|15|CODE(0x41d9b34)|3|...|1|3
5|TRACE|1|2|14|REF(0x41d9ee4)|15|CODE(0x41dbee4)|3|...|1|3
4|WARN|1|2|14|REF(0x41d717c)|15|CODE(0x41d9b34)|3|...|1|3
10|additivity|1|1|17|SCALAR(0x41d52bc)|3|'1'|1|3
Never mind. There is indeed a setting for that under Preferences/Perl EPIC/Enable Debugger Console. I enabled that setting previously because of another weird problem with the debugger (threads wouldn't terminate any more and hang there until Eclipse was restarted), which seemed to work. Disabling it stops the unwanted output. Maybe there's something really weird going on with Perl/EPIC on my computer...
Using this Perl module, how would I wait for the page to load all the images? (I am trying to screenshot image which is part of the age)
I tried $agent->success and this only checks if the page is loaded (doesn't check the images).
I don't think there is a wait method in the module but you can check if the element is visible yet and loop through and wait for it to be visible.
FOO: {while(1){
if($gent->is_visible($element)){
print "Image Downloaded\n";
last FOO;
}
else{
print "Downloading image...\n"; #This might print until image is downloaded
sleep(10);
}
}}
To get started, I am inexperienced scripting in perl, or using gtk, but I've been googling and researching how to for the past two or so weeks. It was difficult just figuring out where I could find the PMs for gtk on windows, and then even more so getting it to some semblance of 'working'. However, there have of course still been problems.
Skipping the above, I have two problems. For a slight bit of relevant background, I am trying to port an mirc script over to xchat, but to do that I obviously need to learn a whole 'nother language.. but anyway. The two problems are thus:
The window consists of several buttons, labels, and text areas. However, the window is.. 'frozen' in time unless one clicks on the title bar and holds. Clicking a button does nothing, not even to show it has been clicked, unless of course the title bar is clicked and held.
I have no idea how to initialize multiple instances of the same window. I have of course tried researching, but it's either not out there or I just haven't found it yet. To be specific.. My mirc script requires that multiple instances be allowed to exist, but I need the buttons for the specific instance to only affect that instance.. and so on.
In regards to problem 1, I do not know if the .xml glade file is important, so I won't post it immediately. I will however post the code that calls it:
my $glade_file = "window3.xml";
my $glade = Gtk2::Builder->new();
$glade->add_from_file($glade_file);
sub charopen {
my $window = $glade->get_object('window1');
$glade->connect_signals(undef, $window);
my $hp_cur = $glade->get_object('HP_Cur');
$window->set("title"=>$_[0][1]);
$hp_cur->set("label"=>$ini->val($_[0][1],"HPC"));
$window->show();
}
Graphical interface design relies on event processing. To work properly, it is important to reserve a thread to process user events (keyboard, mouse clicks...). That is the aim of calling Gtk2->main() when user interface is ready to accept user interaction.
To make the event thread exiting the event loop, an event callback method may invoke Gtk2->main_quit()
The Gtk2::Builder creates Gtk widget hierarchy from XML. To get multiple instance of the same window, you have to create a builder for each one.
Then your event callback methods has to get information about which window has sent the event, and the $user_data parameter may be used in that aim.
Here is a code proposal with a simple button click callback which use Perl reference to a hash so you can pass as many information as you want between window creator code and event callbacks:
sub createWindow($)
my $windowTitle = $_[0];
my $windowBuilder = Gtk2::Builder->new();
$windowBuilder->add_from_file($glade_file);
my $window = $windowBuilder->get_object('window1');
my $hp_cur = $windowBuilder->get_object('HP_Cur');
# Create hash with data (alternative: use Class::Struct for better code)
my %window_user_data = {
"title" => $windowTitle,
"window" => $window,
"hp_cur" => $hp_cur };
# Pass hash reference as user data
$windowBuilder->connect_signals(\%window_user_data);
# prepare interface: set data model into view and then...
$window->show();
}
# Click callback method defined on a button in window
sub button_click_callback($$) {
my $button = $_[0];
my $window_user_data_ref = $_[1];
# get back data model from view
print "Click received from button on "
. $window_user_data_ref->{"title"} . "\n";
}
There is another way to handle callbacks per window but it requires more Perl skills: you can design a Perl package to create an object instance for a specific window, and use $windowbuilder->connect_signals ($user_data, $windowcallbackinstance). In that case, such an object is called controller, and you have built your graphical interface based on Model-View-Controller (MVC) pattern.
I am trying to load a configuration file into a hash during my PerlChildInitHandler and then access the values from PerlResponseHandler. However, even though the process number is the same, it seems that variables changed during the child_init() call revert back to their default values when handler() gets called.
The basic scenario is:
package StartupLog;
# the variable I'm testing
my $sticky = 0;
sub child_init {
$sticky = 1;
return 0;
}
sub handler {
warn __PACKAGE__ . " sticky = $sticky\n"; ### always says "0" but should say "1"
return 0;
}
1;
This was never answered, so eventually I moved on to using the PerlPostConfigHandler, which seemed to work acceptably. I can only assume it's something about the forking that happens in the PerlChildInitiHandler but, sorry to say, I gave up. Hope this helps someone in the future.
Generally, if you want to load something at childinit time, and access it in the response phase, you'd stuff it into a package global (like $My::variable = 'lols'). I've never tried to do it the way you are here. Did you try using our instead of my maybe?.
I am Using WaitforComplete() in watiN but it doesnt seems to work well. As it executes the next statement even if you have given longer time to wait. I am using thread.sleep() to stop my application until it gets the desired page or element. But the thing is pages are so much dynamic that sometimes it takes much longer time as specified.
Any better solution. Any thing that will catch the page return dynamically and dont go to execute next statments in application.
Sample of Code
'Show Details page
Assert.AreEqual("Confirmation", _internetExplorer.Title)
If (_internetExplorer.Button(Find.ById(New Regex("btnFinish"))).Exists) Then
_internetExplorer.Button(Find.ById(New Regex("btnFinish"))).Click()
Else
Assert.Fail("Could not Find Finish Booking Button on Confirmation Page")
End If
System.Threading.Thread.Sleep(100000)
'Show Booking Summary page
Assert.AreEqual("Display Booking", _internetExplorer.Title)
I want something that detect the return of page dynamically. instead of giving some constant value.
WaitForComplete only works well if there is a postback after some action. Otherwise you have to find something else to wait for. Following an example on how to wait for the specified title:
_internetExplorer.Element("title", "Confirmation").WaitUntilExists();
I would always prefer to use one of the WaitXXX methods instead of Thread.Sleep cause the WaitXXX methods do only wait until the contraint is met. Where as Sleep waits for the time you specified. If its to long, time is waisted. If its to short, problems arise.
HTH,
Jeroen
The WaitForComplete method esentially moves on once the browser has set it's readystate to comllete and the busy state to false.
What I typically do is to try and access what you need to, then perform a thread.sleep for say half a second, then try again. I also have a global timeout that quits after say 10 seconds.
int timeout = 20;
bool controlFound = false;
for (int i = 0; i < timeout; i++)
{
if (_internetExplorer.Button(Find.ById(New Regex("btnFinish"))).Exists)
{
_internetExplorer.Button(Find.ById(New Regex("btnFinish"))).Click();
controlFound = true;
break;
}
else
{
System.Threading.Thread.Sleep(500);
}
}
if (!controlFound)
{
Assert.Fail("Control not found");
}
If it is executing the next statement, it should be finding the corresponding element. I suggest posting a sample of the code you are trying.