How to debug Bugzilla extension that doesn't appear to run? - perl

I need some help getting a bugzilla extension off the ground.
I want to hook into bug_format_comment (FWIW: to change some plain text automatically added as a comment when I commit to SVN to links to the respective SCM commit.)
Right now, nothing seems to happen when I manually add a comment to a bug.
Is there anything special I need to do to make the extension run, besides putting it in the /extensions/my-ext-name/ dir ?
How can I test if the extension is called at all?
I use an old version of Bugzilla (3.2.x). Is that hook even supported? (I can't find that info in the documentation).
Here's my complete Extension.pm file (I have no experience in Perl. I took the example of the hook from the example extension and ran from there)
package Bugzilla::Extension::Websvn-scmbug-autolink;
use strict;
use base qw(Bugzilla::Extension);
# This code for this is in ./extensions/Websvn-scmbug-autolink/lib/Util.pm
use Bugzilla::Extension::Websvn-scmbug-autolink::Util;
use URI::Escape;
our $VERSION = '0.01';
# See the documentation of Bugzilla::Hook ("perldoc Bugzilla::Hook"
# in the bugzilla directory) for a list of all available hooks.
sub install_update_db {
my ($self, $args) = #_;
}
sub bug_format_comment {
my ($self, $args) = #_;
my $regexes = $args->{'regexes'};
# push(#$regexes, { match => qr/\bfoo\b/, replace => 'bar' });
# 6665 --> 6666
# CTUFramework:trunk/CTUCsharpRuntime/CtuFramework/text1-renamed.txt
#my $bar_match = qr/\b(bar)\b/;
my $bar_match = qr/(?:^|\r|\n)(\d+|NONE) (-->) (\d+|NONE)[ \r\n\t]+([^:]+):(.*?)[\r\n]/s; #/s - treat as single line
push(#$regexes, { match => $bar_match, replace => \&_replace_bar });
my $scm_match2 = qr/(?:^|\r|\n)(\d+|NONE) (-->) (\d+|NONE)[ \r\n\t]+([^:]+):(.*?)[\r\n]/s; #/s - treat as single line
push(#$regexes, { match => $scm_match2, replace => \&_replace_bar });
}
# Used by bug_format_comment--see its code for an explanation.
sub _replace_bar {
my $args = shift;
my $scmFromVer = $args->{matches}->[0];
my $scmToVer = $args->{matches}->[1];
my $scmArrow = $args->{matches}->[2];
my $scmProject = $args->{matches}->[3];
my $scmFile = $args->{matches}->[4];
# Remember, you have to HTML-escape any data that you are returning!
my $websvnRoot = "http://devlinux/websvn";
my $websvnRepo = uri_escape($scmProject); #maybe do a mapping
my $websvnFilePath = uri_escape("/".$scmFile);
my $fromRevUrl = sprintf("%s/revision.php?repname=%s&rev=%s",
$websvnRoot, $websvnRepo, $scmFromVer);
my $toRevUrl = sprintf("%s/revision.php?repname=%s&rev=%s",
$websvnRoot, $websvnRepo, $scmToVer);
my $diffUrl = sprintf("%s/diff.php?repname=%s&path=%s&rev=%s",
$websvnRoot, $websvnRepo, $websvnFilePath, $scmToVer);
my $fileUrl = sprintf("%s/filedetails.php?repname=%s&path=%s&rev=%s",
$websvnRoot, $websvnRepo, $websvnFilePath, $scmToVer);
# TODO no link for 'NONE'
my $fromRevLink = sprintf(qq{%s}, $fromRevUrl, $scmFromVer);
my $toRevLink = sprintf(qq{%s}, $toRevUrl, $scmToVer);
my $diffLink = sprintf(qq{%s}, $diffUrl, $scmArrow);
my $fileLink = sprintf(qq{%s}, $fileUrl, $scmFilePath);
# $match = html_quote($match);
return "$fromRevLink $diffLink $toRevLink:$fileLink";
};
__PACKAGE__->NAME;

As written, your extension won't even load: dashes are not valid in Perl package names.
Change the name from Websvn-scmbug-autolink to Websvn_scmbug_autolink.

I searched the Bugzilla sources, and found that the hook is simply not supported in version 3.2.x. The hook was introduced in Bugzilla 3.6: http://bzr.mozilla.org/bugzilla/3.6/revision/6762
PS. I hacked the regex replaces right in the template script for comments. Hacky, but it works.

Related

"sh: 1: file: not found" thrown in Perl

So this is an issue I see thrown around on several coding help-sites that always have a slight variation. I'm not entirely familiar with what it means, and what's even more curious is that this error is thrown midway through a larger Upload.pm script, and does not cause any sort of fatal error. It gets tossed into my error log somewhere during this unless conditional snippet
# If this is the first slice, validate the file extension and mime-type. Mime-type of following slices should be "application/octet-stream".
unless ( defined $response{'error'} ) {
if ( $slice->{'index'} == 1 ) {
my ($filename, $directory, $extension) = fileparse($path.$parent_file, qr/\.[^.]*/);
unless ( is_valid_filetype($slice->{'tmp_file'}, $extension) ) {
$response{'error'} = "Invalid file type.";
$response{'retry'} = 0;
}
}
}
Now, let me be perfectly honest. I don't really understand the error message, and I could really use some help understanding it, as well as solving it.
Our Perl based web app has refused to let us upload files correctly since upgrading to Debian Bullseye, and I've been stuck debugging this code I didn't write for a few days now. I'm wondering if the upgrade depreciated some Perl modules, or if the directories to said modules are no longer working?
I'm testing this in a Ubuntu based Docker environment running Debian Bullseye on an Apache 2 server.
If you need any more context, clarification, etc, please let me know.
is_valid_filetype() looks like this:
sub is_valid_filetype
{
my ($tmp_file, $extension) = #_;
if ( $tmp_file && $extension ) {
# Get temp file's actual mime-type.
my $mime = qx/file --mime-type -b '${tmp_file}'/;
$mime =~ s/^\s+|\s+$//g;
# Get valid mime-types matching this extension.
my $dbh = JobTracker::Common::dbh or die("DBH not available.");
my $mime_types = $dbh->selectrow_array('SELECT `mime_types` FROM `valid_files` WHERE `extension` = ?', undef, substr($extension, 1));
if ( $mime && $mime_types ) {
if ( $mime_types !~ /,/ ) {
# Single valid mime-type for this extension.
if ( $mime eq $mime_types ) {
return 1;
}
} else {
# Multiple valid mime-types for this extension.
my %valid_mimes = map { $_ => 1 } split(/,/, $mime_types);
if ( defined $valid_mimes{$mime} ) {
return 1;
}
}
}
}
return 0;
}
It's a message from sh (not Perl). It concerns an error on line 1 of the script, which was apparently an attempt to run the file utility. But sh couldn't find it.
The code in question executes this command using
qx/file --mime-type -b '${tmp_file}'/
Install file or adjust the PATH so it can be found.
Note that this code suffers from a code injection bug. It will fail if the string in $tmp_path contains a single quote ('), possibly resulting in the unintentional execution of code.
Fixed:
use String::ShellQuote qw( shell_quote );
my $cmd = shell_quote( "file", "--mime-type", "-b", $tmp_file" );
qx/$cmd/
Debian Bullseye was reading our CSV files as the wrong mime-type. It was interpreting the file command as application/csv, despite obviously not being an application.
This may be an actual bug in Bullseye, because both my boss and I have scoured the internet with no lucky finding anyone else with this issue. I may even report to Bullseye's devs for further awareness.
The fix was manually adding in our own mime-types that interpreted this file correctly.
It took us dumping the tmp directory to confirm the files existed, and triple checking I had my modules installed.
This was such a weird and crazy upstream issue that either of us could not have imaged it would be the file type interpretation at an OS level in Bullseye.
I really hope this helps someone, saves them the time it took us to find this.

Can't locate object method <var name> via package <file name> error

# Object.pm
sub update {
$table = $self->DB_TABLE;
...
}
The update function is triggered when a value is updated and seems to be executed multiple times by other files whose relevant parts look like:
# Status.pm
use constant DB_TABLE => 'Status';
# Flag.pm
use constant DB_TABLE => 'flag';
I don't know the inner workings of this project, but modified Flag.pm and Object.pm as below because I need to use a different table for updating flag.
# Flag.pm
use constant DB_TABLE => 'flag';
use constnat DB_UPDATE_TABLE => '<Table to use when updating flag>';
# Object.pm
sub update {
my $table = undef;
if($self->DB_UPDATE_TABLE) {
$table = $self->DB_UPDATE_TABLE;
} else {
$table = $self->DB_TABLE;
}
}
When I triggered sub update, I get
Can't locate object method "DB_UPDATE_TABLE" via package "<Status.pm>" at Object.pm.
Is there any way I can check if DB_UPDATE_TABLE exists in each file without error? I can add the following line to Status.pm, but there are a couple dozen of files like Status.pm.
use constant DB_UPDATE_TABLE => '';
I don't know why it is $self->DB_TABLE not $self->{DB_TABLE} but with the assumption that it is a method... tried the following, but it also had its own error.
if( my $ref = eval { $self->can( DB_UPDATE_TABLE ) } ) {
$table = $self->DB_UPDATE_TABLE;
} else {
$table = $self->DB_TABLE;
}
Bareword "DB_UPDATE_TABLE" not allowed while "strict subs" in use at Object.pm => I couldn't find the part to set 'strict subs'
Super close!
$self->can( DB_UPDATE_TABLE )
should be
$self->can("DB_UPDATE_TABLE")

Route to static file in Mojo

I have small app based on mojolicious. And I have index.html in public dir. I want to have route to this file when user asks for '/'.
I wrote two solution, but I don't like them.
First solution - add simple controller.
sub stratup {
//...
$r->get('/')->to('general#index_html');
//...
}
package MyPackage::General;
use Mojo::Base 'Mojolicious::Controller';
use strict;
use warnings;
sub index_html {
my $self = shift;
$self->render_static('index.html');
return;
}
1;
Second solution - add hook
sub startup {
my $self = shift;
$self->hook(before_dispatch => sub {
my $self = shift;
if ($self->req->url eq '/') {
$self->req->url( Mojo::URL->new('/index.html') );
}
});
What I want:
$r->get('/')->to('/index.html');
or something like that.
P.S. I know, than usualy nginx/apache do it, but I use morbo to run code.
You want:
$r->get('...')->to(cb => sub {
my $c = shift;
$c->reply->static('index.html')
});
(As long as you're after Mojolicous 5.45 2014-09-26)
By far the simplest way is
get "/" => "index";
I'll dig this up from the graveyard, why not.
I found myself similarly trying to serve a static html file in a docker container that I had using to serve both a Mojolicious REST API and a Vue.js front end. After searching around and piecing sporadic information together, this is what seems to work for me.
** disclaimer: I have not fully tested this with Vue routing and other aspects as yet.
My directory structure:
/app
/app/script
/app/modules/ui
/app/modules/ui/dist
From the command line the app directory, using morbo to test:
morbo script/ui.pl
ui.pl script
#!/usr/bin/env perl
use Mojolicious::Lite -signatures;
use Mojo::File qw(curfile);
use v5.25;
my $app = app;
my $static = $app->static;
push #{$static->paths}, curfile->dirname->sibling('modules/ui/dist')->to_string;
any '/' => sub {
my $c = shift;
my $content = $static->file("/index.html")->slurp;
$c->render(text => $content);
};
$app->start;
Using a combo of information from https://metacpan.org/pod/Mojolicious::Static and basic routing information at https://docs.mojolicious.org/Mojolicious/Lite, I could get the vue.js index page to render as expected.
** UPDATED A DAY LATER **
As it turns out, there is an easier way, though not clearly documented. If you place the static files inside your public folder, you can use the default helpers included with Mojolicious to render the files. The documentation refers to it here, https://docs.mojolicious.org/Mojolicious/Guides/Rendering#Serving-static-files, but it's not very clear on how to make it happen.
I tooled around some, but it took browsing the code of Controller.pm of for Mojolicious to sort it out. This section of the POD led me to determine how to get the reply object:
=head2 helpers
my $helpers = $c->helpers;
Return a proxy object containing the current controller object and on which helpers provided by /app can be called. This includes all helpers from Mojolicious::Plugin::DefaultHelpers and Mojolicious::Plugin::TagHelpers.
# Make sure to use the "title" helper and not the controller method
$c->helpers->title('Welcome!');
# Use a nested helper instead of the "reply" controller method
$c->helpers->reply->not_found;
Based on this, I can drop my files into the public folder:
/app/public/index.html
Then modify my controller to match:
# https://docs.mojolicious.org/Mojolicious/Guides/Rendering#Serving-static-files
any '/' => sub {
my $c = shift;
$c->helpers->reply->static('index.html');
};

Win32::IEAutomation will not click on links when IE >= 9

On a modern setup (Windows 7 64-bit, IE 11, ActiveState Perl 5.16 64-bit), the Click method in Win32::IEAutomation (v0.5) does not seem to work. Here is an example, lightly adapted from the documentation:
use Win32::IEAutomation;
my $ie = Win32::IEAutomation->new( visible => 1);
$ie->gotoURL('http://www.google.com');
$ie->getLink('linktext:', "About")->Click;
At this point, I should see the "About" page in IE. But I still see Google's home page in IE, and I cannot use the Content method in Win32::IEAutomation to get the source of the "About" page.
I have the same problem on an older setup (Vista SP2 64-bit, IE 9, ActiveState Perl 5.10.1). But the problem does not arise when I use a similar setup with IE8 instead of IE9. The problem thus seems to lie with a difference between IE8 and subsequent IE versions.
Is there anything that I can do to get the example script working with more recent versions of IE?
Win32::IEAutomation is a thin wrapper around the various interfaces exposed by InternetExplorer.Application and MSHTML.
Therefore, I tried to replicate the problem by writing a script to do the navigation without using Win32::IEAutomation. Using the click method on a link did not initiate navigation whereas passing its href to Navigate2 did.
The click method "Simulates a click by causing the HTMLFrameSiteEvents::onclick event to fire," meaning any onClick handlers defined on the page will be involved. I am not sure why specifically navigation is not being initiated.
However, the problem is not specific to Google's home page: I tried it with example.com, and invoking the click method on a link on that page did not initiate navigation either.
Here is the script I used as a testbed:
#!/usr/bin/env perl
use strict;
use warnings;
use feature 'say';
use Win32::OLE qw(EVENTS in valof);
$Win32::OLE::Warn = 3;
my $url = 'https://www.google.com/';
my %event_handler = (
DocumentComplete => \&onDocumentComplete,
);
my %page_handler = (
'https://www.google.com/'
=> \&onPageGoogleHome,
'https://www.google.com/intl/en/about/'
=> \&onPageGoogleAbout,
);
my $ie = Win32::OLE->new(
"InternetExplorer.Application", sub { $_[0]->Quit }
);
Win32::OLE->WithEvents($ie, \&Event, 'DWebBrowserEvents2');
$ie->{Visible} = 1;
$ie->Navigate2($url);
Win32::OLE->MessageLoop;
Win32::OLE->SpinMessageLoop;
$ie->Quit;
sub Event {
my ($ie, $event, #argv) = #_;
if (exists $event_handler{$event}) {
$event_handler{$event}->($ie, \#argv);
}
else {
# unhandled event
}
return;
}
sub onDocumentComplete {
my ($ie, $argv) = #_;
my $url = valof($argv->[-1]);
if (exists $page_handler{$url}) {
$page_handler{$url}->($ie, $argv);
}
else {
# unhandled page
}
return;
}
sub onPageGoogleHome {
my ($ie, $argv) = #_;
say "We are on Google's home page";
my $links = $ie->Document->links;
my $about_link;
for my $link (in $links) {
if ($link->innerText eq 'About') {
say "Found 'About' link";
$about_link = $link;
last;
}
}
if ($about_link) {
# Doesn't work:
# $about_link->click;
$ie->Navigate2($about_link->href);
}
return;
}
sub onPageGoogleAbout {
my ($ie, $argv) = #_;
say "Yay, we are on the about page!";
Win32::OLE->QuitMessageLoop;
return;
}
Version information:
This is perl 5, version 19, subversion 12 (v5.19.12) built for MSWin32-x64-multi-thread
Internet Explorer 11
Windows 8.1 Pro 64-bit
I observe the same faulty behavior with the ->Click() in Strawberry Perl v5.18.2 and Win32::IEAutomation v0.5 and IE v11.0.9600.17105.
My work around is to use the gotoURL() method directly. This obviously would not work with javascript actions, but does for this particular example.
use strict;
use warnings;
use Win32::IEAutomation;
my $ie = Win32::IEAutomation->new( visible => 1);
$ie->gotoURL('http://www.google.com');
my $about = $ie->getLink('linktext:' => 'About')
or die "Unable to find About";
# $about->Click(); # <--- does not work, using alternative method
$ie->gotoURL($about->linkUrl());

Perl mechanize Find all links array loop issue

I am currently attempting to create a Perl webspider using WWW::Mechanize.
What I am trying to do is create a webspider that will crawl the whole site of the URL (entered by the user) and extract all of the links from every page on the site.
But I have a problem with how to spider the whole site to get every link, without duplicates
What I have done so far (the part im having trouble with anyway):
foreach (#nonduplicates) { #array contain urls like www.tree.com/contact-us, www.tree.com/varieties....
$mech->get($_);
my #list = $mech->find_all_links(url_abs_regex => qr/^\Q$urlToSpider\E/); #find all links on this page that starts with http://www.tree.com
#NOW THIS IS WHAT I WANT IT TO DO AFTER THE ABOVE (IN PSEUDOCODE), BUT CANT GET WORKING
#foreach (#list) {
#if $_ is already in #nonduplicates
#then do nothing because that link has already been found
#} else {
#append the link to the end of #nonduplicates so that if it has not been crawled for links already, it will be
How would I be able to do the above?
I am doing this to try and spider the whole site to get a comprehensive list of every URL on the site, without duplicates.
If you think this is not the best/easiest method of achieving the same result I'm open to ideas.
Your help is much appreciated, thanks.
Create a hash to track which links you've seen before and put any unseen ones onto #nonduplicates for processing:
$| = 1;
my $scanned = 0;
my #nonduplicates = ( $urlToSpider ); # Add the first link to the queue.
my %link_tracker = map { $_ => 1 } #nonduplicates; # Keep track of what links we've found already.
while (my $queued_link = pop #nonduplicates) {
$mech->get($queued_link);
my #list = $mech->find_all_links(url_abs_regex => qr/^\Q$urlToSpider\E/);
for my $new_link (#list) {
# Add the link to the queue unless we already encountered it.
# Increment so we don't add it again.
push #nonduplicates, $new_link->url_abs() unless $link_tracker{$new_link->url_abs()}++;
}
printf "\rPages scanned: [%d] Unique Links: [%s] Queued: [%s]", ++$scanned, scalar keys %link_tracker, scalar #nonduplicates;
}
use Data::Dumper;
print Dumper(\%link_tracker);
use List::MoreUtils qw/uniq/;
...
my #list = $mech->find_all_links(...);
my #unique_urls = uniq( map { $_->url } #list );
Now #unique_urls contains the unique urls from #list.