"Insecure dependency error while running with -T switch" using cicindela2 - perl

I am applying the cicindela2 recommendation engine
It uses Apache mod_perl and the Perl DBI module.
Here is the rough flow of how it works
Data input by Record Handler
Data is passed through the filter chain for batch processing
Temporary tables are output from batch processing
Recommendation result is requested by accessing the Recommend Handler which trigger the action of Recommender
I configured an aggregation and ran the project batch script. I know that the batch processing succeeded because I saw the output of processing from DB. But when I tried to access the recommendation result with URL that triggers the Recommend Handler, I saw a blank white page and the log said
FATAL: Insecure dependency in parameter 1 of DBIx::ContextualFetch::db=HASH(0x7f2a76169e78)->prepare_cached method call while running with -T switch at /usr/local/share/perl5/Ima/DBI.pm line 398.
This is where the error was thrown from the
Ima::DBI
base module
/usr/local/share/perl5/Ima/DBI.pm.
sub _mk_sql_closure {
my ($class, $sql_name, $statement, $db_meth, $cache) = #_;
return sub {
my $class = shift;
my $dbh = $class->$db_meth();
# Everything must pass through sprintf, even if #_ is empty.
# This is to do proper '%%' translation.
my $sql = $class->transform_sql($statement => #_);
return $cache # Line 398
? $dbh->prepare_cached($sql)
: $dbh->prepare($sql);
};
}
It seems that the SQL query prepared by the program is insecure, right?
What is reason for this error?
Is it related to the function of cache management of DBI?
Would it be solved if I clear the cache regularly?
Also, I tried to log the SQL statement generated, but the output failed even when I placed something like $LOGGER->warn("123") in the handle subroutine of the Recommend Handler.
How come the log failed and how to log it correctly?

Insecure dependency... while running with -T switch is Perl's way of telling you that you're running with taint mode active and attempting to do something with tainted data which could be potentially unsafe. In this particular case, $sql is tainted, because some or all of its content came from sources external to the program - probably user input, although it could also have been read from a file.
To fix this, you need to think about where $sql came from, so that you can work out the appropriate way to clean it up.
In the most likely scenario, you've asked a user to supply search terms and then inserted those terms directly into your SQL string. This is a bad idea in general, as it opens you up to the possibility of SQL injection attacks. (Obligatory Bobby Tables link.) Revise your SQL handling to make use of SQL placeholders instead of inserting user input into the WHERE clause and this vulnerability should go away.
If tainted data is making its way into $sql in some other way, you need to clean up the tainted data by using a regular expression to validate it and capture the validated data, then assign the captured data to your variable. e.g.,
my $tainted = <STDIN>;
$tainted =~ /([A-Z]*)/; # Only allow uppercase characters
my $clean = $1; # No longer tainted because it came from $1
If you need to take this route, DO NOT use .* as your regex to untaint the data without serious, serious consideration, because, if you just blindly accept any and all data, you will be discarding any and all benefit provided by taint mode.

Related

Connectivity issue to Sybase DB from Perl script

I have to connect my Perl script to a newly constructed Sybase server version - 16.0.03.08.1019
Error - login Failed (due to encrypt password issue)
Previously the script was written in Perl:
$conn = Sybase::DBlib->new($user,$pass,$server,"$dbase Handle");
$conn->sql("use $dbase");
I searched online every where it is written put EncryptPassword=1.
I tried two ways shown below, but couldn't succeed.
$conn = Sybase::DBlib->new($user,$pass,$server,"$dbase Handle","EncryptPassword=1");
$conn = Sybase::DBlib->new("EncryptPassword=1",$user,$pass,$server,"$dbase Handle");
My question is, where to use EncryptPassword=1 in Perl script. Am I using it in correct place.
Wow! DBlib - that takes me back. When I last worked with DBlib (in about 1995), one of the tasks on my list was to replace all use of DBlib with CTlib - which was Sybase's new technology that was intended to replace DBlib. Soon after that, the system was rewritten again to use DBI and DBD::Sybase - which has been the recommended way to talk to Sybase databases from Perl programs for over twenty years. You'll note that the most recent release of sybperl (which is the CPAN distribution containing Sybase::DBlib and Sybase::CTlib) was over ten years ago. I'm pretty sure that Sybase themselves haven't supported DBlib since about the start of this millennium.
So, bearing in mind that you're using ancient technology that is largely dead and shouldn't be used, is there anything that can be done to help you without rewriting the whole system?
Perhaps.
Looking at the documentation for Sybase::DBlib, I see this example of how to write calls to new():
$dbh = new Sybase::DBlib [$user [, $pwd [, $server [, $appname [, {additional attributes}]]]]]
Ignore the fact that it's using the new Class syntax that any rational programmer would avoid - the Class->new() version is this:
$dbh = Sybase::DBlib->new([$user [, $pwd [, $server [, $appname [, {additional attributes}]]]]])
Note the "additional attributes" field at the end. I bet that's where your stuff needs to go. Note also, that it's { additional attributes } - so it looks like it expects a hash reference.
So it seems likely that the syntax you want is this:
$conn = Sybase::DBlib->new($user, $pass, $server, "$dbase Handle", {
EncryptPassword => 1,
});
Note that there are huge caveats in this. Not least, given that Sybase::DBlib has been unsupported for ten years, I wouldn't be at all surprised if it didn't support encrypted passwords at all.
But it might work. It's probably your best hope.
And please do whatever you can to update your codebase to use tools and libraries that haven't been unsupported for such a long time.

assigning a scalar the result of an SQL query

It seems to me that there simply has to be a better way of doing this, but i still haven't found one. And i'm sure i'm not the only one who could use a way to do this: Run an SQL query that only produces one field in one row, then assign that field to a scalar. (In my case, if the query results in more than one field/row, then i have bigger things to worry about than the script breaking).
For example, to get the timestamp from the SQL server, one could use:
my $timestamp;
my $cmd = $dbh->prepare('SELECT cast(now() AS timestamp);') or die $!;
$cmd->execute();
while (my #asd = $cmd->fetchrow_array) { $timestamp = $asd[0] }
Dirty, but it works. But using 4 lines seem a bit much for a simple assignment, especially considering how well perl and postgresql can communicate with eachother via DBI. Sure, i could write a subroutine for it, but isn't there something native that allows me to fetch data as easily as i submit data with $dbh->do() ?
And yes, i did try google.
Usually I write:
$value = $dbh->selectall_arrayref($sql)->[0]->[0];
There's always selectrow_array:
selectrow_array
#row_ary = $dbh->selectrow_array($statement);
#row_ary = $dbh->selectrow_array($statement, \%attr);
#row_ary = $dbh->selectrow_array($statement, \%attr, #bind_values);
This utility method combines prepare, execute and fetchrow_array into a single call.
So something like this:
my $timestamp = $dbh->selectrow_array('select cast(now() as timestamp)');
There's also selectrow_arrayref and selectrow_hashref for similar situations.
From perldoc DBI:
"selectrow_arrayref"
$ary_ref = $dbh->selectrow_arrayref($statement);
$ary_ref = $dbh->selectrow_arrayref($statement, \%attr);
$ary_ref = $dbh->selectrow_arrayref($statement, \%attr, #bind_values);
This utility method combines "prepare", "execute" and
"fetchrow_arrayref" into a single call. It returns the first row of
data from the statement. The $statement parameter can be a previously
prepared statement handle, in which case the "prepare" is skipped.
If any method fails, and "RaiseError" is not set, "selectrow_array"
will return undef.
That will get you most of the way. You still need to do some error checking, but you would be doing that anyway.
Wouldn't fetchrow_array actually only return a scalar as you're only asking for one column?

perl log db query errors into a log file

So I started to get familiar with Perl and I wrote my first Db script.
Now I am trying to select data from atable which is huge and trying to insert into a summary table based on some criteria.
Now there are chances , that select query may fail or the insert query may fail due to timeout or other database issues that is beyond my control.
Eventually my script is going to be cron script.
Can I log just the errors that i encounter for the connection,inserts and selects into a file generated in the script?
$logfile = $path.'logs/$currdate.log';
here is my code:
my $SQL_handled="SELECT division_id,region_id, NVL(COUNT(*),0) FROM super_tab GROUP BY division_id,region_id;";
my $result_handled = $dbh->prepare($SQL_handled);
$result_handled->execute();
while (my ($division_id,$region_id,$count ) = $result_handled->fetchrow_array()){
my $InsertHandled="INSERT INTO summary_tab (date_hour, division_id, region_id,volume) VALUES ('$current',$division_id,$region_id,$market_id,'$service_type','$handled',$count);";
my $result_insert_handled = $dbh->prepare($InsertHandled);
$result_insert_handled->execute();
}
something like
if(DBI-query failed ) {
// log the error onto the above logpath
}
Its usually done like this
my $SQL_handled="SELECT division_id,region_id, NVL(COUNT(*),0) FROM super_tab GROUP BY division_id,region_id;";
my $result_handled = $dbh->prepare($SQL_handled);
my $retval = $result_handled->execute();
if(!$retval){
#open a log file and write errors
writelog();
die "Error executing SQL SELECT - $dbh->errstr";
}
while(my ($division_id,$region_id,$count ) = $result_handled->fetchrow_array()){....
}
---------------------------------
sub writelog{
my $path = "/path/to/logfile";
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);
$year += 1900;
$mon++;
my $currdate = "$mon$mday$year";
$logfile = $path . "/$currdate.log";
open (OUT, ">>$logfile");
print OUT "There was an error encountered while executing SQL- $dbh->errstr \n";
close(OUT);
}
You can also use $dbh->err; which returns the native Oracle error code to trap the error and exit accordingly.
The above, basic exception handling can be performed for every execute() method call in your script. Remember, DBI will have AutoCommit set to 1 (enabled) by default, unless explicitly disabled. So your transactions would be auto committed per insert, in order to handle the ATOMICITY of the entire transaction, you can disable autocommit and use $dbh->commit and $dbh->rollback to handle when you want to commit, or may be use some custom commit point (for larger sets of data).
Or the below can be used while connecting to the DB
$dbh = DBI->connect( "dbi:Oracle:abcdef", "username", "password" , {
PrintError => 0, ### Don't report errors via warn( )
RaiseError => 1 ### Do report errors via die( )
} );
this would automatically report all errors via die. The RaiseError is usually turned off by default.
Also if I understand you correctly then, by cron you mean you would be calling it from a shell cron job. In that case, call to your perl script from the cron itself can be redirected to log files something like below
perl your_perl.pl >> out.log 2>> err.log
out.log will contain regular logs and err.log will contain errors (specifically thrown by DBI prepare() or execute() methods too). In this case, you also need to make sure you use proper verbiage in print or die so that the logs look meaningful.
First, bear in mind that if you put an email address at the top of your crontab file any output from the cron job will be emailed to you:
MAILTO=me#mydomain.com
Second, if you set DBI's RaiseError to 1 when you connect you do not need to check every call, DBI will raise an error whenever one happens.
Third, DBI has an error handler callback. You register a handler and it is called whenever an error occurs with the handle in error and error text etc. If you return false from the error handler, DBI works as it would without the handler and goes on to die or warn. As a result, it is easier to set RaiseError and create an error handler than as Annjawn suggested.
Lastly, if you don't want to do this yourself, you can use something like DBIx::Log4perl and simply ask for it to log errors and nothing else. Any errors will be written to your Log4perl file and they include the SQL being executed, parameters etc.

Special character handling when fetching data from MS SQL Server using Perl DBD

I have an MS SQL Server 2008 Database, from which I am fetching data using perl DBD::Sybase module. But there are some special characters in the DB, like the Copyright symbol, Trademark symbol etc., which are not getting imported properly. Perl seems to change all of these special characters to a Question mark character. Is there a way to fix this?
I have tried specifying charset=utf8 in the connection string. The doc mentions a syb_enable_utf8 (bool) setting, but whenever I try that, I get an error:
Can't locate object method "syb_enable_utf8" via package "DBI::db"
One solution I found was this:
use Encode qw(encode_utf8);
Then, wherever you are writing data to a file or anywhere else, use Encode::encode_utf8($data);
where $data is the column/value which you have fetched from MSSQL.
I don't use DBD::Sybase but a) I use a lot of other DBDs and b) I am currently collecting information about unicode support in DBDs. According to the pod you need at least OpenClient 15.x when using syb_enable_utf8. Are you using 15.x or later? Perhaps syb_enable_utf8 is not defined if your client is less than 15.x or perhaps you have too old a version of DBD::Sybase. Unfortunately I cannot see from the Changes file when syb_enable_utf8 was added.
However, when you say "can't locate method" I think that is a clue as syb_enable_utf8 is not a method, it is an attribute (it is under Sybase Specific Attributes) in the pod. So you need to add it to your connect call or set it via a connection handle like this:
my $h = DBI->connect("dbi:Sybase:something","user","password", {syb_enable_utf8 => 1});
or
$h->{syb_enable_utf8} = 1;
You should also read the bits in the pod describing what happens when syb_enable_utf8 is set as it appears from the documents it only applies to UNIVARCHAR, UNICHAR, and UNITEXT columns.
Lastly, you need to ensure you insert the data correctly in the first place. I'd guess if it is not inserted from Perl with syb_enable_utf8 and charset=utf8 and your data is not proper unicode characters in Perl before you insert you'll get garbage back.
The comment Raze2dust made had nothing to do with your issue but is worth heeding if you are going to write the data retrieved from your database elsewhere. Just remember to decode any data input to your script and encode any data output.

After querying DB I can't print data as well as text anymore to browser

I'm in a web scripting class, and honestly and unfortunately, it has come second to my networking and design and analysis classes. Because of this I find I encounter problems that may be mundane but can't find the solution to it easily.
I am writing a CGI form that is supposed to work with a MySQL DB. I can insert and delete into the DB just fine. My problem comes when querying the DB.
My code compiles fine and I don't get errors when trying to "display" the info in the DB through the browser but the data and text doesn't in fact display. The code in question is here:
print br, 'test';
my $dbh = DBI->connect("DBI:mysql:austinc4", "*******", "*******", {RaiseError => 1} );
my $usersstatement = "select * from users";
my $projstatment = "select * from projects";
# Get the handle
my $userinfo = $dbh->query($usersstatement);
my $projinfo = $dbh->query($projstatement);
# Fetch rows
while (#userrow = $userinfo->fetchrow()) {
print $userrow[0], br;
}
print 'end';
This code is in an if statement that is surrounded by the print header, start_html, form, /form, end_html. I was just trying to debug and find out what was happening and printed the statements test and end. It prints out test but doesn't print out end. It also doesn't print out the data in my DB, which happens to come before I print out end.
What I believe I am doing is:
Connecting to my DB
Forming a string the contains the command/request to the DB
Getting a handle for my query I perform on the DB
Fetching a row from my handle
Printing the first field in the row I fetched from my table
But I don't see why my data wouldn't print out as well as the end text. I looked in DB and it does in fact contain data in the DB and the table that I am trying to get data from.
This one has got me stumped, so I appreciate any help. Thanks again. =)
Solution:
I was using a that wasn't supported by the modules I was including. This leads me to another question. How can I detect errors like this? My program does in fact compile correctly and the webpage doesn't "break". Aside from me double checking that all the methods I do use are valid, do I just see something like text not being displayed and assume that an error like this occurred?
Upon reading the comments, the reason your program is broken is because query() does not execute an SQL query. Therefore you are probably calling an undefined subroutine unless this is a wrapper you have defined elsewhere.
Here is my original posting of helpful hints, which still apply:
I hope you have use CGI, use DBI, etc... and use CGI::Carp and use strict;
Look in /var/log/apache2/access.log or error.log for the bugs
Realize that the first thing a CGI script prints MUST be a valid header or the web server and browser become unhappy and often nothing else displays.
Because of #3 print the header first BEFORE you do anything, especially before you connect to the database where the script may die or print something else because otherwise the errors or other messages will be emitted before the header.
If you still don't see an error go back to #2.
CGIs that use CGI.pm can be run from a command line in a terminal session without going through the webserver. This is also a good way to debug.