Forward Slash issue with DBI - perl

I'm new to using DBI for SQL queries in a perl script. The issue I'm having pertains to data in fields that have a forward slash. I'm wanting to use variables as input for my where clause, but it is doing what DBI intends a forward slash to do: stop the query. I tried numerous different work arounds from binds, quotes, etc. but none worked, is it even possible? Data in this is consistent. The line with the my $sql variable is where the trouble is.
#!/usr/bin/perl
# Modules
use DBI;
use DBD::Oracle;
use strict;
use warnings;
# Connection Info
$platform = "Oracle";
$database = "mydb";
$user = "user";
$pw = "pass";
# Data Source
$ds = "dbi:Oracle:$database";
my $dbh = DBI->connect($ds, $user, $pw);
# my $dbh = DBI->connect();
my $XCOD = $dbh->quote('cba');
my $a = $dbh->quote('abc');
my $b = $dbh->quote('123');
# tried this as well my $pid = $dbh->quote('$a/$b');
my $sql = "SELECT P_ID FROM MyTable WHERE P_ID=$a/$b AND XCOD=$XCOD";
my $sth = $dbh->prepare($sql);
$sth->execute();
my $outfile = 'superunique.txt';
open OUTFILE, '>', $outfile or die "Unable to open $outfile: $!";
while(my #re = $sth->fetchrow_array) {
print OUTFILE #re,"\n";
}
close OUTFILE;
$sth->finish();
$dbh->disconnect();

I don't like to see folks use variable interpolation in SQL queries. Try using placeholders:
[ snip ]
my $P_ID = "$a/$b"
my $sql = "SELECT P_ID FROM MyTable WHERE P_ID = ? AND XCOD = ?";
my $sth = $dbh->prepare($sql);
$sth->execute($P_ID, $XCOD);
[ snip ]

You have been given the correct solution to your problem (use placeholders) but you might be interested to see why what you are doing doesn't work.
The problem is that you seem to misunderstand the quote method. The documentation says this:
Quote a string literal for use as a literal value in an SQL statement,
by escaping any special characters (such as quotation marks) contained
within the string and adding the required type of outer quotation
marks.
You use quote in these three lines.
my $XCOD = $dbh->quote('cba');
my $a = $dbh->quote('abc');
my $b = $dbh->quote('123');
It would be instructive to print out the values of $XCOD, $a and $b (as an aside $a and $b are really bad names for variables - apart from their non-descriptive nature, they are also special variables used in sorting).
I suspect that you'll see "cba", "abd" and "123". The method has found no special characters to escape, so all it has done is to add quote marks around the strings.
You then interpolate these values into your SQL.
my $sql = "SELECT P_ID FROM MyTable WHERE P_ID=$a/$b AND XCOD=$XCOD";
Again, you should take a close look at what $sql contains after this statement has been executed. It will look something like this:
SELECT P_ID FROM MyTable WHERE P_ID="abc"/"123" AND XCOD="cba"
It's probably the first part of the WHERE clause that is a problem. Oracle is treating that as a division. And who knows what Oracle does when you divide one string by another. So you end up looking for a row where P_ID is some strange (perhaps undefined) value.
So this looks to be an example where the simplest of debugging techniques (a few print statements in the code) would have guided you in the right direction.

Related

Data::Dumper wraps second word's output

I'm experiencing a rather odd problem while using Data::Dumper to try and check on my importing of a large list of data into a hash.
My Data looks like this in another file.
##Product ID => Market for product
ABC => Euro
XYZ => USA
PQR => India
Then in my script, I'm trying to read in my list of data into a hash like so:
open(CONFIG_DAT_H, "<", $config_data);
while(my $line = <CONFIG_DAT_H>) {
if($line !~ /^\#/) {
chomp($line);
my #words = split(/\s*\=\>\s/, $line);
%product_names->{$words[0]} = $words[1];
}
}
close(CONFIG_DAT_H);
print Dumper (%product_names);
My parsing is working for the most part that I can find all of my data in the hash, but when I print it using the Data::Dumper it doesn't print it properly. This is my output.
$VAR1 = 'ABC';
';AR2 = 'Euro
$VAR3 = 'XYZ';
';AR4 = 'USA
$VAR5 = 'PQR';
';AR6 = 'India
Does anybody know why the Dumper is printing the '; characters over the first two letters on my second column of data?
There is one unclear thing in the code: is *product_names a hash or a hashref?
If it is a hash, you should use %product_names{key} syntax, not %product_names->{key}, and need to pass a reference to Data::Dumper, so Dumper(\%product_names).
If it is a hashref then it should be labelled with a correct sigil, so $product_names->{key} and Dumper($product_names}.
As noted by mob if your input has anything other than \n it need be cleaned up more explicitly, say with s/\s*$// per comment. See the answer by ikegami.
I'd also like to add, the loop can be simplified by loosing the if branch
open my $config_dat_h, "<", $config_data or die "Can't open $config_data: $!";
while (my $line = <$config_dat_h>)
{
next if $line =~ /^\#/; # or /^\s*\#/ to account for possible spaces
# ...
}
I have changed to the lexical filehandle, the recommended practice with many advantages. I have also added a check for open, which should always be in place.
Humm... this appears wrong to me, even you're using Perl6:
%product_names->{$words[0]} = $words[1];
I don't know Perl6 very well, but in Perl5 the reference should be like bellow considering that %product_names exists and is declared:
$product_names{...} = ... ;
If you could expose the full code, I can help to solve this problem.
The file uses CR LF as line endings. This would become evident by adding the following to your code:
local $Data::Dumper::Useqq = 1;
You could convert the file to use unix line endings (seeing as you are on a unix system). This can be achieved using the dos2unix utility.
dos2unix config.dat
Alternatively, replace
chomp($line);
with the more flexible
$line =~ s/\s+\z//;
Note: %product_names->{$words[0]} makes no sense. It happens to do what you want in old versions of Perl, but it rightfully throws an error in newer versions. $product_names{$words[0]} is the proper syntax for accessing the value of an element of a hash.
Tip: You should be using print Dumper(\%product_names); instead of print Dumper(%product_names);.
Tip: You might also find local $Data::Dumper::Sortkeys = 1; useful. Data::Dumper has such bad defaults :(
Tip: Using split(/\s*=>\s*/, $line, 2) instead of split(/\s*=>\s*/, $line) would permit the value to contain =>.
Tip: You shouldn't use global variable without reason. Use open(my $CONFIG_DAT_H, ...) instead of open(CONFIG_DAT_H, ...), and replace other instances of CONFIG_DAT_H with $CONFIG_DAT_H.
Tip: Using next if $line =~ /^#/; would avoid a lot of indenting.

Perl cgi bind dynamic number of columns

I'm trying to make a simple select from a database, the thing is that I want the same script to be able to select any of the tables in it. I have gotten everything solved up until the point when I need to bind the columns to variables, since they must be generated dynamically I just don't know how to do it.
here's the code:
if($op eq "SELECT"){
if ($whr){
$query1 = "SELECT $colsf FROM $tab WHERE $whr";
}else{
$query1 = "SELECT $colsf FROM $tab";
}
$seth = $dbh->prepare($query1);
$seth->execute();
foreach $cajas(#columnas){
$seth->bind_col(*$dynamically_generated_var*);
}
print $q->br();
print $q->br();
print $q->br();
The variable #columans contains the name of the selected columns (which varies a lot), and I need a variable assigned for each of the columns on the $seth->bind_col().
How can I acheive this?
Using bind_col will not gain you anything here. As you have already figured out, that's used to bind a fixed number of results to a set of variables. But you do not have a fixed set.
Thinking in terms of oh, I can just create them dynamically is a very common mistake. It will get you into all kinds of trouble later. Perl has a data structure specifically for this use case: the hash.
DBI has a bunch of functions built in for retrieving data after execute. One of those is fetchrow_hashref. It will return the results as a hash reference, with one key per column, one row at a time.
while (my $res = $sth->fetchrow_hashref) {
p $res; # p is from Data::Printer
}
Let's assume the result looks like this:
$res = {
id => 1,
color => 'red',
}
You can access the color by saying $res->{color}. The perldocs on perlref and perlreftut have a lot of info about this.
Note that the best practice for naming statement handle variables is $sth.
In your case, you have a dynamic number of columns. Those have to be joined to be in the format of col1, col2, col3. I guess you have already done that in $colsf. The table is pretty obvious in $tab, so we only have the $whr left.
This part is tricky. It's important to always sanitize your input, especially in a CGI environment. With DBI this is best done by using placeholders. They will take care of all the escaping for you, and they are easy to use.
my $sth = $dbi->prepare('select cars from garage where color=?');
$sth->execute($color);
Now we don't need to care if the color is red, blue or ' and 1; --, which might have broken stuff. If it's all very dynamic, use $dbi->quote instead.
Let's put this together in your code.
use strict;
use warnings;
use DBI;
# ...
# the columns
my $colsf = join ',', #some_list_of_column_names; # also check those!
# the table name
my $table = $q->param('table');
die 'invalid table name' if $table =~ /[^a-zA-Z0-9_]/; # input checking
# where
# I'm skipping this part as I don't know where it is comming from
if ($op eq 'SELECT') {
my $sql = 'SELECT $colsf FROM $table';
$sql .= ' WHERE $whr' if $whr;
my $sth = $dbh->prepare($sql) or die $dbi->errstr;
$sth->execute;
my #headings = $sth->{NAME}; # see https://metacpan.org/pod/DBI#NAME1
while (my $res = $sth->fetchrow_hashref) {
# do stuff here
}
}

MySQL Query with variable in Perl

My Perl is fairly rusty so please forgive. Trying to write a query using a variable. Have tried reformatting, just can't seem to get it correctly. Here is my code, not sure what I'm doing wrong.
my $d_var = "$3\n";
my $query="SELECT id FROM `accounts` WHERE (`accounts`.`named` = ?) LIMIT 1";
my $st_h = $db_h->prepare($query);
$st_h->bind_param(1, '$d_var');
$st_h->execute;
my $row = $st_h->fetchrow_array();
Please double check:
$3 contains something reasonable
concatenating $3's value and "\n" (by interpolation) is correct ("\n" in field?)
as ' doesn't interpolate => my $st_ht->bind_param(1, $d_var);
(I don't understand the DBI Docs as Chris Ledet does.)
On 2nd thought:
This code snippet:
my $v = "nix nix 1001";
print "$v\n";
print '$v\n', "\n";
if ($v =~ m/(nix) (nix) (\d+)/) {
print 'found: ', $3, "\n";
$sth = $dbh->prepare('SELECT * FROM sample01.csv WHERE GRUPPE=?');
$sth->bind_param(1, $3);
$sth->execute;
while(my #row = $sth->fetchrow_array()) {
print '|', join( '|', #row ), "|\n";
}
} else {
print "no match\n";
}
and the output:
DBI: 1.616 DBD::CSV: 0.33
|00000089-6d83-486d-9ddf-30bbbf722583|2011-09-17 16:25:09|1001|
|000004c9-92c6-4764-b320-b1403276321e|2011-11-09 13:52:30|2000|
nix nix 1001
$v\n
found: 1001
|00000089-6d83-486d-9ddf-30bbbf722583|2011-09-17 16:25:09|1001|
should illustrate:
' does not interpolate, your '$d_var' will pass this variable name literally to DBI
a valid match needs no "\n" to 'work'
the param sequence for bind_param is number, value
Not sure why you're even using bind_param. In my opinion, it's far simpler to just pass extra values into execute.
my $d_var = "$3\n";
my $query = 'SELECT id FROM accounts` WHERE (`accounts`.`named` = ?) LIMIT 1';
my $st_h = $db_h->prepare($query);
$st_h->execute($d_var);
my $row = $st_h->fetchrow_array();
Have you considered switching to DBIx::Class?
What does this mean?
my $st_ht->bind_param(1, '$d_var');
There is no variable being introduced, so why the my?
I have $3 printing out prior to execution and there is data contained within the string.
I have d_var="$3\n" as the variable $3 is being generated by a Regex string and doesn't seem to work without \n.
Tried what Chris suggested above, however did not work.
Aside from what Dan said (removing the single quote), you binded your param to a possibly none Statement handle object $st_ht->bind_param(...) it should be $st_h->bind_param(...).

Perl split() Function Not Handling Pipe Character Saved As A Variable

I'm running into a little trouble with Perl's built-in split function. I'm creating a script that edits the first line of a CSV file which uses a pipe for column delimitation. Below is the first line:
KEY|H1|H2|H3
However, when I run the script, here is the output I receive:
Col1|Col2|Col3|Col4|Col5|Col6|Col7|Col8|Col9|Col10|Col11|Col12|Col13|
I have a feeling that Perl doesn't like the fact that I use a variable to actually do the split, and in this case, the variable is a pipe. When I replace the variable with an actual pipe, it works perfectly as intended. How could I go about splitting the line properly when using pipe delimitation, even when passing in a variable? Also, as a silly caveat, I don't have permissions to install an external module from CPAN, so I have to stick with built-in functions and modules.
For context, here is the necessary part of my script:
our $opt_h;
our $opt_f;
our $opt_d;
# Get user input - filename and delimiter
getopts("f:d:h");
if (defined($opt_h)) {
&print_help;
exit 0;
}
if (!defined($opt_f)) {
$opt_f = &promptUser("Enter the Source file, for example /qa/data/testdata/prod.csv");
}
if (!defined($opt_d)) {
$opt_d = "\|";
}
my $delimiter = "\|";
my $temp_file = $opt_f;
my #temp_file = split(/\./, $temp_file);
$temp_file = $temp_file[0]."_add-headers.".$temp_file[1];
open(source_file, "<", $opt_f) or die "Err opening $opt_f: $!";
open(temp_file, ">", $temp_file) or die "Error opening $temp_file: $!";
my $source_header = <source_file>;
my #source_header_columns = split(/${delimiter}/, $source_header);
chomp(#source_header_columns);
for (my $i=1; $i<=scalar(#source_header_columns); $i++) {
print temp_file "Col$i";
print temp_file "$delimiter";
}
print temp_file "\n";
while (my $line = <source_file>) {
print temp_file "$line";
}
close(source_file);
close(temp_file);
The first argument to split is a compiled regular expression or a regular expression pattern. If you want to split on text |. You'll need to pass a pattern that matches |.
quotemeta creates a pattern from a string that matches that string.
my $delimiter = '|';
my $delimiter_pat = quotemeta($delimiter);
split $delimiter_pat
Alternatively, quotemeta can be accessed as \Q..\E inside double-quoted strings and the like.
my $delimiter = '|';
split /\Q$delimiter\E/
The \E can even be omitted if it's at the end.
my $delimiter = '|';
split /\Q$delimiter/
I mentioned that split also accepts a compiled regular expression.
my $delimiter = '|';
my $delimiter_re = qr/\Q$delimiter/;
split $delimiter_re
If you don't mind hardcoding the regular expression, that's the same as
my $delimiter_re = qr/\|/;
split $delimiter_re
First, the | isn't special inside doublequotes. Setting $delimiter to just "|" and then making sure it is quoted later would work or possibly setting $delimiter to "\\|" would be ok by itself.
Second, the | is special inside regex so you want to quote it there. The safest way to do that is ask perl to quote your code for you. Use the \Q...\E construct within the regex to mark out data you want quoted.
my #source_header_columns = split(/\Q${delimiter}\E/, $source_header);
see: http://perldoc.perl.org/perlre.html
It seems as all you want to do is count the fields in the header, and print the header. Might I suggest something a bit simpler than using split?
my $str="KEY|H1|H2|H3";
my $count=0;
$str =~ s/\w+/"Col" . ++$count/eg;
print "$str\n";
Works with most any delimeter (except alphanumeric and underscore), it also saves the number of fields in $count, in case you need it later.
Here's another version. This one uses the character class brackets instead, to specify "any character but this", which is just another way of defining a delimeter. You can specify delimeter from the command-line. You can use your getopts as well, but I just used a simple shift.
my $d = shift || '[^|]';
if ( $d !~ /^\[/ ) {
$d = '[^' . $d . ']';
}
my $str="KEY|H1|H2|H3";
my $count=0;
$str =~ s/$d+/"Col" . ++$count/eg;
print "$str\n";
By using the brackets, you do not need to worry about escaping metacharacters.
#!/usr/bin/perl
use Data::Dumper;
use strict;
my $delimeter="\\|";
my $string="A|B|C|DD|E";
my #arr=split(/$delimeter/,$string);
print Dumper(#arr)."\n";
output:
$VAR1 = 'A';
$VAR2 = 'B';
$VAR3 = 'C';
$VAR4 = 'DD';
$VAR5 = 'E';
seems you need define delimeter as \\|

Using perl to parse a file and insert specific values into a database

Disclaimer: I'm a newbie at scripting in perl, this is partially a learning exercise (but still a project for work). Also, I have a much stronger grasp on shell scripting, so my examples will likely be formatted in that mindset (but I would like to create them in perl). Sorry in advance for my verbosity, I want to make sure I am at least marginally clear in getting my point across
I have a text file (a reference guide) that is a Word document converted to text then swapped from Windows to UNIX format in Notepad++. The file is uniform in that each section of the file had the same fields/formatting/tables.
What I have planned to do, in a basic way is grab each section, keyed by unique batch job names and place all of the values into a database (or maybe just an excel file) so all the fields can be searched/edited for each job much easier than in the word file and possibly create a web interface later on.
So what I want to do is grab each section by doing something like:
sed -n '/job_name_1_regex/,/job_name_2_regex/' file.txt --how would this be formatted within a perl script?
(grab the section in total, then break it down further from there)
To read the file in the script I have open FORMAT_FILE, 'test_format.txt'; and then use foreach $line (<FORMAT_FILE>) to parse the file line by line. --is there a better way?
My next problem is that since I converted from a word doc with tables, which looks like:
Table Heading 1 Table Heading 2
Heading 1/Value 1 Heading 2/Value 1
Heading 1/Value 2 Heading 2/Value 2
but the text file it looks like:
Table Heading 1
Table Heading 2Heading 1/Value 1Heading 1/Value 2Heading 2/Value 1Heading 2/Value 2
So I want to have "Heading 1" and "Heading 2" as a columns name and then put the respective values there. I just am not sure how to get the values in relation to the heading from the text file. The values of Heading 1 will always be the line number of Heading 1 plus 2 (Heading 1, Heading 2, Values for heading 1). I know this can be done in awk/sed pretty easily, just not sure how to address it inside a perl script.
---EDIT---
For this I was thinking of doing an array something like:
my #heading1 = ($value1, $value2, etc.)
my #heading2 = ($value1, $value2, etc.)
I just need to be able to associate the correct values and headings together. So that heading1 = the line after heading2 (where the values start).
Like saying (in shell):
x=$(grep -n "Heading 1" file.txt | cut -d":" -f1) #gets the line that "Heading 1" is on in the file
(( x = x+2 )) #adds 2 to the line (where the values will start)
#print values from file.txt from the line where they start to the
#last one (I'll figure that out at some point before this)
sed -n "$x,$last_line_of_values p" file.txt
This is super-hacked together for the moment, to try to elaborate what I want to do...let me know if it clears it up a little...
---/EDIT---
After I have all the right values and such, linking it up to a database may be an issue as well, I haven't started looking at the way perl interacts with DBs yet.
Sorry if this is a bit scatterbrained...it's still not fully formed in my head.
http://perlmeme.org/tutorials/connect_to_db.html
#!/usr/bin/perl
use strict;
use warnings;
use DBI;
my $driver = "mysql"; # Database driver type
my $database = "test"; # Database name
my $user = ""; # Database user name
my $password = ""; # Database user password
my $dbh = DBI->connect(
"DBI:$driver:$database",
$user, $password,
{
RaiseError => 1,
PrintError => 1,
}
) or die $DBI::errstr;
my $sth = $dbh->prepare("
INSERT INTO test
(col1, col2)
VALUES (?, ?)
") or die $dbh->errstr;
my $intable = 0;
open my $file, "file.txt" or die "can't open file $!";
while (<$file>) {
if (/job_name_1_regex/../job_name_2_regex/) { # job 1 section
$intable = 1 if /Table Heading 1/; # table start
if ($intable) {
my $next_line = <$file>; # heading 2 line
chomp; chomp $next_line;
$sth->execute($_, $next_line) or die $dbh->errstr;
}
}
}
close $file or die "can't close file $!";
$dbh->disconnect;
Several things in this post... First, the basic "best practices" :
use modern perl. start your scripts with
use strict; use warnings;
don't use global filehandles, use lexical filehandles (declare them in a variable).
always check "open" for return values.
open my $file, "/some/file" or die "can't open file : $!"
Then, about pattern matching : I don't understand your example at all but I suppose you want something like :
foreach my $line ( <$file> ) {
if ( $line =~ /regexp1/) {
# do something...
}
}
Edit : about table, I suppose the best thing is to build two arrays, one for each column.
If I understand correctly when reading the file you need to split the line and put one part in the #col1 array, and the second part in the #col2 array. The clear and easy way is to use two temporary variables :
my ( $val1, $val2 ) = split /\s+/, $line;
push #col1, $val1;
push #col2, $val2;