How to insert mysql data into perl graph (Editted) - perl

So my task is creating a graph of the data that is inside MySQL table (which is around 41 data with 7 rows) . I only did a basic graph ... so is it possible for me to even create a graph using the data that is inside MySQL table using perl script.
sorry for the lack of codding though since i dont even know how to create a perl graph using MySQL data
Edit
i tried doing the graph but it seems that the data isnt showing up as it attended . it only shown only an empty graph and the values start with a negative for some reason... is there something that i did wrong ?
my sql table
create table Top_1 (
CPU_User float, CPU_System float, CPU_Waiting float);
my script
#!/usr/bin/perl
use DBI;
use warnings;
use strict;
use autodie;
use Data::Dumper;
use GD::Graph::bars;
use GD::Graph::Data;
my $username = "root";
my $password = "";
my $db = "Top_Data_1";
my $host = "127.0.0.1";
my $port = "3306";
my $dsn = "DBI:mysql:database=$db;host=$host;port=$port";
my %attr = (PrintError=>0,RaiseError=>1 );
my $dbh = DBI->connect($dsn,$username,$password,\%attr) or die $DBI::errstr;
my $sth = $dbh->prepare('CPU_User, CPU_System, CPU_Waiting from Top_1');
$sth->execute();
my #row;
while ( #row = $sth->fetchrow_array) {
print "this is CPU_User\n";
print "$row[0]\n";
print "this is CPU_System \n";
print "$row[1]\n";
print "this is CPU_Waiting \n";
print "$row[2]\n";
}
my $data = GD::Graph::Data->new([
["8 am","10 pm","12 pm"],
['$row[0]'],
['$row[1]'],
['$row[2]'],
]) or die GD::Graph::Data->error;
my $graph = GD::Graph::bars->new;
$graph->set(
x_label => 'File_Name',
y_label => 'Value',
title => 'TOP CPU DISPLAY',
x_labels_vertical => 1,
bar_spacing => 10,
bar_width => 3,
long_ticks => 1,
) or die $graph->error;
$graph->set_legend_font(GD::gdMediumBoldFont);
$graph->set_legend('CPU USER','CPU_System','CPU_Waiting');
$graph->plot($data) or die $graph->error;
my $file = 'bars.png';
print "Your Picture Has Been Added To Your Directory\n";
open(my $out, '>', $file) or die "Cannot open '$file' for write: $!";
binmode $out;
print $out $graph->gd->png;
close $out;
$sth->finish();
$dbh->disconnect();

This is possible to do. One can use DBI to get data from MySQL. Then, you can arrange the data into a format to graph it using a graphing library. I have used GD::Graph and is easy to use.
Links:
MySQL: https://metacpan.org/pod/DBD::mysql
Graph: https://metacpan.org/pod/GD::Graph

The data structure is not correct.
Not sure of your database table, but if you have the time field in it then something like below should work.
my $data = GD::Graph::Data->new();
my $sth = $dbh_mysql->prepare('SELECT hour, CPU_USER, CPU_System, CPU_Waiting FROM top1');
$sth->execute();
while (my #row = $sth->fetchrow_array)
{
$data->add_point(#row);
}

Related

how to print the names of tables of database using perl scripting using unix

How to print the names of tables of database using Perl scripting using unix. Connected to DB using DBI module.
I tried scripting using
my $driver= "Oracle";
my $dsn = "DBI:$driver:sid=as;host=asdsa";
my $dbh = DBI->connect($dsn, "a", "b") || die( $DBI::errstr . "\n" );
my $notables = $dbh->tables();
print "No of tables : $notables" ;
Getting error:
Can't call method "tables" on an undefined value at hello.pl line 16.
Please help.
Looks like you not connected to DB.
Read DBI documentation and try something like this:
use DBI;
use Data::Dumper;
my $dbh = DBI->connect($data_source, $username, $password)
or die $DBI::errstr;
my #names = $dbh->tables( $catalog, $schema, $table, $type );
print Dumper #names;
$dbh->tables; without args is deprecated

Display full taxon path from NCBI GI number

I prepared the following script that takes a GI ID number from NCBI that I prepared in my tsv file and prints the scientific name associated with the ID:
#!/usr/bin/perl
use strict;
use warnings;
use Bio::DB::Taxonomy;
my ($filename) = #ARGV;
open my $fh, '<', $filename or die qq{Unable to open "$filename": $!};
while(<>) {
my ($taxonid, $counts) = (split /\t/);
for my $each($taxonid) {
print "$each\n";
my $db = Bio::DB::Taxonomy->new(-source => 'entrez');
my $taxon = $db->get_taxon(-taxonid => $taxonid);
print "Taxon ID is $taxon->id, \n";
print "Scientific name is ", $taxon->scientific_name, "\n";
}
}
With this script, I receive the following:
1760
Taxon ID is Bio::Taxon=HASH(0x33a91f8)->id,
Scientific name is Actinobacteria
What I want to do
Now the next step is for me to list the full taxon path of the bacteria in question. So for the above example, I want to see k__Bacteria; p__ Actinobacteria; c__ Actinobacteria as output. Furthermore, I want the GI IDs on my table to be repliaced with this full taxon path.
In which direction should I go?
First, I notice you open $filename which is your first command line argument, but you don't use the file pointer $fh you created.
So, these two lines are not needed in your case because you already do the trick with <>
my ($filename) = #ARGV;
open my $fh, '<', $filename or die qq{Unable to open "$filename": $!};
Next. I don't know what is inside your filename and your database so I cannot help you more. Can you provide an example of what is inside your database and your file?
One more thing, what I can see here is that you may not need to create your $db instance inside the loop.
#!/usr/bin/perl
use strict;
use warnings;
use Bio::DB::Taxonomy;
my $db = Bio::DB::Taxonomy->new(-source => 'entrez');
while(<>) {
my ($taxonid, $counts) = (split /\t/);
for my $each($taxonid) {
print "$each\n";
my $taxon = $db->get_taxon(-taxonid => $taxonid);
print "Taxon ID is $taxon->id, \n";
print "Scientific name is ", $taxon->scientific_name, "\n";
}
}
Edit
From your command Is is hard to help you. When you write
my $taxon = $db->get_taxon(-taxonid => $taxonid);
You receive a Bio::Taxon node where the documentation ca be found here
I don't know what k__Bacteria; p__ Actinobacteria; c__ Actinobacteria representy for you. Is it an information offered by a Bio::Taxon node?
Anyway, you can still explore $taxon with this:
#!/usr/bin/env perl
# Author: Yves Chevallier
# Date:
use strict;
use warnings;
use Data::Dumper;
use Bio::DB::Taxonomy;
my $db = Bio::DB::Taxonomy->new(-source => 'entrez');
while(<DATA>) {
my ($taxonid, $counts) = (split /\t/);
for my $each($taxonid) {
print "$each\n";
my $taxon = $db->get_taxon(-taxonid => $taxonid);
print Dumper $taxon;
print "Taxon ID is $taxon->id, \n";
print "Scientific name is ", $taxon->scientific_name, "\n";
}
}
__DATA__
12 1760

How can I redirect the output of a SQL query to a file?

I need a little help in redirecting the output of a SQL query to a file. My code looks like this:
my $sth = $dbh->prepare(
"select count(parameter2),
parameter2 as file_type
from KCRT_TABLE_ENTRIES where request_id = $mycrnum
group by parameter2"
) or die "Can't prepare SQL statement: ", $dbh->errstr(), "\n";
$sth->execute > $mydir\\file_detail.txt
or die "Can't execute SQL statement: ", $sth->errstr(), "\n";
I've had to invent a lot of code as you don't show much of your program, but the program below gives you the rough idea
Once you've called execute you have to call one of the fetch methods to retrieve the data in whatever form is most useful to you. Here I've just asked for a reference to an array containing each row's data
Then it's simply a matter of opening the required file for output and printing the rows of data to it
I've removed the status checks on each DBI call and replaced it with the RaiseError flag which does the same thing automatically. I've also replaced the parameter $mycrnum in the SQL statement with a placeholder and passed its value to execute. That way DBI looks after any necessary quoting etc.
use strict;
use warnings;
use DBI;
my ($dsn, $user, $pass);
my ($mycrnum, $mydir);
my $dbh = DBI->connect($dsn, $user, $pass);
#{$dbh}{qw/ PrintError RaiseError /} = (0, 1);
my $sth = $dbh->prepare(
"SELECT COUNT(parameter2),
parameter2 AS file_type
FROM kcrt_table_entries
WHERE request_id = ?
GROUP BY parameter2"
);
$sth->execute($mycrnum);
open my $fh, '>', "$mydir/file_detail.txt" or die $!;
select $fh;
while ( my $row = $sth->fetchrow_arrayref ) {
printf "%5d %s\n", #$row;
}
After the execute, open the output file:
open my $of, ">", "$mydir\\file_detail.txt";
Then read each line (or row) in the results:
while ( #row = $sth->fetchrow_array ) {
Printing the output to the opened file handle:
print $of "#row\n"; # NO COMMA AFTER $of!
Close the while() loop:
}
Finally, close your opened file handle:
close $of;
Now your done.
Something like this, perhaps?
my $sth = $dbh->prepare(q{
select count(parameter2),
parameter2 as file_type
from KCRT_TABLE_ENTRIES where request_id = ?
group by parameter2
}) or die "Can't prepare SQL statement: ", $dbh->errstr(), "\n";
$sth->execute($mycrnum);
open my $OUT, '>', "$mydir/file_detail.txt" or die;
while (my #row = $sth->fetchrow_array) {
print $OUT #row, "\n"; # or whatever...
}
close $OUT;
$sth->finish;
This is a little bit of overkill, since you are only reading a single value, but it at least demonstrates a boilerplate for getting it done for future queries.
If you ever have a guaranteed single row, you can do something like this:
my ($val1, $val1) = $dbh->selectrow_array(q{
select foo, bar
});

How to take only latest uniq record based on any column

I am writing a script in perl. but got stuck in one part. Below is the sample of my csv files.
"MP","918120197922","20150806125001","prepaid","prepaid","3G","2G"
"GJ","919904303790","20150806125002","prepaid","prepaid","2G","3G"
"MH","919921990805","20150806125003","prepaid","prepaid","2G"
"MP","918120197922","20150806125004","prepaid","prepaid","2G"
"MUM","919904303790","20150806125005","prepaid","prepaid","2G","3G"
"MUM","918652624178","20150806125005","","prepaid","","2G","NEW"
"MP","918120197922","20150806125005","prepaid","prepaid","2G","3G"
Now I need to take unique records on the basis of 2nd column (i.e. mobile numbers ) but considering only the latest value of 3rd column (ie timestamp)
eg: for mobile number "918120197922".
"MP","918120197922","20150806125001","prepaid","prepaid","3G","2G"
"MP","918120197922","20150806125004","prepaid","prepaid","2G"
"MP","918120197922","20150806125005","prepaid","prepaid","2G","3G"
it should select the 3rd record as it has the latest value of timestamp (20150806125005). Please help.
Additional Info:
Sorry for inconsistency in data..I have rectified it now.
Yes data is in order which means latest timestamp will appear in the latest rows.
One more thing that my file has the size of more than 1 gb so is there any way to efficiently do this? Will awk work faster than perl in this case. Please help?
Use Text::CSV to process CSV files.
Hash the lines by the 2nd column, only keep the most recent one in the hash.
#!/usr/bin/perl
use warnings;
use strict;
use Text::CSV;
my $csv = 'Text::CSV'->new() or die 'Text::CSV'->error_diag;
my %hash;
open my $CSV, '<', '1.csv' or die $!;
while (my $row = $csv->getline($CSV)) {
my ($number, $timestamp) = #$row[1, 2];
# Store the row if the timestamp is more recent than the stored one.
$hash{$number} = $row if $timestamp gt ($hash{$number}[2] || q());
}
$csv->eol("\n");
$csv->always_quote(1);
open my $OUT, '>', 'uniq.csv' or die $!;
for my $row (values %hash) {
$csv->print($OUT, $row);
}
close $OUT or die $!;
If you know your data is ordered by timestamp you can exploit this and read them backwards and transform your task into a problem to output the first occurrence of each phone number.
#!/usr/bin/env perl
use strict;
use warnings;
use autodie;
use Text::CSV_XS;
use constant PHONENUM_FIELD => 1;
my $filename = shift;
die "Usage: $0 <filename>\n" unless defined $filename;
open my $in, '-|', 'tac', $filename;
my $csv = Text::CSV_XS->new( { binary => 1, auto_diag => 1, eol => $/ } );
my %seen;
while ( my $row = $csv->getline($in) ) {
$csv->print( *STDOUT, $row ) unless $seen{ $row->[PHONENUM_FIELD] }++;
}
If you would like to have output in same order as input you can write into tac as well:
#!/usr/bin/env perl
use strict;
use warnings;
use autodie;
use Text::CSV_XS;
use constant PHONENUM_FIELD => 1;
my $filename = shift;
die "Usage: $0 <filename>\n" unless defined $filename;
open my $in, '-|', 'tac', $filename;
open my $out, '|-', 'tac';
my $csv = Text::CSV_XS->new( { binary => 1, auto_diag => 1, eol => $/ } );
my %seen;
while ( my $row = $csv->getline($in) ) {
$csv->print( $out, $row ) unless $seen{ $row->[PHONENUM_FIELD] }++;
}
1GB should not be a problem at any decent HW. On my old notebook, it took 2m3.393s for processing 29360128 rows and 1.8GB. It is more than 230krows/s but YMMV. Add always_quote => 1 to $csv constructor parameters if you are interested to gain quoted all values at the output.

CSV import to MySQL

Hi I keep getting an error when trying to run the following perl script to import a csv file into an existing mysql database table. Every time I run it I get the message "Died at /home/perl/dep_import_2.pl line 10.
Any help would be appreciated
Thanks
#!/usr/bin/perl
use DBI;
use DBD::mysql;
use warnings "all";
if ($#ARGV != 0) {
print "Usage: dep_import_2.pl filename\n";
die;
}
$filename = $ARGV[0];
# MySQL CONFIG VARIABLES
$host = "localhost";
$user = "standard";
$pw = "standard";
$database = "data_track";
$dsn = "DBI:mysql:database=" . $database . ";host=" . $host;
$dbh = DBI->connect($dsn, $user, $pw)
or die "Can't connect to the DB: $DBI::errstr\n";
print "Connected to DB!\n";
open FILE, "/home/dep/new_study_forms_2.csv", $filename or die $!;
$_ = <FILE>;
$_ = <FILE>;
while (<FILE>) {
#f = split(/,/, $_);
$sql = "INSERT INTO dep (date, subject, weight, size, time, hi_pre, hi_post, hi_afternoon, hi_test, actical_on, actical_off, saggital_1, saggital_2, crown_heel1, crown_heel2, crown_rump1, crown_rump2, scan, record_number, tap, sample, dye, left_chip, right_chip) VALUES('$f[0]', '$f[1]', '$f[2]', '$f[3]' '$f[4]', '$f[5]', '$f[6]', '$f[7]', '$f[8]', '$f[9]', '$f[10]', '$f[11]', '$f[12]', '$f[13]', '$f[14]', '$f[15]', '$f[16]', '$f[17]', '$f[18]', '$f[19]', '$f[20]', '$f[21]', '$f[22]', '$f[23]')";
print "$sql\n";
my $query = $dbh->do($sql);
}
There are a few issues with your code. First, and most importantly, you are not using
use strict;
use warnings;
This is bad because you will not get information about errors in your code without them.
As others have pointed out, the reason the script dies is because $#ARGV is not zero. Meaning that you have either passed too few or too many arguments to the script. The arguments to the script must be exactly one, like the usage statement says.
However, that would not solve your problem, because your open statement below is screwed up. My guess is that you tried to add your file name directly. This line:
open FILE, "/home/dep/new_study_forms_2.csv", $filename or die $!;
It will probably give you the error unknown open() mode .... It should probably be
open FILE, "<", $filename or die $!;
And then you pass /home/dep/new_study_forms_2.csv to the script on the command line, assuming that is the correct file to use.
Also, in your query string, you should not interpolate variables, you should use placeholders, as is described in the documentation for DBI. The placeholders will take care of the quoting for you and avoid any data corruption. To make your query line a bit simpler, you can do something like:
my $sth = $dbh->prepare(
"INSERT INTO dep (date, subject, weight, size, time, hi_pre, hi_post,
hi_afternoon, hi_test, actical_on, actical_off, saggital_1, saggital_2,
crown_heel1, crown_heel2, crown_rump1, crown_rump2, scan, record_number,
tap, sample, dye, left_chip, right_chip)
VALUES(" . join(",", ("?") x #f) . ")");
$sth->execute(#f);
Here's a script which uses Text::CSV to properly parse CSV. It assumes that the first row contains column names, and then loads the CSV in batches, commiting after every 100 inserts. Every parameter (user, password, database) is configurable via command-line options. Usage is an in-line POD document.
#!/usr/bin/env perl
use strict;
use warnings qw(all);
use DBI;
use Getopt::Long;
use Pod::Usage;
use Text::CSV_XS;
=pod
=head1 SYNOPSIS
dep_import_2.pl --filename=file.csv --host=localhost --user=standard --pw=standard --database=data_track
=head1 DESCRIPTION
Loads a CSV file into the specified MySQL database.
=cut
my $host = 'localhost';
my $user = 'standard';
my $pw = 'standard';
my $database = 'data_track';
my $commit = 100;
GetOptions(
'help' => \my $help,
'filename=s' => \my $filename,
'host=s' => \$host,
'user=s' => \$user,
'pw=s' => \$pw,
'database=s' => \$database,
'commit=i' => \$commit,
) or pod2usage(q(-verbose) => 1);
pod2usage(q(-verbose) => 2) if $help;
my $dbh = DBI->connect("DBI:mysql:database=$database;host=$host", $user => $pw)
or die "Can't connect to the DB: $DBI::errstr";
my $csv = Text::CSV_XS->new
or die "Text::CSV error: " . Text::CSV->error_diag;
open(my $fh, '<:utf8', $filename)
or die "Can't open $filename: $!";
my #cols = #{$csv->getline($fh)};
$csv->column_names(\#cols);
my $query = "INSERT INTO dep (#{[ join ',', #cols ]}) VALUES (#{[ join ',', ('?') x (scalar #cols) ]})";
my $sth = $dbh->prepare($query);
my $i = 0;
while (my $row = $csv->getline_hr($fh)) {
$sth->execute(#{$row}{#cols});
$dbh->commit if ((++$i % $commit) == 0);
}
$dbh->commit;
$dbh->disconnect;
$csv->eof or $csv->error_diag;
close $fh;