Why am I getting warnings about an uninitialized value in this Perl program? [closed] - perl

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 8 years ago.
Improve this question
I get these errors
Use of uninitialized value $vm_name in pattern match (m//) at testscript.pl line 164, <STDIN> line 1.
Use of uninitialized value $vm_name in concatenation (.) or string at testscript.pl line 173, <STDIN> line 1.
Can anyone help? I have a feeling it is to do with the subroutine but cannot figure it out exactly. I am new to Perl and have been poring over this for hours.
# !/usr/bin/perl
use strict;
use warnings;
use feature ":5.10";
###############################################################################
# Static Variables
my $templateID = "XXXXXXXXXX";
my $serviceID = "XXXXXXXXXXX";
###############################################################################
###############################################################################
# Global Variables
my $zone;
my $input;
my $zone_ID;
my $vm_name;
my $net_default;
my $net_secondary;
my $vm_ID;
my $start;
my $char_count;
my $ipaddr_1;
my $ipaddr_2;
###############################################################################
# ----------------------- BASIC CONFIGURATIONS [ZONES] ------------------------
print " *** ";
print "
ZONES: paris, milan, berlin, geneva, amsterdam, london, slough";
print " which zone would you like to deploy to? \n ";
# Capture input for ZONE
$input = <STDIN>;
# Data formatting
$input =~ s/^\s+|\s+$|^\t//g;
$zone = lc $input;
# Declare ZONE to USER
print "You are deploying to $zone \n";
# Map input to function
# ZONE PARIS
if ($zone eq "paris") {
print "Deploying to $zone\n";
# assign zone id
$zone_ID = "374b937d-2051-4440-b02c-a314dd9cb27e";
}
# ZONE MILAN
elsif ($zone eq "milan") {
print "Deploying to $zone\n";
# assign zone id
$zone_ID = "58848a37-db49-4518-946a-88911db0ee2b";
}
# ZONE BERLIN
elsif ($zone eq "berlin") {
print "Deploying to $zone\n";
# assign zone id
$zone_ID = "fc129b38-d490-4cd9-acf8-838cf7eb168d";
}
# ZONE GENEVE
elsif ($zone eq "geneva") {
print "Deploying to $zone\n";
# assign zone id
$zone_ID = "1ef96ec0-9e51-4502-9a81-045bc37ecc0a";
}
# ZONE AMSTERDAM
elsif ($zone eq "amsterdam") {
print "Deploying to $zone\n";
# assign zone id
$zone_ID = "3c43b32b-fadf-4629-b8e9-61fb7a5b9bb8";
}
# ZONE SLOUGH
elsif ($zone eq "slough") {
print "Deploying to $zone\n";
# assign zone id
$zone_ID = "5343ddc2-919f-4d1b-a8e6-59f91d901f8e";
}
# ZONE LONDON
elsif ($zone eq "london") {
print "Deploying to $zone\n";
# assign zone id
$zone_ID = "f6b0d029-8e53-413b-99f3-e0a2a543ee1d";
}
# ELSE
else {
print "Please choose a ZONE";
system('./deploy_vm');
}
# SUBROUTINE: Assigning Virtual Machine name
sub vm_name {
# Name of Virtual Machine
print "Enter name of Virtual Machine: \n";
print "[VDC Bug: Please do not include '_' in the name]\n";
my $vm_name = <STDIN>;
# Data formatting
$vm_name =~ s/^\s+|\s+$//g;
print("VM Name: $vm_name");
}
# Apply input filters (no underscores, length constraints)
if (my $vm_name =~ /_/) {
return ('underscore error');
print "Cannot use underscore in VM name, please try again";
vm_name();
}
# DisPlay config
print "
$vm_name will be provisioned with:
1GB HDD and compute offering of 2CPUs with 2GB-RAM";
# ----------------------- NETWORK CONFIGURATIONS ------------------------------
# Gather network information
print "available networks in $zone";
system('cloudmonkey list networks zoneid=$zone_id filter=name,id');
# Users to choose Networks through display
print "";
# Default NIC & IP address
print "please enter Network ID for Default NIC:\n";
$net_default = <STDIN>;
# IP Address DEFAULT
print "please enter IP address\n";
$ipaddr_1 = <STDIN>;
# formatting IP address
$ipaddr_1 =~ m/^\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b/;
# SECONDARY NIC & IP address
print "please enter Network ID for secondary NIC";
$net_secondary = <STDIN>;
# IP Address SECONDARY
print "please enter IP address\n";
$ipaddr_2 = <STDIN>;
# Formatting IP address
$ipaddr_2 =~ m/^\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b/;
print "IP ADDRESS is $ipaddr_2\n";
# Deployment notice
print "... deploying ... \n";
# Provision Machine
# + startvm=false due to provisioning constraints
system('cloudmonkey deploy virtualmachine startvm=false name=$vm_name diplayname=$vm_name zoneid=$zone_id templateid=$templateID serviceofferingid=$serviceID networkids=$net_default ipaddress=$ipaddr_1');
# Add second network to VM
print "Adding second network to VM";
system('cloudmonkey addNicToVirtualMachine networkid=$network_id_second ipaddress=$ip_address_second virtualmachineid=$vm_id');
# Display machine ID
print "$vm_ID";
my $feedback = system('cloudmonkey list networks zoneid=$zone_id filter=name,id');
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# # Sotiris Comments # #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# if (lc($feedback) =~ /error/) {
# Warn user and fall back to known state. }
# User to copy&paste ID
print "Paste VM ID:\n";
$vm_ID = <STDIN>;
# Data Validation
$vm_ID =~ s/^\s+|\s+$//g;
$vm_ID = lc $vm_ID;
$char_count = length($vm_ID);
# IF statement to impliment length validation
if ($char_count != 36) {
print "your ID is too long/short";
}
# Start Virtual Machine?
print "Start Virtual Machine $vm_name?";
$start = <STDIN>;
if ($start eq "yes") {
system('');
}
elsif ($start eq "no") {
system('');
}
# done!

You have posted a poor piece of Perl code that you have presumably written all in one chunk. Programming doesn't work like that, especially in script-like languages, and you should write just two or three lines at a time before testing that it works as you expect. That way, if you have a question at all, it can be about just a few lines of code whose input you know but whose behaviour you can't explain. In this case your Use of uninitialized value errors are the least of your worries.
You really shouldn't delcare everything at the top of your program, C-style. Declarations should be as close as possible to their first point of use.
Comments should be kept to the absolute minimum. It is very rare that a program contains a construct that is incomprehensible without the addition of comments, and your program is inflated around 100% by unnecessary commenting
The mapping between zones and zone IDs is best implemented as a Perl hash, not as an if/elsif/else chain
The vm_name subroutine is never called except in the test if (my $vm_name =~ /_/) { ... }, and your use of my here declares a new $vm_name which will always be undef
The lines you have commented with formatting IP address do no formatting at all. They test whether the variables' contents match the regular expression, and then ignore and discard the result of the test
The calls to system use single quotes, so the string is passed verbatim to the cloudmonkey program, and no interpolations are done. For instance, the variable names $vm_name, $zone_id, $templateID, $serviceID etc. are not replaced by their values
There are several more problems, but I strongly suggest that, as I said at the start, you should write the program incrementally in very small parts. That way you have the encouraging pleasure of seeing your code in action and wortking, as well as knowing for sure that, as you write, you have a firm foundation to build on.

if (my $vm_name =~ /_/)
will always warn Use of uninitialized value $vm_name in pattern match under warnings, as you're mistakenly declaring another lexical with my inside if condition.
Also you should sanitize user input as
my $vm_name = <STDIN>;
returned undefined if the user presses Ctrl-D or there was end of piped input.

You're reached the end of the input, but you pretend you actually read something in. Maybe you want to add
die "Premature EOF\n" if !defined($vm_name);
after
my $vm_name=<STDIN>;

Related

Regular expression to print a string from a command outpout

I have written a function that uses regex and prints the required string from a command output.
The script works as expected. But it's does not support a dynamic output. currently, I use regex for "icmp" and "ok" and print the values. Now, type , destination and return code could change. There is a high chance that command doesn't return an output at all. How do I handle such scenarios ?
sub check_summary{
my ($self) = #_;
my $type = 0;
my $return_type = 0;
my $ipsla = $self->{'ssh_obj'}->exec('show ip sla');
foreach my $line( $ipsla) {
if ( $line =~ m/(icmp)/ ) {
$type = $1;
}
if ( $line =~ m/(OK)/ ) {
$return_type = $1;
}
}
INFO ($type,$return_type);
}
command Ouptut :
PSLAs Latest Operation Summary
Codes: * active, ^ inactive, ~ pending
ID Type Destination Stats Return Last
(ms) Code Run
-----------------------------------------------------------------------
*1 icmp 192.168.25.14 RTT=1 OK 1 second ago
Updated to some clarifications -- we need only the last line
As if often the case, you don't need a regex to parse the output as shown. You have space-separated fields and can just split the line and pick the elements you need.
We are told that the line of interest is the last line of the command output. Then we don't need the loop but can take the last element of the array with lines. It is still unclear how $ipsla contains the output -- as a multi-line string or perhaps as an arrayref. Since it is output of a command I'll treat it as a multi-line string, akin to what qx returns. Then, instead of the foreach loop
my #lines = split '\n', $ipsla; # if $ipsla is a multi-line string
# my #lines = #$ipsla; # if $ipsla is an arrayref
pop #lines while $line[-1] !~ /\S/; # remove possible empty lines at end
my ($type, $return_type) = (split ' ', $lines[-1])[1,4];
Here are some comments on the code. Let me know if more is needed.
We can see in the shown output that the fields up to what we need have no spaces. So we can split the last line on white space, by split ' ', $lines[-1], and take the 2nd and 5th element (indices 1 and 4), by ( ... )[1,4]. These are our two needed values and we assign them.
Just in case the output ends with empty lines we first remove them, by doing pop #lines as long as the last line has no non-space characters, while $lines[-1] !~ /\S/. That is the same as
while ( $lines[-1] !~ /\S/ ) { pop #lines }
Original version, edited for clarifications. It is also a valid way to do what is needed.
I assume that data starts after the line with only dashes. Set a flag once that line is reached, process the line(s) if the flag is set. Given the rest of your code, the loop
my $data_start;
foreach (#lines)
{
if (not $data_start) {
$data_start = 1 if /^\s* -+ \s*$/x; # only dashes and optional spaces
}
else {
my ($type, $return_type) = (split)[1,4];
print "type: $type, return code: $return_type\n";
}
}
This is a sketch until clarifications come. It also assumes that there are more lines than one.
I'm not sure of all possibilities of output from that command so my regular expression may need tweaking.
I assume the goal is to get the values of all columns in variables. I opted to store values in a hash using the column names as the hash keys. I printed the results for debugging / demonstration purposes.
use strict;
use warnings;
sub check_summary {
my ($self) = #_;
my %results = map { ($_,undef) } qw(Code ID Type Destination Stats Return_Code Last_Run); # Put results in hash, use column names for keys, set values to undef.
my $ipsla = $self->{ssh_obj}->exec('show ip sla');
foreach my $line (#$ipsla) {
chomp $line; # Remove newlines from last field
if($line =~ /^([*^~])([0-9]+)\s+([a-z]+)\s+([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)\s+([[:alnum:]=]+)\s+([A-Z]+)\s+([^\s].*)$/) {
$results{Code} = $1; # Code prefixing ID
$results{ID} = $2;
$results{Type} = $3;
$results{Destination} = $4;
$results{Stats} = $5;
$results{Return_Code} = $6;
$results{Last_Run} = $7;
}
}
# Testing
use Data::Dumper;
print Dumper(\%results);
}
# Demonstrate
check_summary();
# Commented for testing
#INFO ($type,$return_type);
Worked on the submitted test line.
EDIT:
Regular expressions allow you to specify patterns instead of the exact text you are attempting to match. This is powerful but complicated at times. You need to read the Perl Regular Expression documentation to really learn them.
Perl regular expressions also allow you to capture the matched text. This can be done multiple times in a single pattern which is how we were able to capture all the columns with one expression. The matches go into numbered variables...
$1
$2

New to Perl - Parsing file and replacing pattern with dynamic values

I am very new to Perl and i am currently trying to convert a bash script to perl.
My script is used to convert nmon files (AIX / Linux perf monitoring tool), it takes nmon files present in a directory, grep and redirect the specific section to a temp file, grep and redirect the associated timestamp to aother file.
Then, it parses data into a final csv file that will be indexed by a a third tool to be exploited.
A sample NMON data looks like:
TOP,%CPU Utilisation
TOP,+PID,Time,%CPU,%Usr,%Sys,Threads,Size,ResText,ResData,CharIO,%RAM,Paging,Command,WLMclass
TOP,5165226,T0002,10.93,9.98,0.95,1,54852,4232,51220,311014,0.755,1264,PatrolAgent,Unclassified
TOP,5365876,T0002,1.48,0.81,0.67,135,85032,132,84928,38165,1.159,0,db2sysc,Unclassified
TOP,5460056,T0002,0.32,0.27,0.05,1,5060,616,4704,1719,0.072,0,db2kmchan64.v9,Unclassified
The field "Time" (Seen as T0002 and really called ZZZZ in NMON) is a specific NMON timestamp, the real value of this timestamp is present later (in a dedicated section) in the NMON file and looks like:
ZZZZ,T0001,00:09:55,01-JAN-2014
ZZZZ,T0002,00:13:55,01-JAN-2014
ZZZZ,T0003,00:17:55,01-JAN-2014
ZZZZ,T0004,00:21:55,01-JAN-2014
ZZZZ,T0005,00:25:55,01-JAN-2014
The NMON format is very specific and can't be exploited directly without being parsed, the timestamp has to be associated with the corresponding value. (A NMON file is almost like a concatenation of numerous different csv files with each a different format, different fileds and so on.)
I wrote the following bash script to parse the section i'm interested in (The "TOP" section which represents top process cpu, mem, io stats per host)
#!/bin/bash
# set -x
################################################################
# INFORMATION
################################################################
# nmon2csv_TOP.sh
# Convert TOP section of nmon files to csv
# CAUTION: This script is expected to be launched by the main workflow
# $DST and DST_CONVERTED_TOP are being exported by it, if not this script will exit at launch time
################################################################
# VARS
################################################################
# Location of NMON files
NMON_DIR=${DST}
# Location of generated files
OUTPUT_DIR=${DST_CONVERTED_TOP}
# Temp files
rawdatafile=/tmp/temp_rawdata.$$.temp
timestampfile=/tmp/temp_timestamp.$$.temp
# Main Output file
finalfile=${DST_CONVERTED_TOP}/NMON_TOP_processed_at_date_`date '+%F'`.csv
###########################
# BEGIN OF WORK
###########################
# Verify exported var are not null
if [ -z ${NMON_DIR} ]; then
echo -e "\nERROR: Var NMON_DIR is null!\n" && exit 1
elif [ -z ${OUTPUT_DIR} ]; then
echo -e "\nERROR: Var OUTPUT_DIR is null!\n" && exit 1
fi
# Check if temp and output files already exists
if [ -s ${rawdatafile} ]; then
rm -f ${rawdatafile}
elif [ -s ${timestampfile} ]; then
rm -f ${timestampfile}
elif [ -s ${finalfile} ]; then
rm -f ${finalfile}
fi
# Get current location
PWD=`pwd`
# Go to NMON files location
cd ${NMON_DIR}
# For each NMON file present:
# To restrict to only PROD env: `ls *.nmon | grep -E -i 'sp|gp|ge'`
for NMON_FILE in `ls *.nmon | grep -E -i 'sp|gp|ge'`; do
# Set Hostname identification
serialnum=`grep 'AAA,SerialNumber,' ${NMON_FILE} | awk -F, '{print $3}' OFS=, | tr [:lower:] [:upper:]`
hostname=`grep 'AAA,host,' ${NMON_FILE} | awk -F, '{print $3}' OFS=, | tr [:lower:] [:upper:]`
# Grep and redirect TOP Section
grep 'TOP' ${NMON_FILE} | grep -v 'AAA,version,TOPAS-NMON' | grep -v 'TOP,%CPU Utilisation' > ${rawdatafile}
# Grep and redirect associated timestamps (ZZZZ)
grep 'ZZZZ' ${NMON_FILE}> ${timestampfile}
# Begin of work
while IFS=, read TOP PID Time Pct_CPU Pct_Usr Pct_Sys Threads Size ResText ResData CharIO Pct_RAM Paging Command WLMclass
do
timestamp=`grep ${Time} ${timestampfile} | awk -F, '{print $4 " "$3}' OFS=,`
echo ${serialnum},${hostname},${timestamp},${Time},${PID},${Pct_CPU},${Pct_Usr},${Pct_Sys},${Threads},${Size},${ResText},${ResData},${CharIO},${Pct_RAM},${Paging},${Command},${WLMclass} \
| grep -v '+PID,%CPU,%Usr,%Sys,Threads,Size,ResText,ResData,CharIO,%RAM,Paging,Command,WLMclass' >> ${finalfile}
done < ${rawdatafile}
echo -e "INFO: Done for Serialnum: ${serialnum} Hostname: ${hostname}"
done
# Go back to initial location
cd ${PWD}
###########################
# END OF WORK
###########################
This works as wanted and generate a main csv file (you'll see in the code that i voluntary don't keep the csv header in the file) wich is a concatenation of all parsed hosts.
But, i have a very large amount of host to treat each day (around 3000 hosts), with this current code and in worst cases, it can takes a few minutes to generate data for 1 host, multiplicated per number of hosts minutes becomes easily hours...
So, this code is really not performer enough to deal with such amount of data
10 hosts represents around 200.000 lines, which represents finally around 20 MB of csv file.
That's not that much, but i think that a shell script is probably not the better choice to manage such a process...
I guess that perl shall be much better at this task (even if the shell script could probably be improved), but my knowledge in perl is (currently) very poor, this is why i ask your help... I think that code should be quite simple to do in perl but i can't get it to work as for now...
One guy used to develop a perl script to manage NMON files and convert them to sql files (to dump these data into a database), i staged it to use its feature and with the help of some shell scripts i manage the sql files to get my final csv files.
But the TOP section was not integrated into that perl script and can't be used to that without being redeveloped.
The code in question:
#!/usr/bin/perl
# Program name: nmon2mysql.pl
# Purpose - convert nmon.csv file(s) into mysql insert file
# Author - Bruce Spencer
# Disclaimer: this provided "as is".
# Date - March 2007
#
$nmon2mysql_ver="1.0. March 2007";
use Time::Local;
#################################################
## Your Customizations Go Here ##
#################################################
# Source directory for nmon csv files
my $NMON_DIR=$ENV{DST_TMP};
my $OUTPUT_DIR=$ENV{DST_CONVERTED_CPU_ALL};
# End "Your Customizations Go Here".
# You're on your own, if you change anything beyond this line :-)
####################################################################
############# Main Program ############
####################################################################
# Initialize common variables
&initialize;
# Process all "nmon" files located in the $NMON_DIR
# #nmon_files=`ls $NMON_DIR/*.nmon $NMON_DIR/*.csv`;
#nmon_files=`ls $NMON_DIR/*.nmon`;
if (#nmon_files eq 0 ) { die ("No \*.nmon or csv files found in $NMON_DIR\n"); }
#nmon_files=sort(#nmon_files);
chomp(#nmon_files);
foreach $FILENAME ( #nmon_files ) {
#cols= split(/\//,$FILENAME);
$BASEFILENAME= $cols[#cols-1];
unless (open(INSERT, ">$OUTPUT_DIR/$BASEFILENAME.sql")) {
die("Can not open /$OUTPUT_DIR/$BASEFILENAME.sql\n");
}
print INSERT ("# nmon version: $NMONVER\n");
print INSERT ("# AIX version: $AIXVER\n");
print INSERT ("use nmon;\n");
$start=time();
#now=localtime($start);
$now=join(":",#now[2,1,0]);
print ("$now: Begin processing file = $FILENAME\n");
# Parse nmon file, skip if unsuccessful
if (( &get_nmon_data ) gt 0 ) { next; }
$now=time();
$now=$now-$start;
print ("\t$now: Finished get_nmon_data\n");
# Static variables (number of fields always the same)
##static_vars=("LPAR","CPU_ALL","FILE","MEM","PAGE","MEMNEW","MEMUSE","PROC");
##static_vars=("LPAR","CPU_ALL","FILE","MEM","PAGE","MEMNEW","MEMUSE");
#static_vars=("CPU_ALL");
foreach $key (#static_vars) {
&mk_mysql_insert_static($key);;
$now=time();
$now=$now-$start;
print ("\t$now: Finished $key\n");
} # end foreach
# Dynamic variables (variable number of fields)
##dynamic_vars=("DISKBSIZE","DISKBUSY","DISKREAD","DISKWRITE","DISKXFER","ESSREAD","ESSWRITE","ESSXFER","IOADAPT","NETERROR","NET","NETPACKET");
#dynamic_vars=("");
foreach $key (#dynamic_vars) {
&mk_mysql_insert_variable($key);;
$now=time();
$now=$now-$start;
print ("\t$now: Finished $key\n");
}
close(INSERT);
# system("gzip","$FILENAME");
}
exit(0);
############################################
############# Subroutines ############
############################################
##################################################################
## Extract CPU_ALL data for Static fields
##################################################################
sub mk_mysql_insert_static {
my($nmon_var)=#_;
my $table=lc($nmon_var);
my #rawdata;
my $x;
my #cols;
my $comma;
my $TS;
my $n;
#rawdata=grep(/^$nmon_var,/, #nmon);
if (#rawdata < 1) { return(1); }
#rawdata=sort(#rawdata);
#cols=split(/,/,$rawdata[0]);
$x=join(",",#cols[2..#cols-1]);
$x=~ s/\%/_PCT/g;
$x=~ s/\(MB\)/_MB/g;
$x=~ s/-/_/g;
$x=~ s/ /_/g;
$x=~ s/__/_/g;
$x=~ s/,_/,/g;
$x=~ s/_,/,/g;
$x=~ s/^_//;
$x=~ s/_$//;
print INSERT (qq|insert into $table (serialnum,hostname,mode,nmonver,time,ZZZZ,$x) values\n| );
$comma="";
$n=#cols;
$n=$n-1; # number of columns -1
for($i=1;$i<#rawdata;$i++){
$TS=$UTC_START + $INTERVAL*($i);
#cols=split(/,/,$rawdata[$i]);
$x=join(",",#cols[2..$n]);
$x=~ s/,,/,-1,/g; # replace missing data ",," with a ",-1,"
print INSERT (qq|$comma("$SN","$HOSTNAME","$MODE","$NMONVER",$TS,"$DATETIME{#cols[1]}",$x)| );
$comma=",\n";
}
print INSERT (qq|;\n\n|);
} # end mk_mysql_insert
##################################################################
## Extract CPU_ALL data for variable fields
##################################################################
sub mk_mysql_insert_variable {
my($nmon_var)=#_;
my $table=lc($nmon_var);
my #rawdata;
my $x;
my $j;
my #cols;
my $comma;
my $TS;
my $n;
my #devices;
#rawdata=grep(/^$nmon_var,/, #nmon);
if ( #rawdata < 1) { return; }
#rawdata=sort(#rawdata);
$rawdata[0]=~ s/\%/_PCT/g;
$rawdata[0]=~ s/\(/_/g;
$rawdata[0]=~ s/\)/_/g;
$rawdata[0]=~ s/ /_/g;
$rawdata[0]=~ s/__/_/g;
$rawdata[0]=~ s/,_/,/g;
#devices=split(/,/,$rawdata[0]);
print INSERT (qq|insert into $table (serialnum,hostname,time,ZZZZ,device,value) values\n| );
$n=#rawdata;
$n--;
for($i=1;$i<#rawdata;$i++){
$TS=$UTC_START + $INTERVAL*($i);
$rawdata[$i]=~ s/,$//;
#cols=split(/,/,$rawdata[$i]);
print INSERT (qq|\n("$SN","$HOSTNAME",$TS,"$DATETIME{$cols[1]}","$devices[2]",$cols[2])| );
for($j=3;$j<#cols;$j++){
print INSERT (qq|,\n("$SN","$HOSTNAME",$TS,"$DATETIME{$cols[1]}","$devices[$j]",$cols[$j])| );
}
if ($i < $n) { print INSERT (","); }
}
print INSERT (qq|;\n\n|);
} # end mk_mysql_insert_variable
########################################################
### Get an nmon setting from csv file ###
### finds first occurance of $search ###
### Return the selected column...$return_col ###
### Syntax: ###
### get_setting($search,$col_to_return,$separator)##
########################################################
sub get_setting {
my $i;
my $value="-1";
my ($search,$col,$separator)= #_; # search text, $col, $separator
for ($i=0; $i<#nmon; $i++){
if ($nmon[$i] =~ /$search/ ) {
$value=(split(/$separator/,$nmon[$i]))[$col];
$value =~ s/["']*//g; #remove non alphanum characters
return($value);
} # end if
} # end for
return($value);
} # end get_setting
#####################
## Clean up ##
#####################
sub clean_up_line {
# remove characters not compatible with nmon variable
# Max rrdtool variable length is 19 chars
# Variable can not contain special characters (% - () )
my ($x)=#_;
# print ("clean_up, before: $i\t$nmon[$i]\n");
$x =~ s/\%/Pct/g;
# $x =~ s/\W*//g;
$x =~ s/\/s/ps/g; # /s - ps
$x =~ s/\//s/g; # / - s
$x =~ s/\(/_/g;
$x =~ s/\)/_/g;
$x =~ s/ /_/g;
$x =~ s/-/_/g;
$x =~ s/_KBps//g;
$x =~ s/_tps//g;
$x =~ s/[:,]*\s*$//;
$retval=$x;
} # end clean up
##########################################
## Extract headings from nmon csv file ##
##########################################
sub initialize {
%MONTH2NUMBER = ("jan", 1, "feb",2, "mar",3, "apr",4, "may",5, "jun",6, "jul",7, "aug",8, "sep",9, "oct",10, "nov",11, "dec",12 );
#MONTH2ALPHA = ( "junk","jan", "feb", "mar", "apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec" );
} # end initialize
# Get data from nmon file, extract specific data fields (hostname, date, ...)
sub get_nmon_data {
my $key;
my $x;
my $category;
my %toc;
my #cols;
# Read nmon file
unless (open(FILE, $FILENAME)) { return(1); }
#nmon=<FILE>; # input entire file
close(FILE);
chomp(#nmon);
# Cleanup nmon data remove trainig commas and colons
for($i=0; $i<#nmon;$i++ ) {
$nmon[$i] =~ s/[:,]*\s*$//;
}
# Get nmon/server settings (search string, return column, delimiter)
$AIXVER =&get_setting("AIX",2,",");
$DATE =&get_setting("date",2,",");
$HOSTNAME =&get_setting("host",2,",");
$INTERVAL =&get_setting("interval",2,","); # nmon sampling interval
$MEMORY =&get_setting(qq|lsconf,"Good Memory Size:|,1,":");
$MODEL =&get_setting("modelname",3,'\s+');
$NMONVER =&get_setting("version",2,",");
$SNAPSHOTS =&get_setting("snapshots",2,","); # number of readings
$STARTTIME =&get_setting("AAA,time",2,",");
($HR, $MIN)=split(/\:/,$STARTTIME);
if ($AIXVER eq "-1") {
$SN=$HOSTNAME; # Probably a Linux host
} else {
$SN =&get_setting("systemid",4,",");
$SN =(split(/\s+/,$SN))[0]; # "systemid IBM,SN ..."
}
$TYPE =&get_setting("^BBBP.*Type",3,",");
if ( $TYPE =~ /Shared/ ) { $TYPE="SPLPAR"; } else { $TYPE="Dedicated"; }
$MODE =&get_setting("^BBBP.*Mode",3,",");
$MODE =(split(/: /, $MODE))[1];
# $MODE =~s/\"//g;
# Calculate UTC time (seconds since 1970)
# NMON V9 dd/mm/yy
# NMON V10+ dd-MMM-yyyy
if ( $DATE =~ /[a-zA-Z]/ ) { # Alpha = assume dd-MMM-yyyy date format
($DAY, $MMM, $YR)=split(/\-/,$DATE);
$MMM=lc($MMM);
$MON=$MONTH2NUMBER{$MMM};
} else {
($DAY, $MON, $YR)=split(/\//,$DATE);
$YR=$YR + 2000;
$MMM=$MONTH2ALPHA[$MON];
} # end if
## Calculate UTC time (seconds since 1970). Required format for the rrdtool.
## timelocal format
## day=1-31
## month=0-11
## year = x -1900 (time since 1900) (seems to work with either 2006 or 106)
$m=$MON - 1; # jan=0, feb=2, ...
$UTC_START=timelocal(0,$MIN,$HR,$DAY,$m,$YR);
$UTC_END=$UTC_START + $INTERVAL * $SNAPSHOTS;
#ZZZZ=grep(/^ZZZZ,/,#nmon);
for ($i=0;$i<#ZZZZ;$i++){
#cols=split(/,/,$ZZZZ[$i]);
($DAY,$MON,$YR)=split(/-/,$cols[3]);
$MON=lc($MON);
$MON="00" . $MONTH2NUMBER{$MON};
$MON=substr($MON,-2,2);
$ZZZZ[$i]="$YR-$MON-$DAY $cols[2]";
$DATETIME{$cols[1]}="$YR-$MON-$DAY $cols[2]";
} # end ZZZZ
return(0);
} # end get_nmon_data
It almost (i say almost because with recent NMON versions it can sometimes have some issue when no data present) does the job, and it does it much much faster that would do my shell script if i would use it for these section
This is why i think perl shall be a perfect solution.
Off course, i don't ask anyone to convert my shell script into something final in perl, but at least to give me to right direction :-)
I really thank anyone in advance for your help !
Normally i am strongly opposed to questions like this but our production systems are down and until they are fixed i do not really have all that much to do...
Here is some code that might get you started. Please consider it pseudo code as it is completely untested and probably won't even compile (i always forget some parantheses or semicolons and as i said, the actual machines that can run code are unreachable) but i commented a lot and hopefully you will be able to modify it to your actual needs and get it to run.
use strict;
use warnings;
open INFILE, "<", "path/to/file.nmon"; # Open the file.
my #topLines; # Initialize variables.
my %timestamps;
while <INFILE> # This will walk over all the lines of the infile.
{ # Storing the current line in $_.
chomp $_; # Remove newline at the end.
if ($_ =~ m/^TOP/) # If the line starts with TOP...
{
push #topLines, $_; # ...store it in the array for later use.
}
elsif ($_ =~ m/^ZZZZ/) # If it is in the ZZZZ section...
{
my #fields = split ',', $_; # ...split the line at commas...
my $timestamp = join ",", $fields(2), $fields(3); # ...join the timestamp into a string as you wish...
$timestamps{$fields(1)} = $timestamp; # ...and store it in the hash with the Twhatever thing as key.
}
# This iteration could certainly be improved with more knowledge
# of how the file looks. For example the search could be cancelled
# after the ZZZZ section if the file is still long.
}
close INFILE;
open OUTFILE, ">", "path/to/output.csv"; # Open the file you want your output in.
foreach (#topLines) # Iterate through all elements of the array.
{ # Once again storing the current value in $_.
my #fields = split ',', $_; # Probably not necessary, depending on how output should be formated.
my $outstring = join ',', $fields(0), $fields(1), $timestamps{$fields(2)}; # And whatever other fields you care for.
print OUTFILE $outstring, "\n"; # Print.
}
close OUTFILE;
print "Done.\n";

Whats wrong with this code to read file?

I have been trying to read a file called "perlthisfile.txt" which is basically the output of nmap on my computer.
I want to get only the ip addresses printed out, so i wrote the following code but it is not working:
#!/usr/bin/perl
use strict;
use warnings;
use Scalar::Util qw(looks_like_number);
print"\n running \n";
open (MYFILE, 'perlthisfile.txt') or die "Cannot open file\n";
while(<MYFILE>) {
chomp;
my #value = split(' ', <MYFILE>);
print"\n before foreach \n";
foreach my $val (#value) {
if (looks_like_number($val)) {
print "\n looks like number block \n";
if ($val == /(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\:\d{1,5})/) {
print "\n$val\n";
}
}
}
}
close(MYFILE);
exit 0;
And when i ran this code the output was:
running
before foreach
before foreach
looks like number block
before foreach
looks like number block
before foreach
looks like number block
My perlthisfile.txt:
Starting Nmap 6.00 ( http://nmap.org ) at 2013-10-16 22:59 EST
Nmap scan report for BoB2.iiNet (10.1.1.1)
Nmap scan report for android-fbff3c3812154cdc (10.1.1.3)
All 1000 scanned ports on android-fbff3c3812154cdc (10.1.1.3) are closed
Nmap scan report for 10.1.1.5
All 1000 scanned ports on 10.1.1.5 are open|filtered
Nmap scan report for 10.1.1.6
All 1000 scanned ports on 10.1.1.6 are closed
Several issues here. As #toolic said, calling <MYFILE> inside the split is probably not what you want - it will read the next record from the file, use $_ instead.
Also, you are using == with a regex, you should use the binding operator, =~ (== is only used for numeric comparisons in Perl):
if ($val =~ /(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\:\d{1,5})/){
I suggest that looks_like_number is redundant if the regex works. I suspect that you are using it because == gives something like isn't numeric in numeric eq (==) depending on the version of perl you are using.
You had a few errors, one of which is regex which should have optional part for port number (: and following \d{1,5})
#!/usr/bin/perl
use strict;
use warnings;
open (my $MYFILE, '<', 'perlthisfile.txt') or die $!;
my $looks_like_ip = qr/( \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3} (?: : \d{1,5})? )/x;
while (<$MYFILE>) {
chomp;
my #value = split;
print"\n before foreach \n";
foreach my $val (#value) {
if (my ($match) = $val =~ /$looks_like_ip/){
print "\n$match\n";
}
# else { print "$val doesn't contain IP\n" }
}
}
close($MYFILE) or warn $!;
If this is what it looks to be, which is a quick hack to extract IPs, you might get away with something simple such as:
perl -nlwe '/((?:\d+\.)+\d+)/ && print $1' perlthisfile.txt
Which is to say, not a very strict regex by any means, it just matches numbers joined by periods. If you'd like to only print unique IPs, you can make use of a hash to dedupe:
perl -nlwe '/((?:\d+\.)+\d+)/ && !$seen{$1}++ && print $1" perlthisfile.txt
With a slightly tighter regex that also matches port numbers:
perl -nlwe '/((?:\d+[\.:]){3,4}\d+)/ && print $1' perlthisfile.txt
This will disallow shorter chains of numbers, and allow for a port number.
This last regex explained:
/( # opening parenthesis, starts a string capture
(?: # a non-capturing parenthesis
\d+ # match a number, repeated one or more times
[\.:] # [ ... ] is a character class, it matches one of the literal
# characters inside it, and only one time
){3,4} # closing the non-capturing parenthesis, adding a quantifier
# that says this parenthesis can match 3 or 4 times
\d+ # match one or more numbers
)/x # close capturing parenthesis (added `/x` switch)
The /x switch is just so that you can use the above regex as-is, with comments and whitespace.
The logic behind this is simply: We want a string consisting of a number followed by a period or a colon. We want this string 3 or 4 times. End with another number.
The + and {3,4} are quantifiers, they dictate how many times the item to the left of it is supposed to match. By default, every item matches one time, but by using a quantifier you can change that. + is shorthand for {1,}, and you also have:
? -> {1,0}
* -> {0,}
The syntax is {min,max}, and when a number is missing, that means as many times as possible.

Pattern matching an array with specific start and end words/characters

Have some output in an array which I am trying to pull details from where the output begins with a specific word and ends with a specific word/character. This output is then to be printed to the screen.
The output in the array which I am working with is:
router rip
version 2
redistribute bgp 45134 metric 3
passive-interface Serial1/3:1.333
passive-interface Serial3/1:3.333
passive-interface Serial3/1:5.333
passive-interface Serial3/2:1.333
passive-interface Serial3/4:1.333
passive-interface Serial3/4:17.333
passive-interface Serial6/1:1.333
no auto-summary
!
address-family ipv4 vrf TestVRF-0001
redistribute bgp 45134 metric 3
network 10.0.0.0
no auto-summary
version 2
exit-address-family
!
The perl code I have generated so far is below:
elsif ( $action eq "show_vrf1" ) {
my $cmd = "show run | begin router rip";
my #lines = $s->cmd(String => $cmd,
Prompt => "/$enableprompt/",
Timeout => 10);
foreach my $line (#lines) {
if(/address-family ipv4 vrf TestVRF-0001.*?!/){
$result=$1;
print $result;
}
}
}
Which I am wanting to only pull the below out of the array:
address-family ipv4 vrf TestVRF-0001
redistribute bgp 45134 metric 3
network 10.0.0.0
no auto-summary
version 2
exit-address-family
!
For some reason when I run the script, I just get a blank screen with no data pulled from the array.
if($line =~ /address-family ipv4 vrf TestVRF-0001.*?!/){
instead of
if(/address-family ipv4 vrf TestVRF-0001.*?!/){
?
You match $line against the regular expression. The output you expect is not on one line, so no line can match it.
You can store the whole multiline string in a scalar:
my $output = join q(), $s->cmd(...);
And then, you can retrieve the output, if you use parentheses to really capture a part of the string:
if ($output =~ /(address-family ipv4 vrf TestVRF-0001.*?!)/s) {
my $result = $1;
Note that the /s modifier is needed to make dot match a newline, which it normaly does not.

Getting all hostnames from IP address in Perl

I'm trying to find a way to get all hostnames that resolve to an IP address.
The gethostbyaddr function appears to only retrieve the first record from DNS (no matter if it's in scalar or list context).
Example:
my $hostname = gethostbyaddr(inet_aton($ip_to_check), AF_INET);
$print($hostname); //output: joe.example.com
my #hostnames = gethostbyaddr(inet_aton($ip_to_check), AF_INET);
foreach my $hostname (#hostnames){
print "(", join(',',#hostnames), ")"; //output: (joe.example.com,,2,4,?)
}
From the terminal:
$ host 192.168.1.5
5.1.168.192.in-addr.arpa domain name pointer joe.example.com.
5.1.168.192.in-addr.arpa domain name pointer john.example.com.
I've heard that Net::DNS is a little more robust, but I haven't had any luck getting that to pull all entries as well.
I used a combination of answers given here and elsewhere on stack overflow to find the answer I was looking for.
# create new Resolver Object
my $res = Net::DNS::Resolver->new;
# change IP from 192.168.1.15 to 15.1.168.192.in-addr.arpa for searching
my $target_IP = join('.', reverse split(/\./, $ip_to_check)).".in-addr.arpa";
# query DNS
my $query = $res->query("$target_IP", "PTR");
# if a result is found
if ($query){
print("Resolves to:\n");
# for every result, print the IP address
foreach my $rr ($query->answer){
# show all unless the type is PTR (pointer to a canonical name)
next unless $rr->type eq "PTR";
# remove the period at the end
printf(substr($rr->rdatastr, 0, -1));
}
}
The gethostby... interface is quite old and clunky, being defined back in primeval times before Perl got references and pretensions to OO. And it doesn't work the way you're trying to use it. When used in list context, it returns the primary name as the first element and a space-separated(!) list of aliases as the second:
my ($hostname, $aliases) = gethostbyaddr($addr, AF_INET);
my #hostname = ($hostname, split ' ', $aliases);
say join ' ', #hostname;
Now that's the theory; I didn't locate any IP addresses with multiple PTR records offhand, so I can't test if gethostbyaddr will actually return them -- it probably depends on your underlying C runtime as well -- but it does work if you use gethostbyname with a CNAMEd name, for instance.
Here's a small program I use to lookup all PTR records for a netmask (for example 192.0.2.0/28 ) when doing abuse tracking tasks. It sends up to 15 queries a second and when they are all sent then starts reading the responses (so it'd need a little work to function properly for bigger net blocks).
#!/usr/bin/env perl
use strict;
use warnings;
use Net::Netmask;
use Net::DNS;
#ARGV or die "$0 ip/cidr\n";
my $block = Net::Netmask->new(shift);
my $res = Net::DNS::Resolver->new;
my %sockets;
my $i = 0;
for my $i (1 .. $block->size - 1) {
my $ip = $block->nth($i);
my $reverse_ip = join ".", reverse split m/\./, $ip;
$reverse_ip .= ".in-addr.arpa";
#print "$ip\n";
my $bgsock = $res->bgsend($reverse_ip, 'PTR');
$sockets{$ip} = $bgsock;
sleep 1 unless $i % 15;
}
$i = 0;
for my $i (1 .. $block->size - 1) {
my $ip = $block->nth($i);
my $socket = $sockets{$ip};
my $wait = 0;
until ($res->bgisready($socket)) {
print "waiting for $ip\n" if $wait > 0;
sleep 1 + $wait;
$wait++;
}
my $packet = $res->bgread($socket);
my #rr = $packet->answer;
printf "%-15s %s\n", $ip, $res->errorstring
unless #rr;
for my $rr (#rr) {
printf "%-15s %s\n", $ip, $rr->string;
}
}
I don't think this is a well-formed problem statement. In the general case, there's a nearly infinite number of DNS names that could resolve to any IP address, even unknown to the party that holds the address. Reverse-lookups are fundamentally unreliable, and are not capable of answering the question the poster would like, since all names for an IP do not need to be in the visible reverse map.
The first answer, which enumerates the reverse map, is the best one can do, but it will miss any names that have not been entered in the map.
This is what I have used:
sub getauthoritivename
{
my ($printerdns)=#_;
my $res = Net::DNS::Resolver->new(searchlist=>$config->{searchlist});
my $query = $res->search($printerdns);
if ($query)
{
foreach my $rr ($query->answer)
{
next unless $rr->type eq "A";
print $rr->name;
}
}
else
{
warn "query failed: ", $res->errorstring, "\n";
return 0;
}
}
As long as $rr->name finds names, it keeps adding them.