CSV File Formatting - perl

I wrote a perl script to output a text file filled with ip addresses and ports that i scanned into microsoft excel. Now that the data is in excel my boss wants me to organize the file in csv format such as
Server, port, protocol, random, random, random
ns1, 25, tcp, stuff, stuff, stuff
Can any one help me with this Please?
Code:
#!/usr/bin/perl
$input = `Cat /cygdrive/c/Windows/System32/test11.txt | grep -v 'SYN Stealth'`;
chomp input;
$output =" /cygdrive/c/Users/bpaul/Desktop/194.csv ";
if (! -e "$output")
{
`touch $output`;
}
open (OUTPUTFILE, ">$output") || die "Can't Open file $output";
print OUTPUTFILE "$input\n";
close (OUTPUTFILE);
Here is a piece of my file
Nmap scan report for 69.25.194.2 Host is up (0.072s latency). Not shown: 9992 filtered ports PORT STATE SERVICE 25/tcp open smtp
80/tcp open http
82/tcp open xfer
443/tcp open
https 4443/tcp closed
pharos 5666/tcp closed
nrpe 8080/tcp closed
http-proxy 9443/tcp closed tungsten-https
So far my code took my txt file and outputted it to excel now I need to format the data like this:
Desired Output:
Server, port, protocol, random, random, random
ns1, 25, tcp, stuff, stuff, stuff

I'm assuming you meant 69.25.194.2 when you said ns1.
use strict;
use warnings;
use Text::CSV_XS qw( );
my $csv = Text::CSV_XS->new({ binary => 1, eol => "\n" });
$csv->print(\*STDOUT, [qw(
Server
port
protocol
random
random
random
)]);
my $host = '[unknown]';
while (<>) {
$host = $1 if /Nmap scan report for (\S+)/;
my ($port, $protocol) = m{(\d+)/(\w+) (?:open|closed)/
or next;
$csv->print(\*STDOUT, [
$host,
$port,
$protocol,
'stuff',
'stuff',
'stuff',
]);
}
Usage:
grep -v 'SYN Stealth' /cygdrive/c/Windows/System32/test11.txt | perl to_csv.pl > /cygdrive/c/Users/bpaul/Desktop/194.csv
Text::CSV_XS
Update: Replaced hardcoded ns1 with address of scanned machine.
Update: Replaced generic usage with what the OP would use.

Related

Error 500 when with perl cgi - but not any of the common pitfalls

I have a very tricky to diagnose perl problem, that has been seriously hampering my ability to maintain a perl/cgi website. It usually occurs when editing a script - after a change I get error 500, and then after I revert it it wont work again unless I delete the file and start from scratch, however I currently have a state which it can be reproduced by the following simple two scripts which show just how crazy this bug is:
file1.pl
#! /usr/bin/perl
use CGI::Carp qw(fatalsToBrowser warningsToBrowser);
print "content-type: text/html\n\nIt works";
file2.pl
#! /usr/bin/perl
use CGI::Carp qw(fatalsToBrowser warningsToBrowser);
print "content-type: text/html\n\nIt works";
(Ie... they're identical)
server.com/cgi-bin/file1.pl works
server.com/cgi-bin/file2.pl results in error 500
Both files have the same size and md5 hash.
Both have the same permissions (755) and the same owner and group.
Both are in the correct folder (hosting supplied cgi-bin).
Both were uploaded in text mode.
Both work with local perl interpreters.
If i rename file1 -> file3, file2 -> file1, and file3->file2, (ie swapping both files), now file2.pl works and file1.pl doesn't. So my guess is some state is attached to the files themselves.
If i edit the files in filezilla and re-upload (eg add some whitespace after a semicolon), same behaviour occurs with the re-uploaded files.
My error 500 page is set to auto-retry using a meta refresh (in case of out memory errors, etc), and it doesn't go away after countless refreshes. It doesn't seem to matter which ones is accessed first.
I do not have access to the http error_log on this hosting so do not know the reason for the failure. The error also occurs without the "use error messages to browser" diagnostic line.
Can anyone give me a hint as to what this could be and help me fix it?
What you describe can be either caused by some problem on your hosting provider side (some bad caching, or transparent proxies, or any other magic), or—and that is what I think it is—still caused by wrong file permissions or line breaks, even if your file manager reports that everything is good.
If I'm reading your description correctly you basically
can put a script and it will work, but
cannot edit it as it will stop working after that.
As you don't have shell access, just put the following small script to the same directory and run it (hope it will run as you are not going to edit it):
#!/usr/bin/perl
use strict;
use warnings;
print "Content-Type: text/plain\n\n";
opendir( my $dirh, "." );
my #files = grep { -f $_; } readdir $dirh;
closedir $dirh;
foreach my $file (#files) {
my #stat = stat $file;
my ($dev, $ino, $mode, $nlink, $uid, $gid, $rdev,
$size, $atime, $mtime, $ctime, $blksize, $blocks
) = stat($file);
my $octmode = sprintf "%04o", $mode & 07777;
print "$file\tmode=$octmode\tuid=$uid\tgid=$gid\tsize=$size\t";
if ( -r $file ) {
open( my $fh, $file );
my $firstline = <$fh>;
print $firstline =~ /\r\n/ ? "crlf\n" : "lf\n";
close $fh;
} else {
print "can't read\n";
}
}
It will show the real permissions, linebreaks, and size of the files—those taken from the server's filesystem, not which your FTP client shows.
Maybe it's worth adding MD5 or SHA1 hash calculation to this script but not sure if you have Digest::MD5 or Digest::SHA1 available.
If you see the same output for test1.pl and test2.pl, just go ahead and contact your hosting provider's support.
My guess: the files don't use the same newline convention.
You can check this (in a Unix shell) using the file command.
Not being able to inspect the errorlog is a big headache.
Nevertheless, I suspect that the cause is still most likely line endings. I would upload a script to examine all of your files like the following:
#!/usr/bin/env perl
use strict;
use warnings;
use autodie;
use CGI qw(header);
use CGI::Carp qw(fatalsToBrowser warningsToBrowser);
use File::stat;
print header('text/plain');
my $fmt = "%-15s %4s %4s %4s %7s %4s %4s\n";
printf $fmt, qw(File mode uid gid size lf crlf);
printf $fmt, map { '-' x $_ } $fmt =~ /(\d+)/g;
opendir my $dh, '.';
while ( my $file = readdir $dh ) {
next unless -f $file;
my $stat = stat $file;
my %chars;
my $data = do { local ( #ARGV, $/ ) = $file; <> };
$chars{$1}++ while $data =~ /(\R)/g;
printf $fmt, $file, sprintf( "%04o", $stat->mode & 07777 ), $stat->uid,
$stat->gid, $stat->size, map { $_ // 0 } #chars{ "\n", "\r\n" };
}
Outputs:
Content-Type: text/plain; charset=ISO-8859-1
File mode uid gid size lf crlf
--------------- ---- ---- ---- ------- ---- ----
env.cgi 0775 0 0 266 25 0
le.pl 0775 501 0 696 28 0
lineendings.pl 0755 501 0 516 30 0
mywiki.pl 0755 501 0 226947 0 6666
test.cgi 0755 0 0 2071 65 0
wiki.pl 0755 0 0 219231 6494 0
For additional testing, I would recommend executing each of the scripts using system and inspecting the error conditions if there are any.
I have had the same problem, got help from user BOC as below:
"You may have problem with encoding of characters. Some editors replace some characters by very close characters when you save files (for example " by “). Try changing editor (notepad++ works well on windows). – BOC"
I downloaded and used Notepad++ instead of Notepad and Winword; It works now for me.

How to find Subnet of the given IP using perl cgi?

I need to find the subnet of any IP that user entered. I am executing the "ypcat -k netmasks.byaddr" unix command in perl to find netmasks. It is working fine in command line execution. But it is not working while executing in webserver.
The below is the code.
use strict;
use CGI;
my $page=new CGI;
print $page->header;
my $ipaddress=$page->param("ip");
my #splitted=split(/\./,$ipaddress);
my $part1=$splitted[0].".".$splitted[1].".".$splitted[2];
my $part2=$splitted[3];
my $comma1="ypcat -k netmasks.byaddr|grep -w $part1|awk '{print \$1}'|awk -F. '{print \$4}'|sort -g";
my $comm2="ypcat -k netmasks.byaddr|grep -w $part1|sort";
my #out=`$comm1`;
my #out2=`$comm2`;
my $match;my $sub;my $found;
foreach my $i(#out){
chomp($i);
if($part2 > $i){
$sub=$i;
$found=$part1.".".$sub;
}
}
my (#matched) = grep $_=~m/$found/, #out2;
chomp(#matched);
print "#matched\n";
The above is the code I am using to find subnet of the given IP. In that the execution of "$comm1" and "$comm2" is failing.
Is any other way to find subnet of the user entered IP using Perl script?
Thanks,
Madhan
You can use NetAddr::IP module for this job as given below:
#!/usr/local/bin/perl
use strict;
use warnings;
use CGI;
use NetAddr::IP;
my $page = new CGI;
print $page->header;
my $ipaddress = $page->param("ip");
my $ip = NetAddr::IP->new($ipaddress);
print "The address is ", $ip->addr, " with mask ", $ip->mask, "\n" ;

Use a perl script to parse a file then update /etc/hosts

Im working on one last perl script to update my /etc/hosts file, but am stuck and wondered if someone can help please?
I have a text file with an IP in it, and need to have my perl script read this, which iv done, but now im stuck on updating the /etc/hosts file.
here is my script so far:
#!/usr/bin/perl
use strict;
my $ip_to_update;
$ip_to_update = `cat /web_root/ip_update/ip_update.txt | awk {'print \$5'}` ;
print "ip = $ip_to_update";
I then need to find an entry in /etc/hosts like
remote.host.tld 192.168.0.20
so i know i need to parse it for remote.host.tld and then replace the second bit, but because the ip wont be the same i cant just do a straight replace.
Can anyone help with the last bit please as im stuck :(
Thankyou!
Your substitution will look like this:
s#^.*\s(remote\.host\.tld)\s*$#$ip_to_update\t$1#
Replacement can be done in one line:
perl -i -wpe "BEGIN{$ip=`awk {'print \$5'} /web_root/ip_update/ip_update.txt`} s#^.*\s(remote\.host\.tld)\s*$#$ip\t$1#"'
Ok, I updated my script to include the file edit etc all in one. Might not be the best way to do it, but it works :)
#!/usr/bin/perl
use strict;
use File::Copy;
my $ip_to_update; # IP from file
my $fh_r; # File handler for reading
my $fh_w; # File handler for writing
my $file_read = "/etc/hosts"; # File to read in
my $file_write = "/etc/hosts.new"; # File to write out
my $file_backup = "/etc/hosts.bak"; # File to copy original to
# Awks the IP from text file
$ip_to_update = `/bin/awk < /web_root/ip_update/ip_update.txt {'print \$5'}` ;
# Open File Handlers
open( $fh_r, '<', $file_read ) or die "Can't open $file_read: $!";
open( $fh_w, '>', $file_write ) or die "Can't open $file_write: $!";
while ( my $line = <$fh_r> )
{
if ( $line =~ /remote.host.tld/ )
{
#print $fh_w "# $line";
}
else
{
print $fh_w "$line";
}
}
chomp($ip_to_update); # Remove newlines
print $fh_w "$ip_to_update remote.host.tld\n";
# Prints out new line with new ip and hostname
# Close file handers
close $fh_r;
close $fh_w;
move("$file_read","$file_backup"); # Moves original file to .bak
move("$file_write","$file_read"); # Moves new file to original file loaction

How can I replace only complete IP addresses in a file using Perl?

I used the following Perl syntax in order to replace strings or IP address in a file:
OLD=aaa.bbb.ccc.ddd (old IP address)
NEW=yyy.zzz.www.qqq (new IP address)
export OLD
export NEW
perl -pe 'next if /^ *#/; s/\Q$ENV{OLD }\E/$1$ENV{NEW }$2/' file
example of problem:
I want to change the IP address in file from 1.1.1.1 to 5.5.5.5
But I get the following:
more file (before change)
11.1.1.10 machine_moon1
more file (after change)
15.5.5.50 machine_moon1
According to "after change example) the IP "11.1.1.10" must to stay as it is , because I want to change only the 1.1.1.1 and not 11.1.1.10
I need help about my perl one line syntax:
How to change my perl syntax only according to the following rule:
RULE: Not change the IP address if:left IP side or right IP side have number/s
Example
IP=1.1.1.1
IP=10.10.1.11
IP=yyy.yyy.yyy.yyy
[number]1.1.1.1[number] - then not replace
[number]10.10.1.11[number] - then not replace
[number]yyy.yyy.yyy.yyy[number] - then not replace
Other cases:
[any character beside number ]yyy.yyy.yyy.yyy[[any character beside number ]] - then replace
Here's what you start with:
OLD=1.1.1.1
NEW=5.5.5.5
export OLD
export NEW
~/sandbox/$ cat file
1.1.1.10 machine1
11.1.1.10 machine2
11.1.1.1 machine3
1.1.1.1 machine4
A1.1.1.1 machine5
A1.1.1.1 machine6
1.1.1.1Z machine7
If you anchor the patterns to only match on word boundaries or non-digits (see perlre), you should only match a complete IP address:
~/sandbox/$ perl -pe 'next if /^ *#/; s/(\b|\D)$ENV{OLD}(\b|\D)/$1$ENV{NEW}$2/' file
1.1.1.10 machine1
11.1.1.10 machine2
11.1.1.1 machine3
5.5.5.5 machine4
A5.5.5.5 machine5
A5.5.5.5Z machine6
5.5.5.5Z machine7
You should use look-behind and look-ahead syntax, see a good article on perlmonks : http://www.perlmonks.org/?node_id=518444
It might be easier to write a short script to do this.
use strict;
use autodie;
my $old_ip = 10.1.1.1; # or $ENV{'OLD'}
my $new_ip = 50.5.5.5; # or $ENV{'NEW'}
open my $infh, '<', $ARGV[0];
open my $outfh, '>', $ARGV[1];
while ( my $line = <$infh> ) {
chomp $line;
my #elems = split '\s+', $line;
next unless $elems[0] eq $old_ip;
print $outfh $new_ip . join(" ", #elems[1..$#elems]) . "\n";
}
close $outfh;
close $infh;

Need Help parsing stream data from php socket server with perl

I'm working on a project for my company that uses a socket server (php) to gather data from a remote device. How can I make this perl program run directly on the stream instead of first having the server write to a tmp file then running this script on that file then writing out a csv file for insertion into a database?
I thought about using IO::Socket but I'm not sure how to go about it.
Also, If anyone has any tips/pointers on how to clean up this code it would be appreciated. (I'm still very much a Perl n00b ;-))
Here's what I have so far for the perl prog.( I wind up with two tmp files after this is run):
#vim set sw=2 ts=2
#!/usr/bin/perl
use warnings;
use strict;
use Data::Dumper;
&convert;
#open the source file, strip out any unneeded chars, reformat the data
sub convert{
my $source = "$ARGV[0]";
my $dest = "$ARGV[0]"."_tmp.txt";
chomp $source;
open SOURCE, '', $dest or die "Could not open '$dest' $!";
# move the data from the source file into an array
my %fields;
my #field_names = qw/FIELD1 FIELD2 FIELD3 FIELD4 FIELD5 FIELD6/;
my $pack_definition = 'a4 a2 a1 a4 a4 a8A*';
while(){
# strip out the packet header
s#T18##g;
s#T00##g;
s/^(FF14).*$//g;
s/^(FF18).*$//g;
s/(^|\n)[\n\s]*/$1/g;
# arrange the data into the necessary order
#fields{#field_names} = unpack($pack_definition, $_);
s#(FF15)(.{2})(.{1})(.{2})(.{2})(.{2})(.{2})(.{2})(.{2})(.{2})(.{2})#$1\t$2\t$3\t$5$4\t$7$6\t$11$10$9$8#g;
s#(FF16)(.{2})(.{1})(.{2})(.{2})(.{2})(.{2})(.{2})(.{2})(.{2})(.{2})#$1\t$2\t$3\t$4\t$5\t$6\t$7\t$8\t$9\t$10\t$11#g;
s#(FF17)(.{2})(.{1})(.{2})(.{2})(.{2})(.{2})(.{4})(.{4})#$1\t$2\t$3\t$5$4\t$7$6\t$8\t$9#g;
my #spds = /(FF15)\t(.{2})\t(.{1})\t(.{4})\t(.{4})\t(.{8})/;
# convert the data from hex to ascii
foreach my $data (#spds){
my $replacement = hex($data);
s#$data#$replacement#g;
}
my #psis = /(FF16)\t(.{2})\t(.{1})\t(.{2})\t(.{2})\t(.{2})\t(.{2})\t(.{2})\t(.{2})\t(.{2})\t(.{2})/;
foreach my $data1(#psis){
my $replacement1 = hex($data1);
s#$data1#$replacement1#g;
}
my #rates= /(FF17)\t(.{2})\t(.{1})\t(.{4})\t(.{1,4})\t(.{4})\t(.{4})/;
foreach my $data2 (#rates){
my $replacement2 = hex($data2);
s#$data2#$replacement2#g;
}
# print the converted data to the destination file
print DEST;
}
# close the files
close SOURCE;
close DEST;
}
&create_vals;
# perform conversion from raw values to human readable output
sub create_vals{
my $source = "$ARGV[0]"."_tmp.txt";
my $dest = "$ARGV[0]"."_converted.txt";
chomp $source;
open SOURCE, '', $dest or die "Cannot open '$dest' for writing $!";
while(){
s#(65301)\t(.{2})\t(8)\t(.{1,5})\t(.{1,5})\t(.{1,4})#"'".$1."','". $2."','". $3."','". ($4/8)."','". ($5/8)."','". ($6/20)."'"#eg;
s#(65302)\t(.{2})\t(8)\t(.{1,3})\t(.{1,3})\t(.{1,3})\t(.{1,3})\t(.{1,3})\t(.{1,3})\t(.{1,3})\t(.{1,3})#"'".$1."','". $2."','". $3."','". $4."','". (($5*1.8)-40)."','". (($6*1.8)-40)."','". ($7*.58)."','".($8*.58)."','".($9*.29008)."','".(($10*1.8)-40)."','".$11."'"#eg;
s#(65303)\t(.{2})\t(8)\t(.{1,5})\t(.{1,5})\t(.{1,5})\t(.{1,5})#"'".$1."','". $2."','". $3."','". ($4*0.014)."','". ($5*.05)."'"#eg;
print DEST;
}
}
close SOURCE;
close DEST;
__END__
If "run directly on the stream" means "accept connections from a remote device instead of the PHP server" then IO::Socket is a way to go. Google has plenty of examples and you can also check out perlipc documentation.