I am working on a perl script that finds will take customer service information and change the service ids. The code i have now can take a 2 column csv file for reference, and swap the numbers, but when i try to use "\n" or "\|" like i need to, it will give me the `ol:
"sed: -e expression #1, char 29: unterminated `s' command"
Here's my original code that works for changing JUST the number within the quotes:
#!/usr/bin/perl
use strict;
use warnings;
use Text::CSV; #load Text::CSV module for parsing CSV
use LWP::Simple; #load LWP module for doing HTTP get
my $file = 'sid_swap.csv'; #CSV file to parse and load
my $csv = Text::CSV->new(); #create a new Text::CSV object
open (CSV, "<", $file) or die $!; #open CSV file for parsing
while (<CSV>) {
if ($csv->parse($_)) {
my #columns = $csv->fields(); #parse csv files and load into an array for each row
my $newSID = $columns[0];
my $oldSID = $columns[1];
system("sed -i 's/\<row SERVICE_MENU_ID=\"$oldSID\"/\<row SERVICE_MENU_ID=\"$newSID\"/g' customer_data.txt");
} else {
my $err = $csv->error_input;
print "Failed to parse line: $err";
}
}
close CSV;
And here is the new code that throws the error:
#!/usr/bin/perl
use strict;
use warnings;
use Text::CSV; #load Text::CSV module for parsing CSV
use LWP::Simple; #load LWP module for doing HTTP get
my $file = 'sid_swap.csv'; #CSV file to parse and load
my $csv = Text::CSV->new(); #create a new Text::CSV object
open (CSV, "<", $file) or die $!; #open CSV file for parsing
while (<CSV>) {
if ($csv->parse($_)) {
my #columns = $csv->fields(); #parse csv files and load into an array for each row
my $newSID = $columns[0];
my $oldSID = $columns[1];
system("sed -i 's/\<row SERVICE_MENU_ID=\"$oldSID\"/\n$newSID\|/g' customer_data.txt");
} else {
my $err = $csv->error_input;
print "Failed to parse line: $err";
}
}
close CSV;
Thanks for any help you can provide!
To debug, change system("sed ...") to die("sed ...");. You'll see you're what trying to execute
sed -i 's/<row SERVICE_MENU_ID="OLDSID"/
NEWSID|/g' customer_data.t
I guess sed doesn't like actual newlines in the middle of its arguments? That can be fixed using proper escaping, but your approach is... insane. You're processing the entire file for each row of the CSV!
open(my $CSV, "<", $file) or die $!; # Let's not use global vars!
my %fixes;
while (<$CSV>) {
$csv->parse($_)
or die "Failed to parse line: ".$csv->error_input."\n";
my ($newSID, $oldSID) = $csv->fields();
$fixes{$oldSID} = $newSID;
}
my $pat = join "|", map quotemeta, keys(%fixes);
my $re = qr/$pat/;
{
local #ARGV = 'customer_data.txt';
local $^I = ''; # "perl -i" mode
while (<>) {
s/<row SERVICE_MENU_ID="($re)"/\n$fixes{$1}|/g;
print;
}
}
Doing it in one pass also solve the problem of
1111,2222
2222,3333
being the same as
1111,3333
2222,3333
Related
Seems fairly simple but with the "$" in the name causes the name to split. I tried escaping the character out but when I try to open the file I get GLOB().
my $path = 'C:\dir\name$.txt';
open my $file, '<', $path || die
print "file = $file\n";
It should open the file so I can traverse the entries.
It has nothing to do with the "$". Just follow standard file handling procedure.
use strict;
use warnings;
my $path = 'C:\dir\name$.txt';
open my $file_handle, '<', $path or die "Can't open $path: $!";
# read and print the file line by line
while (my $line = <$file_handle>) {
# the <> in scalar context gets one line from the file
print $line;
}
# reset the handle
seek $file_handle, 0, 0;
# read the whole file at once, print it
{
# enclose in a block to localize the $/
# $/ is the line separator, so when it's set to undef,
# it reads the whole file
local $/ = undef;
my $file_content = <$file_handle>;
print $file_content;
}
Consider using the CPAN modules File::Slurper or Path::Tiny which will handle the exact details of using open and readline, checking for errors, and encoding if appropriate (most text files are encoded to UTF-8).
use strict;
use warnings;
use File::Slurper 'read_text';
my $file_content = read_text $path;
use Path::Tiny 'path';
my $file_content = path($path)->slurp_utf8;
If it's a data file, use read_binary or slurp_raw.
I have a CSV file like this:
id,item,itemtype,date,service,level,message,action,user
"344","-1","IRM","2008-08-22 13:01:57","login","1","Failed login: \'irm\', database \'irmD\'",NULL,NULL
"346","-1","IRM","2008-08-27 10:58:59","login","1","Ошибка входа:\'\', база данных \'irmD\'",NULL,NULL
It's Okay with the second line, but Text::CSV just skips the third one. The third line consists Cyrillic characters, but the file is encoded in UTF-8 and Perl shouldn't have any problems with that.
And the code:
#!/usr/bin/perl
use warnings;
use strict;
use Text::CSV;
use utf8;
my $file = 'Test.csv'; my $csv = Text::CSV->new();
open (CSV, "<", $file) or die $!;
while (<CSV>) {
if ($csv->parse($_)) {
if ($. == 1) {
next;
}
my #columns = $csv->fields();
my $id=$columns[0];
print $id." ";
}
}
print "\n";
close CSV;
Any help or hint will be appreciated.
Did you read the documentation of Text::CSV?
If your
data contains newlines embedded in fields, or characters above 0x7e
(tilde), or binary data, you must set "binary => 1"
Also, use utf8 tells Perl you're going to use UTF-8 in the source code, not in the data. Remove it.
Using <> to read in CSV is also mentioned in the documentation:
while (<>) { # WRONG!
Here is a working version:
#!/usr/bin/perl
use warnings;
use strict;
use Text::CSV;
my $file = 'Test.csv';
my $csv = 'Text::CSV'->new({ binary => 1 }) or die 'Text::CSV'->error_diag;
open my $CSV, '<', $file or die $!;
while (my $line = $csv->getline($CSV)) {
next if 1 == $.;
my #columns = #$line;
my $id = $columns[0];
print $id . " ";
}
print "\n";
close $CSV;
I think your problem will be, that whilst you've useed UTF8, that's only really for perl's uses.
From:
http://perldoc.perl.org/utf8.html
utf8 - Perl pragma to enable/disable UTF-8 (or UTF-EBCDIC) in source code
Looking at Text::CSV
You probably want:
$csv = Text::CSV::Encoded->new ({ encoding => "utf8" });
You will also - probably - need to specify that you're opening a UTF-8 file. You can either do this as part of the open or with binmode
open ( my $filehandle, "<:encoding(UTF-8)", "Test.csv" );
I have a script that converts xlsx files to csv files. but how can i just convert a specific(named) worksheet from an xlsx workbook (with multiple worksheets) to a csv file.
#!/usr/bin/perl
use strict;
use warnings;
use Spreadsheet::BasicRead;
use Excel::Writer::XLSX;
use Spreadsheet::ParseExcel;
my $xlsx = ("c:\\acb.xlsx"),,1;
my $csv = ("c:\\acb.csv"),,1;
if (-e $xlsx) {
my $ss2 = new Spreadsheet::BasicRead($xlsx) or die;
my $name2 = '';
my $row2 = 0;
open(FILE2, ">$csv") or die ;
binmode(FILE2, ":utf8"); # Wide character in print warnings
while (my $data2 = $ss2->getNextRow()){
$row2++;
$name2 = join(',', #$data2);
print FILE2 $name2."\n" if ($name2 ne "");
}
close FILE2;
}
Add $ss2->setCurrentSheetNum(num) between binmode and while. That should work for you.
HTH!
TheJester1977
I am new to Perl and still trying to figure out how to code in this language.
I am currently trying to split a long single string of csv into multiple lines.
Data example
a,b,c<br />x,y,x<br />
which I so far have manage to split up, adding in quotes, to add into a CSV file again later on:
"a,b,c""x,y,z"
By having the quotes it just signifies which sets of CSV are together as such.
The problem I am having is that when I try and create a CSV file, passing in data in a string i am getting an error
"Can't call method "parse" on an undefined variable.
When I print out the string which I am passing in, it is defined and holds data. I am hoping that this is something simple which I am doing wrong through lack of experience.
The CSV code which I am using is:
use warnings;
use Text::CSV;
use Data::Dumper;
use constant debug => 0;
use Text::CSV;
print "Running CSV editor......\n";
#my $csv = Text::CSV->new({ sep_char => ',' });
my $file = $ARGV[0] or die "Need to get CSV file on the command line\n";
my $fileextension = substr($file, -4);
#If the file is a CSV file then read in the file.
if ($fileextension =~ m/csv/i)
{
print "Reading and formating: $ARGV[0] \n";
open(my $data, '<', $file) or die "Could not open '$file' $!\n";
my #fields;
my $testline;
my $line;
while ($line = <$data>)
{
#Clears the white space at the end of the line.
chomp $line;
#Splits the line up and removes the <br />.
$testline = join "\" \" ", split qr{<br\s?/>}, $line;
#my $newStr = join $/, #lines;
#print $newStr;
my $q1 = "\"";
$testline = join "", $q1,$testline,$q1;
print "\n printing testline: \n $testline \n";
}
$input_string = $testline;
print "\n Testing input string line:\n $input_string";
if ($csv->parse ($input_string))
{
my #field = $csv->fields;
foreach my $col (0 .. $#field) {
my $quo = $csv->is_binary ($col) ? $csv->{quote_char} : "";
printf "%2d: %s%s%s\n", $col, $quo, $field[$col], $quo;#
}
}
else
{
print STDERR "parse () failed on argument: ",
$csv->error_input, "\n";
$csv->error_diag ();
}
#print $_,$/ for #lines;
print "\n Finished reading and formating: $ARGV[0] \n";
}else
{
print "Error: File is not a CSV file\n"
}
You did not create a Text::CSV object, but you try to use it.
"Can't call method "parse" on an undefined variable
This means that your $csv is not there, thus it does not have a method called parse. Simply create a Text::CSV object first, at the top of your code below all the use lines.
my $csv = Text::CSV->new;
Pleae take a look at the CPAN documentation of Text::CSV.
Also, did I mention you should use strict?
Well.. I'm stuck again. I've read up quite a few topic with similar problems but not finding a solution for mine. I have a ; delimited csv file and the strings at the 8th column ($elements[7]) is as following: "aaaa;bb;cccc;ddddd;eeee;fffff;gg;". What i'm trying is to split the string based on ; and capture the outputs to variables. Then use those variables in the main csv file in their own column.
So now the file is like:
3d;2f;7j;8k;4s;2b;5g;"aaaa;bb;cccc;ddddd;eeee;fffff;gg;";4g;1a;5g;2g;7h;3d;2f;7j
3c;9k;5l;4g;1a;5g;3d;"aaaa;bb;cccc;ddddd;eeee;fffff;gg;";4g;1a;5g;2g;7h;3d;2f;7j
4g;1a;5g;2g;7h;3d;8k;"aaaa;bb;cccc;ddddd;eeee;fffff;gg;";3d;2f;7j;8k;4s;2b;4g;1a
And i want it like:
3d;2f;7j;8k;4s;2b;5g;4g;1a;5g;2g;7h;3d;2f;7j;aaaa;bb;cccc;ddddd;eeee;fffff;gg
3c;9k;5l;4g;1a;5g;3d;4g;1a;5g;2g;7h;3d;2f;7j;aaaa;bb;cccc;ddddd;eeee;fffff;gg;
4g;1a;5g;2g;7h;3d;8k;3d;2f;7j;8k;4s;2b;4g;1a;aaaa;bb;cccc;ddddd;eeee;fffff;gg;
This is my code i've been trying it with. I know.. it's terrible! But i'm hoping someone can help me?
use strict;
use warnings;
my $inputfile = shift || die "Give files\n";
my $outputfile = shift || die "Give output\n";
open my $INFILE, '<', $inputfile or die "In use / Not found :$!\n";
open my $OUTFILE, '>', $outputfile or die "In use :$!\n";
while (<$INFILE>) {
s/"//g;
my #elements = split /;/, $_;
my ($varA, $varB, $varC, $varD, $varE, $varF, $varG, $varH) split (';', $elements[10]);
$elements[16] = $varA;
$elements[17] = $varB;
$elements[18] = $varC;
$elements[19] = $varD;
$elements[20] = $varE;
$elements[21] = $varF;
$elements[22] = $varG;
$elements[23] = $varH;
my $output_line = join(";", #elements);
print $OUTFILE $output_line;
}
close $INFILE;
close $OUTFILE;
exit 0;
I'm confused about the my statement as well, it shouldn't be possible right? I mean the $vars are in a closed part so it shouldn't be possible to write them to $elements?
EDIT
This is how i adjusted the code with TLP's suggestions:
use strict;
use warnings;
use Text::CSV;
my $inputfile = shift || die "Give files\n";
my $outputfile = shift || die "Give output\n";
open my $INFILE, '<', $inputfile or die "In use / Not found :$!\n";
open my $OUTFILE, '>', $outputfile or die "In use :$!\n";
my $csv = Text::CSV->new({ # create a csv object
sep_char => ";", # delimiter
eol => "\n", # adds newline to print
});
while (my $row = $csv->getline($INFILE)) { # $row is an array ref
my $line = splice(#$row, 10, 1); # remove 8th line
$csv->parse($line); # parse the line
push #$row, $csv->fields(); # push newly parsed fields onto main array
$csv->print($OUTFILE, $row);
}
close $INFILE;
close $OUTFILE;
exit 0;
You should use a CSV module, e.g. Text::CSV to parse your data. Here's a brief example on how it can be done. You can replace the file handles I used below with your own.
use strict;
use warnings;
use Text::CSV;
my $csv = Text::CSV->new({ # create a csv object
sep_char => ";", # delimiter
eol => "\n", # adds newline to print
});
while (my $row = $csv->getline(*DATA)) { # $row is an array ref
my $line = splice(#$row, 7, 1); # remove 8th line
$csv->parse($line); # parse the line
push #$row, $csv->fields(); # push newly parsed fields onto main array
$csv->print(*STDOUT, $row);
}
__DATA__
3d;2f;7j;8k;4s;2b;5g;"aaaa;bb;cccc;ddddd;eeee;fffff;gg;";4g;1a;5g;2g;7h;3d;2f;7j
3c;9k;5l;4g;1a;5g;3d;"aaaa;bb;cccc;ddddd;eeee;fffff;gg;";4g;1a;5g;2g;7h;3d;2f;7j
4g;1a;5g;2g;7h;3d;8k;"aaaa;bb;cccc;ddddd;eeee;fffff;gg;";3d;2f;7j;8k;4s;2b;4g;1a
Output:
3d;2f;7j;8k;4s;2b;5g;4g;1a;5g;2g;7h;3d;2f;7j;aaaa;bb;cccc;ddddd;eeee;fffff;gg;
3c;9k;5l;4g;1a;5g;3d;4g;1a;5g;2g;7h;3d;2f;7j;aaaa;bb;cccc;ddddd;eeee;fffff;gg;
4g;1a;5g;2g;7h;3d;8k;3d;2f;7j;8k;4s;2b;4g;1a;aaaa;bb;cccc;ddddd;eeee;fffff;gg;