for ($i=0; $i<10; $i++)
{
my $v1 = $sel->get_text("//body[\#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/**div**/table/tbody/tr/td/div/div");
my $v2 = $sel->get_text("//body[#\id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/**div**/table/tbody/tr/td[2]/div/div")
print ($v1 . $v2);
}
For every iteration, it has to find the 14th element starting from div[10] & replace it with the increased div[ ] element (Ex: if 14th element is div, replace it by div[2]. In the next iterartion find 14th element i.e., div[2] & replace it by div[3] & so on ).
By using PATTERN matching, it can't. Is there any method by using regex for finding that particular element & replacing it ? how can i do it ?
my $a = "//body[\#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/**div**/table/tbody/tr/td/div/div";
my #arr = split ('/' , $a);
print "#arr \n";
my $size1 = #arr;
print "$size1\n";
print $arr[16];
foreach my $a2 (#arr)
{
print "$a2 \n";
}
my $b = "//body[\#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/**div**/table/tbody/tr/td[2]/div/div";
Two variables as mentioned in the above question as v1 & v2 (edited as $a and $b), the modification has to apply for both of them. I think i'm almost near to what you've told. Can yoy please help me further
use 5.010;
my $xpath = q(//body[#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div/table/tbody/tr/td/div/div);
for my $i (0..10) {
my #nodes = split qr'/', $xpath;
$nodes[16] .= "[$i]" unless 0 == $i;
say join '/', #nodes;
}
Results:
//body[#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div/table/tbody/tr/td/div/div
//body[#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div[1]/table/tbody/tr/td/div/div
//body[#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div[2]/table/tbody/tr/td/div/div
//body[#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div[3]/table/tbody/tr/td/div/div
//body[#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div[4]/table/tbody/tr/td/div/div
//body[#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div[5]/table/tbody/tr/td/div/div
//body[#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div[6]/table/tbody/tr/td/div/div
//body[#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div[7]/table/tbody/tr/td/div/div
//body[#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div[8]/table/tbody/tr/td/div/div
//body[#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div[9]/table/tbody/tr/td/div/div
//body[#id='ext-gen3']/div[10]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div[10]/table/tbody/tr/td/div/div
Ummm, all elements are separated by /, right? So you can use the native split method to split the portion of the text following div[10] based on /. Store it in an array $arr. Merge it to find the length of the string, say $len. Find the index of the div[10], say $orig_index. Then you find the 14th element, do a regex match to see which format it is in:
$arr[13] =~ /div([\d+])?/;
if ($1) {
$arr[13] =~ /div[$1]/div[($1+1)]/e;
}
else {
$arr[13] = div[2];
}
Now that the 14th element is changed, concatenate the array to get the new output string for the portion from the portion between div[10] and the 14th one:
{
local $" = '';
$newstring = "#arr";
}
splice($originalstring,$orig_index,$len,$newstring);
I think that will do.
Related
I was trying to think in the right way to tackle this:
-I would to pass say, n elements array as argument to a subroutine. And for each element match two char types S and T and print for each element, the count of these letters. So far I did this but I am locked and found some infinite loops in my code.
use strict;
use warnings;
sub main {
my #array = #_;
while (#array) {
my $s = ($_ = tr/S//);
my $t = ($_ = tr/T//);
print "ST are in total $s + $t\n";
}
}
my #bunchOfdata = ("QQQRRRRSCCTTTS", "ZZZSTTKQSST", "ZBQLDKSSSS");
main(#bunchOfdata);
I would like the output to be:
Element 1 Counts of ST = 5
Element 2 Counts of ST = 6
Element 3 Counts of ST = 4
Any clue how to solve this?
while (#array) will be an infinite loop since #array never gets smaller. You can't read into the default variable $_ this way. For this to work, use for (#array) which will read the array items into $_ one at a time until all have been read.
The tr transliteration operator is the right tool for your task.
The code needed to get your results could be:
#!/usr/bin/perl
use strict;
use warnings;
my #data = ("QQQRRRRSCCTTTS", "ZZZSTTKQSST", "ZBQLDKSSSS");
my $i = 1;
for (#data) {
my $count = tr/ST//;
print "Element $i Counts of ST = $count\n";
$i++;
}
Also, note that my $count = tr/ST//; doesn't require the binding of the transliteration operator with $_. Perl assumes this when $_ holds the value to be counted here. Your code tried my $s = ($_ = tr/S//); which will give the results but the shorter way I've shown is the preferred way.
(Just noticed you had = instead of =~ in your statement. That is an error. Has to be $s = ($_ =~ tr/S//);)
You can combine the 2 sought letters as in my code. Its not necessary to do them separately.
I got the output you want.
Element 1 Counts of ST = 5
Element 2 Counts of ST = 6
Element 3 Counts of ST = 4
Also, you can't perform math operations in a quoted string like you had.
print "ST are in total $s + $t\n";
Instead, you would need to do:
print "ST are in total ", $s + $t, "\n";
where the operation is performed outside of the string.
Don't use while to traverse an array - your array gets no smaller, so the condition is always true and you get an infinite loop. You should use for (or foreach) instead.
for (#array) {
my $s = tr/S//; # No need for =~ as tr/// works on $_ by default
my $t = tr/T//;
print "ST are in total $s + $t\n";
}
Why tr///??
sub main {
my #array = #_;
while (#array) {
my $s = split(/S/, $_, -1) - 1;
my $t = split(/T/, $_, -1) - 1;
print "ST are in total $s + $t\n";
}
}
I have been using this perl script (thanks to Jeff Schaller) to match 3 or more words in the title fields of two separate csv files.
Original question here:
https://unix.stackexchange.com/questions/283942/matching-3-or-more-words-from-fields-in-separate-csv-files?noredirect=1#comment494461_283942
I have also added some exception functionality following advice from meuh:
#!/bin/perl
my #csv2 = ();
open CSV2, "<csv2" or die;
#csv2=<CSV2>;
close CSV2;
my %csv2hash = ();
for (#csv2) {
chomp;
my ($title) = $_ =~ /^.+?,\s*([^,]+?),/; #/ match the title
$csv2hash{$_} = $title;
}
open CSV1, "<csv1" or die;
while (<CSV1>) {
chomp;
my ($title) = $_ =~ /^.+?,\s*([^,]+?),/; #/ match the title
my #titlewords = split /\s+/, $title; #/ get words
my #new; #add exception words which shouldn't be matched
foreach my $t (#titlewords){
push(#new, $t) if $t !~ /^(and|if|where)$/i;
}
#titlewords = #new;
my $desired = 3;
my $matched = 0;
foreach my $csv2 (keys %csv2hash) {
my $count = 0;
my $value = $csv2hash{$csv2};
foreach my $word (#titlewords) {
++$count if $value =~ /\b$word\b/i;
last if $count >= $desired;
}
if ($count >= $desired) {
print "$csv2\n";
++$matched;
}
}
print "$_\n" if $matched;
}
close CSV1;
During my testing, one issue I've found that I would like to tweak is that if csv2 contains a single common word such as the, if this is replicated in csv1 three or more times then three positive matches is found. To clarify:
If csv1 contains:
1216454,the important people feel the same way as the others, 15445454, 45445645
^ i.e. there are three insatnces of the in the above line
If csv2 contains:
14564564,the tallest man on earth,546456,47878787
^ i.e. there is one instance of the in this line
Then I would like only one word to be classed as matching, and there be no output (based on my desired number of matching words- 3 ) because there is only one instance of the matching word in one of the files.
However if:
csv1 contained:
1216454,the important people feel the same way as the others,15445454, 45445645
and csv2 contained:
15456456,the only way the man can sing the blues,444545,454545
Then, as there are three matching words in each (i.e. 3 instances of the word the in each title, then I would like this to be classed as a matching title based on my desired number of matching words being 3 or more, thus generating the output:
1216454,the important people feel the same way as the others,15445454, 45445645
15456456,the only way the man can sing the blues,444545,454545
I would like to amend the script so that if there is one instance of a word in a csv, and multiple instances of the same word in the other csv then that is classed as only one match. However, if there were say 3 instance of the word the in both files, then it should still be classed as three matches. Basically I would like matches to be on a word for word basis.
Everything about the script other than this is perfect so I would rather not go back to the drawing board completely as I am happy with everything other than this.
I hope I've explained it ok, if anyone need any clarification let me know.
If you just wan to count unique matches, you can use a hash instead of a list to collect the words from csv1, just like you do for csv2, and then also count the occurrences of each word separately:
#!/usr/bin/env perl
my #csv2 = ();
open CSV2, "<csv2" or die;
#csv2=<CSV2>;
close CSV2;
my %csv2hash = ();
for (#csv2) {
chomp;
my ($title) = $_ =~ /^.+?,\s*([^,]+?),/; #/ match the title
$csv2hash{$_} = $title;
}
open CSV1, "<csv1" or die;
while (<CSV1>) {
chomp;
my ($title) = $_ =~ /^.+?,\s*([^,]+?),/; #/ match the title
my %words;
$words{$_}++ for split /\s+/, $title; #/ get words
## Collect unique words
my #titlewords = keys(%words);
my #new; #add exception words which shouldn't be matched
foreach my $t (#titlewords){
push(#new, $t) if $t !~ /^(and|if|where)$/i;
}
#titlewords = #new;
my $desired = 3;
my $matched = 0;
foreach my $csv2 (keys %csv2hash) {
my $count = 0;
my $value = $csv2hash{$csv2};
foreach my $word (#titlewords) {
my #matches = ( $value=~/\b$word\b/ig );
my $numIncsv2 = scalar(#matches);
#matches = ( $title=~/\b$word\b/ig );
my $numIncsv1 = scalar(#matches);
++$count if $value =~ /\b$word\b/i;
if ($count >= $desired || ($numIncsv1 >= $desired && $numIncsv2 >= $desired)) {
$count = $desired+1;
last;
}
}
if ($count >= $desired) {
print "$csv2\n";
++$matched;
}
}
print "$_\n" if $matched;
}
close CSV1;
I have two arrays:
#file_list holds a list of files in a directory, and
#name_list holds some names.
For example, these arrays could contain
#file_list = ('Bob_car', 'Bob_house', 'Bob_work', 'Fred_car', 'Fred_house', 'Fred_work', ...);
#name_list = ('Bob', 'Fred', ...);
(the real data is not that simple).
My goal is to compare each file with every name and see if they match. They match if the file string starts with the name.
I could then use these matches to sort the files into new directories, based on their corresponding name.
Here is my code:
for ( my $i = 0; $i < scalar #file_list ; $i++ )
{
for ( my $j = 0; $j < #name_list ; $j++ )
{
if ( $file_list[ $i ] =~ m/^$name_list[ $j ]/ )
{
print "$file_list[ $i ] goes with $name_list[ $j ]\n";
}
else
{
print "no match\n";
}
}
}
However, I don't get any matches. I've tested the individual loops and they are working. Else, is there something off about the regex?
About how the arrays were made:
For #name_list, the file containing the names is organized in a seemingly random way, just because of how it was used for something else. The names in that file are on several different lines, with lots of blank lines in between and lots of blank entries within lines. Names can appear more than once.
I used the following code to make #name_list:
while (my $line = <$OriginalFILE>)
{
chomp $line;
my #current_line = split( "\t", $line );
for ( my $i = 0; $i < scalar #current_line ; $i ++ )
{
if ( $current_line[ $i ] =~ m/^\s*$/ )
{
# print "$current_line[$i] is blank\n";
}
else
{
push( #raw_name_list, $current_line[ $i ] );
}
} # end of for
} # while
# collect list without repeat instances of the same name
my %unique = ();
foreach my $name (#raw_name_list)
{
$unique{$name} ++;
}
my #name_list = keys %unique;
foreach my $name ( #name_list )
{
# print "$name\n";
chomp $name;
unless(mkdir $name, 0700)
{
die "Unable to create directory called $name\n";
}
}
The array #file_list was made using:
opendir(DIR, $ARGV[1]);
my #file_list = grep ! /^\./, readdir DIR;
closedir(DIR);
# print #file_list;
#amon, here is what i did to test the loops and regex:
FILE: for my $file (#transposed_files) {
print "$file\n";
for my $name (#transposedunique) {
print "i see this $name\n";
if ($file =~ /^\Q$name\E/) {
print "$file goes with $name\n";
next FILE;
}
}
#print "no match for $file\n";
}
oh, and I transposed the arrays, so that they would print to an outfile into separate rows.
Short version: You are building your name array wrong. Look at this line:
$unique{name} ++;
You are just incrementing the name entry of the hash. You probably wanted the $name variable.
The Longer Version
On English, and Foreach-Loops
Your code is a bit unperlish and looks more like C than like Perl. Perl is much closer to English than you might think. From the original wording of your question:
take the first element from #file_list and then to compare that to each element in #name_list
You wrote this as
for (my $i = 0; $i < #file_list; $i++) {
for (my $j = 0; $j < #name_list; $j++) {
...; # compare $file_list[$i] with $name_list[$j]
}
}
I'd rather do
for my $file (#file_list) {
for my $name (#name_list) {
...; # compare $file with $name
}
}
and save myself from the hassle of array subscripting.
Building Correct Regexes
Your code contains the following test:
$file_list[ $i ] =~ m/^$name_list[ $j ]/
This will not do what you think if $name_list[$j] contains special characters like (, ., +. You can match the literal contents of a variable by enclosing it in \Q ... \E. This would make the code
$file =~ /^\Q$name\E/
(if used with my variant of the loop).
You could also go the nifty route and compare the leading substring directly:
$name eq substr $file, 0, length($name)
This expresses the same condition.
On Loop Control
I will make two assumptions:
You are only interested in the first matching name for any file
You only want to print the no match message if no name was found
Perl allows us to break out of arbitrary loops, or restart the current iteration, or go directly to the next iteration, without using flags, as you would do in other languages. All we have to do is to label our loops like LABEL: for (...).
So once we have a match, we can start our search for the next file. Also, we only want to print no match if we left the inner loop without going to the next file. This code does it:
FILE: for my $file (#file_list) {
for my $name (#name_list) {
if ($file =~ /^\Q$name\E/) {
print "$file goes with $name\n";
next FILE;
}
}
print "no match for $file\n";
}
The Zen of Negation
In your file parsing code, you express a condition
if ($field =~ /^\s*$/) {
} else {
# do this stuff only if the field does not consist only of
# zero or more whitespace characters
}
That description is far to complex. How about
if ($field =~ /\S/) {
# do this stuff only if the field contains a non-whitespace character.
}
The same condition, but simpler, and more efficient.
Simplify your Parse
In short, your file parsing code can be condensed to
my %uniq;
while (<$OriginalFILE>) {
chomp;
$uniq{$_} = undef for grep /\S/, split /\t/;
}
my #name_list = sort { length($b) <=> length($a) } keys %uniq;
The split function takes a regex as first argument, and will split on $_ if no other string is specified. It returns a list of fields.
The grep function takes a condition and a list, and will return all elements of a list that match the condition. The current element is in $_, which regexes match by default. For explanation of the regex, see above.
Note: This still allows for the fields to contain whitespace, even in leading position. To split on all whitespace, you can give split the special argument of a string containing a single space: split ' '. This would make the grep unneccessary.
The for loop can also be used as a statement modifier, i.e. like EXPR for LIST. The current element is in $_. We assign something to the $_ entry in our %uniq hash (which is already initialized to the empty hash). This could be a number, but undef works as well.
The keys are returned in a seemingly random order. But as multiple names could match a file, but we only want to select one match, we will have to match the most specific name first. Therefore, I sort the names after their length in descending order.
Your code seems to work for me. All I did was construct two arrays like this:
my #file_list = qw/Bob_car Bob_house Bob_work Fred_car Fred_house Fred_work/;
my #name_list = qw/Fred Bob Mary/;
Then running your code produces output like this:
no match
Bob_car goes with Bob
no match
no match
Bob_house goes with Bob
no match
no match
Bob_work goes with Bob
no match
Fred_car goes with Fred
no match
no match
Fred_house goes with Fred
no match
no match
Fred_work goes with Fred
no match
no match
So it looks like it's working.
A common problem with reading input from files or from a user is forgetting to strip the newline character from the end of the input. This could be your problem. If so, have a read about perldoc -f chomp, and just chomp each value as you add it to your array.
I'm always interested in doing things in efficient way so every time I see O(N^2) algorithm rings bells for me. Why it should be O(N*M) and not O(N+M)?
my $re = join('|',map quotemeta, #name_list);
$re = qr/$re/;
for my $file (#file_list) {
if($file =~ /^($re)/) {
my $name = $1;
... do what you need
}
}
its look something wrong in loop.
follow comments in code
for ( my $i = 0; $i < scalar #file_list ; $i++ )
{
#use some string variable assign it ""
for ( my $j = 0; $j < #name_list ; $j++ )
{
if ( $file_list[ $i ] =~ m/^$name_list[ $j ]/ )
{
# assign string variable to founded name_list[$j]
break loop
}
}
# check condition if string not equal to "" match found print your requirement with string value else match not found
}
I have made one script which will extract all the Row data from HTML <TR> tags. I am having 30 HTML <TR> tags on my HTML page. Based on count, my code will fetch particular row data. Let's say if I need data present in 5th <tr>...</tr>, then my condition is if(count =5) {(go inside and get that data)}
But my problem here is I need the selected rows' data one at a time. Let's say I need data for rows 5, 6, and 14.
Could you please help me sort it out?
$te = new HTML::TableExtract(count => 0 );
$te->parse($content);
# Examine all matching tables
foreach $ts ($te->table_states) {
#print "Table (", join(',', $ts->coords), "):\n";
$cnt = 1;
foreach $row($ts->rows) {
# print " ---- Printing Row $cnt ----\n";
$PrintLine= join("\t", #$row);
#RowData=split(/\t/,$PrintLine);
$PrintLine =~ s/\r//ig;
$PrintLine =~ s/\t//ig;
$cnt = $cnt + 1;
# if ($PrintLine =~ /Site ID/ig || $PrintLine =~ /Site name/ig){print " Intrest $PrintLine $cnt =====================\n"};
if ( $cnt == 14) {
$arraycnt = 1;
my $SiteID="";
my $SiteName="";
foreach (#RowData) {
# print " Array element $arraycnt\n";
chomp;
$_ =~ s/\r//ig;
$_ =~ s/[\xC3\xA1\xC3\xA0\xC3\xA2\xC3\xA3]//ig;
if ($arraycnt== 17 ) { $SiteID= $_;}
if ($arraycnt== 39 ) { $SiteName= $_;}
$arraycnt = $arraycnt + 1;
}
#$PrintLineFinal = $BridgeCase."\t".$PrintLine;
$PrintLineFinal = $BridgeCase."\t".$SiteID."\t".$SiteName;
#print "$PrintLineFinal\n";
print MYFILE2 "$PrintLineFinal\n";
last;
}
}
}
A few suggestions:
Always:
use strict;
use warnings;
This will force you to declare your variables with my. e.g.
foreach my $ts ($te->table_states) {
my $cnt = 1;
(warnings will let you know about most silly mistakes. strict prevents mistakes by requiring you to use better practices in certain cases).
In several places, you are using your own counter variables as you go through the array. You don't need to do this. Instead, just get the array element you want directly. e.g. $array[3] to get the third element.
Perl also allows array slices to get just certain elements you want. #array[4,5,13] gets the fifth, sixth, and fourteenth elements of the array. You can use this to process only the rows you want, instead of looping through all of them:
my #rows = $ts->rows;
foreach my $row (#rows[4,5,13]) #process only the 5th, 6th, and 14th rows.
{
...
}
Here is a shortcut version of the same thing, using an anonymous array:
foreach my $row (#{[$ts->rows]}[4,5,13])
Also, perhaps you want to define the rows you want elsewhere in your code:
my #wanted_rows = (4,5,13);
...
foreach my $row (#{[$ts->rows]}[#wanted_rows])
This code is quite confused:
$PrintLine= join("\t", #$row);
#RowData=split(/\t/,$PrintLine);
$PrintLine =~ s/\r//ig;
$PrintLine =~ s/\t//ig;
First you are joining an array with tab characters, then you are splitting the array you just joined to get the array back again. Then you remove all tab characters from the line anyway.
I suggest you get rid of all that code. Just use #$row whenever you need the array, instead of making a copy of it. If you need to print the array for debugging (which is all you seem to be doing with $PrintLine, you can print an array directly:
print #$row; #print an array, nothing between each element.
print "#$row"; #print an array with spaces between each element.
With all of these changes, your code would be something like this:
use strict;
use warnings;
my #wanted_rows = (4,5,13);
my $te = new HTML::TableExtract(count => 0);
$te->parse($content);
# Examine all matching tables
foreach my $ts ($te->table_states) {
foreach my $row (#{[$ts->rows]}[#wanted_rows]) {
s/[\xC3\xA1\xC3\xA0\xC3\xA2\xC3\xA3\r\n]//ig for (#$row);
my $SiteID = $$row[16] // ''; #set to empty strings if not defined.
my $SiteName = $$row[38] // '';
print MYFILE2 $BridgeCase."\t".$SiteID."\t".$SiteName;
}
}
You could access the results like this:
foreach $ts ($te->table_states) {
#you need 14th rows
#my 14throws = $ts->rows->[13];#starting with zero!
#17th col from the 14th row
#my $17colfrom14throws = $ts->rows->[13]->[16];
my $SiteName = $ts->rows->[13]->[38];
my $SiteID = $ts->rows->[13]->[16];
my $PrintLineFinal = $BridgeCase."\t".$SiteID."\t".$SiteName;
print MYFILE2 "$PrintLineFinal\n";
}
my $V1 = $sel->get_text("//body[\#id='ext-gen3']/div[14]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div/table/tbody/tr/td/div/div");
print($V1 . "\n");
When i execute the above command, i could print the v1 without error.
My aim is to print the such values ( say around 20). But xpath for each variable varies by the div element only (i.e., div element preeceeding table should be replaced by div[2], and div[2] should be replaced by div[3] in every iteration.
Following snippet does the replacing div[] element. But saying as invalid xpath.
my $xpath = q(//body[\#id='ext-gen3']/div[14]/div[2]/div/div/div/div/div/div/div/div/div/div[2]/div/div/table/tbody/tr/td/div/div);
for my $i (0,2)
{
my #nodes = split qr'/', $xpath;
$nodes[16] .= "[$i]" unless 0 == $i;
#say join '/', #nodes; }
my #somenames = join('/',#nodes);
#print "\n";
#print #somenames ;
foreach my $val (#somenames) {
chomp($val);
#my #new_arr = $sel->get_text($val);
print "\n";
print $val; ***this prints the xpath value which i needed***
my $V1 = $sel->get_text("$val"); ***#i'm passing the same xpath here within double quotes but says as invalid xpath***
print($V1 . "\n");
my #new_arr = $sel->get_text("$val");
}
}
Am i following wrong way of passing xpath ? How can i do it ?
Compare your definition of the XPath string with mine in the previous answer. You inserted a backslash in front of the commercial-at that does not belong.