I'm writing a perl script which needs to parse a file (or more specifically, a known sub in the file) find a specific function call, and retrieve all the arguments for that call. For example:
sub MySub {
# some code here...
# ...
MyFunction ([q{I}, q{want}], [qw(to get)],
["these", "arguments"]);
# ...
# more code...
}
I've been able to parse through the file grab the appropriate line/lines for the function call, and then I wrote my own local "MyFunction" which can grab the args. Then I use a stringy eval to call that function and get the arguments. Example:
sub MyFunction {
return #_;
}
# Not actually doing this, but this is what I might get from parsing:
my $lines = 'MyFunction ({arg1 => "hashref"}, ["arg2", "arrayref"], "etc...");';
my #arguments = eval $lines;
This works great, except for the fact that I'm using a stringy eval. The problem with parsing the argument directly out of the string is there are many different formats (including multiple lines) that are possible. So I guess is there an easy way to parse the arguments directly out of the string, or to call the function without a stringy eval?
Thanks a lot for any help!
You don't have to eval if you don't want to. You can use the name to get the symbol and then use that to get the code ref.
my $func = do {
no strict 'refs';
*{'main::MyFunction'}{CODE};
};
my #arguments = $func->( {arg1 => "hashref"}, ["arg2", "arrayref"], "etc..." );
Parsing Perl is hard (and there's a folk theorem that only Perl can parse Perl). Parsing perl with your own hand-rolled string hack is likely to end in disaster in all but the simplest cases. What you really want is a tool that can parse Perl and give you the arguments.
Our DMS Software Reegineering Toolkit can do this. See parse tree below,
obtained from a file that contains exactly OP's MySub example text:
C:\DMS\Domains\Perl\Tools\Parser>run ..\domainparser ++AST C:\temp\test.pl
Perl~Perl5 Domain Parser Version 2.6.15 DEBUG BUILD
Copyright (C) 1996-2015 Semantic Designs, Inc; All Rights Reserved; SD Confidential
Powered by DMS (R) Software Reengineering Toolkit
Using encoding Unicode-UTF-8?ANSI +CRLF +1 /^I
432 tree nodes in tree.
(Perl#Perl~Perl5=1#5c21d20^0 Line 1 Column 1 File C:/temp/test.pl
(statements#Perl~Perl5=10#5c21c80^1#5c21d20:1 Line 1 Column 1 File C:/temp/test.pl
(subroutine_head#Perl~Perl5=551#5c73680^1#5c21c80:1 Line 1 Column 1 File C:/temp/test.pl
(IDENTIFIER#Perl~Perl5=573#5c735e0^1#5c73680:1[`MySub'] Line 1 Column 5 File C:/temp/test.pl)IDENTIFIER
(prototype#Perl~Perl5=552#5c73640^1#5c73680:2 Line 1 Column 11 File C:/temp/test.pl)prototype
(attributes#Perl~Perl5=554#5c73660^1#5c73680:3 Line 1 Column 11 File C:/temp/test.pl)attributes
)subroutine_head#5c73680
(block#Perl~Perl5=4#5c21640^1#5c21c80:2 Line 1 Column 11 File C:/temp/test.pl
precomment 3:1 `# ...'
precomment 3:2 `# more code...'
(statement#Perl~Perl5=21#5c7cf40^1#5c21640:1 Line 5 Column 5 File C:/temp/test.pl
|(call_statement#Perl~Perl5=561#5c209a0^1#5c7cf40:1 Line 5 Column 5 File C:/temp/test.pl
| (IDENTIFIER#Perl~Perl5=573#5c73920^13#5c210a0:1#5c73ec0:1#5c778c0:1#5c21080:1#5c73e00:1#5c77860:1#5c209a0:1#5c7f860:1#5c73d60:1#5c77840:1#5c7f840:1#5c73560:1#5c777e0:1[`MyFunction'] Line 5 Column 5 File C:/temp/test.pl
| precomment 0:1 `# some code here...'
| precomment 0:2 `# ...')IDENTIFIER
| (L20#Perl~Perl5=514#5c7f240^3#5c206c0:2#5c209a0:2#5c21020:1 Line 5 Column 17 File C:/temp/test.pl
| (L20#Perl~Perl5=514#5c7b540^3#5c7f240:1#5c7c640:1#5c738e0:1 Line 5 Column 17 File C:/temp/test.pl
| (L1_term#Perl~Perl5=187#5c7a700^11#5c7abc0:1#5c7b540:1#5c7c860:1#5c7d280:1#5c7b620:1#5c7ad60:1#5c7a9a0:1#5c20840:1#5c7c1c0:1#5c7b8a0:1#5c7aaa0:1 Line 5 Column 17 File C:/temp/test.pl
| |(L20#Perl~Perl5=514#5c77da0^2#5c7a700:1#5c7a380:2 Line 5 Column 18 File C:/temp/test.pl
| | (constant#Perl~Perl5=101#5c73180^6#5c77bc0:1#5c77da0:1#5c77d60:1#5c77a00:1#5c7a420:1#5c77ae0:1 Line 5 Column 18 File C:/temp/test.pl
| | (string_prefix#Perl~Perl5=120#5c731a0^1#5c73180:1 Line 5 Column 18 File C:/temp/test.pl)string_prefix
| | (ORDINARY_STRING#Perl~Perl5=604#5c77400^1#5c73180:2[`I'] Line 5 Column 19 File C:/temp/test.pl)ORDINARY_STRING
| | )constant#5c73180
| | (constant#Perl~Perl5=101#5c77e00^6#5c7a6a0:2#5c7a740:2#5c7a2a0:2#5c7a160:2#5c77da0:2#5c7a140:2 Line 5 Column 24 File C:/temp/test.pl
| | (string_prefix#Perl~Perl5=120#5c7a020^1#5c77e00:1 Line 5 Column 24 File C:/temp/test.pl)string_prefix
| | (ORDINARY_STRING#Perl~Perl5=604#5c7a000^1#5c77e00:2[`want'] Line 5 Column 25 File C:/temp/test.pl)ORDINARY_STRING
| | )constant#5c77e00
| |)L20#5c77da0
| )L1_term#5c7a700
| (L1_term#Perl~Perl5=187#5c7b420^14#5c7b600:2#5c7b780:2#5c7b620:2#5c7c7c0:1#5c7e880:1#5c20600:1#5c7d160:1#5c7c0c0:1#5c7bf00:2#5c7bf80:2#5c20c20:1#5c7c360:1#5c7b540:2#5c7b560:2 Line 5 Column 34 File C:/temp/test.pl
| |(qw_constant#Perl~Perl5=113#5c7b260^1#5c7b420:1 Line 5 Column 35 File C:/temp/test.pl
| | (WORD_LIST_STRING_START#Perl~Perl5=630#5c7b180^1#5c7b260:1["("] Line 5 Column 37 File C:/temp/test.pl)WORD_LIST_STRING_START
| | (word_list#Perl~Perl5=117#5c7b240^1#5c7b260:2 Line 5 Column 38 File C:/temp/test.pl
| | (word_list#Perl~Perl5=117#5c7b200^1#5c7b240:1 Line 5 Column 38 File C:/temp/test.pl
| | (word_list#Perl~Perl5=116#5c7b1c0^1#5c7b200:1 Line 5 Column 38 File C:/temp/test.pl)word_list
| | (WORD#Perl~Perl5=633#5c7b1a0^1#5c7b200:2[`to'] Line 5 Column 38 File C:/temp/test.pl)WORD
| | )word_list#5c7b200
| | (WORD#Perl~Perl5=633#5c7b1e0^1#5c7b240:2[`get'] Line 5 Column 41 File C:/temp/test.pl)WORD
| | )word_list#5c7b240
| | (STRING_END#Perl~Perl5=620#5c7b220^1#5c7b260:3[")"] Line 5 Column 44 File C:/temp/test.pl)STRING_END
| |)qw_constant#5c7b260
| )L1_term#5c7b420
| )L20#5c7b540
| (L1_term#Perl~Perl5=187#5c20260^12#5c20e80:2#5c20ea0:2#5c20f20:2#5c20f60:2#5c205c0:2#5c20600:2#5c20620:2#5c20380:2#5c7f240:2#5c7f260:2#5c7e880:2#5c7e8a0:2 Line 6 Column 9 File C:/temp/test.pl
| (L20#Perl~Perl5=514#5c7e7e0^1#5c20260:1 Line 6 Column 10 File C:/temp/test.pl
| |(constant#Perl~Perl5=103#5c7d820^5#5c7de60:1#5c7e7e0:1#5c7e140:1#5c7f940:1#5c7dd60:1 Line 6 Column 10 File C:/temp/test.pl
| | (INTERPOLATED_STRING_START#Perl~Perl5=621#5c7d5e0^2#5c7d840:1#5c7d820:1["""] Line 6 Column 10 File C:/temp/test.pl)INTERPOLATED_STRING_START
| | (interpolated_string_content#Perl~Perl5=130#5c7d800^2#5c7d840:2#5c7d820:2 Line 6 Column 11 File C:/temp/test.pl
| | (interpolated_string_content#Perl~Perl5=129#5c7d7c0^1#5c7d800:1 Line 6 Column 11 File C:/temp/test.pl)interpolated_string_content
| | (ORDINARY_INTERPOLATED_STRING_CONTENT#Perl~Perl5=642#5c7c6e0^1#5c7d800:2[`these'] Line 6 Column 11 File C:/temp/test.pl)ORDINARY_INTERPOLATED_STRING_CONTENT
| | )interpolated_string_content#5c7d800
| | (STRING_END#Perl~Perl5=620#5c7d7e0^2#5c7d840:3#5c7d820:3["""] Line 6 Column 16 File C:/temp/test.pl)STRING_END
| |)constant#5c7d820
| |(constant#Perl~Perl5=103#5c7dc40^3#5c7e7e0:2#5c7e840:2#5c7e8e0:2 Line 6 Column 19 File C:/temp/test.pl
| | (INTERPOLATED_STRING_START#Perl~Perl5=621#5c76320^4#5c7dec0:1#5c7e420:1#5c7e400:1#5c7dc40:1["""] Line 6 Column 19 File C:/temp/test.pl)INTERPOLATED_STRING_START
| | (interpolated_string_content#Perl~Perl5=130#5c76800^4#5c7dec0:2#5c7e420:2#5c7e400:2#5c7dc40:2 Line 6 Column 20 File C:/temp/test.pl
| | (interpolated_string_content#Perl~Perl5=129#5c76140^1#5c76800:1 Line 6 Column 20 File C:/temp/test.pl)interpolated_string_content
| | (ORDINARY_INTERPOLATED_STRING_CONTENT#Perl~Perl5=642#5c76120^1#5c76800:2[`arguments'] Line 6 Column 20 File C:/temp/test.pl)ORDINARY_INTERPOLATED_STRING_CONTENT
| | )interpolated_string_content#5c76800
| | (STRING_END#Perl~Perl5=620#5c76180^4#5c7dec0:3#5c7e420:3#5c7e400:3#5c7dc40:3["""] Line 6 Column 29 File C:/temp/test.pl)STRING_END
| |)constant#5c7dc40
| )L20#5c7e7e0
| )L1_term#5c20260
| )L20#5c7f240
|)call_statement#5c209a0
)statement#5c7cf40
)block#5c21640
)statements#5c21c80
)Perl#5c21d20
You still have to walk over the parse tree and pick out the arguments. If you do pick up subtrees, they can be prettyprinted to generate the surface syntax string.
Related
I'm having an issue trying to accurately parse log files which time stamps may (or may not parse multiple files.
Example:
3 log files that get rotated so oldest logs are in time.log.3 and newest logs are in time.log.1
snapshot of the 3 log files
time.log.1
line 17 | 2021-03-17 21:08
line 18 | test5
line 19 | 2021-03-17 21:09
line 20 | test6
line 21 | 2021-03-18 22:10
line 22 | test7
line 23 | 2021-03-18 22:11
line 24 | test8
time.log.2
line 09 | 2021-03-16 21:04
line 10 | test9
line 11 | 2021-03-16 22:05
line 12 | test10
line 13 | 2021-03-17 19:06
line 14 | test11
line 15 | 2021-03-17 19:07
line 16 | test12
time.log.3
line 01 | 2021-03-16 19:00
line 02 | test1
line 03 | 2021-03-16 19:01
line 04 | test2
line 05 | 2021-03-16 20:02
line 06 | test3
line 07 | 2021-03-16 20:03
line 08 | test4
and so I parse them in reverse order using for loop and sed pattern match
for (( i = 3; i >= 1; --i )); do sudo cat time.log.$i | sed -nre '/2021-03-16 19:00/,/2021-03-16 19:01/p;'; done
all is good if you know exact timestamp:
line 01 | 2021-03-16 19:00
line 02 | test1
line 03 | 2021-03-16 19:01
but if I only know approx time range (like changeing it to 19:02) then because that exact pattern is not found it prints the rest of file...
for (( i = 3; i >= 1; --i )); do sudo cat time.log.$i | sed -nre '/2021-03-16 19:00/,/2021-03-16 19:02/p;'; done
yields this, but the 20:XX timestamps are outside the desire window of 19:00 thru 19:02... this is undesirable.
line 01 | 2021-03-16 19:00
line 02 | test1
line 03 | 2021-03-16 19:01
line 04 | test2
line 05 | 2021-03-16 20:02
line 06 | test3
line 07 | 2021-03-16 20:03
line 08 | test4
I need it to print only timestamps within a range, but am having trouble finding the answer... I've tried many google searches for awk, sed, perl, yet cannot land an answer... I feel I need to get past this hurdle before expecting larger time windows spanning multiple files to work...
Expounding on some of the criteria. The parse cannot happen upon the pipe (|) character, those Line number exists only to give context to the reverse order nature log file rotations and log info within the files. The parse can only happen upon the timestamps alone, and needs to contain all the "test#" info too in between the the timestamps, for not every line will have a timestamp.
This type of timestamp can be compared as a string. sed doesn't do comparisons, but awk does:
awk -F' [|] ' '$NF>="2021-03-16 19:00" && $NF<"2021-03-16 19:02"' time.log.{3,2,1}
Note that {3,2,1} is not a filename generating pattern (aka glob), and awk will quit if it encounters a non-existent file argument. It would be better if the files were named in such a way that log.* would work (i.e. sorted in ascending order lexicographically).
When not every line contains a timestamp, detecting if there is a timestamp might complicate things:
awk -F ' [|] ' '
$NF~/^[0-9]{4}(-[0-9]{2}){2} [0-9]{2}:[0-9]{2}$/ {x=$NF} x>=a; x>=b{exit}
' a='2021-03-16 21:00' b='2021-03-17 19:00' time.log.{3,2,1}
Or maybe the lines are grouped in pairs:
awk -F ' [|] ' '
NR%2==1 {x=$NF} x>=a; x>=b{exit}
' a='2021-03-16 21:00' b='2021-03-17 19:00' time.log.{3,2,1}
Maybe the input doesn't really contain ... | :
awk '
NR%2==1 {x=$0} x>=a; x>=b{exit}
' a='2021-03-16 21:00' b='2021-03-17 19:00' time.log.{3,2,1}
It's regarding powershell.
I have csv file with name a and it contains values 1 2 3 4 5 in first column and I have another csv file with name b and it contains values 6 7 8 9 10.
b.csv file output as follows.
6
7
8
9
10
a.csv file output as follows.
1
2
3
4
5
Desired result should be as follows
1 6
2 7
3 8
4 9
5 10
We are copying the output of b.csv file to a.csv file in column2.
here's one way to do the job. [grin] it presumes the two collections have the same number of items. if not, there will be errors.
# fake reading in a CSV file
# in real life, use Import-CSV
$A_In = #'
1
2
3
4
5
'# | ConvertFrom-Csv -Header 'A_Col'
# fake reading in another CSV file
$B_In = #'
6
7
8
9
10
'# | ConvertFrom-Csv -Header 'B_Col'
# the ".GetUpperBound(0)" stuff gets the upper bound of the zero axis of the collection
# that is the same as ".Count - 1"
$Results = foreach ($Index in 0..$A_In.GetUpperBound(0))
{
[PSCustomObject]#{
A_Col = $A_In[$Index].A_Col
B_Col = $B_In[$Index].B_Col
}
}
$Results
output ...
A_Col B_Col
----- -----
1 6
2 7
3 8
4 9
5 10
at this point, the $Results collection can be gracefully sent to a CSV via Export-CSV.
I have a file with the below contents:
0
ABC
1
181.12
2
05/07/16
3
1002
4
1211511108
6
1902
7
1902
10
hello
-1
0
ABC
1
1333.21
2
02/02/16
3
1294
4
1202514258
6
1294
7
1294
10
HAI
-1
...
I want to transpose the above file contents like below. The '-1' in above lists is the record separator which indicates the start of the next record.
ABC,181.12,05/07/16,1002,1211511108,1902,1902,hello
ABC,1333.21,02/02/16,1294,1202514258,1294,1294,HAI
...
Please let me know how to achieve this.
Read the file as a single string:
$txt = Get-Content 'C:\path\to\your.txt' | Out-String
Split the content at -1 lines:
$txt -split '(?m)^-1\r?\n'
Split each block at line breaks:
... | ForEach-Object {
$arr = $_ -split '\r?\n'
}
Select the values at odd indexes (skip the number lines) and join them by commas:
$indexes = 1..$($arr.Count - 1) | Where-Object { ($_ % 2) -ne 0 }
$arr[$indexes] -join ','
My example code output:
time | name | status | s_used | s_max |
+------------+-------------+-----------+------------+-----------+
| 1482222363 | asf | Closed | 0/16 | 0 |
| 1482222363 | as0 | Available | 4/16 | 4 |
I have attached the part of my output which is generated using perl cgi script and mysql database.
My query is how to take denominator value from the column s_used and store only the denominator values in the s_max column using perl.
3.I had attached the following part of code which i tried.
if($i == 4){
if(/s_used/){
print;
}
else{
chomp();
my($num,$s_max)=split /\//,$table_data{2}{'ENTRY'};
print $s_max;
}
}
Code Explanation:
$i == 4 is the column where should I store the variable.
I got time column from the sql database $time, name I got from $table_data{0}{'ENTRY'}, status from $table_data{1}{'ENTRY'}, s_used from $table_Data{2}{'ENTRY'}.
Expected output:
time | name | status | s_used | s_max |
+------------+-------------+-----------+------------+-----------+
| 1482222363 | asf | Closed | 0/16 | 16 |
| 1482222363 | as0 | Available | 4/16 | 16 |
Seems your code "my($num,$s_max)=split /\//,$table_data{2}{'ENTRY'};" is right.
Somehow the value $s_max at the time it's writing to the DB is incorrect. Since you did not post the portion of code to show the part $s_max writing back to the DB, you need to check what value is in $s_max (e.g. printing the $s_max value) at the time right before writing it back to DB. From there, please try to trace back why an incorrect value is assigned to $s_max. Then, the problem would be solved.
I have got text file:
1 2 4 5 6 7
1 3 5 6 7 8
1 2 3 4 5 6
1 2 4 5 6 7
Here first and last line are simmilar. I have a lot of files that have double lines. I need to delete all dublicate.
All these seem really complicated. It is as simple as:
gc $filename | sort | get-unique > $output
Using actual file names instead of variables:
gc test.txt| sort | get-unique > unique.txt
To get unique lines:
PS > Get-Content test.txt | Select-Object -Unique
1 2 4 5 6 7
1 3 5 6 7 8
1 2 3 4 5 6
To remove the duplicate
PS > Get-Content test.txt | group -noelement | `
where {$_.count -eq 1} | select -expand name
1 3 5 6 7 8
1 2 3 4 5 6
If order is not important:
Get-Content test.txt | Sort-Object -Unique | Set-Content test-1.txt
If order is important:
$set = #{}
Get-Content test.txt | %{
if (!$set.Contains($_)) {
$set.Add($_, $null)
$_
}
} | Set-Content test-2.txt
Try something like this:
$a = #{} # declare an arraylist type
gc .\mytextfile.txt | % { if (!$a.Contains($_)) { $a.add($_)}} | out-null
$a #now contains no duplicate lines
To set the content of $a to mytextfile.txt:
$a | out-file .\mytextfile.txt
$file = "C:\temp\filename.txt"
(gc $file | Group-Object | %{$_.group | select -First 1}) | Set-Content $file
The source file now contains only unique lines
The already posted options did not work for me for some reason