I have 2 commands that I need to run back to back 16 times for 2 sets of data. I have labeled the files used as file#a1_100.gen (set 1) and file#a2_100.gen (set 2). The 100 is then replaced by multiples of 100 upto 1600 (100,200,...,1000,...,1600).
Example 1: For first set
Command 1: perl myprogram1.pl file#a1.pos abc#a1.ref xyz#a1.ref file#a1_100.gen file#a1_100.out
Command 2: perl my program2.pl file#a1_100.out file#a1_100.out.long
Example 2: For first set
Command 1: perl myprogram1.pl file#a1.pos abc#a1.ref xyz#a1.ref file#a1_200.gen file#a1_200.out
Command 2: perl my program2.pl file#a1_200.out file#a1_200.out.long
These 2 commands are repeated 16 times for both set 1 and set 2. For set 2 the filename changes to File#a2...
I need a command that will run this on its own by changing the filename for the 2 sets, running it 16 times for each set.
Any help will be greatly appreciated! Thanks!
This is probably most easily done with a shell script. As with Perl, TMTOWTDI — there's more than one way to do it.
for num in $(seq 1 16)
do
perl myprogram1.pl file#a1.pos abc#a1.ref xyz#a1.ref file#a1_${num}00.gen file#a1_${num}00.out
perl myprogram2.pl file#a1_${num}00.out file#a1_${num}00.out.long
done
(You could use {1..16} in place of $(seq 1 16) to generate the numbers. You might also note that the # characters in the file names discombobulate the SO Markdown system.)
Or you could use:
for num in $(seq 100 100 1600)
do
perl myprogram1.pl file#a1.pos abc#a1.ref xyz#a1.ref file#a1_${num}.gen file#a1_${num}.out
perl myprogram2.pl file#a1_${num}.out file#a1_${num}.out.long
done
(I don't think there's a {...} expansion for that.)
Or, better, you could use variables to hold values to avoid repetition:
POS="file#a1.pos"
ABC="abc#a1.ref"
XYZ="xyz#a1.ref"
for num in $(seq 100 100 1600)
do
PFX="file#a1_${num}"
GEN="${PFX}.gen"
OUT="${PFX}.out"
LONG="${OUT}.long"
perl myprogram1.pl "${POS}" "${ABC}" "${XYZ}" "${GEN}" "${OUT}"
perl myprogram2.pl "${OUT}" "${LONG}"
done
In this code, the braces around the parameter names are all optional; in the first block of code, the braces around ${num} were mandatory, but optional in the second set. Enclosing names in double quotes is also optional here, but recommended.
Or, if you must do it in Perl, then:
use warnings;
use strict;
my $POS = "file#a1.ref";
my $ABC = "abc#a1.ref";
my $XYZ = "xyz#a1.ref";
for (my $num = 100; $num <= 1600; $num += 100)
{
my $PFX = "file#a1_${num}";
my $GEN = "${PFX}.gen";
my $OUT = "${PFX}.out";
my $LONG = "${OUT}.long";
system("perl", "myprogram1.pl", "${POS}", "${ABC}", "${XYZ}", "${GEN}", "${OUT}");
system("perl", "myprogram2.pl", "${OUT}", "${LONG}");
}
This is all pretty basic coding. And you can guess that it didn't take me long to generate this from the last shell script. Note the use of multiple separate strings instead on one long string in the system calls. That avoids running a shell interpreter — Perl runs perl directly.
You could use $^X instead of "perl" to ensure that you run the same Perl executable as ran the script shown. (If you have /usr/bin/perl on your PATH but you run $HOME/perl/v5.20.1/bin/perl thescript.pl, the difference might matter, but probably wouldn't.)
Related
I am a beginer in perl and I need to modify a txt file by keeping all the previous data in it and only modify the file by adding 4 to every number related to a specific tag (< COMPRESSED-SIZE >). The file have many lines and tags and looks like below, I need to find all the < COMPRESSED-SIZE > tags and add 4 to the number specified near the tag:
< SOURCE-START-ADDRESS >01< /SOURCE-START-ADDRESS >
< COMPRESSED-SIZE >132219< /COMPRESSED-SIZE >
< UNCOMPRESSED-SIZE >229376< /UNCOMPRESSED-SIZE >
So I guess I need to do something like: search for the keyword(match) and store the number 132219 in a variable and add the second number (4) to it, replace the result 132219 with 132223, the rest of the file must remain unchanged, only the numbers related to this tag must change. I cannot search for the number instead of the tag because the number could change while the tag will remain always the same. I also need to find all the tags with this name and replace the numbers near them by adding 4 to them. I already have the code for finding something after a keyword, because I needed to search also for another tag, but this script does something else, adds a number in front of a keyword. I think I could use this code for what i need, but I do not know how to make the calculation and keep the rest of the file intact or if it is posible in perl.
while (my $row = <$inputFileHandler>)
{
if(index($row,$Data_Pattern) != -1){
my $extract = substr($row, index($row,$Data_Pattern) + length($Data_Pattern), length($row));
my $counter_insert = sprintf "%08d", $counter;
my $spaces = " " x index($row,$Data_Pattern);
$data_to_send ="what i need to add" . $extract;
print {$outs} $spaces . $Data_Pattern . $data_to_send;
$counter = $counter + 1;
}
else
{
print {$outs} $row;
next;
}
}
Maybe you could help me with a block of code for my needs, $Data_Pattern is the match. Thank you very much!
This is a classic one-liner Perl task. Basically you would do something like
$ perl -i.bak -pe's/^< COMPRESSED-SIZE >\K(\d+)/$1 + 4/e' yourfile.txt
Which will in essence copy and replace your file with a new, edited file. This can be very dangerous, especially if you are a Perl newbie. The -i switch is here used with the .bak extension which saves a backup in yourfile.txt.bak. This does not make this operation safe, however, as running the command twice will overwrite the backup.
It is advisable to make a separate backup of the target file before using this command.
-i.bak edit "in-place", the file is overwritten, a backup of the original is created with extension .bak.
-p argument is treated as a file name, which is read, and printed back.
s/ // the substitution operator, which is applied to all lines of the file.
^ inside the regex looks for beginning of line.
\K keep the match that is to the left.
(\d+) capture () 1 or more digits \d+ and store them in $1
/e treat the right hand side of the substitution operator as an expression and use the result as the replacement string. In this case it will increase your number and return the sum.
The long version of this command is
while (<>) {
s/^< COMPRESSED-SIZE >\K(\d+)/$1 + 4/e
}
Which can be placed in a file and run with the -i switch.
-- Edit : Resolved. See answer.
Background:
I'm writing a shell that will perform some extra actions required on our system when someone resizes a database.
The shell is written in ksh (requirement), the OS is Solaris 5.10 .
The problem is with one of the checks, which verifies there's enough free space on the underlying OS.
Problem:
The check reads the df -k line for root, which is what I check in this step, and prints it to a file. I then "read" the contents into variables which I use in calculations.
Unfortunately, when I try to run an arithmetic operation on one of the variables, I get an error indicating it is null. And a debug output line I've placed after that line verifies that it is null... It lost it's value...
I've tried every method of doing this I could find online, they work when I run it manually, but not inside the shell file.
(* The file does have #!/usr/bin/ksh)
Code:
df -k | grep "rpool/ROOT" > dftest.out
RPOOL_NAME=""; declare -i TOTAL_SIZE=0; USED_SPACE=0; AVAILABLE_SPACE=0; AVAILABLE_PERCENT=0; RSIGN=""
read RPOOL_NAME TOTAL_SIZE USED_SPACE AVAILABLE_SPACE AVAILABLE_PERCENT RSIGN < dftest.out
\rm dftest.out
echo $RPOOL_NAME $TOTAL_SIZE $USED_SPACE $AVAILABLE_SPACE $AVAILABLE_PERCENT $RSIGN
((TOTAL_SIZE=$TOTAL_SIZE/1024))
This is the result:
DBResize.sh[11]: TOTAL_SIZE=/1024: syntax error
I'm pulling hairs at this point, any help would be appreciated.
The code you posted cannot produce the output you posted. Most obviously, the error is signalled at line 11 but you posted fewer than 11 lines of code. The previous lines may matter. Always post complete code when you ask for help.
More concretely, the declare command doesn't exist in ksh, it's a bash thing. You can achieve the same result with typeset (declare is a bash equivalent to typeset, but not all options are the same). Either you're executing this script with bash, or there's another error message about declare, or you've defined some additional commands including declare which may change the behavior of this code.
None of this should have an impact on the particular problem that you're posting about, however. The variables created by read remain assigned until the end of the subshell, i.e. until the code hits a ), the end of a pipe (left-hand side of the pipe only in ksh), etc.
About the use of declare or typeset, note that you're only declaring TOTAL_SIZE as an integer. For the other variables, you're just assigning a value which happens to consist exclusively of digits. It doesn't matter for the code you posted, but it's probably not what you meant.
One thing that may be happening is that grep matches nothing, and therefore read reads an empty line. You should check for errors. Use set -e in scripts to exit at the first error. (There are cases where set -e doesn't catch errors, but it's a good start.)
Another thing that may be happening is that df is splitting its output onto multiple lines because the first column containing the filesystem name is too large. To prevent this splitting, pass the option -P.
Using a temporary file is fragile: the code may be executed in a read-only directory, another process may want to access the same file at the same time... Here a temporary file is useless. Just pipe directly into read. In ksh (unlike most other sh variants including bash), the right-hand side of a pipe runs in the main shell, so assignments to variables in the right-hand side of a pipe remain available in the following commands.
It doesn't matter in this particular script, but you can use a variable without $ in an arithmetic expression. Using $ substitutes a string which can have confusing results, e.g. a='1+2'; $((a*3)) expands to 7. Not using $ uses the numerical value (in ksh, a='1+2'; $((a*3)) expands to 9; in some sh implementations you get an error because a's value is not numeric).
#!/usr/bin/ksh
set -e
typeset -i TOTAL_SIZE=0 USED_SPACE=0 AVAILABLE_SPACE=0 AVAILABLE_PERCENT=0
df -Pk | grep "rpool/ROOT" | read RPOOL_NAME TOTAL_SIZE USED_SPACE AVAILABLE_SPACE AVAILABLE_PERCENT RSIGN
echo $RPOOL_NAME $TOTAL_SIZE $USED_SPACE $AVAILABLE_SPACE $AVAILABLE_PERCENT $RSIGN
((TOTAL_SIZE=TOTAL_SIZE/1024))
Strange...when I get rid of your "declare" line, your original code seems to work perfectly well (at least with ksh on Linux)
The code :
#!/bin/ksh
df -k | grep "/home" > dftest.out
read RPOOL_NAME TOTAL_SIZE USED_SPACE AVAILABLE_SPACE AVAILABLE_PERCENT RSIGN < dftest.out
\rm dftest.out
echo $RPOOL_NAME $TOTAL_SIZE $USED_SPACE $AVAILABLE_SPACE $AVAILABLE_PERCENT $RSIGN
((TOTAL_SIZE=$TOTAL_SIZE/1024))
print $TOTAL_SIZE
The result :
32962416 5732492 25552588 19% /home
5598
Which are the value a simple df -k is returning. The variables seem to last.
For those interested, I have figured out that it is not possible to use "read" the way I was using it.
The variable values assigned by "read" simply "do not last".
To remedy this, I have applied the less than ideal solution of using the standard "while read" format, and inside the loop, echo selected variables into a variable file.
Once said file was created, I just "loaded" it.
(pseudo code:)
LOOP START
echo "VAR_A="$VAR_A"; VAR_B="$VAR_B";" > somefile.out
LOOP END
. somefile.out
I have made two different programs in QBasic and they both are saved in different .bas files, i.e one is 1.bas and the other 2.bas.
How to open program 1.bas while I am in program 2.bas, without closing it?
Program 1 should run inside program 2 for some time, and when it ends I should again be in program 2. Is there any way to do that?
I would like to know if there is a syntax for this that works in QBasic and/or QB64.
In Qbasic you can use CHAIN command to pass control to another .BAS file and when it is finished it will return to the first .BAS file. You can combine it with COMMON to also share variables between the two programs.
You could also use RUN but in QBasic you can't pass variables (not sure but I think the control will not return). And in QB64 it is possible to pass variables using RUN
See the standard COM1_EX.BAS and COM2_EX.BAS as an example, contents of COM1_EX.BAS:
' == COM1_EX.BAS - COMMON statement programming example ==
DIM Values(1 TO 50)
COMMON Values(), NumValues
PRINT "Enter values one per line. Type 'END' to quit."
NumValues = 0
DO
INPUT "-> ", N$
IF I >= 50 OR UCASE$(N$) = "END" THEN EXIT DO
NumValues = NumValues + 1
Values(NumValues) = VAL(N$)
LOOP
PRINT "Leaving COM1_EX.BAS to chain to COM2_EX.BAS"
PRINT "Press any key to chain... "
DO WHILE INKEY$ = ""
LOOP
CHAIN "com2_ex"
contents of COM2_EX.BAS:
' == COM2_EX.BAS - COMMON statement programming example ==
' Notice that the variables Values() and NumValues from COM1_EX
' will be called X() and N here in COM2_EX
DIM X(1 TO 50)
COMMON X(), N
PRINT
PRINT "Now executing file com2_ex.bas, reached through a CHAIN command"
IF N > 0 THEN
Sum = 0
FOR I = 1 TO N
Sum = Sum + X(I)
NEXT I
PRINT "The average of the values is"; Sum / N
END IF
I get zero for average sum. It does not pass the array values
Formerly, I used the Qbasic "Join" command successfully for my lengthy Structural programs but it never returned to the original program. In Structural Designing, you have to create a Member Matrix,Property Matrix,Do some Matrix Multiplications, and Load Matrix. Using another program to Inverse for the resulting Matrix. The "Chain" command in any Basic software does not work at all. There ought to be a command to run one program and go to execute another program and return the values/Text it created. As I am running Win 10, 64 bit and I'm not a Comp. expert,
I've been searching all over the place and since I'm taking my first steps in PERL this might be one of he dumbest questions but here it goes.
So I'm creating a script to manage my windows and later bind it to keyboard shortcuts, so I I'm trying to run a command and passing some variables:
my $command = `wmctrl -r :ACTIVE: -e 0,0,0,$monitors->{1}->{'width'}/2,$monitors->{1}->{'height'}`;
But I get an error saying I'm not passing the right parameters to the command, but if I do this, everything works great:
my $test = $monitors->{1}->{'width'}/2;
my $command = `wmctrl -r :ACTIVE: -e 0,0,0,$test,$monitors->{1}->{'height'}`;
So do I really have to do this? assign it first to a variable and then pass it, or there's a more elegant way of doing it?
The backticks operator (or the qx{}) accepts A string which is (possibly) interpolated. So accepts string and not expression like $var/2.
Thats mean than the $variables ($var->{1}->{some} too) are expanded but not the arithmetic expressions.
Therefore your 2 step variant works, but not the first.
If you want evaluate an expression inside the string you can use the next:
my $ans=42;
print "The #{[ $ans/2 ]} is only the half of answer\n";
prints
The 21 is only the half of answer
but it is not very readable, so better and elegant is what you're already doing - calculate the command argument in andvace, and to the qx{} or backticks only pass the calculated $variables.
Lets say i have a file like below:
And i want to store all the decimal numbers in a hash.
hello world 10 20
world 10 10 10 10 hello 20
hello 30 20 10 world 10
i was looking at this
and this worked fine:
> perl -lne 'push #a,/\d+/g;END{print "#a"}' temp
10 20 10 10 10 10 20 30 20 10 10
Then what i need was to count number of occurrences of each regex.
for this i think it would be better to store all the matches in a hash and assign an incrementing value for each and every key.
so i tried :
perl -lne '$a{$1}++ for ($_=~/(\d+)/g);END{foreach(keys %a){print "$_.$a{$_}"}}' temp
which gives me an output of:
> perl -lne '$a{$1}++ for ($_=~/(\d+)/g);END{foreach(keys %a){print "$_.$a{$_}"}}' temp
10.4
20.7
Can anybody correct me whereever i was wrong?
the output i expect is:
10.7
20.3
30.1
although i can do this in awk,i would like to do it only in perl
Also order of the output is not a concern for me.
$a{$1}++ for ($_=~/(\d+)/g);
This should be
$a{$_}++ for ($_=~/(\d+)/g);
and can be simplified to
$a{$_}++ for /\d+/g;
The reason for this is that /\d+/g creates a list of matches, which is then iterated over by for. The current element is in $_. I imagine $1 would contain whatever was left in there by the last match, but it's definitely not what you want to use in this case.
Another option would be this:
$a{$1}++ while ($_=~/(\d+)/g);
This does what I think you expected your code to do: loop over each successful match as the matches happen. Thus the $1 will be what you think it is.
Just to be clear about the difference:
The single argument for loop in Perl means "do something for each element of a list":
for (#array)
{
#do something to each array element
}
So in your code, a list of matches was built first, and only after the whole list of matches was found did you have the opportunity to do something with the results. $1 got reset on each match as the list was being built, but by the time your code was run, it was set to the last match on that line. That is why your results didn't make sense.
On the other hand, a while loop means "check if this condition is true each time, and keep going until the condition is false". Therefore, the code in a while loop will be executed on each match of a regex, and $1 has the value for that match.
Another time this difference is important in Perl is file processing. for (<FILE>) { ... } reads the entire file into memory first, which is wasteful. It is recommended to use while (<FILE>) instead, because then you go through the file line by line and keep only the information you want.