I see this awk idiom all the time and I would like to know exactly what it means
while (getline <"FILE" > 0)
I understand what getline is doing, but I don't get redirection to 0.
According to The Awk Programming Language, you do this to avoid a infinite loop in the case of FILE being nonexistant.
Why does redirection get rid of that error? What exactly is being redirected? It can't be the return value of getline, otherwise, what expression is while evaluating? I'm missing something!
EDIT: Thanks for the comments. I got redirection operators confused with relational operators. Perhaps it would be clearer if it was written like this
while ((getline <"FILE") > 0)
Or even clearer
while (0 < (getline <"FILE"))
The > sign is testing the return value from getline (for greater than zero). It is not a redirection sign.
From https://www.gnu.org/software/gawk/manual/html_node/Getline.html :
The getline command returns one if it finds a record and zero if it
encounters the end of the file. If there is some error in getting a
record, such as a file that cannot be opened, then getline returns -1.
In this case, gawk sets the variable ERRNO to a string describing the
error that occurred.
The confusion may come from the fact that output redirection to a file is not used by getline. (It is used only to read data). So there should be no ambiguity regarding the > sign. (It cannot mean output redirection)
To print something to a file you can use output redirection from the print or printf commands.
Related
When I run a file which is begin with #!/usr/bin/perl -w, I get a error:
syntax error at line 153, near "=~ ?"
I try to add "#!/bin/bash", this error is not append, but I get another
error:
"line 34: syntax error near unexpected token `('"
line 153 in my file:
($output_volume =~ ?^([\S]+).mnc?) && ($base_name = $1) ||
die "sharpen_volume failed: output volume does not appear to be"
." a minc volume.\n";
line34 in my file:
use MNI::Startup qw(nocputimes);
$output_volume =~ ?^([\S]+).mnc?
This used to be valid perl and thus might appear in old code and instructional material.
From perlop:
In the past, the leading m in m?PATTERN? was optional, but omitting it would produce a deprecation warning. As of v5.22.0, omitting it produces a syntax error. If you encounter this construct in older code, you can just add m.
That is Perl code so the first error message is meaningful.
With delimiters other than // in the match operator you must have the explicit m for it, so
$output_volume =~ m?^([\S]+).mnc?
It is only with // delimiters that the m may be omitted; from Regex Quote-Like Operators (perlop)
If "/" is the delimiter then the initial m is optional.
See perlretut for a tutorial introduction to regex and perlre for reference.
Also note that the particular delimiters of ? trigger specific regex behavior in a special case. This is discussed by the end of the documentation section in perlop linked above.
You already have two answers that explain the problem.
? ... ? is no longer valid syntax for a match operator. You need m? ... ? instead.
Until Perl 5.22, your syntax generated a warning. Now it's a fatal error (which is what you are seeing). So I assume you're now running this on a more recent version of Perl.
There are, however, a few other points it is probably worth making.
You say you tried to investigate this by changing the first line of your file from #!/usr/bin/perl -w to #!/bin/bash. I'm not sure how you think this was going to help. This line defines the program that is used to run your code. As you have Perl code, you need to run it with Perl. Trying to run it with bash is very unlikely to be useful.
The m? ... ? (or, formerly, ? ... ?) syntax triggers an obscure and specialised behaviour. It seems to me that this behaviour isn't required in your case, so you can probably change it to the more usual / ... /.
Your regex contains an unescaped dot character. Given that you seem to be extracting the basename from a filename that has an extension, it seems likely that this should be escaped (using \.) so that it matches an actual dot (rather than any character).
If you are using this code to extract a file's basename, then using a regex probably isn't the best approach. Perhaps take a look at File::Basename instead.
-- Edit : Resolved. See answer.
Background:
I'm writing a shell that will perform some extra actions required on our system when someone resizes a database.
The shell is written in ksh (requirement), the OS is Solaris 5.10 .
The problem is with one of the checks, which verifies there's enough free space on the underlying OS.
Problem:
The check reads the df -k line for root, which is what I check in this step, and prints it to a file. I then "read" the contents into variables which I use in calculations.
Unfortunately, when I try to run an arithmetic operation on one of the variables, I get an error indicating it is null. And a debug output line I've placed after that line verifies that it is null... It lost it's value...
I've tried every method of doing this I could find online, they work when I run it manually, but not inside the shell file.
(* The file does have #!/usr/bin/ksh)
Code:
df -k | grep "rpool/ROOT" > dftest.out
RPOOL_NAME=""; declare -i TOTAL_SIZE=0; USED_SPACE=0; AVAILABLE_SPACE=0; AVAILABLE_PERCENT=0; RSIGN=""
read RPOOL_NAME TOTAL_SIZE USED_SPACE AVAILABLE_SPACE AVAILABLE_PERCENT RSIGN < dftest.out
\rm dftest.out
echo $RPOOL_NAME $TOTAL_SIZE $USED_SPACE $AVAILABLE_SPACE $AVAILABLE_PERCENT $RSIGN
((TOTAL_SIZE=$TOTAL_SIZE/1024))
This is the result:
DBResize.sh[11]: TOTAL_SIZE=/1024: syntax error
I'm pulling hairs at this point, any help would be appreciated.
The code you posted cannot produce the output you posted. Most obviously, the error is signalled at line 11 but you posted fewer than 11 lines of code. The previous lines may matter. Always post complete code when you ask for help.
More concretely, the declare command doesn't exist in ksh, it's a bash thing. You can achieve the same result with typeset (declare is a bash equivalent to typeset, but not all options are the same). Either you're executing this script with bash, or there's another error message about declare, or you've defined some additional commands including declare which may change the behavior of this code.
None of this should have an impact on the particular problem that you're posting about, however. The variables created by read remain assigned until the end of the subshell, i.e. until the code hits a ), the end of a pipe (left-hand side of the pipe only in ksh), etc.
About the use of declare or typeset, note that you're only declaring TOTAL_SIZE as an integer. For the other variables, you're just assigning a value which happens to consist exclusively of digits. It doesn't matter for the code you posted, but it's probably not what you meant.
One thing that may be happening is that grep matches nothing, and therefore read reads an empty line. You should check for errors. Use set -e in scripts to exit at the first error. (There are cases where set -e doesn't catch errors, but it's a good start.)
Another thing that may be happening is that df is splitting its output onto multiple lines because the first column containing the filesystem name is too large. To prevent this splitting, pass the option -P.
Using a temporary file is fragile: the code may be executed in a read-only directory, another process may want to access the same file at the same time... Here a temporary file is useless. Just pipe directly into read. In ksh (unlike most other sh variants including bash), the right-hand side of a pipe runs in the main shell, so assignments to variables in the right-hand side of a pipe remain available in the following commands.
It doesn't matter in this particular script, but you can use a variable without $ in an arithmetic expression. Using $ substitutes a string which can have confusing results, e.g. a='1+2'; $((a*3)) expands to 7. Not using $ uses the numerical value (in ksh, a='1+2'; $((a*3)) expands to 9; in some sh implementations you get an error because a's value is not numeric).
#!/usr/bin/ksh
set -e
typeset -i TOTAL_SIZE=0 USED_SPACE=0 AVAILABLE_SPACE=0 AVAILABLE_PERCENT=0
df -Pk | grep "rpool/ROOT" | read RPOOL_NAME TOTAL_SIZE USED_SPACE AVAILABLE_SPACE AVAILABLE_PERCENT RSIGN
echo $RPOOL_NAME $TOTAL_SIZE $USED_SPACE $AVAILABLE_SPACE $AVAILABLE_PERCENT $RSIGN
((TOTAL_SIZE=TOTAL_SIZE/1024))
Strange...when I get rid of your "declare" line, your original code seems to work perfectly well (at least with ksh on Linux)
The code :
#!/bin/ksh
df -k | grep "/home" > dftest.out
read RPOOL_NAME TOTAL_SIZE USED_SPACE AVAILABLE_SPACE AVAILABLE_PERCENT RSIGN < dftest.out
\rm dftest.out
echo $RPOOL_NAME $TOTAL_SIZE $USED_SPACE $AVAILABLE_SPACE $AVAILABLE_PERCENT $RSIGN
((TOTAL_SIZE=$TOTAL_SIZE/1024))
print $TOTAL_SIZE
The result :
32962416 5732492 25552588 19% /home
5598
Which are the value a simple df -k is returning. The variables seem to last.
For those interested, I have figured out that it is not possible to use "read" the way I was using it.
The variable values assigned by "read" simply "do not last".
To remedy this, I have applied the less than ideal solution of using the standard "while read" format, and inside the loop, echo selected variables into a variable file.
Once said file was created, I just "loaded" it.
(pseudo code:)
LOOP START
echo "VAR_A="$VAR_A"; VAR_B="$VAR_B";" > somefile.out
LOOP END
. somefile.out
I am experiencing strange results with Perl's short circuited and, that is &&.
I am trying to solve a Project Euler problem where I want a certain number to be divisible by a list of numbers.
$b=42;
if($b%21==0 && b%2==0 && b%5==2){print "Why not?"};
Should print "Why not" as far as I can see, but keeps silent.
$b=42;
if($b%21==0 && b%2==0 && b%5==0){print "WTF?"};
Should keep silent, but prints "WTF?".
What gives?
As Rohit answered, the solution is to add the $ before the b. The exact reason it doesn't print "Why not?" but prints "WTF" is this: when you give the b without the $ sign (and without use strict; in force), Perl treats the b as the string "b". Then when you apply the operator % on it, since % is a numerical operator, Perl looks within the string "b" and checks whether it starts with a number. Since it doesn't, Perl takes the numerical value of "b" as 0, and then applies the mod (%) operation. 0%5 is 0 and not 2, so WTF is printed and not "Why not?"
Always use use strict and use warnings.
You are using your last two b's as bareword, which will be shown as warning - "Unquoted string "b" may clash with future reserved word"
You need to change your if to: -
if($b%21==0 && $b%2==0 && $b%5==2){print "Why not?"};
and: -
if($b%21==0 && $b%2==0 && $b%5==0){print "WTF?"};
gives expected results.
if($b%21==0 && $b%2==0 && $b%5==2){print "Why not?"};
works over here, you forgot the $, but apearntly you already found it :)
I am writing a program where if no command line arguments are supplied i.e #ARGV == 0, the program takes in three inputs. But, the program has the feature to read any files given as arguments, thus
calculate input1 input2
runs the formula on the numbers found in file1 and file2.
The problem I am running into is when I run
calculate < input1
#ARGV returns 0, thus it runs the code for user input.
How do I get around this so that the program can read input1 and use the values inside for calculations?
calculate < input1 is equivalent to cat input1 | calculate.
You need to read from <STDIN> and not look for command line arguments.
That should not be a problem. If you read reading from <> (which is really <ARGV>), then there is no difference.
You must be doing something wrong if redirection changes things. Are you actually opening files yourself???
You might consider using a module like Getopt::Euclid or Getopt::Long to make the argument passing more explicit. That might make the program easier to understand for other users too.
I got this strange line of code today, it tells me 'empty' or 'not empty' depending on whether the CWD has any items (other than . and ..) in it.
I want to know how it works because it makes no sense to me.
perl -le 'print+(q=not =)[2==(()=<.* *>)].empty'
The bit I am interested in is <.* *>. I don't understand how it gets the names of all the files in the directory.
It's a golfed one-liner. The -e flag means to execute the rest of the command line as the program. The -l enables automatic line-end processing.
The <.* *> portion is a glob containing two patterns to expand: .* and *.
This portion
(q=not =)
is a list containing a single value -- the string "not". The q=...= is an alternate string delimiter, apparently used because the single-quote is being used to quote the one-liner.
The [...] portion is the subscript into that list. The value of the subscript will be either 0 (the value "not ") or 1 (nothing, which prints as the empty string) depending on the result of this comparison:
2 == (()=<.* *>)
There's a lot happening here. The comparison tests whether or not the glob returned a list of exactly two items (assumed to be . and ..) but how it does that is tricky. The inner parentheses denote an empty list. Assigning to this list puts the glob in list context so that it returns all the files in the directory. (In scalar context it would behave like an iterator and return only one at a time.) The assignment itself is evaluated in scalar context (being on the right hand side of the comparison) and therefore returns the number of elements assigned.
The leading + is to prevent Perl from parsing the list as arguments to print. The trailing .empty concatenates the string "empty" to whatever came out of the list (i.e. either "not " or the empty string).
<.* *>
is a glob consisting of two patterns: .* are all file names that start with . and * corresponds to all files (this is different than the usual DOS/Windows conventions).
(()=<.* *>)
evaluates the glob in list context, returning all the file names that match.
Then, the comparison with 2 puts it into scalar context so 2 is compared to the number of files returned. If that number is 2, then the only directory entries are . and .., period. ;-)
<.* *> means (glob(".*"), glob("*")). glob expands file patterns the same way the shell does.
I find that the B::Deparse module helps quite a bit in deciphering some stuff that throws off most programmers' eyes, such as the q=...= construct:
$ perl -MO=Deparse,-p,-q,-sC 2>/dev/null << EOF
> print+(q=not =)[2==(()=<.* *>)].empty
> EOF
use File::Glob ();
print((('not ')[(2 == (() = glob('.* *')))] . 'empty'));
Of course, this doesn't instantly produce "readable" code, but it surely converts some of the stumbling blocks.
The documentation for that feature is here. (Scroll near the end of the section)