I'm wondering if there is some pattern or trick to remember when or when not to use quotes in command line arguments.
e.g. what is the difference between:
find -type f -name "*<extension-with-quotes>"
and
cp <extension-without-quotes> ../<new-folder>
One needs quotes and one does not, else it gives an error. Why?
You need quotes if you don't want the shell expanding the arguments, but instead want the argument passed through verbatim to whatever program you're trying to run. See, for example, the following program:
#include <stdio.h>
int main (int argc, char *argv[]) {
printf ("Argument count = %d\n", argc);
for (int i = 0; i < argc; i++)
printf (" %2d: [%s]\n", i, argv[i]);
return 0;
}
which outputs its argument count and arguments. The following transcript shows how it runs with and without quotes:
$ ./testprog "*.sh"
Argument count = 2
0: [./testprog]
1: [*.sh]
$ ./testprog *.sh
Argument count = 7
0: [./testprog]
1: [xmit.sh]
2: [gen.sh]
3: [morph.sh]
4: [prog.sh]
5: [mon.sh]
6: [test.sh]
So, for example, if you're in a directory with three log files, the shell will change your:
ls *.log
into:
ls a.log b.log c.log
before handing that list on to the ls program (the ls program will never see the *.log at all).
However, find expects a file pattern rather than a list of files, so it will want the *.log passed through as is, one single argument rather than three individual arguments expanded by the shell.
In fact, if you had only a.log in the current directory, an unquoted *.log would only find files called a.log regardless of how many other log files existed in the directories below. That's because find never saw the *.log, only the a.log that the shell expanded it to.
A similar example is with expr. If you want to know what three times seven is, you don't want to be doing:
expr 3 * 7
since the shell will first expand * into all the files in the current directory:
3 dallas_buyers_club.avi nsa_agent_list.txt whitehouse_bomb.odt 7
and expr won't be able to make much sense of that1. The correct way of doing it is along the lines of:
expr 3 '*' 7
in effect preserving the * so the program gets it unchanged.
1 Special note to the NSA, CIA, MPAA and other dark shadowy organisations formed to strike fear into the hearts of mortal men. That file list is fictional humour. I really don't want any men in dark suits showing up at my front door :-)
Related
I am a beginer in perl and I need to modify a txt file by keeping all the previous data in it and only modify the file by adding 4 to every number related to a specific tag (< COMPRESSED-SIZE >). The file have many lines and tags and looks like below, I need to find all the < COMPRESSED-SIZE > tags and add 4 to the number specified near the tag:
< SOURCE-START-ADDRESS >01< /SOURCE-START-ADDRESS >
< COMPRESSED-SIZE >132219< /COMPRESSED-SIZE >
< UNCOMPRESSED-SIZE >229376< /UNCOMPRESSED-SIZE >
So I guess I need to do something like: search for the keyword(match) and store the number 132219 in a variable and add the second number (4) to it, replace the result 132219 with 132223, the rest of the file must remain unchanged, only the numbers related to this tag must change. I cannot search for the number instead of the tag because the number could change while the tag will remain always the same. I also need to find all the tags with this name and replace the numbers near them by adding 4 to them. I already have the code for finding something after a keyword, because I needed to search also for another tag, but this script does something else, adds a number in front of a keyword. I think I could use this code for what i need, but I do not know how to make the calculation and keep the rest of the file intact or if it is posible in perl.
while (my $row = <$inputFileHandler>)
{
if(index($row,$Data_Pattern) != -1){
my $extract = substr($row, index($row,$Data_Pattern) + length($Data_Pattern), length($row));
my $counter_insert = sprintf "%08d", $counter;
my $spaces = " " x index($row,$Data_Pattern);
$data_to_send ="what i need to add" . $extract;
print {$outs} $spaces . $Data_Pattern . $data_to_send;
$counter = $counter + 1;
}
else
{
print {$outs} $row;
next;
}
}
Maybe you could help me with a block of code for my needs, $Data_Pattern is the match. Thank you very much!
This is a classic one-liner Perl task. Basically you would do something like
$ perl -i.bak -pe's/^< COMPRESSED-SIZE >\K(\d+)/$1 + 4/e' yourfile.txt
Which will in essence copy and replace your file with a new, edited file. This can be very dangerous, especially if you are a Perl newbie. The -i switch is here used with the .bak extension which saves a backup in yourfile.txt.bak. This does not make this operation safe, however, as running the command twice will overwrite the backup.
It is advisable to make a separate backup of the target file before using this command.
-i.bak edit "in-place", the file is overwritten, a backup of the original is created with extension .bak.
-p argument is treated as a file name, which is read, and printed back.
s/ // the substitution operator, which is applied to all lines of the file.
^ inside the regex looks for beginning of line.
\K keep the match that is to the left.
(\d+) capture () 1 or more digits \d+ and store them in $1
/e treat the right hand side of the substitution operator as an expression and use the result as the replacement string. In this case it will increase your number and return the sum.
The long version of this command is
while (<>) {
s/^< COMPRESSED-SIZE >\K(\d+)/$1 + 4/e
}
Which can be placed in a file and run with the -i switch.
-- Edit : Resolved. See answer.
Background:
I'm writing a shell that will perform some extra actions required on our system when someone resizes a database.
The shell is written in ksh (requirement), the OS is Solaris 5.10 .
The problem is with one of the checks, which verifies there's enough free space on the underlying OS.
Problem:
The check reads the df -k line for root, which is what I check in this step, and prints it to a file. I then "read" the contents into variables which I use in calculations.
Unfortunately, when I try to run an arithmetic operation on one of the variables, I get an error indicating it is null. And a debug output line I've placed after that line verifies that it is null... It lost it's value...
I've tried every method of doing this I could find online, they work when I run it manually, but not inside the shell file.
(* The file does have #!/usr/bin/ksh)
Code:
df -k | grep "rpool/ROOT" > dftest.out
RPOOL_NAME=""; declare -i TOTAL_SIZE=0; USED_SPACE=0; AVAILABLE_SPACE=0; AVAILABLE_PERCENT=0; RSIGN=""
read RPOOL_NAME TOTAL_SIZE USED_SPACE AVAILABLE_SPACE AVAILABLE_PERCENT RSIGN < dftest.out
\rm dftest.out
echo $RPOOL_NAME $TOTAL_SIZE $USED_SPACE $AVAILABLE_SPACE $AVAILABLE_PERCENT $RSIGN
((TOTAL_SIZE=$TOTAL_SIZE/1024))
This is the result:
DBResize.sh[11]: TOTAL_SIZE=/1024: syntax error
I'm pulling hairs at this point, any help would be appreciated.
The code you posted cannot produce the output you posted. Most obviously, the error is signalled at line 11 but you posted fewer than 11 lines of code. The previous lines may matter. Always post complete code when you ask for help.
More concretely, the declare command doesn't exist in ksh, it's a bash thing. You can achieve the same result with typeset (declare is a bash equivalent to typeset, but not all options are the same). Either you're executing this script with bash, or there's another error message about declare, or you've defined some additional commands including declare which may change the behavior of this code.
None of this should have an impact on the particular problem that you're posting about, however. The variables created by read remain assigned until the end of the subshell, i.e. until the code hits a ), the end of a pipe (left-hand side of the pipe only in ksh), etc.
About the use of declare or typeset, note that you're only declaring TOTAL_SIZE as an integer. For the other variables, you're just assigning a value which happens to consist exclusively of digits. It doesn't matter for the code you posted, but it's probably not what you meant.
One thing that may be happening is that grep matches nothing, and therefore read reads an empty line. You should check for errors. Use set -e in scripts to exit at the first error. (There are cases where set -e doesn't catch errors, but it's a good start.)
Another thing that may be happening is that df is splitting its output onto multiple lines because the first column containing the filesystem name is too large. To prevent this splitting, pass the option -P.
Using a temporary file is fragile: the code may be executed in a read-only directory, another process may want to access the same file at the same time... Here a temporary file is useless. Just pipe directly into read. In ksh (unlike most other sh variants including bash), the right-hand side of a pipe runs in the main shell, so assignments to variables in the right-hand side of a pipe remain available in the following commands.
It doesn't matter in this particular script, but you can use a variable without $ in an arithmetic expression. Using $ substitutes a string which can have confusing results, e.g. a='1+2'; $((a*3)) expands to 7. Not using $ uses the numerical value (in ksh, a='1+2'; $((a*3)) expands to 9; in some sh implementations you get an error because a's value is not numeric).
#!/usr/bin/ksh
set -e
typeset -i TOTAL_SIZE=0 USED_SPACE=0 AVAILABLE_SPACE=0 AVAILABLE_PERCENT=0
df -Pk | grep "rpool/ROOT" | read RPOOL_NAME TOTAL_SIZE USED_SPACE AVAILABLE_SPACE AVAILABLE_PERCENT RSIGN
echo $RPOOL_NAME $TOTAL_SIZE $USED_SPACE $AVAILABLE_SPACE $AVAILABLE_PERCENT $RSIGN
((TOTAL_SIZE=TOTAL_SIZE/1024))
Strange...when I get rid of your "declare" line, your original code seems to work perfectly well (at least with ksh on Linux)
The code :
#!/bin/ksh
df -k | grep "/home" > dftest.out
read RPOOL_NAME TOTAL_SIZE USED_SPACE AVAILABLE_SPACE AVAILABLE_PERCENT RSIGN < dftest.out
\rm dftest.out
echo $RPOOL_NAME $TOTAL_SIZE $USED_SPACE $AVAILABLE_SPACE $AVAILABLE_PERCENT $RSIGN
((TOTAL_SIZE=$TOTAL_SIZE/1024))
print $TOTAL_SIZE
The result :
32962416 5732492 25552588 19% /home
5598
Which are the value a simple df -k is returning. The variables seem to last.
For those interested, I have figured out that it is not possible to use "read" the way I was using it.
The variable values assigned by "read" simply "do not last".
To remedy this, I have applied the less than ideal solution of using the standard "while read" format, and inside the loop, echo selected variables into a variable file.
Once said file was created, I just "loaded" it.
(pseudo code:)
LOOP START
echo "VAR_A="$VAR_A"; VAR_B="$VAR_B";" > somefile.out
LOOP END
. somefile.out
I wrote a program in Java that accepts input via command line arguments.
I get an input of two numbers and an operator from the command line.
To multiply two numbers, I have to give input as e.g. 5 3 *, but it's not working as written.
Why is it not accepting * from the command line?
That's because * is a shell wildcard: it has a special meaning to the shell, which expands it before passing it on to the command (in this case, java).
Since you need a literal *, you need to escape it from the shell. The exact way of escaping varies depending on your shell, but you can try:
java ProgramName 5 3 "*"
Or:
java ProgramName 5 3 \*
By the way, if you want to know what the shell does with the *, try printing the content of String[] args to your main method. You'll find that it will contain names of the files in your directory.
This can be handy if you need to pass some filenames as command line arguments.
See also
Wikipedia: glob
For example, if a directory contains two files, a.log and b.log then the command cat *.log will be expanded by the shell to cat a.log b.log
Wikipedia: Escape character
In Bourne shell (sh), the asterisk (*) and question mark (?) characters are wildcard characters expanded via globbing. Without a preceding escape character, an * will expand to the names of all files in the working directory that don't start with a period if and only if there are such files, otherwise * remains unexpanded. So to refer to a file literally called "*", the shell must be told not to interpret it in this way, by preceding it with a backslash (\).
Under MS WINDOWS not quite true: "java.exe" silently expands command line arguments with the wildcards
*
?
[abc]
, but only in the last component, so
a/*/*
does not work as you may expect.
It also ignores the entries "." and "..", but does honor other file names starting with ".".
To avoid misunderstandings: If I look at the command line of the running JAVA process with PROCEXP, I see the unexpanded args!
I found no way to work around this. In other words: As long as you have at least one file or directory in the current directory, "java Calc 3 * 7" will NOT work!
This is VERY ugly, and seems to always having been there in all JRE versions up to and including Java 8.
Does anybody have an idea how to disable Java's nasty command line expansion?
* has special meaning in shell interpreters. How to get a * literally is depending on what shell interpreter you are using. For Bash, you should put single quotes around the *, i.e. '*', instead of double quotes like "*".
Try surrounding the * with quotes like "*". The star is a reserved symbol on the command line.
Use single quotes:
java FooBar 5 3 '*'
This works with most of the popular shells (including bash and ksh).
Expanding on #Arno Unkrig's answer:
On Windows, some JVMs definitely do expand the "*" character, and it is not the shell expanding the path. You can confirm this by writing a small Java program that prints out the arguments:
public class TestArgs {
public static void main(String[] args) {
for (int i = 0; i < args.length; i++) {
System.out.println("Arg " + i + ": " + args[i]);
}
}
}
The good news is, there is a workaround! You can use #filename as an argument to JVM like this:
java #args.txt where args.txt is a text file that contains the arguments for each line. Example content:
TestArgs
*
This is equivalent to calling java with two arguments TestArgs and *. Most importantly, * is not expanded when it is included using the #filename method. I was able to find the details from this page.
GCC version 4.6
The Problem: To find a way to feed in parameters to the executable, say a.out, from the command line - more specifically feed in an array of double precision numbers.
Attempt: Using the READ(*,*) command, which is older in the standard:
Program test.f -
PROGRAM MAIN
REAL(8) :: A,B
READ(*,*) A,B
PRINT*, A+B, COMMAND_ARGUMENT_COUNT()
END PROGRAM MAIN
The execution -
$ gfortran test.f
$ ./a.out 3.D0 1.D0
This did not work. On a bit of soul-searching, found that
$./a.out
3.d0,1.d0
4.0000000000000000 0
does work, but the second line is an input prompt, and the objective of getting this done in one-line is not achieved. Also the COMMAND_ARGUMENT_COUNT() shows that the numbers fed into the input prompt don't really count as 'command line arguments', unlike PERL.
If you want to get the arguments fed to your program on the command line, use the (since Fortran 2003) standard intrinsic subroutine GET_COMMAND_ARGUMENT. Something like this might work
PROGRAM MAIN
REAL(8) :: A,B
integer :: num_args, ix
character(len=12), dimension(:), allocatable :: args
num_args = command_argument_count()
allocate(args(num_args)) ! I've omitted checking the return status of the allocation
do ix = 1, num_args
call get_command_argument(ix,args(ix))
! now parse the argument as you wish
end do
PRINT*, A+B, COMMAND_ARGUMENT_COUNT()
END PROGRAM MAIN
Note:
The second argument to the subroutine get_command_argument is a character variable which you'll have to parse to turn into a real (or whatever). Note also that I've allowed only 12 characters in each element of the args array, you may want to fiddle around with that.
As you've already figured out read isn't used for reading command line arguments in Fortran programs.
Since you want to read an array of real numbers, you might be better off using the approach you've already figured out, that is reading them from the terminal after the program has started, it's up to you.
The easiest way is to use a library. There is FLAP or f90getopt available. Both are open source and licensed under free licenses.
The latter is written by Mark Gates and me, just one module and can be learned in minutes but contains all what is needed to parse GNU- and POSIX-like command-line options. The first is more sophisticated and can be used even in closed-source projects. Check them out.
Furthermore libraries at https://fortranwiki.org/fortran/show/Command-line+arguments
What READ (*,*) does is that it reads from the standard input. For example, the characters entered using the keyboard.
As the question shows COMMAND_ARGUMENT_COUNT() can be used to get the number of the command line arguments.
The accepted answer by High Performance Mark show how to retrieve the individual command line arguments separated by blanks as individual character strings using GET_COMMAND_ARGUMENT(). One can also get the whole command line using GET_COMMAND(). One then has to somehow parse that character-based information into the data in your program.
I very simple cases you just need the program requires, for example, two numbers, so you read one number from arg 1 and another form arg 2. That is simple. Or you can read a triplet of numbers from a single argument if they are comma-separated like 1,2,3 using a simple read(arg,*) nums(1:3).
For general complicated command line parsing one uses libraries such as those mentioned in the answer by Hani. You have set them up so that the library knows the expected syntax of the command line arguments and the data it should fill with the values.
There is a middle ground, that is still relatively simple, but one already have multiple arguments, that correspond to Fortran variables in the program, that may or may not be present. In that case one can use the namelist for the syntax and for the parsing.
Here is an example, the man point is the namelist /cmd/ name, point, flag:
implicit none
real :: point(3)
logical :: flag
character(256) :: name
character(1024) :: command_line
call read_command_line
call parse_command_line
print *, point
print *, "'",trim(name),"'"
print *, flag
contains
subroutine read_command_line
integer :: exenamelength
integer :: io, io2
command_line = ""
call get_command(command = command_line,status = io)
if (io==0) then
call get_command_argument(0,length = exenamelength,status = io2)
if (io2==0) then
command_line = "&cmd "//adjustl(trim(command_line(exenamelength+1:)))//" /"
else
command_line = "&cmd "//adjustl(trim(command_line))//" /"
end if
else
write(*,*) io,"Error getting command line."
end if
end subroutine
subroutine parse_command_line
character(256) :: msg
namelist /cmd/ name, point, flag
integer :: io
if (len_trim(command_line)>0) then
msg = ''
read(command_line,nml = cmd,iostat = io,iomsg = msg)
if (io/=0) then
error stop "Error parsing the command line or cmd.conf " // msg
end if
end if
end subroutine
end
Usage in bash:
> ./command flag=T name=\"data.txt\" point=1.0,2.0,3.0
1.00000000 2.00000000 3.00000000
'data.txt'
T
or
> ./command flag=T name='"data.txt"' point=1.0,2.0,3.0
1.00000000 2.00000000 3.00000000
'data.txt'
T
Escaping the quotes for the string is unfortunately necessary, because bash eats the first quotes.
I got this strange line of code today, it tells me 'empty' or 'not empty' depending on whether the CWD has any items (other than . and ..) in it.
I want to know how it works because it makes no sense to me.
perl -le 'print+(q=not =)[2==(()=<.* *>)].empty'
The bit I am interested in is <.* *>. I don't understand how it gets the names of all the files in the directory.
It's a golfed one-liner. The -e flag means to execute the rest of the command line as the program. The -l enables automatic line-end processing.
The <.* *> portion is a glob containing two patterns to expand: .* and *.
This portion
(q=not =)
is a list containing a single value -- the string "not". The q=...= is an alternate string delimiter, apparently used because the single-quote is being used to quote the one-liner.
The [...] portion is the subscript into that list. The value of the subscript will be either 0 (the value "not ") or 1 (nothing, which prints as the empty string) depending on the result of this comparison:
2 == (()=<.* *>)
There's a lot happening here. The comparison tests whether or not the glob returned a list of exactly two items (assumed to be . and ..) but how it does that is tricky. The inner parentheses denote an empty list. Assigning to this list puts the glob in list context so that it returns all the files in the directory. (In scalar context it would behave like an iterator and return only one at a time.) The assignment itself is evaluated in scalar context (being on the right hand side of the comparison) and therefore returns the number of elements assigned.
The leading + is to prevent Perl from parsing the list as arguments to print. The trailing .empty concatenates the string "empty" to whatever came out of the list (i.e. either "not " or the empty string).
<.* *>
is a glob consisting of two patterns: .* are all file names that start with . and * corresponds to all files (this is different than the usual DOS/Windows conventions).
(()=<.* *>)
evaluates the glob in list context, returning all the file names that match.
Then, the comparison with 2 puts it into scalar context so 2 is compared to the number of files returned. If that number is 2, then the only directory entries are . and .., period. ;-)
<.* *> means (glob(".*"), glob("*")). glob expands file patterns the same way the shell does.
I find that the B::Deparse module helps quite a bit in deciphering some stuff that throws off most programmers' eyes, such as the q=...= construct:
$ perl -MO=Deparse,-p,-q,-sC 2>/dev/null << EOF
> print+(q=not =)[2==(()=<.* *>)].empty
> EOF
use File::Glob ();
print((('not ')[(2 == (() = glob('.* *')))] . 'empty'));
Of course, this doesn't instantly produce "readable" code, but it surely converts some of the stumbling blocks.
The documentation for that feature is here. (Scroll near the end of the section)