In a Perl script, I want to execute a system command and write the output in the console. Here is a snippet who reproduce the behavior of my script:
use strict;
use warnings FATAL => 'all';
sub safe_run_cmd {
my ($cmd) = #_;
my $pid;
my $sleep_count;
my $fcmd;
do {
$pid = open($fcmd, "$cmd 2>&1 |");
unless(defined $pid) {
warn("Cannot fork: $!\n");
die("Bailing out\n") if $sleep_count++ > 6;
sleep(1);
}
} until (defined $pid);
if($pid) {
while ( my $line = <$fcmd> ) {
print $line;
}
close $fcmd;
} else {
exit(0);
}
print("End safe_run_cmd\n");
}
eval{safe_run_cmd("bad_command")};
print(`ps aux | egrep open`);
print("-----\n");
eval{safe_run_cmd("echo good_command")};
print(`ps aux | egrep open`);
I called the function safe because I follow what is described in the documentation.
If I run my script, I get this:
pierre 146161 21.0 0.0 21916 4548 pts/1 S+ 14:32 0:00 perl open.pl
pierre 146163 0.0 0.0 21916 2816 pts/1 S+ 14:32 0:00 perl open.pl
pierre 146164 0.0 0.0 4320 756 pts/1 S+ 14:32 0:00 sh -c ps aux | egrep open
pierre 146166 0.0 0.0 12752 1008 pts/1 S+ 14:32 0:00 grep -E open
-----
good_command
End safe_run_cmd
pierre 146161 10.5 0.0 21916 4548 pts/1 S+ 14:32 0:00 perl open.pl
pierre 146163 0.0 0.0 21916 3516 pts/1 S+ 14:32 0:00 perl open.pl
pierre 146168 0.0 0.0 4320 756 pts/1 S+ 14:32 0:00 sh -c ps aux | egrep open
pierre 146170 0.0 0.0 12752 996 pts/1 S+ 14:32 0:00 grep -E open
End safe_run_cmd
pierre 146161 10.5 0.0 21916 4744 pts/1 S+ 14:32 0:00 perl open.pl
pierre 146171 0.0 0.0 4320 708 pts/1 S+ 14:32 0:00 sh -c ps aux | egrep open
pierre 146173 0.0 0.0 12752 1008 pts/1 S+ 14:32 0:00 grep -E open
-----
good_command
End safe_run_cmd
pierre 146161 10.5 0.0 21916 4744 pts/1 S+ 14:32 0:00 perl open.pl
pierre 146175 0.0 0.0 4320 788 pts/1 S+ 14:32 0:00 sh -c ps aux | egrep open
pierre 146177 0.0 0.0 12752 1012 pts/1 S+ 14:32 0:00 grep -E open
We can see, when I print the list of the processes after running the bad command, I have two perl forks. When the first one ends, the second one continue from the call of open. But, when the command is right, open doesn't fork.
What can I do to avoid this fork (or to manage it) and to display an error message when the command is bad?
The use of warnings FATAL => 'all' has a side-effect on the open function. Indeed, if open gets a warning, it immediately dies. So, if I remove it from the code, I get a correct output:
Cannot fork: No such file or directory
Cannot fork: No such file or directory
Cannot fork: No such file or directory
Cannot fork: No such file or directory
Cannot fork: No such file or directory
Cannot fork: No such file or directory
Cannot fork: No such file or directory
Cannot fork: No such file or directory
pierre 207725 2.3 0.0 21644 4432 pts/1 S+ 15:28 0:00 perl open.pl
pierre 207750 0.0 0.0 4320 816 pts/1 S+ 15:28 0:00 sh -c ps aux | egrep open
pierre 207752 0.0 0.0 12752 984 pts/1 S+ 15:28 0:00 grep -E open
-----
good_command
End safe_run_cmd
pierre 207725 2.3 0.0 21644 4448 pts/1 S+ 15:28 0:00 perl open.pl
pierre 207754 0.0 0.0 4320 748 pts/1 S+ 15:28 0:00 sh -c ps aux | egrep open
pierre 207756 0.0 0.0 12752 996 pts/1 S+ 15:28 0:00 grep -E open
To automaticly die if the command doesn't exist, it is possible to use autodie instead of the do block:
sub safe_run_cmd {
my ($cmd) = #_;
my $pid;
my $sleep_count;
my $fcmd;
use autodie;
$pid = open($fcmd, "$cmd 2>&1 |");
while ( my $line = <$fcmd> ) {
print $line;
}
close $fcmd;
print("End safe_run_cmd\n");
}
I get:
pierre 211968 11.5 0.0 26544 7244 pts/1 S+ 15:32 0:00 perl open.pl
pierre 211971 0.0 0.0 4320 768 pts/1 S+ 15:32 0:00 sh -c ps aux | egrep open
pierre 211973 0.0 0.0 12752 1064 pts/1 S+ 15:32 0:00 grep -E open
-----
good_command
End safe_run_cmd
pierre 211968 11.5 0.0 26544 7264 pts/1 S+ 15:32 0:00 perl open.pl
pierre 211975 0.0 0.0 4320 792 pts/1 S+ 15:32 0:00 sh -c ps aux | egrep open
pierre 211977 0.0 0.0 12752 1032 pts/1 S+ 15:32 0:00 grep -E open
Related
I use sed -e '$s/.$//' to trim the last character of a stream. Is it the correct way to do so? Are there other better ways to do so with other command line tools?
$ builtin printf 'a\nb\0' | sed -e '$s/.$//' | od -c -t x1 -Ax
000000 a \n b
61 0a 62
000003
EDIT: It seems that this command is not robust. The expected output is a\nb for the following example. Better methods (but not too verbose) are needed.
$ builtin printf 'a\nb\n' | sed -e '$s/.$//' | od -c -t x1 -Ax
000000 a \n \n
61 0a 0a
000003
You may use head -c -1:
printf 'a\nb\0' | head -c -1 | od -c -t x1 -Ax
000000 a \n b
61 0a 62
000003
printf 'a\nb\n' | head -c -1 | od -c -t x1 -Ax
000000 a \n b
61 0a 62
000003
It seems you can't rely on any line-oriented tools (like sed) that automatically remove and re-add newlines.
Perl can slurp the whole stream into a string and can remove the last char:
$ printf 'a\nb\0' | perl -0777 -pe chop | od -c -t x1 -Ax
000000 a \n b
61 0a 62
000003
$ printf 'a\nb\n' | perl -0777 -pe chop | od -c -t x1 -Ax
000000 a \n b
61 0a 62
000003
The tradeoff is that you need to hold the entire stream in memory.
I want to invoke the "find" command inside myscript, using the arguments that I pass to myscript. But if I use wildcard characters with the "-name" argument and any file happens to match the string, the string is expanded to all the filenames, no matter what kind of "quotes" I 'use'.
In the shell, I want globbing, but not when I pass arguments to this script. Example:
~/u/tmp/JNK> ls -latTr
total 32
drwxr-xr-x 160 BNW staff 5440 Jan 27 18:47:37 2018 ../
-rw-r--r-- 1 BNW staff 0 Jan 27 19:30:10 2018 Henry-James.txt
-rw-r--r-- 1 BNW staff 0 Jan 27 19:30:17 2018 Emily-Dickinson.txt
-rw-r--r-- 1 BNW staff 0 Jan 27 19:30:21 2018 for-Henry-James.txt
-rwxr-xr-x 1 BNW staff 97 Jan 27 19:31:55 2018 myscript*
-rw-r--r--# 1 BNW staff 6148 Jan 27 19:43:15 2018 .DS_Store
drwxr-xr-x 2 BNW staff 68 Jan 27 19:44:51 2018 DIRECTORY/
lrwxr-xr-x 1 BNW staff 19 Jan 27 19:45:09 2018 softlink# -> for-Henry-James.txt
drwxr-xr-x 9 BNW staff 306 Jan 27 19:45:09 2018 ./
~/u/tmp/JNK> find . -type f
./.DS_Store
./Emily-Dickinson.txt
./for-Henry-James.txt
./Henry-James.txt
./myscript
~/u/tmp/JNK> find . -type f -name "*James*"
./for-Henry-James.txt
./Henry-James.txt
~/u/tmp/JNK> cat myscript
#!/bin/bash
for arg in $#
do
printf '%s' "arg="
printf '%s' "$arg"
printf '\n'
done
find . $#
~/u/tmp/JNK> ./myscript -type f
arg=-type
arg=f
./.DS_Store
./Emily-Dickinson.txt
./for-Henry-James.txt
./Henry-James.txt
./myscript
~/u/tmp/JNK> ./myscript -type f -name "*James*"
arg=-type
arg=f
arg=-name
arg=Henry-James.txt
arg=for-Henry-James.txt
find: for-Henry-James.txt: unknown primary or operator
~/u/tmp/JNK> ./myscript -type f -name '*James*'
arg=-type
arg=f
arg=-name
arg=Henry-James.txt
arg=for-Henry-James.txt
find: for-Henry-James.txt: unknown primary or operator
~/u/tmp/JNK>
Is there an elegant solution?
The solution appears to be to add one line
set -o noglob
to the script. Now we have
#!/bin/bash
set -o noglob
for arg in $#
do
printf '%s' "arg="
printf '%s' "$arg"
printf '\n'
done
find . $#
and
~/u/tmp/JNK> ./myscript -name "*Jam*"
arg=-name
arg=*Jam*
./for-Henry-James.txt
./Henry-James.txt
~/u/tmp/JNK>
Can anybody help me please?
grep " 287 " file.txt | grep "HI" | sed -i 's/HIS/HID/g'
sed: no input files
Tried also xargs
grep " 287 " file.txt | grep HI | xargs sed -i 's/HIS/HID/g'
sed: invalid option -- '6'
This works fine
grep " 287 " file.txt | grep HI
If you want to keep your pipeline:
f=file.txt
tmp=$(mktemp)
grep " 287 " "$f" | grep "HI" | sed 's/HIS/HID/g' > "$tmp" && mv "$tmp" "$f"
Or, simplify:
sed -i -n '/ 287 / {/HI/ s/HIS/HID/p}' file.txt
That will filter out any line that does not contain " 287 " and "HI" -- is that what you want? I suspect you really want this:
sed -i '/ 287 / {/HI/ s/HIS/HID/}' file.txt
For lines that match / 287 /, execute the commands in braces. In there, for lines that match /HI/, search for the first "HIS" and replace with "HID". sed implicitly prints all lines if -n is not specified.
Other commands that do the same thing:
awk '/ 287 / && /HI/ {sub(/HIS/, "HID")} {print}' file.txt > new.txt
perl -i -pe '/ 287 / and /HI/ and s/HIS/HID/' file.txt
awk does not have an "in-place" option (except gawk -i inplace for recent gawk versions)
Considering this value:
my $value = hex('0x12345678');
And I would like my hexdump to be like this (same bits order):
0000000 1234 5678
I used this method but it mixes up my value:
open(my $out, '>:raw', 'foo') or die "Unable to open: $!";
print $out pack('l', $value); # Test in little endian
print $out pack('l>', $value); # Test in big endian
Here's what I get:
0000000 5678 1234 3412 7856
How can I get the bits in order?
EDIT
So the problem might come from my hexdump, because I get the same output with the suggested answer.
$ perl -e 'print pack $_, 0x12345678 for qw( l> N )' | hexdump
0000000 3412 7856 3412 7856
I got the correct result with hexdump -C:
$ perl -e 'print pack $_, 0x12345678 for qw( l> N )' | hexdump -C
00000000 12 34 56 78 12 34 56 78 |.4Vx.4Vx|
And I found the explanation here:
hexdump confusion
The 'l>' option works for me (note there's no call to hex, though). Also, N as the template works:
perl -e 'print pack $_, 0x12345678 for qw( l> N )' | xxd
0000000: 1234 5678 1234 5678
Before:
eng-vshakya:scripts vshakya$ ls
American Samoa.png Faroe Islands.png Saint Barthelemy.png
After:
eng-vshakya:scripts vshakya$ ls
AmericanSamoa.png FaroeIslands.png SaintBarthelemy.png
Tried below prototype, but it does not work :( Sorry, not very good when it comes to awk/sed :(
ls *.png | sed 's/\ /\\\ /g' | awk '{print("mv "$1" "$1)}'
[ Above is prototype, real command, I guess, would be:
ls *.png | sed 's/\ /\\\ /g' | awk '{print("mv "$1" "$1)}' | sed 's/\ //g'
]
No need to use awk or sed when you can do this in pure bash.
[ghoti#pc ~/tmp1]$ ls -l
total 2
-rw-r--r-- 1 ghoti wheel 0 Aug 1 01:19 American Samoa.png
-rw-r--r-- 1 ghoti wheel 0 Aug 1 01:19 Faroe Islands.png
-rw-r--r-- 1 ghoti wheel 0 Aug 1 01:19 Saint Barthelemy.png
[ghoti#pc ~/tmp1]$ for name in *\ *; do mv -v "$name" "${name// /}"; done
American Samoa.png -> AmericanSamoa.png
Faroe Islands.png -> FaroeIslands.png
Saint Barthelemy.png -> SaintBarthelemy.png
[ghoti#pc ~/tmp1]$
Note that the ${foo/ /} notation is bash, and does not work in classic Bourne shell.
ghoti's solution is the right thing to do. Since you ask how to do it in sed, here's one way:
for file in *; do newfile=$( echo "$file" | tr -d \\n | sed 's/ //g' );
test "$file" != "$newfile" && mv "$file" "$newfile"; done
The tr is there to remove newlines in the filename, and is necessary to ensure that sed sees the entire filename in one line.