I can track the rsync progress via Whiptail, using Awk to parse the rsync output, however, I'm puzzled by why the Perl counterpart doesn't work (the Whiptail gauge stays stuck at 0).
This is the working Awk commandline:
rsync --info=progress2 --no-inc-recursive --human-readable <source> <destination> |
stdbuf -o0 awk -v RS='\r' '$2 ~ /%$/ { print substr($2, 0, length($2) - 1) }' |
whiptail --gauge Syncing 20 80 0
This is the Perl (I assume) equivalent:
rsync --info=progress2 --no-inc-recursive --human-readable <source> <destination> |
stdbuf -o0 perl -lne 'BEGIN { $/ = "\r" } print /(\d+)%/' |
whiptail --gauge Syncing 20 80 0
If I remove the Whiptail command from the Perl version, the percentage numbers are printed as expected.
How do I need to modify the Perl version?
You may be suffering from buffering. Try setting autoflush on STDOUT.
BEGIN { $/ = "\r"; $|++ }
or if Perl is at least version 5.14, or otherwise with adding the -MIO::Handle switch, you can be more explicit:
BEGIN { $/ = "\r"; *STDOUT->autoflush }
Related
I want to patch a binary file with perl. The command doesn't work today, but in the past I used it a lot .
The command below doesn't work on Mac Os X:
perl -pi -e 's|\xA0\x37\x96\x30\xDE\x90|\xA7\x70\x92\x30\xD5\x9B|' /file.bin
If I use
perl -MO=Deparse -pi -e 's|\xA0\x37\x96\x30\xDE\x90|\xA7\x70\x92\x30\xD5\x9B|' /file.bin
the result is:
BEGIN { $^I = ""; }
LINE: while (defined($_ = <ARGV>)) {
s/\xA0\x37\x96\x30\xDE\x90/\247p\2220\325\233/;
}
continue {
die "-p destination: $!\n" unless print $_;
}
-e syntax OK
Why the replace section was modified like this?
I checked 1000 time the syntax is correct; why it doesn't work as expected?
It wasn't modified.
"\xA7\x70\x92\x30\xD5\x9B"
and
"\247p\2220\325\233"
are equivalent.
$ perl -e'CORE::say "\xA7\x70\x92\x30\xD5\x9B" eq "\247p\2220\325\233" ? "same" : "diff"'
same
2478 = A716
p's ASCII encoding is 7016
2228 = 9216
0's ASCII encoding is 3016
3258 = D516
2338 = 9B16
I have the following program snippet
my $nfdump_command = "nfdump -M /data/nfsen/profiles-data/live/upstream1 -T -R ${syear}/${smonth}/${sday}/nfcapd.${syear}${smonth}${sday}0000:${eyear}/${emonth}/${eday}/nfcapd.${eyear}${emonth}${eday}2355 -n 100 -s ip/bytes -N -o csv -q | awk 'BEGIN { FS = \",\" } ; { if (NR > 1) print \$5, \$10 }'";
syslog("info", $nfdump_command);
my %args;
Nfcomm::socket_send_ok ($socket, \%args);
my #nfdump_output = `$nfdump_command`;
my %domain_name_to_bytes;
my %domain_name_to_ip_addresses;
syslog("info", Dumper(\#nfdump_output));
foreach my $a_line (#nfdump_output) {
syslog("info", "LINE: " . $a_line);
}
Bug: #nfdump_output is empty.
The $nfdump_command is correct and it printing output when ran individually
This program was working for sometime and then it broke. Couldn't figure out why. After moving my development setup to another virtual machine, I found out that using absolute path to nfdump fixes it
I need to add a timestamp in front of the output of a long-executing command (a "tcpdump", in my use-case...).
It - very simplified - looks like this one:
(echo A1; sleep 3; echo B2) | perl -MPOSIX -pe 'print strftime "%T ", localtime $^T; s/\d//'
which gives this kind of output:
16:10:24 A
16:10:24 B
i.e.: perl's localtime is (obviously) called when perl is invoked.
Instead I need this kind of result:
16:10:24 A
16:10:27 B
i.e.: time stamp should be relative to the input's generation time...
Any smart (or no so smart :-) solution?
Just remove the $^T from your Perl command. That way, you will use the current time instead of the process start time. See the docs for $^T.
However, a more elegant formulation with Perl would be:
... | perl -MPOSIX -ne's/\d//; print strftime("%T ", localtime), $_'
You could pipe the output to:
awk '{ print strftime("%T"), $0; }'
Example:
while : ; do echo hey; sleep 1; done | awk '{ print strftime("%T"), $0; }'
20:49:58 hey
20:49:59 hey
20:50:00 hey
20:50:01 hey
20:50:02 hey
20:50:03 hey
20:50:04 hey
20:50:05 hey
Alternatively, you could use ts:
ts '%T'
(echo A1; sleep 3; echo B2) | perl -MPOSIX -pe 'print strftime "%T ", localtime; s/\d//'
Works excellent for me. Why you added $^T there?
on Linux, you can use tcpdump -tttt to print the timestamp before each output line.
# tcpdump -tttt -c 1 2>/dev/null
2013-12-13 23:42:12.044426 IP 10.0.2.15.ssh > 10.0.2.2.53466: Flags [P.], seq 464388005:464388121, ack 16648998, win 65535, length 116
If your tcpdump doesn't have the -tttt option, you should use awk from devnull
Say I have a file like so:
+jaklfjdskalfjkdsaj
fkldsjafkljdkaljfsd
-jslakflkdsalfkdls;
+sdjafkdjsakfjdskal
I only want to find and count the amount of times during this file a line that starts with - is immediately followed by a line that starts with +.
Rules:
No external scripts
Must be done from within a bash script
Must be inline
I could figure out how to do this in a Python script, for instance, but I've never had to do something this extensive in Bash.
Could anyone help me out? I figure it'll end up being grep, perl, or maybe a talented sed line -- but these are things I'm still learning.
Thank you all!
grep -A1 "^-" $file | grep "^+" | wc -l
The first grep finds all of the lines starting with -, and the -A1 causes it to also output the line after the match too.
We then grep that output for any lines starting with +. Logically:
We know the output of the first grep is only the -XXX lines and the following lines
We know that a +xxx line cannot also be a -xxx line
Therefore, any +xxx lines must be following lines, and should be counted, which we do with wc -l
Easy in Perl:
perl -lne '$c++ if $p and /^\+/; $p = /^-/ }{ print $c' FILE
awk one-liner:
awk -v FS='' '{x=x sprintf("%s", $1)}END{print gsub(/-\+/,"",x)}' file
e.g.
kent$ cat file
+jaklfjdskalfjkdsaj
fkldsjafkljdkaljfsd
-jslakflkdsalfkdls;
+sdjafkdjsakfjdskal
-
-
-
+
-
+
foo
+
kent$ awk -v FS='' '{x=x sprintf("%s", $1)}END{print gsub(/-\+/,"",x)}' file
3
Another Perl example. Not as terse as choroba's, but more transparent in how it works:
perl -e'while (<>) { $last = $cur; $cur = $_; print $last, $cur if substr($last, 0, 1) eq "-" && substr($cur, 0, 1) eq "+" }' < infile
Output:
-jslakflkdsalfkdls;
+sdjafkdjsakfjdskal
Pure bash:
unset c p
while read line ; do
[[ $line == +* && $p == 0 ]] && (( c++ ))
[[ $line == -* ]]
p=$?
done < FILE
echo $c
In awk I can write: awk -F: 'BEGIN {OFS = FS} ...'
In Perl, what's the equivalent of FS? I'd like to write
perl -F: -lane 'BEGIN {$, = [what?]} ...'
update with an example:
echo a:b:c:d | awk -F: 'BEGIN {OFS = FS} {$2 = 42; print}'
echo a:b:c:d | perl -F: -ane 'BEGIN {$, = ":"} $F[1] = 42; print #F'
Both output a:42:c:d
I would prefer not to hard-code the : in the Perl BEGIN block, but refer to wherever the -F option saves its argument.
To sum up, what I'm looking for does not exist:
there's no variable that holds the argument for -F, and more importantly
Perl's "FS" is fundamentally a different data type (regular expression) than the "OFS" (string) -- it does not make sense to join a list of strings using a regex.
Note that the same holds true in awk: FS is a string but acts as regex:
echo a:b,c:d | awk -F'[:,]' 'BEGIN {OFS=FS} {$2=42; print}'
outputs "a[:,]42[:,]c[:,]d"
Thanks for the insight and workarounds though.
You can use perl's -s (similar to awk's -v) to pass a "FS" variable, but the split becomes manual:
echo a:b:c:d | perl -sne '
BEGIN {$, = $FS}
#F = split $FS;
$F[1] = 42;
print #F;
' -- -FS=":"
If you know the exact length of input, you could do this:
echo a:b:c:d | perl -F'(:)' -ane '$, = $F[1]; #F = #F[0,2,4,6]; $F[1] = 42; print #F'
If the input is of variable lengths, you'll need something more sophisticated than #f[0,2,4,6].
EDIT: -F seems to simply provide input to an automatic split() call, which takes a complete RE as an expression. You may be able to find something more suitable by reading the perldoc entries for split, perlre, and perlvar.
You can sort of cheat it, because perl is actually using the split function with your -F argument, and you can tell split to preserve what it splits on by including capturing parens in the regex:
$ echo a:b:c:d | perl -F'(:)' -ane 'print join("/", #F);'
a/:/b/:/c/:/d
You can see what perl's doing with some of these "magic" command-line arguments by using -MO=Deparse, like this:
$ perl -MO=Deparse -F'(:)' -ane 'print join("/", #F);'
LINE: while (defined($_ = <ARGV>)) {
our(#F) = split(/(:)/, $_, 0);
print join('/', #F);
}
-e syntax OK
You'd have to change your #F subscripts to double what they'd normally be ($F[2] = 42).
Darnit...
The best I can do is:
echo a:b:c:d | perl -ne '$v=":";#F = split("$v"); $F[1] = 42; print join("$v", #F) . "\n";'
You don't need the -F: this way, and you're only stating the colon once. I was hoping there was someway of setting variables on the command line like you can with Awk's -v switch.
For one liners, Perl is usually not as clean as Awk, but I remember using Awk before I knew of Perl and writing 1000+ line Awk scripts.
Trying things like this made people think Awk was either named after the sound someone made when they tried to decipher such a script, or stood for AWKward.
There is no input record separator in Perl. You're basically emulating awk by using the -a and -F flags. If you really don't want to hard code the value, then why not just use an environmental variable?
$ export SPLIT=":"
$ perl -F$SPLIT -lane 'BEGIN { $, = $ENV{SPLIT}; } ...'