svn diff through perltidy - perl

I want to run perltidy before i look for diff in my subversion working copy.
in svn config i wrote:
diff-cmd = /usr/bin/d.sh
As David W said in this answer https://stackoverflow.com/a/5834900/1927848 i make a script /usr/bin/d.sh:
#!/usr/local/bin/bash
/usr/local/bin/perltidy "$1" > "/tmp/$1"
/usr/local/bin/perltidy "$2" > "/tmp/$2"
/usr/bin/diff "$1" "$2"
/bin/rm "/tmp/$1" "/tmp/$2"
exit 0
and when i make svn diff in working copy i got some errors:
dev# svn diff
Index: nodeny/new_month.pl
===================================================================
Unknown option: u
Error on command line; for help try 'perltidy -h'
Option l is ambiguous (libpods, line-up-parentheses, logfile, logfile-gap, long-block-line-count, look-for-autoloader, look-for-hash-bang, look-for-selfloader)
Error on command line; for help try 'perltidy -h'
diff: option requires an argument -- L
/usr/bin/diff: Try `/usr/bin/diff --help' for more information.
where is my errors?
UPD: $1 and $2 are not file names, $6 and $7 contains file names. i made some modifications to code, thanks to ikegami comment
#!/usr/local/bin/bash
/usr/local/bin/perltidy "$6" -st > "/tmp/tidy001"
/usr/local/bin/perltidy "$7" -st > "/tmp/tidy002"
/usr/bin/diff "/tmp/tidy001" "/tmp/tidy002"
/bin/rm "/tmp/tidy001" "/tmp/tidy002"
exit 0
but now script only does first perltidy command and wait... whats wrong?
UPD2: perl script, that works:
#!/usr/bin/perl
use Text::Diff;
if (-e $ARGV[-2] && -e $ARGV[-1]) {
my $str1 = `/usr/local/bin/perltidy -npro -pbp -nst -se -et=4 -bar -l=200 $ARGV[-2] -st`;
my $str2 = `/usr/local/bin/perltidy -npro -pbp -nst -se -et=4 -bar -l=200 $ARGV[-1] -st`;
my $diff = diff(\$str1, \$str2);
print $diff;
}
else {
print "Error file $ARGV[-2] or $ARGV[-1] not exists\n";
}
exit 0;

I'm not an experienced bash code, so the following may not be optimal, especially given the redundancy, but it solves your problem by assuming the last two args are the file names.
#!/bin/bash
args=("$#")
f1_idx=$(( ${#args[#]} - 2 ))
f1="${args[$f1_idx]}"
/usr/local/bin/perltidy "$f1" -st > "/tmp/$f1"
args[$f1_idx]="/tmp/$f1"
f2_idx=$(( ${#args[#]} - 1 ))
f2="${args[$f2_idx]}"
/usr/local/bin/perltidy "$f2" -st > "/tmp/$f2"
args[$f2_idx]="/tmp/$f2"
/usr/bin/diff "$args[#]"
/bin/rm "/tmp/$f1" "/tmp/$f2"
exit 0
Or if you don't care about the actual file names (as your update implies), you can avoid the temporary files altogether.
#!/bin/bash
args=("$#")
last_idx=$(( ${#args[#]} - 1 ))
f2="${args[$last_idx]}"
unset args[$last_idx]
last_idx=$(( ${#args[#]} - 1 ))
f1="${args[$last_idx]}"
unset args[$last_idx]
/usr/bin/diff "$args[#]" \
<( /usr/local/bin/perltidy "$f1" -st ) \
<( /usr/local/bin/perltidy "$f2" -st )
exit 0

Related

Replacing characters in a sh script

I am writing an sh script and need to replace the . and - with a _
Current:
V123_45_678_910.11_1213-1415.sh
Wanted:
V123_45_678_910_11_1213_1415.sh
I have used a few mv commands, but I am having trouble.
for file in /virtualun/rest/scripts/IOL_Extra/*.sh ; do mv $file ${file//V15_IOL_NVMe_01./V15_IOL_NVMe_01_} ; done
You don't need to match any of the other parts of the file name, just the characters you want to replace. To avoid turning foo.sh into foo-sh, remove the extension first, then add it back to the result of the replacement.
for file in /virtualun/rest/scripts/IOL_Extra/*.sh ; do
base=${file%.sh}
mv -i -- "$file" "${base//[-.]/_}".sh
done
Use the -i option to make sure you don't inadvertently replace one file with another when the modified names coincide.
This should work:
#!/usr/bin/env sh
# Fail on error
set -o errexit
# Disable undefined variable reference
set -o nounset
# Enable wildcard character expansion
set +o noglob
# ================
# CONFIGURATION
# ================
# Pattern
PATTERN="/virtualun/rest/scripts/IOL_Extra/*.sh"
# ================
# LOGGER
# ================
# Fatal log message
fatal() {
printf '[FATAL] %s\n' "$#" >&2
exit 1
}
# Info log message
info() {
printf '[INFO ] %s\n' "$#"
}
# ================
# MAIN
# ================
{
# Check directory exists
[ -d "$(dirname "$PATTERN")" ] || fatal "Directory '$PATTERN' does not exists"
for _file in $PATTERN; do
# Skip if not file
[ -f "$_file" ] || continue
info "Analyzing file '$_file'"
# File data
_file_dirname=$(dirname -- "$_file")
_file_basename=$(basename -- "$_file")
_file_name="${_file_basename%.*}"
_file_extension=
case $_file_basename in
*.*) _file_extension=".${_file_basename##*.}" ;;
esac
# New file name
_new_file_name=$(printf '%s\n' "$_file_name" | sed 's/[\.\-][\.\-]*/_/g')
# Skip if equals
[ "$_file_name" != "$_new_file_name" ] || continue
# New file
_new_file="$_file_dirname/${_new_file_name}${_file_extension}"
# Rename
info "Renaming file '$_file' to '$_new_file'"
mv -i -- "$_file" "$_new_file"
done
}
You can try this:
for f in /virtualun/rest/scripts/IOL_Extra/*.sh; do
mv "$f" $(sed 's/[.-]/_/g' <<< "$f")
done
The sed command is replacing all characters .- by _.
I prefer using sed substitute as posted by oliv.
However, if you have not familiar with regular expression, using rename is faster/easier to understand:
Example:
$ touch V123_45_678_910.11_1213-1415.sh
$ rename -va '.' '_' *sh
`V123_45_678_910.11_1213-1415.sh' -> `V123_45_678_910_11_1213-1415_sh'
$ rename -va '-' '_' *sh
`V123_45_678_910_11_1213-1415_sh' -> `V123_45_678_910_11_1213_1415_sh'
$ rename -vl '_sh' '.sh' *sh
`V123_45_678_910_11_1213_1415_sh' -> V123_45_678_910_11_1213_1415.sh'
$ ls *sh
V123_45_678_910_11_1213_1415.sh
Options explained:
-v prints the name of the file before -> after the operation
-a replaces all occurrences of the first argument with the second argument
-l replaces the last occurrence of the first argument with the second argument
Note that this might not be suitable depending on the other files you have in the given directory that would match *sh and that you do NOT want to rename.

posix sh: how to count number of occurrences in a string without using external tools?

In bash, it can be done like this:
#!/bin/bash
query='bengal'
string_to_search='bengal,toyger,bengal,persian,bengal'
delimiter='|'
replace_queries="${string_to_search//"$query"/"$delimiter"}"
delimiter_count="${replace_queries//[^"$delimiter"]}"
delimiter_count="${#delimiter_count}"
echo "Found $delimiter_count occurences of \"$query\""
Output:
Found 3 occurences of "bengal"
The caveat of course is that the delimiter cannot occur in 'query' or 'string_to_search'.
In POSIX sh, string replacement is not supported. Is there a way this can be done in POSIX sh using only shell builtins?
#!/bin/sh
query='bengal'
string_to_search='bengal,toyger,bengal,persian,bengal'
ct() (
n=0
IFS=,
q=$1
set $2
for t in "$#"; do
if [ "$t" = "$q" ]; then
n=$((n + 1))
fi
done
echo $n
)
n=$(ct "$query" "$string_to_search")
printf "found %d %s\n" $n $query
Though I'm not sure what the point is. If you've got a posix shell,
you also almost certainly have printf, sed, grep, and wc.
printf '%s\n' "$string_to_search" | sed -e 's/,/\n/g' | grep -Fx "$query" | wc -l
Think I got it...
#!/bin/sh
query='bengal'
string_to_search='bengal,toyger,bengal,persian,bengal'
i=0
process_string="$string_to_search"
while [ -n "$process_string" ]; do
case "$process_string" in
*"$query"*)
process_string="${process_string#*"$query"}"
i="$(( i + 1 ))"
;;
*)
break
;;
esac
done
echo "Found $i occurences of \"$query\""

Perl backticks not capturing output

I have the following program snippet
my $nfdump_command = "nfdump -M /data/nfsen/profiles-data/live/upstream1 -T -R ${syear}/${smonth}/${sday}/nfcapd.${syear}${smonth}${sday}0000:${eyear}/${emonth}/${eday}/nfcapd.${eyear}${emonth}${eday}2355 -n 100 -s ip/bytes -N -o csv -q | awk 'BEGIN { FS = \",\" } ; { if (NR > 1) print \$5, \$10 }'";
syslog("info", $nfdump_command);
my %args;
Nfcomm::socket_send_ok ($socket, \%args);
my #nfdump_output = `$nfdump_command`;
my %domain_name_to_bytes;
my %domain_name_to_ip_addresses;
syslog("info", Dumper(\#nfdump_output));
foreach my $a_line (#nfdump_output) {
syslog("info", "LINE: " . $a_line);
}
Bug: #nfdump_output is empty.
The $nfdump_command is correct and it printing output when ran individually
This program was working for sometime and then it broke. Couldn't figure out why. After moving my development setup to another virtual machine, I found out that using absolute path to nfdump fixes it

Print all line between the search pattern into different files using perl or any method

Could someone help out on this
I want to print all line between the search pattern (START & END) to different files (new_file_name can be any incremental name provided)
But the search pattern repeats in file hence each time it finds the pattern it should dump the line b/w them into different files
The file is something like this
START --- ./body1/b1
##########################
123body1
abcbody1
##########################
END --- ./body1/b1
START --- ./body2/b2
##########################
123body2
defbody2
##########################
END --- ./body2/b2
perl solution,
perl -MFile::Basename -MFile::Path -ne '
($a) = /^START.+?(\S+)$/;
$b = /^END/;
$a..$b or next;
if ($a){ mkpath(dirname $a); open STDOUT,">",$a; }
$a||$b or print;
' file
Here is my awk solution:
# print_between_patterns.awk
/^START/ { filename = $NF ; next } # On START, use the last field as file name
/^END/ { next } # On END, skip
{ print > filename } # For the rest of the lines, print to file
Assume your data file is called data.txt, the following will do what you want:
awk -f print_between_patterns.awk data.txt
Discussion
After the script ran, you will have ./body1, ./body2, and so on.
If you don't want to skip the BEGIN and END parts, remove the next commands.
Update
If you want to control the output filename in a sequential way:
/^START/ { filename = sprintf("out%04d.txt", ++count) ; next }
/^END/ { next }
{ print > filename }
To get automatically generated incremental file names:
awk '
/^END/ { inBlock=0 }
inBlock { print > outfile }
/^START/ { inBlock=1; outfile = "outfile" ++count }
' file
To use the file names from your input:
awk '
/^END/ { inBlock=0 }
inBlock { print > outfile }
/^START/ {
inBlock=1
outdir = outfile = $NF
sub(/\/[^\/]+$/,"",outdir)
system("mkdir -p \"" outdir "\"")
}
' file
The problem #JamesBond was having below was that I wasn't escaping the "/" within the character list in the sub() so I've updated my answer above to do that now. There's absolutely no reason why that should need to be escaped but apparently both nawk and /usr/xpg4/bin/awk require it:
$ cat file
the
quick/brown
dog
$ gawk '/[/]/' file
quick/brown
$ nawk '/[/]/' file
nawk: nonterminated character class [
source line number 1
context is
>>> /[/ <<< ]/
$ /usr/xpg4/bin/awk '/[/]/' file
/usr/xpg4/bin/awk: /[/: [ ] imbalance or syntax error Context is:
>>> /[/ <<<
and gawk doesn't care either way:
$ gawk --lint --posix '/[/]/' file
quick/brown
$ gawk --lint '/[/]/' file
quick/brown
$ gawk --lint --posix '/[\/]/' file
quick/brown
$ gawk --lint '/[\/]/' file
quick/brown
They all work just fine if I escape the backslash without putting it in a character list:
$ /usr/xpg4/bin/awk '/\//' file
quick/brown
$ nawk '/\//' file
quick/brown
$ gawk '/\//' file
quick/brown
So I guess that's something worth remembering for portability in future!
Using awk:
awk 'sub(/^START/, ""){out=sprintf("out%d", c++); p=1}
sub(/^END/, ""){print > out; p=0} p{print > out}' file
This will find and store each match between START and END into separate files named out1, out2 etc.
This is one way to do it in Bash.
#!/bin/bash
[ -n "$BASH_VERSION" ] || {
echo "You need Bash to run this script."
exit 1
}
shopt -s extglob || {
echo "Unable to enable extglob shell option."
exit 1
}
IFS=$' \t\n' ## Use default.
while read KEY DASH FILENAME; do
if [[ $KEY == START && $DASH == --- && -n $FILENAME ]]; then
CURRENT_FILENAME=$FILENAME
DIRNAME=${FILENAME%%+([^/])}
if [[ -n $DIRNAME ]]; then
mkdir -p "$DIRNAME" || {
echo "Unable to create directory $DIRNAME."
exit 1
}
fi
exec 4>"$CURRENT_FILENAME" || {
echo "Unable to open $CURRENT_FILENAME for output."
exit 1
}
for (( ;; )); do
IFS= read -r LINE || {
echo "End of file reached finding END block of $CURRENT_FILENAME."
exec 4>&-
exit 1
}
read -r KEY DASH FILENAME <<< "$LINE"
if [[ $KEY == END && $DASH == --- && $FILENAME == "$CURRENT_FILENAME" ]]; then
break
else
echo "$LINE" >&4
fi
done
exec 4>&-
fi
done
Make sure you save the script in UNIX file format then run it as bash script.sh < file.
I guess you need to see this.
perl -lne 'print if((/START/../END/) and ($_!~/START/ and $_!~/END/))' your_file
Tested below:
> cat temp
START --- ./body1
##########################
123body1
abcbody1
##########################
END --- ./body1
START --- ./body2
##########################
123body2
defbody2
##########################
END --- ./body2
> perl -lne 'print if((/START/../END/) and ($_!~/START/ and $_!~/END/))' temp
##########################
123body1
abcbody1
##########################
##########################
123body2
defbody2
##########################
>
This might work for you:
csplit -z file '/^START/' '{*}'
Files will be named xx00 xx01 xx..

perl backticks: use bash instead of sh

I noticed that when I use backticks in perl the commands are executed using sh, not bash, giving me some problems.
How can I change that behavior so perl will use bash?
PS. The command that I'm trying to run is:
paste filename <(cut -d \" \" -f 2 filename2 | grep -v mean) >> filename3
The "system shell" is not generally mutable. See perldoc -f exec:
If there is more than one argument in LIST, or if LIST is an array with more than one value, calls execvp(3) with the arguments in LIST. If
there is only one scalar argument or an array with one element in it, the argument is checked for shell metacharacters, and if there are any, the
entire argument is passed to the system's command shell for parsing (this is "/bin/sh -c" on Unix platforms, but varies on other platforms).
If you really need bash to perform a particular task, consider calling it explicitly:
my $result = `/usr/bin/bash command arguments`;
or even:
open my $bash_handle, '| /usr/bin/bash' or die "Cannot open bash: $!";
print $bash_handle 'command arguments';
You could also put your bash commands into a .sh file and invoke that directly:
my $result = `/usr/bin/bash script.pl`;
Try
`bash -c \"your command with args\"`
I am fairly sure the argument of -c is interpreted the way bash interprets its command line. The trick is to protect it from sh - that's what quotes are for.
This example works for me:
$ perl -e 'print `/bin/bash -c "echo <(pwd)"`'
/dev/fd/63
To deal with running bash and nested quotes, this article provides the best solution: How can I use bash syntax in Perl's system()?
my #args = ( "bash", "-c", "diff <(ls -l) <(ls -al)" );
system(#args);
I thought perl would honor the $SHELL variable, but then it occurred to me that its behavior might actually depend on your system's exec implementation. In mine, it seems that exec
will execute the shell
(/bin/sh) with the path of the
file as its first argument.
You can always do qw/bash your-command/, no?
Create a perl subroutine:
sub bash { return `cat << 'EOF' | /bin/bash\n$_[0]\nEOF\n`; }
And use it like below:
my $bash_cmd = 'paste filename <(cut -d " " -f 2 filename2 | grep -v mean) >> filename3';
print &bash($bash_cmd);
Or use perl here-doc for multi-line commands:
$bash_cmd = <<'EOF';
for (( i = 0; i < 10; i++ )); do
echo "${i}"
done
EOF
print &bash($bash_cmd);
I like to make some function btck (which integrates error checking) and bash_btck (which uses bash):
use Carp;
sub btck ($)
{
# Like backticks but the error check and chomp() are integrated
my $cmd = shift;
my $result = `$cmd`;
$? == 0 or confess "backtick command '$cmd' returned non-zero";
chomp($result);
return $result;
}
sub bash_btck ($)
{
# Like backticks but use bash and the error check and chomp() are
# integrated
my $cmd = shift;
my $sqpc = $cmd; # Single-Quote-Protected Command
$sqpc =~ s/'/'"'"'/g;
my $bc = "bash -c '$sqpc'";
return btck($bc);
}
One of the reasons I like to use bash is for safe pipe behavior:
sub safe_btck ($)
{
return bash_btck('set -o pipefail && '.shift);
}