How do I use argv with a wildcard? - fish

I'm trying to write a function which will change the extension for all matching files in the current directory:
function chext
if count $args -eq 2 > /dev/null
for f in (ls *$argv[1])
mv (basename $f $argv[2]) (basename $f $argv[1])
end
else
echo "Error: use the following syntax"
echo "chext oldext newext"
end
end
I keep getting this output though:
(*master) λ chext markdown txt
No matches for wildcard '*$argv[1]'. (Tip: empty matches are allowed in 'set', 'count', 'for'.)
~/.config/fish/config.fish (line 1): ls *$argv[1]
^
in command substitution
called on line 60 of file ~/.config/fish/config.fish
in function 'chext'
called on standard input
with parameter list 'markdown txt'
I know there must be a way to interpolate the variable and keep * as a wildcard, but I can't figure it out. I have tried using (string join '*' $argv[1]), but it turned the wildcard into a string.
I did get this to almost work:
function chext
if count $args -eq 2 > /dev/null
for f in (ls (string join * $argv[1]))
mv (basename $f $argv[2]) (basename $f $argv[1])
end
else
echo "Error: use the following syntax"
echo "chext oldext newext"
end
end
It removed the extensions from all of my files but didn't add the new one, then gave me an error which was all of the file names combined and said file name too long.
UPDATE:
I'm sure there is a way to get the correct ls working, but I did get this working successfully:
if count $args -eq 2 > /dev/null
for f in (ls)
mv $f (string replace -r \.$argv[1]\$ .$argv[2] (basename $f))
end
else
echo "Error: use the following syntax"
echo "chext oldext newext"
end
end

The answer is in the message:
No matches for wildcard '*$argv[1]'. (Tip: empty matches are allowed in 'set', 'count', 'for'.)
That means
for i in *argv[1]
works, as does
set -l expanded *argv[1]
If you try to launch any other command with a glob that doesn't match anything, fish will skip the execution and print an error.
Your function has a number of other issues, allow me to go through them one by one.
if count $args -eq 2 > /dev/null
count does not take the "-eq 2" argument. It will return 0 (i.e. true) whenever it is given an argument, so this will always be true.
What you tried to do was if test (count $args) -eq 2.
What also works (and what I happen to like more) is if set -q argv[2]
(also, it's "argv", not "args").
for f in (ls *$argv[1])
Please do not use the output of ls. It doesn't have as much problems in fish as it does in bash (where it'll break once a file has a space, tab or newline in the name), but it's unnecessary (it launches an additional process) and still has an issue - it'll break when a file has a newline in its name.
Just do for f in *$argv[1], which will also nicely solve your question.
mv (basename $f $argv[2]) (basename $f $argv[1])
I'm not sure what basename you are using here, but mine just removes a suffix. So if you did chext .opus .ogg, this would execute
mv (basename song.opus .ogg) (basename song.opus .opus)
which would resolve to
mv song.opus song
Personally, I'd use string replace for this. Something like
mv $f (string replace -- $argv[1] $argv[2] $f)
(Note: This would do the wrong thing if the extension string appears before, e.g. if you had a file called "f.ogg-banana.ogg" only the first ".ogg" would be changed. You might want to use regular expressions for this: string replace -r -- "$argv[1]\$" $argv[2] $f)

Related

ZSH autocomplete function using existing autocompletions

I have a function mycmd to launch a program that I wrote. The program needs the first argument to be foo, ssh or ls. The second argument depends on the first argument as follows,
foo -> No second argument
ssh -> Something to ssh to
ls -> A file
I want to write zsh autocomplete function for mycmd which suggest the second argument depending on the first argument. In the simplest form, I know that I can do the following for the first argument
_mycmd() {
compadd foo ssh ls
}
compdef _mycmd mycmd
I have a hard time understanding what to do for the second argument from here. How do I use _ssh autocomplete for ssh argument and _ls autocomplete for ls argument? (And nothing for foo as well)
To inspect the current command line, it could be used $words and $CURRENT that are the completion special parameters.
CURRENT
This is the number of the current word, i.e. the word the cursor is currently on in the words array.
...
words
This array contains the words present on the command line curently being edited.
--- zshcompwid(1), completion special parameters, zshcompwid - zsh completion widgets
The completion function could modify $words and $CURRENT (and/or other variables) and then start the entire completion system based with its modified command line. For example:
$ mycmd ls -al<TAB> ;# This is the input, and
;# $words == ("mycmd" "ls" "-al") ;# original value for $words.
;# $words=("ls" "-al") ;# We could update $words for making zsh
;# $ ls -al<TAB> ;# to start the completion system with
;# its modified command line.
;# Finally, _ls would be called.
The utility function _normal could be used.
_normal
...
A second use is to reexamine the command line specified by the $words array and the $CURRENT parameter after those have been modified.
-- zshcompsys(1), utility function, _normal
_mycmd could be listed below:
_mycmd () {
if ((CURRENT == 2)); then
compadd foo ssh ls
elif ((CURRENT > 2)); then
case "$words[2]" in
(ssh|ls)
shift words
((CURRENT--))
_normal -p mycmd
;;
(foo)
_nothing
;;
(*)
_message "mycmd: invalid subcommand or arguments"
;;
esac
fi
return $?
}
or even, more use of the completion builtin/utility functions like below:
_mycmd () {
local curcontext="${curcontext}" state context line
local -A opt_args
_arguments '*:: :->subcmd'
if [[ "$state" == "subcmd" ]]; then
if ((CURRENT == 1)); then
_describe -t mycmd-subcmd "mycmd command" '(foo ssh ls)'
else
curcontext="${curcontext%:*:*}:mycmd-$words[1]:"
case "$words[1]" in
(ssh|ls)
compset -n 1
_normal -p $service
;;
(foo)
_nothing
;;
(*)
_message "mycmd: invalid subcommand or arguments"
;;
esac
fi
fi
return $?
}

How to rename multiple files in a folder with a specific format?

I have many files in a folder with the format '{galaxyID}-cutout-HSC-I-{#}-pdr2_wide.fits', where {galaxyID} and {#} are different numbers for each file. Here are some examples:
2185-cutout-HSC-I-9330-pdr2_wide.fits
992-cutout-HSC-I-10106-pdr2_wide.fits
2186-cutout-HSC-I-9334-pdr2_wide.fits
I want to change the format of all files in this folder to match the following:
2185_HSC-I.fits
992_HSC-I.fits
2186_HSC-I.fits
namely, I want to take out "cutout", the second number, and "pdr2_wide" from each file name. I would prefer to do this in either Perl or Python. For my Perl script, so far I have the following:
rename [-n];
my #parts=split /-/;
my $this=$parts[0].$parts[1].$parts[2].$parts[3].$parts[4].$parts[5];
$_ = $parts[0]."_".$parts[2]."_".$parts[3];
*fits
which gives me the error message
Not enough arguments for rename at ./rename.sh line 3, near "];" Execution of ./rename.sh aborted due to compilation errors.
I included the [-n] because I want to make sure the changes are what I want before actually doing it; either way, this is in a duplicated directory just for safety.
It looks like you are using the rename you get on Ubuntu (it's not the one that's on my ArchLinux box), but there are other ones out there. But, you've presented it oddly. The brackets around -n shouldn't be there and the ; ends the command.
The syntax, if you are using what I think you are, is this:
% rename -n -e PERL_EXPR file1 file2 ...
The Perl expression is the argument to the -e switch, and can be a simple substitution. Note that this expression is a string that you give to -e, so that probably needs to be quoted:
% rename -n -e 's/-\d+-pdr2_wide//' *.fits
rename(2185-cutout-HSC-I-9330-pdr2_wide.fits, 2185-cutout-HSC-I.fits)
And, instead of doing this in one step, I'd do it in two:
% rename -n -e 's/-cutout-/-/; s/-\d+-pdr2_wide//' *.fits
rename(2185-cutout-HSC-I-9330-pdr2_wide.fits, 2185-HSC-I.fits)
There are other patterns that might make sense. Instead of taking away parts, you can keep parts:
% rename -n -e 's/\A(\d+).*(HSC-I).*/$1-$2.fits/' *.fits
rename(2185-cutout-HSC-I-9330-pdr2_wide.fits, 2185-HSC-I.fits)
I'd be inclined to use named captures so the next poor slob knows what you are doing:
% rename -n -e 's/\A(?<galaxy>\d+).*(HSC-I).*/$+{galaxy}-$2.fits/' *.fits
rename(2185-cutout-HSC-I-9330-pdr2_wide.fits, 2185-HSC-I.fits)
From your description {galaxyID}-cutout-HSC-I-{#}-pdr2_wide.fits, I assume that cutout-HSC-I is fixed.
Here's a script that will do the rename. It takes a list of files on stdin. But, you could adapt to take the output of readdir:
#!/usr/bin/perl
master(#ARGV);
exit(0);
sub master
{
my($oldname);
while ($oldname = <STDIN>) {
chomp($oldname);
# find the file extension/suffix
my($ix) = rindex($oldname,".");
next if ($ix < 0);
# get the suffix
my($suf) = substr($oldname,$ix);
# only take filenames of the expected format
next unless ($oldname =~ /^(\d+)-cutout-(HSC-I)/);
# get the new name
my($newname) = $1 . "_" . $2 . $suf;
printf("OLDNAME: %s NEWNAME: %s\n",$oldname,$newname);
# rename the file
# change to "if (1)" to actually do it
if (0) {
rename($oldname,$newname) or
die("unable to rename '$oldname' to '$newname' -- $!\n");
}
}
}
For your sample input file, here's the program output:
OLDNAME: 2185-cutout-HSC-I-9330-pdr2_wide.fits NEWNAME: 2185_HSC-I.fits
OLDNAME: 992-cutout-HSC-I-10106-pdr2_wide.fits NEWNAME: 992_HSC-I.fits
OLDNAME: 2186-cutout-HSC-I-9334-pdr2_wide.fits NEWNAME: 2186_HSC-I.fits
The above is how I usually do things but here's one with just a regex. It's fairly strict in what it accepts [for safety], but you can adapt as desired:
#!/usr/bin/perl
master(#ARGV);
exit(0);
sub master
{
my($oldname);
while ($oldname = <STDIN>) {
chomp($oldname);
# only take filenames of the expected format
next unless ($oldname =~ /^(\d+)-cutout-(HSC-I)-\d+-pdr2_wide([.].+)$/);
# get the new name
my($newname) = $1 . "_" . $2 . $3;
printf("OLDNAME: %s NEWNAME: %s\n",$oldname,$newname);
# rename the file
# change to "if (1)" to actually do it
if (0) {
rename($oldname,$newname) or
die("unable to rename '$oldname' to '$newname' -- $!\n");
}
}
}

tcl script to compare 2 texts and return the line numbers of the lines that differ only. Also, How to avoid "child process exit abnormally"?

can anybody guide me how to write tcl script to compare 2 texts and return the line numbers of the lines that differ only ?
I know how to do it in bash, but to include the bash in tcl doesnt seem to be very neat, here's the bash command :
diff --old-line-format '%L' --new-line-format '' --unchanged-line-format '' <(nl File1) <(nl File2) | awk '{print $1 }' > difflines
To include this in tcl, i did the following :
exec cat nl File1 > File11
exec cat nl File2 > File22
exec diff --old-line-format {%L} --new-line-format {} --unchanged-line-format {} <
File11 < File22 | awk {{print $1 }} > difflines
Is there a cleaner way ?
Also if there's difference i get the "child exit abnormally", how can i avoid this ?
Thanks
The struct::list package in Tcllib has tools for computing longest-common-subsequences, which is the key part of a diff tool. To use, you load your data into Tcl and split it into a list of lines:
proc getLines {filename} {
set f [open $filename]
set result [split [read $f] "\n"]
close $f
return $result
}
Then you can get the information about the common elements (== common lines):
set sharedLineInfo [struct::list longestCommonSubsequence $file1_lines $file2_lines]
This returns a pair of lists, where each list is the indices (counting from zero) of the common lines; the first list will be for the first file, and the second list for the second file. Any line number not listed will be one that has changed.
There's also a function to invert the information provided to get instructions on how to change one sequence into the other:
set changes [struct::list lcsInvert $sharedLineInfo \
[llength $file1_lines] [llength $file2_lines]]
This returns a list of triples, where the first is the operation performed (added, changed or deleted) and the second and third are the ranges of indices in each of the relevant lists (i.e., zero-based line numbers).
I'm not quite sure how to take this information and produce what you are looking for, but I guess we could put it together like this:
package require struct::list
proc getLines {filename} {
set f [open $filename]
set result [split [read $f] "\n"]
close $f
return $result
}
proc variedLines {filename1 filename2} {
set l1 [getLines $filename1]
set l2 [getLines $filename2]
lassign [struct::list longestCommonSubsequence $l1 $l2] common1
set result {}
for {set i 0} {$i < [llength $l1]} {incr i} {
if {$i ni $common1} {
lappend result [expr {$i + 1}]
}
}
return $result
}
If you want the results written to a file, puts $f [join $someList "\n"] is likely to feature, but I'll leave that as an exercise…
Regarding "child process exited abnormally", from the exec man page (emphasis mine):
If any of the commands in the pipeline exit abnormally or are killed or suspended, then exec will return an error and the error message will include the pipeline's output followed by error messages describing the abnormal terminations; the -errorcode return option will contain additional information about the last abnormal termination encountered. If any of the commands writes to its standard error file and that standard error is not redirected and -ignorestderr is not specified, then exec will return an error; the error message will include the pipeline's standard output, followed by messages about abnormal terminations (if any), followed by the standard error output.
"commands exit abnormally" means that the command exits with a non-zero status. Some common commands like grep and diff return a non-zero exit status to indicate something normal, so you have to wrap that exec call in a catch
set rc [catch {exec bash -c {
diff --old-line-format '%L' --new-line-format '' --unchanged-line-format '' <(nl File1) <(nl File2) | awk '{print $1}' > difflines
}} output]
if {$rc == 0} {
puts "no differences found"
} elseif {$rc == 1} {
puts "differences found:"
puts $output
} else {
puts "diff returned an error: $output"
}

Script to find all similarly named files (differing only by case?)

I having been working on an SVN repo using command line only. I now have to bring in users that require a GUI to interface with the repo, however this is presenting a number of problems with similarly named files.
As it so happens a large number of images have been duplicated for reasons due to lack of communication or laziness.
I would like to be able to search for all files recursively from a given folder, and identify all files that differ only by case/capitalization, and must have the same file size, as it is certainly possible conflicts exist between different files, although I've not encountered any yet.
I don't mind to hammer out a Perl script to handle this myself, however I'm wonder if such a thing already exists or if anybody has any tips before I roll my sleeves up?
Thanks :D
I lean on md5sum for this type of problem:
find * -type f | xargs md5sum | sort | uniq -Dw32
If you are using svn, you'll want to exclude your .svn directories. This will print out all files with their paths that have identical content.
If you really want to only match files that differ by case, you can add a few more things to the above pipeline:
find * -type f | xargs md5sum | sort | uniq -Dw32 | awk -F'[ /]' '{ print $NF }' | sort -f | uniq -Di
myimage_23.png
MyImage_23.png
A shell script to list all filenames in a Subversion working directory that differ only in case from another filename in the same directory, and therefore will cause problems for Subversion clients on case-insensitive file systems, which cannot distinguish between such filenames:
find . -name .svn -type d -prune -false -o -print | \
perl -ne 'push #{$f{lc($_)}}, $_; END{map{print #{$f{$_}}} grep {#{$f{$_}}>1} sort keys %f}'
I have not used it personally but the Duplicate Files Finder looks like it would be suitable.
However, it will identify any duplicate files, regardless of file name, so you might have to filter the results if you only want duplicates with case-insensitive-matching file names.
It is open source, available on Windows and Linux, has both command line and GUI interfaces, and from the description the algorithm sounds very fast (only compares files with the same size rather than producing a checksum for every file).
I guess it would be something like:
#!perl
use File::Spec;
sub check_dir {
my ($dir, $out) = #_;
$out ||= [];
opendir DIR, $dir or die "Impossible to read dir: $!";
my #files = sort grep { /^[^\.]/ } readdir(DIR); # Ignore files starting with dot
closedir DIR;
my #nd = map { ! -d $_ ? File::Spec->catfile($dir, $_) : () } #files;
for my $i (0 .. $#nd-1){
push #$out, $nd[$i]
if lc $nd[$i] eq lc $nd[$i+1]
and -s $nd[$i] == -s $nd[$i+1];
}
map { -d $_ ? &check_dir($_, $out) : () } #files;
return $out;
}
print join "\n", #{&check_dir(shift #ARGV)}, "";
Please check it before using it, I have no access to windows machines (this does not happen in Un*x). Also, note that in the case of two files with the same name (except for the case) and the same size, only the first will be printed. In the case of three, only the first two, and so on (of course, you will need to keep one!).
As far as I know what you want doesn't exist as such. However, here's an implementation in bash:
#!/bin/bash
dir=("$#")
matched=()
files=()
lc(){ tr '[:upper:]' '[:lower:]' <<< ${*} ; }
in_list() {
local search="$1"
shift
local list=("$#")
for file in "${list[#]}" ; do
[[ $file == $search ]] && return 0
done
return 1
}
while read -r file ; do
files=("${files[#]}" "$file")
done < <(find "${dir[#]}" -type f | sort)
for file1 in "${files[#]}" ; do
for file2 in "${files[#]}" ; do
if
# check that the file did not match already
! in_list "$file1" "${matched[#]}" &&
# check that the files are not the same file
! [ $(stat -f %i "${file1}") -eq $(stat -f %i "${file2}") ] &&
# check that the size of the files are the same
[ $(stat -f %z "${file1}") = $(stat -f %z "${file2}") ] &&
# check that the non-directory part (aka file name) of the two
# files match case insensitively
grep -q $(lc "${file1##*/}") <<<$(lc "${file2##*/}")
then
matched=("${matched[#]}" "$file1")
echo "$file1"
break
fi
done
done
EDIT: Added comments and, inspired by TLP's comment, made only the file part of the path matter for equality comparisons. This has now been tested to a reasonable minimum degree and I expect that it won't explode in your face.
Here's a Ruby script to recursively search for files that differ only in case.
#!/usr/bin/ruby
# encoding: utf-8
def search( directory )
set = {}
Dir.entries( directory ).each do |entry|
next if entry == '.' || entry == '..'
path = File.join( directory, entry )
key = path.upcase
set[ key ] = [] unless set.has_key?( key )
set[ key ] << entry
search( path ) if File.directory?( path )
end
set.delete_if { |key, entries| entries.size == 1 }
set.each do |key, entries|
entries.each do |entry|
puts File.join( directory, entry )
end
end
end
search( File.expand_path( ARGV[ 0 ] ) )

How do you climb up the parent directory structure of a bash script?

Is there a neater way of climbing up multiple directory levels from the location of a script.
This is what I currently have.
# get the full path of the script
D=$(cd ${0%/*} && echo $PWD/${0##*/})
D=$(dirname $D)
D=$(dirname $D)
D=$(dirname $D)
# second level parent directory of script
echo $D
I would like a neat way of finding the nth level. Any ideas other than putting in a for loop?
dir="/path/to/somewhere/interesting"
saveIFS=$IFS
IFS='/'
parts=($dir)
IFS=$saveIFS
level=${parts[3]}
echo "$level" # output: somewhere
#!/bin/sh
ancestor() {
local n=${1:-1}
(for ((; n != 0; n--)); do cd $(dirname ${PWD}); done; pwd)
}
Usage:
$ pwd
/home/nix/a/b/c/d/e/f/g
$ ancestor 3
/home/nix/a/b/c/d
A solution without loops would be to use recursion. I wanted to find a config file for a script by traversing backwards up from my current working directory.
rtrav() { test -e $2/$1 && echo $2 || { test $2 != / && rtrav $1 `dirname $2`;}; }
To check if the current directory is in a GIT repo: rtrav .git $PWD
rtrav will check the existence of a filename given by the first argument in each parent folder of the one given as the second argument. Printing the directory path where the file was found or exiting with an error code if the file was not found.
The predicate (test -e $2/$1) could be swapped for checking of a counter that indicates the traversal depth.
If you're OK with including a Perl command:
$ pwd
/u1/myuser/dir3/dir4/dir5/dir6/dir7
The first command lists the directory containing first N (in my case 5) directories
$ perl-e 'use File::Spec; \
my #dirs = File::Spec->splitdir( \
File::Spec->rel2abs( File::Spec->curdir() ) ); \
my #dirs2=#dirs[0..5]; print File::Spec->catdir(#dirs2) . "\n";'
/u1/myuser/dir3/dir4/dir5
The second command lists the directory N levels up (in my case 5) directories (I think you wanted the latter).
$ perl -e 'use File::Spec; \
my #dirs = File::Spec->splitdir( \
File::Spec->rel2abs( File::Spec->curdir() ) ); \
my #dirs2=#dirs[0..$#dir-5]; print File::Spec->catdir(#dirs2)."\n";'
/u1/myuser
To use it in your bash script, of course:
D=$(perl -e 'use File::Spec; \
my #dirs = File::Spec->splitdir( \
File::Spec->rel2abs( File::Spec->curdir() ) ); \
my #dirs2=#dirs[0..$#dir-5]; print File::Spec->catdir(#dirs2)."\n";')
Any ideas other than putting in a for loop?
In shells, you can't avoid the loop, because traditionally they do not support regexp, but glob matching instead. And glob patterns do not support the any sort of repeat counters.
And BTW, simplest way is to do it in shell is: echo $(cd $PWD/../.. && echo $PWD) where the /../.. makes it strip two levels.
With tiny bit of Perl that would be:
perl -e '$ENV{PWD} =~ m#^(.*)(/[^/]+){2}$# && print $1,"\n"'
The {2} in the Perl's regular expression is the number of directory entries to strip. Or making it configurable:
N=2
perl -e '$ENV{PWD} =~ m#^(.*)(/[^/]+){'$N'}$# && print $1,"\n"'
One can also use Perl's split(), join() and splice() for the purpose, e.g.:
perl -e '#a=split("/", $ENV{PWD}); print join("/", splice(#a, 0, -2)),"\n"'
where -2 says that from the path the last two entries has to be removed.
Two levels above the script directory:
echo "$(readlink -f -- "$(dirname -- "$0")/../..")"
All the quoting and -- are to avoid problems with tricky paths.
This method uses the actual full path to the perl script itself ... TIMTOWTDI
You could just easily replace the $RunDir with the path you would like to start with ...
#resolve the run dir where this scripts is placed
$0 =~ m/^(.*)(\\|\/)(.*)\.([a-z]*)/;
$RunDir = $1 ;
#change the \'s to /'s if we are on Windows
$RunDir =~s/\\/\//gi ;
my #DirParts = split ('/' , $RunDir) ;
for (my $count=0; $count < 4; $count++) { pop #DirParts ; }
$confHolder->{'ProductBaseDir'} = $ProductBaseDir ;
This allows you to work your way up until whatever condition is desired
WORKDIR=$PWD
until test -d "$WORKDIR/infra/codedeploy"; do
# get the full path of the script
WORKDIR=$(dirname $WORKDIR)
done