I am trying to return an array from a function. The code below doesn't work.
function testArray
echo 1 2 3 4
end
set r (testArray)
echo $r[2]
# error
What is the proper way to return multiple values from a function using fish shell?
The result of a command substitution becomes a list by splitting on newlines (technically the contents of $IFS, but modifying IFS is discouraged).
So you could replace spaces with newlines, perhaps with tr:
function testArray
echo 1 2 3 4
end
set r (testArray | tr ' ' \n)
echo $r[2]
Or modify the function to just output newlines directly:
function testArray
echo \n1\n2\n3\n4
end
set r (testArray)
echo $r[2]
https://github.com/fish-shell/fish-shell/issues/445 tracks better mechanisms for generating lists.
As explained by ridiculous_fish you can use native line-splitting mechanism.
result of a command substitution becomes a list by splitting on newlines ($IFS)
So
list one item per line ;
then capture that in a command substitution.
Listing
function __list_public_keys
for key in ~/.ssh/*.pub
echo $key
end
end
output
$ __list_public_keys
/home/ed8/.ssh/id_rsa.blabla.pub
/home/ed8/.ssh/id_rsa.pub
/home/ed8/.ssh/id_rsa.test.pub
Capturing
function __add_keys
set keys (__list_public_keys)
echo "[" $keys "]"
end
output
$ __add_keys
[/home/ed8/.ssh/id_rsa.blabla.pub] [/home/ed8/.ssh/id_rsa.pub] [/home/ed8/.ssh/id_rsa.test.pub]
I don't think this is possible yet. Compare this issue on GitHub.
Related
I define a function, an array and a variable:
set fnctn = "F(x)=Vx1*(1+cos(1*x-pi))"
set Vx = ( 1 1 1 1 )
set Vx1 = $Vx[1]
The following commands do what I want:
echo "$fnctn" | sed "s/Vx1/$Vx1/"
set fnctn2 = `echo "$fnctn" | sed "s/Vx1/$Vx1/"`
echo "$fnctn2"
or even:
echo "$fnctn" | sed "s/Vx1/$Vx[1]/"
But storing the answer to the later command in a variable such as:
set fnctn2 = `echo "$fnctn" | sed "s/Vx1/$Vx[1]/"`
reports the following error message:
set: No match.
Where is the trick?
ps: please do not suggest me to switch to bash :-) -
Because of the square brackets, csh interprets the word as a pattern and tries to do filename substitution ("globbing") on it. Since you don't have any files with names that match that "pattern", it tells you that it can't find a match.
Just inhibit filename substitution like this:
set noglob
before you attempt the assignment.
The catch here is that for $Vx[1], filename substitution is for some reason attempted twice: apparently, first on evaluation of the variable, then on the evaluation of the result of the command substitution. While for $Vx1, it's only attempted once, on variable substitution:
> ls
f1 f2 f3
> echo *
f1 f2 f3
> set v=("*" "?2")
> set v1="$v[1]"
> set echo=1
> echo `echo ${v1}`
echo `echo ${v1}`
echo *
f1 f2 f3
> echo `echo "${v1}"`
echo `echo "${v1}"`
echo *
*
> echo "${v[1]}"
echo *
*
> echo `echo "${v[1]}"`
echo `echo "${v[1]}"`
echo *
f1 f2 f3
My guess about the reason is because array indices are also subject of variable substitution, $Vx[1] is marked "substitute twice" or something, and the resulting "*" has "one substitution left yet". The man page doesn't say anything relevant, so if it's by design, the link is too subtle for me. Though it is definitely a side effect of the existing implementation, whatever it is. This is a bug in my book -- at least, the fact that this behavior is not documented.
The way around that I've found is to quote the command substitution clause. Now, escaping the quotes inside with a backslash doesn't work reliably and is prone to giving parsing errors depending on the expression inside. The way that worked for me in this case was to use single quotes inside:
> echo "`echo '$fnctn' | sed 's/Vx1/$Vx[1]/'`"
echo `echo 'F(x)=Vx1*(1+cos(1*x-pi))' | sed 's/Vx1/1/'`
sed s/Vx1/1/
echo F(x)=Vx1*(1+cos(1*x-pi))
F(x)=1*(1+cos(1*x-pi))
This is just one of the examples of csh's poor/unpolished design that causes people to recommend against using it.
To me, it looks like perl split can never give the result (""), i.e. a single-element list whose single element is the empty string. No matter what -- any input, any arguments to split. Can anyone show otherwise? And if not, is this a feature or a bug?
I wanted split to be able to for consistency, but alas:
Note that splitting an EXPR that evaluates to the empty string always
produces zero fields, regardless of the LIMIT specified.
http://perldoc.perl.org/functions/split.html
E.g.:
$ echo ""|perl -ne 'chomp;print 0+split/x/,$_,-1'
0
$ echo "x"|perl -ne 'chomp;print 0+split/x/,$_,-1'
2
$ echo "xx"|perl -ne 'chomp;print 0+split/x/,$_,-1'
3
$ echo "xxx"|perl -ne 'chomp;print 0+split/x/,$_,-1'
4
And if not, is this a feature or a bug?
Not returning an empty string is not a bug. As per the documentation,
Note that splitting an EXPR that evaluates to the empty string always produces zero fields, regardless of the LIMIT specified.
Can anyone show otherwise?
It's highly unlikely that anyone will be able to find an input for which split return an empty string when it's documented to never return an empty string.
It sounds like you want a list of one item when the input is an empty string, so
length($_) ? split(..., $_, -1) : ""
This is a tentative answer to my own question, pending any further information/correction that may come from others:
There are no inputs and arguments to perl split for which the result will ever be a single-element list containing the empty string.
In order to get a result consistent with the promises 1) "size of result will always be one more than the number of separators (regexp matches)" and 2) "if there are no separators, the result will always be a single-element list whose element is the whole original string", what would normally be a clean function call expression
split /.../
instead needs to be wrapped as follows, including an additional auxiliary array:
#s = split /.../, $_, -1 or push #s, "";
and then #s used where the split /.../ normally would have been.
E.g.:
$ echo ""|perl -ne 'chomp;#s=split/x/,$_,-1 or push #s,"";print 0+#s'
1
$ echo "x"|perl -ne 'chomp;#s=split/x/,$_,-1 or push #s,"";print 0+#s'
2
$ echo "xx"|perl -ne 'chomp;#s=split/x/,$_,-1 or push #s,"";print 0+#s'
3
$ echo "xxx"|perl -ne 'chomp;#s=split/x/,$_,-1 or push #s,"";print 0+#s'
4
Or, alternatively, any code using bare split /.../ and relying on either of the above "promises" needs to be put inside a guard if (length) {...} and the case of length==0 handled in separate code.
I need to search for a specific word in a file starting from specific line and return the line numbers only for the matched lines.
Let's say I want to search a file called myfile for the word my_word and then store the returned line numbers.
By using shell script the command :
sed -n '10,$ { /$my_word /= }' $myfile
works fine but how to write that command on tcl shell?
% exec sed -n '10,$ { /$my_word/= }' $file
extra characters after close-brace.
I want to add that the following command works fine on tcl shell but it starts from the beginning of the file
% exec sed -n "/$my_word/=" $file
447431
447445
448434
448696
448711
448759
450979
451006
451119
451209
451245
452936
454408
I have solved the problem as follows
set lineno 10
if { ! [catch {exec sed -n "/$new_token/=" $file} lineFound] && [string length $lineFound] > 0 } {
set lineNumbers [split $lineFound "\n"]
foreach num $lineNumbers {
if {[expr {$num >= $lineno}] } {
lappend col $num
}
}
}
Still can't find a single line that solve the problem
Any suggestions ??
I don't understand a thing: is the text you are looking for stored inside the variable called my_word or is the literal value my_word?
In your line
% exec sed -n '10,$ { /$my_word/= }' $file
I'd say it's the first case. So you have before it something like
% set my_word wordtosearch
% set file filetosearchin
Your mistake is to use the single quote character ' to enclose the sed expression. That character is an enclosing operator in sh, but has no meaning in Tcl.
You use it in sh to group many words in a single argument that is passed to sed, so you have to do the same, but using Tcl syntax:
% set my_word wordtosearch
% set file filetosearchin
% exec sed -n "10,$ { /$my_word/= }" $file
Here, you use the "..." to group.
You don't escape the $ in $my_word because you want $my_word to be substitued with the string wordtosearch.
I hope this helps.
After a few trial-and-error I came up with:
set output [exec sed -n "10,\$ \{ /$myword/= \}" $myfile]
# Do something with the output
puts $output
The key is to escape characters that are special to TCL, such as the dollar sign, curly braces.
Update
Per Donal Fellows, we do not need to escape the dollar sign:
set output [exec sed -n "10,$ \{ /$myword/= \}" $myfile]
I have tried the new revision and found it works. Thank you, Donal.
Update 2
I finally gained access to a Windows 7 machine, installed Cygwin (which includes sed and tclsh). I tried out the above script and it works just fine. I don't know what your problem is. Interestingly, the same script failed on my Mac OS X system with the following error:
sed: 1: "10,$ { /ipsum/= }": extra characters at the end of = command
while executing
"exec sed -n "10,$ \{ /$myword/= \}" $myfile"
invoked from within
"set output [exec sed -n "10,$ \{ /$myword/= \}" $myfile]"
(file "sed.tcl" line 6)
I guess there is a difference between Linux and BSD systems.
Update 3
I have tried the same script under Linux/Tcl 8.4 and it works. That might mean Tcl 8.4 has nothing to do with it. Here is something else that might help: Tcl comes with a package called fileutil, which is part of the tcllib. The fileutil package contains a useful tool for this case: fileutil::grep. Here is a sample on how to use it in your case:
package require fileutil
proc grep_demo {myword myfile} {
foreach line [fileutil::grep $myword $myfile] {
# Each line is in the format:
# filename:linenumber:text
set lineNumber [lindex [split $line :] 1]
if {$lineNumber >= 10} { puts $lineNumber}
}
}
puts [grep_demo $myword $myfile]
Here is how to do it with awk
awk 'NR>10 && $0~f {print NR}' f="$my_word" "$myfile"
This search for all line larger than line number 10 that contains word in variable $my_word in file name stored in variable myfile
I have a list of strings and I want to pass those strings as arguments in a single Bash command line call. For simple alphanumeric strings it suffices to just pass them verbatim:
> script.pl foo bar baz yes no
foo
bar
baz
yes
no
I understand that if an argument contains spaces or backslashes or double-quotes, I need to backslash-escape the double-quotes and backslashes, and then double-quote the argument.
> script.pl foo bar baz "\"yes\"\\\"no\""
foo
bar
baz
"yes"\"no"
But when an argument contains an exclamation mark, this happens:
> script.pl !foo
-bash: !foo: event not found
Double quoting doesn't work:
> script.pl "!foo"
-bash: !foo: event not found
Nor does backslash-escaping (notice how the literal backslash is present in the output):
> script.pl "\!foo"
\!foo
I don't know much about Bash yet but I know that there are other special characters which do similar things. What is the general procedure for safely escaping an arbitrary string for use as a command line argument in Bash? Let's assume the string can be of arbitrary length and contain arbitrary combinations of special characters. I would like an escape() subroutine that I can use as below (Perl example):
$cmd = join " ", map { escape($_); } #args;
Here are some more example strings which should be safely escaped by this function (I know some of these look Windows-like, that's deliberate):
yes
no
Hello, world [string with a comma and space in it]
C:\Program Files\ [path with backslashes and a space in it]
" [i.e. a double-quote]
\ [backslash]
\\ [two backslashes]
\\\ [three backslashes]
\\\\ [four backslashes]
\\\\\ [five backslashes]
"\ [double-quote, backslash]
"\T [double-quote, backslash, T]
"\\T [double-quote, backslash, backslash, T]
!1
!A
"!\/'" [double-quote, exclamation, backslash, forward slash, apostrophe, double quote]
"Jeff's!" [double-quote, J, e, f, f, apostrophe, s, exclamation, double quote]
$PATH
%PATH%
&
<>|&^
*#$$A$##?-_
EDIT:
Would this do the trick? Escape every unusual character with a backslash, and omit single or double quotes. (Example is in Perl but any language can do this)
sub escape {
$_[0] =~ s/([^a-zA-Z0-9_])/\\$1/g;
return $_[0];
}
If you want to securely quote anything for Bash, you can use its built-in printf %q formatting:
cat strings.txt:
yes
no
Hello, world
C:\Program Files\
"
\
\\
\\\
\\\\
\\\\\
"\
"\T
"\\T
!1
!A
"!\/'"
"Jeff's!"
$PATH
%PATH%
&
<>|&^
*#$$A$##?-_
cat quote.sh:
#!/bin/bash
while IFS= read -r string
do
printf '%q\n' "$string"
done < strings.txt
./quote.sh:
yes
no
Hello\,\ world
C:\\Program\ Files\\
\"
\\
\\\\
\\\\\\
\\\\\\\\
\\\\\\\\\\
\"\\
\"\\T
\"\\\\T
\!1
\!A
\"\!\\/\'\"
\"Jeff\'s\!\"
\$PATH
%PATH%
\&
\<\>\|\&\^
\*#\$\$A\$##\?-_
These strings can be copied verbatim to for example echo to output the original strings in strings.txt.
What is the general procedure for safely escaping an arbitrary string for use as a command line argument in Bash?
Replace every occurrence of ' with '\'', then put ' at the beginning and end.
Every character except for a single quote can be used verbatim in a single-quote-delimited string. There's no way to put a single quote inside a single-quote-delimited string, but that's easy enough to work around: end the string ('), then add a single quote by using a backslash to escape it (\'), then begin a new string (').
As far as I know, this will always work, with no exceptions.
You can use single quotes to escape strings for Bash. Note however this does not expand variables within quotes as double quotes do. In your example, the following should work:
script.pl '!foo'
From Perl, this depends on the function you are using to spawn the external process. For example, if you use the system function, you can pass arguments as parameters so there"s no need to escape them. Of course you"d still need to escape quotes for Perl:
system("/usr/bin/rm", "-fr", "/tmp/CGI_test", "/var/tmp/CGI");
sub text_to_shell_lit(_) {
return $_[0] if $_[0] =~ /^[a-zA-Z0-9_\-]+\z/;
my $s = $_[0];
$s =~ s/'/'\\''/g;
return "'$s'";
}
See this earlier post for an example.
Whenever you see you don't get the desired output, use the following method:
"""\special character"""
where special character may include ! " * ^ % $ # # ....
For instance, if you want to create a bash generating another bash file in which there is a string and you want to assign a value to that, you can have the following sample scenario:
Area="(1250,600),(1400,750)"
printf "SubArea="""\""""${Area}"""\""""\n" > test.sh
printf "echo """\$"""{SubArea}" >> test.sh
Then test.sh file will have the following code:
SubArea="(1250,600),(1400,750)"
echo ${SubArea}
As a reminder to have newline \n, we should use printf.
Bash interprets exclamation marks only in interactive mode.
You can prevent this by doing:
set +o histexpand
Inside double quotes you must escape dollar signs, double quotes, backslashes and I would say that's all.
This is not a complete answer, but I find it useful sometimes to combine two types of quote for a single string by concatenating them, for example echo "$HOME"'/foo!?.*' .
FWIW, I wrote this function that invokes a set of arguments using different credentials. The su command required serializing all the arguments, which required escaping them all, which I did with the printf idiom suggested above.
$ escape_args_then_call_as myname whoami
escape_args_then_call_as() {
local user=$1
shift
local -a args
for i in "$#"; do
args+=( $(printf %q "${i}") )
done
sudo su "${user}" -c "${args[*]}"
}
Dear friends
I have the following:
PARAM=1,2,3=,4,5,6,=,7#,8,9
How to count by sed/awk the even "=" character between PARAM until "#" character
For example
PARAM=1,2,3=,4,5,6,=,7#,8,9
Then sed/awk should return 3
OR
PARAM=1,2,3=,4=,5=,6,=,7#,=8,9
Then sed/awk should return 5
THX
yael
you can use this one liner. No need to use split() as in the answer. Just use gsub(). It will return the count of the thing that is replaced. Also, set the field delimiter to "#", so you only need to deal with the first field.
$ echo "PARAM=1,2,3=,4,5,6,=,7#,8,9" | awk -F"#" '{print gsub("=","",$1)}'
3
$ echo "PARAM=1,2,3=,4=,5=,6,=,7#,=8,9" | awk -F"#" '{print gsub("=","",$1)}'
5
Here is an awk script that finds the count using field separators/split. IT sets the field separator to the # symbol and then splits the first word (the stuff to the left of the first # on the = character. An odd approach possibly, but it is one method. Note that it assumes there are no = characters to the left of param. If that is a bad assumption, this will not work.
BEGIN{ FS="#" }
/PARAM.*#/{
n = split( $1, a, "=" );
printf( "Count = %d\n", n-1 );
}
It can be done with one line as well:
[]$ export LINE=PARAM=1,2=3,4=5#=6
[]$ echo $LINE | awk 'BEGIN{FS="#"}/PARAM.*#/{n=split($1,a,"="); print n-1;}'
3