The following example prints "SAME":
if (q/\\a/ eq q/\a/) {
print "SAME\n";
}
else {
print "DIFFERENT\n";
}
I understand this is consistent with the documentation. But I think this behavior is undesirable. Is there a need to escape a backlash lilteral in single-quoted string? If I wanted 2 backlashes, I'd need to specify 4; this does not seem convenient.
Shouldn't Perl detect whether a backslash serves as an escape character or not? For instance, when a backslash does not precede a delimiter, it should be treated as a literal; and if that were the case, I wouldn't need 3 backslashes to express two, e.g.,
q<a\\b>
instead of
q<a\\\b>.
Is there a need to escape a backlash in single-quoted string?
Yes, if the backslash is followed by another backslash, or is the last character in the string:
$ perl -e'print q/C:\/'
Can't find string terminator "/" anywhere before EOF at -e line 1.
$ perl -e'print q/C:\\/'
C:\
This makes it possible to include any character in a single-quoted string, including the delimiter and the escape character.
If I wanted 2 backlashes, I'd need to specify 4; this does not seem convenient.
Actually, you only need three (because the second backslash isn't followed by another backslash). But as an alternative, if your string contains a lot of backslashes you can use a single-quoted heredoc, which requires no escaping:
my $path = <<'END';
C:\a\very\long\path
END
chomp $path;
print $path; # C:\a\very\long\path
Note that the heredoc adds a newline to the end, which you can remove with chomp.
In single-quoted string literals,
A backslash represents a backslash unless followed by the delimiter or another backslash, in which case the delimiter or backslash is interpolated.
In other words,
You must escape delimiters.
You must escape \ that are followed by \ or the delimiter.
You may escape \ that aren't followed by \ or the delimiter.
So,
q/\// ⇒ /
q/\\\\a/ ⇒ \\a
q/\\\a/ ⇒ \\a
q/\\a/ ⇒ \a
q/\a/ ⇒ \a
Is there a need to escape a backlash in single-quoted string?
Yes, if it's followed by another backslash or the delimiter.
If I wanted 2 backlashes, I'd need to specify 4
Three would suffice.
this does not seem convenient.
It's more convenient than double-quoted strings, where backslashes must always be escaped. Single-quoted string require the minimum amount of escaping possible without losing the ability to produce the delimiter.
Related
In Bash, what are the differences between single quotes ('') and double quotes ("")?
Single quotes won't interpolate anything, but double quotes will. For example: variables, backticks, certain \ escapes, etc.
Example:
$ echo "$(echo "upg")"
upg
$ echo '$(echo "upg")'
$(echo "upg")
The Bash manual has this to say:
3.1.2.2 Single Quotes
Enclosing characters in single quotes (') preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
3.1.2.3 Double Quotes
Enclosing characters in double quotes (") preserves the literal value of all characters within the quotes, with the exception of $, `, \, and, when history expansion is enabled, !. The characters $ and ` retain their special meaning within double quotes (see Shell Expansions). The backslash retains its special meaning only when followed by one of the following characters: $, `, ", \, or newline. Within double quotes, backslashes that are followed by one of these characters are removed. Backslashes preceding characters without a special meaning are left unmodified. A double quote may be quoted within double quotes by preceding it with a backslash. If enabled, history expansion will be performed unless an ! appearing in double quotes is escaped using a backslash. The backslash preceding the ! is not removed.
The special parameters * and # have special meaning when in double quotes (see Shell Parameter Expansion).
The accepted answer is great. I am making a table that helps in quick comprehension of the topic. The explanation involves a simple variable a as well as an indexed array arr.
If we set
a=apple # a simple variable
arr=(apple) # an indexed array with a single element
and then echo the expression in the second column, we would get the result / behavior shown in the third column. The fourth column explains the behavior.
#
Expression
Result
Comments
1
"$a"
apple
variables are expanded inside ""
2
'$a'
$a
variables are not expanded inside ''
3
"'$a'"
'apple'
'' has no special meaning inside ""
4
'"$a"'
"$a"
"" is treated literally inside ''
5
'\''
invalid
can not escape a ' within ''; use "'" or $'\'' (ANSI-C quoting)
6
"red$arocks"
red
$arocks does not expand $a; use ${a}rocks to preserve $a
7
"redapple$"
redapple$
$ followed by no variable name evaluates to $
8
'\"'
\"
\ has no special meaning inside ''
9
"\'"
\'
\' is interpreted inside "" but has no significance for '
10
"\""
"
\" is interpreted inside ""
11
"*"
*
glob does not work inside "" or ''
12
"\t\n"
\t\n
\t and \n have no special meaning inside "" or ''; use ANSI-C quoting
13
"`echo hi`"
hi
`` and $() are evaluated inside "" (backquotes are retained in actual output)
14
'`echo hi`'
`echo hi`
`` and $() are not evaluated inside '' (backquotes are retained in actual output)
15
'${arr[0]}'
${arr[0]}
array access not possible inside ''
16
"${arr[0]}"
apple
array access works inside ""
17
$'$a\''
$a'
single quotes can be escaped inside ANSI-C quoting
18
"$'\t'"
$'\t'
ANSI-C quoting is not interpreted inside ""
19
'!cmd'
!cmd
history expansion character '!' is ignored inside ''
20
"!cmd"
cmd args
expands to the most recent command matching "cmd"
21
$'!cmd'
!cmd
history expansion character '!' is ignored inside ANSI-C quotes
See also:
ANSI-C quoting with $'' - GNU Bash Manual
Locale translation with $"" - GNU Bash Manual
A three-point formula for quotes
If you're referring to what happens when you echo something, the single quotes will literally echo what you have between them, while the double quotes will evaluate variables between them and output the value of the variable.
For example, this
#!/bin/sh
MYVAR=sometext
echo "double quotes gives you $MYVAR"
echo 'single quotes gives you $MYVAR'
will give this:
double quotes gives you sometext
single quotes gives you $MYVAR
Others explained it very well, and I just want to give something with simple examples.
Single quotes can be used around text to prevent the shell from interpreting any special characters. Dollar signs, spaces, ampersands, asterisks and other special characters are all ignored when enclosed within single quotes.
echo 'All sorts of things are ignored in single quotes, like $ & * ; |.'
It will give this:
All sorts of things are ignored in single quotes, like $ & * ; |.
The only thing that cannot be put within single quotes is a single quote.
Double quotes act similarly to single quotes, except double quotes still allow the shell to interpret dollar signs, back quotes and backslashes. It is already known that backslashes prevent a single special character from being interpreted. This can be useful within double quotes if a dollar sign needs to be used as text instead of for a variable. It also allows double quotes to be escaped so they are not interpreted as the end of a quoted string.
echo "Here's how we can use single ' and double \" quotes within double quotes"
It will give this:
Here's how we can use single ' and double " quotes within double quotes
It may also be noticed that the apostrophe, which would otherwise be interpreted as the beginning of a quoted string, is ignored within double quotes. Variables, however, are interpreted and substituted with their values within double quotes.
echo "The current Oracle SID is $ORACLE_SID"
It will give this:
The current Oracle SID is test
Back quotes are wholly unlike single or double quotes. Instead of being used to prevent the interpretation of special characters, back quotes actually force the execution of the commands they enclose. After the enclosed commands are executed, their output is substituted in place of the back quotes in the original line. This will be clearer with an example.
today=`date '+%A, %B %d, %Y'`
echo $today
It will give this:
Monday, September 28, 2015
Since this is the de facto answer when dealing with quotes in Bash, I'll add upon one more point missed in the answers above, when dealing with the arithmetic operators in the shell.
The Bash shell supports two ways to do arithmetic operation, one defined by the built-in let command and the other the $((..)) operator. The former evaluates an arithmetic expression while the latter is more of a compound statement.
It is important to understand that the arithmetic expression used with let undergoes word-splitting, pathname expansion just like any other shell commands. So proper quoting and escaping need to be done.
See this example when using let:
let 'foo = 2 + 1'
echo $foo
3
Using single quotes here is absolutely fine here, as there isn't any need for variable expansions here. Consider a case of
bar=1
let 'foo = $bar + 1'
It would fail miserably, as the $bar under single quotes would not expand and needs to be double-quoted as
let 'foo = '"$bar"' + 1'
This should be one of the reasons, the $((..)) should always be considered over using let. Because inside it, the contents aren't subject to word-splitting. The previous example using let can be simply written as
(( bar=1, foo = bar + 1 ))
Always remember to use $((..)) without single quotes
Though the $((..)) can be used with double quotes, there isn't any purpose to it as the result of it cannot contain content that would need the double quote. Just ensure it is not single quoted.
printf '%d\n' '$((1+1))'
-bash: printf: $((1+1)): invalid number
printf '%d\n' $((1+1))
2
printf '%d\n' "$((1+1))"
2
Maybe in some special cases of using the $((..)) operator inside a single quoted string, you need to interpolate quotes in a way that the operator either is left unquoted or under double quotes. E.g., consider a case, when you are tying to use the operator inside a curl statement to pass a counter every time a request is made, do
curl http://myurl.com --data-binary '{"requestCounter":'"$((reqcnt++))"'}'
Notice the use of nested double quotes inside, without which the literal string $((reqcnt++)) is passed to the requestCounter field.
There is a clear distinction between the usage of ' ' and " ".
When ' ' is used around anything, there is no "transformation or translation" done. It is printed as it is.
With " ", whatever it surrounds, is "translated or transformed" into its value.
By translation/ transformation I mean the following:
Anything within the single quotes will not be "translated" to their values. They will be taken as they are inside quotes. Example: a=23, then echo '$a' will produce $a on standard output. Whereas echo "$a" will produce 23 on standard output.
A minimal answer is needed for people to get going without spending a lot of time as I had to.
The following is, surprisingly (to those looking for an answer), a complete command:
$ echo '\'
whose output is:
\
Backslashes, surprisingly to even long-time users of bash, do not have any meaning inside single quotes. Nor does anything else.
I want to replace specific strings in php files automatically using sed. Some work, and some do not. I already investigated this is not an issue with the replacement string but with the string that is to be replaced. I already tried to escape [ and ] with no success. It seems to be the whitespace within the () - not whitespaces in general. The first whitespaces (around the = ) do not have any problems. Please can someone point me to the problem:
sed -e "1,\$s/$adm = substr($path . rawurlencode($upload['name']) , 16);/$adm = rawurlencode($upload['name']); # fix 23/g" -i administration/identify.php
I already tried to shorten the string which should be replaced and the result was if I cut it directly behind $path it works, with the following whitespace it does not. Escaping whitespace has no effect...
what must be escaped for sed
The following characters have special meaning in sed and have to be escaped with \ for the regex to be taken literally:
\
[
the character used in separating s command parts, ie. / here
.
*
& only replacement string
Newline character is handled specially as the end of the string, but can be replaced for \n.
So first escape all special characters in input and then pass it to sed:
rgx="$adm = substr($path . rawurlencode($upload['name']) , 16);"
rgx_escaped=$(sed 's/[\\\[\.\*\/&]/\\&/g' <<<"$rgx")
sed "s/$rgx_escaped/ etc."
See Escape a string for a sed replace pattern for a generic escaping solution.
You may use
sed -i 's/\$adm = substr(\$path \. rawurlencode(\$upload\['"'"'name'"'"']) , 16);/$adm = rawurlencode($upload['"'"'name'"'"']); # fix 23/g' administration/identify.php
Note:
the sed command is basically wrapped in single quotes, the variable expansion won't occur inside single quotes
In the POSIX BRE syntax, ( matches a literal (, you do not need to escape ) either, but you need to escape [ and . that must match themselves
The single quotes require additional quoting with concatenation.
Perl's q function or single quote is supposed to return the string literal as typed (except \'). But it doesn't work as expected for the following scenario.
I want to print the following UNC path
\\dir1\dir2\dir3
So I have used
my $path = q(\\dir1\dir2\dir3);
OR
my $path = '\\dir1\dir2\dir3';
But this skips one backslash at the front.
So if I print it i.e. print $path; it prints
\dir1\dir2\dir3
I want to know why? I have to type 3 or 4 backslashes at the beginning of the UNC path to make it work as expected. What am I missing?
From perldoc perlop:
q/STRING/
'STRING'
A single-quoted, literal string. A backslash represents a backslash unless followed by the delimiter or another backslash, in which case the delimiter or backslash is interpolated.
Change:
my $path = q(\\dir1\dir2\dir3);
to:
my $path = q(\\\dir1\dir2\dir3);
As for why, it's because Perl lets you include the quote delimiter in your string by escaping it with a backslash:
my $single_quote = 'This is a single quote: \'';
But if a backslash before the delimiter always escaped the delimiter, there would be no way to end a string with a backslash:
my $backslash = 'This is a backslash: \'; # nope
Allowing backslashes to be escaped too takes care of that:
my $backslash = 'This is a backslash: \\';
Interestingly enough, there is only one way to type in double backslashes in a perl string without it being interpolated as a single backslash.
As all the other answers showed, any of the quote operators treat backslashes as a backslash unless there is another one following it directly.
The only way to get the double backslashes to display exactly as you have typed them is to use a single quote here-doc.
my $path = <<'VISTA';
\\dir1\dir2\dir3
VISTA
chomp $path;
print $path."\n";
Would print it exactly as you've typed it in.
So, I saw in another post that to split using \\ as a delimiter, you need to split on \\\\\\\\. This didn't really make sense to me, but when I attempted to split using \\\\, this happened:
my $string="a\\\\b\\\\c";
my #ra=split("\\\\",$string);
Array is:
a
<empty>
b
<empty>
c
As the other poster said, using \\\\\\\\ works perfectly. Why is this the case?
Also, I got curious and started messing with '' vs "" and got unexpected results. I thought that I understood what the difference is, but I guess I didn't, at least not in the following context:
my $string="a\.\.b\.\.c";
my #ra=split("\.\.",$string);
Array is:
<empty>
<empty>
<empty>
c
Yet,
my $string="a\.\.b\.\.c";
my #ra=split('\.\.',$string);
Array is:
a
b
c
Thanks in advance.
Oh, quoting rules and regexes.
Backslash rules with different quotes
In q() and related, all backslashes are left in the string, unless they escape the string delimiter or another backslash:
say '\a\\b\''; # »\a\b'«
In qq() and related, all backslashes that do not form a known string escape sequence are silently removed:
say "\d\\b\"\."; # »d\b."«
Ditto in qr// and regex literals, except that there are different escapes compared to double quoted strings.
If a string is used in place of a regex, then during compilation the escape rules for that kind of string are performed. However, a second level of escapes is processed when it is used as a regex, hence backslashes have to be double-escaped in the worst cases. Regex literals don't suffer from this problem; there is only one level of escaping.
Explanations for your examples
Therefore, "a\\\\b\\\\c"; is a\\b\\c, and "\\\\" is \\ which matches \ as a regex. So it splits on every backslash, thus producing zero-length fields in between the double backslashes.
The '\\\\\\\\' of the other question you meant is \\\\ which as a regex matches \\.
The "a\.\.b\.\.c" is a..b..c, and "\.\." is .. which as a regex matches two non-newline characters. It first matches a., then .b, then ... This produces the string fragments "", "", "", "c".
The string '\.\.' is \.\., which as a regex matches two literal periods in sequence.
The solution is to use regexes where regexes are due. split takes a regex as first argument like split /foo/, in other scenarios the regex quote qr/foo/ is useful. This avoids mind-bending[1] double escaping.
[1]: for small values of ”mind bending”, once you grok the rules.
In single-quoted strings literals,
\ followed by the string delimiter (' by default) results in the string delimiter.
'That\'s fool\'s gold!' -> That's fool's gold!
q!That's fool's gold\!! -> That's fool's gold!
\ followed by \ results in \.
'c:\\foo' -> c:\foo
\ followed by anything else results in those two characters.
'c:\foo' -> c:\foo
In double-quoted strings literals,
\ followed by non-word character results in that character.
"c:\\foo" -> c:\foo
"Can't open \"foo\"" -> Can't open "foo"
\ followed by word character has a special meaning.
"foo\n" -> foo{newline}
In regular expressions literals,
\ followed by the delimiter is replaced results in the delimiter.
qr/\// -> /
\ followed by anything else results in those two characters.
qr/\\/ -> \\
qr/\_/ -> \_
qr/\$/ -> \$
qr/\n/ -> \n
When applying a regular expressions,
\ followed by non-word character matches that character.
/c:\\foo/ -> Matches strings containing: c:\foo
\ followed by word character has a special meaning.
/foo\z/ -> Matches strings ending with: foo
Looking at your cases:
my $string="a\\\\b\\\\c";
my #ra=split("\\\\",$string);
"\\\\" results in the string \\, so you first create the string a\\b\\c and you pass \\ to split.
The first argument of split is used as a regular expression, and the regex pattern \\ matches a single \. There are 4 \ in a\\b\\c, so it gets split into 4+1 pieces.
If you use regex literals instead of double-quoted string literals, there will be less confusion.
split(/\\/, $string); # Passes pattern \\ to split. Matches singles
split("\\\\", $string); # Passes pattern \\ to split. Matches singles
split(/\\\\/, $string); # Passes pattern \\\\ to split. Matches doubles
split("\\\\\\\\", $string); # Passes pattern \\\\ to split. Matches doubles
In short, don't use split "..."!
Your other two cases should be obvious to you by now.
my $string="a\.\.b\.\.c"; # String a..b..c
my #ra=split("\.\.",$string); # Pattern .., which matches any two chars.
my $string="a\.\.b\.\.c"; # String a..b..c
my #ra=split('\.\.',$string); # Pattern \.\., which matches two periods.
Split using /\\\\/ instead of "\\\\" and avoid all the worries,
e.g.
use Data::Dumper;
my $string= "a\\\\b\\\\c";
my #ra = split /\\\\/, $string;
print Dumper #ra;
will output
$VAR1 = [
'a',
'b',
'c'
];
/\\/ will match a two \ in a row
or you can be cute and do
split /\\{2}/, $string
Is there some way to replace a string such as #or * or ? or & without needing to put a "\" before it?
Example:
perl -pe 'next if /^#/; s/\#d\&/new_value/ if /param5/' test
In this example I need to replace a #d& with new_value but the old value might contain any character, how do I escape only the characters that need to be escaped?
You have several problems:
You are using \b incorrectly
You are replacing code with shell variables
You need to quote metacharacters
From perldoc perlre
A word boundary ("\b") is a spot between two characters that has a "\w" on one side of it
Neither of the characters # or & are \w characters. So your match is guaranteed to fail. You may want to use something like s/(^|\s)\#d\&(\s|$)/${1}new text$2/
(^|\s) says to match either the start of the string (^)or a whitespace character (\s).
(\s|$) says to match either the end of the string ($) or a whitespace character (\s).
To solve the second problem, you should use %ENV.
To solve the third problem, you should use the \Q and \E escape sequences to escape the value in $ENV{a}.
Putting it all together we get:
#!/bin/bash
export a='#d&'
export b='new text'
echo 'param5 #d&' |
perl -pe 'next if /^#/; s/(^|\s)\Q$ENV{a}\E(\s|$)/$1$ENV{b}$2/ if /param5/'
Which prints
param5 new text
As discussed at perldoc perlre:
...Today it is more common to use the quotemeta() function or the "\Q" metaquoting
escape sequence to disable all metacharacters' special meanings like this:
/$unquoted\Q$quoted\E$unquoted/
Beware that if you put literal backslashes (those not inside interpolated variables) between "\Q" and "\E", double-quotish backslash interpolation may
lead to confusing results. If you need to use literal backslashes within "\Q...\E", consult "Gory details of parsing quoted constructs" in perlop.
You can also use a ' as the delimiter in the s/// operation to make everything be parsed literally:
my $text = '#';
$text =~ s'#'1';
print $text;
In your example, you can do (note the single quotes):
perl -pe 's/\b\Q#f&\E\b/new_value/g if m/param5/ and not /^ *#/'
The other answers have covered the question, now here's your meta-problem: Leaning Toothpick Syndrome. Its when the delimiter and escapes start to blur together:
s/\/foo\/bar\\/\/bar\/baz/
The solution is to use a different delimiter. You can use just about anything, but balanced braces work best. Most editors can parse them and you generally don't have to worry about escaping.
s{/foo/bar\\}{/bar/baz}
Here's your regex with braced delimiters.
s{\#d\&}{new_value}
Much easier on the eyeholes.
If you really want to avoid typing the \s, put your search string into a variable and then use that in your regex instead. You don't need quotemeta or \Q ... \E in that case. For example:
my $s = '#d&';
s/$s/new_value/g;
If you must use this in a one-liner, bear in mind that you will have to escape the $s if you use "s to contain your perl code, or escape the 's if you use 's to contain your perl code.
If you have a string like
my $var1 = abc$123
and you want to replace it with abcd then you have to use \Q \E. If you don't then no matter what perl doesn't replace the string.
This is the only thing that worked for me.
my $var2 = s/\Q$var1\E/abcd/g;