My PowerShell script has one parameter. It is invoked by a tool, which feeds it the argument. The argument contains special characters such as ', " and %. With special characters, PowerShell expects the argument to be surrounded by single or double quotes. What if the argument contains both single and double quotes? No problem. If the argument is surrounded by single quotes, use two single quotes instead of one inside the argument. If it surrounded by double quotes, use two double quotes instead of one inside the argument.
Problem: I cannot modify the argument before passing it to the script, i.e., I cannot double the single/double quotes in the argument. Is there anything I can do?
-Rohan.
EDIT #1:
Reason I cannot modify the argument is that it is automatically passed to the script by the tool.
EDIT #2:
The argument is a password and so, I need to accept and store it as a secure string in the script.
One option is to pass it as a base64-encoded string, and then decode it in the script:
Using Powershell -encodedcommand to pass parameters
Related
I am writing a program which expects command line arguments of the form
-foo ${foo}
However, ${name} is Eclipse’s notation for variables. Passing the above command line argument causes Eclipse to look for an internal variable named foo and inserting that instead of the ${foo} variable specification before running the program.
What is the proper way of escaping ${foo} so Eclipse will pass it literally, rather than trying to expand it?
TL;DR: $\{foo}
The escape character for Eclipse is \; prepending this will cause Eclipse to pass the following character literally.
Of a string resembling a variable specification, only the opening brace needs to be escaped in order to pass the whole thing literally.
Escaping $ and } is not needed but will do no harm – $\{foo}, \$\{foo} and \$\{foo\} will all result in ${foo} being passed to the program.
In Bash, what are the differences between single quotes ('') and double quotes ("")?
Single quotes won't interpolate anything, but double quotes will. For example: variables, backticks, certain \ escapes, etc.
Example:
$ echo "$(echo "upg")"
upg
$ echo '$(echo "upg")'
$(echo "upg")
The Bash manual has this to say:
3.1.2.2 Single Quotes
Enclosing characters in single quotes (') preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
3.1.2.3 Double Quotes
Enclosing characters in double quotes (") preserves the literal value of all characters within the quotes, with the exception of $, `, \, and, when history expansion is enabled, !. The characters $ and ` retain their special meaning within double quotes (see Shell Expansions). The backslash retains its special meaning only when followed by one of the following characters: $, `, ", \, or newline. Within double quotes, backslashes that are followed by one of these characters are removed. Backslashes preceding characters without a special meaning are left unmodified. A double quote may be quoted within double quotes by preceding it with a backslash. If enabled, history expansion will be performed unless an ! appearing in double quotes is escaped using a backslash. The backslash preceding the ! is not removed.
The special parameters * and # have special meaning when in double quotes (see Shell Parameter Expansion).
The accepted answer is great. I am making a table that helps in quick comprehension of the topic. The explanation involves a simple variable a as well as an indexed array arr.
If we set
a=apple # a simple variable
arr=(apple) # an indexed array with a single element
and then echo the expression in the second column, we would get the result / behavior shown in the third column. The fourth column explains the behavior.
#
Expression
Result
Comments
1
"$a"
apple
variables are expanded inside ""
2
'$a'
$a
variables are not expanded inside ''
3
"'$a'"
'apple'
'' has no special meaning inside ""
4
'"$a"'
"$a"
"" is treated literally inside ''
5
'\''
invalid
can not escape a ' within ''; use "'" or $'\'' (ANSI-C quoting)
6
"red$arocks"
red
$arocks does not expand $a; use ${a}rocks to preserve $a
7
"redapple$"
redapple$
$ followed by no variable name evaluates to $
8
'\"'
\"
\ has no special meaning inside ''
9
"\'"
\'
\' is interpreted inside "" but has no significance for '
10
"\""
"
\" is interpreted inside ""
11
"*"
*
glob does not work inside "" or ''
12
"\t\n"
\t\n
\t and \n have no special meaning inside "" or ''; use ANSI-C quoting
13
"`echo hi`"
hi
`` and $() are evaluated inside "" (backquotes are retained in actual output)
14
'`echo hi`'
`echo hi`
`` and $() are not evaluated inside '' (backquotes are retained in actual output)
15
'${arr[0]}'
${arr[0]}
array access not possible inside ''
16
"${arr[0]}"
apple
array access works inside ""
17
$'$a\''
$a'
single quotes can be escaped inside ANSI-C quoting
18
"$'\t'"
$'\t'
ANSI-C quoting is not interpreted inside ""
19
'!cmd'
!cmd
history expansion character '!' is ignored inside ''
20
"!cmd"
cmd args
expands to the most recent command matching "cmd"
21
$'!cmd'
!cmd
history expansion character '!' is ignored inside ANSI-C quotes
See also:
ANSI-C quoting with $'' - GNU Bash Manual
Locale translation with $"" - GNU Bash Manual
A three-point formula for quotes
If you're referring to what happens when you echo something, the single quotes will literally echo what you have between them, while the double quotes will evaluate variables between them and output the value of the variable.
For example, this
#!/bin/sh
MYVAR=sometext
echo "double quotes gives you $MYVAR"
echo 'single quotes gives you $MYVAR'
will give this:
double quotes gives you sometext
single quotes gives you $MYVAR
Others explained it very well, and I just want to give something with simple examples.
Single quotes can be used around text to prevent the shell from interpreting any special characters. Dollar signs, spaces, ampersands, asterisks and other special characters are all ignored when enclosed within single quotes.
echo 'All sorts of things are ignored in single quotes, like $ & * ; |.'
It will give this:
All sorts of things are ignored in single quotes, like $ & * ; |.
The only thing that cannot be put within single quotes is a single quote.
Double quotes act similarly to single quotes, except double quotes still allow the shell to interpret dollar signs, back quotes and backslashes. It is already known that backslashes prevent a single special character from being interpreted. This can be useful within double quotes if a dollar sign needs to be used as text instead of for a variable. It also allows double quotes to be escaped so they are not interpreted as the end of a quoted string.
echo "Here's how we can use single ' and double \" quotes within double quotes"
It will give this:
Here's how we can use single ' and double " quotes within double quotes
It may also be noticed that the apostrophe, which would otherwise be interpreted as the beginning of a quoted string, is ignored within double quotes. Variables, however, are interpreted and substituted with their values within double quotes.
echo "The current Oracle SID is $ORACLE_SID"
It will give this:
The current Oracle SID is test
Back quotes are wholly unlike single or double quotes. Instead of being used to prevent the interpretation of special characters, back quotes actually force the execution of the commands they enclose. After the enclosed commands are executed, their output is substituted in place of the back quotes in the original line. This will be clearer with an example.
today=`date '+%A, %B %d, %Y'`
echo $today
It will give this:
Monday, September 28, 2015
Since this is the de facto answer when dealing with quotes in Bash, I'll add upon one more point missed in the answers above, when dealing with the arithmetic operators in the shell.
The Bash shell supports two ways to do arithmetic operation, one defined by the built-in let command and the other the $((..)) operator. The former evaluates an arithmetic expression while the latter is more of a compound statement.
It is important to understand that the arithmetic expression used with let undergoes word-splitting, pathname expansion just like any other shell commands. So proper quoting and escaping need to be done.
See this example when using let:
let 'foo = 2 + 1'
echo $foo
3
Using single quotes here is absolutely fine here, as there isn't any need for variable expansions here. Consider a case of
bar=1
let 'foo = $bar + 1'
It would fail miserably, as the $bar under single quotes would not expand and needs to be double-quoted as
let 'foo = '"$bar"' + 1'
This should be one of the reasons, the $((..)) should always be considered over using let. Because inside it, the contents aren't subject to word-splitting. The previous example using let can be simply written as
(( bar=1, foo = bar + 1 ))
Always remember to use $((..)) without single quotes
Though the $((..)) can be used with double quotes, there isn't any purpose to it as the result of it cannot contain content that would need the double quote. Just ensure it is not single quoted.
printf '%d\n' '$((1+1))'
-bash: printf: $((1+1)): invalid number
printf '%d\n' $((1+1))
2
printf '%d\n' "$((1+1))"
2
Maybe in some special cases of using the $((..)) operator inside a single quoted string, you need to interpolate quotes in a way that the operator either is left unquoted or under double quotes. E.g., consider a case, when you are tying to use the operator inside a curl statement to pass a counter every time a request is made, do
curl http://myurl.com --data-binary '{"requestCounter":'"$((reqcnt++))"'}'
Notice the use of nested double quotes inside, without which the literal string $((reqcnt++)) is passed to the requestCounter field.
There is a clear distinction between the usage of ' ' and " ".
When ' ' is used around anything, there is no "transformation or translation" done. It is printed as it is.
With " ", whatever it surrounds, is "translated or transformed" into its value.
By translation/ transformation I mean the following:
Anything within the single quotes will not be "translated" to their values. They will be taken as they are inside quotes. Example: a=23, then echo '$a' will produce $a on standard output. Whereas echo "$a" will produce 23 on standard output.
A minimal answer is needed for people to get going without spending a lot of time as I had to.
The following is, surprisingly (to those looking for an answer), a complete command:
$ echo '\'
whose output is:
\
Backslashes, surprisingly to even long-time users of bash, do not have any meaning inside single quotes. Nor does anything else.
This question was asked many times on SO and yet...
All I've seen were solutions where the input string has to be modified. Either by replacing all double quotes with single quotes or by using backticks.
But I have no control over the input string since I have no access to the source. I cannot change Hello "W"orld to Hello 'W'orld or Hello """W"""orld
What I can do is to wrap the whole string with any escaping characters. For example with single quotes around 'Hello "W"orld'. But none of thoses escaping mechanisms I tried worked. And I can change my PowerShell script
Q: How can I pass a string with double quotes to PowerShell as argument and retain the quotes?
How to reproduce
Save this
cls
write-host $args[0]
as PowerShell script echoArgs1.ps1 on your desktop.
Open a CMD window, navigate to your desktop folder and enter
powershell -file echoArgs1.ps1 "Hello "W"orld"
Current Output
Desired Output
You're using the $(CurText) macro to pass the currently selected text in Visual Studio to a PowerShell script file via an external tools definition.
Unfortunately, Visual Studio doesn't offer a way to escape double quotes in the selected text to guarantee that it is ultimately seen as-is by whatever external executable you pass it to.
(For most executables, including PowerShell, embedding literal " chars. in a "..."-enclosed argument requires escaping them as \" - see this answer for the full story.)
Due to this lack of proper escaping, PowerShell won't parse text passed as an argument to a script file (*.ps1) via the -File CLI parameter as expected if it contains literal " chars.
This is Visual Studio's shortcoming, but there is a workaround:
With just one argument being passed, inspect the raw command line via [Environment]::CommandLine, and consider everything after the *.ps1 file the argument, verbatim.
To simplify that process, pass $(CurText) without enclosing it in "..." in the external-tool definition (and make sure that it is separated from the previous token by just one space char.).
Inside of echoArgs1.ps1, use the following command to retrieve the argument verbatim:
$rawText = ([Environment]::CommandLine -split '\.ps1 ', 2)[-1]
The problem is that the command line interpreter has already removed the quotes. In other words, the quotes are already gone before the command reaches the PowerShell interpreter.
What you might try to do is: pulling the original bare command line ($MyInvocation.Line) and resolve the arguments by removing the command itself:
$FileName = [System.IO.Path]::GetFileName($MyInvocation.MyCommand.Path)
$Arguments = $MyInvocation.Line -Replace ("^.*\\" + $FileName.Replace(".", "\.") + "['"" ]\s*")
Write-Host $Arguments
Note that there are a few pitfalls with regards to the command filename in the command line:
it might contain a relative or absolute path
it might be quoted or not
I need to be able to have a URL stored in a variable in powershell, but the issue comes to the correct formating.
For example I need to format this URL
https://graph.windows.net/mytenantname/users?$top=999?api-version=1.6
But i need to replace mytenantname with a variable
"https://graph.windows.net/$mytenantname/users?$top=999?api-verion=1.6"
but doing this i can't keep the $top as PowerShell treats is as a variable too....
Use a backtick (`) on the dollar sign you want, and leave the variable dollar sign:
"https://graph.windows.net/$mytenantname/users?`$top=999?api-verion=1.6"
I have a Windows application and on events, it calls a command like this:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass "G:\test.ps1 -name '%x' -data '%y'"
The name parameter sometimes has ' in it. Is it possible to escape that somehow?
This is actually a lot trickier than you'd think. Escaping nested quotes in strings passed from cmd to PowerShell is a major headache. What makes this one especially tricky is that you need to make the replacement in a variable expanded by cmd in the quoted argument passed to powershell.exe within a single-quoted argument passed to a PowerShell script parameter. AFAIK cmd doesn't have any native functionality for even basic string replacements, so you need PowerShell to do the replacement for you.
If the argument to the -data paramater (the one contained in the cmd variable x) doesn't necessarily need to be single-quoted, the simplest thing to do is to double-quote it, so that single quotes within the value of x don't need to be escaped at all. I say "simplest", but even that is a little tricky. As Vasili Syrakis indicated, ^ is normally the escape character in cmd, but to escape double quotes within a (double-)quoted string, you need to use a \. So, you can write your batch command like this:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass "G:\test.ps1 -name \"%x%\" -data '%y%'"
That passes the following command to PowerShell:
G:\test.ps1 -name "value of x, which may contain 's" -data 'value of y'
If, however, x can also contain characters that are special characters in PowerShell interpolated strings (", $, or `), then it becomes a LOT more complicated. The problem is that %x is a cmd variable that gets expanded by cmd before PowerShell has a chance to touch it. If you single-quote it in the command you're passing to powershell.exe and it contains a single quote, then you're giving the PowerShell session a string that gets terminated early, so PowerShell doesn't have the opportunity to perform any operations on it. The following obviously doesn't work, because the -replace operator needs to be supplied a valid string before you can replace anything:
'foo'bar' -replace "'", "''"
On the other hand, if you double-quote it, then PowerShell interpolates the string before it performs any replacements on it, so if it contains any special characters, they're interpreted before they can be escaped by a replacement. I searched high and low for other ways to quote literal strings inline (something equivalent to perl's q//, in which nothing needs to be escaped but the delimiter of your choice), but there doesn't seem to be anything.
So, the only solution left is to use a here string, which requires a multi-line argument. That's tricky in a batch file, but it can be done:
setlocal EnableDelayedExpansion
set LF=^
set pscommand=G:\test.ps1 -name #'!LF!!x!!LF!'# -data '!y!'
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass "!pscommand!"
This assumes that x and y were set earlier in the batch file. If your app can only send a single-line command to cmd, then you'll need to put the above into a batch file, adding the following two lines to the beginning:
set x=%~1
set y=%~2
Then invoke the batch file like this:
path\test.bat "%x%" "%y%"
The ~ strips out the quotes surrounding the command line arguments. You need the quotes in order to include spaces in the variables, but the quotes are also added to the variable value. Batch is stupid that way.
The two blank lines following set LF=^ are required.
That takes care of single quotes which also interpreting all other characters in the value of x literally, with one exception: double quotes. Unfortunately, if double quotes may be part of the value as you indicated in a comment, I don't believe that problem is surmountable without the use of a third party utility. The reason is that, as mentioned above, batch doesn't have a native way of performing string replacements, and the value of x is expanded by cmd before PowerShell ever sees it.
BTW...GOOD QUESTION!!
UPDATE:
In fact, it turns out that it is possible to perform static string replacements in cmd. Duncan added an answer that shows how to do that. It's a little confusing, so I'll elaborate on what's going on in Duncan's solution.
The idea is that %var:hot=cold% expands to the value of the variable var, with all occurrences of hot replaced with cold:
D:\Scratch\soscratch>set var=You're such a hot shot!
D:\Scratch\soscratch>echo %var%
You're such a hot shot!
D:\Scratch\soscratch>echo %var:hot=cold%
You're such a cold scold!
So, in the command (modified from Duncan's answer to align with the OP's example, for the sake of clarity):
powershell G:\test.ps1 -name '%x:'=''%' -data '%y:'=''%'
all occurrences of ' in the variables x and y are replaced with '', and the command expands to
powershell G:\test.ps1 -name 'a''b' -data 'c''d'
Let's break down the key element of that, '%x:'=''%':
The two 's at the beginning and the end are the explicit outer quotes being passed to PowerShell to quote the argument, i.e. the same single quotes that the OP had around %x
:'='' is the string replacement, indicating that ' should be replaced with ''
%x:'=''% expands to the value of the variable x with ' replaced by '', which is a''b
Therefore, the whole thing expands to 'a''b'
This solution escapes the single quotes in the variable value much more simply than my workaround above. However, the OP indicated in an update that the variable may also contain double quotes, and so far this solution still doesn't pass double quotes within x to PowerShell--those still get stripped out by cmd before PowerShell receives the command.
The good news is that with the cmd string replacement method, this becomes surmountable. Execute the following cmd commands after the initial value of x has already been set:
Replace ' with '', to escape the single quotes for PowerShell:
set x=%x:'=''%
Replace " with \", to escape the double quotes for cmd:
set x=%x:"=\"%
The order of these two assignments doesn't matter.
Now, the PowerShell script can be called using the syntax the OP was using in the first place (path to powershell.exe removed to fit it all on one line):
powershell.exe -ExecutionPolicy Bypass "G:\test.ps1 -name '%x' -data '%y'"
Again, if the app can only send a one-line command to cmd, these three commands can be placed in a batch file, and the app can call the batch file and pass the variables as shown above (first bullet in my original answer).
One interesting point to note is that if the replacement of " with \" is done inline rather than with a separate set command, you don't escape the "s in the string replacement, even though they're inside a double-quoted string, i.e. like this:
powershell.exe -ExecutionPolicy Bypass "G:\test.ps1 -name '%x:"=\"' -data '%y'"
...not like this:
powershell.exe -ExecutionPolicy Bypass "G:\test.ps1 -name '%x:\"=\\"' -data '%y'"
I'm slightly unclear in the question whether %x and %y are CMD variables (in which case you should be using %x% to substitute it in, or a substitution happening in your other application.
You need to escape the single quote you are passing to PowerShell by doubling it in the CMD.EXE command line. You can do this by replacing any quotes in the variable with two single quotes.
For example:
C:\scripts>set X=a'b
C:\scripts>set Y=c'd
C:\scripts>powershell .\test.ps1 -name '%x:'=''%' '%y:'=''%'
Name is 'a'b'
Data is 'c'd'
where test.ps1 contains:
C:\scripts>type test.ps1
param($name,$data)
write-output "Name is '$name'"
write-output "Data is '$data'"
If the command line you gave is being generated in an external application, you should still be able to do this by assigning the string to a variable first and using & to separate the commands (be careful to avoid trailing spaces on the set command).
set X=a'b& powershell .\test.ps1 -name '%x:'=''%'
The CMD shell supports both a simple form of substitution, and a way to extract substrings when substituting variables. These only work when substituting in a variable, so if you want to do multiple substitutions at the same time, or substitute and substring extraction then you need to do one at a time setting variables with each step.
Environment variable substitution has been enhanced as follows:
%PATH:str1=str2%
would expand the PATH environment variable, substituting each occurrence
of "str1" in the expanded result with "str2". "str2" can be the empty
string to effectively delete all occurrences of "str1" from the expanded
output. "str1" can begin with an asterisk, in which case it will match
everything from the beginning of the expanded output to the first
occurrence of the remaining portion of str1.
May also specify substrings for an expansion.
%PATH:~10,5%
would expand the PATH environment variable, and then use only the 5
characters that begin at the 11th (offset 10) character of the expanded
result. If the length is not specified, then it defaults to the
remainder of the variable value. If either number (offset or length) is
negative, then the number used is the length of the environment variable
value added to the offset or length specified.
%PATH:~-10%
would extract the last 10 characters of the PATH variable.
%PATH:~0,-2%
would extract all but the last 2 characters of the PATH variable.
I beleive you can escape it with ^:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass "G:\test.ps1 -name ^'%x^' -data ^'%y^'"
Try encapsulating your random single quote variable inside a pair of double quotes to avoid the issue.
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass "G:\test.ps1 -name `"%x`" -data `"%y`""
The problem arises because you used single quotes and the random extra single quote appearing inside the single quotes fools PowerShell. This shouldn't occur if you double quote with backticks, as the single quote will not throw anything off inside double quotes and the backticks will allow you to double/double quote.
Just FYI, I ran into some trouble with a robocopy command in powershell and wanting to exclude a folder with a single quote in the name (and backquote didn't help and ps and robocopy don't work with double quote); so, I solved it by just making a variable for the folder with a quote in it:
$folder="John Doe's stuff"
robocopy c:\users\jd\desktop \\server\folder /mir /xd 'normal folder' $folder
One wacky way around this problem is to use echo in cmd to pipe your command to 'powershell -c "-" ' instead of using powershell "arguments"
for instance:
ECHO Write-Host "stuff'" | powershell -c "-"
output:
stuff'
Couple of notes:
Do not quote the command text you are echoing or it won't work
If you want to use any pipes in the PowerShell command they must be triple-escaped with carats to work properly
^^^|