I am trying to download a Google sheet via a batch file. This works:
powershell -Command "Invoke-WebRequest https://docs.google.com/spreadsheets/d/SPREADSHEET_ID/export?exportFormat=tsv -OutFile output.tsv"
When I specify which sheet/tab I want by adding &gid=1234, this breaks:
powershell -Command "Invoke-WebRequest https://docs.google.com/spreadsheets/d/SPREADSHEET_ID/export?exportFormat=tsv&gid=1234 -OutFile output.tsv"
The error is:
The ampersand (&) character is not allowed. The & operator is reserved
for future use; wrap an ampersand in double quotation marks ("&") to
pass it as part of a string.
How do I wrap the ampersand in quotes without breaking the outer quotes for the Command parameter?
The URL embedded inside the "..." string passed to powershell -Command must be quoted too, because an unquoted & has special meaning to PowerShell too (though in Windows PowerShell it is currently only reserved for future use; in PowerShell Core it can be used post-positionally to run a command as a background job).
The simplest option is to use embedded '...' quoting, as suggested by Olaf, because ' chars. don't need escaping inside "...". '...' strings in PowerShell are literal strings, which is fine in this case, given that the URL contains no variable references.
powershell -Command "Invoke-WebRequest 'https://docs.google.com/spreadsheets/d/SPREADSHEET_ID/export?exportFormat=tsv&gid=1234' -OutFile output.tsv"
If embedded "..." quoting is needed for string interpolation, use \" (sic) to escape the embedded (") chars. (note that inside PowerShell, you'd need to use `" or "" instead):
powershell -Command "Invoke-WebRequest \"https://docs.google.com/spreadsheets/d/SPREADSHEET_ID/export?exportFormat=tsv&gid=1234\" -OutFile output.tsv"
Related
I want to call a small PowerShell (v5.1) script from Windows 10 Pro command line with arguments.
param($ReqURL, $JSONString)
Write-Host $ReqURL # for debug
Write-Host $JSONString # for debug
Invoke-RestMethod -Uri $ReqURL -Method Patch -ContentType "application/json" -Body $JSONString
Calling the script in a PowerShell console works fine:
.\http-patch.ps1 -ReqURL myRESTApi -JSONString '{"lastname": "Bunny"}'
Calling the script from the Windows Command Prompt (cmd.exe) works when I escape the double quotes, like this:
PowerShell .\http-patch.ps1 -ReqURL myRESTApi -JSONString '{\"lastname\": \"Bunny\"}'
But when I use the -File option, it fails because the $JSONString inside the script is equal to '{"lastname"::
PowerShell -File "C:\work\LV Demo\http-patch.ps1" -ReqURL myRESTApi -JSONString '{\"lastname\": \"Bunny\"}'
I assume here are some problems with quotes, but I can't find the right way.
When you use the -File parameter of powershell.exe, the Windows PowerShell CLI, only double quotes (") are recognized as having syntactic function when calling from outside PowerShell, such as from cmd.exe:
Therefore, switch from ' to " (the embedded " chars. require escaping as \" (sic) either way):
:: From cmd.exe
PowerShell -File "C:\work\LV Demo\http-patch.ps1" -JSONString "{\"lastname\": \"Bunny\"}" -ReqURL myRESTApi
By contrast, when you use -Command, '...' strings are recognized, after unescaped " chars. - the only ones with syntactic function during initial command-line parsing - have been stripped, because the resulting tokens are then interpreted as PowerShell code.
For guidance on when to use -Command vs. -File and the differences in resulting parsing, see this answer.
From inside PowerShell, there's rarely a need to to call another PowerShell instance, given that .ps1 files can be invoked directly, in-process.
In cases where you do need to call the CLI from inside PowerShell - say you need to call the CLI of the other PowerShell edition, PowerShell (Core)'s pwsh.exe, from powershell.exe or vice versa - the best choice is to use a script block ({ ... }) (which works only when calling from inside PowerShell), because that:
allows you to focus solely on PowerShell's own quoting and escaping requirements.
supports receiving typed output (objects other than strings), behind-the-scenes XML-based serialization, albeit with limited type fidelity - see this answer.
# From PowerShell
# Note how the " chars. now need NO escaping.
# (If you were to use a "..." string, you'd escape them as `" or "")
PowerShell {
& C:\work\LV Demo\http-patch.ps1" -JSONString '{"lastname": "Bunny"}' -ReqURL myRESTApi
}
While you can call via individual arguments, analogous to how you must call from outside PowerShell, you'll not only lose the benefit of typed output, but you'll also run in a longstanding bug up to PowerShell 7.2.x that requires manual escaping of embedded " chars. as \ when calling external programs - see this answer - as evidenced by one of your own attempts (here, using '...' is perfectly fine, because PowerShell recognizes it):
# !! Note the unexpected need to \-escape the embedded " chars.
PowerShell -File .\http-patch.ps1 -JSONString '{\"lastname\": \"Bunny\"}' -ReqURL myRESTApi
# Variant with double quotes.
# !! Note the *double* escaping: first with ` - for the sake of PowerShell itself -
# !! then with \ due to the bug.
PowerShell -File .\http-patch.ps1 -JSONString "{\`"lastname\`": \`"Bunny\`"}" -ReqURL myRESTApi
I have a powershell script that takes a parameter from the command line.
The passed parameter is a string with new line characters but I cannot get it to work as I wish.
Example running from command line:
powershell.exe -command "& {& 'C:\My Scripts\Script.ps1' "Some Text`nWoo Hoo"}"
The script above is only passed the text Some. If I use single quotes around the passed text the `n newline character is interpreted literally, not as a newline.
I'm sure there is some way to solve this!
thanks
Since your overall command string is (of necessity when calling from cmd.exe) double-quoted, you must escape the embedded double quotes as \" (even though inside PowerShell it is ` that serves as the escape character, as used in the embedded `n escape sequence):
# From cmd.exe
powershell.exe -command "& 'C:\My Scripts\Script.ps1' \"Some Text`nWoo Hoo\""
Also note that there's never a need to use -Command "& { ... }" when calling the PowerShell CLI - just -Command "..." is sufficient.
In your case, embedded double-quoting (\"...\"), i.e. use of an (escaped) expandable string is a must in order to get interpolation of the `n escape sequence to an actual newline.
In cases where a verbatim (literal) string is sufficient, you can bypass the need to escape by using single-quoting ('...').
See the bottom section of this answer for more information about string literals in PowerShell.
I'm trying to run a PowerShell script from Windows cmd.exe. The input to the PowerShell script is a string, which contains newline characters using PowerShell backtick escaping - i.e:
`r`n
For demonstration purposes, the input string is then written to the console, and also dumped to a file.
The issue I have is that when the script is run from cmd.exe using the syntax
powershell.exe script.ps1 "TEST`r`nTEST"
The newline characters in the string are not treated as newline, and are included literally in both the console output and the output text file.
TEST`r`nTEST
However, if I run this from a PowerShell environment, I get the expected result (i.e. the newline characters are parsed correctly, and a newline is inserted in the appropriate location).
TEST
TEST
Similarly, if I pass in \r\n instead of the escaped newline characters through Windows cmd.exe, and do a .replace in the PowerShell script
$date = $data.replace("\r\n","`r`n")
I get the expected output:
TEST
TEST
Is anyone able to shed some light on why this happens?
The test script is as follows:
param([string]$data) # data to send
Write-Host $data
[IO.File]::WriteAllText('d:\temp.txt', $data)
return 0
And the file is called from the command line as:
powershell.exe script.ps1 "TEST`r`nTEST"
The script is running on Windows Server 2012 R2, using PowerShell v4.0
tl;dr
Use -Command and pass the entire PowerShell command as a single string; e.g.:
C:\> powershell -NoProfile -Command "script.ps1 \"TEST`r`nTEST\""
TEST
TEST
Note how the internal " instances are escaped as \", which PowerShell requires when called from the outside (alternatively, for full robustness, use "^"" (sic) in Windows PowerShell and "" in PowerShell (Core) v6+).
In your specific case,
powershell -NoProfile -Command script.ps1 "TEST`r`nTEST" would have worked too, but generally that only works as intended if the string has no embedded spaces.
Given that -Command is the default up to PSv5.1, your command - as currently posted - should work as-is.
As of PowerShell v5.1, arguments passed to powershell.exe from the outside:
ARE subject to interpretation by PowerShell, including string interpolation, by default and when you use -Command (i.e., specifying neither -File nor -Command currently defaults to -Command).
Caveat: The default behavior will change in v6: -File will be the default then - see the relevant change on GitHub.
are NOT subject to interpretation if you use -File to invoke a script - (after potential interpretation by cmd.exe) PowerShell treats all arguments as literal strings.
Caveat: This behavior is currently being discussed with respect to v6, given that it is overtly problematic in at least one case: trying to pass Boolean values.
Optional reading: Why you should pass the entire PowerShell command as a single argument when using -Command:
When you use -Command with multiple arguments, PowerShell essentially assembles them into a single command line behind the scenes before executing it.
Any "..."-quoting around the individual arguments is lost in the process, which can have unexpected results; e.g.:
C:\> powershell -NoProfile -Command "& { $args.count }" "line 1`r`nline 2"
3 # !! "line 1`r`nline 2" was broken into 3 arguments
Given that the outer "..." quoting was removed in the process of parsing the command line, the actual command line that PowerShell ended up executing was:
C:\ PS> & { $args.Count } line 1`r`nline 2
3
To illustrate why, let's look at an equivalent command that uses explicit quoting:
C:\ PS> & { $args.Count } "line" "1`r`nline" "2"
In other words: After the enclosing " were removed, the resulting token was broken into multiple arguments by spaces, as usual.
The parameter will need to be reinterpreted as a PowerShell string. Will this get you down the road?
The reason your -replace did not work is that the original string actually contains a backtick. It needs to be escaped in the search string.
C:\src\t>type p1.ps1
Param([string]$s)
Write-Host $s
$p = Invoke-Expression `"$s`"
Write-Host $p
$p2 = $s -replace "``r``n","`r`n"
Write-Host $p2
C:\src\t>powershell -noprofile -file .\p1.ps1 "TEST`r`nTEST"
TEST`r`nTEST
TEST
TEST
TEST
TEST
Carriage return and Linefeed are bytes with values 13 and 10, you can't write them, you can't see them.
As a convenience, when writing PowerShell code, the language will let you write:
"`r`n"
in a double quoted string, and when processing PowerShell source code (and at no other time), it will read those and replace them with bytes value 13 and 10.
It is this line of code in the PowerShell tokenizer which does it.
There is nothing special about backtick-n to the cmd.exe interpreter, and nothing special about having it in a string - you can put it there in a single quoted string
'`n'
or replacing it in a string - except that you have to note when the replacement happens. e.g. in your comment:
For example, if you pass in 'r'n and then replace 'r'n with 'r'n, the 'r'n is still output literally
Because your code
-replace "`r`n"
becomes
-replace "[char]13[char]10"
and your string passed in from outside contains
`r`n
and they don't match. Backtick-n in a string isn't magic, strings are not all interpreted by the PowerShell engine as PowerShell code, nor are parameters, or anything. And it's only in that context - when you write your -replace code, that is when the swap for actual newline characters happens.
I can download a file to a constant location from cmd console using powershell like this
C:\Users\Weijun>powershell -command "(new-object System.Net.WebClient).DownloadFile('https://example.com/example.zip', 'D:\example.zip')"
but i need to download it to system temp directory,using the powershell variable $env:temp,the following doesn't work.(variable not expanded in single-quote)
powershell -command "(new-object System.Net.WebClient).DownloadFile('https://example.com/example.zip', '$env:temp\example.zip')"
how to pass variable as parameter in method invoke?
UPDATE: it is a typo for $env:temp:,I amend it.
tl;dr:
From outside of PowerShell (cmd.exe - Command Prompt or batch file), use escaped double-quoted strings - \"...\" - inside the overall command string - "..." - passed to powershell -command for strings that need interpolation (e.g., expansion of variable references such as $env:temp):
powershell -command "(new-object System.Net.WebClient).DownloadFile('https://example.com/example.zip', \"$env:temp\example.zip\")"
You not only need to fix the incidental problem that Nkosi points out - removing the extraneous : at the end of $env:temp: - but you must generally either not quote the path at all (argument syntax), or use escaped double quotes (expression syntax or values with shell metacharacters) - single-quoted strings don't work, because PowerShell treats their content as a literal and won't expand the environment-variable reference.[1]
For a description of PowerShell's two fundamental parsing modes - argument syntax vs. expression syntax - see Get-Help about_Parsing.
Simplified examples:
Without quoting (argument syntax):
c:\>powershell.exe -noprofile -command "write-output $env:temp\example.zip"
C:\Users\jdoe\AppData\Local\Temp\example.zip
With double-quoting (expression syntax or if the value contains shell metacharacters such as , or |):
c:\>powershell.exe -noprofile -command "write-output \"$env:temp\example.zip\""
C:\Users\jdoe\AppData\Local\Temp\example.zip
Note: When called from the outside, PowerShell requires embedded " chars. to be escaped as \" - unlike inside PowerShell, where `" must be used.
Applied to your command: with (escaped) double-quoting, which is needed, because you're using .NET method syntax, which is parsed with expression syntax:
powershell -command "(new-object System.Net.WebClient).DownloadFile('https://example.com/example.zip', \"$env:temp\example.zip\")"
[1] If you run the same command from PowerShell, you won't see the problem, because the command as a whole is enclosed in "...", which means that the current PowerShell session will expand $env:temp - before it is passed to powershell.exe.
However, when called from cmd.exe, this interpolation is NOT applied - hence the need to either omit quoting or use embedded (escaped) double-quoting.
I would use Invoke-WebRequest with -OutFile:
powershell -command "Invoke-WebRequest -Uri 'http://example.com/file-to-get.txt' -OutFile $env:temp\file-to-get.txt"
NB this requires PowerShell 3.0 (if I remember correctly).
I have a script like this:
param(
[string]$root,
[string]$bin,
[string]$out,
[string]$zdir
)
echo "args..."
echo "Root: $root", "zdir: $zdir", "out: $out", "bin: $bin"
I invoke it like follows:
powershell.exe -nologo -noprofile -file "C:\Users\arun_jayapal\Documents\Visual Studio 2013\Projects\OutlookCompass\OutlookCompass\zip.ps1" -root "C:\Users\arun_jayapal\Documents\Visual Studio 2013\Projects\OutlookCompass\OutlookCompass\" -zdir "Zip" -out "output.zip" -bin "C:\Users\arun_jayapal\Documents\Visual Studio 2013\Projects\OutlookCompass\OutlookCompass\bin\Debug\"
But my output is quite to the contrary:
C:\code\misc>powershell.exe -nologo -noprofile -file "C:\Users\arun_jayapal\Docu
ments\Visual Studio 2013\Projects\OutlookCompass\OutlookCompass\zip.ps1" -root "
C:\Users\arun_jayapal\Documents\Visual Studio 2013\Projects\OutlookCompass\Outlo
okCompass\" -zdir "Zip" -out "output.zip" -bin "C:\Users\arun_jayapal\Documents\
Visual Studio 2013\Projects\OutlookCompass\OutlookCompass\bin\Debug\"
args...
Root: C:\Users\arun_jayapal\Documents\Visual Studio 2013\Projects\OutlookCompas
s\OutlookCompass" -zdir Zip -out output.zip -bin C:\Users\arun_jayapal\Document
s\Visual
zdir:
out: 2013\Projects\OutlookCompass\OutlookCompass\bin\Debug"
bin: Studio
Your problem is that you are invoking your powershell script from cmd.exe, and for many legacy .exe programs the sequence \" is used to escape a quote into a parameter. Powershell respects this convention so when you invoke Powershell.exe you have to follow this escaping convention.
So:
-root "something\" -zdir "Zip" -out "output.zip" -bin "something\"
Is a single argument containing two double quotes escaped into the string with \". cmd removes the other double quotes, so Zip and output.zip are strings outside the quotes, but as they don't contain any spaces they don't split the argument.
This should work:
-root "something\\" -zdir "Zip" -out "output.zip" -bin "something\\"
Doubling the backslashes before the quotes mean a single backslash is passed through and the quote mark loses its special meaning. Don't double any of the other backslashes though as cmd only regards the backslash as special when a sequence of one or more of them precede a double quote.
Alternatively leave off the trailing backslashes and insert them where needed in your script.
Or just use powershell itself as your main command prompt and ditch cmd altogether.
BTW, if you invoke another copy of powershell from inside itself it is recognised as something special and the arguments are encoded in base64 and passed with the -EncodedCommand command-line option so in that case there is no need to worry about escaped quotes.
Update: The problem still affects Windows PowerShell, but has been fixed in PowerShell (Core) 7+.
Duncan's helpful answer contains the crucial pointer (and helpful background information as well): \" at the end of the parameter values is interpreted as an escaped ", which causes parameter boundaries to be misinterpreted.
Your options are:
Use \\" instead of \" at the end of each parameter value, as Duncan suggests (e.g., "c:\foo bar\\")
Omit the trailing \ from the paths altogether (e.g., "c:\foo bar" instead of "c:\foo bar\")
If you want to avoid having to modify the parameter values, use the following, single-quoted solution (verified on v3).
Single-quoted solution:
This assumes that you're calling this from cmd.exe, whether from a batch file or at the command prompt (inside PowerShell, you could just use & <script> ...)
use single quotes instead of double quotes
use the -Command parameter instead of -File
prefix the script path with & (escaped with ^ to have cmd.exe treat it as a literal)
powershell.exe -noprofile -command ^& 'C:\Users\arun_jayapal\Documents\Visual Studio 2013\Projects\OutlookCompass\OutlookCompass\zip.ps1' -root 'C:\Users\arun_jayapal\Documents\Visual Studio 2013\Projects\OutlookCompass\OutlookCompass\' -zdir 'Zip' -out 'output.zip' -bin 'C:\Users\arun_jayapal\Documents\Visual Studio 2013\Projects\OutlookCompass\OutlookCompass\bin\Debug\'
This works, because by using -Command (and PowerShell's & (call) operator to invoke a script whose path has embedded spaces), the rest of the command line can be quoted PowerShell-style with single-quoted strings that aren't subject to interpretation (not even by the C-style argument parsing that underlies PowerShell's own startup command-line parsing, with the exception of embedded " chars. - see below).
The only characters that need escaping with this approach are:
Use '' to embed a literal ' inside a parameter.
Use \" to embed a literal " inside a parameter.
If you need to escape % chars. to protect them from interpretation by cmd.exe as part of environment-variable references (which you'd have to do irrespective of whether you use single- or double-quoted strings), see this answer of mine.
Note that in order to execute a script in the current directory you must .\-prefix the script filename, as you would inside PowerShell.