Why does backtick in Set-Content not create multiple lines? - powershell

Both of these commands doesn't create multiple line text:
Set-Content .\test.md 'Hello`r`nWorld'
Set-Content .\test.md 'Hello\r\nWorld'
Only this can
Set-Content .\test.md #("Hello`nWorld")
Do you know why is that?

Escape sequences such as `r`n only work inside "...", i.e, expandable (interpolating) strings.
By contrast, '...' strings are verbatim strings that do not interpret their contents - even ` instances are used as verbatim (literally).
Only ` (the so-called backtick) serves as the escape character in PowerShell, not \.
That is, in both "..." and '...' strings a \ is a literal.
(However, \ is the escape character in the context of regexes (regular expressions), but it is then the .NET regex engine that interprets them, not PowerShell; e.g.,
"`r" -match '\r' is $true: the (interpolated) literal CR char. matched its escaped regex representation).
As for what you tried:
It is the fact that "Hello`nWorld" in your last command is a "..." string that made it work.
By contrast, enclosing the string in #(...), the array-subexpression operator, is incidental to the solution. (Set-Content's (positionally implied) -Value parameter is array-valued anyway (System.Object[]), so even a single string getting passed is coerced to an array).
Finally, note that Set-Content by default adds a trailing, platform-native newline to the output file; use -NoNewLine to suppress that, but note that doing so also places no newline between the (string representations of) multiple input objects, if applicable (in your case there's only one).
Therefore (note the -NoNewLine and the trailing `n):
Set-Content -NoNewLine .\test.md "Hello`nWorld`n"
Optional reading: design rationale for PowerShell's behavior:
Why doesn't PowerShell use the backslash (\) as the escape character, like other languages?
Because PowerShell must (also) function on Windows (it started out as Windows-only), use of \ as the escape character - as known from Unix (POSIX-compatible) shells such as Bash - is not an option, given that \ is used as the path separator on Windows.
If \ were the escape character, you'd have to use Get-ChildItem C:\\Windows\\System32 instead of Get-ChildItem C:\Windows\System32, which is obviously impractical in a shell, where dealing with file-system paths is very common.
Thus, a different character had to be chosen, which turned out to be `, the so-called backtick: At least on US keyboards, it is easy to type (just like \), and it has the benefit of occurring rarely (as itself) in real-world strings, so that the need to escape it rarely arises.
Note that the much older legacy shell on Windows, cmd.exe, too had to pick a different character: it chose ^, the so-called caret.
Why doesn't it use single quote and double quote interchangeably, like other languages?
Different languages made different design choices, but in making "..." strings interpolating, but '...' strings not, PowerShell did follow existing languages here, namely that of POSIX-compatible shells such as Bash.
As an improvement on the latter PowerShell also supports embedding verbatim ' inside '...', escaped as '' (e.g., '6'' tall')
Given PowerShell's commitment to backward compatibility, this behavior won't change, especially given how fundamental it is to the language.
Conceptually speaking, you could argue that the aspect of what quoting character a string uses should be separate from whether it is interpolating, so that you'd be free to situationally choose one or the other quoting style for syntactic convenience, while separately controlling whether interpolation should occur.
Thus, hypothetically, PowerShell could have used a separate sigil to make a string interpolating, say $"..." and $'...' (similar to what C# now offers, though it notably only has one string-quoting style).
(As an aside: Bash and Ksh do have this syntax form, but it serves a different purpose (localization of strings) and is rarely used in pratice).
In practice, however, once you know how "..." and '...' work in PowerShell, it isn't hard to make them work as intended.
See this answer for a juxtaposition of PowerShell, cmd.exe, and POSIX-compatible shells with respect to fundamental features.

Related

`doskey /macros:all` produces quoted string I can't get rid of

First, I'm in PowerShell and I've entered the doskey /exename=powershell.exe option.
Second, I did something that I now realize doesn't quite work:
doskey envpath=$env:Path -split ';'
The goal was to have it print the path environment variable (whatever it is at the time I later enter envpath). However, it seems to have evaluated $env:Path while defining the macro, so the macro now appears to be all the paths in my path environment variable followed by '-split ;'. So that's a problem, but only listed here for context. I'll figure that out separately. The purpose of this question (one question per post) is the following:
I was following this and getting something weird...
If I now enter doskey /macros:all I get:
"envpath=C:\WINDOWS\system32;C:\WINDOWS;<etc>;" -split ;
Please note the quotes.
Now, if, per the above-linked other answer, I enter doskey envpath=something (literally) then doskey /macros:all returns:
"envpath=C:\WINDOWS\system32;C:\WINDOWS;<etc>;" -split ;
envpath=something
(which is expected except for the quoted part).
And when I do doskey envpath= it clears/deletes that macro, and doskey /macros:all, returns the first result again.
So my question: What is this entry in the quotes and how do I get rid of that please?
Hopefully I've explained that clearly enough. If confused please feel free to ask for clarification. Thanks in advance for help!
As noted in the answer to your related question,
it's best to avoid use of doskey.exe in PowerShell, because getting it to work requires forgoing PowerShell's own, rich command-line editing experience, by unloading the PSReadLine module first (Remove-Module PSReadLine) - see this answer for background information.
the better alternative is to define a PowerShell function and add it to your $PROFILE file, as shown in the linked answer.
If you want to use doskey.exe nonetheless, define your macro as follows (for PowerShell (Core) 7+, replace powershell.exe with pwsh.exe):
doskey /exename=powershell.exe envpath = `$env:Path -split "';'"
The tokens that make up the PowerShell command must be passed as individual arguments to doskey.exe, and be sure to follow the = with a space.
If you accidentally pass the PowerShell command as a single, quoted argument, doskey.exe stores enclosing "..." as part of the macro and includes these quotes when it expands the macro.
If you additionally also include the macro name in that single, quoted argument, you not only get a virtually unusable macro,[1] you also cannot remove it in-session (neither individually, with doskey /exename=powershell.exe envpath=, nor as part of clearing all macros with Alt-F10) - you must start a new PowerShell session to get rid of it.
Note that the macro name is also included if you attempt partial quoting in PowerShell, e.g., doskey envpath="`$env:Path -split ','" is effectively the same as doskey "envpath=`$env:Path -split ','", due to how PowerShell rebuilds the command line behind the scenes (see below).
To avoid instant expansion of $env:Path, the $ character is preceded by PowerShell's escape character, the so-called backtick (`).
To preserve the '...'-quoting around ;, outer "..." quoting is used.
This is necessary, because PowerShell rebuilds the command line behind the scenes when it invokes external programs, which involves translating '...' quoting into "..." quoting if necessary; that is, irrespective of what quoting was originally used on the PowerShell side, an argument is enclosed in "..." if it contains spaces and used unquoted otherwise; thus, ';' by itself would turn into just ; on the behind-the-scenes command line; an originally partially quoted argument that contains spaces ends up being double-quoted as a whole.
[1] In a macro definition that doskey /macros or doskey /macros:all lists as "envpath=C:\WINDOWS\system32;C:\WINDOWS;<etc>;" -split ;, the macro name is "envpath verbatim, i.e. including the opening ". The - then unbalanced - closing " is retained in the text to expanded to.

Split text after each end of line [duplicate]

This question already has answers here:
Powershell: split string with newline, then use -contains
(3 answers)
Closed 1 year ago.
I have a script that works perfectly fine with Powershell 5.x, but does not work anymore on Powershell Core (7.2.1)
The problem happens when I try to split a text (copy&past from an email)..
It all comes down to this part of the code:
$test="blue
green
yellow
"
#$test.Split([Environment]::NewLine)
$x = $test.Split([Environment]::NewLine)
$x[0]
$x[1]
In Powershell 5 the value for $x[0]==blue and $x[1]==green
But in Powershell Core the split doesn't do anything and $x[1] is "non existent".
In Powershell 7 the line breaks are handled differently (that's at least what I assume), but I couldn't find a solution to it..
I tried it with changing the code to
$rows = $path.split([Environment]::NewLine) and $path.Split([System.Environment]::NewLine, [System.StringSplitOptions]::RemoveEmptyEntries) but that doesn't change anything..
Also, when I use a "here-string"
$test = #'
green
yellow
blue
white
'#
$x= $test -split "`r`n", 5, "multiline"
Everything excepts $x[0] is empty (i.e $x[2])
I was already looking here: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_split?view=powershell-7.2
And here: powershell -split('') specify a new line
And here: WT: Paste multiple lines to Windows Terminal without executing
So far I have not found a solution to my problem.
Any help is appreciated.
EDIT: I found a hint about that problem, but don't understand the implications of it yet: https://n-v-o.github.io/2021-06-10-String-Method-in-Powershell-7/
EDIT 2:
Thanks everyone for participating in answering my question.
First I thought I'm going to write a long explanation why my question is different then the duplicated answer from #SantiagoSquarzon. But while reading the answers to my question and the other question I noticed I was doing something differently..
Apparently there is something differnt when I use
$splits = $test -split '`r?`n' # doesn't work in 5.1 and 7.2.1
$splits = $test -split '\r?\n' # works in 5.1 and 7.2.1 as suggested from Santiago and others
BUT
$splits = $test.Split("\r?\n") # doesn't work in 5.1 and 7.2.1
$splits = $test.Split("`r?`n") # doesn't work in 5.1 and 7.2.1
$splits = $test.Split([char[]]"\r\n") # doesnt' work in 7.2.1
$splits = $test.Split([char[]]"`r`n") # works in 7.2.1
tl;dr:
Use -split '\r?\n to split multiline text into lines irrespective of whether Windows-format CRLF or Unix-format LF newlines are used (it even handles a mix of these formats in a single string).
If you additionally want to handle CR-only newlines (which would be unusual, but appears to be the case for you), use -split '\r?\n|\r'
On Windows, with CRLF newlines only, .Split([Environment]::NewLine) only works as intended in PowerShell (Core) 7+, not in Windows PowerShell (and, accidentally, in Windows PowerShell only with CR-only newlines, as in your case.) To explicitly split by CR only, .Split("`r") would happen to work as intended in both editions, due to splitting by a single character only.
# Works on both Unix and Windows, in both PowerShell editions.
# Input string contains a mix of CRLF and LF and CR newlines.
"one`r`ntwo`nthree`rfour" -split '\r?\n|\r' | ForEach-Object { "[$_]" }
Output:
[one]
[two]
[three]
[four]
This is the most robust approach, as you generally can not rely on input text to use the platform-native newline format, [Environment]::NewLine; see the bottom section for details.
Note:
The above uses PowerShell's -split operator, which operates on regexes (regular expressions), which enables the flexible matching logic shown above.
This regex101.com page explains the \r?\n|\r regex and allows you to experiment with it.
By contrast, the System.String.Split() .NET method only splits by literal strings, which, while faster, limits you to finding verbatim separators.
The syntax implications are:
Regex constructs such as escape sequences \r (CR) and \n (LF) are only supported by the .NET regex engine and therefore only by -split (and other PowerShell contexts where regexes are being used); ditto for regex metacharacters ? (match the preceding subexpression zero or one time) and | (alternation; match the subexpression on either side).
Inside strings (which is how regexes must be represented in PowerShell, preferably inside '...'), these sequences and characters have no special meaning, neither to PowerShell itself nor to the .Split() method, which treats them all verbatim.
By contrast, the analogous escape sequences "`r" (CR) and "`n" (LF) are PowerShell features, available in expandable strings, i.e. they work only inside "..." - not also inside verbatim strings, '...' - and are expanded to the characters they represent before the target operator, method, or command sees the resulting string.
This answer discusses -split vs. .Split() in more depth and recommends routine use of -split.
As for what you tried:
Use [Environment]::NewLine only if you are certain that the input string uses the platform-native newline format. Notably, multiline string literals entered interactively at the PowerShell prompt use Unix-format LF newlines even on Windows (the only exception is the obsolescent Windows-only ISE, which uses CRLF).
String literals in script files (*.ps1) use the same newline format that the script is saved in - which may or may not be the platform's format.
Additionally, as you allude to in your own answer, the addition of a string parameter overload in the System.String.Split() method in .NET Core / .NET 5+ - and therefore PowerShell (Core) v6+ - implicitly caused a breaking change relative to Windows PowerShell: specifically, .Split('ab') splits by 'a' or 'b' - i.e. by any of the individual characters that make up the string - in Windows PowerShell, whereas it splits by the whole string, 'ab', in PowerShell (Core) v6+.
Such implicit breaking changes are rare, but they do happen, and they're outside PowerShell's control.
For that reason, you should always prefer PowerShell-native features for long-term stability, which in this case means preferring the -split operator to the .Split() .NET method.
That said, sometimes .NET methods are preferable for performance reasons; you can make them work robustly, but only if carefully match the exact data types of the method overloads of interest, which may require cast; see below.
See this answer for more information, including a more detailed explanation of the implicit breaking change.
Your feedback on -split '\r?\n' not working for you and the solutions in your own answer suggest that your input string - unusually - uses CR-only newlines.
Your answer's solutions would not work as expected with Windows-format CRLF-format text, because splitting would happen for each CR and LF in isolation, which would result in extra, empty elements in the output array (each representing the empty string "between" a CRLF sequence).
If you did want to split by [Environment]::NewLine on Windows - i.e. by CRLF - and you wanted to stick with the .Split() method, in order to make it work in Windows PowerShell too, you'd need to call the overload that expects a [string[]] argument, indicating that each string (even if only one) is to be used as a whole as the separator - as opposed to splitting by any of its individual characters:
# On Windows, split by CRLF only.
# (Would also work on Unix with LF-only text.)
# In PowerShell (Core) 7+ only, .Split([Environment]::NewLine) would be enough.
"one`r`ntwo`r`nthree".Split([string[]] [Environment]::NewLine, [StringSplitOptions]::None) |
ForEach-Object { "[$_]" }
Output:
[one]
[two]
[three]
While this is obviously more ceremony than using -split '\r?\n', it does have the advantage of performing better - although that will rarely matter. See the next section for a generalization of this approach.
Using an unambiguous .Split() call for improved performance:
Note:
This is only necessary if -split '\r?\n' or -split '\r?\n|\r' turns out to be too slow in practice, which won't happen often.
To make this work robustly, in both PowerShell editions as well as long-term, you must carefully match the exact data types of the .Split() overload of interest.
The command below is the equivalent of -split '\r?\n|\r', i.e. it matches CRLF, LF, and CR newlines. Adapt the array of strings for more restrictive matching.
# Works on both Unix and Windows, in both PowerShell editions
"one`r`ntwo`nthree`rfour".Split(
[string[]] ("`r`n", "`n", "`r"),
[StringSplitOptions]::None
) | ForEach-Object { "[$_]" }
The reason: When pasting text into the terminal, it matters which terminal you are using. The default powershell 5.1, ISE terminals, and most other Windows software separates new lines with both carriage return \r and newline \n characters. We can check by converting to bytes:
# 5.1 Desktop
$test = "a
b
c"
[byte[]][char[]]$test -join ','
97,13,10,98,13,10,99
#a,\r,\n, b,\r,\n, c
Powershell Core separates new lines with only a newline \n character
# 7.2 Core
$test = "a
b
c"
[byte[]][char[]]$test -join ','
97,10,98,10,99
On Windows OS, [Environment]::NewLine is \r\n no matter which console. On Linux, it is \n.
The solution: split multiline strings on either \r\n or \n (but not on only \r). The easy way here is with regex like #Santiago-squarzon suggests:
$splits = $test -split '\r?\n'
$splits[0]
a
$splits[1]
b
Thanks to this site I found a solution:
https://n-v-o.github.io/2021-06-10-String-Method-in-Powershell-7/
In .NET 4, the string class only had methods that took characters as
parameter types. PowerShell sees this and automagically converts it,
to make life a little easier on you. Note there’s an implied ‘OR’ (|)
here as it’s an array of characters.
Why is PowerShell 7 behaving differently? In .NET 5, the string class
has some additional parameters that accept strings. PowerShell 7 does
not take any automatic action.
In order to fix my problem, I had to use this:
$test.Split("`r").Split("`n") #or
$test.Split([char[]]"`r`n")

Powershell escaping quotation marks in a variable that is used in another variable

I've found plenty of post explaining how to literally escape both single and double quotation marks using either """" for one double quotation mark, or '''' for a single quotation mark (or just doing `"). I find myself in a situation where I need to search through a list of names that is input in a different query:
Foreach($Username in $AllMyUsers.Username){
$Filter = "User = '$Username'"
# do some more code here using $Filter
}
The problem occurs when I reach a username like O'toole or O'brian which contains quotation marks. If the string is literal I could escape it with
O`'toole
O''brian
etc.
But, since it's in a loop I need to escape the quotation mark for each user.
I tried to use [regex]::Escape() but that doesn't escape quotation marks.
I could probably do something like $Username.Replace("'","''") but it feels like there should be a more generic solution than having to manually escape the quotation marks. In other circumstances I might need to escape both single and double, and just tacking on .Replace like so $VariableName.Replace("'","''").Replace('"','""') doesn't feel like it's the most efficient way to code.
Any help is appreciated!
EDIT: This feels exactly like a "how can I avoid SQL injection?" question but for handling strings in Powershell. I guess I'm looking for something like mysqli_real_escape_string but for Powershell.
I could probably do something like $Username.Replace("'","''") but it feels like there should be a more generic solution than having to manually escape the quotation marks
As implied by Mathias R. Jessen's comment, there is no generic solution, given that you're dealing with an embedded '...' string, and the escaping requirements entirely depend on the ultimate target API or utility - which is unlikely to be PowerShell itself (where ' inside a '...' string must be doubled, i.e. represented as '').
In the case at hand, where you're passing the filter string to a System.Data.DataTable's .DataView's .RowFilter property, '' is required as well.
The conceptually cleanest way to handle the required escaping is to use -f, the format operator, combined with a separate string-replacement operation:
$Filter = "User = '{0}'" -f ($UserName -replace "'", "''")
Note how PowerShell's -replace operator - rather than the .NET [string] type's .Replace() method - is used to perform the replacement.
Aside from being more PowerShell-idiomatic (with respect to: syntax, being case-insensitive by default, accepting arrays as input, converting to strings on demand), -replace is regex-based, which also makes performing multiple replacements easier.
To demonstrate with your hypothetical .Replace("'","''").Replace('"','""') example:
#'
I'm "fine".
'# -replace '[''"]', '$0$0'
Output:
I''m ""fine"".

Unable to move forward using cd

I'm having a problem moving forward through a path with PowerShell. I am able to move up the directory but not down. Here's the situation:
I open PowerShell and type in the "pwd" command and it shows that I am currently in PS C:\Users\Robert Inspiron14>
I type the command "cd.." and now I move to PS C:\Users>
I then attempt to change directories by typing: "cd C:\Users\Robert Inspiron14" and I am unable to. Unfortunately, I can't post a picture yet due to lack of reputation.
I'm able to perform the change in CMD but not PowerShell. Also, I don't know how to change the User from "Robert Inspiron14" to just "Robert". Any help is appreciated!
Before PowerShell can execute your cd command, it needs to parse it, and PowerShell's parser interprets your command like this:
cd C:\Users\Robert Inspiron14
\/ \_____________/ \________/
Command Name | |
argument 1 |
argument 2
In other words, C:\Users\Robert and Inspiron14 are interpreted as separate arguments.
Neither argument is a path to a valid directory, so cd (or rather Set-Location for which cd is an alias) throws an error.
You can force PowerShell to recognize C:\Users\Robert Inspiron14 as a single string argument by qualifying its boundaries using quotation marks (both " and ' will work):
cd 'C:\Users\Robert Inspiron14'
You can read more about how PowerShell parses command expressions in the about_Parsing help topic
To complement Mathias R. Jessen's helpful answer with more background information:
Quoting an argument that contains spaces is a general syntactic necessity, in all shells, because unquoted spaces are used to separate multiple arguments.
It isn't only spaces that require quoting, but any of PowerShell's so-called metacharacters (characters that, when used unquoted, have syntactic function); for instance, passing the path to a directory literally named a;b requires quoting as well, given that ; would otherwise be interpreted as a statement separator.
There are multiple quoting styles:
Since your path value is a literal - it contains no variable references or expressions - a verbatim (single-quoted) string ('...') is the best choice.
cd 'C:\Users\Robert Inspiron14'
If your path contains variables or subexpressions, you must use an expandable (double-quoted) string ("...")[1]
cd "$HOME\Documents"
Another, less common solution is to individually escape the space characters with `, the so-called backtick, PowerShell's escape character:
cd C:\Users\Robert` Inspiron14
Also note:
PowerShell's tab-completion automatically applies quoting as necessary.
cd.. is the name of a built-in function in PowerShell, whose sole purpose is to emulate cmd.exe's (questionably permissive) behavior (see below); the function performs a syntactically correct Set-Location .. call (verify by executing ${function:cd..}), with a space separating the command name from its argument.
Contrast with cmd.exe:
Unfortunately, cmd.exe's built-in cd command decided not to enforce its usual syntax rules, and enabled calls such as cd C:\Program Files.
It should never have done that: While convenient at first glance, it constitutes a problematic exception from the usual rules that invites confusion.
Note that cmd.exe's tab completion properly quotes arguments that contain spaces.
Similarly, cd.. was unfortunately allowed as as syntactically exceptional alternative to the correct cd .. - see the comments on this answer for details.
[1] Note "..."-quoting isn't strictly necessary if you use variable references in a path, as long as any literal components do not require quoting; e.g., $HOME\foo is fine without quoting, whereas the " around "$HOME\foo bar" are required. With subexpressions ($(...)), the rules get more complicated, so the simplest approach is to always use "..."-quoting with them.

Execute Windows Powershell command copy text to clipboard (incl. `r`n ')

I am trying to execute a powershell command to copy text to the windows clipboard including carriage returns and ALL special characters. I can execute the command ok using:
powershell.exe -command Set-Clipboard 'TEXT'
This is not in the powershell console directly so syntax differs.
I was using double quotes around text, substituting carriage returns in original text with `r`n and escaping all other special characters with `
It worked up until I got to a single ' which I understand is used by powershell to mean a literal text string.
So I changed approaches and wrapped un-escaped text in single quotes (except substituting 1 ' for 2 ''). Of course `r`n within the single quoted text are interpreted literally so doesn't work. I have tried stringing them together outside single quoted text like:
'some text here' "`r`n" 'more text here'
This works in the console but not in the command. Tried adding + either side but still does not work.
User "TessellatingHeckler" suggested -EncodedCommand but unfortunately I am unable to produce any version of base 64 encoded strings (to include in the command) which match the same string encoded via the PS console. So that is not going to work.
I have been attempting to simply substitute carriage returns in the original text with an obscure string, wrap the text in single quotes (literal) and then substitute it back to `r`n within PS. I have gotten the substitution to work in the console directly but cannot figure out how to actually send it as a command.
powershell.exe -command Set-Clipboard $Str = 'this is a test--INSERT_CRLF_HERE--1234'; $Car = '--INSERT_CRLF_HERE--'; $clr = "`r`n"; $Str = $Str -replace $Car, $clr
Can the above command be modified to work? Is it possible to achieve the intended outcome without writing to a temp file? It is preferable to be able to use single quoted text blocks as it is more robust and lightweight than trying to escape everything (even spaces) in the original text.
I was informed about a rather tidy solution by Rob Simmers on another forum, which I am currently employing.
Original Text:
Test text
!##$%^&*()_+-=[]\{}|;':",./<>?
String with 5x characters substituted ({ = {{, } = }}, ' = '', crlf = {0}, " = {1}):
Test text{0}!##$%^&*()_+-=[]\{{}}|;'':{1},./<>?
Powershell.exe command:
powershell.exe -command "$str = 'Test text{0}!##$%^&*()_+-=[]\{{}}|;'':{1},./<>?' -f [System.Environment]::NewLine, [Char] 0x22 ; Set-Clipboard $str"
Output (literal text placed on the clipboard) - same as the input, as desired:
Test text
!##$%^&*()_+-=[]\{}|;':",./<>?
As an aside: A fully robust solution that works in any invocation scenario would indeed require use of -EncodedCommand with a string that is the Base64 encoding of the command string's UTF16-LE byte representation - but you've stated that creating such a string is not an option for you.
If you were to call from PowerShell, you could more simply use a script block (see bottom).
Update: The OP's own answer now contains a robust, if nontrivial, solution based on careful string substitutions.
The answer below may still be of interest for simpler scenarios and for background information on quoting requirements and pitfalls.
Using the scenario from your question, this simpler solution should do (verified from cmd.exe - we still don't know where you're calling PowerShell from, but I expect it to work if there's no shell involved):
powershell.exe -command "Set-Clipboard \"this is`r`na test\""
As for other special characters:
' can be embedded as-is - no escaping needed
" requires escaping as `\"
$ as `$, but only if you want it to be treated as a literal (the same escaping that would apply in a regular double-quoted PowerShell string)
If your use case requires passing an arbitrary preexisting string, you'd have to employ string substitution to perform the above escaping - including replacing embedded newlines with literal `r`n.
If there is no shell involved (such as with subprocess.check_output() from Python), the above rules should suffice and make for a robust solution (assuming you only use printable characters and there are no character-encoding issues).
From cmd.exe, however, a fully robust solution that doesn't use -EncodedCommand requires extra, nontrivial work, due to its lack of proper parsing of embedded double quotes:
The following cmd.exe metacharacters typically require ^-escaping, but sometimes the mustn't be escaped:
& | < >
In the following example, & requires ^-escaping:
powershell.exe -command "Set-Clipboard \"this is`r`na ^& test\""
However, if your string also has embedded (and escaped) " chars., whether these characters require^-escaping depends on their placement relative to the embedded "; note how in the following example & need not be ^-escaped and indeed must not be, because the ^ would then become part of the string:
powershell.exe -command "Set-Clipboard \"this is`r`na 3`\" & test\""
Anticipating these variations algorithmically is a nontrivial undertaking.
Additionally, if your string had %...% tokens that look like cmd.exe-style environment variables - e.g., %FOO% - but you want to treat them as literals, the % cannot be escaped.
There is a workaround that may work, but it again depends on the presence and placement of embedded " chars.
In the following example, the "^ disrupter" trick can be used to prevent expansion of %OS%:
powershell.exe -command "Set-Clipboard \"do not expand: %^OS%\""
However, if an embedded " precedes it, the workaround becomes ineffective (the ^ is retained), and there's no direct fix that I know of in this scenario:
powershell.exe -command "Set-Clipboard \"do not expand: 3`\" %^OS%\""
You'd have to split the string into multiple pieces to break the %OS% token apart and concatenate the pieces via a PowerShell expression:
powershell.exe -command "Set-Clipboard (\"do not expand: 3`\" %\" + \"OS%\")"
Algorithmically, you could use placeholder chars. that you then replace as part of the PowerShell command, a technique you've used in your own answer.
As an aside:
Extra escaping would be required if you wanted to execute this from PowerShell:
powershell.exe -command "Set-Clipboard \`"this is`r`na test\`""
However, from PowerShell it's not hard to construct a Base64-encoded string to pass to
-EncodedCommand, but there's an even easier method: use a script block to pass your command; no extra quoting requirements apply in that case, because -EncodedCommand is then automatically used behind the scenes (along with -OutputFormat Xml).
Do note that this only works from within PowerShell.
powershell.exe -command { Set-Clipboard "this is`r`na test" }