I have a daily process that downloads and parses pipe-delimited files into a database. Sometimes these files contain a single carriage return (no newline) at the end of one of the fields within a row, which breaks my process. My scheduler (not a windows scheduler, internal to our company) allows me to run batch or PowerShell processes on these files before loading them into the database, so I want to trim out any instance of orphaned carriage returns at the end of a field (\r|).
This PowerShell command works when run manually:
(Get-Content CarriageReturnTest.txt -Raw).Replace("`r|","|") | Set-Content CarriageReturnTest.txt -Force
However when I put this in the scheduler, I get an error that '","' is not recognized as an internal or external command. I realized the scheduler must be running a windows command of PowerShell "{command}" so the double quotes are making the command line think that I'm running several different commands. I tried replacing the double quotes in the PowerShell command with single quotes like so:
(Get-Content CarriageReturnTest.txt -Raw).Replace('`r|','|') | Set-Content CarriageReturnTest.txt -Force
This runs through the scheduler, but doesn't actually trim anything out of the file because PowerShell interprets the ` as an escape character when it's enclosed in single quotes.
I've tried escaping the double quotes with "", `", \", and ^" but nothing seems to work.
I realize there's some longer batch scripts I could write to handle this but ideally this would be a one liner to fit in the scheduler. I've also looked into saving the script as a ps1 file and running that with the file paths as arguments but I haven't gotten that to work either.
Based off what #JosefZ said I tried this and it worked:
(Get-Content CarriageReturnTest.txt -Raw) -replace('\r\|','|') | Set-Content CarriageReturnTest.txt -Force
The -replace let me use regex in single quotes '\r' instead of "`r"in double, which kept me from escaping
Related
I want to replace some text in a file after Jenkins does its build step.
I have the code below in my post build step.
powershell.exe -Command(Get-Content "./PublishedWebsites/app.config") |
Foreach-Object {$_.replace({{MyDummValue}}, "${env.BUILD_NUMBER}")} |
Set-Content "./PublishedWebsites/app.config"
The first part works (Get-Content "./PublishedWebsites/app.config") up until the pipe ("|").
After the pipe it throws an error that Foreach-Object is not recognized by an internal or external command.
I am thinking it has something to do with the escape sequence and the syntax.
So I have tried this as well
"""powershell.exe -Command(Get-Content "./PublishedWebsites/app.config") |
Foreach-Object {$_.replace({{MyDummValue}}, "${env.BUILD_NUMBER}")} |
Set-Content "./PublishedWebsites/app.config"""
Then for some reason jenkins is trying to evaluate the $_.replace({{MyDummValue}} before the post build step is even reached and gives me the error of 'Cannot get property replace on null object'
So I am at a loss. All I want to do is on a post build step, rename a value in a file with the Build Number.
When calling PowerShell (or other) scripts from Groovy, always use single-quoted (or triple-single-quoted) string literals for the script. With double-quoted (or triple-double-quoted) strings, Groovy tries to interpolate the string, so you'd have to escape $ as \$, to use it literally. While this is just a nuisance, double-quoted strings are also prone to script injection, where a pipeline user might run arbitrary PowerShell scripts, when pipeline parameters are interpolated by Groovy within the script.
Another mistake: the value {{MyDummValue}} isn't quoted.
Also, I don't know why you are calling powershell.exe explicitly. Use the powershell step instead:
post {
always { // or one of the other available sections
powershell '''
Get-Content "./PublishedWebsites/app.config" |
Foreach-Object {$_.replace("{{MyDummValue}}", $env:BUILD_NUMBER)} |
Set-Content "./PublishedWebsites/app.config"
'''
}
}
I have one powershell command inside that both Double quotes and single quotes are present and it's needed for sure in order to make the command execute successfully in dos prompt. I am not sure how to make it escape in pipeline script.
bat "powershell -Command "(Get-Content "${Physical_FolderLoc}\\${Stord_Process_Name}.txt") | ForEach-Object { $_ -replace [RegEx]::Escape('E:\\config'), 'I:\\config' } | Set-Content "${Physical_FolderLoc}\\${Stord_Process_Name}.txt" " "
In the above command you can see the second last " is ending quote for Get Content and last one is for bat command.
I tried the above command with triple slash but getting groovy error.
groovy.lang.MissingPropertyException: No such property: _ for class: groovy.lang.Binding
at groovy.lang.Binding.getVariable(Binding.java:63)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty
Please help me to resolve this.
Change each internal double quoted string, eg:
"${Physical_FolderLoc}\\${Stord_Process_Name}.txt"
to:
('{0}\\{1}.txt' -f $Physical_FolderLoc, $Stord_Process_Name)
I got the exact issue why the command was not escaping. There is a stand alone $ symbol in the powershell command which is also a system character for groovy, so that need to be escaped as well.(The $ character which is after ForEach-Object {). It's now prefixed with \, so escaped.
Now the complete command is :
bat "powershell -Command \"(Get-Content ${Physical_FolderLoc}\\${Stord_Process_Name}.txt) | ForEach-Object { \$_ -replace [RegEx]::Escape('E:\\config'), 'I:\\config' } | Set-Content ${Physical_FolderLoc}\\${Stord_Process_Name}.txt\" "
Though no one had tried to help much, but still thanks a lot for the suggestion.
I'm currently working on an automated setup program. It has to execute SQL command. I decided to create it with a simple batch file which allows me to use commands like sqlcmd or Powershell.
I'm working on a XML file that will get insert into another file. First, I have to replace some strings in it. I successfully used the -replace command like that :
powershell -Command "(gc source_file.xml) -replace 'utf-8', 'utf-16' | sc updated_file.xml"
Now, the updated_file.xml still contains single quotes that I'd like to replace by '' (two single quotes). I'd like to use the replace command again to make the substitution. I tried to escape the single quote character but it didn't work :
powershell -Command "(gc source_file.xml) -replace "`'", "`'`'" | sc updated_file.xml"
The error shows that the single quote isn't escaped. What's more, in my case, it seems that the replace command only reads string between single quotes. I also tried with a regex::Escape("`'") without success.
Do you have any idea ?
Thanks a lot !
try this:
powershell -Command "(gc source_file.xml) -replace 'utf-8', 'utf-16' -replace '''', '''''' | sc updated_file.xml"
For cmd.exe you have to escape inner double quotes with a backslash, so this should work:
powershell -NoP -C "(gc source_file.xml) -replace 'utf-8','utf-16' -replace \"'\",\"''\" | Set-Content updated_file.xml"
Hint: the alias sc is removed in upcoming releases of PowerShell, so avoid it.
Also changing content text utf8 => utf-16 won't change the encoding (possibly append -Encoding UTF8 to the gc.
I'm currently using the following command to do a find and replace in a file (I googled the code and just added the -encoding UTF8 because otherwise Apache refused to read the file as a php file):
powershell -Command "(gc app.php) -replace '/../', '/../new_project_name/' | Out-File -encoding UTF8 app.php"
The code is working as long as the folder is "new_project_name". new_project_name should actually be a variable name though. E.g. if SET new_project_name=example then the powershell would be as follows:
powershell -Command "(gc app.php) -replace '/../', '/../example/' | Out-File -encoding UTF8 app.php"
I've tried passing a variable to the powershell command but either get errors or no changes are made.
Variables defined in your batch script are available in the env: scope in PowerShell commands started from the batch script (as it inherits the parent script's environment):
Set "new_name=example"
powershell -Command "(gc app.php) -replace '/../', \"/../$env:new_name/\" | ..."
Note that if you use the variable inside a (PowerShell) string you must put that string in double quotes. Single quotes will not work. Escape the nested double quotes with backslashes for CMD.
I am calling PowerShell from within a Java application (through the Windows command prompt) to read various file attributes.
For example,
powershell (Get-Item 'C:\Users\erlpm\Desktop\Temp\s p a c e s.txt').creationTime
I am enclosing the file path in single quotes, because it may contain spaces.
It worked fine, until I encountered a file path containing square brackets, which seem to be interpreted as a wildcard character. I managed to solve it by adding the -literalPath parameter:
powershell (Get-Item -literalpath 'C:\Users\erlpm\Desktop\Temp\brackets[].txt').creationTime
So far, so good... But file paths may also contain single quotes, dollar signs, ampersands, etc., and all these characters seem to have a specific function in PowerShell for which the -literalPath parameter does not seem to work.
I tried enclosing the path with double quotes or escaping with the `character, but that did not solve my problem either :-(
Any suggestions on how to pass a file path to PowerShell which may contain spaces, single quotes, square brackets, ampersands, dollar signs, etc.?
Someone here already showed me how to get it working from within PowerShell, but somehow the answer has been removed(?).
Anyway, I did create a file called $ & ' [].txt.
This works form within PowerShell (needed to escape the &):
Get-Item -LiteralPath "C:\Users\erlpm\Desktop\Temp\`$ & ' [].txt"
Directory: C:\Users\erlpm\Desktop\Temp
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 2012-08-23 14:22 0 $ & ' [].txt
But when I execute the same PowerShell command through the Windows command prompt, ...
powershell Get-Item -LiteralPath "C:\Users\erlpm\Desktop\Temp\`$ & ' [].txt"
... I get this error:
Ampersand not allowed. The & operator is reserved for future use; use "&" to pass ampersand as a string.
At line:1 char:55 \
Get-Item -LiteralPath C:\Users\erlpm\Desktop\Temp`$ & <<<< ' [].txt \
CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException \
FullyQualifiedErrorId : AmpersandNotAllowed
Using the -command parameter and putting the PowerShell command between {} gives exactly the same error message ...
powershell -command {Get-Item -LiteralPath "C:\Users\erlpm\Desktop\Temp\`$ & ' [].txt"}
This is really a question about cmd.exe string escaping, then. Combining cmd and PowerShell string escaping is a total nightmare. After quite a few attempts, I got your specific example to work:
powershell.exe -nologo -noprofile -command ^&{ dir -LiteralPath ^"""".\`$ & ' [].txt"" }
Directory: C:\Users\latkin
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 8/23/2012 8:46 AM 0 $ & ' [].txt
Totally intuitive, right?
You can spend 15 minutes wrestling with escape sequences every time, or you can try some other approaches.
Put the file name in a text file, and read it out of there.
powershell.exe -command "&{dir -literal (gc .\filename.txt) }"
or
Use a script
powershell.exe -file .\ProcessFiles.ps1 # In processfiles.ps1 you are in fully powershell environment, so escaping is easier
or
Use -EncodedCommand
See powershell.exe -? (last item shown) or http://dmitrysotnikov.wordpress.com/2011/07/06/passing-parameters-to-encodedcommand/
If you can write the path to a temporary .txt file, the following works OK:
(Get-Item -literalpath (gc 'C:\Temp\path.txt')).creationTime
The file # C:\Temp\path.txt contains the path with special characters in it, other than this I think you would have to escape each special character on a per path basis.
In addition to the hack above, it appears PowerShell V3 may help out here with the addition of a new 'magic parameter' language feature. Specifically, see the section headed 'Easier Reuse of Command Lines From Cmd.exe' here and because I hate link-rot, here it is, shamelessly reproduced below:
Easier Reuse of Command Lines From Cmd.exe
The web is full of command lines written for Cmd.exe. These commands
lines work often enough in PowerShell, but when they include certain
characters, e.g. a semicolon (;) a dollar sign ($), or curly braces,
you have to make some changes, probably adding some quotes. This
seemed to be the source of many minor headaches.
To help address this scenario, we added a new way to “escape” the
parsing of command lines. If you use a magic parameter --%, we stop
our normal parsing of your command line and switch to something much
simpler. We don’t match quotes. We don’t stop at semicolon. We
don’t expand PowerShell variables. We do expand environment variables
if you use Cmd.exe syntax (e.g. %TEMP%). Other than that, the
arguments up to the end of the line (or pipe, if you are piping) are
passed as is. Here is an example:
echoargs.exe --% %USERNAME%,this=$something{weird}
Arg 0 is <jason,this=$something{weird}>
This thread is 5 years old so maybe times have changed, but the current answer that worked for me was to use the backtick (`) symbol to escape the special character.
In my case, it was a dollar sign in a directory path that was failing. By putting a backtick before the dollar sign, everything worked.
Before:
$dir = "C:\folder$name\dir" # failed
After:
$dir = "C:\folder`$name\dir" # succeeded