Is there anyway for PowerShell to output a file without ANSI control characters like color control, e.x. [1;xxm or [xm], before outputting to a file,
[1;35mStarting selenium server... [0m[1;35mstarted - PID: [0m 22860
[0;36m[Signin Test] Test Suite[0m
[0;35m================================[0m
Running: [0;32mstep 1 - launch the browser[0m
[1;35m[40mINFO[0m [1;36mRequest: POST /wd/hub/session[0m
The output displays correctly with color in PowerShell terminal, (I've used chcp, not working)
You could try something like this:
... | ForEach-Object {
$_ -replace '\[\d+(;\d+)?m' | Add-Content 'C:\path\to\output.txt'
$_
}
or wrap it in a function:
function Tee-ObjectNoColor {
[CmdletBinding()]
Param(
[Parameter(Position=0, Mandatory=$true, ValueFromPipeline=$true)]
[string]$InputObject,
[Parameter(Position=1, Mandatory=$true)]
[string]$FilePath
)
Process {
$InputObject -replace '\[\d+(;\d+)?m' | Add-Content $FilePath
$InputObject
}
}
... | Tee-ObjectNoColor -FilePath 'C:\path\to\output.txt'
For the windows system one could use the Replace command available as a part of Powershell 3.0.The powershell makes use of regex expression that helps to replace the ANSI Color codes. (In case of UNIX one could use the sed command )
Using Regex
Below is the standard Regex for removing ANSI color codes (can be used in Linux and windows both)
'\x1b\[[0-9;]*m'
\x1b (or \x1B) is the escape special character
(sed does not support alternatives \e and \033)
\[ is the second character of the escape sequence
[0-9;]* is the color value(s) regex
m is the last character of the escape sequence
Final Command
I am here outputting the logs of docker to a log file.One could do the same for other commands
docker logs container | ForEach-Object { $_ -replace '\x1b\[[0-9;]*m','' }| Out-File -FilePath .\docker-logs.log
ForEach-Object refers to each object from the pipped stream and $_ refers to current object.
The above command will remove the special characters like [1;35m , [0m[1;3 and ^[[37mABC from the output stream.
Related
I have a text file with a large number of log messages.
I want to extract the messages between two string patterns. I want the extracted message to appear as it is in the text file.
I tried the following methods. It works, but doesn't support Get-Content's -Wait and -Tail options. Also, the extracted results are displayed in one line, but not like the text file. Inputs are welcome :-)
Sample Code
function GetTextBetweenTwoStrings($startPattern, $endPattern, $filePath){
# Get content from the input file
$fileContent = Get-Content $filePath
# Regular expression (Regex) of the given start and end patterns
$pattern = "$startPattern(.*?)$endPattern"
# Perform the Regex opperation
$result = [regex]::Match($fileContent,$pattern).Value
# Finally return the result to the caller
return $result
}
# Clear the screen
Clear-Host
$input = "THE-LOG-FILE.log"
$startPattern = 'START-OF-PATTERN'
$endPattern = 'END-OF-PATTERN'
# Call the function
GetTextBetweenTwoStrings -startPattern $startPattern -endPattern $endPattern -filePath $input
Improved script based on Theo's answer.
The following points need to be improved:
The beginning and end of the output is somehow trimmed despite I adjusted the buffer size in the script.
How to wrap each matched result into START and END string?
Still I could not figure out how to use the -Wait and -Tail options
Updated Script
# Clear the screen
Clear-Host
# Adjust the buffer size of the window
$bw = 10000
$bh = 300000
if ($host.name -eq 'ConsoleHost') # or -notmatch 'ISE'
{
[console]::bufferwidth = $bw
[console]::bufferheight = $bh
}
else
{
$pshost = get-host
$pswindow = $pshost.ui.rawui
$newsize = $pswindow.buffersize
$newsize.height = $bh
$newsize.width = $bw
$pswindow.buffersize = $newsize
}
function Get-TextBetweenTwoStrings ([string]$startPattern, [string]$endPattern, [string]$filePath){
# Get content from the input file
$fileContent = Get-Content -Path $filePath -Raw
# Regular expression (Regex) of the given start and end patterns
$pattern = '(?is){0}(.*?){1}' -f [regex]::Escape($startPattern), [regex]::Escape($endPattern)
# Perform the Regex operation and output
[regex]::Match($fileContent,$pattern).Groups[1].Value
}
# Input file path
$inputFile = "THE-LOG-FILE.log"
# The patterns
$startPattern = 'START-OF-PATTERN'
$endPattern = 'END-OF-PATTERN'
Get-TextBetweenTwoStrings -startPattern $startPattern -endPattern $endPattern -filePath $inputFile
You need to perform streaming processing of your Get-Content call, in a pipeline, such as with ForEach-Object, if you want to process lines as they're being read.
This is a must if you're using Get-Content -Wait, because such a call doesn't terminate by itself (it keeps waiting for new lines to be added to the file, indefinitely), but inside a pipeline its output can be processed as it is being received, even before the command terminates.
You're trying to match across multiple lines, which with Get-Content output would only work if you used the -Raw switch - by default, Get-Content reads its input file(s) line by line.
However, -Raw is incompatible with -Wait.
Therefore, you must stick with line-by-line processing, which requires that you match the start and end patterns separately, and keep track of when you're processing lines between those two patterns.
Here's a proof of concept, but note the following:
-Tail 100 is hard-coded - adjust as needed or make it another parameter.
The use of -Wait means that the function will run indefinitely - waiting for new lines to be added to $filePath - so you'll need to use Ctrl-C to stop it.
While you can use a Get-TextBetweenTwoStrings call itself in a pipeline for object-by-object processing, assigning its result to a variable ($result = ...) won't work when terminating with Ctrl-C, because this method of termination also aborts the assignment operation.
To work around this limitation, the function below is defined as an advanced function, which automatically enables support for the common -OutVariable parameter, which is populated even in the event of termination with Ctrl-C; your sample call would then look as follows (as Theo notes, don't use the automatic $input variable as a custom variable):
# Look for blocks of interest in the input file, indefinitely,
# and output them as they're being found.
# After termination with Ctrl-C, $result will also contain the blocks
# found, if any.
Get-TextBetweenTwoStrings -OutVariable result -startPattern $startPattern -endPattern $endPattern -filePath $inputFile
Per your feedback you want the block of lines to encompass the full lines on which the start and end patterns match, so the regexes below are enclosed in .*
The word pattern in your $startPattern and $endPattern parameters is a bit ambiguous in that it suggests that they themselves are regexes that can therefore be used as-is or embedded as-is in a larger regex on the RHS of the -match operator.
However, in the solution below I am assuming that they are be treated as literal strings, which is why they are escaped with [regex]::Escape(); simply omit these calls if these parameters are indeed regexes themselves; i.e.:
$startRegex = '.*' + $startPattern + '.*'
$endRegex = '.*' + $endPattern + '.*'
The solution assumes there is no overlap between blocks and that, in a given block, the start and end patterns are on separate lines.
Each block found is output as a single, multi-line string, using LF ("`n") as the newline character; if you want a CRLF newline sequences instead, use "`r`n"; for the platform-native newline format (CRLF on Windows, LF on Unix-like platforms), use [Environment]::NewLine.
# Note the use of "-" after "Get", to adhere to PowerShell's
# "<Verb>-<Noun>" naming convention.
function Get-TextBetweenTwoStrings {
# Make the function an advanced one, so that it supports the
# -OutVariable common parameter.
[CmdletBinding()]
param(
$startPattern,
$endPattern,
$filePath
)
# Note: If $startPattern and $endPattern are themselves
# regexes, omit the [regex]::Escape() calls.
$startRegex = '.*' + [regex]::Escape($startPattern) + '.*'
$endRegex = '.*' + [regex]::Escape($endPattern) + '.*'
$inBlock = $false
$block = [System.Collections.Generic.List[string]]::new()
Get-Content -Tail 100 -Wait $filePath | ForEach-Object {
if ($inBlock) {
if ($_ -match $endRegex) {
$block.Add($Matches[0])
# Output the block of lines as a single, multi-line string
$block -join "`n"
$inBlock = $false; $block.Clear()
}
else {
$block.Add($_)
}
}
elseif ($_ -match $startRegex) {
$inBlock = $true
$block.Add($Matches[0])
}
}
}
First of all, you should not use $input as self-defined variable name, because this is an Automatic variable.
Then, you are reading the file as a string array, where you would rather read is as a single, multiline string. For that append switch -Raw to the Get-Content call.
The regex you are creating does not allow fgor regex special characters in the start- and end patterns you give, so it I would suggest using [regex]::Escape() on these patterns when creating the regex string.
While your regex does use a group capturing sequence inside the brackets, you are not using that when it comes to getting the value you seek.
Finally, I would recommend using PowerShell naming convention (Verb-Noun) for the function name
Try
function Get-TextBetweenTwoStrings ([string]$startPattern, [string]$endPattern, [string]$filePath){
# Get content from the input file
$fileContent = Get-Content -Path $filePath -Raw
# Regular expression (Regex) of the given start and end patterns
$pattern = '(?is){0}(.*?){1}' -f [regex]::Escape($startPattern), [regex]::Escape($endPattern)
# Perform the Regex operation and output
[regex]::Match($fileContent,$pattern).Groups[1].Value
}
$inputFile = "D:\Test\THE-LOG-FILE.log"
$startPattern = 'START-OF-PATTERN'
$endPattern = 'END-OF-PATTERN'
Get-TextBetweenTwoStrings -startPattern $startPattern -endPattern $endPattern -filePath $inputFile
Would result in something like:
blahblah
more lines here
The (?is) makes the regex case-insensitive and have the dot match linebreaks as well
Nice to see you're using my version of the Get-TextBetweenTwoStrings function, however I believe you are mistaking the output in the console to output as in a dedicated text editor. In the console, too long lines will be truncated, whereas in a text editor like notepad, you can choose to wrap long lines or have a horizontal scrollbar.
If you simply append
| Set-Content -Path 'X:\wherever\theoutput.txt'
to the Get-TextBetweenTwoStrings .. call, you will find the lines are NOT truncated when you open it in Word or notepad for instance.
In fact, you can have that line folowed by
notepad 'X:\wherever\theoutput.txt'
to have notepad open that file straight away.
When pulling this install string from the registry, there is an invisible leading character.
I am unable to run the uninstall string or strip this character. Various iterations of split, replace, join, etc work against the string, but do nothing to change the errant behaviour. I have tried within PowerShell or the Windows console.
Write-Output $uninst shows the correct string:
MsiExec.exe /x {1F4D7BAB-E816-43DF-B4B1-5A41A2DA13E8} /qn
When executing that string in PowerShell, the msiexec help bubble pops up. When executing that string at the Windows CMD shell, a white square character is at the beginning of the line.
# pull ESET uninstall string
$esetVer = Get-ChildItem -Path HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall, HKLM:\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall |
Get-ItemProperty |
Where-Object { $_.DisplayName -match "ESET Endpoint Antivirus" } |
Select-Object -Property DisplayName, UninstallString
foreach ($ver in $esetVer) {
if ($ver.UninstallString) {
$uninst = $ver.UninstallString
$uninst = $uninst.Replace('/I{',' /x {').Replace('}','} /qn')
Invoke-Expression $uninst
Write-Output $uninst
}
}
Removing first char only removes the M.
In my question, I focus on some errant character before the msiexec.exe command line. That apparently was not the issue. The issue was the braces around the app ID ( {1F4D7BAB-E816-43DF-B4B1-5A41A2DA13E8} ). They required a backtick. So, simply including the backtick before the brace in my substitution line fixed the code.
Old: $uninst = $uninst.Replace('/I{',' /x {').Replace('}','} /qn')
New: $uninst = $uninst.Replace('/I{',' /x {').Replace('}','} /qn')
This modification worked on 2 Windows 7 Pro computers.
One of my scripts can be stripped down to the following code.
function Replace
{
[CmdletBinding()]
Param
(
[Parameter(Mandatory, Position=0)]
[string]
$LiteralPath,
[Parameter(Mandatory, Position=1)]
[string]
$Expression,
[Parameter(Mandatory, Position=2)]
[string]
$Replacement
)
Get-Content $LiteralPath | ForEach-Object {$_ -replace $Expression, $Replacement } | Set-Content $LiteralPath + ".temp"
}
An example call to the script would be
Replace ".\MyFile.txt" "^#define abc.*" "#define abc 1"
I have run into a situation where the string I need to find and replace contains both dollar signs and underscores. The dollars signs must be escaped to prevent PowerShell from expanding the variable. One string contains a dollar sign followed by an underscore. This is causing an issue because PowerShell is not expanding the variable name but then is expanding the $_ piping variable. How can I prevent PowerShell from expanding both variable name and piping token.
This is an example cal to the function with a string I need to escape.
Replace ".\MyFile.txt" "^\#\`$__LIBRARY_DIR\\prj.gpj" "`$__LIBRARY_DIR\prj.gpj"
In this example the line of text which reads #$__LIBRARY_DIR\prj.gpj is getting changed to #$__LIBRARY_DIR\prj.gpj_LIBRARY_DIR\prj.gpj. I am looking for the text to be changed to $__LIBRARY_DIR\prj.gpj
Notice the $_ is expanded which I do not want it to expand. I have tried adding more escape characters but that only causes them to appear in the file. How can the string be escaped to prevent $_ from expanding?
In powershell, if you don't want variables to expand in your string, use 'single quotes' instead if "double quotes", that saves you the trouble of escaping the $ sign with backticks.
Now in your case you have the additional challenge, that the -replace operator will also want to expand expressions that start with the $ sign in the replacement string, regardless of the types of quotes that you use.
To tell -replace that you really want to see that $ in your replacement string, you need to write $$:
'#$__LIBRARY_DIR\prj.gpj' -replace '^#\$__LIBRARY_DIR\\prj.gpj','$$__LIBRARY_DIR\prj.gpj'
Note: As others have correctly pointed out in the comments, if your task is to strip expressions from a leading #, you can do that in a more simple way:
'#$__LIBRARY_DIR\prj.gpj' -replace '^#'
Or alternatively with the good old "trim":
'#$__LIBRARY_DIR\prj.gpj'.TrimStart('#')
I tried to create a function that emulates Linux's head:
Function head( )
{
[CmdletBinding()]
param (
[parameter(mandatory=$false, ValueFromPipeline=$true)] [Object[]] $inputs,
[parameter(position=0, mandatory=$false)] [String] $liness = "10",
[parameter(position=1, ValueFromRemainingArguments=$true)] [String[]] $filess
)
$lines = 0
if (![int]::TryParse($liness, [ref]$lines)) {
$lines = 10
$filess = ,$liness + (#{$true=#();$false=$filess}[$null -eq $filess])
}
$read = 0
$input | select-object -First $lines
if ($filess) {
get-content -TotalCount $lines $filess
}
}
The problem is that this will actually read all the content (whether by reading $filess or from $input) and then print the first, where I'd want head to read the first lines and forget about the rest so it can work with large files.
How can this function be rewritten?
Well, as far as I know, you are overdoing it slightly...
"Beginning in Windows PowerShell 3.0, Select-Object includes an optimization feature that prevents commands from creating and processing objects that are not used. When you include a Select-Object command with the First or Index parameter in a command pipeline, Windows PowerShell stops the command that generates the objects as soon as the selected number of objects is generated, even when the command that generates the objects appears before the Select-Object command in the pipeline. To turn off this optimizing behavior, use the Wait parameter."
So all you need to do is:
Get-Content -Path somefile | Select-Object -First 10 #or pass a variable
I have a powershell script and I want to write to a console and write to a log file with one call.
I was doing this...
Start-Transcript -Path $TargetDir\Log.log
Write-Host "Stuff"
... which works great, except that the newlines it generates are LF, which means my logs look great in every text editor on earth, except for notepad.
Here's what I have for this...
function global:Write-Notepad
(
[string] $Message,
[string] $ForegroundColor = 'Gray'
)
{
Write-Host "$Message`r" -ForegroundColor $ForegroundColor
}
...which writes a CR to the end of every message, but it doesn't seem to write out lines like this...
&$ACommand | Write-Notepad
I'm not sure what syntax the piping operator expects, but I would greatly appreciate help.
Try this:
& $ACommand | Tee-Object -FilePath $TargetDir\Log.log | Write-Host
Tee-Object will send a copy of the pipeline objects to a file or a variable, and output at the same time.
Here's the solution I settled on...
# This method adds a CR character before the newline that Write-Host generates.
# This is necesary, because notepad is the only text editor in the world that
# doesn't recognize LF newlines, but needs CR LF newlines.
function global:Write-Notepad
(
[string] $Message,
[string] $ForegroundColor = 'Gray'
)
{
Process
{
if($_){ Write-Host "$_`r" }
}
End
{
if($Message){ Write-Host "$Message`r" -ForegroundColor $ForegroundColor }
}
}