I am attempting to use PowerShell to grab useful output from wsl.exe.
For the life of me, I cannot seem to match for strings in the output (see below).
I get the same behavior in both PowerShell 5 and 7.
Does anyone have a reliable means to capture the output of wsl.exe?
Thanks
PS C:\Users\gsamuel> $foo = (wsl -l -v)
PS C:\Users\gsamuel> $foo
NAME STATE VERSION
* Ubuntu-20.04 Running 2
PS C:\Users\gsamuel> $foo | Select-String -Pattern 'Ubuntu'
PS C:\Users\gsamuel> $foo -match 'Ubuntu'
PS C:\Users\gsamuel>
PS C:\Users\gsamuel>
PS C:\Users\gsamuel> [string]$foo | Select-String -Pattern 'Ubuntu'
PS C:\Users\gsamuel> [string]$foo -match 'Ubuntu'
False
This appears to be an encoding issue of some sort: there is a null between each letter of the output, probably because a UTF-16 output is being interpreted as UTF-8.
You can see this by replacing all null characters with #:
$foo = (wsl -l -v)
$foo -Replace "`0", '#'
Which gives this:
# #N#A#M#E# # # # # # # # # # # # #S#T#A#T#E# # # # # # # # # # # #V#E#R#S#I#O#N#
#
#*# #U#b#u#n#t#u#-#2#0#.#0#4# # # # #R#u#n#n#i#n#g# # # # # # # # # #2#
#
#
So the output doesn't actually contain the string "Ubuntu"!
A simple workaround would be to replace those with empty strings:
$foo -Replace "`0", '' | Select-String 'Ubuntu'
Which outputs the expected match:
* Ubuntu-18.04 Stopped 2
This is now fixed in the latest WSL Preview release 0.64.0, but you do have to "opt-in" to the fix so that older workarounds don't inadvertently break.
Adding the environment variable WSL_UTF8 with a value of 1 (and only that value) will now prevent wsl.exe from generating this "mangled" UTF-16 that was causing the issue.
For your example:
> $env:WSL_UTF8=1
> $foo = (wsl -l -v)
> $foo | Select-String -NoEmphasis -Pattern 'Ubuntu'
Ubuntu20.04_WSL1 Stopped 1
Ubuntu20.04_WSL2 Stopped 2
Ubuntu_21.10_WSL2 Stopped 2
Ubuntu-22.04 Stopped 2
Ubuntu Running 2
> $foo -match 'Ubuntu'
# Same
I'll also point to another solution in this answer for those who can't yet upgrade to Windows 11 and/or the WSL Preview. That solution changes the console's OutputEncoding in PowerShell to force wsl.exe's output to be usable.
Related
Can I pipe back from:
$OUTPUT = $(flutter build ios --release --no-codesign | tail -1)
I would like to get both the last line from the build AND show progress, something like
$OUTPUT = $(flutter build ios --release --no-codesign | out | tail -1)
where the hypothetical out utility would also send the output to the terminal.
Do you know how?
Note:
On Unix-like platforms, with external-program output, js2010's elegant tee /dev/tty solution is the simplest.
The solutions below, which also work on Windows, may be of interest for processing external-program output line by line in PowerShell.
A general solution that also works with the complex objects that PowerShell-native commands can output, requires different approaches:
In PowerShell (Core) 7+, use the following:
# PS v7+ only. Works on both Windows and Unix
... | Tee-Object ($IsWindows ? 'CON' : '/dev/tty')
In Windows PowerShell, where Tee-Object unfortunately doesn't support targeting CON, a proxy function that utilizes Out-Host is required - see this answer.
A PowerShell solution (given that the code in your question is PowerShell[1]):
I'm not sure how flutter reports its progress, but the following may work:
If everything goes to stdout:
$OUTPUT = flutter build ios --release --no-codesign | % {
Write-Host $_ # print to host (console)
$_ # send through pipeline
} | select -Last 1
Note: % is the built-in alias for ForEach-Object, and select the one for Select-Object.
If progress messages go to stderr:
$OUTPUT = flutter build ios --release --no-codesign 2>&1 | % {
Write-Host $_.ToString() # print to host (console)
if ($_ -is [string]) { $_ } # send only stdout through pipeline
} | select -Last 1
[1] As evidenced by the $ sigil in the variable name in the LHS of an assignment and the spaces around =
($OUTPUT = ), neither of which would work as intended in bash / POSIX-like shells.
I assume you mean bash because to my knowledge there is no tail in powershell.
Here's how you can see a command's output while still capturing it into a variable.
#!/bin/bash
# redirect the file descriptor 3 to 1 (stdout)
exec 3>&1
longRunningCmd="flutter build ios --release --no-codesign"
# use tee to copy the command's output to file descriptor 3 (stdout) while
# capturing 1 (stdout) into a variable
output=$(eval "$longRunningCmd" | tee >(cat - >&3) )
# last line of output
lastline=$(printf "%s" "$output" | tail -n 1)
echo "$lastline"
I use write-progress in the pipeline.
In order to keep readable pipeline, I wrote a function
function Write-PipedProgress{
<#
.SYNOPSIS
Insert this function in a pipeline to display progress bar to user
.EXAMPLE
$Result = (Get-250Items |
Write-PipedProgress -PropertyName Name -Activity "Audit services" -ExpectedCount 250 |
Process-ItemFurther)
>
[cmdletBinding()]
param(
[parameter(Mandatory=$true,ValueFromPipeline=$true)]
$Data,
[string]$PropertyName=$null,
[string]$Activity,
[int]$ExpectedCount=100
)
begin {
Write-Verbose "Starting $($MyInvocation.MyCommand)"
$ItemCounter = 0
}
process {
Write-Verbose "Start processing of $($MyInvocation.MyCommand)($Data)"
try {
$ItemCounter++
# (3) mitigate unexpected additional input volume"
if ($ItemCounter -lt $ExpectedCount) {
$StatusProperty = if ($propertyName) { $Data.$PropertyName } > > else { ""}
$StatusMessage = "Processing $ItemCounter th $StatusProperty"
$statusPercent = 100 * $ItemCounter / $ExpectedCount
Write-Progress -Activity $Activity -Status $StatusMessage -> > PercentComplete $statusPercent
} else {
Write-Progress -Activity $Activity -Status "taking longer than expected" -PercentComplete 99
}
# return input data to next element in pipe
$Data
} catch {
throw
}
finally {
Write-Verbose "Complete processing of $Data in > $($MyInvocation.MyCommand)"
}
}
end {
Write-Progress -Activity $Activity -Completed
Write-Verbose "Complete $($MyInvocation.MyCommand) - processed $ItemCounter items"
}
}
Hope this helps ;-)
I believe this would work, at least in osx or linux powershell (or even Windows Subsystem for Linux) that have these commands available. I tested it with "ls" instead of "flutter". Is there actually an "out" command?
$OUTPUT = bash -c 'flutter build ios --release --no-codesign | tee /dev/tty | tail -1'
Or, assuming tee isn't aliased to tee-object. Actually, tee-object would work too.
$OUTPUT = flutter build ios --release --no-codesign | tee /dev/tty | tail -1
It would work with the $( ) too, but you don't need it. In powershell, it's used to combine multiple pipelines.
My question may seem duplicate of PowerShell "echo on", but it is not.
I am not interested in capturing the command output, but in the command line itself of every command executed by the script, including the native commands.
This is what "echo on" in cmd does and this is what I am looking for. Set-PSDebug -Trace 1 does not do it and neither passing the -Verbose flag.
So far I have not see a way except outputing them myself, which is a huge pain in itself.
So, can Powershell do what "echo on" does in cmd?
EDIT 1
Not ideal, but I would accept an answer suggesting to use a wrapper function which would receive a command (native or powershell) with parameters and run the command while faithfully logging the respective command line. Of course, the wrapper function code should be part of the answer.
EDIT 2
The following trivial example demonstrates why Set-PSDebug -Trace 1 does not do it:
tasklist `
/fi "status eq running" | Select-Object -First 4
Please, observe:
C:\> cat C:\temp\1.ps1
tasklist `
/fi "status eq running" | Select-Object -First 4
C:\> Set-PSDebug -Trace 1
C:\> C:\temp\1.ps1
DEBUG: 1+ >>>> C:\temp\1.ps1
DEBUG: 1+ >>>> tasklist `
Image Name PID Session Name Session# Mem Usage
========================= ======== ================ =========== ============
csrss.exe 756 Console 1 2,816 K
C:\>
EDIT 3
For comparison, observe an equivalent script in cmd with echo on:
C:\>type c:\temp\1.cmd
#echo on
tasklist ^
/fi "status eq running" |findstr/n ^^|findstr "^[1-4]:"
C:\>c:\temp\1.cmd
C:\>tasklist /fi "status eq running" | findstr/n ^ | findstr "^[1-4]:"
1:
2:Image Name PID Session Name Session# Mem Usage
3:========================= ======== ================ =========== ============
4:csrss.exe 756 Console 1 2,328 K
C:\>
EDIT 4
start-transcript does not do it either:
C:\WINDOWS\system32> cat c:\temp\1.ps1
tasklist `
/fi "status eq running" | Select-Object -First 4 | Out-Default
C:\WINDOWS\system32> Start-Transcript
Transcript started, output file is ~\Documents\PowerShell_transcript.L-PF0TBKV7.Sr1ntThx.20190611143800.txt
C:\WINDOWS\system32> c:\temp\1.ps1
Image Name PID Session Name Session# Mem Usage
========================= ======== ================ =========== ============
csrss.exe 756 Console 1 2,936 K
C:\WINDOWS\system32> Stop-Transcript
Transcript stopped, output file is ~\Documents\PowerShell_transcript.L-PF0TBKV7.Sr1ntThx.20190611143800.txt
C:\WINDOWS\system32> cat ~\Documents\PowerShell_transcript.L-PF0TBKV7.Sr1ntThx.20190611143800.txt
**********************
Windows PowerShell transcript start
Start time: 20190611143800
Username: xyz\me
RunAs User: xyz\me
Configuration Name:
Machine: L-PF0TBKV7 (Microsoft Windows NT 10.0.16299.0)
Host Application: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
Process ID: 25508
PSVersion: 5.1.16299.1004
PSEdition: Desktop
PSCompatibleVersions: 1.0, 2.0, 3.0, 4.0, 5.0, 5.1.16299.1004
BuildVersion: 10.0.16299.1004
CLRVersion: 4.0.30319.42000
WSManStackVersion: 3.0
PSRemotingProtocolVersion: 2.3
SerializationVersion: 1.1.0.1
**********************
Transcript started, output file is ~\Documents\PowerShell_transcript.L-PF0TBKV7.Sr1ntThx.20190611143800.txt
C:\WINDOWS\system32
>
PS>c:\temp\1.ps1
Image Name PID Session Name Session# Mem Usage
========================= ======== ================ =========== ============
csrss.exe 756 Console 1 2,936 K
C:\WINDOWS\system32
>
PS>Stop-Transcript
**********************
Windows PowerShell transcript end
End time: 20190611143810
**********************
C:\WINDOWS\system32>
As you can see it does not contain the command line.
Firstly, the reason you're dissatisfied with the built-in options is because you're going against the grain; your requirement is like asking how to put sacks of gravel in the back of a Porsche. Powershell comes with Verbose and Debug output streams and a fantastic debugger.
If you have any ability to influence coding standards, look at splatting as an alternative to backtick-continuations.
If you can count on versions of Windows that are not years past EoL, consider Get-ScheduledTask | Where-Object State -eq 'Ready' instead of tasklist.
That said, yes, what you want is possible. Here's a script that will echo across line continuations:
# Echo.ps1
function Disable-Echo
{
param
(
[Parameter(Mandatory)]
[string]$Path
)
$Path = ($Path | Resolve-Path -ErrorAction Stop).Path
Get-PSBreakpoint -Script $Path | Remove-PSBreakpoint
}
function Enable-Echo
{
param
(
[Parameter(Mandatory)]
[string]$Path
)
$Path = ($Path | Resolve-Path -ErrorAction Stop).Path
Disable-Echo $Path
$Ast = [System.Management.Automation.Language.Parser]::ParseFile($Path, [ref]$null, [ref]$null)
$Statements = $Ast.BeginBlock, $Ast.ProcessBlock, $Ast.EndBlock |
Select-Object -ExpandProperty Statements |
Write-Output |
Where-Object {$_.Extent}
foreach ($Statement in $Statements)
{
$Action = {
$Text = $Statement.Extent.Text
$Text = $Text -replace '`\r?\n' # Concatenate lines that are escaped with backtick
# Alternative to remove superfluous whitespace:
# $Text = $Text -replace '\s+`\r?\n\s*', ' '
Write-Host "ECHO: $Text" -ForegroundColor Yellow
continue # or 'break' to stop on the line
}.GetNewClosure() # Create a closure, to capture the value of $Statement
$Params = #{
Script = $Path
Line = $Statement.Extent.StartLineNumber
Column = $Statement.Extent.StartColumnNumber
Action = $Action
}
$null = Set-PSBreakpoint #Params
}
}
Sample script:
# foo.ps1
gci `
-Name `
-File `
-Filter Victor.*
gci -File -Name *.md; gci -File -Name *.psd1
Usage:
# Without echo
❯ .\foo.ps1
Victor.build.ps1
Victor.psd1
Victor.psm1
README.md
Victor.psd1
❯ . .\Echo.ps1
❯ Enable-Echo .\foo.ps1
❯ .\foo.ps1
ECHO: gci -Name -File -Filter Victor.*
Victor.build.ps1
Victor.psd1
Victor.psm1
ECHO: gci -File -Name *.md
README.md
ECHO: gci -File -Name *.psd1
Victor.psd1
Tested on PSv5 and PSv7. Should work on PSv2, although the sample foo.ps1 is PSv3+ (IIRC).
This will not echo calls to other scripts. For that, you'd probably want to do more AST inspection, identify CommandAsts that call scripts, and recursively enable echo on those scripts too. Alternatively, there might be joy in Set-PSBreakpoint -Variable - the $$ variable might be a good place to start - but this would likely be a PITA to work with as it would invoke while you're trying to debug the echo function. You could inspect Get-PSCallStack to skip the action while you're at the command prompt.
I expect four answers and you have already mentioned three that do not work for you (Set-PSDebug, Start-Transaction, -Verbose). As much as they may be viable but not in the format you are looking for, I will not talk more of them.
For the third option, try using Get-History. Now, this will not print out each command as you execute it (only when you call it) like I assume you want. It will also likely not print out each of the lines inside another script (you would want a trace but you did not like that because it prints out more than just the execution).
You can try asking around the PowerShell repository but I do not expect you to find what you are seeking.
If Event logs is an option, start tracing by enabling this Group Policy.
Administrative Templates -> Windows Components -> Windows PowerShell
See Microsoft Docs - Script Tracing and Logging
Then you would of course need to parse the Event logs accordingly...
values.ini looks like
[default]
A=1
B=2
C=3
foo.txt looks like
Now is the %A% for %a% %B% men to come to the %C% of their %c%
I want to use Powershell to search for all of the %x% values in values.ini and then replace every matching instance in foo.txt with the corresponding value, case insensitively; generating the following:
Now is the 1 for 1 2 men to come to the 3 of their 3
Assuming PowerShell version 3.0 or newer, you can use the ConvertFrom-StringData cmdlet to parse the key-value pair in your ini file, but you'll need to filter out the [default] directive:
# grab relevant lines from file
$KeyValPairs = Get-Content .\values.ini | Where {$_ -like "*=*" }
# join strings together as one big string
$KeyValPairString = $KeyValPairs -join [Environment]::NewLine
# create hashtable/dictionary from string with ConvertFrom-StringData
$Dictionary = $KeyValPairString |ConvertFrom-StringData
You can then use the [regex]::Replace() method to do a lookup against the dictionary for each match you want to replace:
Get-Content .\foo.txt |ForEach-Object {
[Regex]::Replace($_, '%(\p{L}+)%', {
param($Match)
# look term up in dictionary
return $Dictionary[$Match.Groups[1].Value]
})
}
To complement Mathias R. Jessen's excellent answer with alternative approaches that also take the later requirement change of limiting values to a specific INI-file section into account (PSv2+, except for Get-Content -Raw; in PSv2, use (Get-Content ...) -join "`n" instead.)
Using PsIni\Get-IniContent and [environment]::ExpandEnvironmentVariables():
# Translate key-value pairs from section the section of interest
# into environment variables.
# After this command, the following environment variables are defined:
# $env:A, with value 1 (cmd.exe equivalent: %A%)
# $env:B, with value 2 (cmd.exe equivalent: %B%)
# $env:C, with value 3 (cmd.exe equivalent: %C%)
$section = 'default' # Specify the INI-file section of interest.
(Get-IniContent values.ini)[$section].GetEnumerator() |
ForEach-Object { Set-Item "env:$($_.Name)" -Value $_.Value }
# Read the template string as a whole from file foo.txt, and expand the
# environment-variable references in it, using the .NET framework.
# With the sample input, this yields
# "Now is the 1 for 1 2 men to come to the 3 of their 3".
[environment]::ExpandEnvironmentVariables((Get-Content -Raw foo.txt))
The 3rd-party Get-IniContent cmdlet, which conveniently reads an INI file (*.ini) into a nested, ordered hashtable, can easily be installed with Install-Module PsIni from an elevated console (alternatively, add -Scope CurrentUser), if you have PS v5+ (or v3 or v4 with PackageManagement installed).
This solution takes advantage of the fact that the placeholders (e.g., %a%) look like cmd.exe-style environment-variable references.
Note the assumptions and caveats:
All ini-file keys / placeholder names are legal environment-variable names.
Preexisting variables may be overwritten, which can be problematic with names such as PATH.
Cross-platform caveat: on Unix-like platforms, environment-variable references are case-sensitive, so the solution won't work the same there.
Using custom INI-file parsing and [environment]::ExpandEnvironmentVariables():
If installing a module for INI-file parsing is not an option, the following solution uses a - rather complex - regular expression to extract the section of interest via the -replace operator.
$section = 'default' # Specify the INI-file section of interest.
# Get all non-empty, non-comment lines from the section using a regex.
$sectLines = (Get-Content -Raw values.ini) -replace ('(?smn)\A.*?(^|\r\n)\[' + [regex]::Escape($section) + '\]\r\n(?<sectLines>.*?)(\r\n\[.*|\Z)'), '${sectLines}' -split "`r`n" -notmatch '(^;|^\s*$)'
# Define the key-value pairs as environment variables.
$sectlines | ForEach-Object { $tokens = $_ -split '=', 2; Set-Item "env:$($tokens[0].Trim())" -Value $tokens[1].Trim() }
# Read the template string as a whole, and expand the environment-variable
# references in it, as before.
[environment]::ExpandEnvironmentVariables((Get-Content -Raw foo.txt))
I found a simpler solution using this INI script called Get-IniContent.
#read from Setup.ini
$INI = Get-IniContent .\Setup.ini
$sec="setup"
#REPLACE VARIABLES
foreach($c in Get-ChildItem -Path .\Application -Recurse -Filter *.config)
{
Write-Output $c.FullName
Write-Output $c.DirectoryName
$configFile = Get-Content $c.FullName -Raw
foreach($v in $INI[$sec].Keys)
{
$k = '%'+$v+'%'
$match = [regex]::IsMatch($configFile, $k)
if($match)
{
$configFile = $configFile -ireplace [regex]::Escape($k), $INI[$sec][$v]
}
}
Set-Content $c.FullName -Value $configFile
}
I basically want to use powershell and get the console user like
"query session | findstr console | awk '{print $2}'"
but not using awk, but I can't get it to work.
$out = query session | findstr console # good
$consoleuser = $out.split('\s+')[1] # doesn't work
$out looks something like:
>console joe 2 Active
$consoleuser ends up being:
ole joe 2 Active
As others have suggested try the following
$out = query session | findstr console
$consoleuser = $($out -split('\s+'))[1]
Or you could try
$consoleuser = $ENV:username
.Split() is a .Net string method, it doesn't use regexes. -split is the PowerShell operator and uses regexes.
And calling out to findstr is usable, but there's no need to leave PS to use it. e.g.
$out = query session | Where {$_ -match 'console'}
$consoleuser = ($out -split '\s+')[1]
((query session) -match 'console' -split '\s+')[1]
To complement TessellatingHeckler's helpful answer with a further optimization (but note that armorall171's helpful recommendation to simply use $env:USERNAME may be all you need):
(-split ((query session) -match '^>console'))[1]
Output from external command query session is returned as an array of strings by PowerShell, and the -match operator filters that array down to only the matching elements (just 1 line in this case).
The -split operator has a unary form whose behavior is similar to awk's default field-parsing behavior:
It splits the input into array elements by runs of whitespace, ignoring leading and trailing whitespace.
Example:
> -split " ab `t `t cde `n `n efgh "
ab
cde
efgh
Try
($out -Split '\s+')[1]
More useful stuff here
Is ther any way in Powershell to pipe in from an virable instead of a file?
There are commands that I need to pipe into another command, right now that is done by first creating a file with the additional commands, and then piping that file into the original command. Code looks somehting like this now:
$val = "*some command*" + "`r`n" + "*some command*" + "`r`n" + "*some command*"
New-Item -name Commands.txt -type "file" -value $val
$command = #'
db2cmd.exe /C '*custom db2 command* < \Commands.txt > \Output.xml'
'#
Invoke-Expression -Command:$command
So instead of creating that file, can I somehow just pipe in $val insatead of Commands.txt?
Try this
$val = #("*some command*1","*some command2*","*some command3*")
$val | % { db2cmd.exe /C $_ > \Output.xml }
You should be able to pipe in from $val provided you use Write-Output or its shorthand echo, but it may also be worth trying passing the commands directly on the command line. Try this (and if it doesn't work I can delete the answer):
PS C:\> filter db2cmd() { $_ | db2cmd.exe ($args -replace '(\\*)"','$1$1\"') }
PS C:\> $val = #"
>> *custom db2 command*
>> *some command*
>> *some command*
>> *some command*
>> "#
>>
PS C:\> db2cmd /C $val > \Output.xml
What happens here is that Windows executables receive their command line from a single string. If you run them from cmd.exe you cannot pass newlines in the argument string, but Powershell doesn't have that restriction so with many programs you can actually pass multiple lines as a single argument. I don't know db2cmd.exe so it might not work here.
The strange bit of string replacement is to handle any double quotes in the arguments: Powershell doesn't quote them and the quoting rules expected by most exe files are a bit bizarre.
The only limitation here would be that $val must not exceed about 32,600 characters and cannot contain nulls. Any other restrictions (such as whether non-ascii unicode characters work) would depend on the application.
Failing that:
echo $val | db2cmd.exe /C '*custom db2 command*' > \Output.xml
may work, or you can use it in combination with the filter I defined at the top:
echo $val | db2cmd /C '*custom db2 command*' > \Output.xml