Powershell fails to run multi-line commands from stdin? - powershell

I'm wanting to pass arbitrary scripts to Powershell via stdin.
(In practice, I'd like to avoid having to put the script into a temporary file, but for the purposes of this question I will pipe the contents of a file to powershell.)
So I'm doing something like so (in this example, from a Windows cmd shell):
type myfile.txt | powershell -
It works if myfile.txt contains something like this:
1..3 | % { $_ *2 }
echo done
(It outputs 2\n4\n6\ndone.)
However, if I split this first statement across multiple lines like so, then Powershell simply exists without generating any output at all:
1..3 |
% { $_ *2 }
echo done
This seems to fail for any multiline statement. For example, this also fails to produce output:
1..3 | % {
$_ *2 }
echo done
I'm surprised by this since each are legal Powershell scripts that would work normally if placed into a .ps1 file and run as normal.
I've tried various things including escaping the EOL using line continuation chars, to no avail. The same effect occurs if the parent shell is Powershell, or even Python (using subprocess.Popen with stdin=PIPE). In each case, Powershell exits without any error, and the exit code is 0.
Interestingly, if I run the following, only "before.txt" gets created.
"before" | out-file before.txt
1..3 |
% { $_ *2 }
"after" | out-file after.txt
echo done
Any ideas why Powershell would have trouble reading a multi-line command, if read from stdin?

I'm going to consider this answered by this:
How to end a multi-line command in PowerShell since it shows that an extra newline is required.
However, I'm going to raise this to MS as a bug since this should not be required when reading from a non-tty, or when -NonInteractive switch is specified.
Please vote on my bug report to the Powershell team.

This is not a complete answer, but from what I can tell, the problem has to do with the input being sent in line by line.
To demonstrate the line-by-line issue, I invoke powershell this way:
powershell.exe -command "gc myfile.txt" | powershell.exe -
vs
powershell.exe -command "gc myfile.txt -raw" | powershell.exe -
The first example replicates what you see with type, the second reads the entire contents of the file, and it works as expected.
It also works from within PowerShell if you put the script contents in a string and pipe it into powershell.exe -.
I had a theory that it had to do with line-by-line input lacking line breaks, but it's not so clear cut. If that were the case, why would the first option work but not the second (removing the line break splitting the single pipeline should have no effect, while removing the line break between the pipeline and the echo should make it fail). Maybe there's something unclear about the way powershell is handling the input with or without line breaks.

Related

Avoid Line break at end of cmd output?

when I use this command pwsh -c echo hello in cmd I get the following output:
C:\>pwsh -c echo hello
hello
C:\>
I do not get that line break at the end
when I run it on powershell:
PS C:\> pwsh -c echo hello
hello
PS C:\>
So I think the problem is in cmd. I know this is not such a problem and have an easy fix but I have some programs uses cmd to access powershell and removing that line break is not that fun.
So is there any fix to prevent cmd to add that line ?
Mofi has provided the crucial pointers in comments:
When executing a command interactively, cmd.exe unconditionally appends a a newline (line break) to the command's output, presumably for readability and perhaps also to ensure that the next prompt always starts on a new line.
This applies irrespective of what that command is. In other words: It doesn't matter that your command happens to be a PowerShell command.
However, that trailing newline does not become part of the command's output, therefore programmatic processing of a command's output is not affected, such as when you redirect > to a file or process the output lines one by one with for /f.
In other words: for programmatic processing you need not remove the trailing newline, because it isn't part of the actual command output.
Conversely, if you really need to in effect suppress the trailing newline for display, you'll have to modify the command's output - if that is even an option - so that the output itself doesn't end in a newline, as shown in this SuperUser answer for cmd.exe's own echo command; for PowerShell, you could do pwsh -c Write-Host -NoNewLine hello.
Edge case:
When capturing output from a batch file that is running without #echo off (or with echo on) - in which case the trailing newlines do become part of the output - you can filter out empty lines by piping to findstr /r /v /c:"^$" (as also shown in the linked answer); e.g.
foo.cmd | findstr /r /v /c:"^$"
However, note that all empty lines are filtered out this way - potentially including actual empty lines in the output from commands executed by the batch file.
If preventing that is required, a more sophisticated approach is required, which, however (a) relies on the standard prompt string (e.g., C:\>) being used and (b) can still yield false positives:
foo.cmd | powershell -nop -c "#($Input) -join \"`n\" -replace '\n(?=[a-z]:\\.*?>)'"
Finally note that if you execute the above commands without capturing or redirecting their output, their overall output in the cmd.exe console will again have a trailing newline.

Powershell does not render my script help correctly

I have a Powershell script with rather a big help at the beginning of it to be displayed with the Get-Help command - https://gist.github.com/MarkKharitonov/91698caf8c695902eacde2b6c7825bf1
I have two problems with the way the help is rendered. Specifically with the .EXAMPLE section. Here is an abbreviated version of the help (see the hyperlink for the full version of the script and the help):
<#
.SYNOPSIS
Runs a WinDBG command or a script file.
...
.NOTES
The default is to scan for images using the .imgscan /l. One has to check first with the WinDBG GUI to see if this is necessary, because this is a time consuming operation.
.EXAMPLE
Please, find below a PowerShell session transcript showing the script invoked with the -Command parameter:
PS D:\tmp\cantestr52 - 06-09-2017> cat C:\Users\mkharitonov\runcdb.config.ps1
$Sosex = "E:\Utils\sosex\64\sosex.dll"
$CDB = "e:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe"
$SymbolsCache = "E:\Symbols"
PS D:\tmp\cantestr52 - 06-09-2017> cat .\runcdb.config.ps1
$dump = "d:\tmp\cantestr52 - 06-09-2017\Quartz.Server.DMP"
$ImagePath = "d:\tmp\cantestr52 - 06-09-2017\BJE"
$NoScanForImages = $false
PS D:\tmp\cantestr52 - 06-09-2017> runcdb.ps1 -Command "!dumpheap -mt 00007fff11d90f78 -live"
...
PS D:\tmp\cantestr52 - 06-09-2017>
The script produced two log files:
c:\Users\MKHARI~1\AppData\Local\Temp\!dumpheap_-mt_00007fff11d90f78_-live.log.init
c:\Users\MKHARI~1\AppData\Local\Temp\!dumpheap_-mt_00007fff11d90f78_-live.log
Here is the WinDBG code executed by the script:
| .logopen "c:\Users\MKHARI~1\AppData\Local\Temp\!dumpheap_-mt_00007fff11d90f78_-live.log.init"
| .imgscan /l
| .load E:\Utils\sosex\64\sosex.dll
| !lhi
| .logclose
| .logopen "c:\Users\MKHARI~1\AppData\Local\Temp\!dumpheap_-mt_00007fff11d90f78_-live.log"
| !dumpheap -mt 00007fff11d90f78 -live
| .logclose
| q
#>
Problem 1
Notice that the last lines start with |. This is because of .logopen - PowerShell attempts to treat it as a help section keyword, like .NOTES or .EXAMPLE. Of course, there is no such keyword and so PowerShell just truncates the help output and stops short of showing the last lines. I had to escape the dot somehow, but could not figure out the way.
Problem 2
Please, observe:
Notice the C:\PS> in the output. Of course, it is not part of my help.
So, two questions - how to escape the first dot in a help line and what is the source of the C:\PS> in the help output above?
Problem 1 - (lines beginning with dot .)
I don't think there's any getting around this with comment based help, but if you use XML based help then I don't think it would be an issue. That's a big change though.
Problem 2 - (prompt string)
It seems to do this with all examples, and isn't likely to be avoided with XML help. For what it's worth, I don't like this behavior either.
However, I will say that your example, I think, does not properly follow the format of what an example is. Your explanation that looks awkward in front of a prompt shouldn't be in the example section.
After .EXAMPLE you should just put the line of code (a short explanation could maybe go after the code).
More explanation can be given in .NOTES, or in linked documentation.

Compressing to tar.xz using 7-zip through a pipe on windows

My command line is this (powershell):
$7z ="`"c:\Program Files\7-Zip\7z.exe`""
&$7z a -r -ttar -bd -so . | &$7z a -r -txz -bd $archive -si
The produced archive file indeed contains a tar file, but that tar file is corrupt.
Note, that breaking the pipe into two commands works correctly:
&$7z a -r -ttar -bd ${archive}.tmp .
&$7z a -r -txz -bd $archive ${archive}.tmp
The produced archive is perfectly valid.
So, what is wrong with my pipeline?
(I am using Powershell)
Nothing is wrong with your pipeline it is the way that the pipeline works that's causing the error.
PowerShell pipe works in an asynchronous way. Meaning that output of the first command is available to the second command immediately one object at the time even if the first one has not finished executing, See here.
Both Unix and PowerShell pipes operate in the same way. The reason why you might be seeing a difference from Unix to PowerShell is the way in which they go about it is different.
Unix passes Strings between the commands. Where as a Powershell pipe will pass full-fledged .net object between commands. This difference in the data type being past between command will be why it works on unix and not in PowerShell. If 7z.exe can not huddle these .net objects correctly the files will be come corrupt, See here.
Try adding | %{ "$_" } in between the pipes like
&$7z a -r -ttar -bd -so . | %{ "$_" } | &$7z a -r -txz -bd $archive -si
The point is that the second call to 7z expects unmodified data on STDIN, but PowerShell is converting the output from the first call to 7z to (multiple) (string) objects. % is an alias for foreach-object, so what the additional command does is to loop over each object and convert it to a plain string before passing it on to the second call to 7z.
Edit: Reading through PowerShell’s Object Pipeline Corrupts Piped Binary Data it looks to me now as if my suggestion would not work, and there's also no way to fix it. Well, other than wrapping the whole pipeline into a cmd /c "..." call to make cmd and not PowerShell handle the pipeline.
Edit2: I also was trying this solution from the PowerShell Cookbook, but it was very slow.
In the end, I created a .cmd script with the 7z pipes that I'm calling from my PowerShell script.

Mercurial - How to configure multiline hook on Windows

Running Windows 7, configuring a commit hook in my .hgrc file.
The hook calls an external powershell script and passes it a few parameters. When I put all the parameters on one long line, the script receives them just fine. However, when I try to put each parameter on its own line, the hook can't figure out that they are all part of the same call to my external script.
[hooks]
commit.working_one_liner = PowerShell.exe -ExecutionPolicy Bypass -File .\MyScript.ps1 -hg %HG% -updatedToChangeset %HG_NODE% -dbName 'Test'
commit.multi_line_hook_not_working = PowerShell.exe
-ExecutionPolicy Bypass
-File .\MyScript.ps1
-hg %HG%
-updatedToChangeset %HG_NODE%
-dbName 'Test'
According to the Hg documentation, I should be able to do this:
"A configuration file consists of sections, led by a [section] header and followed by name = value entries (sometimes called configuration keys):
[spam]
eggs=ham
green=
eggs
Each line contains one entry. If the lines that follow are indented, they are treated as continuations of that entry. Leading whitespace is removed from values. Empty lines are skipped. Lines beginning with # or ; are ignored and may be used to provide comments."
Despite indenting, the multi_line_hook seems to ignore everything after the first line. I've tried various escape characters (`, ^, \, etc.) after each new line. Any ideas what I'm missing here?
The example from the documentation only works for the first line to follow and not the rest.
I don't have access to windows at the moment, but on linux I need to use a backslash:
[hooks]
commit.working = echo \
test

How do I reference variables when executing a shell command in PowerShell?

I'm a newbie to PowerShell. What's wrong with my script below? It's not wanting to emit the value of $config. However, when I wrap that command in double quotes, everything looks okay.
param($config, $logfolder)
# Must run log analysis in chronological order.
ls $logfolder | Sort-Object LastWriteTime | % {
perl D:\Websites\_awstats\wwwroot\cgi-bin\awstats.pl -LogFile="$($_.FullName)" -config=$config update
}
# Execute with - .\regen-logs.ps1 webgenesis "C:\inetpub\logs\LogFiles\W3SVC5"
# Returns for each file - Error: Couldn't open config file "awstats.config.conf" nor "awstats.conf" after searching in path "D:\Websites\_awstats\wwwroot\cgi-bin,/etc/awstats,/usr/local/etc/awstats,/etc,/etc/opt/awstats": No such file or directory
As-is, what gets emitted and executed seems to have "-config=$config" passed as an argument. At least, that's my best guess. I don't know if $_ is working correctly either.
If I put quotes around the perl command like so, I get the command I do want to execute.
ls $logfolder | Sort-Object LastWriteTime | % {
"perl D:\Websites\_awstats\wwwroot\cgi-bin\awstats.pl -LogFile=`"$($_.FullName)`" -config=$config update"
}
# Outputs for each log file something like - perl D:\Websites\_awstats\wwwroot\cgi-bin\awstats.pl -LogFile="C:\inetpub\logs\LogFiles\W3SVC5\u_ex110602.log" -config=webgenesis update
If putting quotes around it produces the correct commandline, one way to execute the contents of a string is with Invoke-Expression (alias iex):
$v = "myexe -myarg1 -myarg2=$someVar"
iex $v
Put double quotes around "-config=$config". Without this, PowerShell will interpret -config=$config as one string argument that just happens to contain a $ sign in it.
I think you need to start your perl command out with & so that PowerShell interprets things as a command and not a string.
& perl D:\Websites\_awstats\wwwroot\cgi-bin\awstats.pl -LogFile=`"$($_.FullName)`" -config=$config update
Also, see: Run a program in a foreach