I am writing a program prog.exe that retrieves all arguments that are passed to it (in the form of a "sentence", not standalone arguments).
I just realized that in some cases only part of the line is retrieved, and this is when there are #parameters:
PS > ./prog.exe this is a #nice sentence
Only this, is and a are retrieved. In case I do not use # I get all of them. I presume this is because everything after the # is interpreted by Powershell as a comment.
Is there a way to retrieve everything that is on the command line?
If this makes a difference, I code in Go and get the arguments via os.Args[1:].
You can prevent PowerShell from interpreting # as a comment token by explicitly quoting the input arguments:
./prog.exe one two three '#four' five
A better way exists, though, especially if you don't control the input: split the arguments into individual strings then use the splatting operator # on the array containing them:
$paramArgs = -split 'one two three #four five'
./prog.exe #paramArgs
Finally, using the --% end-of-parsing token in a command context will cause the subsequent arguments on the same line to be passed as-is, no parsing of language syntax:
./prog.exe --% one two three #four five
Related
I downloaded the npm package for merge junit reports - https://www.npmjs.com/package/junit-merge.
The problem is that I have multiple files to merge and I am trying to use string variable to hold file names to merge.
When I write the script myslef like:
junit-merge a.xml b.xml c.xml
This works, the merged file is being created, but when I do it like
$command = "a.xml b.xml c.xml"
junit-merge $command
This does not work. The error is
Error: File not found
Has anyone faced similar issues?
# WRONG
$command = "a.xml b.xml c.xml"; junit-merge $command
results in command line junit-merge "a.xml b.xml c.xml"[1], i.e. it passes a string with verbatim value a.xml b.xml c.xml as a single argument to junit-merge, which is not the intent.
PowerShell does not act like POSIX-like shells such as bash do in this regard: In bash, the value of variable $command - due to being referenced unquoted - would be subject to word splitting (one of the so-called shell expansions) and would indeed result in 3 distinct arguments (though even there an array-based invocation would be preferable).
PowerShell supports no bash-like shell expansions[2]; it has different, generally more flexible constructs, such as the splatting technique discussed below.
Instead, define your arguments as individual elements of an array, as justnotme advises:
# Define the *array* of *individual* arguments.
$command = "a.xml", "b.xml", "c.xml"
# Pass the array to junit-merge, which causes PowerShell
# to pass its elements as *individual arguments*; it is the equivalent of:
# junit-merge a.xml b.xml c.xml
junit-merge $command
This is an application of a PowerShell technique called splatting, where you specify arguments to pass to a command via a variable:
Either (typically only used for external programs, as in your case):
As an array of arguments to pass individually as positional arguments, as shown above.
Or (more typically when calling PowerShell commands):
As a hashtable to pass named parameter values, in which you must replace the $ sigil in the variable reference with #; e.g., in your case #command; e.g., the following is the equivalent of calling Get-ChildItem C:\ -Directory:
$paramVals = #{ LiteralPath = 'C:\'; Directory = $true }; Get-ChildItem #paramVals
Caveat re array-based splatting:
Due to a bug detailed in GitHub issue #6280, PowerShell doesn't pass empty arguments through to external programs (applies to all Windows PowerShell versions / and as of PowerShell (Core) 7.2.x; a fix may be coming in 7.3, via the $PSNativeCommandArgumentPassing preference variable, which in 7.2.x relies on an explicitly activated experimental feature).
E.g., foo.exe "" unexpectedly results in just foo.exe being called.
This problem equally affects array-based splatting, so that
$cmdArgs = "", "other"; foo.exe $cmdArgs results in foo.exe other rather than the expected foo.exe "" other.
Optional use of # with array-based splatting:
You can use the # sigil also with arrays, so this would work too:
junit-merge #command
There is a subtle distinction, however.
While it will rarely matter in practice,
the safer choice is to use $, because it guards against (the however hypothetical) accidental misinterpretation of a --% array element you intend to be a literal.
Only the # syntax recognizes an array element --% as the special stop-parsing symbol, --%
Said symbol tells PowerShell not to parse the remaining arguments as it normally would and instead pass them through as-is - unexpanded, except for expanding cmd.exe-style variable references such as %USERNAME%.
This is normally only useful when not using splatting, typically in the context of being able to use command lines that were written for cmd.exe from PowerShell as-is, without having to account for PowerShell's syntactical differences.
In the context of splatting, however, the behavior resulting from --% is non-obvious and best avoided:
As in direct argument passing, the --% is removed from the resulting command line.
Argument boundaries are lost, so that a single array element foo bar, which normally gets placed as "foo bar" on the command line, is placed as foo bar, i.e. effectively as 2 arguments.
[1] Your call implies the intent to pass the value of variable $command as a single argument, so when PowerShell builds the command line behind the scenes, it double-quotes the verbatim a.xml b.xml c.xml string contained in $command to ensure that. Note that these double quotes are unrelated to how you originally assigned a value to $command.
Unfortunately, this automatic quoting is broken for values with embedded " chars. - see this answer, for instance.
[2] As a nod to POSIX-like shells, PowerShell does perform one kind of shell expansion, but (a) only on Unix-like platforms (macOS, Linux) and (b) only when calling external programs: Unquoted wildcard patterns such as *.txt are indeed expanded to their matching filenames when you call an external program (e.g., /bin/echo *.txt), which is feature that PowerShell calls native globbing.
I had a similar problem. This technique from powershell worked for me:
Invoke-Expression "junit-merge $command"
I also tried the following (from a powershell script) and it works:
cmd / c "junit-merge $command"
I'm trying to automate creating certificates via a Perl script.
The command I want to run is:
easyrsa build-client-full $clientname nopass
The way I thought it should be done in Perl is:
my $arguments = ("build-client-full $clientname nopass");
my $cmd = "$easyrsa_path/easyrsa"." "."$arguments";
system("bash", $cmd);
However, this yields
"file not found"
on execution. I triple checked that the path is correct.
If I try it like this:
my #arguments = ("bash", $easyrsa_path,"build-client-full $clientname nopass");
system(#arguments);
Bash returns
"Unknown command 'build-client-full test nopass'. Run without commands
for usage help."
Background
When you use system(LIST) where LIST has more than one element, Perl will not call the shell, and instead directly invoke the program given by the first element in the LIST, and use the rest of the list as command line arguments to be passed verbatim, with no interpolation by the shell, including no splitting arguments on whitespace.
So in your first example, Perl is running the command bash and passing the string "$easyrsa_path/easyrsa build-client-full $clientname nopass", literally as one big long argument, and in your second example, it's running the command bash and passing the two arguments $easyrsa_path and "build-client-full $clientname nopass". However, I assume that easyrsa needs the three arguments as separate strings in its argument list, which the shell would normally split, but since both of your calls to system aren't using the shell, it's not working.
system (and exec) have four ways of interpreting their arguments, as per the documentation:
If you pass a single string (including a LIST with only one element) that does not contain any shell metacharacters, it is split into words and passed directly to execvp(3) (meaning it bypasses the shell).
Warning: This invocation is easily confused with the following - a single metacharacter will cause the shell to be invoked, which can be dangerous especially when unchecked variables are interpolated into the command string.
If you pass a single string (including a LIST with only one element) that does contain shell metacharacters, the entire argument is passed to the system's command shell for parsing. Normally, that's /bin/sh -c on Unix platforms, but the idea of the "default shell" is problematic, and there is certainly no guarantee that it'll be bash (though it could be).
Warning: In this invocation of system, you have the full power of the shell, which also means you're responsible for correctly quoting and escaping any shell metacharacters and/or whitespace. I recommend you only use this form if you explicitly want the power of the shell, and otherwise, it's usually best to use one of the following two.
If there is more than one argument in LIST, this calls execvp(3) with the arguments in LIST, meaning the shell is avoided.
(See below for caveats on Windows.)
The form system {EXPR} LIST always runs the program named by EXPR and avoids the shell, no matter what's in LIST.
(See below for caveats on Windows.)
The latter two are desirable if you want to pass special characters that the shell would normally interpret, and I'd actually always recommend doing this, since blindly passing user input into system can open up a security hole - I wrote a longer article about that over on PerlMonks.
Solutions
#Borodin and #AnFi have already pointed out: If you simply split up the elements of the LIST properly, it should work - it doesn't look like you need any features of bash or any shell here. And don't forget to check for errors!
system("$easyrsa_path/easyrsa","build-client-full",$clientname,"nopass") == 0
or warn "system failed: \$? = $?";
Note that there are good modules that provide alternatives to system and qx, my go-to module is usually IPC::Run3. These modules are very helpful if you want to capture output from the external command. In this case, IPC::System::Simple might be easier since it provides a drop-in replacement for system with better error handling, as well as systemx which always avoids the shell. (That module is what autodie uses when you say use autodie ':all';.)
use IPC::System::Simple qw/systemx/;
systemx("$easyrsa_path/easyrsa","build-client-full",$clientname,"nopass");
Note that if you really wanted to call bash, you'd need to add the -c option and say system("bash","-c","--","$easyrsa_path/easyrsa build-client-full $clientname nopass"). But as I a said above, I strongly recommend against this, since if $easyrsa_path or $clientname contain any shell metacharacters or malicious content, you may end up having a huge problem.
Windows
Windows is more complicated than the above. The documentation says that the only "reliable" way to avoid calling the shell there is the system PROGRAM LIST form, but on Windows, command line arguments are not passed as a list, but a single big string, and it's up to the called command, not the shell, to interpret that string, and different commands may do that differently - see also. (I have heard good things about Win32::ShellQuote, though.)
Plus, there's the special system(1, #args) form documented in perlport.
If you pass multiple parameters to system then each one forms a separate parameter to the command line. So it is as though you had entered
easyrsa "build-client-full test nopass"
and you correctly get the error
Unknown command 'build-client-full test nopass'
You also don't need to add bash: perl will run the shell for you if necessary
You can either pass the whole command to system
system($cmd)
and perl will pass it to the shell to be processed as if you'd entered it at the command prompt. Or you can split the parameters properly
system("$easyrsa_path/easyrsa", "build-client-full", $clientname, "nopass")
which will make perl call easyrsa directly unless the command contains things that need the shell to process, like output redirection
I need to build a string that is actually a command-line, and then execute the contents of that command-line. I'd hoped the call operator (&) would help, but it appears not. Here is a simple contrived example. The following works as expected, it pings a website:
$command = "ping"
$website = "www.bbc.co.uk"
& $command $website
however if I change it to this:
$command = "ping"
$website = "www.bbc.co.uk"
$cmd = "$command $website"
& $cmd
I get an error:
The term 'ping www.bbc.co.uk' is not recognized as the name of a
cmdlet, function, script file, or operable program.
Is there a way to dynamically build up a command-line as a string, and then execute it?
Yes, but you need to use Invoke-Expression (which is just like eval), instead of the call operator. Note that you also need to ensure that all your quoting is correct in that case. E.g.
$cmd = "'$command' '$website'"
would work in your trivial example, unless $command or $website contained single quotes. The problem here is essentially that everything you put into the string is subject to the usual parsing rules of PowerShell.
Generally, if you can, stay as far away from Invoke-Expression as you can. There are a few problems that need it, but invoking external programs ... not so much.
A much better alternative, especially if you have an arbitrary number of arguments, is to just collect the arguments in an array and use the splat operator (note the # in the code example below):
$command = 'ping'
$arguments = '-t','www.bbc.co.uk'
&$command #arguments
This ensures that arguments are properly quoted when necessary and generally avoids a lot of headaches you're going to get with Invoke-Expression.
(Side note: Whenever you have a problem in PowerShell and think »Oh, I'm just going to use a string«, it's often time to rethink that. This includes handling file names, or command lines. PowerShell has objects, reducing it to the capabilities of earlier shells just yields the same pain you have elsewhere too, e.g. multiple levels of escaping, sometimes with different syntaxes, etc. And most of the time there are better ways of solving the problem.)
This is a weird one. Normally when I execute an external command from powershell I use the & operator like this:
& somecommand.exe -p somearguments
However, today I came across the . operator used like this:
.$env:systemdrive\chocolatey\chocolateyinstall\chocolatey.cmd install notepadplusplus
What purpose does the period serve in this scenario? I don't get it.
The "." dot sourcing operator will send AND receive variables from other scripts you have called. The "&" call operator will ONLY send variables.
For instance, considering the following:
Script 1 (call-operator.ps1):
clear
$funny = "laughing"
$scriptpath = split-path -parent $MyInvocation.MyCommand.Definition
$filename = "laughing.ps1"
"Example 1:" # Call another script. Variables are passed only forward.
& $scriptpath\$filename
"Example 2:" # Call another script. Variables are passed backwards and forwards.
. $scriptpath\$filename
$variableDefinedInOtherScript
Script 2 (laughing.ps1):
# This is to test the passing of variables from call-operator.ps1
"I am $funny so hard. Passing variables is so hilarious."
$variableDefinedInOtherScript = "Hello World!"
Create both scripts and ONLY run the first one. You'll see that the "." dot sourcing operator sends and receives variables.
Both have their uses, so be creative. For instance, the "&" call operator would be useful if you wanted to modify the value(s) of variables in another script while preserving the original value(s) in the your current script. Kinda a safeguard. ;)
The Short:
It is a Special Operator used to achieve what regular operators cannot achieve. This particular operator . actually has two distinctively different Special Operator use cases.
The Long:
As with any other language, scripting or otherwise, PowerShell script also supports many different types of Operators to help manipulate values. These regular operators include:
Arithmetic
Assignment
Comparison
Logical
Redirection
List item
Split and Join
Type
Unary
However, PowerShell also supports whats known as Special Operators which are used to perform tasks that cannot be performed by the other types of operators.
These Special Operators Include:
#() Array subexpression operator
& Call operator
[ ] Cast operator
, Comma operator
. Dot sourcing operator
-f Format operator
[ ] Index operator
| Pipeline operator
. Property dereference operator
.. Range operator
:: Static member operator
$( ) Subexpression operator
. Dot sourcing operator: is used in this context to allow a script to run in the current scope essentially allowing any functions, aliases, and variables which has been created by the script to be added to the current script.
Example:
. c:\scripts.sample.ps1
NoteThat this application of the . Special Operator is followed by a space to distinguish it from the (.) symbol that represents the current directory
Example:
. .\sample.ps1
. Property dereference operator: Allows access to the properties and methods of of an object which follows the . by indicating that the expression on the left side of the . character is an object and the expression on the right side of the is an object member (a property or method).
Example:
$myProcess.peakWorkingSet
(get-process PowerShell).kill()
Disclaimer & Sources:
I had the same question while looking at a PowerShell script that I was trying to expand on its feature sets and landed here when doing my research for the answer. However I managed to find my answer using this magnificent write up on the Microsoft Development Network supplemented with this further expansion of the same ideas from IT Pro.
Cheers.
The dot is a call operator:
$a = "Get-ChildItem"
. $a # (executes Get-ChildItem in the current scope)
In your case, however, I don't see what it does.
.Period or .full stop for an objects properties; like
$CompSys.TotalPhysicalMemory
See here: http://www.computerperformance.co.uk/powershell/powershell_syntax.htm#Operators_
This answer is to expand slightly upon those already provided by David Brabant and his commenters. While those remarks are all true and pertinent, there is something that has been missed.
The OPs use of & when invoking external commands is unnecessary. Omitting the & would have no effect (on the example of his usage). The purpose of & is to allow the invocation of commands whose names are the values of a (string) expression. By using the & above, powershell then (essentially) treats the subsequent arguments as strings, the first of which is the command name that & duly invokes. If the & were omitted, powershell would take the first item on the line as the command to execute.
However, the . in the second example is necessary (although, as noted by others, & would work just as well in this case). Without it, the command line would begin with a variable access ($env:systemdrive) and so powershell would be expecting an expression of some form. However, immediately following the variable reference is a bare file path which is not a valid expression and will generate an error. By using the . (or &) at the beginning of the line, it is now treated as a command (because the beginning doesn't look like a valid expression) and the arguments are processed as expandable strings (" "). Thus, the command line is treated as
. "$env:systemdrive\chocolatey\chocolateyinstall\chocolatey.cmd" "install" "notepadplusplus"
The first argument has $env:systemdrive substituted into it and then . invokes the program thus named.
Note: the full description of how powershell processes command line arguments is way more complicated than that given here. This version was cut down to just the essential bits needed to answer the question. Take a look at about_Parsing for a comprehensive description. It is not complete but should cover most normal usage. There are other posts on stackoverflow and github (where powershell now resides) that cover some of the seemingly quirky behaviour not listed in the official documentation. Another useful resource is about_Operators though again this isn't quite complete. An example being the equivalence of . and & when invoking something other than a powershell script/cmdlet getting no mention at all.
I am writing a release script in powershell and need to invoke a custom executable and pass in a bunch of parameters to the executable. Few of these parameters are assigned values from various conditions and executions in the script. I see that the values are not getting evaluated when the executable is invoked. Some of the variables need to be passed in as a series of arguments inside double quotes as well. Shown below is a smaple call
mysample.exe '-forcerebuild' '-release=$CALC_VERSION' '-projfile=sample.proj' '-buildoptions="/p:AllowDevDependencies=true /p:AssemblyVersion=$CALC_ASSEMBLY_VERSION /flp:LogFile=$env:temp\build.log /p:BaseIntermediateOutputPath=$PROJECT_BASE\build\ /clp:nosummary"'
All the variables that need to be evaluated are highlighted. The exe expects all the options provided to -buildoptions inside quoted -buildoptions="all build options"
The values don't seem to get evaluated. However, if i put build options as a separate variable and print it, the values seem to get evaluated. I am new to powershell and would really appreciate any help with this.
Note that powershell doesn't do variable expansion inside single quoted strings. For example:
$test_variable = "test"
write-host "this is a $test_variable"
write-host 'this is a $test_variable'
will print
this is a test
this is a $test_variable
Also, powershell has problems in passing arguments to exes. There is a util called EchoArgs.exe that comes with Powershell Community Extensions. Use that and pass your argument to that to see if they are passed fine.