Basically I'm trying to get the below "inline if-statement" function working (credit here)
Function IIf($If, $Then, $Else) {
If ($If -IsNot "Boolean") {$_ = $If}
If ($If) {If ($Then -is "ScriptBlock") {&$Then} Else {$Then}}
Else {If ($Else -is "ScriptBlock") {&$Else} Else {$Else}}
}
Using PowerShell v5 it doesn't seem to work for me and calling it like
IIf "some string" {$_.Substring(0, 4)} "no string found :("
gives the following error:
You cannot call a method on a null-valued expression.
At line:1 char:20
+ IIf "some string" {$_.Substring(0, 4)} "no string found :("
+ ~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
So, as a more general question, how do you make $_ available to the scriptblock passed into a function?
I kind of tried following this answer, but it seems it's meant for passing it to a separate process, which is not what I'm looking for.
Update:
It seems the issue is that I have the function in a module rather than directly in a script/PS session. A workaround would be to avoid putting it in the module, but I feel a module is more portable, so I'd like to figure out a solution for that.
There are two changes worth making, which make your problem go away:
Do not try to assign to $_ directly; it is an automatic variable under PowerShell's control, not meant to be set by user code (even though it may work situationally, it shouldn't be relied upon).
Instead, use the ForEach-Object cmdlet to implicitly set $_ via its -InputObject parameter.
Note that use of ForEach-Object with -InputObject rather than with input from the pipeline is unusual, because it results in atypical behavior: even collections passed to -InputObject are passed as a single object to the -Process block; that is, the usual enumeration does not take place; however, in the context at hand, this is precisely what is desired here: whatever $If represents should be passed as-is to the -Process script block, even if it happens to be a collection.
Use the -is operator with type literals such as [Boolean], not type names such as "Boolean".
Function IIf($If, $Then, $Else) {
If ($If) {
If ($Then -is [scriptblock]) { ForEach-Object -InputObject $If -Process $Then }
Else { $Then }
} Else {
If ($Else -is [scriptblock]) { ForEach-Object -InputObject $If -Process $Else }
Else { $Else }
}
}
As for what you tried:
In a later update you state that your IIf function is defined in a module, which explains why your attempt to set $_ by direct assignment ($_ = $If, which, as stated, is to be avoided in general), was ineffective:
It created a function-local $_ instance, which the $Then script block, due to being bound to the scope of the (module-external) caller, does not see.
The reason is that each module has its own scope domain (hierarchy of scopes aka session state), which only shares the global scope with non-module callers - see the bottom section of this answer for more information about scopes in PowerShell.
Related
I've encountered this issue in a longer script and have simplified here to show the minimal code required to reproduce it (I think). It outputs numbers followed by letters:
1 a
1 b
1 c...
2 a
2 b
2 c...
all the way to "500 z"
Function Write-HelloWorld
{
Param($number)
write-host -Object $number
}
$numbers = 1..500
$letters = "a".."z"
$Function = get-command Write-HelloWorld
$numbers | ForEach-Object -Parallel {
${function:Write-HelloWorld} = $using:Function
foreach($letter in $using:letters) {
Write-HelloWorld -number "$_ $letter"
}
}
I'm seeing 2 types of sporadically (not every time I run it):
"The term 'write-host' is not recognized as a name of a cmdlet, function, script file, or executable program." As understand it, write-host should always be available. Adding the line "Import-Module Microsoft.PowerShell.Utility" just before the call to write-host didn't help
Odd output like the below, specifically all the "write-host :" lines.
Santiago Squarzon's helpful answer demonstrates the problem with your approach well and links to a GitHub issue explaining the underlying problem (runspace affinity); however, that demonstration isn't the right solution (it wasn't meant to be), as it uses explicit synchronization to allow only one thread at a time to call the function, which negates the benefits of parallelism.
As for a solution:
You must pass a string representation of your Write-HelloWorld's function body to the ForEach-Object -Parallel call:
Function Write-HelloWorld
{
Param($number)
write-host -Object $number
}
$numbers = 1..500
$letters = "a".."z"
# Get the body of the Write-HelloWorld function *as a string*
# Alternative, as suggested by #Santiago:
# $funcDefString = (Get-Command -Type Function Write-HelloWorld).Definition
$funcDefString = ${function:Write-HelloWorld}.ToString()
$numbers | ForEach-Object -Parallel {
# Redefine the Write-HelloWorld function in this thread,
# using the *string* representation of its body.
${function:Write-HelloWorld} = $using:funcDefString
foreach($letter in $using:letters) {
Write-HelloWorld -number "$_ $letter"
}
}
${function:Write-HelloWorld} is an instance of namespace variable notation, which allows you to both get a function (its body as a [scriptblock] instance) and to set (define) it, by assigning either a [scriptblock] or a string containing the function body.
By passing a string, the function is recreated in the context of each thread, which avoids cross-thread issues that can arise when you pass a [System.Management.Automation.FunctionInfo] instance, as output by Get-Command, which contains a [scriptblock] that is bound to the runspace in which it was defined (i.e., the caller's; that script block is said to have affinity with the caller's runspace), and calling this bound [scriptblock] instance from other threads (runspaces) isn't safe.
By contrast, by redefining the function in each thread, via a string, a thread-specific [scriptblock] instance bound to that thread is created, which can safely be called.
In fact, you appear to have found a loophole, given that when you attempt to use a [scriptblock] instance directly with the $using: scope, the command by design breaks with an explicit error message:
A ForEach-Object -Parallel using variable cannot be a script block.
Passed-in script block variables are not supported with ForEach-Object -Parallel,
and can result in undefined behavior
In other words: PowerShell shouldn't even let you do what you attempted to do, but unfortunately does, as of PowerShell Core 7.2.7, resulting in the obscure failures you saw - see GitHub issue #16461.
Potential future improvement:
An enhancement is being discussed in GitHub issue #12240 to support copying the caller's state to the parallel threads on demand, which would automatically make the caller's functions available, without the need for manual redefinition.
Note, this answer is meant to prove a point but does not provide the correct solution to the problem.
See mklement0's helpful answer for the proper way to solve this by simply passing the function's definition as string to the runspaces. See also GitHub Issue #4003 for more details.
It's a very bad idea to pass in a reference object and use it without thread safety, here is proof that by simply adding thread safety to your code the problem is solved:
function Write-HelloWorld {
param($number)
Write-Host -Object $number
}
$numbers = 1..500
$letters = "a".."z"
$Function = Get-Command Write-HelloWorld
$numbers | ForEach-Object -Parallel {
$refObj = $using:Function
[System.Threading.Monitor]::Enter($refObj)
${function:Write-HelloWorld} = $using:Function
foreach($letter in $using:letters) {
Write-HelloWorld -number "$_ $letter"
}
[System.Threading.Monitor]::Exit($refObj)
}
To be precise, this issue is related to Runspace Affinity, all Runspaces are trying to send the invocation back to the origin Runspace thread hence poor PowerShell collapses.
I've encountered this issue in a longer script and have simplified here to show the minimal code required to reproduce it (I think). It outputs numbers followed by letters:
1 a
1 b
1 c...
2 a
2 b
2 c...
all the way to "500 z"
Function Write-HelloWorld
{
Param($number)
write-host -Object $number
}
$numbers = 1..500
$letters = "a".."z"
$Function = get-command Write-HelloWorld
$numbers | ForEach-Object -Parallel {
${function:Write-HelloWorld} = $using:Function
foreach($letter in $using:letters) {
Write-HelloWorld -number "$_ $letter"
}
}
I'm seeing 2 types of sporadically (not every time I run it):
"The term 'write-host' is not recognized as a name of a cmdlet, function, script file, or executable program." As understand it, write-host should always be available. Adding the line "Import-Module Microsoft.PowerShell.Utility" just before the call to write-host didn't help
Odd output like the below, specifically all the "write-host :" lines.
Santiago Squarzon's helpful answer demonstrates the problem with your approach well and links to a GitHub issue explaining the underlying problem (runspace affinity); however, that demonstration isn't the right solution (it wasn't meant to be), as it uses explicit synchronization to allow only one thread at a time to call the function, which negates the benefits of parallelism.
As for a solution:
You must pass a string representation of your Write-HelloWorld's function body to the ForEach-Object -Parallel call:
Function Write-HelloWorld
{
Param($number)
write-host -Object $number
}
$numbers = 1..500
$letters = "a".."z"
# Get the body of the Write-HelloWorld function *as a string*
# Alternative, as suggested by #Santiago:
# $funcDefString = (Get-Command -Type Function Write-HelloWorld).Definition
$funcDefString = ${function:Write-HelloWorld}.ToString()
$numbers | ForEach-Object -Parallel {
# Redefine the Write-HelloWorld function in this thread,
# using the *string* representation of its body.
${function:Write-HelloWorld} = $using:funcDefString
foreach($letter in $using:letters) {
Write-HelloWorld -number "$_ $letter"
}
}
${function:Write-HelloWorld} is an instance of namespace variable notation, which allows you to both get a function (its body as a [scriptblock] instance) and to set (define) it, by assigning either a [scriptblock] or a string containing the function body.
By passing a string, the function is recreated in the context of each thread, which avoids cross-thread issues that can arise when you pass a [System.Management.Automation.FunctionInfo] instance, as output by Get-Command, which contains a [scriptblock] that is bound to the runspace in which it was defined (i.e., the caller's; that script block is said to have affinity with the caller's runspace), and calling this bound [scriptblock] instance from other threads (runspaces) isn't safe.
By contrast, by redefining the function in each thread, via a string, a thread-specific [scriptblock] instance bound to that thread is created, which can safely be called.
In fact, you appear to have found a loophole, given that when you attempt to use a [scriptblock] instance directly with the $using: scope, the command by design breaks with an explicit error message:
A ForEach-Object -Parallel using variable cannot be a script block.
Passed-in script block variables are not supported with ForEach-Object -Parallel,
and can result in undefined behavior
In other words: PowerShell shouldn't even let you do what you attempted to do, but unfortunately does, as of PowerShell Core 7.2.7, resulting in the obscure failures you saw - see GitHub issue #16461.
Potential future improvement:
An enhancement is being discussed in GitHub issue #12240 to support copying the caller's state to the parallel threads on demand, which would automatically make the caller's functions available, without the need for manual redefinition.
Note, this answer is meant to prove a point but does not provide the correct solution to the problem.
See mklement0's helpful answer for the proper way to solve this by simply passing the function's definition as string to the runspaces. See also GitHub Issue #4003 for more details.
It's a very bad idea to pass in a reference object and use it without thread safety, here is proof that by simply adding thread safety to your code the problem is solved:
function Write-HelloWorld {
param($number)
Write-Host -Object $number
}
$numbers = 1..500
$letters = "a".."z"
$Function = Get-Command Write-HelloWorld
$numbers | ForEach-Object -Parallel {
$refObj = $using:Function
[System.Threading.Monitor]::Enter($refObj)
${function:Write-HelloWorld} = $using:Function
foreach($letter in $using:letters) {
Write-HelloWorld -number "$_ $letter"
}
[System.Threading.Monitor]::Exit($refObj)
}
To be precise, this issue is related to Runspace Affinity, all Runspaces are trying to send the invocation back to the origin Runspace thread hence poor PowerShell collapses.
Here's what I'm trying to do:
param([Switch]$myparameter)
If($myparamter -eq $true) {$export = Export-CSV c:\temp\temp.csv}
Get-MyFunction | $export
If $myparameter is passed, export the data to said location. Else, just display the normal output (in other words, ignore the $export). What doesn't work here is setting $export to the "Export-csv...". Wrapping it in quotes does not work.
I'm trying to avoid an if, then statement saying "if it's passed, export this. If it's not passed, output data"
I have a larger module that everything works in so there is a reason behind why I am looking to do it this way. Please let me know if any additional information is needed.
Thank you everyone in advance.
tl;dr:
param([Switch] $myparameter)
# Define the core command as a *script block* (enclosed in { ... }),
# to be invoked later, either with operator . (no child variable scope)
# or & (with child variable scope)
$scriptBlock = { Get-MyFunction }
# Invoke the script block with . (or &), and pipe it to the Export-Csv cmdlet,
# if requested.
If ($myparameter) { # short for: ($myparameter -eq $True), because $myparameter is a switch
. $scriptBlock | Export-Csv c:\temp\temp.csv
} else {
. $scriptBlock
}
TessellatingHeckler's answer is concise, works, and uses a number of advanced features cleverly - however, while it avoids an if statement, as requested, doing so may not yield the best or most readable solution in this case.
What you're looking for is to store a command in a variable for later execution, but your own attempt to do so:
If ($myparameter -eq $true) { $export = Export-CSV c:\temp\temp.csv }
results in immediate execution, which is not only unintended, but fails, because the Export-Csv cmdlet is missing input in the above statement.
You can store a snippet of source code for later execution in a variable via a script block, simply by enclosing the snippet in { ... }, which in your case would mean:
If ($myparameter -eq $true) { $export = { Export-Csv c:\temp\temp.csv } }
Note that what you pass to if is itself a script block, but it is by definition one that is executed as soon as the if condition is found to be true.
A variable containing a script block can then be invoked on demand, using one of two operators:
., the "dot-sourcing" operator, which executes the script block in the current scope.
&, the call operator, which executes the script block in a child scope with respect to potential variable definitions.
However, given that you only need the pipeline with an additional command if switch $myparameter is specified, it's better to change the logic:
Store the shared core command, Get-MyFunction, in a script block, in variable $scriptBlock.
Invoke that script block in an if statement, either standalone (by default), or by piping it to Export-Csv (if -MyParameter was specified).
I'm trying to avoid an if, then statement
Uh, if you insist...
param([Switch]$myparameter)
$cmdlet, $params = (('Write-output', #{}),
('Export-Csv', #{'LiteralPath'='c:\temp\temp.csv'}))[$myparameter]
Get-MyFunction | & $cmdlet #params
What works -
Lets say I have a scriptblock which I use with Select-Object cmdlet.
$jobTypeSelector = `
{
if ($_.Type -eq "Foo")
{
"Bar"
}
elseif ($_.Data -match "-Action ([a-zA-Z]+)")
{
$_.Type + " [" + $Matches[1] + "]"
}
else
{
$_.Type
}
}
$projectedData = $AllJobs | Select-Object -Property State, #{Name="Type"; Expression=$jobTypeSelector}
This works fine, and I get the results as expected.
What I am trying to do -
However, at a later point in code, I want to reuse the scriptblock defined as $jobTypeSelector.
For example, I expected below code to take $fooJob (note that it is a single object) passed as parameter below, and be used for $_ automatic variable in the scriptblock and return me the same result, as it returns when executed in context of Select-Object cmdlet.
$fooType = $jobTypeSelector.Invoke($fooJob)
What doesn't work -
It does not work as I expected and I get back empty value.
What I have already tried -
I checked, the properties are all correctly set, and it's not due to the relevant property itself being blank or $null.
I looked on the internet, and it's seemed pretty hard to find any relevant page on the subject, but I eventually found one which was close to explaining the issue in a slightly different context - calling the script blocks in PowerShell. The blog doesn't directly answer my question, and any relevant explanation only leads to a solution which would be very ugly, hard to read and maintain in my opinion.
Question -
So, what is the best way to invoke a scriptblock for a single object, which uses $_ automatic variable as parameter, (instead of param block)
After fiddling around with varoius options, I ended up sloving the problem, in a sort of Hackish way.. But I find it to be the best solution because it's small, easy to read, maintain and understand.
Even though we are talking about single object, use it in the pipeline (which is when PowerShell defines the $_ automatic variable) with ForEach-Object cmdlet
$fooType = $fooJob | ForEach-Object $jobTypeSelector
You can certainly use foreach or ForEach-Object as you mention.
You can also pipe to the ScriptBlock directly, if you change it from a function ScriptBlock to a filter ScriptBlock by setting IsFilter to $true:
$jobTypeSelector.IsFilter = $true
$fooType = $fooJob | $jobTypeSelector
But, what would be even better is if you used a named function instead of an anonymous ScriptBlock, for example:
function Get-JobType
{
Param (
[object] $Job
)
if ($Job.Type -eq "Foo")
{
"Bar"
}
elseif ($Job.Data -match "-Action ([a-zA-Z]+)")
{
$Job.Type + " [" + $Matches[1] + "]"
}
else
{
$Job.Type
}
}
Then you can use it with Select-Object aka select like this:
$projectedData = $AllJobs |
select -Property State, #{Name="Type"; Expression={Get-JobType $_}}
Or with a single job, like this:
$fooType = Get-JobType $fooJob
I'm trying to understand Powershell, but find somethings not so intuitive. What I understand of it is that in the pipeline objects are passed, instead of traditionally text. And $_ refers to the current object in the pipeline. Then, why is the following not working:
get-date|Write-Host "$_"
The errormessage is:
Write-Host : The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the input and its properties do not matc
h any of the parameters that take pipeline input.
At line:1 char:10
+ get-date|Write-Host $_
+ ~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (10-9-2014 15:17:00:PSObject) [Write-Host], ParameterBindingException
+ FullyQualifiedErrorId : InputObjectNotBound,Microsoft.PowerShell.Commands.WriteHostCommand
$_ is the current single item in the pipeline. To write each item in the pipeline you would write
get-data | foreach { Write-Host $_ }
Or in the short form
get-data |% { Write-Host $_ }
Conceptually, Foreach is a cmdlet that receives a function parameter, a pipeline input and applies the function on each item of the pipeline. You can't just write code with $_ - you need to have a function explicitly states that it agrees to receive pipeline input
And $_ refers to the current object in the pipeline
Indeed, the automatic $_ variable refers to the current pipeline object, but only in script blocks { ... }, notably those passed to the ForEach-Object and Where-Object cmdlets.
Outside of script blocks it has no meaningful value.
Therefore, the immediate fix to your command is the following:
Get-Date | ForEach-Object { Write-Host $_ }
However, note that:
Write-Host is is typically the wrong tool to use, unless the intent is to write to the display only, bypassing the success output stream and with it the ability to send output to other commands, capture it in a variable, or redirect it to a file.
To output a value, use it by itself; e.g, $value, instead of Write-Host $value (or use Write-Output $value); see this answer. To explicitly print only to the display but with rich formatting, use Out-Host.
Therefore, if merely outputting each pipeline input object is the goal, Get-Date | ForEach-Object { $_ } would do, where the ForEach-Object call is redundant if each input object is to simply be passed through (without transformation); that is, in the latter case just Get-Date would do.
As for what you tried:
get-date|Write-Host "$_"
As noted, the use of $_ in this context is pointless, but the reason for the error message you saw is unrelated to that problem:
Instead, the reason for the error is that you're mistakenly trying to provide input to Write-Host both via the pipeline Get-Date | Write-Host ... and by way of an argument (... | Write-Host "...")
Given that the argument ("$_") (positionally) binds to the -Object parameter, the pipeline input then has no parameter left to bind to, which causes the error at hand.