Unexpected behavior in function processing PowerShell pipeline - powershell

I created a function to process items I want from an object and sort them into a new PSCustomObject. If I pass the object through the pipeline I get some duplicated and odd results versus passing the object as a parameter into the function and using a ForEach-Object loop.
Here is my example (this would produce 3 records):
$audioSessions | Where-Object {$_.QoeReport.FeedbackReports}
Versus this (which produces six and some are duplicated:
$audioSessions | Where-Object {$_.QoeReport.FeedbackReports} | ProcessFeedback
Here is the difference in the output:
Any idea why this would be happening? There are 3 objects I'm passing to the ProcessFeedback function, no? Why are some items duplicated and some are not?
If I choose to pass the entire variable into the function and loop within it, I get the 3 objects back from my function as expected:
ProcessFeedback -feedbackInput $audioSessions
Then, inside my function I do the filter with the Where-Object statement resulting in something like this:
function ProcessFeedback{
[cmdletbinding()]
Param(
[Parameter(mandatory=$true, valuefrompipeline=$true)]
$feedbackInput
)
begin{}
process{
$feedbackInput | Where-Object {$_.QoeReport.FeedbackReports} | ForEach-Object{
[array]$newObject += [PSCustomObject][ordered]#{
FromUri = $_.FromUri
ToUri = $_.ToUri
CaptureTime = $_.QoeReport.FeedbackReports.CaptureTime
Rating = $_.QoeReport.FeedBackReports.Rating
}
}
return $newObject
}
}
NOTE: When I pass the object through the pipeline, I remove the Where-Object statement in the ProcessFeedback function as I only ever see one object passed to it at a time.

Okay so I think I figured this out...
If you're passing multiple objects through the pipeline, there is no need to add the results together.
I simply changed the code to remove the [array] and += values from $newObject as follows:
function ProcessFeedback{
begin{}
process{
$newObject = [PSCustomObject][ordered]#{
FromUri = $_.FromUri
ToUri = $_.ToUri
CaptureTime = $_.QoeReport.FeedbackReports.CaptureTime
Rating = $_.QoeReport.FeedBackReports.Rating
}
}
return $newObject
}
}
Then run it like this:
[array]$arrFeedbackResults += $audioSessions | Where-Object {$_.QoeReport.FeedbackReports} | ProcessFeedback
I also removed the input parameter from the function as the object is passed through the pipeline anyway.

Related

Tab-complete a parameter value based on another parameter's already specified value

This question addresses the following scenario:
Can custom tab-completion for a given command dynamically determine completions based on the value previously passed to another parameter on the same command line, using either a parameter-level [ArgumentCompleter()] attribute or the Register-ArgumentCompleter cmdlet?
If so, what are the limitations of this approach?
Example scenario:
A hypothetical Get-Property command has an -Object parameter that accepts an object of any type, and a -Property parameter that accepts the name of a property whose value to extract from the object.
Now, in the course of typing a Get-Property call, if a value is already specified for -Object, tab-completing -Property should cycle through the names of the specified object's (public) properties.
$obj = [pscustomobject] #{ foo = 1; bar = 2; baz = 3 }
Get-Property -Object $obj -Property # <- pressing <tab> here should cycle
# through 'foo', 'bar', 'baz'
#mklement0, regarding first limitation stated in your answer
The custom-completion script block ({ ... }) invoked by PowerShell fundamentally only sees values specified via parameters, not via the pipeline.
I struggled with this, and after some stubbornness I got a working solution.
At least good enough for my tooling, and I hope it can make life easier for many others out there.
This solution has been verified to work with PowerShell versions 5.1 and 7.1.2.
Here I made use of $cmdAst (called $commandAst in the docs), which contains information about the pipeline. With this we can get to know the previous pipeline element and even differentiate between it containing only a variable or a command. Yes, A COMMAND, which with help of Get-Command and the command's OutputType() member method, we can get (suggested) property names for such as well!
Example usage
PS> $obj = [pscustomobject] #{ foo = 1; bar = 2; baz = 3 }
PS> $obj | Get-Property -Property # <tab>: bar, baz, foo
PS> "la", "na", "le" | Select-String "a" | Get-Property -Property # <tab>: Chars, Context, Filename, ...
PS> 2,5,2,2,6,3 | group | Get-Property -Property # <tab>: Count, Values, Group, ...
Function code
Note that apart from now using $cmdAst, I also added [Parameter(ValueFromPipeline=$true)] so we actually pick the object, and PROCESS {$Object.$Property} so that one can test and see the code actually working.
param(
[Parameter(ValueFromPipeline=$true)]
[object] $Object,
[ArgumentCompleter({
param($cmdName, $paramName, $wordToComplete, $cmdAst, $preBoundParameters)
# Find out if we have pipeline input.
$pipelineElements = $cmdAst.Parent.PipelineElements
$thisPipelineElementAsString = $cmdAst.Extent.Text
$thisPipelinePosition = [array]::IndexOf($pipelineElements.Extent.Text, $thisPipelineElementAsString)
$hasPipelineInput = $thisPipelinePosition -ne 0
$possibleArguments = #()
if ($hasPipelineInput) {
# If we are in a pipeline, find out if the previous pipeline element is a variable or a command.
$previousPipelineElement = $pipelineElements[$thisPipelinePosition - 1]
$pipelineInputVariable = $previousPipelineElement.Expression.VariablePath.UserPath
if (-not [string]::IsNullOrEmpty($pipelineInputVariable)) {
# If previous pipeline element is a variable, get the object.
# Note that it can be a non-existent variable. In such case we simply get nothing.
$detectedInputObject = Get-Variable |
Where-Object {$_.Name -eq $pipelineInputVariable} |
ForEach-Object Value
} else {
$pipelineInputCommand = $previousPipelineElement.CommandElements[0].Value
if (-not [string]::IsNullOrEmpty($pipelineInputCommand)) {
# If previous pipeline element is a command, check if it exists as a command.
$possibleArguments += Get-Command -CommandType All |
Where-Object Name -Match "^$pipelineInputCommand$" |
# Collect properties for each documented output type.
ForEach-Object {$_.OutputType.Type} | ForEach-Object GetProperties |
# Group properties by Name to get unique ones, and sort them by
# the most frequent Name first. The sorting is a perk.
# A command can have multiple output types. If so, we might now
# have multiple properties with identical Name.
Group-Object Name -NoElement | Sort-Object Count -Descending |
ForEach-Object Name
}
}
} elseif ($preBoundParameters.ContainsKey("Object")) {
# If not in pipeline, but object has been given, get the object.
$detectedInputObject = $preBoundParameters["Object"]
}
if ($null -ne $detectedInputObject) {
# The input object might be an array of objects, if so, select the first one.
# We (at least I) are not interested in array properties, but the object element's properties.
if ($detectedInputObject -is [array]) {
$sampleInputObject = $detectedInputObject[0]
} else {
$sampleInputObject = $detectedInputObject
}
# Collect property names.
$possibleArguments += $sampleInputObject | Get-Member -MemberType Properties | ForEach-Object Name
}
# Refering to about_Functions_Argument_Completion documentation.
# The ArgumentCompleter script block must unroll the values using the pipeline,
# such as ForEach-Object, Where-Object, or another suitable method.
# Returning an array of values causes PowerShell to treat the entire array as one tab completion value.
$possibleArguments | Where-Object {$_ -like "$wordToComplete*"}
})]
[string] $Property
)
PROCESS {$Object.$Property}
Update: See betoz's helpful answer for a more complete solution that also supports pipeline input.
The part of the answer below that clarifies the limitations of pre-execution detection of the input objects' data type still applies.
The following solution uses a parameter-specific [ArgumentCompleter()] attribute as part of the definition of the Get-Property function itself, but the solution analogously applies to separately defining custom-completion logic via the Register-CommandCompleter cmdlet.
Limitations:
[See betoz's answer for how to overcome this limitation] The custom-completion script block ({ ... }) invoked by PowerShell fundamentally only sees values specified via parameters, not via the pipeline.
That is, if you type Get-Property -Object $obj -Property <tab>, the script block can determine that the value of $obj is to be bound to the -Object parameter, but that wouldn't work with
$obj | Get-Property -Property <tab> (even if -Object is declared as pipeline-binding).
Fundamentally, only values that can be evaluated without side effects are actually accessible in the script block; in concrete terms, this means:
Literal values (e.g., -Object ([pscustomobject] #{ foo = 1; bar = 2; baz = 3 })
Simple variable references (e.g., -Object $obj) or property-access or index-access expressions (e.g., -Object $obj.Foo or -Object $obj[0])
Notably, the following values are not accessible:
Method-call results (e.g., -Object $object.Foo())
Command output (via (...), $(...), or #(...), e.g.
-Object (Invoke-RestMethod http://example.org))
The reason for this limitation is that evaluating such values before actually submitting the command could have undesirable side effects and / or could take a long time to complete.
function Get-Property {
param(
[object] $Object,
[ArgumentCompleter({
# A fixed list of parameters is passed to an argument-completer script block.
# Here, only two are of interest:
# * $wordToComplete:
# The part of the value that the user has typed so far, if any.
# * $preBoundParameters (called $fakeBoundParameters
# in the docs):
# A hashtable of those (future) parameter values specified so
# far that are side effect-free (see above).
param($cmdName, $paramName, $wordToComplete, $cmdAst, $preBoundParameters)
# Was a side effect-free value specified for -Object?
if ($obj = $preBoundParameters['Object']) {
# Get all property names of the objects and filter them
# by the partial value already typed, if any,
# interpreted as a name prefix.
#($obj.psobject.Properties.Name) -like "$wordToComplete*"
}
})]
[string] $Property
)
# ...
}

How to pass foreach-object output to a function accepting pipeline input?

This is a very simple issue as far as I understand. There are plenty of similar questions on here, but I haven't been able to find exactly what I need. What am I missing?
Expected output
1 2 3
Actual output (error)
cmdlet ForEach-Object at command pipeline position 1
Supply values for the following parameters:
Process[0]:
Code
function processItem {
param($item)
Process {
$item
}
}
$collection = #(1,2,3)
$collection | foreach-object | processItem
First, you don't have to use Foreach-Object here because the pipeline will directly unwrap $items and send one value at a time to your function processItem.
Passing Arrays to Pipeline
If a function returns more than one value, PowerShell wraps them in an array. However, if you pass the results to another function inside a pipeline, the pipeline automatically "unwraps" the array and processes one array element at a time.
The parameter $item in the function doesn't accept pipeline input in your code, you should use ValueFromPipeline like this:
function processItem {
param([parameter(ValueFromPipeline=$true)]$item)
Process {
$item
}
}
Use like this:
$items = #(1, 2, 3)
$items | processItem

Pipe a single object and process it without For-EachObject

Original Question
I a piping a single string and processing it with For-EachObject as follows:
"Test" | % { $_.Substring(0,1) }
It seems wrong to process a single piped item using For-EachObject, partly because it's misleading to future code maintainers. I don't know any other way, though, to capture the string while saying "it's just a single item." For instance, this doesn't work.
"Test" | $_.Substring(0,1)
"Test" | { $_.Substring(0,1) }
How can I process a single object while indicating that I expect only one?
Edit: Add the actual use case
The above is a simplified version of what I'm actually trying to accomplish. I am getting the first paragraph of a Wikipedia article, which is part of a larger function that saves the result to a file.
curl "www.wikipedia.org/wiki/Hope,_British_Columbia" |
select -expand allelements |
? { $_.id -eq "mw-content-text" } |
select -expand innerHTML |
% {
$i = $_.IndexOf("<P>");
$j = $_.IndexOf("</P>");
$_.Substring($i, $j - $i) -replace '<[^>]*>'
}
The part that needs to process a single object follows the select -expand innerHtml expression. Piping is my preferred way because putting multiple parenthesis around the curl part seems ugly.
Aliases
curl is Invoke-WebRequest
select is Select-Object
-expand is ExplandProperty
? is Where-Object
% is For-EachObject
If you are creating single-purpose code where you control both the input and the output, and there will always be only one object, then using the pipeline is overkill and not really appropriate. Just pass the string as a parameter to your function.
function f([String]$s) {
$s.Substring(0,1)
}
PS> f "Test"
T
If you're building a general-purpose function to take input from the pipeline, your function needs to account for more than one object in the stream. Fortunately PowerShell has a natural way to do this using the Process{} block, which is executed once for each item in the input pipeline.
function f {
param(
[Parameter(ValueFromPipeline=$true)]
[String]$item
)
process {
$item.Substring(0,1)
}
}
PS> '123','abc','#%#' | f
1
a
#
This is a common enough function that PowerShell has a shorthand for writing a function that takes one parameter from the pipeline and only contains a process block.
filter f {
$_.SubString(0,1)
}
PS> '123','abc','#%#' | f
1
a
#

How to Pass Multiple Objects via the Pipeline Between Two Functions in Powershell

I am attempting to pass a list of objects from one function to another, one by one.
First function: generate a list of users (objects) near expiry;
Second function: send an email to each user (object)
The first function works fine and outputs a group of objects (or so it would seem) and the second function will accept input and email a single user without issue.
Issues arise only when multiple objects are passed from the first function to the second.
Relevant code snippets are below:
The First function creates a custom object for each located user and adds it to an array, which is then outputted in the end block. Below is an extremely simplified snippet of the code with the essential object creation step:
Function 01
{
#param block goes here etc...
Foreach ($user in $users)
{
$userOutput = #()
$userTable = New-Object PSObject -Property #{
name = $User.Name
SamAccountName = $User.SamAccountName
emailAddress = $User.EmailAddress
expired = $user.PasswordExpired
expiryDate = $ExpiryDate.ToShortDateString()
daysTillExpiry = $daysTillExpiry
smtpRecipientAddress = $User.EmailAddress
smtpRecipientName = $User.Name
}
$userOutput += $userTable
}
Write-Output $userOutput
}
I have also tried writing each custom object ($userTable) straight to the console within each iteration of the Foreach (users) loop.
The Second function accepts pipeline input for a number of matching parameters from the first function, e.g:
[Parameter(Mandatory=$true,ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)][string]$smtpRecipientName
The second function also calls a third function designed specifically to send smtp mail and contains no loops, it just takes the current object from the pipeline and deals with it.
I haven't included the full code for either mail function because it is largely irrelevant. I just want to know whether the objects outputted from the first function can be dealt with one-by-one by the second.
At present, the mail function deals with the first object passed to it, and no others.
Update:
This is what I have in mind (but the second function only deals with the last object that was piped in:
Function Test-UserExp
{
$iteration = 0
For ($i=0;$i -le 9;$i++)
{
$iteration ++
$userTable = New-Object PSObject -Property #{
expiryDate = "TestExpDate_$iteration"
daysTillExpiry = "TestDaysTillExpiry_$iteration"
smtpRecipientAddress = "TestSMTPRecipientAddress_$iteration"
smtpRecipientName = "TestSMTPRecipientName_$iteration"
}
$userTable
}
}
Function Test-MailSend
{
Param
(
[Parameter(ValueFromPipelineByPropertyName=$true)][string]$expiryDate,
[Parameter(ValueFromPipelineByPropertyName=$true)][string]$daysTillExpiry,
[Parameter(ValueFromPipelineByPropertyName=$true)][string]$smtpRecipientAddress,
[Parameter(ValueFromPipelineByPropertyName=$true)][string]$smtpRecipientName
)
Write-Host 'Output from Test-MailSend:'
$expiryDate
$daysTillExpiry
$smtpRecipientAddress
$smtpRecipientName
}
First of all: if you want to process objects in a pipeline, one at the time, do not kill experience by collecting all the objects - that's only necessary if you intend to do something about whole collection at some point. If not than just output objects as soon as you get them:
foreach ($user in $users) {
New-Object PSObject -Property #{
name = $User.Name
SamAccountName = $User.SamAccountName
emailAddress = $User.EmailAddress
# ...
}
}
In your case you output whole collection at the end. That's hardly a pipeline experience if you would ask me.
For the second command: if you intend to create parameter for each property, just leave the part 'ValueFromPipeline' out. Otherwise you may end up with whole object converted to string... If you want to take an object as a whole, leave out 'ValueFromPipelineByPropertyName' and specify correct type. And make sure you have process {} wrapped around the code that uses parameters taken from pipeline.
And finally: why would you write a function to send mails? You have Send-MailMessage, so unless you do something this cmdlet doesn't cover, you probably don't need hand-crafted replacement...
In function 1 you want to create the array before the ForEach loop, so you aren't re-creating the array every iteration.
In the param block for the second function, you want to declare the parameter as an array of strings, not just a string.
Finally, when accepting pipeline input for the second function you will need to use the Begin, Process, and End blocks. The part of the function that repeats for each item should be in the Process block.
Here is a short working sample below:
Function fun1{
$users = #(1,2,3)
$userOutput = #()
Foreach ($user in $users){
$userTable = New-Object PSObject -Property #{
emailAddress = "$user#blah.com"
}
$userOutput += $userTable
}
$userOutput
}
Function fun2{
param(
[parameter(ValueFromPipeLine=$true)]
[String[]]$Recipients
)
begin{}
process{
ForEach ($Recipient in $Recipients){
$_
}
}
end{}
}
fun1 | Select emailAddress | fun2
This will give you the output below:
emailAddress
------------
1#blah.com
2#blah.com
3#blah.com
Here is a great breakdown of how the Begin/Process/End blocks work in PowerShell http://technet.microsoft.com/en-us/magazine/hh413265.aspx
function Set-UserExpiry {
1..10 | foreach {
[PSCustomObject]#{
ExpiryDate = "TestExpDate_$_"
DaysTillExpiry = "TestDaysTillExpiry_$_"
SmtpRecipientAddress = "TestSMTPRecipientAddress_$_"
SmtpRecipientName = "TestSMTPRecipientName_$_"
}
}
}
function Test-UserExpiry {
param
(
[Parameter(ValueFromPipelineByPropertyName = $true)]
[string]$ExpiryDate,
[Parameter(ValueFromPipelineByPropertyName = $true)]
[string]$DaysTillExpiry,
[Parameter(ValueFromPipelineByPropertyName = $true)]
[string]$SmtpRecipientAddress,
[Parameter(ValueFromPipelineByPropertyName = $true)]
[string]$SmtpRecipientName
)
process {
Write-Output 'Output from Test-MailSend:'
$expiryDate
$daysTillExpiry
$smtpRecipientAddress
$smtpRecipientName
Write-Output ''
}
}
Set-UserExpiry | Test-UserExpiry

ArrayList Unrolling

Powershell unrolling is driving me crazy.
I have the following code to retrieve email addresses from an exchange recipient. I'm using the ArrayList because it is suggested by many people when you want the ability to remove items from the array.
$aliases = New-Object System.Collections.ArrayList
$smtpAddresses = (Get-Recipient $this.DN).EmailAddresses | ?{$_.Prefix.ToString() -eq 'smtp' }
foreach ($smtpAddress in $smtpAddresses) {
$aliases.Add($smtpAddress.SmtpAddress)
}
return $aliases
The value of $aliases is correct at the end of the function (i.e. will contain x email addresses and is type ArrayList) but after returning it becomes System.Object[] and has 2x entries. There x Int32's followed by x Strings (i.e. {0, 1, bob#here, bob#there} ). Why does this happen and how to I keep my ArrayList intact? Am I wrong for using ArrayList?
Out of curiosity, with all the questions/problems resulting from PS unrolling, what is its purpose? The big benefit of powershell is that you work directly with objects instead of their textual projections, unfortunately, I never know what kind of object I'm working with - and even when I check, it doesn't seem to hold its shape for more than a few lines of code.
-- Edit
The function is called as part of a PSObject
$get_aliases = { ... }
$obj | Add-Member -MemberType ScriptProperty -Name Aliases -Value $get_aliases -SecondValue $set_aliases
Part of the problem is how the array is being used inside the function. Remember, a function in PowerShell doesn't actually return anything. It writes objects to the pipeline. Therefore, the return is superfluous, but not actually causing any problems. The use of the Add function is causing the problem because Add returns the index at which the value was added and therefore writes to the pipeline as well.
function get-myarray
{
$al = New-Object System.Collections.ArrayList
$al.Add( 0 )
$al.Add( 1 )
$al.Add( 'me#co.com' )
$al.Add( 'you.co.com' )
return $al
}
$array = get-myarray
$array.Count
8
Note how the size is 8. What needs to be done is to suppress the writing of what is returned by the Add function. There are a few ways to do this but here is one:
function get-myarray
{
$al = New-Object System.Collections.ArrayList
$al.Add( 0 ) | out-null
$al.Add( 1 ) | out-null
$al.Add( 'me#co.com' ) | out-null
$al.Add( 'you.co.com' ) | out-null
return $al
}
$array = get-myarray
$array.Count
4
I don't believe the use of `ArrayList' is a wrong one if you want to remove items from it.
As far as unrolling goes, this deserves a whole other question and has been already addressed.
How are you return the $aliases?
like this?
$a = MyFunctionName # In this way `$a` is of type [OBJECT[]]
You can try this way:
[System.Collections.ArrayList]$a = MyFunctionName
after you can know the type in this way:
$a.item(0).gettype()