How can I effectively conserve resources using the PowerShell pipeline? - powershell

Question

I have several PowerShell scripts that I use to manage a medium sized Microsoft Exchange organization (~10,000 mailboxes). Several of the scripts process all of the organization's mailboxes in some way. One common problem I run into while running these scripts is resource exhaustion. These scripts end up using gigabytes of RAM.
My research suggests that using the pipeline avoids memory consumption because the results aren't loaded into an array prior to processing. However, under certain conditions, Get-Mailbox still seems to load the entire list of results into memory before it attempts to pass those results to the next command in the pipeline.
For instance, I assumed the following example code would start listing the mobile devices associated with each mailbox as soon as the the command is executed:
EXAMPLE 1
function GetMailboxDevices
{
process
{
Write-Host $_.Alias -ForegroundColor Green
Get-MobileDevice -Mailbox $_
}
}
Get-Mailbox -ResultSize Unlimited | GetMailboxDevices
However, this code does not appear to process the results in real time while the Get-Mailbox cmdlet is running. Instead, Get-Mailbox appears to take a few minutes to run and then passes all of the results to the second command in the pipeline at once. The PowerShell session's RAM usage climbs to 1.5 GB or higher during this process.
Nevertheless, I can work around the issue using code similar to the following:
EXAMPLE 2
function GetMailboxAliases
{
process
{
Write-Host $_.Alias -ForegroundColor Green
$_.Alias
}
}
$aliases = Get-Mailbox -ResultSize Unlimited | GetMailboxAliases
foreach ($alias in $aliases)
{
Get-MobileDevice -Mailbox $alias
}
In the second example, Get-Mailbox does pass each result down the pipeline in real time as opposed to all at once (Write-Host confirms this) and the RAM usage does not increase significantly. Of course, this code is not as elegant as I have to collect the aliases into an array and then process the array with a foreach statement.
The pipeline seems to be effective if I do something simple in the function (such as simply returning the alias of each mailbox), but the behavior changes as soon as I introduce another Exchange cmdlet into the function (such as Get-MobileDevices).
My question is this: why doesn't the code in example 1 leverage the pipeline efficiently but example 2 does? What steps can be taken to ensure the pipeline is leveraged efficiently?

Answer 1

I'am not using Exchange so mush, but in my scripts I would do this :
function GetMailboxDevices ($mb)
{
process
{
Write-Host $_.Alias -ForegroundColor Green
Get-MobileDevice -Mailbox $mb
}
}
Get-Mailbox -ResultSize Unlimited | Foreach-object { GetMailboxDevices $_}
or
Get-Mailbox -ResultSize Unlimited | % { GetMailboxDevices $_}

Answer 2

When you are running a script that returns a lot of objects, loading them to a variable is a great way to help memory usage. This is why version 2 performs better.
Also, running this script remotely from another server or workstation could also make code execution easier and make resource usage easier to troubleshoot.
You can try this to check your impressions:
Measure-Command { script1.ps1 }
Measure-command { script2.ps1 }