Get-ChildItem in an enormous directory and RAM usage - powershell

Question

I have created a PS script on a domain controller (SRV2012R2).
The script checks every shared folder (mapped drive) to see if there are any files present larger than 2GB:
foreach($dir in $Dirs)
{
$files = Get-ChildItem $dir -Recurse
foreach ($item in $files)
{
#Check if $item.Size is greater than 2GB
}
}
I have the following problem:
The shares are pretty filled with over 800GB of (sub)folders, files and most of them are just normal documents.
Whenever I run my script, I see that the CPU+RAM consumes enormous amounts while running my script (after 5 minutes into the Get-Childitem-line, the RAM has already reached >4GB).
My question is, why does Get-ChildItem need so many resources? What alternative can I use? Because I haven't manage to run my script succesfully.
I have seen that I can use | SELECT fullname, length after my Get-ChildItem-clause as an improvement, but this hasn't helped me at all (query still consumes enormous RAM).
Is there anything I can do in order that I can loop through the directories without much resistance from the machine resources?

Answer 1

Instead of saving every single file to a variable, use the pipeline to filter out the ones you don't need (ie. those smaller than 2GB):
foreach($dir in $Dirs)
{
$bigFiles = Get-ChildItem $dir -Recurse |Where Length -gt 2GB
}
If you need to process or analyze these big files further, I'd suggest you extend that pipeline with ForEach-Object:
foreach($dir in $Dirs)
{
Get-ChildItem $dir -Recurse |Where Length -gt 2GB |ForEach-Object {
# do whatever else you must
# $_ contains the current file
}
}

Answer 2

Try this:
# Create an Array
$BigFilesArray = #()
#Populate it with your data
$BigFilesArray = #(ForEach-Object($dir in $dirs) {Get-ChildItem $dir -Recurse | Where-Object Length -GT 2GB})
#Number of items in the array
$BigFilesArray.Count
#Looping to get the name and size of each item in the array
ForEach-Object ($bf in $BigFilesArray) {"$($bf.name) - $($bf.length)"}
Hope this helps!