Referencing targeted object in ForEach-Object in Powershell - powershell

I am fairly new to Powershell. I am attempting to re-write the names of GPO backup folders to use their friendly name rather than their GUID by referencing the name in each GPO backup's 'gpresult.xml' file that is created as part of the backup. However, I do not understand how I can reference the specific object (in this case, the folder name) that is being read into the ForEach-Object loop in order to read into the file beneath this folder.
function Backup_GPO {
$stamp = Get-Date -UFormat "%m%d"
New-Item -ItemType "directory" -Name $stamp -Path \\dc-16\share\GPO_Backups -Force | out-null # create new folder to specify day backup is made
Backup-GPO -All -Path ("\\dc-16\share\GPO_Backups\" + "$stamp\" )
Get-ChildItem -Path \\dc-16\share\GPO_Backups\$stamp | ForEach-Object {
# I want to reference the current folder here
[xml]$file = Get-Content -Path (folder that is being referenced in for loop)\gpresult.xml
$name = $file.GPO.Name
}
I'm coming from Python, where if I want to reference the object I'm currently iterating on, I can do so very simply -
for object in list:
print(object)
How do you reference the currently in-use object in Powershell's ForEach-Object command?

I'm coming from Python, where if I want to reference the object I'm currently
iterating on, I can do so very simply -
for object in list:
print(object)
The direct equivalent of that in PowerShell is the foreach statement (loop):
$list = 1..3 # create a 3-element array with elements 1, 2, 3
foreach ($object in $list) {
$object # expression output is *implicitly* output
}
Note that you cannot directly use a foreach statement in a PowerShell pipeline.
In a pipeline, you must use the ForEach-Object cmdlet instead, which - somewhat confusingly - can also be referred to as foreach, via an alias - it it is only the parsing mode that distinguishes between the statement and the cmdlet's alias.
You're using the ForEach-Object cmdlet in the pipeline, where different rules apply.
Script blocks ({ ... }) passed to pipeline-processing cmdlets such as ForEach-Object and Where-Object do not have an explicit iteration variable the way that the foreach statement provides.
Instead, by convention, such script blocks see the current pipeline input object as automatic variable $_ - or, more verbosely, as $PSItem.
While the foreach statement and the ForEach-Object cmdlet operate the same on a certain level of abstraction, there's a fundamental difference:
The foreach statement operates on collections collected up front in memory, in full.
The ForEach-Object cmdlet operates on streaming input, object by object, as each object is being received via the pipeline.
This difference amounts to the following trade-off:
Use the foreach statement for better performance, at the expense of memory usage.
Use the ForEach-Object cmdlet for constant memory use and possibly also for the syntactic elegance of a single pipeline, at the expense of performance - however, for very large input sets, this may be the only option (assuming you don't also collect a very large dataset in memory on output).

Inside the ForEach-Object scriptblock, the current item being iterated over is copied to $_:
Get-ChildItem -Filter gpresult.xml |ForEach-Object {
# `$_` is a FileInfo object, `$_.FullName` holds the absolute file system path
[xml]$file = Get-Content -LiteralPath $_.FullName
}
If you want to specify a custom name, you can either specify a -PipelineVariable name:
Get-ChildItem -Filter gpresult.xml -PipelineVariable fileinfo |ForEach-Object {
# `$fileinfo` is now a FileInfo object, `$fileinfo.FullName` holds the absolute file system path
[xml]$file = Get-Content -LiteralPath $fileinfo.FullName
}
or use a foreach loop statement, much like for object in list in python:
foreach($object in Get-ChildItem -Filter gpresult.xml)
{
[xml]$file = Get-Content -LiteralPath $object.FullName
}

Another way...
$dlist = Get-ChildItem -Path "\\dc-16\share\GPO_Backups\$stamp"
foreach ($dir in $dlist) {
# I want to reference the current folder here
[xml]$file = Get-Content -Path (Join-Path -Path $_.FullName -ChildPath 'gpresult.xml')
$name = $file.GPO.Name
}

Here's my solution. It's annoying that $_ doesn't have the full path. $gpath is easier to work with than $_.fullname for joining the two strings together on the next line with get-content. I get a gpreport.xml file when I try backup-gpo. Apparently you can't use relative paths like .\gpo_backups\ with backup-gpo.
mkdir c:\users\js\gpo_backups\
get-gpo -all | where displayname -like '*mygpo*' |
backup-gpo -path c:\users\js\gpo_backups\
Get-ChildItem -Path .\GPO_Backups\ | ForEach-Object {
$gpath = $_.fullname
[xml]$file = Get-Content -Path "$gpath\gpreport.xml"
$file.GPO.Name
}

Related

Using Variables with Directories & Filtering

I'm new to PowerShell, and trying to do something pretty simple (I think). I'm trying to filter down the results of a folder, where I only look at files that start with e02. I tried creating a variable for my folder path, and a variable for the filtered down version. When I get-ChildItem for that filtered down version, it brings back all results. I'm trying to run a loop where I'd rename these files.
File names will be something like e021234, e021235, e021236, I get new files every month with a weird extension I convert to txt. They're always the same couple names, and each file has its own name I'd rename it to. Like e021234 might be Program Alpha.
set-location "C:\MYPATH\SAMPLE\"
$dir = "C:\MYPATH\SAMPLE\"
$dirFiltered= get-childItem $dir | where-Object { $_.baseName -like "e02*" }
get-childItem $dirFiltered |
Foreach-Object {
$name = if ($_.BaseName -eq "e024") {"Four"}
elseif ($_.BaseName -eq "e023") {"Three"}
get-childitem $dirFiltered | rename-item -newname { $name + ".txt"}
}
There are a few things I can see that could use some adjustment.
My first thought on this is to reduce the number of places a script has to be edited when changes are needed. I suggest assigning the working directory variable first.
Next, reduce the number of times information is pulled. The Get-ChildItem cmdlet offers an integrated -Filter parameter which is usually more efficient than gathering all the results and filtering afterward. Since we can grab the filtered list right off the bat, the results can be piped directly to the ForEach block without going through the variable assignment and secondary filtering.
Then, make sure to initialize $name inside the loop so it doesn't accidentally cause issues. This is because $name remains set to the last value it matched in the if/elseif statements after the script runs.
Next, make use of the fact that $name is null so that files that don't match your criteria won't be renamed to ".txt".
Finally, perform the rename operation using the $_ automatic variable representing the current object instead of pulling the information with Get-ChildItem again. The curly braces have also been replaced with parenthesis because of the change in the Rename-Item syntax.
Updated script:
$dir = "C:\MYPATH\SAMPLE\"
Set-Location $dir
Get-ChildItem $dir -Filter "e02*" |
Foreach-Object {
$name = $null #initialize name to prevent interference from previous runs
$name = if ($_.BaseName -eq "e024") {"Four"}
elseif ($_.BaseName -eq "e023") {"Three"}
if ($name -ne $null) {
Rename-Item $_ -NewName ($name + ".txt")
}
}

Powershell - Find # of nested folders in long file path of a directory

Can anyone help me w/ a code puzzle in powershell? I'm trying to look at a specific directory on several remote servers, and find the deepest nested subfolder in that directory and then count number of parent folders. Pseudo code below.
$servers = get-content (list of servers) and $path = (targetdir on remote machine)
for each $s in $servers:
find the longest path
count the # of \ (to identify # of subfolders)
Write output to file $Servername $countOfNestedFolders
Sorry I'm just good enough w/ posh to be a little dangerous.
Since you're trying to find the biggest count, it sounds like you'll want to do a comparative. Basically, start with a size of 0 - if the folder you're looking at is bigger than that, then it becomes the biggest. You do this for all the folders until you're left with the biggest folder. Note, this method won't work if there are any ties, but it doesn't sound like that's what you're looking for. I should add this is the main code for looking at a single computer. You can wrap a foreach {$server in $servers} around this for multiple servers.
$folders = Get-ChildItem -Path "C:\Directory" -Directory -Recurse
$n = 0
$biggest = ""
foreach ($folder in $folders)
{
$splitout = $folder.FullName.split("\")
if ($splitout.count -gt $n)
{
$n = $splitout.count
$biggest = $folder
}
}
Write-host "Count $n - $biggest"
here's a slight variant of the "count the path parts" solutions. [grin] it counts the delimiters. if your paths are UNC paths OR local paths, this will still give you the deepest nested dir.
however, it will not work with mixed UNC [\\SysName\ShareName] and local [c:\] paths.
also, it does not remove the starting dir from the result.
also also, i am unsure how you want to count number of parent folders. so i just posted the delimiter count.
what it does ...
sets the top dir to work from
gets the dir delimiter char
creates a regex escaped version of that char
grabs all the dirs in the target dir tree
sorts [in descending order] them by the string length of what is left over when you remove everything except the dir delimiters
grabs the 1st of those dirs
displays the .FullName of that dir
displays the number of dir delimiters in the above string
the code ...
$TargetTopDir = $env:APPDATA
$DirDelim = [System.IO.Path]::DirectorySeparatorChar
$RegexDD = [regex]::Escape($DirDelim)
$DirList = Get-ChildItem -LiteralPath $TargetTopDir -Directory -Recurse
$DeepestNestedDir = ($DirList |
Sort-Object {$_.FullName -replace "[^$RegexDD]"} -Descending)[0]
$DeepestNestedDir.FullName
'DirDelimCount = {0}' -f ($DeepestNestedDir.FullName -replace "[^$RegexDD]").Length
output ...
C:\Users\MyUserName\AppData\Roaming\Thunderbird\Profiles\shkjhmpc.default\extensions\{e2fda1a4-762b-4020-b5ad-a41df1933103}\chrome\calendar-gd\locale\gd\calendar\dialogs
DirDelimCount = 15
This got it done; thanks again for all the help!
$servers = gc C:\serverlist.txt
ForEach ($server in $servers){
$folder = "\\$server\x$\share"
$TargetTopDir = $folder
$DirDelim = [System.IO.Path]::DirectorySeparatorChar
$RegexDD = [regex]::Escape($DirDelim)
$DirList = Get-ChildItem -LiteralPath $TargetTopDir -Directory -Recurse -ErrorAction
SilentlyContinue
$DeepestNestedDir = ($DirList | Sort-Object {$_.FullName -replace "[^$RegexDD]"} -
Descending)[0]
$DepthCount = '{0}' -f ($DeepestNestedDir.FullName -replace "[^$RegexDD]").Length
$arrayItems = #{
"Depth Count" = $DepthCount - 3
"Path Name" = $DeepestNestedDir.FullName
"Server Name" = $server
}
$output= #()
$output += New-Object -TypeName PSObject -Property $arrayItems
$output | Export-CSV C:\Output.csv -NoTypeInformation -Append
}
To solve your core problem:
For a given $path, you can find the maximum directory depth in its subtree - expressed as the number of path separators (\ on Windows, / on Unix) plus one in the full path of the most deeply nested subdirectories inside $path - as follows:
# Outputs the number of path components of the most deeply nested folder in $path.
(Get-ChildItem $path -Recurse -Directory |
Measure-Object -Maximum { ($_.FullName -split '[\\/]').Count }
).Maximum
Note: If you wanted to know the relative depth - relative to $path, add -Name to the Get-ChildItem call and replace $_.FullName with $_ inside the script block ({ ... }) passed to Measure-Object. A result of 0 then means that $path has no subdirectories at all, 1 means that there are only immediate subdirectories, 2 means that the immediate subdirectories have (only) subdirectories themselves, ...
Get-ChildItem -Recurse -Directory $path outputs all subdirectories (-Directory) in the entire subtree of (-Recurse) of directory $path; add -Force to include hidden subdirs. - see Get-ChildItem.
Measure-Object -Maximum { ($_.FullName -split '[\\/]').Count } calculates the count of path separators ([\\/] is a regex that matches both a single \ and / char.) in each directory's full path ($_.FullName) - using a script block {...} as the (implied) -Property argument inside of which $_ represents the input path at hand - and determines the maximum (-Maximum); given that Measure-Object outputs a Microsoft.PowerShell.Commands.GenericMeasureInfo instance, the raw maximum value is accessed via the .Maximum property.
All incidental tasks - applying this calculation to multiple servers, writing the results to server-specific files - can be accomplished with the usual cmdlets (Get-Content, ForEach-Object, Set-Content or Out-File / >).
A faster alternative:
The above command is concise and PowerShell-idiomatic, but somewhat slow.
Here's a significantly faster alternative that uses LINQ and .NET APIs directly:
# Note: Makes sure that $path is a *full* path, because .NET's current
# directory usually differs from PowerShell's.
1 + [Linq.Enumerable]::Max(
([System.IO.Directory]::GetDirectories(
$path, '*', 'AllDirectories'
) -replace '[^\\/]').ForEach('Length')
)
Note: The above invariably includes hidden directories too. In .NET Core / .NET 5+, [System.IO.Directory]::GetDirectories() now provides an additional overload that provides more control over the enumeration.
Listing the maximum-depth directories too:
If you want not just to calculate the maximum depth, but also want to list all directories that have the maximum depth (note that there can be more than one):
# Sample input path.
# Note: Makes sure that $path is a *full* path, because .NET's current
# directory usually differs from PowerShell's.
$path = $PWD
# Extract all directories with the max. depth using Group-Object:
# Group by the calculated depth and extract the last group, which relies on
# Group-Object outputting the results sorted by grouping criteria.
$maxDepthGroup =
[System.IO.Directory]::GetDirectories($path, '*', 'AllDirectories') |
Group-Object { ($_ -split '[\\/]').Count } |
Select-Object -Last 1
# Construct the output object.
[pscustomobject] #{
MaxDepth = $maxDepthGroup.Values[0] # The grouping criterion, i.e. the depth.
MaxDepthDirs = $maxDepthGroup.Group # The paths comprising the group.
}
The output is a custom object with .MaxDepth and .MaxDepthDirs (an array of the full paths of those dirs. that have the max. depth) properties. If you pipe it to Format-List, you'll get something like:
MaxDepth : 6
MaxDepthDirs : {/Users/jdoe/Documents/Ram Dass Audio Collection/The Path of Service, /Users/jdoe/Documents/Ram Dass Audio Collection/Conscious Aging,
/Users/jdoe/Documents/Ram Dass Audio Collection/Cultivating the Heart of Compassion, /Users/jdoe/Documents/Cheatsheets/YAML Ain't
Markup Language_files}

Compress File per file, same name

I hope you are all safe in this time of COVID-19.
I'm trying to generate a script that goes to the directory and compresses each file to .zip with the same name as the file, for example:
sample.txt -> sample.zip
sample2.txt -> sample2.zip
but I'm having difficulties, I'm not that used to powershell, I'm learning and improving this script. In the end it will be a script that deletes files older than X days, compresses files and makes them upload in ftp .. the part of excluding with more than X I've already managed it for days, now I grabbed a little bit on this one.
Last try at moment.
param
(
#Future accept input
[string] $InputFolder,
[string] $OutputFolder
)
#test folder
$InputFolder= "C:\Temp\teste"
$OutputFolder="C:\Temp\teste"
$Name2 = Get-ChildItem $InputFolder -Filter '*.csv'| select Name
Set-Variable SET_SIZE -option Constant -value 1
$i = 0
$zipSet = 0
Get-ChildItem $InputFolder | ForEach-Object {
$zipSetName = ($Name2[1]) + ".zip "
Compress-Archive -Path $_.FullName -DestinationPath "$OutputFolder\$zipSetName"
$i++;
$Name2++
if ($i -eq $SET_SIZE) {
$i = 0;
$zipSet++;
}
}
You can simplify things a bit, and it looks like most of the issues are because in your script example $Name2 will contain a different set of items than the Get-ChildItem $InputFolder will return in the loop (i.e. may have other objects other than .csv files).
The best way to deal with things is to use variables with the full file object (i.e. you don't need to use |select name). So I get all the CSV file objects right away and store in the variable $CsvFiles.
We can additionally use the special variable $_ inside the ForEach-Object which represents the current object. We also can use $_.BaseName to give us the name without the extension (assuming that's what you want, otherwise use $_Name to get a zip with the name like xyz.csv).
So a simplified version of the code can be:
$InputFolder= "C:\Temp\teste"
$OutputFolder="C:\Temp\teste"
#Get files to process
$CsvFiles = Get-ChildItem $InputFolder -Filter '*.csv'
#loop through all files to zip
$CsvFiles | ForEach-Object {
$zipSetName = $_.BaseName + ".zip"
Compress-Archive -Path $_.FullName -DestinationPath "$OutputFolder\$zipSetName"
}

trying to get unique list of extension in a directory with a lot of files is going very slowly

I am trying to get the list of unique extension, and example file of each, in a dataset that is about 9TB and has several hundred thousand files. I try to use the get-child item and it works when I filter to folders that don't have a lot of files but when I filter it to one with a lot of files it seems like it will never start. below are two examples that I have been trying.
$Extensions = New-Object System.Collections.ArrayList
$filesReviewed = 0
Get-ChildItem \\server\folder -Exclude 'excludeFolder'| Get-ChildItem | Where-Object {$_.Name.Equals('files')} | Get-ChildItem -OutBuffer 1000 |
foreach{
Write-Progress -Activity "Files Reviewed: " -Status "$filesReviewed"
$filesReviewed++
if( $Extensions.contains($_.Extension) -eq $False) {
$Extensions.add($_.Extension)
Write-Host $_.Extension
Write-Host $Path = $_.FullName
}
}
I started to try to use dir thinking it might be faster but it has the same problem
set-location \\server\folder
dir | dir | Where-Object {$_.Name.Equals('files')} | dir -OutBuffer 10
Get-ChildItem retrieves a lot of info about a file that you don't need in this case and is slowing you down. You could try using [System.IO.Directory]::GetFiles to speed things up
$extensions=#{}
[System.IO.Directory]::GetFiles("\\server\folder", "*.*", [System.IO.SearchOption]::AllDirectories) | %
{
$extensions[[System.IO.DirectoryInfo]::new($_).Extension]++
}
$extensions | ft -a
you may try the following:
(Get-ChildItem -Path C:\windows -File -Recurse).Extension | Select-Object -Unique
Of course, replace the path with the one you would like to use.
More details about get-childitem could be found in: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-childitem?view=powershell-6.
Hope it helps!
There are two keys to speeding up your code:
Avoid the use of the pipeline and therefore avoid using cmdlets.
If you cannot avoid the pipeline, avoid the use of custom script blocks ({ ... }), because executing one for every input object is time-consuming.
Generally, avoid the use of Write-Progress, which slows things down noticeably.
Avoiding cmdlets requires direct use of .NET framework types instead.
Lieven Keersmaekers' helpful answer is a promising start, although combining the pipeline (%, i.e. the ForEach-Object cmdlet), slows things down, as does the construction of a [System.IO.DirectoryInfo] instance in each iteration, although to a lesser degree.
Note: For brevity and simplicity, the following solution focuses on processing a given directory's entire subtree (the equivalent of Get-ChildItem -Recurse -File).
A performance-optimized solution:
Note the following aspects:
[System.IO.Directory]::EnumerateFiles() rather than Get-ChildItem is used to enumerate the files.
A foreach loop rather than the pipeline with the ForEach-Object cmdlet (%) is used.
Inside the loop, construction of unnecessary objects is avoided, by calling the static [System.IO.Path]::GetExtension() method to extract the filename extension.
$seenExtensions=#{}
foreach ($file in [IO.Directory]::EnumerateFiles($PWD.ProviderPath, '*', 'AllDirectories')) {
if (-not $seenExtensions.ContainsKey(($ext = [IO.Path]::GetExtension($file)))) {
$seenExtensions.Add($ext, $true)
[pscustomobject] #{
Extension = $ext
Example = $file
}
}
}
The above outputs an array of custom objects each representing a unique extension (property .Extension) and the path of the first file with that extension encountered (.Example).
Sample output (note that the output won't be sorted by extension, but you can simply pipe to ... | Sort-Object Extension):
Extension Example
--------- -------
.json C:\temp\foo.json
.txt C:\temp\sub\bar.txt
...
If performance weren't a concern, PowerShell's cmdlets would allow for a much more elegant solution:
Get-ChildItem -File -Recurse |
Group-Object Extension |
Select #{ n='Extension'; e='Name' }, #{ n='Example'; e = { $_.Group[0].Name } }
Note that Group-Object implicitly sorts the output by the grouping property, so the output will be sorted alphabetically by filename extension.

Powershell script to find all backups on all drives and print file path, file name, and file size

I found scripts that all circle around the answer I need but I cannot figure out how to combine them.
Here is a script to find all the backups on all drives but it moves them; I just want to print the details (to file preferably).
foreach ($server in Get-Content c:\scripts\sl.txt){
foreach ($root in 'c$','d$','e$','f$'){
cmd /c dir "\\$server\$root\*.bak" /B /S /A-D |%{
Move-Item $_ -destination C:\users\Scripts
}
}
}
And I found others that will print all files with particular extensions found in a single drive.
$Extensions = #(".bak",".csv",".txt")
Foreach ( $Extension in $Extensions )
{
[System.IO.Directory]::EnumerateFiles("C:\","*$Extension","AllDirectories")
}
I am having trouble combining the two and under tons of pressure. Please help!
That first example uses cmd to call dir which is unnecessary since Get-ChildItem can do a directory listing. Get-ChildItem actually returns much more information and in an object format which is very usable in further scripting. There are even aliases (Get-Help alias) for Get-ChildItem: dir, ls and gci. (Save these for commandline, scripts should use the long form for readability).
The second example is using some kind of .Net roundabout method of enumerating properties of the file objects. MUCH easier to use dot notation, or Select-Object -Property directly with the powershell objects. Use 'Get-Member' to see the list of properties and methods of an object. e.g. gci | gm
PS M:\> $file = gci c:\windows\notepad.exe
PS M:\> $file.DirectoryName
C:\windows
Or
PS M:\> (gci c:\windows\notepad.exe).DirectoryName
C:\windows
If you wanted to do a oneliner, set $server beforehand or insert actual name, and change the output file name each time:
"C$","D$","E$","F$" | %{gci "\\$server\$_\*.bak" -recurse} | %{export-csv -notypeinformation -append c:\temp\filelist.csv}
Another thing to consider would be modifying the objects returned by 'Get-ChildItem' to add a property to hold the 'server' property. Since the DirectoryName property already includes the root drive letter, you could then output all servers and drives .bak file lists into one file.
Bottom Line, use this modified version of what arco444 wrote:
function List-Backups {
foreach ($server in Get-Content c:\scripts\serverlist.txt){
foreach ($root in 'c','d','e','f'){
$outfile = "C:\Temp\FileList-$server-$root.csv"
Get-ChildItem "\\$server\$root"+'$'+"\*.bak" |
Add-Member –MemberType NoteProperty –Name ServerName –Value $server |
export-csv $outfile -NoTypeInformation -Append
}
}
}
This gives you a CSV file with all the files remaining as objects. You can then do what you want with the CSV.
import-csv c:\temp\filelist.csv | select Name, DirectoryName
Later, you can create function(s) to pull information from the text files output by this function.
Try the below:
foreach ($server in Get-Content c:\scripts\sl.txt){
foreach ($root in 'c$','d$','e$','f$'){
$files = Get-ChildItem \\$server\$root\*.bak
foreach($f in $files) {
Write-Output "$($f.directoryname) $($f.name) $($f.length)" | Tee-Object -Append C:\output.log
}
}
}
For each root volume you can use the Get-ChildItem command to get a list of *.bak files. This will return a list of FileInfo objects which contain properties such as length (size), name, LastWriteTime etc...
You can access these properties by looping over the list and accessing using the . notation. Use Write-Output to print the results to the screen, you can optionally pipe to Tee-Object to print to both the screen and a file.