Script to enforce backup retention policy - powershell

I have a need for a script that can look at our backup folders (we take full backups of machines) and ensure that we never store more than the two most recent backups for each computer. I am currently working on a powershell loop that should compare each file name to every other file in the backup folder. Each machine has multiple backup files that can be identified by a matching prefix and a unique ID number separated by a delimiting character.
prefix = machine name and unique ID denotes each backup.
I am working on setting up the outer loop and running into a wall. I am storing the list of files to be checked in $array.
I have a nested for loop that I want to have check each file against every other file in the array. I have a query that should rebuild the array and exclude the current reference file each iteration.
for ($i=0; $i -lt 5; $i++) {
$array = Get-ChildItem -Path C:\Users\Aaron.Trujillo\Desktop\test
$array = Get-ChildItem -Path C:\Users\Aaron.Trujillo\Desktop\test -Exclude $array[$i]
for ($j=0; $j -lt 5; $j++){
Write-Host "i: $i j: $j"
Write-Host "Array i: $($array[$i].name) Array j: $($array[$j].name)"
if($array[$i].name -ne $array[$j].name)
{
Write-Host "truthval=false"
$truthval="false"
}
else
{
Write-Host "truthval=true"
$truthval = "true"
}
}
}
I was expecting this loop to compare the first five files in the array against each other (excluding the current reference file) and change the variable for each file, but for some reason the reference file isn't getting excluded.
Something must be wrong with the logic, but I'm struggling to spot it.

I see several flaws with your code.
As $array is presumably populated by a Get-ChildItem (which on ntfs formatted drives returns a sorted output) the nested loops aren't neccessary at all.
When iterating with one for simply compare with the predessor/follower index +/- 1.
As Names include per your description an ID, they are always different. So you've to split the name at the nebulous delimiter and check if the machine name is the same.
your if command doesn't emit anything, it just overwrites the same variable on every iteration.
With #Lee_Dailey's good idea of grouping you can split the names on the fly and check the count per group, sort files per group by LastWriteTime and only keep the newest 2.
So given a hypothetical tree of files
> tree /F
A:.
└───Backup
Alpha_8.bak
Bravo_2.bak
Bravo_5.bak
Bravo_6.bak
Bravo_7.bak
Bravo_9.bak
Charlie_1.bak
Charlie_10.bak
Charlie_3.bak
Charlie_4.bak
this small script:
$BackupDir = 'A:\Backup'
$Delim = '_'
Get-ChildItem -Path $BackupDir -File |
Group-Object {($_.BaseName -split $Delim)[0]}
yields:
Count Name Group
----- ---- -----
1 Alpha {Alpha_8.bak}
5 Bravo {Bravo_2.bak, Bravo_5.bak, Bravo_6.bak, Bravo_7.bak...}
4 Charlie {Charlie_1.bak, Charlie_10.bak, Charlie_3.bak, Charlie_4.bak}
You can then iterate the groups and delete files per group by whatever means,
for example delete all but the 2 newest (defined per LastWriteTime) files per machine:
Get-ChildItem -Path $BackupDir -File |
Group-Object {($_.BaseName -split $Delim)[0]} | ForEach-Object {
$_.Group | Sort LastWriteTime -Desc | Select -skip 2 | Remove-Item -WhatIf
}
If the output looks OK, remove the trailing -WhatIf

I think it's the brackets around the $array[$j.name] i.e. it should be:
if($array[$i].name -ne $array[$j].name)
Edit:
For troubleshooting try manually outputting your information:
for ($i=0; $i -lt 5; $i++) {
for ($j=$i+1; $j -lt 5; $j++){
Write-Host "i: $i j: $j"
Write-Host "Array i: $($array[$i].name) Array j: $($array[$j].name)"
if($array[$i].name -ne $array[$j].name)
{
Write-Host "truthval=false"
$truthval="false"
}
else
{
Write-Host "truthval=true"
$truthval = "true"
}
}
}

Related

Powershell - Find # of nested folders in long file path of a directory

Can anyone help me w/ a code puzzle in powershell? I'm trying to look at a specific directory on several remote servers, and find the deepest nested subfolder in that directory and then count number of parent folders. Pseudo code below.
$servers = get-content (list of servers) and $path = (targetdir on remote machine)
for each $s in $servers:
find the longest path
count the # of \ (to identify # of subfolders)
Write output to file $Servername $countOfNestedFolders
Sorry I'm just good enough w/ posh to be a little dangerous.
Since you're trying to find the biggest count, it sounds like you'll want to do a comparative. Basically, start with a size of 0 - if the folder you're looking at is bigger than that, then it becomes the biggest. You do this for all the folders until you're left with the biggest folder. Note, this method won't work if there are any ties, but it doesn't sound like that's what you're looking for. I should add this is the main code for looking at a single computer. You can wrap a foreach {$server in $servers} around this for multiple servers.
$folders = Get-ChildItem -Path "C:\Directory" -Directory -Recurse
$n = 0
$biggest = ""
foreach ($folder in $folders)
{
$splitout = $folder.FullName.split("\")
if ($splitout.count -gt $n)
{
$n = $splitout.count
$biggest = $folder
}
}
Write-host "Count $n - $biggest"
here's a slight variant of the "count the path parts" solutions. [grin] it counts the delimiters. if your paths are UNC paths OR local paths, this will still give you the deepest nested dir.
however, it will not work with mixed UNC [\\SysName\ShareName] and local [c:\] paths.
also, it does not remove the starting dir from the result.
also also, i am unsure how you want to count number of parent folders. so i just posted the delimiter count.
what it does ...
sets the top dir to work from
gets the dir delimiter char
creates a regex escaped version of that char
grabs all the dirs in the target dir tree
sorts [in descending order] them by the string length of what is left over when you remove everything except the dir delimiters
grabs the 1st of those dirs
displays the .FullName of that dir
displays the number of dir delimiters in the above string
the code ...
$TargetTopDir = $env:APPDATA
$DirDelim = [System.IO.Path]::DirectorySeparatorChar
$RegexDD = [regex]::Escape($DirDelim)
$DirList = Get-ChildItem -LiteralPath $TargetTopDir -Directory -Recurse
$DeepestNestedDir = ($DirList |
Sort-Object {$_.FullName -replace "[^$RegexDD]"} -Descending)[0]
$DeepestNestedDir.FullName
'DirDelimCount = {0}' -f ($DeepestNestedDir.FullName -replace "[^$RegexDD]").Length
output ...
C:\Users\MyUserName\AppData\Roaming\Thunderbird\Profiles\shkjhmpc.default\extensions\{e2fda1a4-762b-4020-b5ad-a41df1933103}\chrome\calendar-gd\locale\gd\calendar\dialogs
DirDelimCount = 15
This got it done; thanks again for all the help!
$servers = gc C:\serverlist.txt
ForEach ($server in $servers){
$folder = "\\$server\x$\share"
$TargetTopDir = $folder
$DirDelim = [System.IO.Path]::DirectorySeparatorChar
$RegexDD = [regex]::Escape($DirDelim)
$DirList = Get-ChildItem -LiteralPath $TargetTopDir -Directory -Recurse -ErrorAction
SilentlyContinue
$DeepestNestedDir = ($DirList | Sort-Object {$_.FullName -replace "[^$RegexDD]"} -
Descending)[0]
$DepthCount = '{0}' -f ($DeepestNestedDir.FullName -replace "[^$RegexDD]").Length
$arrayItems = #{
"Depth Count" = $DepthCount - 3
"Path Name" = $DeepestNestedDir.FullName
"Server Name" = $server
}
$output= #()
$output += New-Object -TypeName PSObject -Property $arrayItems
$output | Export-CSV C:\Output.csv -NoTypeInformation -Append
}
To solve your core problem:
For a given $path, you can find the maximum directory depth in its subtree - expressed as the number of path separators (\ on Windows, / on Unix) plus one in the full path of the most deeply nested subdirectories inside $path - as follows:
# Outputs the number of path components of the most deeply nested folder in $path.
(Get-ChildItem $path -Recurse -Directory |
Measure-Object -Maximum { ($_.FullName -split '[\\/]').Count }
).Maximum
Note: If you wanted to know the relative depth - relative to $path, add -Name to the Get-ChildItem call and replace $_.FullName with $_ inside the script block ({ ... }) passed to Measure-Object. A result of 0 then means that $path has no subdirectories at all, 1 means that there are only immediate subdirectories, 2 means that the immediate subdirectories have (only) subdirectories themselves, ...
Get-ChildItem -Recurse -Directory $path outputs all subdirectories (-Directory) in the entire subtree of (-Recurse) of directory $path; add -Force to include hidden subdirs. - see Get-ChildItem.
Measure-Object -Maximum { ($_.FullName -split '[\\/]').Count } calculates the count of path separators ([\\/] is a regex that matches both a single \ and / char.) in each directory's full path ($_.FullName) - using a script block {...} as the (implied) -Property argument inside of which $_ represents the input path at hand - and determines the maximum (-Maximum); given that Measure-Object outputs a Microsoft.PowerShell.Commands.GenericMeasureInfo instance, the raw maximum value is accessed via the .Maximum property.
All incidental tasks - applying this calculation to multiple servers, writing the results to server-specific files - can be accomplished with the usual cmdlets (Get-Content, ForEach-Object, Set-Content or Out-File / >).
A faster alternative:
The above command is concise and PowerShell-idiomatic, but somewhat slow.
Here's a significantly faster alternative that uses LINQ and .NET APIs directly:
# Note: Makes sure that $path is a *full* path, because .NET's current
# directory usually differs from PowerShell's.
1 + [Linq.Enumerable]::Max(
([System.IO.Directory]::GetDirectories(
$path, '*', 'AllDirectories'
) -replace '[^\\/]').ForEach('Length')
)
Note: The above invariably includes hidden directories too. In .NET Core / .NET 5+, [System.IO.Directory]::GetDirectories() now provides an additional overload that provides more control over the enumeration.
Listing the maximum-depth directories too:
If you want not just to calculate the maximum depth, but also want to list all directories that have the maximum depth (note that there can be more than one):
# Sample input path.
# Note: Makes sure that $path is a *full* path, because .NET's current
# directory usually differs from PowerShell's.
$path = $PWD
# Extract all directories with the max. depth using Group-Object:
# Group by the calculated depth and extract the last group, which relies on
# Group-Object outputting the results sorted by grouping criteria.
$maxDepthGroup =
[System.IO.Directory]::GetDirectories($path, '*', 'AllDirectories') |
Group-Object { ($_ -split '[\\/]').Count } |
Select-Object -Last 1
# Construct the output object.
[pscustomobject] #{
MaxDepth = $maxDepthGroup.Values[0] # The grouping criterion, i.e. the depth.
MaxDepthDirs = $maxDepthGroup.Group # The paths comprising the group.
}
The output is a custom object with .MaxDepth and .MaxDepthDirs (an array of the full paths of those dirs. that have the max. depth) properties. If you pipe it to Format-List, you'll get something like:
MaxDepth : 6
MaxDepthDirs : {/Users/jdoe/Documents/Ram Dass Audio Collection/The Path of Service, /Users/jdoe/Documents/Ram Dass Audio Collection/Conscious Aging,
/Users/jdoe/Documents/Ram Dass Audio Collection/Cultivating the Heart of Compassion, /Users/jdoe/Documents/Cheatsheets/YAML Ain't
Markup Language_files}

Renaming pdf files in sequential order using powershell

I have a large number of pdf's that need renamed in sequential order. These were originally scanned into a single document, then extracted as separate files. When extracted, the name becomes "444026-444050 1", "444026-444050 2", etc. I am trying to rename all the files to match the document number ("444026-444050 1" would become "444026").
I found the following line of code that I can use in Powershell, but it seems that anything over 9 files there is a problem! Once I try it with 10 files, only the first file is saved correctly. The rest become jumbled (file 444027 has the contents of file 444035, then file 444028 has 444027, and 444029 has 444028, etc.)
I imagine there is some sort of problem with a loop, but am having difficulty fixing it.
Can someone help?
thanks
Dir *.pdf | ForEach-Object -begin { $count=26 } -process { rename-item $_ -NewName "4440$count.pdf"; $count++ }
Alright. Let's see if this makes everybody happy. Maybe you should try this with a backup copy of the files.
# make some test files in a new folder
# 1..20 | foreach {
# if (! (test-path "44026-44050 $_.pdf")) {
# echo "44026-44050 $_" > "44026-44050 $_.pdf" }
# }
# rename and pad the first 9 old filenames with a 0 before the last digit for sorting
# is it less than 100 files?
1..9 | foreach {
ren "44026-44050 $_.pdf" "44026-44050 0$_.pdf" -whatif
}
# make dir complete first with parentheses for powershell 5
# pad $count to 2 digits
# take off the -whatif if it looks ok
(dir *.pdf) | foreach { $count = 1 } {
$padded = $count | foreach tostring 00
rename-item $_ -newname 4440$padded.pdf -whatif; $count++ }
The order in which Dir (which is an alias for Get-ChildItem) retrieves the items does not appear to be strictly guaranteed. Furthermore, if it is sorting it's probably sorting them as strings and "444026-444050 10" comes before "444026-444050 2" as strings. It might be worth inserting SortObject into your pipeline and using Split to get at the sequence number you care about:
Dir *.pdf | Sort-Object -Property {[int]$_.Name.Split()[1].Split(".")[0]} | ForEach-Object -begin { $count=26 } -process { rename-item $_ -NewName "4440$count.pdf"; $count++ }
The key part is this new pipeline stage inserted after Dir and before ForEach-Object:
Sort-Object -Property {[int]$_.Name.Split()[1].Split(".")[0]}
This says to sort the output of Dir according to the whatever comes between the first space and the subsequent period, comparing those things as integers (not strings). This ensures that your results will be ordered and that you'll get them in numeric order, not lexicographic order.

Search and replace files and folders names with txt file support

I have many folders and inside these different files. Each folder and their children files have the same name and different extension, so in the ABC folder there are the ABC.png, ABC.prj, ABC.pgw files, in the DEF folder there are the DEF.png, DEF.prj, DEF.pgw files and so on.
With a script I have created a txt file with the list of png file names. Then I put in row 2 a new name for the name in row1, in row 4 a new name for the name in row 3, and so on.
Now I'm searching a powershell script that:
- scan all folder for the name in row 1 and replace it with name in row2
- scan all folder for the name in row 3 and replace it with name in row4 and so on
I have try with this below, but it doesn't work.
Have you some suggestions? Thank you
$0=0
$1=1
do {
$find=Get-Content C:\1\Srv\MapsName.txt | Select -Index $0
$repl=Get-Content C:\1\Srv\MapsName.txt | Select -Index $1
Get-ChildItem C:\1\newmaps -Recurse | Rename-Item -NewName { $_.name -replace $find, $repl} -verbose
$0=$0+2
$1=$1+2
}
until ($0 -eq "")
I believe there are several things wrong with your code and also the code Manuel gave you.
Although you have a list of old filenames and new filenames, you are not using that in the Get-ChildItem cmdlet, but instead try and replace all files it finds.
Using -replace uses a Regular Expression replace, that means the special character . inside the filename is regarded as Any Character, not simply a dot.
You are trying to find *.png files, but you do not add a -Filter with the Get-ChildItem cmdlet, so now it will return all filetypes.
Anyway, I have a different approach for you:
If your input file C:\1\Srv\MapsName.txt looks anything like this:
picture1.png
ABC_1.png
picture2.png
DEF_1.png
picture3.png
DEF_2.png
The following code will use that to build a lookup Hashtable so it can act on the files mentioned in the input file and leave all others unchanged.
$mapsFile = 'C:\1\Srv\2_MapsName.txt'
$searchPath = 'C:\1\NewMaps'
# Read the input file as an array of strings.
# Every even index contains the file name to search for.
# Every odd index number has the new name for that file.
$lines = Get-Content $mapsFile
# Create a hashtable to store the filename to find
# as Key, and the replacement name as Value
$lookup = #{}
for ($index = 0; $index -lt $lines.Count -1; $index += 2) {
$lookup[$lines[$index]] = $lines[$index + 1]
}
# Next, get a collection of FileInfo objects of *.png files
# If you need to get multiple extensions, remove the -Filter and add -Include '*.png','*.jpg' etc.
$files = Get-ChildItem -Path $searchPath -Filter '*.png' -File -Recurse
foreach ($file in $files) {
# If the file name can be found as Key in the $lookup Hashtable
$find = $file.Name
if ($lookup.ContainsKey($find)) {
# Rename the file with the replacement name in the Value of the lookup table
Write-Host "Renaming '$($file.FullName)' --> $($lookup[$find])"
$file | Rename-Item -NewName $lookup[$find]
}
}
Edit
If the input text file 'C:\1\Srv\MapsName.txt' does NOT contain filenames including their extension, change the final foreach loop into this:
foreach ($file in $files) {
# If the file name can be found as Key in the $lookup Hashtable
# Look for the file name without extension as it is not given in the 'MapsName.txt' file.
$find = [System.IO.Path]::GetFileNameWithoutExtension($file.Name)
if ($lookup.ContainsKey($find)) {
# Rename the file with the replacement name in the Value of the lookup table
# Make sure to add the file's extension if any.
$newName = $lookup[$find] + $file.Extension
Write-Host "Renaming '$($file.FullName)' --> '$newName'"
$file | Rename-Item -NewName $newName
}
}
Hope that helps
The problem in your snippet is that it never ends.
I tried it and it works but keeps looping forever.
I created a folder with the files a.txt, b.txt and c.txt.
And in the map.txt I have this content:
a.txt
a2.md
b.txt
b2.md
c.txt
c2.md
Running the following script I managed to rename every file to be as expected.
$0=0
$1=1
$find=Get-Content D:\map.txt | Select -Index $0
while($find) {
$find=Get-Content D:\map.txt | Select -Index $0
$repl=Get-Content D:\map.txt | Select -Index $1
if(!$find -Or !$repl) {
break;
}
Get-ChildItem D:\Files -Recurse | Rename-Item -NewName { $_.name -replace $find, $repl} -verbose
$0=$0+2
$1=$1+2
}

How to rename files with sequential even and odd numbers in PowerShell?

I would like to know how I can rename the files from a specific folder with a sequence of only even and odds numbers in PowerShell. E.g. Folder1: pag_001.jpg, pag_003.jpg, pag_005.jpg.... pag_201.jpg , Folder2: pag_002.jpg, pag_004.jpg, pag_006.jpg.... pag_200.jpg. It is because I have a document that was scanned first the odds pages and secondly the even pages, therefore their file names are in a consecutive sequence from 1 to 201. Then I separated one half of the files which are the odds pages in a new place: Folder1, and the second half,the even pages in the Folder2. That is why I would like change the names first and the join again together with their new names.
I have tried this based in a similar post:
At the moment I could generate even number sequences like that:
ForEach ($number in 1..100 ) { $number * 2}
and odd numbers like that:
ForEach ($number in 0..100 ) { $number *2+1}
and wanted apply the sequences generated before to rename my files like that:
cd C:\test\Folder1
$i = $number * 2
Get-ChildItem *.jpg | %{Rename-Item $_ -NewName ('pag_{0:D3}.jpg' -f $i++)}
but it doesn't works! Any suggestions are welcome
Regards,
Your $i++ adds 1 each time, this is why it also add even numbers,
You can create array of Odd Numbers then use the $i++ to step one item in the array, like this:
$path = "C:\test\Folder1"
$oddNumbersArray = 0..100 | % {$_ *2 +1}
$i = 0
Get-ChildItem $path -Filter *.jpg | % {Rename-Item $_ -NewName ("pag_$($oddNumbersArray[$i]).jpg") ;$i++}
For Even Numbers change the $oddNumbersArray line to {$_ *2}
Bunch of ways to do this. For mine we add each index as a member so that it is more easily accessible in the rename item script block.
$index = 0
Get-ChildItem $path -Filter "*.jpg" | ForEach-Object{
$index = $index +2
$_ | Add-Member -Name "Index" -MemberType NoteProperty -Value $index -PassThru
} | Rename-Item -NewName {'pag_{0:D3}.jpg' -f $_.Index} -WhatIf
Using Add-Member in a ForEach-Object we update the value of index and then add it as a property of the same name. Then in your rename-item scriptblock we can call that property. Remove the -WhatIf after you verified the new names are what you wanted. Switch $index between 0 and -1 for even and odd respectively.
Another method using a global index variable and mitigating the pipeline by using calculated properties to create the pipeline variables that Rename-Item uses.
$path = "C:\Temp\csv"
$global:index = 0 # Use -1 for odd
Get-ChildItem $path -Filter "*.csv" |
Select-Object #{Name="Path";Expression={$_.FullName}},
#{Name="NewName";Expression={$global:index = $global:index + 2; 'pag_{0:D3}.jpg' -f $global:index}} |
Rename-Item -WhatIf

Powershell - Assigning unique file names to duplicated files using list inside a .csv or .txt

I have limited experience with Powershell doing very basic tasks by itself (such as simple renaming or moving files), but I've never created one that has the need to actually extract information from inside a file and apply that data directly to a file name.
I'd like to create a script that can reference a simple .csv or text file containing a list of unique identifiers and have it assign those to a batch of duplicated files (they all have the same contents) that share a slightly different name in the form of a 3-digit number appended as the prefix of a generic name.
For example, let's say my list of files are something like this:
001_test.txt
002_test.txt
003_test.txt
004_test.txt
005_test.txt
etc.
Then my .csv contains an alphabetical list of what I would like those to become:
Alpha.txt
Beta.txt
Charlie.txt
Delta.txt
Echo.txt
etc.
I tried looking at similar examples, but I'm failing miserably trying to tailor them to get it to do the above.
EDIT: I didn't save what I already modified, but here is the baseline script I was messing with:
$file_server = Read-Host "Enter the file server IP address"
$rootFolder = 'C:\TEMP\GPO\source\5'
Get-ChildItem -LiteralPath $rootFolder -Directory |
Where-Object { $_.Name -as [System.Guid] } |
ForEach-Object {
$directory = $_.FullName
(Get-Content "$directory\gpreport.xml") |
ForEach-Object { $_ -replace "99.999.999.999", $file_server } |
Set-Content "$directory\gpreport.xml"
# ... etc
}
I think this is to replace a string inside a file though. I need to replace the file name itself using a list from another file (that is not getting renamed), while not changing the contents of the files that are being renamed.
So you want to rename similar files with those listed in a text file. Ok, here's what you are going to need for my solution (alias listed in parenthesis): Get-Content (GC), Get-ChildItem (GCI), Where (?), Rename-Item, ForEach (%)
$NewNames = GC c:\temp\Namelist.txt #Path, including file name, to list of new names
$Name = "dog.txt" #File name without the 001_ prefix
$Path = "C:\Temp" #Path to search
$i=0
GCI $path | ?{$_.Name -match "\d{3}_$Name"}|%{Rename-Item $_.FullName $NewNames[$i];$i++}
Tested as working. That gets your list of new names and saves it as an array. Then it defines your file name, path, and sets $i to 0 as a counter. Then for each file that matches your pattern it renames it based off of item number $i in the array of new names, and then increments $i up one number and moves to the next file.
I haven't tested this, but it should be pretty close. It assumes you have a CSV with a column named FileNames and that you have at least as many names in that list as there are on disk.
$newNames = Import-Csv newfilenames.csv | Select -ExpandProperty FileNames
$existingFiles = Get-ChildItem c:\someplace
for ($i = 0; $i -lt $existingFiles.count; $i++)
{
Rename-Item -Path $existingFiles[$i].FullName -NewName $newNames[$i]
}
Basically, you create two arrays and using a basic for loop steping through the list of files on disk and pull the name from the corresponding index in the newNames array.
Does your CSV file map the identifiers to the file names?
Identifier,NewName
001,Alpha
002,Beta
If so, you'll need to look up the identifier before renaming the file:
# Define the naming convention
$Suffix = '_test'
$Extension = 'txt'
# Get the files and what to rename them to
$Files = Get-ChildItem "*$Suffix.$Extension"
$Csv = Import-Csv 'Names.csv'
# Rename the files
foreach ($File in $Files) {
$NewName = ($Csv | Where-Object { $File.Name -match '^' + $_.Identifier } | Select-Object -ExpandProperty NewName)
Rename-Item $File "$NewName.$Extension"
}
If your CSV file is just a sequential list of filenames, logicaldiagram's answer is probably more along the lines of what you're looking for.