Powershell Scan for missing files in multiple folders - powershell

I'm checking for missing XYZ map tiles using Powershell, but coming unstuck in the nested loops. Essentially the map tiles exist from a "base" folder, within this base folder are multiple directories. Within each directory are the map tiles.
e.g.
C:\My Map\17\ # this is the Base folder, zoom level 17
C:\My Map\17\1234\ # this is a folder containing map tiles
C:\My Map\17\1234\30200.png # this is a map tile
C:\My Map\17\1234\30201.png # this is a map tile
C:\My Map\17\1234\30203.png # this is a map tile, but we're missing 30202.png (have tiles either side)
C:\My Map\17\1234\30204.png # this is a map tile
C:\My Map\17\1235\ # this is another folder containing map tiles [...]
So my idea is for each folder, scan for gaps where we have tiles each side and try to download them.
This is what I have so far:
$BasePath = "C:\_test\17\"
$ColumnDirectories = Get-ChildItem $BasePath -Directory
$ColumnDirectories | ForEach-Object {
$ColumnDirectory = $ColumnDirectories.FullName
$MapTiles = Get-ChildItem -Path $ColumnDirectory -Filter *.png -file
$MapTiles | ForEach-Object {
#Write-Host $MapTiles.FullName
$TileName = $MapTiles.Name -replace '.png',''
$TileNamePlus1 = [int]$TileName + 1
$TileNamePlus2 = [int]$TileName + 2
Write-Host $TileName
}
}
But I'm getting Cannot convert the "System.Object[]" value of type "System.Object[]" to type "System.Int32".
Eventually I want to go Test-Path on each of $TileName, TileNamePlus1, $TileNamePlus2, and where the middle one doesn't exist to download it again.
e.g.
C:\My Map\17\1234\30201.png -- Exists
C:\My Map\17\1234\30202.png -- Not exists, download from https://somemapsrv.com/17/1234/30202.png
C:\My Map\17\1234\30203.png -- Exists
Any help appreciated! I'm fairly new to Powershell.

The whole problem here is an understanding of how ForEach-Object loops work. Within the loop the automatic variable $_ represents the current iteration of the loop. So as suggested by the comments by dugas and Santiago Squarzon you need to change this line:
$TileName = $MapTiles.Name -replace '.png',''
to this:
$TileName = $_.Name -replace '\.png',''
Or more simply this (the BaseName property is the file name without the extension):
$TileName = $_.BaseName

Since all your png files have basenames as integer numbers, you could do something like this:
$BasePath = 'C:\_test\17'
$missing = Get-ChildItem -Path $BasePath -Directory | ForEach-Object {
$ColumnDirectory = $_.FullName
# get an array of the files in the folder, take the BaseName only
$MapTiles = (Get-ChildItem -Path $ColumnDirectory -Filter '*.png' -File).BaseName
# create an array of integer numbers taken from the files BaseName
$sequence = $MapTiles | ForEach-Object { [int]$_ } | Sort-Object
$sequence[0]..$sequence[-1] | Where-Object { $MapTiles -notcontains $_ } | ForEach-Object {
Join-Path -Path $ColumnDirectory -ChildPath ('{0}.png' -f $_)
}
}
# missing in this example has only one file, but could also be an array of missing sequential numbered files
$missing # --> C:\_test\17\1234\30202.png
If your file names have leading zero's, this won't work..

Related

Retrieving file information of specific date using powershell

I have a base folder as:
D:\St\Retail\AMS\AMS\FTP-FromClient\AMS
It contains various folders of dates:
2022-04-01
2022-04-02
...
...
2022-02-02
2021-05-05
2019-04-12
And each of these folders contains own files inside the folder. So, I want to retrieve all the filename inside the folder if it has 2022-04. So if the folder has '2022-04' as the base name ,I need to retreive all the file inside the folder like '2022-04-01','2022-04-02','2022-04-03'. The way I tried is:
cls
$folerPath = 'D:\St\Retail\AMS\AMS\FTP-FromClient\AMS'
$files = Get-ChildItem $folerPath
[System.Collections.ArrayList]$data = #()
foreach ($f in $files) {
$a = Get-ChildItem $f.FullName
foreach ($inner in $a) {
echo $inner.FullName
$outfile = $inner.FullName -match '*2022-04*'
$datepart = $inner.FullName.split('\')[-1]
if ($outfile) {
$data.add($datepart + '\' + $inner.Name.Trim())
}
}
}
My final $data may contains like this:
2022-04-01/abc.txt
2022-04-02/cde.pdf
2022-04-03/e.xls
You can do this by first collecting the directories you want to explore and then loop over these to get the files inside.
Using a calculated property you can output in whatever format you like:
$folderPath = 'D:\St\Retail\AMS\AMS\FTP-FromClient\AMS'
$data = Get-ChildItem -Path $folderPath -Filter '2022-04*' -Directory | ForEach-Object {
$dir = $_.Name
(Get-ChildItem -Path $_.FullName -File |
Select-Object #{Name = 'FolderFile'; Expression = {'{0}\{1}' -f $dir, $_.Name}}).FolderFile
}
After this, $data would be a string array with this content:
2022-04-01\abc.txt
2022-04-02\cde.pdf
2022-04-03\e.xls
By using wildcards for both directory and file name, you only need a single Get-ChildItem call:
$folderPath = 'D:\St\Retail\AMS\AMS\FTP-FromClient\AMS'
$folderDate = '2022-04'
[array] $data = Get-ChildItem "$folderPath/$folderDate*/*" -File | ForEach-Object{
# Join-Path's implicit output will be captured as an array in $data.
Join-Path $_.Directory.Name $_.Name
}
$data will be an array of file paths like this:
2022-04-01\abc.txt
2022-04-02\cde.pdf
2022-04-03\e.xls
Notes:
[array] $data makes sure that the variable always contains an array. Otherwise PowerShell would output a single string value when only a single file is found. This could cause problems, e. g. when you want to iterate over $data by index, you would iterate over the characters of the single string instead.
To make this answer platform-independent I'm using forward slashes in the Get-ChildItem call which work as path separators under both Windows and *nix platforms.
Join-Path is used to make sure the output paths use the expected default path separator (either / or \) of the platform.

How to Create a 'Missing' File(s) List

I am trying to compile a list of missing files between two folder/directories which have text/image files with the same basename.
# file directory/folder with .txt files
$filesPathText = "C:\test\test3"
# file directory/folder with .JPG files
$filesPathImage = "C:\test\test4"
Content:
Copy of 0002.txt
Copy of 0003.txt
Copy of 0004.txt
Copy of 0006.txt
Copy of 0002.jpg
Copy of 0003.jpg
Copy of 0004.jpg
Copy of 0005.jpg
Copy of 0006.jpg
I want to output that the 'missing' file is: Copy of 0005.txt
I have tried this kind of thing:
$texts = Get-ChildItem -Path $filesPathText
$images = Get-ChildItem -Path $filesPathImage
$result = $images | Where-Object{$texts -notcontains $images}
$result
To me the logic reads correctly but the result is an output of all the image files.
Even though this is a simple example and would seem common usage I have not been able to find a similar question that has been answered.
Any suggestions would be welcome.
Your Where-Object script block is comparing 1 array of objects ($texts) against the other array of object ($images) instead of comparing each object ($_) against an array of objects. You're also not referencing the Property (.BaseName) you want to compare.
$texts = Get-ChildItem -Path $filesPathText
$images = Get-ChildItem -Path $filesPathImage
# missing text files
$images | Where-Object { $texts.BaseName -notcontains $_.BaseName } | ForEach-Object {
$_.BaseName + '.txt'
}
# missing images
$texts | Where-Object { $images.BaseName -notcontains $_.BaseName } | ForEach-Object {
$_.BaseName + '.jpg'
}

Powershell - Find # of nested folders in long file path of a directory

Can anyone help me w/ a code puzzle in powershell? I'm trying to look at a specific directory on several remote servers, and find the deepest nested subfolder in that directory and then count number of parent folders. Pseudo code below.
$servers = get-content (list of servers) and $path = (targetdir on remote machine)
for each $s in $servers:
find the longest path
count the # of \ (to identify # of subfolders)
Write output to file $Servername $countOfNestedFolders
Sorry I'm just good enough w/ posh to be a little dangerous.
Since you're trying to find the biggest count, it sounds like you'll want to do a comparative. Basically, start with a size of 0 - if the folder you're looking at is bigger than that, then it becomes the biggest. You do this for all the folders until you're left with the biggest folder. Note, this method won't work if there are any ties, but it doesn't sound like that's what you're looking for. I should add this is the main code for looking at a single computer. You can wrap a foreach {$server in $servers} around this for multiple servers.
$folders = Get-ChildItem -Path "C:\Directory" -Directory -Recurse
$n = 0
$biggest = ""
foreach ($folder in $folders)
{
$splitout = $folder.FullName.split("\")
if ($splitout.count -gt $n)
{
$n = $splitout.count
$biggest = $folder
}
}
Write-host "Count $n - $biggest"
here's a slight variant of the "count the path parts" solutions. [grin] it counts the delimiters. if your paths are UNC paths OR local paths, this will still give you the deepest nested dir.
however, it will not work with mixed UNC [\\SysName\ShareName] and local [c:\] paths.
also, it does not remove the starting dir from the result.
also also, i am unsure how you want to count number of parent folders. so i just posted the delimiter count.
what it does ...
sets the top dir to work from
gets the dir delimiter char
creates a regex escaped version of that char
grabs all the dirs in the target dir tree
sorts [in descending order] them by the string length of what is left over when you remove everything except the dir delimiters
grabs the 1st of those dirs
displays the .FullName of that dir
displays the number of dir delimiters in the above string
the code ...
$TargetTopDir = $env:APPDATA
$DirDelim = [System.IO.Path]::DirectorySeparatorChar
$RegexDD = [regex]::Escape($DirDelim)
$DirList = Get-ChildItem -LiteralPath $TargetTopDir -Directory -Recurse
$DeepestNestedDir = ($DirList |
Sort-Object {$_.FullName -replace "[^$RegexDD]"} -Descending)[0]
$DeepestNestedDir.FullName
'DirDelimCount = {0}' -f ($DeepestNestedDir.FullName -replace "[^$RegexDD]").Length
output ...
C:\Users\MyUserName\AppData\Roaming\Thunderbird\Profiles\shkjhmpc.default\extensions\{e2fda1a4-762b-4020-b5ad-a41df1933103}\chrome\calendar-gd\locale\gd\calendar\dialogs
DirDelimCount = 15
This got it done; thanks again for all the help!
$servers = gc C:\serverlist.txt
ForEach ($server in $servers){
$folder = "\\$server\x$\share"
$TargetTopDir = $folder
$DirDelim = [System.IO.Path]::DirectorySeparatorChar
$RegexDD = [regex]::Escape($DirDelim)
$DirList = Get-ChildItem -LiteralPath $TargetTopDir -Directory -Recurse -ErrorAction
SilentlyContinue
$DeepestNestedDir = ($DirList | Sort-Object {$_.FullName -replace "[^$RegexDD]"} -
Descending)[0]
$DepthCount = '{0}' -f ($DeepestNestedDir.FullName -replace "[^$RegexDD]").Length
$arrayItems = #{
"Depth Count" = $DepthCount - 3
"Path Name" = $DeepestNestedDir.FullName
"Server Name" = $server
}
$output= #()
$output += New-Object -TypeName PSObject -Property $arrayItems
$output | Export-CSV C:\Output.csv -NoTypeInformation -Append
}
To solve your core problem:
For a given $path, you can find the maximum directory depth in its subtree - expressed as the number of path separators (\ on Windows, / on Unix) plus one in the full path of the most deeply nested subdirectories inside $path - as follows:
# Outputs the number of path components of the most deeply nested folder in $path.
(Get-ChildItem $path -Recurse -Directory |
Measure-Object -Maximum { ($_.FullName -split '[\\/]').Count }
).Maximum
Note: If you wanted to know the relative depth - relative to $path, add -Name to the Get-ChildItem call and replace $_.FullName with $_ inside the script block ({ ... }) passed to Measure-Object. A result of 0 then means that $path has no subdirectories at all, 1 means that there are only immediate subdirectories, 2 means that the immediate subdirectories have (only) subdirectories themselves, ...
Get-ChildItem -Recurse -Directory $path outputs all subdirectories (-Directory) in the entire subtree of (-Recurse) of directory $path; add -Force to include hidden subdirs. - see Get-ChildItem.
Measure-Object -Maximum { ($_.FullName -split '[\\/]').Count } calculates the count of path separators ([\\/] is a regex that matches both a single \ and / char.) in each directory's full path ($_.FullName) - using a script block {...} as the (implied) -Property argument inside of which $_ represents the input path at hand - and determines the maximum (-Maximum); given that Measure-Object outputs a Microsoft.PowerShell.Commands.GenericMeasureInfo instance, the raw maximum value is accessed via the .Maximum property.
All incidental tasks - applying this calculation to multiple servers, writing the results to server-specific files - can be accomplished with the usual cmdlets (Get-Content, ForEach-Object, Set-Content or Out-File / >).
A faster alternative:
The above command is concise and PowerShell-idiomatic, but somewhat slow.
Here's a significantly faster alternative that uses LINQ and .NET APIs directly:
# Note: Makes sure that $path is a *full* path, because .NET's current
# directory usually differs from PowerShell's.
1 + [Linq.Enumerable]::Max(
([System.IO.Directory]::GetDirectories(
$path, '*', 'AllDirectories'
) -replace '[^\\/]').ForEach('Length')
)
Note: The above invariably includes hidden directories too. In .NET Core / .NET 5+, [System.IO.Directory]::GetDirectories() now provides an additional overload that provides more control over the enumeration.
Listing the maximum-depth directories too:
If you want not just to calculate the maximum depth, but also want to list all directories that have the maximum depth (note that there can be more than one):
# Sample input path.
# Note: Makes sure that $path is a *full* path, because .NET's current
# directory usually differs from PowerShell's.
$path = $PWD
# Extract all directories with the max. depth using Group-Object:
# Group by the calculated depth and extract the last group, which relies on
# Group-Object outputting the results sorted by grouping criteria.
$maxDepthGroup =
[System.IO.Directory]::GetDirectories($path, '*', 'AllDirectories') |
Group-Object { ($_ -split '[\\/]').Count } |
Select-Object -Last 1
# Construct the output object.
[pscustomobject] #{
MaxDepth = $maxDepthGroup.Values[0] # The grouping criterion, i.e. the depth.
MaxDepthDirs = $maxDepthGroup.Group # The paths comprising the group.
}
The output is a custom object with .MaxDepth and .MaxDepthDirs (an array of the full paths of those dirs. that have the max. depth) properties. If you pipe it to Format-List, you'll get something like:
MaxDepth : 6
MaxDepthDirs : {/Users/jdoe/Documents/Ram Dass Audio Collection/The Path of Service, /Users/jdoe/Documents/Ram Dass Audio Collection/Conscious Aging,
/Users/jdoe/Documents/Ram Dass Audio Collection/Cultivating the Heart of Compassion, /Users/jdoe/Documents/Cheatsheets/YAML Ain't
Markup Language_files}

Search and replace files and folders names with txt file support

I have many folders and inside these different files. Each folder and their children files have the same name and different extension, so in the ABC folder there are the ABC.png, ABC.prj, ABC.pgw files, in the DEF folder there are the DEF.png, DEF.prj, DEF.pgw files and so on.
With a script I have created a txt file with the list of png file names. Then I put in row 2 a new name for the name in row1, in row 4 a new name for the name in row 3, and so on.
Now I'm searching a powershell script that:
- scan all folder for the name in row 1 and replace it with name in row2
- scan all folder for the name in row 3 and replace it with name in row4 and so on
I have try with this below, but it doesn't work.
Have you some suggestions? Thank you
$0=0
$1=1
do {
$find=Get-Content C:\1\Srv\MapsName.txt | Select -Index $0
$repl=Get-Content C:\1\Srv\MapsName.txt | Select -Index $1
Get-ChildItem C:\1\newmaps -Recurse | Rename-Item -NewName { $_.name -replace $find, $repl} -verbose
$0=$0+2
$1=$1+2
}
until ($0 -eq "")
I believe there are several things wrong with your code and also the code Manuel gave you.
Although you have a list of old filenames and new filenames, you are not using that in the Get-ChildItem cmdlet, but instead try and replace all files it finds.
Using -replace uses a Regular Expression replace, that means the special character . inside the filename is regarded as Any Character, not simply a dot.
You are trying to find *.png files, but you do not add a -Filter with the Get-ChildItem cmdlet, so now it will return all filetypes.
Anyway, I have a different approach for you:
If your input file C:\1\Srv\MapsName.txt looks anything like this:
picture1.png
ABC_1.png
picture2.png
DEF_1.png
picture3.png
DEF_2.png
The following code will use that to build a lookup Hashtable so it can act on the files mentioned in the input file and leave all others unchanged.
$mapsFile = 'C:\1\Srv\2_MapsName.txt'
$searchPath = 'C:\1\NewMaps'
# Read the input file as an array of strings.
# Every even index contains the file name to search for.
# Every odd index number has the new name for that file.
$lines = Get-Content $mapsFile
# Create a hashtable to store the filename to find
# as Key, and the replacement name as Value
$lookup = #{}
for ($index = 0; $index -lt $lines.Count -1; $index += 2) {
$lookup[$lines[$index]] = $lines[$index + 1]
}
# Next, get a collection of FileInfo objects of *.png files
# If you need to get multiple extensions, remove the -Filter and add -Include '*.png','*.jpg' etc.
$files = Get-ChildItem -Path $searchPath -Filter '*.png' -File -Recurse
foreach ($file in $files) {
# If the file name can be found as Key in the $lookup Hashtable
$find = $file.Name
if ($lookup.ContainsKey($find)) {
# Rename the file with the replacement name in the Value of the lookup table
Write-Host "Renaming '$($file.FullName)' --> $($lookup[$find])"
$file | Rename-Item -NewName $lookup[$find]
}
}
Edit
If the input text file 'C:\1\Srv\MapsName.txt' does NOT contain filenames including their extension, change the final foreach loop into this:
foreach ($file in $files) {
# If the file name can be found as Key in the $lookup Hashtable
# Look for the file name without extension as it is not given in the 'MapsName.txt' file.
$find = [System.IO.Path]::GetFileNameWithoutExtension($file.Name)
if ($lookup.ContainsKey($find)) {
# Rename the file with the replacement name in the Value of the lookup table
# Make sure to add the file's extension if any.
$newName = $lookup[$find] + $file.Extension
Write-Host "Renaming '$($file.FullName)' --> '$newName'"
$file | Rename-Item -NewName $newName
}
}
Hope that helps
The problem in your snippet is that it never ends.
I tried it and it works but keeps looping forever.
I created a folder with the files a.txt, b.txt and c.txt.
And in the map.txt I have this content:
a.txt
a2.md
b.txt
b2.md
c.txt
c2.md
Running the following script I managed to rename every file to be as expected.
$0=0
$1=1
$find=Get-Content D:\map.txt | Select -Index $0
while($find) {
$find=Get-Content D:\map.txt | Select -Index $0
$repl=Get-Content D:\map.txt | Select -Index $1
if(!$find -Or !$repl) {
break;
}
Get-ChildItem D:\Files -Recurse | Rename-Item -NewName { $_.name -replace $find, $repl} -verbose
$0=$0+2
$1=$1+2
}

Renaming a new folder file to the next incremental number with powershell script

I would really appreciate your help with this
I should first mention that I have been unable to find any specific solutions and I am very new to programming with powershell, hence my request
I wish to write (and later schedule) a script in powershell that looks for a file with a specific name - RFUNNEL and then renames this to R0000001. There will only be one of such 'RFUNELL' files in the folder at any time. However when next the script is run and finds a new RFUNNEL file I will this to be renamed to R0000002 and so on and so forth
I have struggled with this for some weeks now and the seemingly similar solutions that I have come across have not been of much help - perhaps because of my admittedly limited experience with powershell.
Others might be able to do this with less syntax, but try this:
$rootpath = "C:\derp"
if (Test-Path "$rootpath\RFUNNEL.txt")
{ $maxfile = Get-ChildItem $rootpath | ?{$_.BaseName -like "R[0-9][0-9][0-9][0-9][0-9][0-9][0-9]"} | Sort BaseName -Descending | Select -First 1 -Expand BaseName;
if (!$maxfile) { $maxfile = "R0000000" }
[int32]$filenumberint = $maxfile.substring(1); $filenumberint++
[string]$filenumberstring = ($filenumberint).ToString("0000000");
[string]$newName = ("R" + $filenumberstring + ".txt");
Rename-Item "$rootpath\RFUNNEL.txt" $newName;
}
Here's an alternative using regex:
[cmdletbinding()]
param()
$triggerFile = "RFUNNEL.txt"
$searchPattern = "R*.txt"
$nextAvailable = 0
# If the trigger file exists
if (Test-Path -Path $triggerFile)
{
# Get a list of files matching search pattern
$files = Get-ChildItem "$searchPattern" -exclude "$triggerFile"
if ($files)
{
# store the filenames in a simple array
$files = $files | select -expandProperty Name
$files | Write-Verbose
# Get next available file by carrying out a
# regex replace to extract the numeric part of the file and get the maximum number
$nextAvailable = ($files -replace '([a-z])(.*).txt', '$2' | measure-object -max).Maximum
}
# Add one to either the max or zero
$nextAvailable++
# Format the resulting string with leading zeros
$nextAvailableFileName = 'R{0:000000#}.txt' -f $nextAvailable
Write-Verbose "Next Available File: $nextAvailableFileName"
# rename the file
Rename-Item -Path $triggerFile -NewName $nextAvailableFileName
}