Using a powershell loop to copy files to pattern-aware directories - powershell

I have a directory A in which containing the sub directories a, b, c, d, each sub-directory contains timestamped folders, example 20230120, and files whose names end with the date and time of day example file-202301200545
What I want is to copy the files whose names end with today's date to the folder with today's date.
Sorry, it's a bit complicated to explain but I hope you understand it.
I wrote a script that performs this action but on the condition that I fix each directory.
$pattern = Get-Date -Format yyyyMMdd
$path = "D:\A\b\"
$Dest = "D:\A\b\\$pattern"
Get-ChildItem -Path $path -Recurse -Filter *$pattern* | Where-Object {$_.PSIsContainer -eq $false} | Move-Item -Destination $Dest
$path = "D:\A\c\"
$Dest = "D:\A\c\\$pattern"
Get-ChildItem -Path $path -Recurse -Filter *$pattern* | Where-Object {$_.PSIsContainer -eq $false} | Move-Item -Destination $Dest
$path = "D:\A\d\"
$Dest = "D:\A\d\\$pattern"
Get-ChildItem -Path $path -Recurse -Filter *$pattern* | Where-Object {$_.PSIsContainer -eq $false} | Move-Item -Destination $Dest
I need help to use a loop or any other techinique that will allow me to do this more easily

Assuming that the target dir. already exists:
Get-ChildItem -Directory -Path D:\A\[a-d] |
Get-ChildItem -File -Recurse -Filter *$pattern* |
Move-Item -Destination {
Join-Path ($_.DirectoryName -split '\\')[0..2] $pattern
} -WhatIf
Note: The -WhatIf common parameter in the command above previews the operation. Remove -WhatIf and re-execute once you're sure the operation will do what you want.
If you want to create the target dir. on demand, use
New-Item -Type Directory -Force (Join-Path ($_.DirectoryName -split '\\')[0..2] $pattern) - New-Item's -Force switch, when combined with -Type Directory, ensures that a preexisting directory, if any, is returned; otherwise, the directory is created; in either case, a System.IO.DirectoryInfo instance representing the directory is returned, which stringifies to its .FullName property, i.e. to its full, file-system-native path.
Note:
The nested Get-ChildItem calls are meant to promote conceptual clarity:
The outer call uses a PowerShell wildcard expression in combination with the -Directory switch to match the top-level directories of interest - note the use of character range [a-d] to match directories names a, b, c, or d.
The inner call acts on each resulting directory and recursively searches only for files matching the pattern, via the -File switch.
This approach avoids the need for your Where-Object {$_.PSIsContainer -eq $false} call, which - incidentally - can be more PowerShell-idiomatically expressed as Where-Object { -not $_.PSIsContainer } and - in PowerShell (Core) 7+ - using simplified syntax, Where-Object -not PSIsContainer.
Note: It is tempting to try a single Get-ChildItem call such as Get-ChildItem -Path D:\A\[a-d] -File -Recurse -Filter *$pattern*, but that doesn't actually work, because the -File switch is then applied to whatever the D:\A\[a-d] wildcard expression resolves to, and since the results are directories, -File effectively excludes them from further processing, resulting in an effective no-op.
That is, -File in combination with -Recurse only works meaningfully for literal input paths, to whose child (descendant) items the -File switch is then applied.
The Move-Item call uses a delay-bind script block to dynamically determine the destination directory:
$_.DirectoryName contains each input file's full directory path, and (... -split '\\')[0..2] extracts the ancestral path that is composed of 3 components, such as D:\A\c.

Related

Get ChildItem from only files of specific folders

I try to export files of specific named Folders:
Get-ChildItem -Path 'C:\Test' -Name -Recurse -File > C:\Test\Test.txt
I get a list like:
content.csv
Test.txt
Folder 1\INTERESTED_FOLDER\a - Kopie.txt
Folder 1\INTERESTED_FOLDER\a.txt
Folder 1\Neuer Ordner\ttt.txt
Folder 1\Neuer Ordner - Kopie\ttt.txt
Folder 2\INTERESTED_FOLDER\b - Kopie.txt
Folder 2\INTERESTED_FOLDER\b.txt
Folder 2\Neuer Ordner\ttt.txt
Folder 2\Neuer Ordner - Kopie\ttt.txt
But what i want is:
Folder 1\INTERESTED_FOLDER\a - Kopie.txt
Folder 1\INTERESTED_FOLDER\a.txt
Folder 2\INTERESTED_FOLDER\b - Kopie.txt
Folder 2\INTERESTED_FOLDER\b.txt
I tried with -Filter "INTERESTED" etc. but then i only get
C:\Test\Folder 1\INTERESTED_FOLDER
C:\Test\Folder 2\INTERESTED_FOLDER
What i do wrong?
If I read your question correctly, you want the FullNames of the files (so their names including the path).
If that is the case, remove the -Name switch and filter on the DirectoryName property like:
(Get-ChildItem -Path 'C:\Test' -Recurse -File |
Where-Object { $_.DirectoryName -match 'INTERESTED_FOLDER' }).FullName | # also matches 'MY_INTERESTED_FOLDER_123'
Set-Content -Path 'C:\Test\Test.txt'
Alternatives for the Where-Object clause:
# also matches 'MY_INTERESTED_FOLDER_123'
Where-Object { $_.DirectoryName -like '*INTERESTED_FOLDER*' }
or if you are looking for a precise match on the complete folder name
# does NOT match 'MY_INTERESTED_FOLDER_123'
Where-Object { $_.DirectoryName -split '[/\\]' -contains 'INTERESTED_FOLDER' }
You can perform a wildcard search using the -Filter parameter:
Get-ChildItem -Path 'C:\Test' -Name -Recurse -File -Filter *INTERESTED_FOLDER* > C:\Test\Test.txt
If for example, you were interested in finding the files in INTERESTED_FOLDER but also only the files that are .txt you could do:
-Filter *INTERESTED_FOLDER*.txt
Using the -Name parameter affects your capabilities because you are returning strings instead of FileInfoObjects. You may then use Where-Object to capture everything.
Get-ChildItem -Path 'C:\Test' -Name -Recurse -File |
Where {$_ -match '\\INTERESTED_FOLDER\\'} |
Set-Content c:\test\test.txt
Note that the matching above assumes INTERESTED_FOLDER is not a direct descendent of your path. If that is a possibility, then your match expression needs to be updated to \\?INTERESTED_FOLDER\\.

How to loop through all subfolders in a directory, and remove files with a specific filename

I am trying to write a script in Powershell to remove some files automatically with a certain file name.
My idea is to get all the folders in the directory, then loop through the subdirectory, and remove all items with the file name, but it doesn't seem to be working as expected.
Here is my script
$folders = Get-ChildItem -path "C:\Website-Backup" -Recurse | Where-Object {$_.PsIsContainer} |Group-Object {$_.FullName.Split('_')[0] }
$subfolders = Get-ChildItem -path $folders -Recurse | Where-Object {$_.PsIsContainer} | Group-Object {$_.FullName.Split('_')[0] }
ForEach($subfolder in $subfolders)
{
Remove-Item * -Include *100x*
}
Any idea why the script doesn't seem to be doing anything?
I think you can simplify your code if I understand correctly to:
Get-ChildItem "C:\Website-Backup" -Recurse -include "*100x*" -file | remove-item
The Group-Object command is likely what's confusing things here - Remove-Item is expecting a path - you're not referencing the subfolder variable in your loop as well, so this is the same as just running the Remove-Item command as many times as there are items in the array.
You can try this instead;
Get-ChildItem -Path "C:\Website-Backup" -Recurse | Where-Object -FilterScript { $_.name -like 'MyFile.txt' } | Remove-Item
This will pipe the returned child items into Where-Object, filter it to the specified file name, then pass that to Remove-Item as a file path.
You can also skip the Where-Object, but you lose a bit of control this way;
Get-ChildItem -Path 'C:\WebSiteBackup\*MyFile.txt' -Recurse | Remove-Item

exclude path from get-childitem search

I'm trying to have all the files in the folder structure copied to a folder that is also part of that structure. So the destination folder is excluded form the search. I also want to exclude any folder that has in its path ".thumbnails" but when I replace the full path in the $Skip with a wild card path such as 'D:\ZZZ_Phone_test*.thumbnails' it won't work.
Secondly, I'd like to make this more efficient if possible so the job can be finished quicker. When the script is running it is mostly the CPU working not so much the harddrive.
Thirdly, is there any way how to generate some output of what was copied, skipped, errors... and save it to a logfile?
$Source = 'D:\ZZZ_Phone_test'
$Dest = 'D:\ZZZ_Phone_test\1\1\BackUp'
$Skip = 'D:\ZZZ_Phone_test\4\.thumbnails'
Get-ChildItem $Source -Directory -Recurse | ? FullName -ne $Dest | ? FullName -ne $Skip | get-ChildItem -File | Copy-Item -Exclude `
*.0,*.1,*.nomedia,*.thumbnail,*.chck,*.crypt12,*.tmp,*.db,*.crypt1,*.ini,*.pdrproj,*.pkpass,*.dat,*.enc,*.lck,*.xml,*.json,*.LOCK,*.443,*.preference `
-Destination $Dest
.
EDIT: the following works but it will only exclude files in directories whose names end with "thumbnails" or "BackUp". If there are any directories with files inside of "thumbnails" folder they will all be processed. I'd like to define the folders to be excluded the way that even if there are subdirectories with files in a directory defined in $Skip they would not be processed.
$Source = 'D:\ZZZ_Phone_test'
$Dest = 'D:\ZZZ_Phone_test\1\1\BackUp'
$Skip = '*thumbnails', '*BackUp'
(Get-ChildItem -Path $Source -Directory -Recurse -Exclude $Skip).FullName |
get-ChildItem -File |
Copy-Item -WhatIf -Exclude `
*.0,*.1,*.nomedia,*.thumbnail,*.chck,*.crypt12,*.tmp,*.db,*.crypt1,*.ini,*.pdrproj,*.pkpass,*.dat,*.enc,*.lck,*.xml,*.json,*.LOCK,*.443,*.preference `
-Destination $Dest
Try this and modify as you wish for that file exclusion section...
$Source = 'D:\Temp'
$Dest = 'D:\Destination'
$Skip = '*est', '*here'
<#
Always build you code one use case at a time to ensure you are getting what
you'd expect before moving ot the next.
#>
# Get all directories off a give path
(Get-ChildItem -Path $Source -Directory -Recurse).FullName |
Select-Object -First 5
# Results
<#
D:\Temp\AddressFiles
D:\Temp\ChildFolder
D:\Temp\est
D:\Temp\here
D:\Temp\hold
#>
# Exclude named directories
(Get-ChildItem -Path $Source -Directory -Recurse -Exclude $Skip).FullName |
Select-Object -First 5
# Results
<#
D:\Temp\AddressFiles
D:\Temp\ChildFolder
D:\Temp\ChildFolder\New folder
D:\Temp\ChildFolder\temp
D:\Temp\hold
#>
# Or include only what you want
(Get-ChildItem -Path $Source -Directory -Recurse -Include $Skip).FullName |
Select-Object -First 5
# Results
<#
D:\Temp\ChildFolder\New folder\est
D:\Temp\est
D:\Temp\here
#>
# Loop directories and process files, trap errors
(Get-ChildItem -Path $Source -Directory -Recurse -Exclude $Skip).FullName |
Select-Object -First 5 |
ForEach {
Try
{
"Processing $PSItem"
$CopyItemSplat = #{
Path = (Get-ChildItem -Path $PSItem -ErrorAction Stop).FullName
Destination = $Dest
Verbose = $true
WhatIf = $true
}
}
Catch
{
Write-Warning -Message 'An error was encountered.'
$PSitem.Exception.Message
}
}
# Results
<#
Processing D:\Temp\AddressFiles
Processing D:\Temp\ChildFolder
Processing D:\Temp\ChildFolder\New folder
Processing D:\Temp\ChildFolder\temp
WARNING: An error was encountered.
The property 'FullName' cannot be found on this object. Verify that the property exists.
Processing D:\Temp\hold
WARNING: An error was encountered.
The property 'FullName' cannot be found on this object. Verify that the property exists.
#>
For complex filtering needs, divide et impera is often useful an approach. That is, simplify the problem in multiple steps instead of trying to write an one-liner.
Let's take a directory listing of all the files and exclude the destination directory $dest. As % (shorthand for Where-Object) parameter -notmatch expects a regular expression, the $dest path is escaped with [regex]::Escape. This needs to be done, as backslash \ is a reserved character in regular expressions. One could write the path in escaped form in the first hand, like c:\\my\\path\\to\\somewhere, but Escape does all the work needed.
Get-ChildItem $source\* -Recurse -File | ? { $_.psparentpath -notmatch [regex]::escape($dest) }
Now that we have all the files except destination, start pruning the list. Since there are lots of file extensions, let's put those on an array. Loop through the array, and remove each match from the $files array.
$excluded = #("*.0", "*.1", "*.nomedia", "*.thumbnail", "*.chck", "*.crypt12", "*.tmp", "*.db",
"*.crypt1", "*.ini", "*.pdrproj", "*.pkpass", "*.dat", "*.enc", "*.lck", "*.xml", "*.json",
"*.LOCK", "*.443", "*.preference")
foreach($ex in $excluded) {
$files = $files | ? {$_.extension -notlike $ex}
}
To remove the $skip, filter the collection again:
$files = $files | ? {$_.DirectoryName -ne $skip)
At this point, all you have is an array that contains files that are to be copied into $dest. Before copying, use -WhatIf switch to see what Copy-Item would do to be sure the copy works as intended:
$files | % { Copy-Item -WhatIf $_ $dest }
To wrap up a complete example,
$source = "c:\temp\phonetest"
$dest = "c:\temp\phonetest\1\1\BackUp"
$Skip = "c:\temp\phonetest\skipme"
# Get list of all the files
Get-ChildItem $source\* -Recurse | ? { $_.psparentpath -notmatch [regex]::escape($dest) }
# Filter by extension
$excluded = #("*.0", "*.1", "*.nomedia", "*.thumbnail", "*.chck", "*.crypt12", "*.tmp", "*.db", "*.crypt1", "*.ini", "*.pdrproj", "*.pkpass", "*.dat", "*.enc", "*.lck", "*.xml", "*.json", "*.LOCK", "*.443", "*.preference")
foreach($ex in $excluded) {
$files = $files | ? {$_.extension -notlike $ex}
}
# Skip specific dir
$files = $files | ? {$_.DirectoryName -ne $skip)
# See what would be copied
$files | % { Copy-Item -WhatIf $_ $dest }

Filenames matching regex but in folder with wildcard

This is a follow up question of: PowerShell concatenate output of Get-ChildItem
This code works fine:
Get-ChildItem -Path "D:\Wim\TM1\TI processes" -Filter "*.vue" -Recurse -File |
Where-Object { $_.BaseName -match '^[0-9]+$' } |
ForEach-Object { ($_.FullName -split '\\')[-2,-1] -join '\' } |
Out-File D:\wim.txt
But I would need to restrict the search folder to only certain folders, basically, this filter: D:\Wim\TM1\TI processes\\*}vues (so all subfolders ending in }vues).
If I add that wildcard condition I get no result. Without the restriction, I get the correct result. Is this possible please?
The idea is to get rid of the 3rd line in the first output (which was a copy/paste by me) and also to minimize the number of folders to look at.
You can nest two Get-ChildItem calls:
An outer Get-ChildItem -Directory -Recurse call to filter directories of interest first,
an inner Get-ChildItem -File call that, for each directory found, examines and processes the files of interest.
Get-ChildItem -Path "D:\Wim\TM1\TI processes" -Filter "*}vues" -Recurse -Directory |
ForEach-Object {
Get-ChildItem -LiteralPath $_.FullName -Filter "*.vue" -File |
Where-Object { $_.BaseName -match '^[0-9]+$' } |
ForEach-Object { ($_.FullName -split '\\')[-2,-1] -join '\' }
} | Out-File D:\wim.txt
Note: The assumption is that all *.vue files of interest are located directly in each *}vues folder.
As for what you tried:
Given that you're limiting items being enumerated to files (-File), your directory-name wildcard pattern *}vues never gets to match any directory names and, in the absence of files matching that pattern, returns nothing.
Generally, with -Recurse it is conceptually cleaner not to append the wildcard pattern directly to the -Path argument, so as to better signal that the pattern will be matched in every directory in the subtree.
In your case you would have noticed your attempt to filter doubly, given that you're also using the -Filter parameter.

Recursively Delete Files and Directories Using a Filter on the Directory Name

I am attempting to delete all directories, sub-directories, and the files contained in them based on a filter that specifies the required directory/sub-directory name.
For example, if I have c:\Test\A\B.doc, c:\Test\B\A\C.doc, and c:\Test\B\A.doc and my filter specifies all directories named 'A', I would expect the remaining folders and files to be c:\Test, c:\Test\B and c:\Test\B\A.doc respectively.
I am trying to do this in PowerShell and am not familiar with it.
The following 2 examples will delete all of the files that match my specified filter, but the files that match the filter as well.
$source = "C:\Powershell_Test" #location of directory to search
$strings = #("A")
cd ($source);
Get-ChildItem -Include ($strings) -Recurse -Force | Remove-Item -Force –Recurse
and
Remove-Item -Path C:\Powershell_Test -Filter A
I would use something like this:
$source = 'C:\root\folder'
$names = #('A')
Get-ChildItem $source -Recurse -Force |
Where-Object { $_.PSIsContainer -and $names -contains $_.Name } |
Sort-Object FullName -Descending |
Remove-Item -Recurse -Force
The Where-Object clause restricts the output from Get-ChildItem to just folders whose names are present in the array $names. Sorting the remaining items by their full name in descending order ensures that child folders get deleted before their parent. That way you avoid errors from attempting to delete a folder that had already been deleted by a prior recursive delete operation.
If you have PowerShell v3 or newer you can do all filtering directly with Get-ChildItem:
Get-ChildItem $source -Directory -Include $names -Recurse -Force |
Sort-Object FullName -Descending |
Remove-Item -Recurse -Force
I don't think you can do it quite that simply. This gets the list of directories, and breaks the path into its constituent parts, and verifies whether the filter matches one of those parts. If so, it removes the whole path.
It adds a little caution to handle if it already deleted a directory because of nesting (the test-path) and the -Confirm helps ensure that if there's a bug here you have a chance to verify the behavior.
$source = "C:\Powershell_Test" #location of directory to search
$filter = "A"
Get-Childitem -Directory -Recurse $source |
Where-Object { $_.FullName.Split([IO.Path]::DirectorySeparatorChar).Contains($filter) } |
ForEach-Object { $_.FullName; if (Test-Path $_) { Remove-Item $_ -Recurse -Force -Confirm } }