Thanks to #Theo yesterday I got my script to work and do exactly all the things I wanted. So now have a lovely menu system, options etc.
But I've decided - like a charm I wanted to do a little more!
So part of what is happening is finding files and renaming them. Theo's example was best:
D:\The Old Man and his dog.mp4 → D:\Old Man and his dog (The).mp4
D:\A Nightmare on Elm Street.mp4 → D:\Nightmare on Elm Street (A).mp4
D:\I, Robot.mkv → D:\Robot (I).mkv
What I wanted to do is change from output to screen the results. To a log file of changes made, and preferably update the file and add or make a new log file every time the script runs. Just in case I ever need to find something or check what happened - I've got something to refer back to!
Below is the code I've used but not to happy with to display the results.
Start-Sleep -s 5
cls
Write-Output "
Ok then, $filter and $rename has now been searched and changed.
With a total of
(Get-ChildItem | ? {$_.CreationTime -ge (Get-Date).Addminutes(-10)}).Count
Write-Output "files being changed.
"
pause
cls}
I would like to get a better display result, i.e.
xx Files have been successfully update
xx Files failed.
Please see log for details.
And then look in the log file and see something like:
Log1.txt:
D:\The Old Man and his dog.mp4 successfully Changed to D:\Old Man and his dog (The).mp4
Total of 1 /1 files successfully changed
I'm thinking of Compare-Object and Out-File, so maybe
Compare-Object $(Get-Content c:\user\documents\List1.txt) $(Get-Content c:\user\documents\List2.txt) > c:\user\documents\diff_output.txt
But for some reason I cannot get my head around this idea and figure out where to start.
If I understand the requirements correctly, extending on my previous code, you could do this:
$rootFolder = 'D:\' #'# the root folder where the files and subfolders are
$logFile = 'D:\renamed_{0:yyyy-MM-dd HH_mm_ss}.txt' -f (Get-Date) #'# the file to log to, including the current date
# create an array of words that start the file name and should be moved towards the back
$words = 'The', 'i', 'a'
# create a regex pattern where all possible words are separated by the regex OR (|) sign
$re = '^({0})[ ,]+' -f ($words -join '|')
# create two integer counter variables
[int]$success, [int]$failed = 0
Get-ChildItem -Path $rootFolder | Where-Object { $_.Name -match $re } | ForEach-Object {
$newName = '{0} ({1}){2}' -f ($_.BaseName -replace $re), $matches[1], $_.Extension
try {
$_ | Rename-Item -NewName $newName -ErrorAction Stop -WhatIf
Add-Content -Path $logFile -Value "Successfully renamed '$($_.FullName)' to '$newName'"
$success++
}
catch {
Add-Content -Path $logFile -Value "Rename failed for file '$($_.FullName)'"
$failed++
}
}
# write summary
$total = $success + $failed
$summary = "Total of $success / $total files successfully changed`r`nTotal of $failed / $total files failed"
# output summary to the log file
Add-Content -Path $logFile -Value "`r`n==========================================================="
Add-Content -Path $logFile -Value $summary
# output summary on screen
Write-Host "`r`n$summary`r`nPlease see '$logFile' for details." -ForegroundColor Yellow
Using the same three examples will output this on screen:
Total of 3 / 3 files successfully changed
Total of 0 / 3 files failed
Please see 'D:\renamed_2019-09-18 12_07_01.txt' for details.
and the log file will contain:
Successfully renamed 'D:\A Nightmare on Elm Street.mp4' to 'Nightmare on Elm Street (A).mp4'
Successfully renamed 'D:\I, Robot.mkv' to 'Robot (I).mkv'
Successfully renamed 'D:\The Old Man and his dog.mp4' to 'Old Man and his dog (The).mp4'
===========================================================
Total of 3 / 3 files successfully changed
Total of 0 / 3 files failed
Related
I'm in the process of writing up a PowerShell script that can take a bunch of .TIF images, rename them, and place them in a new folder structure depending on the original file name.
For example, a folder containing the file named:
ABC-ALL-210316-0001-3001-0001-1-CheckInvoice-Front.TIF
would be renamed to "00011CIF.TIF", and placed in the following folder:
\20220316\03163001\
I've been trying to put together a code to perform this task, and I got one to work where I had two different "ForEach" methods. One would do a bunch of file renaming to remove "-" and shorten "CheckInvoiceFront" to "CIF" and such. Then the second method would again pull all .TIF images, create substrings of the image names, and create folders from those substrings, and then move the image to the new folder, shortening the file name. Like I said, it worked... but I wanted to combine the ForEach methods into one process. However, each time I try to run it, it fails for various reasons... I've tried to change things around, but I just can't seem to get it to work.
Here's the current (non-working) code:
# Prompt user for directory to search through
$sorDirectory = Read-Host -Prompt 'Input source directory to search for images: '
$desDirectory = Read-Host -Prompt 'Input target directory to output folders: '
Set-Location $sorDirectory
# Check directory for TIF images, and close if none are found
Write-Host "Scanning "$sorDirectory" for images... "
$imageCheck = Get-ChildItem -File -Recurse -Path $sorDirectory -include '*.tif'
$imageCount = $imageCheck.count
if ($imageCount -gt 0) {
Write-Host "Total number of images found: $imageCount"
""
Read-Host -Prompt "Press ENTER to continue or CTRL+C to quit"
$count1=1;
# Rename all images, removing "ABCALL" from the start and inserting "20", and then shorten long filetype names, and move files to new folders with new names
Clear-Host
Write-Host "Reformatting images for processing..."
""
Get-ChildItem -File -Recurse -Path $sorDirectory -include '*.tif' |
ForEach-Object {
Write-Progress -Activity "Total Formatted Images: $count1/$imageCount" -Status "0--------10%--------20%--------30%--------40%--------50%--------60%--------70%--------80%--------90%-------100" -CurrentOperation $_ -PercentComplete (($count1 / $imageCount) * 100)
Rename-Item $_ -NewName $_.Name.Replace("-", "").Replace("ABCALL", "20").Replace("CheckInvoiceFront", "CIF").Replace("CheckInvoiceBack", "CIB").Replace("CheckFront", "CF").Replace("CheckBack", "CB") |Out-Null
$year = $_.Name.SubString(0, 4)
$monthday = $_.Name.Substring(4,4)
$batch = $_.Name.SubString(12, 4)
$fulldate = $year+$monthday
$datebatch = $monthday+$batch
$image = $_.Name.SubString(16)
$fullPath = "$desDirectory\$fulldate\$datebatch"
if (-not (Test-Path $fullPath)) { mkdir $fullPath |Out-Null }
Move-Item $_ -Destination "$fullPath\$image" |Out-Null
$count1++
}
# Finished
Clear-Host
Write-Host "Job complete!"
Timeout /T -1
}
# Closes if no images are found (likely bad path)
else {
Write-Host "There were no images in the selected folder. Now closing..."
Timeout /T 10
Exit
}
Usually this results in an error stating that it's can't find the path of the original file name, as if it's still looking for the original non-renamed image. I tried adding some other things, but then it said I was passing null values. I'm just not sure what I'm doing wrong.
Note that if I take the everything after the "Rename-Item" (starting with "$year =") and have that in a different ForEach method, it works. I guess I just don't know how to make the Rename-Item return its results back to "$_" before everything else tries working on it. I tried messing around with "-PassThru" but I don't think I was doing it right.
Any suggestions?
As Olaf points out, situationally you may not need both a Rename-Item and a Move-Item call, because Move-Item can rename and move in single operation.
That said, Move-Item does not support implicit creation of the target directory to move a file to, so in your case you do need separate calls.
You can use Rename-Item's -PassThru switch to make it output a System.IO.FileInfo instance (or, if a directory is being renamed, a System.IO.DirectoryInfo instance) representing the already renamed file; you can directly pass such an instance to Move-Item via the pipeline:
Get-ChildItem -File -Recurse -Path $sorDirectory -include '*.tif' |
ForEach-Object {
# ...
# Use -PassThru with Rename-Item to output a file-info object describing
# the already renamed file.
$renamedFile = $_ | Rename-Item -PassThru -NewName $_.Name.Replace("-", "").Replace("ABCALL", "20").Replace("CheckInvoiceFront", "CIF").Replace("CheckInvoiceBack", "CIB").Replace("CheckFront", "CF").Replace("CheckBack", "CB")
# ...
# Pass $renamedFile to Move-Item via the pipeline.
$renamedFile | Move-Item -Destination "$fullPath\$image"
# ...
}
As for your desire to:
make the Rename-Item return its results back to "$_"
While PowerShell doesn't prevent you from modifying the automatic $_ variable, it is better to treat automatic variables as read-only.
Therefore, a custom variable is used above to store the output from Rename-Item -PassThru
You need -passthru and -destination:
rename-item file1 file2 -PassThru | move-item -Destination dir1
I have a powershell script that's moving files from a source directory over to a target directory every 15 minutes. Files of around 1 meg are moving into the source directory by an SFTP server... so the files can be written at anytime by the SFTP clients.
The Move-Item command is moving files, however it seems that it's moving them without making sure the file isn't still being written (in-use?).
I need some help coming up with a way to write the files from the source to the target and make sure the entire file gets to the target. Anyone run across this issue before with Powershell?
I searched and was able to find a few functions that said they solved the problem but when I tried them out I wasn't seeing the same results.
Existing PowerShell script is below:
Move-Item "E:\SFTP_Server\UserFolder\*.*" "H:\TargetFolder\" -Verbose -Force *>&1 | Out-File -FilePath E:\Powershell_Scripts\LOGS\MoveFilesToTarget-$(get-date -f yyyy-MM-dd-HH-mm-ss).txt
I ended up cobbling together a few things and got this working as I wanted it. Basically I'm looping through the files and checking the length of the file once... then waiting a second and checking the length of the file again to see if it's changed. This seems to be working well. Here's a copy of the script incase it helps anyone in the future!
$logfile ="H:\WriteTest\LogFile_$(get-date -format `"yyyyMMdd_hhmmsstt`").txt"
function log($string, $color)
{
if ($Color -eq $null) {$color = "white"}
write-host $string -foregroundcolor $color
$string | out-file -Filepath $logfile -append
}
$SourcePath = "E:\SFTP_Server\UserFolder\"
$TargetPath = "H:\TargetFolder\"
$Stuff = Get-ChildItem "$SourcePath\*.*" | select name, fullname
ForEach($I in $Stuff){
log "Starting to process $I.name" green
$newfile = $TargetPath + $I.name
$LastLength = 1
$NewLength = (Get-Item $I.fullname).length
while ($NewLength -ne $LastLength) {
$LastLength = $NewLength
Start-Sleep -Seconds 1
log "Waiting 1 Second" green
$NewLength = (Get-Item $I.fullname).length
log "Current File Length = $NewLength" green
}
log "File Not In Use - Ready To Move!" green
Move-Item $I.fullname $TargetPath
}
I wrote the following script to batch process the files in to folders based on the title of the magazine (everything before the first hyphen):
magazine title - year-month.pdf eg National Geographic - 2017-07.pdf
After running the script the magazine(s) are moved from the parent folder to a new sub folder, in this case "National Geographic Magazine".
Three related questions:
The '_Orphans' folder (line 38) is created even if there are no 'orphans'
to file in to it for later manual processing. How do I make the folder
creation conditional?
Duplicate files create an error message during processing. Not a big deal as the script continues to run, but I'd like to handle duplicates the same way 'orphans' are handled, with a new "_Duplicates" folder/move.
How do I comment multiple lines without the # at the beginning of each
line (as at the top of the script, for example). There must be a more elegant
way to handle comments/documentation...?
Bonus Question:
If you're really bored waiting for that multi-TB file copy
you're watching progress like an hour glass, could anyone help with the code
for an array of delimiters (wrong term/name probably) as shown on line 10? I'd
like to be able to specify more than just the hard-coded hyphen I used in my
regex match (line 26, which took me the better part of a day to get working).
$OrigFolder = ".\"
$NewFolder = ".\_Sorted to Move"
# Orphans folder, where files that return null in the regex match will be moved
# Example: file "- title.pdf"
# will be moved to ".\_Orphans" folder
$Orphans = '_Orphans' # Use the underscore to sort the folder to the top of the window
#### How to use an array of values for the delimiters in the regex instead of literals
#### My proposed code, but I am missing how to use the delims in the regex match
#### $delims = "\s-\s" ",\s"\s and\s"
# First count the number of files in the $OrigFolder directory
$numFiles = (Get-ChildItem -Path $OrigFolder).Count
$i=0
# Tell the user what will happen
clear-host;
Write-Host 'This script will copy ' $numFiles ' files from ' $OrigFolder ' to _Sorted to Move'
# Ask user to confirm the copy operation
Read-host -prompt 'Press enter to start copying the files'
# Regex to match filenames
$Regex = [regex]"(?:(.*?)\s-)|(?:(.*?),\s)|(?:(.*?)\sand\s)"
# Loop through the $OrigFolder directory, skipping folders
Get-ChildItem -LiteralPath $OrigFolder | Where-Object {!$_.PsIsContainer} |
ForEach-Object {
if($_.BaseName -match $Regex){
$ChildPath = $_.BaseName -replace $Regex
#Caluclate copy operation progress as a percentage
[int]$percent = $i / $numFiles * 100
# If first part of the file name is empty, move it to the '_Orphans' folder
if(!$Matches[1]){
$ChildPath = $Orphans}
else {
$ChildPath = $Matches[1]
}
# Generate new folder name
$FolderName = Join-Path -Path $NewFolder -ChildPath ($ChildPath + ' Magazine')
# Create folder if it doesn't exist
if(!(Test-Path -LiteralPath $FolderName -PathType Container)){
$null = New-Item -Path $FolderName -ItemType Directory}
# Log progress to the screen
Write-Host "$($_.FullName) -> $FolderName"
# Move the file to the folder
Move-Item -LiteralPath $_.FullName -Destination $FolderName
# Tell the user how much has been moved
Write-Progress -Activity "Copying ... ($percent %)" -status $_ -PercentComplete $percent -verbose
$i++
}
}
Write-Host 'Total number of files in '$OrigFolder ' is ' $numFiles
Write-Host 'Total number of files copied to '$NewFolder ' is ' $i
Read-host -prompt "Press enter to complete..."
clear-host;
Q1: I tested your script on my machine with PDFs that followed the same naming scheme and it didn't create an Orphans folder or move my Orphan PDFs. I noticed you have a if ($_.BaseName -match $Regex) immediately after your foreach. Inside that is where you are looking for Orphans, but orphans wouldn't make it into this if block because they wouldn't match the Regex. In pseudocode, you structure should be something like:
foreach{
if (match)
{$childpath $_.BaseName -replace $Regex}
else
{Childpath = $Orphans}
Create your folders and do your moves.
}
Q2: Try, Catch blocks: https://blogs.technet.microsoft.com/heyscriptingguy/2014/07/05/weekend-scripter-using-try-catch-finally-blocks-for-powershell-error-handling/
Q3: You can comment multiple lines by enclosing them in <# #> pairs.
<#
How to use an array of values for the delimiters in the regex instead of literals
My proposed code, but I am missing how to use the delims in the regex match
$delims = "\s-\s" ",\s"\s and\s"
#>
I have a script that I have put together, for the most part the script does what I want it to do, it hit's a list of servers, looking for log files that are 25+ hours old (indicating that another script isn't doing it's job), this worked perfectly in testing(1 to 5 servers), however once I turned it loose on the 150+ servers I want to check on in this environment, the file size increased, and the email process failed due to the fact the filesize is in excess of 10mb.
So now I need a way to compress the results, I would like to use 7zip, but for some reason I just cannot wrap my head around how to accomplish what I'm trying to do.
Any assistance would be greatly appreciated.
Here is the script I have thus far.
# Specify where the list of servers is located.
$SL = get-content C:\Scripts\lists\AgingLogsServers.txt
# Define logfile age to report on.
$limit = (Get-Date).AddHours(-25)
# Define the current date & time.
$filedate = get-date -f "MM.dd.yy_hh.mm.ss"
$emldate = get-date -f "MM.dd.yy"
# Variable to add current date & time to saved filename.
$filename = "AgingReport_$($filedate).log"
# Files or patterns to exclude from the scan.
$excluded = #(".exe")
# Specify SMTP server
$smtpserver = "mail.yourserver.com"
# Loop to process each server in the pool.
Foreach ($Server in $SL){
$c1++
Write-Progress -Activity 'Looking for Logfiles in excess of 25 hours old' -Status "Processing $($c1) of $($SL.count)" -CurrentOperation $Server -PercentComplete (($c1/$SL.count) * 100)
If (Test-Path "\\$Server\logs") {$SP1 = "\\$Server\Logs"}
Else {$SP1 = "\\$Server\D-Logs"}
Get-ChildItem -ErrorAction SilentlyContinue -Path $SP1 -Exclude $excluded -Include *.zip, *.7z, *.log -Recurse | Where-Object { !$_.PSIsContainer -and $_.CreationTime -lt $limit } | Foreach-Object {write-output $_.CreationTime $server $_.fullname} | Out-file C:\Scripts\data\$filename -Append -Width 160
}
# Zip the $filename and remove the original
# And this is where I believe a 7zip process would go to compress the result file, then I can reference that file and path in the Send-MailMessage line.
# Email the results.
Send-MailMessage -From "Aging.Logs#yourhost.com" -To "user#yourhost.com" -Subject "Aging Logs report for $emldate" -Body "Attached is the listing of aging logs for the environment for $emldate" -SmtpServer $smtpserver -Attachments C:\Scripts\data\$filename
# Give the logfile time to release from the email process.
Start-Sleep -s 120
# Clean up the results file.
#Remove-Item C:\Scripts\data\AgingReport*
Running 7-Zip is pretty easy. The syntax is
7z.exe a <archive path> <file to zip path>
That's easy, we just need to know where 7z.exe is. So, we'll make PowerShell find that, then execute it using the call operator &, with those parameters (by the way, the 'a' means that we're adding a file to an archive). Then we clean up the source file, and email the archive.
# Zip the $filename and remove the original
# Find 7-Zip executable
$7Zip = gci c:\Program* -include '7z.exe' -recurse -ea 4|select -first 1 -expand fullname
# Define archive name and path
$ZipFile = "C:\Scripts\data\$filename" -replace "...$","zip"
# Perform Zip
& $7Zip a "$ZipFile" "C:\Scripts\data\$filename" | Out-Null
# Remove source file
Remove-Item -Path "C:\Scripts\data\$filename"
# Email the results.
Send-MailMessage -From "Aging.Logs#yourhost.com" -To "user#yourhost.com" -Subject "Aging Logs report for $emldate" -Body "Attached is the listing of aging logs for the environment for $emldate" -SmtpServer $smtpserver -Attachments $ZipFile
Your archive, by the way, will be the same as your log file, but with a .zip file extension instead of .log.
#TheMadTechnician, Thank you for your very helpful post, I attempted to integrate what you gave me, but no matter how I went about it I was unable to get the desired action, I took the direction that you provided and finally was able to get it to work, here is the code that does everything that I wanted it to do, in-case anyone else is looking to accomplish the same thing.
<#
Script: AgingLogQuery.ps1
Author: Xander J.
Date: 11/12/2015
Aging log query checks the logs and d-logs shares contained within a text file to see if there are any logfiles older than 25
hours old,if it finds a logfile that is older than 25 hours old it passes the server name, the full path and filename and the
files age to the AgingReport log file.
After checking all of the servers in the list, the script archives the logfile, removes the original logfile, emails the
archive as an attachment, then waits a specified amount of time to remove the archive file.
#>
# Specify where the list of servers is located.
$SL = get-content C:\Scripts\lists\AgingLogsServers.txt
# Define logfile age to report on.
$limit = (Get-Date).AddHours(-25)
# Define the current date & time.
$filedate = get-date -f "MM.dd.yy_hh.mm.ss"
$emldate = get-date -f "MM.dd.yy"
# Variable to add current date & time to saved filename.
$filename = "AgingReport_$($filedate).log"
# Files or patterns to exclude from the scan.
$excluded = #("*.exe*")
# Specify SMTP server
$smtpserver = "mail.email.com"
#Get script path
$ScriptPath = Split-Path -Path $($MyInvocation.MyCommand.Definition)
#Get the path for 7za.exe
$zipexe = $ScriptPath + "\7za.exe"
set-alias sz $zipexe
$archive = "AgingReport_$($filedate).zip"
# Loop to process each server on the list.
Foreach ($Server in $SL){
$c1++
Write-Progress -Activity 'Looking for Logfiles in excess of 25 hours old' -Status "Processing $($c1) of $($SL.count)" -CurrentOperation $Server -PercentComplete (($c1/$SL.count) * 100)
If (Test-Path "\\$Server\logs") {$SP1 = "\\$Server\Logs"}
Else {$SP1 = "\\$Server\D-Logs"}
Get-ChildItem -ErrorAction SilentlyContinue -Path $SP1 -Exclude $excluded -Include *.zip, *.7z, *.log -Recurse | Where-Object { !$_.PSIsContainer -and $_.CreationTime -lt $limit } | Foreach-Object {write-output $_.CreationTime $server $_.fullname} | Out-file C:\Scripts\data\$filename -Append -Width 160
}
# Zip the $filename
& sz a -mmt -tzip c:\Scripts\Data\$archive C:\Scripts\data\AgingReport*.log -stl
# Clean up the results file.
Remove-Item -Force C:\Scripts\data\$filename
# Email the results.
Send-MailMessage -From "Aging.Logs#echopass.com" -To "user#email.com" -Subject "Aging Logs report for $emldate" -Body "Attached is the listing of aging logs for the environment for $emldate" -SmtpServer $smtpserver -Attachments C:\Scripts\data\$archive
# Give the logfile time to release from the email process.
Start-Sleep -s 15
# Clean up the results file.
Remove-Item -Force C:\Scripts\data\$archive
I want to write a PowerShell script that will recursively search a directory, but exclude specified files (for example, *.log, and myFile.txt), and also exclude specified directories, and their contents (for example, myDir and all files and folders below myDir).
I have been working with the Get-ChildItem CmdLet, and the Where-Object CmdLet, but I cannot seem to get this exact behavior.
I like Keith Hill's answer except it has a bug that prevents it from recursing past two levels. These commands manifest the bug:
New-Item level1/level2/level3/level4/foobar.txt -Force -ItemType file
cd level1
GetFiles . xyz | % { $_.fullname }
With Hill's original code you get this:
...\level1\level2
...\level1\level2\level3
Here is a corrected, and slightly refactored, version:
function GetFiles($path = $pwd, [string[]]$exclude)
{
foreach ($item in Get-ChildItem $path)
{
if ($exclude | Where {$item -like $_}) { continue }
$item
if (Test-Path $item.FullName -PathType Container)
{
GetFiles $item.FullName $exclude
}
}
}
With that bug fix in place you get this corrected output:
...\level1\level2
...\level1\level2\level3
...\level1\level2\level3\level4
...\level1\level2\level3\level4\foobar.txt
I also like ajk's answer for conciseness though, as he points out, it is less efficient. The reason it is less efficient, by the way, is because Hill's algorithm stops traversing a subtree when it finds a prune target while ajk's continues. But ajk's answer also suffers from a flaw, one I call the ancestor trap. Consider a path such as this that includes the same path component (i.e. subdir2) twice:
\usr\testdir\subdir2\child\grandchild\subdir2\doc
Set your location somewhere in between, e.g. cd \usr\testdir\subdir2\child, then run ajk's algorithm to filter out the lower subdir2 and you will get no output at all, i.e. it filters out everything because of the presence of subdir2 higher in the path. This is a corner case, though, and not likely to be hit often, so I would not rule out ajk's solution due to this one issue.
Nonetheless, I offer here a third alternative, one that does not have either of the above two bugs. Here is the basic algorithm, complete with a convenience definition for the path or paths to prune--you need only modify $excludeList to your own set of targets to use it:
$excludeList = #("stuff","bin","obj*")
Get-ChildItem -Recurse | % {
$pathParts = $_.FullName.substring($pwd.path.Length + 1).split("\");
if ( ! ($excludeList | where { $pathParts -like $_ } ) ) { $_ }
}
My algorithm is reasonably concise but, like ajk's, it is less efficient than Hill's (for the same reason: it does not stop traversing subtrees at prune targets). However, my code has an important advantage over Hill's--it can pipeline! It is therefore amenable to fit into a filter chain to make a custom version of Get-ChildItem while Hill's recursive algorithm, through no fault of its own, cannot. ajk's algorithm can be adapted to pipeline use as well, but specifying the item or items to exclude is not as clean, being embedded in a regular expression rather than a simple list of items that I have used.
I have packaged my tree pruning code into an enhanced version of Get-ChildItem. Aside from my rather unimaginative name--Get-EnhancedChildItem--I am excited about it and have included it in my open source Powershell library. It includes several other new capabilities besides tree pruning. Furthermore, the code is designed to be extensible: if you want to add a new filtering capability, it is straightforward to do. Essentially, Get-ChildItem is called first, and pipelined into each successive filter that you activate via command parameters. Thus something like this...
Get-EnhancedChildItem –Recurse –Force –Svn
–Exclude *.txt –ExcludeTree doc*,man -FullName -Verbose
... is converted internally into this:
Get-ChildItem | FilterExcludeTree | FilterSvn | FilterFullName
Each filter must conform to certain rules: accepting FileInfo and DirectoryInfo objects as inputs, generating the same as outputs, and using stdin and stdout so it may be inserted in a pipeline. Here is the same code refactored to fit these rules:
filter FilterExcludeTree()
{
$target = $_
Coalesce-Args $Path "." | % {
$canonicalPath = (Get-Item $_).FullName
if ($target.FullName.StartsWith($canonicalPath)) {
$pathParts = $target.FullName.substring($canonicalPath.Length + 1).split("\");
if ( ! ($excludeList | where { $pathParts -like $_ } ) ) { $target }
}
}
}
The only additional piece here is the Coalesce-Args function (found in this post by Keith Dahlby), which merely sends the current directory down the pipe in the event that the invocation did not specify any paths.
Because this answer is getting somewhat lengthy, rather than go into further detail about this filter, I refer the interested reader to my recently published article on Simple-Talk.com entitled Practical PowerShell: Pruning File Trees and Extending Cmdlets where I discuss Get-EnhancedChildItem at even greater length. One last thing I will mention, though, is another function in my open source library, New-FileTree, that lets you generate a dummy file tree for testing purposes so you can exercise any of the above algorithms. And when you are experimenting with any of these, I recommend piping to % { $_.fullname } as I did in the very first code fragment for more useful output to examine.
The Get-ChildItem cmdlet has an -Exclude parameter that is tempting to use but it doesn't work for filtering out entire directories from what I can tell. Try something like this:
function GetFiles($path = $pwd, [string[]]$exclude)
{
foreach ($item in Get-ChildItem $path)
{
if ($exclude | Where {$item -like $_}) { continue }
if (Test-Path $item.FullName -PathType Container)
{
$item
GetFiles $item.FullName $exclude
}
else
{
$item
}
}
}
Here's another option, which is less efficient but more concise. It's how I generally handle this sort of problem:
Get-ChildItem -Recurse .\targetdir -Exclude *.log |
Where-Object { $_.FullName -notmatch '\\excludedir($|\\)' }
The \\excludedir($|\\)' expression allows you to exclude the directory and its contents at the same time.
Update: Please check the excellent answer from msorens for an edge case flaw with this approach, and a much more fleshed out solution overall.
Recently, I explored the possibilities to parameterize the folder to scan through and the place where the result of recursive scan will be stored. At the end, I also did summarize the number of folders scanned and number of files inside as well. Sharing it with community in case it may help other developers.
##Script Starts
#read folder to scan and file location to be placed
$whichFolder = Read-Host -Prompt 'Which folder to Scan?'
$whereToPlaceReport = Read-Host -Prompt 'Where to place Report'
$totalFolders = 1
$totalFiles = 0
Write-Host "Process started..."
#IMP separator ? : used as a file in window cannot contain this special character in the file name
#Get Foldernames into Variable for ForEach Loop
$DFSFolders = get-childitem -path $whichFolder | where-object {$_.Psiscontainer -eq "True"} |select-object name ,fullName
#Below Logic for Main Folder
$mainFiles = get-childitem -path "C:\Users\User\Desktop" -file
("Folder Path" + "?" + "Folder Name" + "?" + "File Name " + "?"+ "File Length" )| out-file "$whereToPlaceReport\Report.csv" -Append
#Loop through folders in main Directory
foreach($file in $mainFiles)
{
$totalFiles = $totalFiles + 1
("C:\Users\User\Desktop" + "?" + "Main Folder" + "?"+ $file.name + "?" + $file.length ) | out-file "$whereToPlaceReport\Report.csv" -Append
}
foreach ($DFSfolder in $DFSfolders)
{
#write the folder name in begining
$totalFolders = $totalFolders + 1
write-host " Reading folder C:\Users\User\Desktop\$($DFSfolder.name)"
#$DFSfolder.fullName | out-file "C:\Users\User\Desktop\PoC powershell\ok2.csv" -Append
#For Each Folder obtain objects in a specified directory, recurse then filter for .sft file type, obtain the filename, then group, sort and eventually show the file name and total incidences of it.
$files = get-childitem -path "$whichFolder\$($DFSfolder.name)" -recurse
foreach($file in $files)
{
$totalFiles = $totalFiles + 1
($DFSfolder.fullName + "?" + $DFSfolder.name + "?"+ $file.name + "?" + $file.length ) | out-file "$whereToPlaceReport\Report.csv" -Append
}
}
# If running in the console, wait for input before closing.
if ($Host.Name -eq "ConsoleHost")
{
Write-Host ""
Write-Host ""
Write-Host ""
Write-Host " **Summary**" -ForegroundColor Red
Write-Host " ------------" -ForegroundColor Red
Write-Host " Total Folders Scanned = $totalFolders " -ForegroundColor Green
Write-Host " Total Files Scanned = $totalFiles " -ForegroundColor Green
Write-Host ""
Write-Host ""
Write-Host "I have done my Job,Press any key to exit" -ForegroundColor white
$Host.UI.RawUI.FlushInputBuffer() # Make sure buffered input doesn't "press a key" and skip the ReadKey().
$Host.UI.RawUI.ReadKey("NoEcho,IncludeKeyUp") > $null
}
##Output
##Bat Code to run above powershell command
#ECHO OFF
SET ThisScriptsDirectory=%~dp0
SET PowerShellScriptPath=%ThisScriptsDirectory%MyPowerShellScript.ps1
PowerShell -NoProfile -ExecutionPolicy Bypass -Command "& {Start-Process PowerShell -ArgumentList '-NoProfile -ExecutionPolicy Bypass -File ""%PowerShellScriptPath%""' -Verb RunAs}";
A bit late, but try this one.
function Set-Files($Path) {
if(Test-Path $Path -PathType Leaf) {
# Do any logic on file
Write-Host $Path
return
}
if(Test-Path $path -PathType Container) {
# Do any logic on folder use exclude on get-childitem
# cycle again
Get-ChildItem -Path $path | foreach { Set-Files -Path $_.FullName }
}
}
# call
Set-Files -Path 'D:\myFolder'
Commenting here as this seems to be the most popular answer on the subject for searching for files whilst excluding certain directories in powershell.
To avoid issues with post filtering of results (i.e. avoiding permission issues etc), I only needed to filter out top level directories and that is all this example is based on, so whilst this example doesn't filter child directory names, it could very easily be made recursive to support this, if you were so inclined.
Quick breakdown of how the snippet works
$folders << Uses Get-Childitem to query the file system and perform folder exclusion
$file << The pattern of the file I am looking for
foreach << Iterates the $folders variable performing a recursive search using the Get-Childitem command
$folders = Get-ChildItem -Path C:\ -Directory -Name -Exclude Folder1,"Folder 2"
$file = "*filenametosearchfor*.extension"
foreach ($folder in $folders) {
Get-Childitem -Path "C:/$folder" -Recurse -Filter $file | ForEach-Object { Write-Output $_.FullName }
}