Recursively count files in subfolders - powershell

I am trying to count the files in all subfolders in a directory and display them in a list.
For instance the following dirtree:
TEST
/VOL01
file.txt
file.pic
/VOL02
/VOL0201
file.nu
/VOL020101
file.jpg
file.erp
file.gif
/VOL03
/VOL0301
file.org
Should give as output:
PS> DirX C:\TEST
Directory Count
----------------------------
VOL01 2
VOL02 0
VOL02/VOL0201 1
VOL02/VOL0201/VOL020101 3
VOL03 0
VOL03/VOL0301 1
I started with the following:
Function DirX($directory)
{
foreach ($file in Get-ChildItem $directory -Recurse)
{
Write-Host $file
}
}
Now I have a question: why is my Function not recursing?

Something like this should work:
dir -recurse | ?{ $_.PSIsContainer } | %{ Write-Host $_.FullName (dir $_.FullName | Measure-Object).Count }
dir -recurse lists all files under current directory and pipes (|) the result to
?{ $_.PSIsContainer } which filters directories only then pipes again the resulting list to
%{ Write-Host $_.FullName (dir $_.FullName | Measure-Object).Count } which is a foreach loop that, for each member of the list ($_) displays the full name and the result of the following expression
(dir $_.FullName | Measure-Object).Count which provides a list of files under the $_.FullName path and counts members through Measure-Object
?{ ... } is an alias for Where-Object
%{ ... } is an alias for foreach

Similar to David's solution this will work in Powershell v3.0 and does not uses aliases in case someone is not familiar with them
Get-ChildItem -Directory | ForEach-Object { Write-Host $_.FullName $(Get-ChildItem $_ | Measure-Object).Count}
Answer Supplement
Based on a comment about keeping with your function and loop structure i provide the following. Note: I do not condone this solution as it is ugly and the built in cmdlets handle this very well. However I like to help so here is an update of your script.
Function DirX($directory)
{
$output = #{}
foreach ($singleDirectory in (Get-ChildItem $directory -Recurse -Directory))
{
$count = 0
foreach($singleFile in Get-ChildItem $singleDirectory.FullName)
{
$count++
}
$output.Add($singleDirectory.FullName,$count)
}
$output | Out-String
}
For each $singleDirectory count all files using $count ( which gets reset before the next sub loop ) and output each finding to a hash table. At the end output the hashtable as a string. In your question you looked like you wanted an object output instead of straight text.

Well, the way you are doing it the entire Get-ChildItem cmdlet needs to complete before the foreach loop can begin iterating. Are you sure you're waiting long enough? If you run that against very large directories (like C:) it is going to take a pretty long time.
Edit: saw you asked earlier for a way to make your function do what you are asking, here you go.
Function DirX($directory)
{
foreach ($file in Get-ChildItem $directory -Recurse -Directory )
{
[pscustomobject] #{
'Directory' = $File.FullName
'Count' = (GCI $File.FullName -Recurse).Count
}
}
}
DirX D:\
The foreach loop only get's directories since that is all we care about, then inside of the loop a custom object is created for each iteration with the full path of the folder and the count of the items inside of the folder.
Also, please note that this will only work in PowerShell 3.0 or newer, since the -directory parameter did not exist in 2.0

Get-ChildItem $rootFolder `
-Recurse -Directory |
Select-Object `
FullName, `
#{Name="FileCount";Expression={(Get-ChildItem $_ -File |
Measure-Object).Count }}

My version - slightly cleaner and dumps content to a file
Original - Recursively count files in subfolders
Second Component - Count items in a folder with PowerShell
$FOLDER_ROOT = "F:\"
$OUTPUT_LOCATION = "F:DLS\OUT.txt"
Function DirX($directory)
{
Remove-Item $OUTPUT_LOCATION
foreach ($singleDirectory in (Get-ChildItem $directory -Recurse -Directory))
{
$count = Get-ChildItem $singleDirectory.FullName -File | Measure-Object | %{$_.Count}
$summary = $singleDirectory.FullName+" "+$count+" "+$singleDirectory.LastAccessTime
Add-Content $OUTPUT_LOCATION $summary
}
}
DirX($FOLDER_ROOT)

I modified David Brabant's solution just a bit so I could evaluate the result:
$FileCounter=gci "$BaseDir" -recurse | ?{ $_.PSIsContainer } | %{ (gci "$($_.FullName)" | Measure-Object).Count }
Write-Host "File Count=$FileCounter"
If($FileCounter -gt 0) {
... take some action...
}

Related

Find similarly-named files, and if present, remove the files without a specific string using PowerShell

In a directory, there are files with the following filenames:
ExampleFile.mp3
ExampleFile_pn.mp3
ExampleFile2.mp3
ExampleFile2_pn.mp3
ExampleFile3.mp3
I want to iterate through the directory, and IF there is a filename that contains the string '_pn.mp3', I want to test if there is a similarly named file without the '_pn.mp3' in the same directory. If that file exists, I want to remove it.
In the above example, I'd want to remove:
ExampleFile.mp3
ExampleFile2.mp3
and I'd want to keep ExampleFile3.mp3
Here's what I have so far:
$pattern = "_pn.mp3"
$files = Get-ChildItem -Path '$path' | Where-Object {! $_.PSIsContainer}
Foreach ($file in $files) {
If($file.Name -match $pattern){
# filename with _pn.mp3 exists
Write-Host $file.Name
# search in the current directory for the same filename without _pn
<# If(Test-Path $currentdir $filename without _pn.mp3) {
Remove-Item -Force}
#>
}
enter code here
You could use Group-Object to group all files by their BaseName (with the pattern removed), and then loop over the groups where there are more than one file. The result of grouping the files and filtering by count would look like this:
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1
Count Name Group
----- ---- -----
2 ExampleFile {ExampleFile.mp3, ExampleFile_pn.mp3}
2 ExampleFile2 {ExampleFile2.mp3, ExampleFile2_pn.mp3}
Then if we loop over these groups we can search for the files that do not end with the $pattern:
#'
ExampleFile.mp3
ExampleFile_pn.mp3
ExampleFile2.mp3
ExampleFile2_pn.mp3
ExampleFile3.mp3
'# -split '\r?\n' -as [System.IO.FileInfo[]] | Set-Variable files
$pattern = "_pn"
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1 | ForEach-Object {
$_.Group.Where({-not $_.BaseName.Endswith($pattern)})
}
This is how your code would look like, remove the -WhatIf switch if you consider the code is doing what you wanted.
$pattern = "_pn.mp3"
$files = Get-ChildItem -Path -Filter *.mp3 -File
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1 | ForEach-Object {
$toRemove = $_.Group.Where({-not $_.BaseName.Endswith($pattern)})
Remove-Item $toRemove -WhatIf
}
I think you can get by here by adding file names into a hash map as you go. If you encounter a file with the ending you are interested in, check if a similar file name was added. If so, remove both the file and the similar match.
$ending = "_pn.mp3"
$files = Get-ChildItem -Path $path -File | Where-Object { ! $_.PSIsContainer }
$hash = #{}
Foreach ($file in $files) {
# Check if file has an ending we are interested in
If ($file.Name.EndsWith($ending)) {
$similar = $file.Name.Split($ending)[0] + ".mp3"
# Check if we have seen the similar file in the hashmap
If ($hash.Contains($similar)) {
Write-Host $file.Name
Write-Host $similar
Remove-Item -Force $file
Remove-Item -Force $hash[$similar]
# Remove similar from hashmap as it is removed and no longer of interest
$hash.Remove($similar)
}
}
else {
# Add entry for file name and reference to the file
$hash.Add($file.Name, $file)
}
}
Just get a list of the files with the _pn then process against the rest.
$pattern = "*_pn.mp3"
$files = Get-ChildItem -Path "$path" -File -filter "$pattern"
Foreach ($file in $files) {
$TestFN = $file.name -replace("_pn","")
If (Test-Path -Path $(Join-Path -Path $Path -ChildPath $TestFN)) {
$file | Remove-Item -force
}
} #End Foreach

Execute get-childitem, but iterate in reverse order?

I have a folder full of 500,00+ files. I'm trying to iterate through this folder and run some logic to determine if we can delete unneeded files. The problem is this process needs to run semi-regularly and the new files that need to be deleted are currently at the end of the list it seems.
I put together the following list of code to sort through it all:
gci $RPT | %{
$flag = 0;
$number = [int]($_.Name | select-string -pattern "\d{12}" -Allmatches).Matches.Value
if ($submidlist -match "^$number$"){
if ($_ -notmatch "acct\.csv|jpd\.csv|jss\.pdf|jman\.pdf|3600\.pdf|cont\.pdf|msl\.txt|pres\.pdf|tray\.pdf|qual\.pdf|zipl\.pdf"){
echo "DELETE SUBMID $_"
remove-item $RPT\$_
$count++
$totalcount++
$flag = 1;
}
}
if ($jobidlist -match "^$number$"){
if ($_ -match "acct\.csv|jpd\.csv|jss\.pdf|jman\.pdf|3600\.pdf|cont\.pdf|msl\.txt|pres\.pdf|tray\.pdf|qual\.pdf|zipl\.pdf"){
echo "DELETE JOBID $_"
remove-item $RPT\$_
$count++
$totalcount++
$flag = 1;
}
}
}
Currently, running the above script takes over 24 hours and it still doesn't make it to the end of the list. Is there a way to optimize this or reverse the order that get-childitem iterates through this folder?
function Delete-Items($List, [string]$ListName){
$DoNotDelete = #("acct.csv","jpd.csv","jss.pdf","jman.pdf","3600.pdf","cont.pdf","msl.txt","pres.pdf","tray.pdf","qual.pdf","zipl.pdf")
$List = $List | %{
"*$_*"
}
Get-ChildItem C:\TEST\56381643\ -Recurse -Include $List -Directory | %{
Get-ChildItem $_.FullName -Exclude $DoNotDelete -Recurse | %{
echo "DELETE $ListName $($_.name | select-string -pattern "\d{12}")"
Remove-Item -Path $_.FullName -WhatIf
}
}
}
#Example Usage
$JobList = #(
098765432109
123456789012
)
$SubmitList = #(
234567890123
)
Delete-Items -List $JobList -ListName JOBID
Delete-Items -List $SubmitList -ListName SUBMID
Lets go over a basic rundown of whats happening in the function.
We have a array of files not to delete
We turn the $list numbers into wildcards by adding a * before and after each item in the array. We then only search for those directories that contain those numbers.
We then use another Get-ChildItem to get the files in each directory but exclude the ones mentioned in$DoNotDelete`.
If you want to delete the files delete the -Whatif on the remove-item

Powershell - Exclude folders in Get-ChildItem

How to exclude folders ? Now I hardcode the folder names but i want it to be more flexible.
foreach($file in Get-ChildItem $fileDirectory -Exclude folderA,folderb)
"How to exclude folders ?" , if you mean all folders :
get-childitem "$fileDirectory\\*" -file
but it works only for the first level of $fileDirectory .
This works recursevly :
Get-ChildItem "$fileDirectory\\*" -Recurse | ForEach-Object { if (!($_.PSIsContainer)) { $_}}
or
Get-ChildItem "$fileDirectory\\*" -Recurse | where { !$_.PSisContainer }
You can do this by using the pipeline and a Where-Object filter.
First of all, the idiomatic way to iterate over a group of files in PowerShell is to pipe Get-Childitem to Foreach-Object. So rewriting your command gets:
Get-ChildItem $fileDirectory | foreach {
$file = $_
...
}
The advantage of using the pipeline is that now you can insert other cmdlets in between. Specifically, we use Where-Object to filter the list of files. The filter will pass on a file only if it isn't contained in a given array.
$excludelist = 'folderA', 'folderB'
Get-Childitem $fileDirectory |
where { $excludeList -notcontains $_ } |
foreach {
$file = $_
...
}
If you're going to use this a lot, you can even write a custom filter function to modify the list of files in an arbitrary way before passing to foreach.
filter except($except, $unless = #()) {
if ($except -notcontains $_ -or $unless -contains $_ ){
$_
}
}
$excludelist = 'folderA', 'folderB'
$alwaysInclude = 'folderC', 'folderD'
Get-ChildItem $fileDirectory |
except $excludeList -unless $alwaysInclude |
foreach {
...
}
#dvjz said that -file works only in the first level of a folder, but not recursively. But it seems to work for me.
get-childitem "$fileDirectory\\*" -file -recurse
For future googlers, I have found that files have a property called PSIsContainer which is $true when they are a directory.
A command listing all files in $fileDirectory would be:
foreach ($file in Get-ChildItem $fileDirectory | Where-Object -Property PSIsContainer -eq $false)
{
Write-Host $file.Name
}
Note that -Property is optional for the cmdlet Where-Object.
The simplest way to exclude your folders recursively:
foreach($file in Get-ChildItem $fileDirectory -Exclude {Get-ChildItem folderA},{Get-ChildItem folderB})
Where:
$fileDirectory - search folder
folderA, folderB - excluded folders

Powershell Script to search for credit card numbers in a folder

I am using the below script to search for credit card numbers inside a folder that contains many subfolders:
Get-ChildItem -rec | ?{ findstr.exe /mprc:. $_.FullName }
| select-string "[456][0-9]{15}","[456][0-9]{3}[-| ][0-9]{4} [-| ][0-9]{4}[-| ][0-9]{4}"
However, this will return all instances found in every folder/subfolder.
How can I amend the script to skip the current folder on the first instance found? meaning that if it finds a credit card number it will stop processing the current folder and move to the next folder.
Appreciate you answers and help.
Thanks in advance,
You could use this recursive function:
function cards ($dir)
Get-ChildItem -Directory $dir | % { cards($_.FullName) }
Get-ChildItem -File $dir\* | % {
if ( Select-String $_.FullName "[456][0-9]{15}","[456][0-9]{3}[-| ][0-9]{4} [-| ][0-9]{4}[-| ][0-9]{4}" ) {
write-host "card found in $dir"
return
}
}
}
cards "C:\path\to\base\dir"
It'll keep going through subdirectories of the top level directory you specify. Whenever it gets to a directory with no subdirectories, or its been through all the subdirectories of the current directory, it'll start looking through the files for the matching regex, but will bail out of the function when the first match is found.
So really what you want is the first file in every folder that has a credit card number in the contents.
Break it into two parts. Get a list of all your folders, recursively. Then, for each folder, get the list of files, non-recursively. Search each file until you find one that matches.
I don't see any easy way to do this with pipes alone. That means more traditional programming techniques.
This requires PowerShell 3.0. I've eliminated ?{ findstr.exe /mprc:. $_.FullName } because all I can see that it does is eliminate folders (and zero length files) and this already handles that.
Get-ChildItem -Directory -Recurse | ForEach-Object {
$Found = $false;
$i = 0;
$Files = $_ | Get-ChildItem -File | Sort-Object -Property Name;
for ($i = 0; ($Files[$i] -ne $null) -and ($Found -eq $false); $i++) {
$SearchResult = $Files[$i] | Select-String "[456][0-9]{15}","[456][0-9]{3}[-| ][0-9]{4} [-| ][0-9]{4}[-| ][0-9]{4}";
if ($SearchResult) {
$Found = $true;
Write-Output $SearchResult;
}
}
}
Didn't have the time to test it fully, but I thought about something like this:
$Location = 'H:\'
$Dirs = Get-ChildItem $Location -Directory -Recurse
$Regex1 = "[456][0-9]{3}[-| ][0-9]{4} [-| ][0-9]{4}[-| ][0-9]{4}"
$Regex2 = "[456][0-9]{15}"
Foreach ($d in $Dirs) {
$Files = Get-ChildItem $d.FullName -File
foreach ($f in $Files) {
if (($f.Name -match $Regex1) -or ($f.Name -match $Regex2)) {
Write-Host 'Match found'
Return
}
}
}
Here is another one, why not, the more the merrier.
I'm assuming that your Regex is correct.
Using break in the second loop will skip looking for a credit card in the remaining files if one is found and continue to the next folder.
$path = '<your path here>'
$folders = Get-ChildItem $path -Directory -rec
foreach ($folder in $folders)
{
$items = Get-ChildItem $folder.fullname -File
foreach ($i in $items)
{
if (($found = $i.FullName| select-string "[456][0-9]{15}","[456][0-9]{3}[-| ][0-9]{4} [-| ][0-9]{4}[-| ][0-9]{4}") -ne $null)
{
break
}
}
}
I think the intention was to look inside each file for the PII data right?
If so, you need to open the load the file and search each line. The code you posted will only run a regex on the name of the file.

How do I recursively rename folders with Powershell?

Recursive renaming files using PS is trivial (variation on example from Mike Ormond's blog):
dir *_t*.gif -recurse
| foreach { move-item -literal $_ $_.Name.Replace("_thumb[1]", "")}
I'm trying to recursively rename a folder structure.
The use case is I'd like to be able to rename a whole VS.NET Solution (e.g. from Foo.Bar to Bar.Foo). To do this there are several steps:
Rename folders (e.g. \Foo.Bar\Foo.Bar.Model => \Bar.Foo\Bar.Foo.Model)
Rename files (e.g. Foo.Bar.Model.csproj => Bar.Foo.Model.csproj)
Find and Replace within files to correct for namespace changes (e.g. 'namespace Foo.Bar' => 'namespace Bar.Foo')
I'm currently working the first step in this process.
I found this posting, which talks about the challenges, and claims a solution but doesn't talk about what that solution is.
I keep running into the recursion wall. If I let PS deal with the recursion using a flag, the parent folder gets renamed before the children, and the script throws an error. If I try to implement the recursion myself, my head get's all achy and things go horribly wrong - for the life of me I cannot get things to start their renames at the tail of the recursion tree.
Here's the solution rbellamy ended up with:
Get-ChildItem $Path -Recurse | %{$_.FullName} |
Sort-Object -Property Length -Descending |
% {
Write-Host $_
$Item = Get-Item $_
$PathRoot = $Item.FullName | Split-Path
$OldName = $Item.FullName | Split-Path -Leaf
$NewName = $OldName -replace $OldText, $NewText
$NewPath = $PathRoot | Join-Path -ChildPath $NewName
if (!$Item.PSIsContainer -and $Extension -contains $Item.Extension) {
(Get-Content $Item) | % {
#Write-Host $_
$_ -replace $OldText, $NewText
} | Set-Content $Item
}
if ($OldName.Contains($OldText)) {
Rename-Item -Path $Item.FullName -NewName $NewPath
}
}
How about this - do a recursive list of the full names, sort it in descending order by the length of the full name, and then run that back through your rename routine.
e.g.
gci <directory> -recurse |
foreach {$_.fullname} |
sort -length -desc
Maybe something in this is useful, here's a snippet that recurses and prepends "pre" to a directory structure
$dirs = Get-ChildItem c:/foldertorecurse -rec | Where-Object {$_.PSIsContainer -eq 1} | sort fullname -descending
foreach ( $dir in $dirs ) { rename-item -path $dir.fullname -newname ("pre" + $dir.name) }