Folders with more than 40.000 files - powershell

I have this script I received to check folders and subfolders on a network drive. I wonder how it could be modified into checking only folders and subfolder and write in the CSV if there is any folder with more then 40.000 files in it and the number of files. The image show a sample output from the script as it is now and I do not need it to show any files as it currently do.
$dir = "D:\test"
$results = #()
gci $dir -Recurse -Depth 1 | % {
$temp = [ordered]#{
NAME = $_
SIZE = "{0:N2} MB" -f ((gci $_.Fullname -Recurse | measure -Property Length -Sum -ErrorAction SilentlyContinue).Sum / 1MB)
FILE_COUNT = (gci -File $_.FullName -Recurse | measure | select -ExpandProperty Count)
FOLDER_COUNT = (gci -Directory $_.FullName -Recurse | measure | select -ExpandProperty Count)
DIRECTORY_PATH = $_.Fullname
}
$results += New-Object PSObject -Property $temp
}
$results | export-csv -Path "C:\temp\output.csv" -NoTypeInformation

Instead of executing so many Get-ChildItem cmdlets, here's an approach that uses robocopy to do the heavy lifting of counting the number of files, folders and total sizes:
# set the rootfolder to search
$dir = 'D:\test'
# switches for robocopy
$roboSwitches = '/L','/E','/NJH','/BYTES','/NC','/NP','/NFL','/XJ','/R:0','/W:0','/MT:16'
# regex to parse the output from robocopy
$regEx = '\s*Total\s*Copied\s*Skipped\s*Mismatch\s*FAILED\s*Extras' +
'\s*Dirs\s*:\s*(?<DirCount>\d+)(?:\s+\d+){3}\s+(?<DirFailed>\d+)\s+\d+' +
'\s*Files\s*:\s*(?<FileCount>\d+)(?:\s+\d+){3}\s+(?<FileFailed>\d+)\s+\d+' +
'\s*Bytes\s*:\s*(?<ByteCount>\d+)(?:\s+\d+){3}\s+(?<BytesFailed>\d+)\s+\d+'
# loop through the directories directly under $dir
$result = Get-ChildItem -Path $dir -Directory | ForEach-Object {
$path = $_.FullName # or if you like $_.Name
$summary = (robocopy.exe $_.FullName NULL $roboSwitches | Select-Object -Last 8) -join [Environment]::NewLine
if ($summary -match $regEx) {
$numFiles = [int64] $Matches['FileCount']
if ($numFiles -gt 40000) {
[PsCustomObject]#{
PATH = $path
SIZE = [int64] $Matches['ByteCount']
FILE_COUNT = [int64] $Matches['FileCount']
FOLDER_COUNT = [int64] $Matches['DirCount']
}
}
}
else {
Write-Warning -Message "Path '$path' output from robocopy was in an unexpected format."
}
}
# output on screen
$result | Format-Table -AutoSize
# output to CSV file
$result | Export-Csv -Path "C:\temp\output.csv" -NoTypeInformation

Related

How can I get the time and date my PowerShell script deletes a file

I am using the following script to read a list of file names which are then deleted. Is there a way can get an output of the date and time each file is deleted?
$targetFolder = "D:\" $fileList = "C:\DeleteList.txt" Get-ChildItem
-Path "$targetFolder\*" -Recurse -Include #(Get-Content $fileList) | Remove-Item -Verbose
Thanks for any help.
You could keep track of the files that are deleted and the time of deletion by outputting an object with the file's fullname and current date.
This output can then be saved as structured CSV file
$targetFolder = "D:\"
$fileList = Get-Content -Path "C:\DeleteList.txt"
$deleted = Get-ChildItem -Path $targetFolder -Recurse -Include $fileList | ForEach-Object {
# output an object with the current date and the file FullName
$_ | Select-Object #{Name = 'DeletedOn'; Expression = {(Get-Date)}}, FullName
$_ | Remove-Item -WhatIf
}
# output on screen
$deleted | Format-Table -AutoSize
# output to csv file
$deleted | Export-Csv -Path 'C:\RemovedFiles.csv' -NoTypeInformation
Remove the -WhatIf safety-switch if you are satisfied with the results shown on screen.
Would this work?
$targetFolder = "D:"
$fileList = "C:\DeleteList.txt"
$Files = Get-ChildItem -Path "$targetFolder" -Recurse -Include #(Get-Content $fileList)
# Once you have the desires files stored in the $Files variable, then run a Foreach loop.
$Obj = #() # create an array called $Obj
Foreach ($File in $Files)
{
# store info in hash table
$hash = #{
DateTime = (get-date)
fileName = $File.name
fullpath = $File.fullname
}
Write-Host "deleting file $($file.name)" -for cyan
Remove-Item $File.fullname # *** BE VERY CAREFUL!!!***
# record information in an array called $Obj
$Obj += New-Object psobject -Property $hash
}
$Obj | select fileName, DateTime | Export-csv C:\...

Creating a Powershell script to get network folder information Problem with capturing and displaying LastWriteTime

Okay i am not a programmer and my Powershell experience is basic. But here goes. I have been asked to collect some info on a Directory we are migrating off our network.
It collects sub dirs names, size, #of files and folders and datestamp and exports to csv.
I cannot for the life of me make the folder creation date work so i gave up on that and have been looking to get the lastwritetime for the folders as i am trying to figure out what has been used recently. It only works for a few folders but the rest in excel have system.object[] in the cell. Super frustrating.
Here is the code. It uses a gui directory picker.
#Refresh network drives for session
net use i: /delete
net use m: /delete
net use i: "\\wfs.queensu.ca\ADV\Workgroups"
net use m: "\\wfs.queensu.ca\ADVMedia"
Function Get-Folder($initialDirectory)
{
[System.Reflection.Assembly]::LoadWithPartialName("System.windows.forms")|Out-Null
$foldername = New-Object System.Windows.Forms.FolderBrowserDialog
$foldername.Description = "Select a folder"
$foldername.rootfolder = "MyComputer"
if($foldername.ShowDialog() -eq "OK")
{
$folder += $foldername.SelectedPath
}
return $folder
}
$forDir = Get-Folder
#Change this to the parent directory that you want counts for
#$forDir = "\\wfs.queensu.ca\adv\workgroups\ADV Services\$seldir"
$Dirs = Get-ChildItem $forDir -Directory -Name
$Tab = [char]9
$results = #()
Write-Host $forDir
foreach($Dir in $Dirs)
{
$dirSize = "{0:N2} MB" -f ((Get-ChildItem $forDir/$Dir -Recurse | Measure-Object -Property Length
-Sum -ErrorAction Stop).Sum / 1MB)
$dirFiles = Get-ChildItem $forDir/$Dir -Recurse -File | Measure-Object | %{$_.Count}
$dirFolders = Get-ChildItem $forDir/$Dir -Recurse -Directory | Measure-Object | %{$_.Count}
#$dirDate = (Get-ChildItem $forDir/$Dir).LastWriteTime.ToString
$dirDate = #(Get-ChildItem $forDir/$Dir | % {$_.LastWriteTime})
$details = [ordered] #{
dir = $Dir
No_Files = $dirFiles
No_Folders = $dirFolders
size = $dirSize
date = $dirDate
}
$results += New-Object PSobject -Property $details
}
#This line finds the last index of the slash and adding one char
$Dirlength = $forDir.LastIndexOf('\') + 1
#This line takes the entire length of the string minus the postion above leaving the directory name
$sublength = $forDir.length - $Dirlength
#Assigns the remaining characters from the substring to the varibale to be used as the filename
$DirName = $forDir.SubString($Dirlength, $sublength)
$results | Export-Csv "C:\$DirName.csv" -NoTypeInformation
Write-Host ("Complete WOW!")
Get-ChildItem .\dir gives you all files contained in the directory .\dir not the directory itself.
That is why the following line in your script creates an array of LastWriteTimes for all files that are contained in the directory that $forDir/$Dir resolves to in your foreach loop:
$dirDate = #(Get-ChildItem $forDir/$Dir | % {$_.LastWriteTime})
The array in $dirDate will return ​System.Object[] when its toString() method is called. This is the reason, why you see this string in your excel, where you expect the folder's timestamp.
I bet that those folders, that seem to work do have exactly one childitem...
To get the LastWriteTime of the directory itself use Get-Item instead of Get-ChildItem.
$dirDate = Get-Item $forDir/$Dir | Select-Object -Expand LastWriteTime
try this...
Get-ChildItem -Path 'D:\temp' -Recurse |
Where-Object { $_.PSIsContainer } |
Select-Object -Property Name, LastWriteTime
<#
# Results
Name LastWriteTime
---- -------------
est 17-Feb-20 15:50:53
LogFiles 11-Mar-20 11:37:28
NewFolder 06-Feb-20 14:56:48
ParentFolder 12-Feb-20 14:24:25
Reference 03-Feb-20 11:55:47
Source 06-Feb-20 14:56:48
Target 24-Feb-20 22:03:56
New folder 03-Feb-20 11:55:24
temp 20-Jan-20 11:17:42
ChildFolder 12-Feb-20 14:08:11
GrandchildFolder 12-Feb-20 14:08:32
#>
# Or in v3 and beyond
Get-ChildItem -Path 'D:\temp' -Directory -Recurse |
Select-Object -Property Name, LastWriteTime
<#
# Results
Name LastWriteTime
---- -------------
est 17-Feb-20 15:50:53
LogFiles 11-Mar-20 11:37:28
NewFolder 06-Feb-20 14:56:48
ParentFolder 12-Feb-20 14:24:25
Reference 03-Feb-20 11:55:47
Source 06-Feb-20 14:56:48
Target 24-Feb-20 22:03:56
New folder 03-Feb-20 11:55:24
temp 20-Jan-20 11:17:42
ChildFolder 12-Feb-20 14:08:11
GrandchildFolder 12-Feb-20 14:08:32
#>
I know this question has already been answered, but for completeness, here's another way of doing this by utilizing the GetFileSystemInfos method every DirInfo object has.
$rootFolder = 'X:\YourRootPath'
Get-ChildItem -Path $rootFolder -Directory -Recurse | ForEach-Object {
# GetFileSystemInfos() (needs .NET 4+) is faster than Get-ChildItem and returns hidden objects by default
# See: https://devblogs.microsoft.com/powershell/why-is-get-childitem-so-slow/
$fsObjects = $_.GetFileSystemInfos('*', 'TopDirectoryOnly') # TopDirectoryOnly --> do not recurse
# you can also use Get-ChildItem here of course.
# To also get hidden files, with Get-ChildItem you need to add the -Force switch
# $fsObjects = Get-ChildItem -Path $_.FullName -Filter '*' -Force
# from the $fsObjects array, filter out the files and directories in order to get the count
$files = $fsObjects | Where-Object { $_ -is [System.IO.FileInfo] } # or: !($_.Attributes -band 'Directory')
$folders = $fsObjects | Where-Object { $_ -is [System.IO.DirectoryInfo] } # or: $_.Attributes -band 'Directory'
# emit a PSObject with all properties you wish to collect
[PsCustomObject]#{
Path = $_.FullName
FileCount = $files.Count
DirCount = $folders.Count
DirSize = "{0:N2} MB" -f (($files | Measure-Object -Sum -Property Length).Sum / 1MB)
DirDate = $_.LastWriteTime
}
} | Export-Csv -Path "X:\YourFolder_Info.csv" -NoTypeInformation -UseCulture

Exporting Object and strings to CSV using Powershell

The purpose of this code is to transfer files from one location to another and to log whether the transfer was a success or a failure.
Everything works except I am having issues with the log. I want the log to be in CSV format and there to be 3 columns: success/failure, from location, and to location. This is outputting the results all into rows with one column.
I've tried the Export-Csv option but that looks for objects/properties so only displays the length(I have strings too). Add-content works but there is only one column. Any suggestions?
#LOCATION OF CSV
$csv = Import-Csv C:\test2.csv
#SPECIFY DATE (EXAMPLE-DELETE FILES > 7 YEARS. 7 YEARS=2555 DAYS SO YOU WOULD ENTER "-2555" BELOW)
$Daysback = "-1"
#FILE DESTINATION
$storagedestination = "C:\Users\mark\Documents\Test2"
#LOG LOCATION
$loglocation = "C:\Users\mark\Documents\filetransferlog.csv"
$s = "SUCCESS"
$f = "FAIL"
$CurrentDate = Get-Date
foreach ($line in $csv) {
$Path = $line | Select-Object -ExpandProperty FullName
$DatetoDelete = $CurrentDate.AddDays($DaysBack)
$objects = Get-ChildItem $Path -Recurse | Select-Object FullName, CreationTime, LastWriteTime, LastAccessTime | Where-Object { $_.LastWriteTime -lt $DatetoDelete }
foreach ($object in $objects) {
try
{
$sourceRoot = $object | Select-Object -ExpandProperty FullName
Copy-Item -Path $sourceRoot -Recurse -Destination $storagedestination
Remove-Item -Path $sourceRoot -Force -Recurse
$temp = $s, $sourceRoot, $storagedestination
$temp | add-content $loglocation
}
catch
{
$temp2 = $f, $sourceRoot, $storagedestination
$temp2 | add-content $loglocation
}
}
}
All your | Select-Object -ExpandProperty are superfluous, simply attach the property name to the variable name => $Path = $line.FullName
Why calculate $DatetoDelete inside the foreach every time?
Output the success/fail to a [PSCustomObject] and gather them in a variable assigned directly to the foreach.
Untested:
$csv = Import-Csv C:\test2.csv
$Daysback = "-1"
$destination = "C:\Users\mark\Documents\Test2"
$loglocation = "C:\Users\mark\Documents\filetransferlog.csv"
$s = "SUCCESS"
$f = "FAIL"
$CurrentDate = Get-Date
$DatetoDelete = $CurrentDate.Date.AddDays($DaysBack)
$Log = foreach ($line in $csv) {
$objects = Get-ChildItem $line.FullName -Rec |
Where-Object LastWriteTime -lt $DatetoDelete
foreach ($object in $objects) {
$Result = $s
$sourceRoot = $object.FullName
try {
Copy-Item -Path $sourceRoot -Recurse -Destination $destination
Remove-Item -Path $sourceRoot -Recurse -Force
} catch {
$Result = $f
}
[PSCustomObject]#{
'Success/Fail' = $Result
Source = $sourceRoot
Destination = $destination
}
}
}
$Log | Export-Csv $loglocation -NoTypeInformation

User Profile Names and Sizes into CSV File

This script gets user profiles on a machine and I am able to provide the output into a text file but I cannot figure out how to export this information to a CSV file.
dir C:\Users | foreach -Begin {} -Process {
$size = (dir $_.FullName -Recurse -Force -EA SilentlyContinue | Measure-Object ‘length’ -Sum -Maximum).Sum
Write-Output ("{0:n2}" -f ($size/1MB) + " MB", $_.FullName) >> "C:\scripts\UserProfiles\UserProfiles.txt"
}
You'll need to create an object with properties and then use Export-Csv
Get-ChildItem C:\Users |
ForEach-Object `
-Begin { Write-Host -Object "Scanning user directories" } `
-Process {
Write-Host "Scanning path '$($_.FullName)'"
$Size = (Get-ChildItem $_.FullName -Recurse -Force -ErrorAction SilentlyContinue | Measure-Object ‘length’ -Sum -Maximum).Sum
[pscustomobject] #{
Name = $_.Name
Path = $_.FullName
Size = '{0:N2} MB' -f ( $Size / 1MB )
}
} |
Export-Csv -Path C:\Scripts\UserProfiles\UserProfiles.csv -NoTypeInformation

Deleting a created file at every iteration

$getFiles = Get-ChildItem -Path $readDir -File -Include "*.doc","*.docx","*.xlsx"-Recurse | %{
if(($_ -match "\.doc$") -or ($_ -match "\.docx$")){
$Doc = $word.Documents.Open($_.fullname)
$nameDoc = $fileSaveLoc + $_.Name.Replace(".docx",".txt").replace(".doc",".txt")
$Doc.saveas([ref] $nameDoc, [ref] 5)
$Doc.close()
if((Get-ChildItem "I:\temp\").length -ne 0){
$locations = (Get-Item "I:\temp\"), (Get-ChildItem "I:\temp\" -Directory -recurse) | % {
Get-ChildItem -File $_.FullName | Select-String -List -Pattern '^\d{3}-?\d{2}-?\d{4}$' |
% Path
}
if($locations -ne $null){
$locations | out-file "I:\temp\SSN_FILES.txt"
#Get-ChildItem "I:\temp\" -exclude "fullpath.txt","SSN_FILES.txt" | remove-item
}else{
Get-ChildItem "I:\temp\" -exclude "fullpath.txt","SSN_FILES.txt" | remove-item
}
}
}
elseif($_ -match "\.xlsx$"){
$workbook = $excel.Workbooks.Open($_.FullName)
$csvFilePath = "I:\temp\" + $_.Name.Replace(".xlsx",".csv")
#$csvFilePath = $_.FullName -replace "\.xlsx$", ".csv"
$workbook.SaveAs($csvFilePath, [Microsoft.Office.Interop.Excel.XlFileFormat]::xlCSV)
$workbook.Close()
if((Get-ChildItem "I:\temp\").length -ne 0){
$locations = (Get-Item "I:\temp\"), (Get-ChildItem "I:\temp\" -Directory -recurse) | % {
Get-ChildItem -File $_.FullName | Select-String -List -Pattern '^\d{3}-?\d{2}-?\d{4}$' |
% Path
}
if($locations -ne $null){
$locations | out-file "I:\temp\SSN_FILES.txt"
#Get-ChildItem "I:\temp\" -exclude "fullpath.txt","SSN_FILES.txt" | remove-item
}else{
Get-ChildItem "I:\temp\" -exclude "fullpath.txt","SSN_FILES.txt" | remove-item
}
}
}
}
So this basically says:
check for a file matching doc/docx/xlsx
convert them into a file that can be parsed through
parse through each file at every iteration and compare it to a regex
if the regex is not null, then output it to a file with the file path
otherwise, delete it and anything else that was created except for two files
restart the process at the next file
Now the problem I am encountering is that the files aren't being deleted. I can't get them to be removed when they are created, when I know they don't match the regex. Setting ($locations -eq $true) doesn't solve that issue because it never goes into the first conditional statement.
The folder should contain only the two files that were created and possibly the ones that match the regex.