PowerShell Script to Validate/compare Folder and files of Two servers - powershell

I am trying to get the number of files inside folders and number of folders along with total size. To a certain extent I have achieved that goal but when I am trying it for checking millions of folders/files and their size may be around 1 TB its getting stuck might be getting into infinite lope.
Also I want to compare the two location size and number of files/folder and get output as bollen value. – true if the size and numbers match, false if they don’t.
Below script is what I tried. Can some one help me with the requirement.
$hostnames = Get-Content C:\server.txt
$directory = "D$\logs"
foreach ($hostname in $hostnames){
$Folderitems = (Get-ChildItem -Path "\\$hostname\$directory" -Directory -Recurse -Force).Count
$FileItems = Get-ChildItem -path "\\$hostname\$directory" -recurse
$Measurement = $FileItems | Measure-Object -property length -sum
$colitems = $FileItems | measure-Object -property length -sum
"$hostname,$directory;{0:N2}" -f ($colitems.sum / 1024MB) + "GB;" + $Measurement.count + " files;" + $items +"Folders;"
}

A performance improvement would be to invoke the Get-ChildItem cmdlet only once and use it to filter the Folders using the Where-Object cmdlet:
$FileItems = Get-ChildItem -path "\\$hostname\$directory" -recurse
$Folderitems = $FileItems | Where-Object PsIsContainer
You also calculate the length twice, remove one of these:
$Measurement = $FileItems | Measure-Object -property length -sum
$colitems = $FileItems | measure-Object -property length -sum
Also i want to compare the two location size and number of
files/folder and get output as bollen value.
Which two locations you want to compare? You are iterating over a list of hostnames and don't store any values...
Edit to your comment:
Regarding the comparison part i want to compare the size and number of
files from server1 to server2 for location d$\website. Its just like
same we do right click on folder and see the files and size.
Currently, you are retrieving the items for D$\logs, not D$\website you probably have to change this?!. To compare the results, you have to store them first. I would recommend an array of PSCustomObjects. The structure looks like this (sample output):
Server Sum Count
------ --- -----
YourServer1 88575206 386
YourServer2 88575206 386
And here the script:
foreach ($hostname in $hostnames){
$FileItems = Get-ChildItem -path "\\$hostname\$directory" -recurse
$Folderitems = $FileItems | Where-Object PsIsContainer
$Measurement = $FileItems | Measure-Object -property length -sum
$serverFileMeasurements += [PSCustomObject]#{
Server = $hostname
Sum = $Measurement.Sum
Count = $Measurement.Count
}
}
# assuming there are only two servers:
if ($serverFileMeasurements[0].Count -eq $serverFileMeasurements[1].Count -and
$serverFileMeasurements[0].Sum -eq $serverFileMeasurements[1].Sum)
{
Write-Host $true
}
else
{
Write-Host $false
}

Related

Creating own object powershell

I ve got a script which get each folder and get name,filescount,size of each folder.
Size with measure-object doesn t work.
My first try using my own object to handle this([pscustomobject]).
May I integrate a command (measure-object) in an object ?
Get-ChildItem d:\ -Directory -Recurse -Depth 2 -ErrorAction SilentlyContinue|
ForEach-Object{
[pscustomobject]#{
FullName = $_.Fullname
FileCount = $_.GetFiles().Count
size=measure-object -sum -property length
}
} | sort -Property filecount -Descending
Thks ,)
Unfortunately folders don't actually have a size (Windows is just kind enough to find out for us when we check the properties of it)
So in your script you need to get all child items of the current iterations folder and measure their combined size.
Get-ChildItem d:\ -Directory -Recurse -Depth 2 -ErrorAction SilentlyContinue |
ForEach-Object {
[pscustomobject]#{
FullName = $_.Fullname
FileCount = $_.GetFiles().Count
size = (Get-Childitem -Path $_.Fullname -recurse | measure-object -property length -sum).sum
}
} | sort -Property filecount -Descending
You will be using the result from .GetFiles() twice, first for getting the total file count and the second time to get the sum of the file's Length, hence I would advise you to store the result of that method call in a variable for further manipulation.
To get the folder size, you can either use Enumerable.Sum:
Get-ChildItem D:\ -Directory -Recurse -Depth 2 -ErrorAction SilentlyContinue | & {
process {
$files = $_.GetFiles()
[pscustomobject]#{
FullName = $_.FullName
FileCount = $files.Count
Size = [Linq.Enumerable]::Sum([int64[]] $files.ForEach('Length'))
}
}
} | Sort-Object -Property Filecount -Descending
Or, if you want to use Measure-Object, the Size property would be:
Size = ($files | Measure-Object Length -Sum).Sum
Lastly, if you want to do a recursive search per folder, you can target the .GetFiles(String, SearchOption) overload using AllDirectories for SearchOption Enum:
$files = $_.GetFiles('*', [IO.SearchOption]::AllDirectories)

Find share sizes across multiple servers -powershell

I have about 1600 old home drives that I need to capture the folder sizes on. They are spread across 12 NAS Appliances and I need to get the size of each folder. I see this asked a lot but they are always asking for all the shares on a single server.
I have used simple one-liners like the the one below which work great.
Get-ChildItem \\Server1\Users\User1 -file -recurse | Measure-Object -Sum Length
How can I feed in a txt or csv from multiple locations?
Like
\\Server1\Users\User1
\\Server2\Users\User1
\\Server3\Users\User1
\\Server4\Users\User1
\\Server5\Users\User1
\\Server6\Users\User1
\\Server7\Users\User1
etc...
Exporting to a csv would be nice as well.
thanks!!
You can use Get-Content to read your text file line-by-line:
Get-Content .\path\to\file.txt
Next up we need to repeat your one-liner for each line:
Get-Content .\path\to\file.txt |ForEach-Object {
Get-ChildItem $_ -File -Recurse | Measure-Object -Sum Length
}
This will give us 1600 individual measurements - but now we won't know which paths they belong to...
To track which measurement belongs to which user directory, let's create a new object containing both all the information instead:
Get-Content .\path\to\file.txt |ForEach-Object {
$Measurement = Get-ChildItem $_ -File -Recurse | Measure-Object -Sum Length
[pscustomobject]#{
Path = $_
FileCount = $Measurement.Count
TotalSize = $Measurement.Sum
}
}
Finally, we can export to CSV using Export-Csv:
Get-Content .\path\to\file.txt |ForEach-Object {
$Measurement = Get-ChildItem $_ -File -Recurse | Measure-Object -Sum Length
[pscustomobject]#{
Path = $_
FileCount = $Measurement.Count
TotalSize = $Measurement.Sum
}
} |Export-Csv .\path\to\output.csv -NoTypeInformation

Add a function to return folder size in Powershell [duplicate]

I have found several resources that use the following script to get folder sizes
$colItems = (Get-ChildItem $startFolder -recurse | Where-Object {$_.PSIsContainer -eq $True} | Sort-Object)
foreach ($i in $colItems)
{
$subFolderItems = (Get-ChildItem $i.FullName | Measure-Object -property length -sum)
$i.FullName + " -- " + "{0:N2}" -f ($subFolderItems.sum / 1MB) + " MB"
}
The problem with that is it also lists the subdirectories ie:
c:\test\1 -- 10mb
c:\test\1\folder -- 10mb
c:\test\1\folder\deep -- 5mb
c:\test\1\folder\tuna -- 5mb
c:\test\2 -- 20bm
c:\test\2\folder -- 20mb
c:\test\2\folder\deep -- 10mb
c:\test\2\folder\tuna -- 10mb
I think you know see where I am going. What I am looking for is just the parent folder's results... SO:
c:\test\1 -- 10mb
c:\test\2 -- 20mb
How can this be accomplished with Powershell?
....
You need to get the total contents size of each directory recursively to output. Also, you need to specify that the contents you're grabbing to measure are not directories, or you risk errors (as directories do not have a Length parameter).
Here's your script modified for the output you're looking for:
$colItems = Get-ChildItem $startFolder | Where-Object {$_.PSIsContainer -eq $true} | Sort-Object
foreach ($i in $colItems)
{
$subFolderItems = Get-ChildItem $i.FullName -recurse -force | Where-Object {$_.PSIsContainer -eq $false} | Measure-Object -property Length -sum | Select-Object Sum
$i.FullName + " -- " + "{0:N2}" -f ($subFolderItems.sum / 1MB) + " MB"
}
This simple solution worked for me as well.
Get-ChildItem -Recurse 'directory_path' | Measure-Object -Property Length -Sum
The solution posted by #Linga:
"Get-ChildItem -Recurse 'directory_path' | Measure-Object -Property Length -Sum" is nice and short. However, it only computes the size of 'directory_path', without sub-directories.
Here is a simple solution for listing all sub-directory sizes. With a little pretty-printing added.
(Note: use the -File option to avoid errors for empty sub-directories)
foreach ($d in gci -Directory -Force) {
'{0,15:N0}' -f ((gci $d -File -Recurse -Force | measure length -sum).sum) + "`t`t$d"
}
Sorry to reanimate a dead thread, but I have just been dealing with this myself, and after finding all sorts of crazy bloated solutions, I managed to come up with this.
[Long]$actualSize = 0
foreach ($item in (Get-ChildItem $path -recurse | Where {-not $_.PSIsContainer} | ForEach-Object {$_.FullName})) {
$actualSize += (Get-Item $item).length
}
Quickly and in few lines of code gives me a folder size in Bytes, than can easily be converted to any units you want with / 1MB or the like.
Am I missing something? Compared to this overwrought mess it seems rather simple and to the point. Not to mention that code doesn't even work since the called function is not the same name as the defined function. And has been wrong for 6 years. ;)
So, any reasons NOT to use this stripped down approach?
This is similar to https://stackoverflow.com/users/3396598/kohlbrr answer, but I was trying to get the total size of a single folder and found that the script doesn't count the files in the Root of the folder you are searching. This worked for me.
$startFolder = "C:\Users";
$totalSize = 0;
$colItems = Get-ChildItem $startFolder
foreach ($i in $colItems)
{
$subFolderItems = Get-ChildItem $i.FullName -recurse -force | Where-Object {$_.PSIsContainer -eq $false} | Measure-Object -property Length -sum | Select-Object Sum
$totalSize = $totalSize + $subFolderItems.sum / 1MB
}
$startFolder + " | " + "{0:N2}" -f ($totalSize) + " MB"
This is something I wind up looking for repeatedly, even though I wrote myself a nice little function a while ago. So, I figured others might benefit from having it and maybe I'll even find it here, myself. hahaha
It's pretty simple to paste into your script and use. Just pass it a folder object.
I think it requires PowerShell 3 just because of the -directory flag on the Get-ChildItem command, but I'm sure it can be easily adapted, if need be.
function Get-TreeSize ($folder = $null)
{
#Function to get recursive folder size
$result = #()
$folderResult = "" | Select-Object FolderPath, FolderName, SizeKB, SizeMB, SizeGB, OverThreshold
$contents = Get-ChildItem $folder.FullName -recurse -force -erroraction SilentlyContinue -Include * | Where-Object {$_.psiscontainer -eq $false} | Measure-Object -Property length -sum | Select-Object sum
$sizeKB = [math]::Round($contents.sum / 1000,3) #.ToString("#.##")
$sizeMB = [math]::Round($contents.sum / 1000000,3) #.ToString("#.##")
$sizeGB = [math]::Round($contents.sum / 1000000000,3) #.ToString("#.###")
$folderResult.FolderPath = $folder.FullName
$folderResult.FolderName = $folder.BaseName
$folderResult.SizeKB = $sizeKB
$folderresult.SizeMB = $sizeMB
$folderresult.SizeGB = $sizeGB
$result += $folderResult
return $result
}
#Use the function like this for a single directory
$topDir = get-item "C:\test"
Get-TreeSize ($topDir)
#Use the function like this for all top level folders within a direcotry
#$topDir = gci -directory "\\server\share\folder"
$topDir = Get-ChildItem -directory "C:\test"
foreach ($folderPath in $topDir) {Get-TreeSize $folderPath}
My proposal:
$dir="C:\temp\"
get-childitem $dir -file -Rec | group Directory | where Name -eq $dir | select Name, #{N='Size';E={(($_.Group.Length | measure -Sum).Sum / 1MB)}}
from sysinternals.com with du.exe or du64.exe -l 1 .
or 2 levels down:
**du -l 2 c:**
Much shorter than Linux though ;)
At the answer from #squicc if you amend this line: $topDir = Get-ChildItem -directory "C:\test" with -force then you will be able to see the hidden directories also. Without this, the size will be different when you run the solution from inside or outside the folder.
I used the great answer of #kohlbrr and improved it a bit. I also turned it into a function you can put in your $PROFILE. This outputs three properties instead of just text: Name, Size (which is a string output of the size converted to MB), and Value which is the raw size value you can use with | sort value
function Get-DirectorySize([string] $rootFolder) {
$colItems = Get-ChildItem $rootFolder | Where-Object {$_.PSIsContainer -eq $true} | Sort-Object
foreach ($i in $colItems) {
$subFolderItems = Get-ChildItem $i.FullName -recurse -force | Where-Object {$_.PSIsContainer -eq $false} | Measure-Object -property Length -sum | Select-Object Sum
[PSCustomObject]#{ Name=$i.Fullname; Size="{0:N2}" -f ($subFolderItems.sum / 1MB) + " MB"; Value=$subFolderItems.sum }
}
}
The following script provides file sizes from a directory(not going deep) and sizes in bytes.(give your path in $($args[0]) in both the scripts)
Get-ChildItem -Path "$($args[0])" -Recurse -Depth 0 -file| select Length,LastWriteTime,FullName | Export-Csv .\FileAndFolderSizes.csv -NoTypeInformation
The following script provides the folder sizes(depth 0 ) in bytes within the given directory.
$array= #()
Get-ChildItem -Path "$($args[0])" -Recurse -Depth 0 | Where-Object { $_.PSIsContainer } |
ForEach-Object {
$obj = New-Object PSObject
$Size = [Math]::Round((Get-ChildItem -Recurse $_.FullName | Measure-Object Length -Sum -ErrorAction SilentlyContinue).Sum, 2)
$obj |Add-Member -MemberType NoteProperty -Name "FullName" $_.FullName
$obj |Add-Member -MemberType NoteProperty -Name "LastWriteTime" $_.LastWriteTime
$obj |Add-Member -MemberType NoteProperty -Name "Length" $Size
$array +=$obj
}
$array | select Length,LastWriteTime,FullName | Export-Csv .\FileAndFolderSizes.csv -NoTypeInformation -Append
Output is as follows
"Length","LastWriteTime","FullName"
"593408","2/17/2022 10:51:01 PM","C:\a_appli\Partners_Ref\Client64_Rws2\compresslayer.dll"
"286720","2/9/2021 11:52:18 PM","C:\a_appli\Partners_Ref\Client64_Rws2\fdsrtd.dll"
"589312","6/19/2019 10:18:41 PM","C:\a_appli\Partners_Ref\Client64_Rws2\FdswCli.dll"
"13658276","3/18/2022 9:52:16 AM","C:\a_appli\Partners_Ref\Client64_Rws2\PartnersTemplateBuilder"
Interesting how powerful yet how helpless PS can be in the same time, coming from a Nix learning PS. after install crgwin/gitbash, you can do any combination in one commands:
size of current folder:
du -sk .
size of all files and folders under current directory
du -sk *
size of all subfolders (including current folders)
find ./ -type d -exec du -sk {} \;

Sum of file folder size based on file/folder name

I have multiple folders across a number of SQL Servers that contain hundreds/thousands of databases. Each database comprises of three elements:
<dbname>.MDF
<dbname>.LDF
<dbname>files (Folder that contains db files/attachments)
I need to marry these files together and add up their total size, does anyone have any advice on how to do this?
EDIT : Just to clarify, I'm currently able to output the filesizes of the MDF/LDF files, I have a separate script that summarises the folder sizes. I need a method of adding together a .MDF/.LDF/DBFiles folder when their name matches. Bearing in mind all of the files are prefixed with the database name.
EDIT #2: The 2 options given so far sum together the .mdf/.ldf files with no problem, but do not add the folder size of the DBFiles folder. Does anyone have any input on how to amend these scripts to include a folder beginning with the same name.
First provided script:
$root = 'C:\db\folder'
Get-ChildItem "$root\*.mdf" | Select-Object -Expand BaseName |
ForEach-Object {
New-Object -Type PSObject -Property #{
Database = $_
Size = Get-ChildItem "$root\$_*" -Recurse |
Measure-Object Length -Sum |
Select-Object -Expand Sum
}
}
Second provided script:
gci "c:\temp" -file -Include "*.mdf", "*.ldf" -Recurse |
group BaseName, DirectoryName |
%{new-object psobject -Property #{FilesAndPath=$_.Name; Size=($_.Group | gci | Measure-Object Length -Sum).Sum } }
EDIT #3:
Thanks to Ansgar (below), the updated solution has done the trick perfectly. Updating question with final solution:
$root = 'C:\db\folder'
Get-ChildItem "$root\*.mdf" | Select-Object -Expand BaseName |
ForEach-Object {
New-Object -Type PSObject -Property #{
Database = $_
Size = Get-ChildItem "$root\$_*\*" -Recurse |
Measure-Object Length -Sum |
Select-Object -Expand Sum
}
}
Enumerate just the .mdf files from your database folder, then enumerate the files and folders for each basename.
$root = 'C:\db\folder'
Get-ChildItem "$root\*.mdf" | Select-Object -Expand BaseName |
ForEach-Object {
New-Object -Type PSObject -Property #{
Database = $_
Size = Get-ChildItem "$root\$_*\*" -Recurse |
Measure-Object Length -Sum |
Select-Object -Expand Sum
}
}
if you want the sum of sise files database by dir and name file (without extension), try it
gci "c:\temp" -file -Include "*.mdf", "*.ldf" -Recurse |
group BaseName, DirectoryName |
%{new-object psobject -Property #{FilesAndPath=$_.Name; Size=($_.Group | gci | Measure-Object Length -Sum).Sum } }
Modifiy a little the include gci if necessary

Powershell Search

Here is what I am trying to do:
Search my computer for files ending with a .doc, .docx, .xls, or .xlsx
Output the filenames and sizes (in groups by file extension) to a text file named “File_Summary.txt”.
I also want the total of the number of files and total file size for each file extension listed in the output.
I can't even get past the check folder part:
$Folder_To_Check = C:\AIU
$Report_File_Location = "File_Summary.txt"
$files= Get-Childitem -Path $Folder_To_Check-Include *doc, *docx, *xls, *xlsx $Report_File_Location
$totalfiles = ($files | Measure-Object).Count
$totalsize = ($files | Measure-Object -Sum Length).Sum
Update. Here is my code again with some changes I made from the suggestions, but I'm still coming up empty.
$Report_File_Location = "File_Summary.txt"
$files= Get-Childitem C:\AIU -include "*doc", "*docx", "*xls", "*xlsx"-recurse | Sort-Object | Get-Unique -asString
$files | Out-File $Report_File_Location
$totalfiles = ($files | Measure-Object).Count
$totalsize = ($files | Measure-Object -Sum Length).Sum
write-host "totalfiles: $totalfiles"
write-host "totalsize: $totalsize"
The more I was looking about this I think I shouldn't use the Sort-Object but to use Group Extension -NoElement | Sort Count -Descending that would give me the total number of files for each type?
UPDATE
Thanks to help of people here I got my code to work. But I had an issue where it was saying that my file didn't exist. The problem? I needed to list the entire folder path and use SINGLE quotes.
This code works:
$Folder_To_Check = 'C:\Users\Sarah\Documents\AIU'
$Report_File_Location = "File_Summary.txt"
$results = Get-ChildItem $Folder_To_Check -Include *.doc,*.docx,*.xls,*.xlsx -Recurse
$results | Group-Object extension | ForEach-Object {
[PSCustomObject]#{
Results = $_.Name
Count = $_.Count
Size = [Math]::Round(($_.Group | Measure-Object -Sum Length | Select-Object - ExpandProperty Sum) / 1MB,2)
}
} | Out-File $Report_File_Location -Append
BIG props to Matt for helping me organize my results so nice. Thank you for helping me learn.
$Folder_To_Check = C:\AIU
$Report_File_Location = "File_Summary.txt"
$results = Get-ChildItem $Folder_To_Check -Include *.doc,*.docx,*.xls,*.xlsx -Recurse
$results | Group-Object extension | ForEach-Object {
[PSCustomObject]#{
Extension = $_.Name
Count = $_.Count
Size = [Math]::Round(($_.Group | Measure-Object -Sum Length | Select-Object -ExpandProperty Sum) / 1MB,2)
}
} | Out-File $Report_File_Location -Append
Get all of the files you are looking for with Get-ChildItem much like you were. Vasja mentioned it as well that you might want to use -Recurse to get results from sub directories as well. Use Group-Object to collect the files by extension. For each collection output a custom object of the extension and file count, which both come Group-Object, and the size of all the files of that particular extension converted to MB and rounded to 2 decimal places.
Update for 2.0
In case you only have 2.0 installed I wanted to provide and answer that works for that.
$results | Group-Object extension | ForEach-Object {
$properties = #{
Extension = $_.Name
Count = $_.Count
Size = [Math]::Round(($_.Group | Measure-Object -Sum Length | Select-Object -ExpandProperty Sum) / 1MB,2)
}
New-Object -TypeName PSObject -Property $properties
}
Added some quotes.
Also you probably want -Recurse on Get-Childitem
$Folder_To_Check = "C:\AIU"
$Report_File_Location = "E:\tmp\File_Summary.txt"
$files = Get-Childitem -Path $Folder_To_Check -Include *doc, *docx, *xls, *xlsx -Recurse
$files | Out-File $Report_File_Location
$totalfiles = ($files | Measure-Object).Count
$totalsize = ($files | Measure-Object -Sum Length).Sum
write-host "totalfiles: $totalfiles"
write-host "totalsize: $totalsize"
Yep, you need a collection of strings for the -Include argument. So, what you tried is one string, that being:
"*doc, *docx, *xls, *xlsx"
While the commas do need to seperate the extensions when you include it within the quotes it thinks that's a part of the one thing to include, so it's seriously looking for files that have anything (as per the asterisk) then "doc," then anything then "docx," then anything then... you see where I'm going. It thinks it has to include all of that. Instead you need a collection of strings like:
-Include "*doc","*docx","*xls","xlsx"
I hope that helps. Here's your line modified to what should work:
$files= Get-Childitem -Path $Folder_To_Check-Include "*doc", "*docx", "*xls", "*xlsx"