I have multiple folders across a number of SQL Servers that contain hundreds/thousands of databases. Each database comprises of three elements:
<dbname>.MDF
<dbname>.LDF
<dbname>files (Folder that contains db files/attachments)
I need to marry these files together and add up their total size, does anyone have any advice on how to do this?
EDIT : Just to clarify, I'm currently able to output the filesizes of the MDF/LDF files, I have a separate script that summarises the folder sizes. I need a method of adding together a .MDF/.LDF/DBFiles folder when their name matches. Bearing in mind all of the files are prefixed with the database name.
EDIT #2: The 2 options given so far sum together the .mdf/.ldf files with no problem, but do not add the folder size of the DBFiles folder. Does anyone have any input on how to amend these scripts to include a folder beginning with the same name.
First provided script:
$root = 'C:\db\folder'
Get-ChildItem "$root\*.mdf" | Select-Object -Expand BaseName |
ForEach-Object {
New-Object -Type PSObject -Property #{
Database = $_
Size = Get-ChildItem "$root\$_*" -Recurse |
Measure-Object Length -Sum |
Select-Object -Expand Sum
}
}
Second provided script:
gci "c:\temp" -file -Include "*.mdf", "*.ldf" -Recurse |
group BaseName, DirectoryName |
%{new-object psobject -Property #{FilesAndPath=$_.Name; Size=($_.Group | gci | Measure-Object Length -Sum).Sum } }
EDIT #3:
Thanks to Ansgar (below), the updated solution has done the trick perfectly. Updating question with final solution:
$root = 'C:\db\folder'
Get-ChildItem "$root\*.mdf" | Select-Object -Expand BaseName |
ForEach-Object {
New-Object -Type PSObject -Property #{
Database = $_
Size = Get-ChildItem "$root\$_*\*" -Recurse |
Measure-Object Length -Sum |
Select-Object -Expand Sum
}
}
Enumerate just the .mdf files from your database folder, then enumerate the files and folders for each basename.
$root = 'C:\db\folder'
Get-ChildItem "$root\*.mdf" | Select-Object -Expand BaseName |
ForEach-Object {
New-Object -Type PSObject -Property #{
Database = $_
Size = Get-ChildItem "$root\$_*\*" -Recurse |
Measure-Object Length -Sum |
Select-Object -Expand Sum
}
}
if you want the sum of sise files database by dir and name file (without extension), try it
gci "c:\temp" -file -Include "*.mdf", "*.ldf" -Recurse |
group BaseName, DirectoryName |
%{new-object psobject -Property #{FilesAndPath=$_.Name; Size=($_.Group | gci | Measure-Object Length -Sum).Sum } }
Modifiy a little the include gci if necessary
Related
Please consider the following directory tree:
root
dir1
dir11
x.L01 12kb
x.L02 10kb
dir12
dir122
a.jpg 5kb
b.xls 3kb
c.bmp 3kb
dir2
a.L01 100kb
a.L02 200kb
a.L03 50kb
dir3
dir31
dir4
There are 3 possible cases:
a (sub)dir is empty; root/dir3/dir31 and root/dir4
a (sub)dir contains (only) L0x files, where x is a number; root/dir1/dir11 and root/dir2
a (sub)dir has files, but not of the L0x-kind
The desired output is a custom directory listing with 3 columns:
filepath
filesize
lefcount (see below)
The logic is as follows:
if a (sub)dir is empty, do not list the dir
if a (sub)dir contains (only) L0x files, only list the first one (root/dir1/dir11/x.L01) but count the number of and total filesize of all L01s
if a (sub)dir has other files, list the dir, but count the number of and total filesize of all files
So the example output would be:
path size count
----------------------------------------
root/dir1/dir11/x.L01 22kb 2
root/dir1/dir12/dir122 11kb 3
root/dir2/a.L01 350kb 3
I'm just beginning powershell, and have come up with the following, which is not much but (a) am I going in the right direction? and (b) how to proceed from here?
Get-ChildItem "C:\root" -Recurse |
Foreach-Object {
If ($_.PSIsContainer) {
Get-ChildItem $_.fullname |
Foreach-Object {
Write-Host $_.fullname
}
}
}
Any help would be greatly appreciated!
This can evolve as your needs change. This will create the desired output as a custom object that you can manipulate and export as required.
$rootPath = "c:\temp"
Get-ChildItem $rootPath -Recurse |
Where-Object {$_.PSIsContainer} |
Where-Object {(Get-ChildItem $_.FullName | Where-Object {!$_.PSIsContainer}| Measure-Object | Select-Object -ExpandProperty Count) -gt 0} |
ForEach-Object{
$files = Get-ChildItem $_.FullName
$props = #{
Path = $_.FullName
Size = "{0:N0} KB" -f (($files | Where-Object {!$_.PSIsContainer} | Measure-Object -Sum Length | Select-Object -ExpandProperty Sum) / 1024)
Count = $files | Measure-Object | Select-Object -ExpandProperty Count
}
If($files.Extension -match "L\d\d"){
# These are special files and we are assuming they are alone in the directory
# Change the path
$props.Path = $files | Where-Object {!$_.PSIsContainer} | Select-Object -First 1 | Select-Object -ExpandProperty FullName
}
New-Object -TypeName PSCustomObject -Property $props
} | Select Path,Size,Count
Get all the folders and files recursively for the $rootPath. Filter out all files and empty folders based on their immediate contents. Then build a custom object with all the requested details. If it turns out the L0X files are present then update the path with the first one found.
Currently I assume that all files are of L0X format. If need be we can confirm.
Sample Output
Path Size Count
---- ---- -----
C:\temp\win64 1,092 KB 2
C:\temp\Empy\Stuff\New Text Document - Copy.L01 0 KB 2
I have a file directory which contains approx. 600 employee image files which have been copied from an alternative source.
The filename format is:
xxxxxx_123456_123_20141212.jpg
When the employee image file is updated it just creates another file in the same location and only the datetime changes at the end.
I need to be able to identify the most recent file, however i need to establish first of all which files are 'duplicated'.
My initial thoughts were to try and match the first 14 characters and, if they matched, work out the recent modified date and then delete the older file.
This requires PowerShell version 3.
$Path = 'C:\Users\madtomvane\Documents\PowerShellTest'
#Get the files #Group them by name #Select the most resent file
$FilesToKeep = Get-ChildItem $Path -Recurse -File | Group-Object -Property {$_.Name[0..14]} | ForEach-Object {$_.Group | Sort-Object -Property LastWriteTime -Descending | Select-Object -First 1}
#Get the files #Group them by name #Where there is more than one file in the group #Select the old ones
$FilesToRemove = Get-ChildItem $Path -Recurse -File | Group-Object -Property {$_.Name[0..14]} | Where-Object {$_.Group.Count -gt 1} | ForEach-Object {$_.Group | Sort-Object -Property LastWriteTime -Descending | Select-Object -Skip 1}
$FilesToRemove | Remove-Item
Here is what I am trying to do:
Search my computer for files ending with a .doc, .docx, .xls, or .xlsx
Output the filenames and sizes (in groups by file extension) to a text file named “File_Summary.txt”.
I also want the total of the number of files and total file size for each file extension listed in the output.
I can't even get past the check folder part:
$Folder_To_Check = C:\AIU
$Report_File_Location = "File_Summary.txt"
$files= Get-Childitem -Path $Folder_To_Check-Include *doc, *docx, *xls, *xlsx $Report_File_Location
$totalfiles = ($files | Measure-Object).Count
$totalsize = ($files | Measure-Object -Sum Length).Sum
Update. Here is my code again with some changes I made from the suggestions, but I'm still coming up empty.
$Report_File_Location = "File_Summary.txt"
$files= Get-Childitem C:\AIU -include "*doc", "*docx", "*xls", "*xlsx"-recurse | Sort-Object | Get-Unique -asString
$files | Out-File $Report_File_Location
$totalfiles = ($files | Measure-Object).Count
$totalsize = ($files | Measure-Object -Sum Length).Sum
write-host "totalfiles: $totalfiles"
write-host "totalsize: $totalsize"
The more I was looking about this I think I shouldn't use the Sort-Object but to use Group Extension -NoElement | Sort Count -Descending that would give me the total number of files for each type?
UPDATE
Thanks to help of people here I got my code to work. But I had an issue where it was saying that my file didn't exist. The problem? I needed to list the entire folder path and use SINGLE quotes.
This code works:
$Folder_To_Check = 'C:\Users\Sarah\Documents\AIU'
$Report_File_Location = "File_Summary.txt"
$results = Get-ChildItem $Folder_To_Check -Include *.doc,*.docx,*.xls,*.xlsx -Recurse
$results | Group-Object extension | ForEach-Object {
[PSCustomObject]#{
Results = $_.Name
Count = $_.Count
Size = [Math]::Round(($_.Group | Measure-Object -Sum Length | Select-Object - ExpandProperty Sum) / 1MB,2)
}
} | Out-File $Report_File_Location -Append
BIG props to Matt for helping me organize my results so nice. Thank you for helping me learn.
$Folder_To_Check = C:\AIU
$Report_File_Location = "File_Summary.txt"
$results = Get-ChildItem $Folder_To_Check -Include *.doc,*.docx,*.xls,*.xlsx -Recurse
$results | Group-Object extension | ForEach-Object {
[PSCustomObject]#{
Extension = $_.Name
Count = $_.Count
Size = [Math]::Round(($_.Group | Measure-Object -Sum Length | Select-Object -ExpandProperty Sum) / 1MB,2)
}
} | Out-File $Report_File_Location -Append
Get all of the files you are looking for with Get-ChildItem much like you were. Vasja mentioned it as well that you might want to use -Recurse to get results from sub directories as well. Use Group-Object to collect the files by extension. For each collection output a custom object of the extension and file count, which both come Group-Object, and the size of all the files of that particular extension converted to MB and rounded to 2 decimal places.
Update for 2.0
In case you only have 2.0 installed I wanted to provide and answer that works for that.
$results | Group-Object extension | ForEach-Object {
$properties = #{
Extension = $_.Name
Count = $_.Count
Size = [Math]::Round(($_.Group | Measure-Object -Sum Length | Select-Object -ExpandProperty Sum) / 1MB,2)
}
New-Object -TypeName PSObject -Property $properties
}
Added some quotes.
Also you probably want -Recurse on Get-Childitem
$Folder_To_Check = "C:\AIU"
$Report_File_Location = "E:\tmp\File_Summary.txt"
$files = Get-Childitem -Path $Folder_To_Check -Include *doc, *docx, *xls, *xlsx -Recurse
$files | Out-File $Report_File_Location
$totalfiles = ($files | Measure-Object).Count
$totalsize = ($files | Measure-Object -Sum Length).Sum
write-host "totalfiles: $totalfiles"
write-host "totalsize: $totalsize"
Yep, you need a collection of strings for the -Include argument. So, what you tried is one string, that being:
"*doc, *docx, *xls, *xlsx"
While the commas do need to seperate the extensions when you include it within the quotes it thinks that's a part of the one thing to include, so it's seriously looking for files that have anything (as per the asterisk) then "doc," then anything then "docx," then anything then... you see where I'm going. It thinks it has to include all of that. Instead you need a collection of strings like:
-Include "*doc","*docx","*xls","xlsx"
I hope that helps. Here's your line modified to what should work:
$files= Get-Childitem -Path $Folder_To_Check-Include "*doc", "*docx", "*xls", "*xlsx"
1)How to create a list of attributes of files with cmdlet get-member and then sort it by last write time?
2)Find total size of files with different extension(for examp total size for all *.html files)
I think the solution for the first task(second task is ok) should be like this(however it doesn't work)
$a=get-childitem . -filter *.html
$n=$a.Length
do{
$isnotsorted=0
for($i=0;$i -lt ($n-1); $i++) {
if ((get-member $a[$i]).LastWriteTime -lt (get-member $a[$i]).LastWRiteTime){
$a[$i],$a[$i+1]=`
$a[$i+1],$a[$i]
$isnotsorted=$i+1
}
}
$n=$isnotsorted
}
until ($n -eq 0)
$a
You don't need to use Get-Member to do this. You can use Sort-Object and Select-Object:
dir C:\ -Force | ? {!$_.PsIsContainer} | Sort LastWriteTime | Select FullName, Attributes
You can use Group-Object and Measure-Object to do this.
((dir D:\Software -Force -Filter *.html | Group Extension).Group | Measure-Object -Sum Length).Sum / 1MB
I'm not sure why you don't want to use Sort-Object -Property LastWriteTime but here is how you would fix your bubble sort code. Remember Get-Member is not the right cmdlet to use to access a properties value.
$a = get-childitem -filter *.html
$n = $a.Length
do {
$isnotsorted = 0
for($i = 0; $i -lt ($n-1); $i++) {
if ( ($a[$i]).LastWriteTime -lt ($a[$i + 1]).LastWRiteTime ) {
$a[$i] , $a[$i+1] = $a[$i+1] , $a[$i]
$isnotsorted = $i + 1
}
}
$n = $isnotsorted
} until ($n -eq 0)
$a
Another thing to note here is that the performance of this algorithm is much worse than just using Sort-Object. My music folder has 1355 files and the above finishes in 83 seconds. Using Sort-Object finishes in 1.7 seconds.
Measure-Command {
get-childitem D:\shares\Music -rec -filter *.m4a | Sort-Object LastWriteTime
}
You don't need Get-Member to display the attributes of files. Just use Get-ChildItem to get the contents of a directory and then pipe them to Sort-Object:
Get-ChildItem -Path $path | Sort-Object -Property 'LastWriteTime'
You can add the -Recurse parameter to Get-ChildItem to list child directories, and add -Force to list files with the Hidden attribute. You can pipe all of this to a Format-* cmdlet if you want to display properties other than those displayed by the standard formatting for files and directories:
Get-ChildItem -Path $path `
| Sort-Object -Property 'LastWriteTime' `
| Format-Table -Property #('Attributes', 'FullName', 'CreationTime')
Get-Member can be used to determine which properties exist on a file or directory object.
You can use Measure-Object with the -Sum switch to add up the Length property of a collection of files:
$htmlFiles = Get-ChildItem -Path $path -Filter '*.html';
$measurement = $htmlFiles | Measure-Object -Property 'Length' -Sum;
$totalHtmlSize = $measurement.Sum;
To generate a table of the total size of each file type you can do something like this:
Get-ChildItem -Path $path `
| Where-Object { $_ -is [IO.FileInfo]; } `
| Group-Object -Property 'Extension' `
| Select-Object `
#{ Name = 'Extension'; Expression = 'Name' }, `
#{ Name = 'TotalSize'; Expression = { `
($_.Group | Measure-Object -Property 'Length' -Sum).Sum } `
} `
| Sort-Object -Property 'Extension';
That retrieves the contents of $path, filters it to only include files, groups the files by the Extension property, projects each group into an object with a property for the extension and a property for the total file size, then sorts the results by extension.
How can I list size of each folder in a directory by the sum of all files in each folder/subfolders?
My latest attempt:
ls | foreach-object { select-object Name, #{Name = "Size"; Expression = { ls $_ -recurse | measure-object -property length -sum } }
I've made other attempts but nothing successful yet. Any suggestions or solutions are very welcome. I feel like I'm missing something very obvious.
The output should look as follows:
Name Size
And it should list each folder in the root folder and the size of the folder counting subfolders of that folder.
I was able to resolve the issue with the following:
param([String]$path)
ls $path | Add-Member -Force -Passthru -Type ScriptProperty -Name Size -Value {
ls $path\$this -recurse | Measure -Sum Length | Select-Object -Expand Sum } |
Select-Object Name, #{Name = "Size(MB)"; Expression = {"{0:N0}" -f ($_.Size / 1Mb)} } | sort "Size(MB)" -descending
I think you've basically got it, honestly.
You could be a bit more elegant by using Add-Member:
ls | Add-Member -Force -Passthru -Type ScriptProperty -Name Length -Value {
ls $this -recurse | Measure -Sum Length | Select -Expand Sum }
PSCX messes with the formatting and will output "" for the size even though you've actually got a size. If you're using PSCX you'll have to add an explicit | Format-Table Mode, LastWriteTime, Length, Name -Auto
It's not particularly elegant but should get the job done:
gci . -force | ?{$_.PSIsContainer} |
%{$res=#{};$res.Name=$_.Name; $res.Size = (gci $_ -r | ?{!$_.PSIsContainer} |
measure Length -sum).Sum; new-object psobject -prop $res}
Note the use of -Force to make sure you're summing up hidden files. Also note the aliases I have used (convenient when typing interactively). There's ? for Where-Object and % for Foreach-Object. Saves the wrists. :-)
Here is a handy Powershell example script that may be adapted to fit what you are looking for.