Improving the speed of Get-FileMetaData - powershell

I'm currently using the below script taken from scriptingguys.com (all credit to them, I just added the bottom 2 lines.) That takes a directory and pulls the file path and comments field from the meta data of the files. Currently the script take's a little over 1.5 minutes to fully run. Is there anyway to speed this up or use a different method to get this data?
I am using this script at the start of some software I have written and 1.5+ minutes is too long for the script to complete. Any thoughts/comments?
Function Get-FileMetaData
{
Param([string[]]$folder)
foreach($sFolder in $folder)
{
$a = 0
$objShell = New-Object -ComObject Shell.Application
$objFolder = $objShell.namespace($sFolder)
foreach ($File in $objFolder.items())
{
$FileMetaData = New-Object PSOBJECT
for ($a ; $a -le 266; $a++)
{
if($objFolder.getDetailsOf($File, $a))
{
$hash += #{$($objFolder.getDetailsOf($objFolder.items, $a)) =
$($objFolder.getDetailsOf($File, $a)) }
$FileMetaData | Add-Member $hash
$hash.clear()
} #end if
} #end for
$a=0
$FileMetaData
} #end foreach $file
} #end foreach $sfolder
} #end Get-FileMetaData
$fileMetaData = Get-FileMetaData -folder "C:\Pics" | select 'Name', 'Path', 'Comments' | Sort-Object 'Name'
$fileMetaData | select 'Name', 'Path', 'Comments' | Export-CSV "C:\SCRIPTS\TestDirectory.txt" -encoding Utf8 -NoTypeInformation
Solved by wOxxOm, thanks for your help! Running the below and now working.
Function Get-FileMetaData(
[string[]]$folders,
[string[]]$properties
) {
$shellApp = New-Object -ComObject Shell.Application
$supportsOrdered = $PSVersionTable.PSVersion.Major -ge 3
$hash = if ($supportsOrdered) { [ordered]#{} } else { #{} }
# walk the folders and get the properties by index found above
$folders | ForEach {
$shellFolder = $shellApp.namespace($_)
# get all headers and find their indexes
$allProps = #{}
foreach ($index in 0..266) {
$allProps[$shellFolder.getDetailsOf($shellFolder.items, $index)] = $index
}
$shellFolder.items() | ForEach {
$file = $_
$hash.Clear()
foreach ($prop in $properties) {
if (($index = $allProps[$prop]) -ne $null) {
if ($value = $shellFolder.getDetailsOf($file, $index)) {
$hash[$prop] = $value
}
}
}
if ($supportsOrdered) {
[PSCustomObject]$hash
} else {
Select $properties -inputObject (
New-Object PSObject -Property $hash
)
}
}
}
}
Get-FileMetaData -folders 'C:\PICS' -properties Name, Path, Comments | Sort-Object Name |
select Name, Path, Comments | Export-Csv 'C:\Scripts\test.txt' -encoding UTF8 -NoTypeInformation

getDetailsOf is slow, and your code needlessly invokes it 267 times for each file when you only need it for 3 properties.
Collect the property names just once at the start of the function, don't do it on every file
Add-Member is slow. Don't invoke it on every property. Collect all found properties in a hashtable and pass it once to Add-Member or, since you create an empty object, directly to New-Object. To enforce the order of properties use Select-Object in PowerShell 2. Note, PowerShell 3.0 and newer support [ordered] and [PSCustomObject] typecast (see the code below).
Use pipelining instead of foreach statements so that the results appear immediately
Files are already sorted by name, at least on NTFS file system in Windows, so no need to sort.
Function Get-FileMetaData(
[string[]]$folders,
[string[]]$properties
) {
$shellApp = New-Object -ComObject Shell.Application
# get all headers and find their indexes
$shellFolder = $shellApp.namespace($folders[0])
$allProps = #{}
foreach ($index in 0..266) {
$allProps[$shellFolder.getDetailsOf($shellFolder.items, $index)] = $index
}
$supportsOrdered = $PSVersionTable.PSVersion.Major -ge 3
$hash = if ($supportsOrdered) { [ordered]#{} } else { #{} }
# walk the folders and get the properties by index found above
$folders | ForEach {
$shellFolder = $shellApp.namespace($_)
$shellFolder.items() | ForEach {
$file = $_
$hash.Clear()
foreach ($prop in $properties) {
if (($index = $allProps[$prop]) -ne $null) {
$hash[$prop] = $shellFolder.getDetailsOf($file, $index)
}
}
if ($supportsOrdered) {
[PSCustomObject]$hash
} else {
Select $properties -inputObject (
New-Object PSObject -Property $hash
)
}
}
}
}
Usage example 1:
Get-FileMetaData -folders 'r:\folder1', 'r:\folder2' -properties Name, Path, Comments
Usage example 2:
Get-FileMetaData -folders 'r:\folder1', 'r:\folder2' -properties Name, Path, Comments |
Export-Csv r:\results.csv -encoding UTF8 -NoTypeInformation
Usage example 3 gets all properties, which is slow:
Get-FileMetaData -folders 'r:\folder1', 'r:\folder2'

Related

dynamically creating key/value of an object and exporting to CSV

After getting a search result from an LDAP Server, i need to create a pscustomobject dynamically.
The Problem here is that some of the attributes are not set for all users.
this is why i cannot create the pscustomobject the traditional way.
Name = $($item.Attributes['givenname'].GetValues('string'))
Surname = $($item.Attributes['sn'].GetValues('string'))
The Attribute Name does not exist for all users and doing this throws an error.
How can i create the pscustomobject in this case where i need to add both key and value dynamically.
Here is what i have so far:
$vals="cn","tel","email","sn","givenname","ou"
$c.Bind()
$r = New-Object System.DirectoryServices.Protocols.SearchRequest -ArgumentList $baseDN,$Filter,$scope,$attrlist
$re = $c.SendRequest($r)
foreach ($item in $re.Entries) {
foreach($attr in $vals){
if($item.Attributes.Keys -contains $attr){
$pskeys += $attr
}}
foreach($pskey in $pskeys){
$data += [pscustomobject]#{
$($pskey) = $($item.Attributes[$pskey].GetValues('string'))
}}
$pskeys = #()
}
givenname does not exist for all the users and this is why the pscustombject must be created dynamically.
I cannot use a HashTable or some kind of a List as duplicate values must be allowed. There are cases where the attributes sn and givenname are equal.
After hours of trying and failing i can only hope for the Wizards of Stackoverflow to show me how this can be achieved.
I need a pscustomobject where i can save the available attributes and skip the missing attributes dynamically. Is there a way to do this?
Regards
Try following :
$table = [System.Collections.ArrayList]::new()
foreach ($item in $re.Entries) {
$newRow = New-Object -TypeName psobject
foreach($attr in $vals){
if($item.Attributes.Keys -contains $attr){
$pskeys += $attr
}}
foreach($pskey in $pskeys){
foreach($item in $item[$pskey].Attributes.GetValues('string'))
{
$newRow | Add-Member -NotePropertyName $item.Name -NotePropertyValue $item.Value
}
}
$table.Add($newRow) | Out-Null
}
$table | Format-Table
Finally!
I have gotten it to work!
The Trick was to enclose $pskey and $item.Attributes[$pskey].GetValues('string') in $()
Without $() Add-Member was adding the properties as Arrays and not as Strings.
Here is the working Code:
$table = New-Object System.Collections.ArrayList
$c.Bind()
$r = New-Object System.DirectoryServices.Protocols.SearchRequest -ArgumentList $baseDN,$Filter,$scope,$attrlist
$re = $c.SendRequest($r)
foreach ($item in $re.Entries) {
$newRow = New-Object -TypeName psobject
foreach($attr in $vals){
if($item.Attributes.Keys -contains $attr){
$pskeys += $attr
}}
foreach($pskey in $pskeys){
$newRow | Add-Member -NotePropertyName $($pskey) -NotePropertyValue $($item.Attributes[$pskey].GetValues('string'))
}
$table.Add($newRow) | Out-Null
$pskeys = #()
}
$table | Export-Csv -Path $ExportPath -NoTypeInformation -Encoding UTF8 -Append -Delimiter ";"
Thank You jdweng for pointing me in the right direction.
$table | Format-Table on the console and the resulting CSV after the Export look flawless now.
My Problem is solved.

Vmware performance metrics into a a single CSV file for each VM

I am trying to create a .csv file containing VmWare Performance metrics for several VMs.
The googlefoo'd examples I found generally separate each performance counter into individual .csv files which is not preferable. I would rather have one .csv per VM that contains all available Performance counters.
I would appreciate if you could please help me wrap my mind over how to get this done.
Here is what I have for now:
Set-PowerCLIConfiguration -InvalidCertificateAction Ignore
Connect-ViServer my-vcenter
$vms = Get-Content list-of-vms.txt
$start = (Get-Date).AddDays(-7)
$vms | ForEach-Object {
$metrics = (Get-VM $_ | Get-StatType)
$output = #() #this is where i collect all the counters per line
$hashtable = Get-Stat -Entity $_ -Stat $metrics -Start $start | Group-Object -Property TimeStamp
For ($i=0; $i -lt $hashtable.Count; $i++) { # iterate through each timestamp
$output = New-Object -TypeName psobject -Property #{TimeStamp = $hashtable[$i].Name}
For ($j=0; $j -lt $hashtable[$i].Group.Count; $j++) { # iterate through each counter in above timestamp
$output| Add-Member -Name $hashtable[$i].Group[$j].MetricId -Type NoteProperty -Value $hashtable[$i].Group[$j].Value
}
$output | Export-Csv "$($_)_results.csv" -Append
}
}
The thinking process here is this:
For each VM, generate a hashtable with all available counters, grouped by time stamp.
Then, for each time stamp, extract each counters name, value type and add that as a LINE to a $output variable. Once all the recursive iterations are done, create an .csv file with the results.
.csv file's content would look like this as an example:
somevm_results.csv
TimeStamp, cpu.usage.average, mem.usage.average, etc
5/20/2020,50%,50%, etc
I managed to get what i wanted. Below is the code that works for me.
if you have a suggestion on how to make it less of a mess of a code, I'd greatly appreciate it.
Set-PowerCLIConfiguration -InvalidCertificateAction Ignore
$vcenter = "my-vcenter"
Connect-ViServer $vcenter
<# use this if you want only specific counters
$metrics = "cpu.usage.average",
"cpu.usagemhz.average",
"mem.usage.average"
#>
$vms = Get-VM (Get-Content -Path .\list.txt)
$start = (Get-Date).AddDays(-7)
$vms | ForEach-Object {
$output = #()
$metrics = ($_ | Get-StatType)
$hashtable = Get-Stat -Entity $_ -Stat $metrics -Start $start | Group-Object -Property TimeStamp
For ($i=0; $i -lt $hashtable.Count; $i++) { # iterate through each TimeStamp
$output = New-Object -TypeName psobject -Property #{TimeStamp = $hashtable[$i].Name}
For ($j=0; $j -lt $hashtable[$i].Group.Count; $j++) { # iterate through each counter in above timestamp
If ($hashtable[$i].Group[$j].Instance){ #check if a group has multiple instance names for the same counter
$counter_name = $($hashtable[$i].Group[$j].MetricId+"-"+$hashtable[$i].Group[$j].Instance+" ("+$hashtable[$i].Group[$j].Unit+")")
} else {
$counter_name = $($hashtable[$i].Group[$j].MetricId+" ("+$hashtable[$i].Group[$j].Unit+")")
}
$counter_value = $hashtable[$i].Group[$j].Value
$output| Add-Member -Name $counter_name -Type NoteProperty -Value $counter_value
}
$output | Export-Csv "$($_).csv" -Append -NoTypeInformation
}
}
Disconnect-VIServer $vcenter -Confirm:$false

Loop Confusion Comparing Objects

I am trying to create my own service comparison script. I see some online but want to do this myself. I've only gotten so far. I keep getting confused.
The desired output is with the following format. It doesn't even have to show what's different. I just want to see what the previous state was compared to the current state. I did a compare-object and it didn't give me the format I desired. I then thought maybe I should just do two nested loops and create a new object with the states I want in it. It didn't work out correctly, it returns an array. So then I thought, maybe a for loop in the foreach loop... I keep confusing myself and it's so close.
You have to provide a csv with some services to compare to to make this work as part of it's paramaters.
Usage
Inspect-ServiceSnapshot -SnapshotPath "C:\YourPath"
Desired output
Name CurrentState PreviousState
app1 Running Stopped
Code So Far
function Inspect-ServiceSnapshot {
[CmdletBinding()]
param (
#Snapshot
[Parameter(Mandatory=$true)]
[ValidatePattern("C:")]
[string]
$SnapshotPath,
# timer
[Parameter(Mandatory=$false)]
[int]
$TimeToWait
)
if($TimeToWait -ne $null) {
Start-Sleep -Seconds $TimeToWait
$list = #()
$old = Import-Csv -Path $SnapshotPath
foreach($entry in (get-service)) {
foreach($oldItem in $old) {
$object = New-Object -TypeName psobject -Property #{
Name = $entry.Name
CurrentStatus = $entry.status
DisplayName = $entry.displayname
PreviousStatus = $oldItem.status
}
$list += $object
}
}
$list
} else {
$list = #()
$old = Import-Csv -Path $SnapshotPath
foreach($entry in (get-service)) {
foreach($oldItem in $old) {
$object = New-Object -TypeName psobject -Property #{
Name = $entry.Name
CurrentStatus = $entry.status
DisplayName = $entry.displayname
PreviousStatus = $oldItem.status
}
$list += $object
}
}
$list
}
}
This should do it because it is actually checking the value for the old service to be the same as the one Get-Service provides at a certain time.
function Inspect-ServiceSnapshot {
[CmdletBinding()]
param (
#Snapshot
[Parameter(Mandatory=$true)]
[ValidateScript({Test-Path $_ -PathType Leaf})]
[string]$SnapshotPath,
# timer
[Parameter(Mandatory=$false)]
[int]$TimeToWait = 0
)
if($TimeToWait) { Start-Sleep -Seconds $TimeToWait }
$list = #()
$old = Import-Csv -Path $SnapshotPath
foreach($entry in (Get-Service)) {
# make sure we are dealing with the SAME service
$oldItem = $old | Where-Object { $_.Name -eq $entry.Name }
$object = New-Object -TypeName psobject -Property #{
Name = $entry.Name
CurrentStatus = $entry.status
DisplayName = $entry.displayname
PreviousStatus = if ($oldItem) { $oldItem.status } else { 'Unknown' }
}
$list += $object
}
$list
}
This answer is assuming there are no new services added on a regular basis.
You almost got it! You can forgo the nested loops, get rid of the else block (it is redundant), and use an index loop. When you're using nested loops like that, you are iterating through all of the array elements in the $old array every time you iterate through one of the (get-service) objects. This can cause issues when your arrays include thousands of objects.
You can easily get to what you want by using a for loop.
e.g.
if($TimeToWait -ne $null) {
Start-Sleep -Seconds $TimeToWait
}
$list = #();
$old = Import-Csv -Path $SnapshotPath | Sort-Object Name;
$new = Get-Service | Sort-Object Name;
for ($i -eq 0; $i -lt $old.length -or $i -lt $new.length; $i++) {
$object = New-Object -TypeName psobject -Property #{
Name = $new[$i].Name
CurrentStatus = $new[$i].status
DisplayName = $new[$i].displayname
PreviousStatus = $old[$i].status
}
$list += $object;
}
$list;
A lot of your code is redundant and you can just do this all in one go. Since you're not technically comparing any objects, you can just populate the fields as they go.

Shell.Application - write new Values?

I have a directory with MP3 files. I can read the properties with this script. How I can write properties (Album, Genre, etc.)?
$com = (New-Object -ComObject Shell.Application).NameSpace('C:\Users\Peter\Music')
for( $index = 0; ((-not $bitrateAttribute) -or (-not $albumAttribute)); ++$index ) {
$name = $com.GetDetailsOf($com.Items,$index)
if ($name -eq 'Album') {$albumAttribute = $index}
if ($name -eq 'Bit rate') {$bitrateAttribute = $index}
}
$com.Items() | ForEach-Object {
New-Object -TypeName PSCustomObject -Property #{
Name = $_.Name
Album = $com.GetDetailsOf($_,$albumAttribute)
BitRate = $com.GetDetailsOf($_,$bitrateAttribute)
} | Select-Object -Property Name,Album,BitRate
}
Or is there a better way to write ID3 tags to MP3 files?

PSCustomObject to Hashtable

What is the easiest way to convert a PSCustomObject to a Hashtable? It displays just like one with the splat operator, curly braces and what appear to be key value pairs. When I try to cast it to [Hashtable] it doesn't work. I also tried .toString() and the assigned variable says its a string but displays nothing - any ideas?
Shouldn't be too hard. Something like this should do the trick:
# Create a PSCustomObject (ironically using a hashtable)
$ht1 = #{ A = 'a'; B = 'b'; DateTime = Get-Date }
$theObject = new-object psobject -Property $ht1
# Convert the PSCustomObject back to a hashtable
$ht2 = #{}
$theObject.psobject.properties | Foreach { $ht2[$_.Name] = $_.Value }
Keith already gave you the answer, this is just another way of doing the same with a one-liner:
$psobject.psobject.properties | foreach -begin {$h=#{}} -process {$h."$($_.Name)" = $_.Value} -end {$h}
Here's a version that works with nested hashtables / arrays as well (which is useful if you're trying to do this with DSC ConfigurationData):
function ConvertPSObjectToHashtable
{
param (
[Parameter(ValueFromPipeline)]
$InputObject
)
process
{
if ($null -eq $InputObject) { return $null }
if ($InputObject -is [System.Collections.IEnumerable] -and $InputObject -isnot [string])
{
$collection = #(
foreach ($object in $InputObject) { ConvertPSObjectToHashtable $object }
)
Write-Output -NoEnumerate $collection
}
elseif ($InputObject -is [psobject])
{
$hash = #{}
foreach ($property in $InputObject.PSObject.Properties)
{
$hash[$property.Name] = ConvertPSObjectToHashtable $property.Value
}
$hash
}
else
{
$InputObject
}
}
}
My extremely lazy approach, enabled by a new feature in PowerShell 6:
$myhashtable = $mypscustomobject | ConvertTo-Json | ConvertFrom-Json -AsHashTable
This works for PSCustomObjects created by ConvertFrom_Json.
Function ConvertConvertFrom-JsonPSCustomObjectToHash($obj)
{
$hash = #{}
$obj | Get-Member -MemberType Properties | SELECT -exp "Name" | % {
$hash[$_] = ($obj | SELECT -exp $_)
}
$hash
}
Disclaimer: I barely understand PowerShell so this is probably not as clean as it could be. But it works (for one level only).
My code:
function PSCustomObjectConvertToHashtable() {
param(
[Parameter(ValueFromPipeline)]
$object
)
if ( $object -eq $null ) { return $null }
if ( $object -is [psobject] ) {
$result = #{}
$items = $object | Get-Member -MemberType NoteProperty
foreach( $item in $items ) {
$key = $item.Name
$value = PSCustomObjectConvertToHashtable -object $object.$key
$result.Add($key, $value)
}
return $result
} elseif ($object -is [array]) {
$result = [object[]]::new($object.Count)
for ($i = 0; $i -lt $object.Count; $i++) {
$result[$i] = (PSCustomObjectConvertToHashtable -object $object[$i])
}
return ,$result
} else {
return $object
}
}
For simple [PSCustomObject] to [Hashtable] conversion Keith's Answer works best.
However if you need more options you can use
function ConvertTo-Hashtable {
<#
.Synopsis
Converts an object to a hashtable
.DESCRIPTION
PowerShell v4 seems to have trouble casting some objects to Hashtable.
This function is a workaround to convert PS Objects to [Hashtable]
.LINK
https://github.com/alainQtec/.files/blob/main/src/scripts/Converters/ConvertTo-Hashtable.ps1
.NOTES
Base ref: https://community.idera.com/database-tools/powershell/powertips/b/tips/posts/turning-objects-into-hash-tables-2
#>
PARAM(
# The object to convert to a hashtable
[Parameter(ValueFromPipeline = $true, Mandatory = $true)]
$InputObject,
# Forces the values to be strings and converts them by running them through Out-String
[switch]$AsString,
# If set, empty properties are Included
[switch]$AllowNulls,
# Make each hashtable to have it's own set of properties, otherwise,
# (default) each InputObject is normalized to the properties on the first object in the pipeline
[switch]$DontNormalize
)
BEGIN {
$headers = #()
}
PROCESS {
if (!$headers -or $DontNormalize) {
$headers = $InputObject | Get-Member -type Properties | Select-Object -expand name
}
$OutputHash = #{}
if ($AsString) {
foreach ($col in $headers) {
if ($AllowNulls -or ($InputObject.$col -is [bool] -or ($InputObject.$col))) {
$OutputHash.$col = $InputObject.$col | Out-String -Width 9999 | ForEach-Object { $_.Trim() }
}
}
} else {
foreach ($col in $headers) {
if ($AllowNulls -or ($InputObject.$col -is [bool] -or ($InputObject.$col))) {
$OutputHash.$col = $InputObject.$col
}
}
}
}
END {
return $OutputHash
}
}
Maybe this is overkill but I hope it Helps
Today, the "easiest way" to convert PSCustomObject to Hashtable would be so:
$custom_obj | ConvertTo-HashtableFromPsCustomObject
OR
[hashtable]$custom_obj
Conversely, you can convert a Hashtable to PSCustomObject using:
[PSCustomObject]$hash_table
Only snag is, these nifty options may not be available in older versions of PS