Returning multiple values from simular hash keys powershell - powershell

I have the following code that pulls in some server information from a text file and spits it into a hashtable.
Get-Content $serverfile | Foreach-Object {
if($_ -match '^([^\W+]+)\s+([^\.+]+)')
{
$sh[$matches[1]] = $matches[2]
}
}
$sh.GetEnumerator()| sort -Property Name
This produces the following:
Name Value
---- -----
Disk0 40
Disk1 40
Disk2 38
Disk3 43
Memory 4096
Name Value
Number_of_disks 1
Number_of_network_cards 2
Number_of_processors 1
ServerName WIN02
Depending on the server there may be one Disk0 or many more.
My challenge here is to pull each Disk* value from each of the varying number of Disk keys and return the values in a comma separated list, for example;
$disks = 40,40,38,43
I have tried varying approaches to this problem however none have met the criteria of being dynamic and including the ',' after each disk.
Any help would be appreciated.

I assume that when you say "Depending on the server there may be one Disk0 or many more", you mean "one Disk or many more", each with a different number? You can't have more than one Disk0, because key names can't be duplicated in a hash.
This will give you a list of all the hash values for keys starting with "Disk":
$sh.Keys | ?{$_ -match '^Disk'} | %{$sh.$_}
If you actually want to get a comma-separated list (a single string value), you can use the -join operator:
$disks = ($sh.Keys | ?{$_ -match '^Disk'} | %{$sh.$_}) -join ','
However, if the reason you want a comma-separated list is in order to get an array of the values, you don't really need the comma-separated list; just assign the results (which are already an array) to the variable:
$disks = $sh.Keys | ?{$_ -match '^Disk'} | %{$sh.$_}
Note, BTW, that hashes are not ordered. There's no guarantee that the order of the keys listed will be the same as the order in which you added them or in ascending alphanumeric order. So, in the above example, your result could be 38,40,43,40. If order does matter (i.e. you're counting on the values in $disks to be in the order of their respective Disk numbers, you have two options.
Filter the listing of the keys through Sort-Object:
$sh.Keys | ?{$_ -match '^Disk'} | sort | %{$sh.$_}
(You can put the | sort between $sh.Keys and | ?{..., but it's more efficient this way...which makes little difference here but would matter with larger data sets.)
Use an ordered dictionary, which functions pretty much the same as a hash, but maintains the keys in the order added:
$sh = New-Object System.Collections.Specialized.OrderedDictionary

Related

How best to speed up powershell processing time (compare-object)

I have a powershell script which uses Compare-Object to diff/compare a list of MD5 checksum's against each-other ... how can I speed this up? its been running for hours!
$diffmd5 =
(Compare-Object -ReferenceObject $localmd5 -DifferenceObject $remotefilehash |
Where-Object { ($_.SideIndicator -eq '=>') } |
Select-Object -ExpandProperty InputObject)
Compare-Object is convenient, but indeed slow; also avoiding the pipeline altogether is important for maximizing performance.
I suggest using a [System.Collections.Generic.HashSet[T] instance, which supports high-performance lookups in a set of unordered[1] values:[2]
# Two sample arrays
$localmd5 = 'Foo1', 'Bar1', 'Baz1'
$remotefilehash = 'foo1', 'bar1', 'bar2', 'baz1', 'more'
# Create a hash set from the local hashes.
# Make lookups case-*insensitive*.
# Note: Strongly typing the input array ([string[]]) is a must.
$localHashSet = [System.Collections.Generic.HashSet[string]]::new(
[string[]] $localmd5,
[System.StringComparer]::OrdinalIgnoreCase
)
# Loop over all remote hashes to find those not among the local hashes.
$remotefilehash.Where({ -not $localHashSet.Contains($_) })
The above yields collection 'bar2', 'more'.
Note that if case-sensitive lookups are sufficient, which is the default (for string elements), a simple cast is sufficient to construct the hash set:
$localHashSet = [System.Collections.Generic.HashSet[string]] $localmd5
Note: Your later feedback states that $remotefilehash is a hashtable(-like) collection of key-value pairs rather than a collection of mere file-hash strings, in which the keys store the hash strings. In that case:
To find just the differing hash strings (note the .Keys property access to get the array of key values):
$remotefilehash.Keys.Where({ -not $localHashSet.Contains($_) })
To find those key-value pairs whose keys are not in the hash set (note the .GetEnumerator() call to enumerate all entries (key-value pairs)):
$remotefilehash.GetEnumerator().Where({ -not $localHashSet.Contains($_.Key) })
Alternatively, if the input collections are (a) of the same size and (b) have corresponding elements (that is, element 1 from one collection should be compared to element 1 from the other, and so on), using Compare-Object with -SyncWindow 0, as shown in js2010's helpful answer, with subsequent .SideIndicator filtering may be an option; to speed up the operation, the -PassThru switch should be used, which forgoes wrapping the differing objects in [pscustomobject] instances (the .SideIndicator property is then added as a NoteProperty member directly to the differing objects).
[1] There is a related type for maintaining sorted values, System.Collections.Generic.SortedSet[T], but - as of .NET 6 - no built-in type for maintaining values in input order, though you can create your own type by deriving from [System.Collections.ObjectModel.KeyedCollection[TKey, TItem]]
[2] Note that a hash set - unlike a hash table - has no values associated with its entries. A hash set is "all keys", if you will - all it supports is testing for the presence of a key == value.
By default, compare-object compares every element in the first array with every element in the second array (up to about 2 billion positions), so the order doesn't matter, but large lists would be very slow. -syncwindow 0 would be much faster but would require matches to be in the same exact positions:
Compare-Object $localmd5 $remotefilehash -syncwindow 0
As a simple demo of -syncwindow:
compare-object 1,2,3 1,3,2 -SyncWindow 0 # not equal
InputObject SideIndicator
----------- -------------
3 =>
2 <=
2 =>
3 <=
compare-object 1,2,3 1,3,2 -SyncWindow 1 # equal
compare-object 1,2,3 1,2,3 -SyncWindow 0 # equal
I feel this should be faster than Compare-Object
$result = [system.collections.generic.list[string]]::new()
foreach($hash in $remotefilehash)
{
if(-not($localmd5.contains($hash)))
{
$result.Add($hash)
}
}
The problem here is that .contains method is case sensitive, I believe all MD5 hashes have uppercase letters but if this was not the case you would need to call the .toupper() or .tolower() methods to normalize the arrays.

Sum various columns to get subtotal depending on a criteria from a row using Powershell

I have a csv file, that contains the next data:
Pages,Pages BN,Pages Color,Customer
145,117,28,Report_Alexis
46,31,15,Report_Alexis
75,27,48,Report_Alexis
145,117,28,Report_Jack
46,31,15,Report_Jack
75,27,48,Report_Jack
145,117,28,Report_Amy
46,31,15,Report_Amy
75,27,48,Report_Amy
So what i need to do , is sum each column based on the report name and the export to another csv file like this
Pages,Pages BN,Pages Color,Customer
266,175,91,Report_Alexis
266,175,91,Report_Jack
266,175,91,Report_Amy
How can i do this?
I tried with this:
$coutnpages = Import-Csv "C:\temp\testcount\final file2.csv" |where {$_.Filename -eq 'Report_Jack'} | Measure-Object -Property Pages -Sum
then
$Countpages.Sum | Set-Content -Path "C:\temp\testcount\final file3.csv"
But this is just one, and then i dont know how to follow.
Can you please help me?
Working code
$IdentityColumns = #('Customer')
$ColumnsToSum = #('Pages', 'Pages BN', 'Pages Color')
$CSVFileInput = 'S:\SCRIPTS\1.csv'
Import-Csv -Path $CSVFileInput |
Group-Object -Property $IdentityColumns |
ForEach-Object {
$resultHT = #{ Customer = $_.Name } # This is result HashTable (Key-Value collection). We add here sum's next line.
#($_.Group | Measure-Object -Property $ColumnsToSum -Sum ) | # Run calculating of sum for all $ColumnsToSum`s in one line
ForEach-Object { $resultHT[$_.Property] = $_.Sum } # For each calculated property we set property in result HashTable
return [PSCustomObject]$resultHT # Convert HashTable to PSCustomObject. This better.
} | # End of ForEach-Object by groups
Select #($ColumnsToSum + $IdentityColumns) | # This sets order of columns. It may be important.
Out-GridView # Or replace with Export-Csv
#Export-Csv ...
Explanation:
Use Group-Object to make collection of groups. Groups have 4 properties:
Name - Name of group, equals to stingified values of property(-ies) you're grouping by
Values - Collection of values of properties you're grouping by (not stringified)
Count - Count of elements grouped into this group
Group - Values of elements grouped into this group
For grouping by single string properties (in this case it is ok), you can easily use Name of group, otherwise, always use Values.
So after Group-Object, you iterate not on collection-of-rows of CSV, but on collection-of-collections-of-rows grouped by some condition.
Measure-Object can process more than one propertiy for single pass (not mixing between values from different properties), we use this actively. This results in array of objects with attribute Property equal to passed to Measure-Object and value (Sum in our case). We move those Property=Sum pairs to hashtable.
[PSCustomObject] converts hashtable to object. Objects are always better for output.

Powershell Group-Object - filter using multiple objects

I have a CSV of devices that are missing security updates along with the date the update was released and kb number.
devicename,date,kb
Desktop1,9/12/17,KB4011055
Desktop1,9/12/17,KB4038866
Desktop2,9/12/17,KB4011055
Desktop2,6/13/17,KB3203467
I am trying to compile a list of devices that are missing updates that have been released in the past 30 days but exclude devices that are also missing older updates. So in the example above, the only device I want is Desktop 1.
I know I could do something like this to see devices that are under that 30 day window but that would still include devices that have other entries which are greater than 30 days.
$AllDevices | Where-Object {[datetime]$_.date_released -gt ((get-date).adddays(-30))}
I was thinking I could use Group-Object devicename to group all the devices together but I'm not sure how to check the dates from there.
Any ideas?
The assumption is that $AllDevices was assigned the output from something like
Import-Csv c:\path\to\some.csv and that PSv3+ is used.
$AllDevices | Group-Object devicename | Where-Object {
-not ([datetime[]] $_.Group.date -le (Get-Date).AddDays(-30))
} | Select-Object #{ l='devicename'; e='Name' }, #{ l='kbs'; e={ $_.Group.kb } }
With the sample input, this yields:
devicename kbs
---------- ---
Desktop1 {KB4011055, KB4038866}
Explanation:
Group-Object devicename groups all input objects by device name, which outputs a collection of [Microsoft.PowerShell.Commands.GroupInfo] instances each representing all input objects sharing a given device name (e.g., Desktop1) - see Get-Help Group-Object.
The Where-Object call is then used to weed out groups that contain objects whose date is older than 30 days.
[datetime[]] $_.Group.date creates an array of date-time objects [datetime[]] from the date-time strings (.date) of every member of the group $_.Group.
Note that $_.Group is the collection of input objects that make up the group, and even though .date is applied directly to $_.Group, the .date property is accessed on each collection member and the results are collected in an array - this handy shortcut syntax is called member-access enumeration and was introduced in PSv3.
-le (Get-Date).AddDays(-30) filters that array to only return members whose dates are older than 30 days; note that -le applied to an array-valued LHS returns a filtered subarray, not a Boolean.
-not negates the result of the -le comparison, which forces interpretation of the filtered array as a Boolean, which evaluates to $False if the array is empty, and $True otherwise; in other words: if one or more group members have dates older than 30 days, the -le comparison evaluates to $True as a Boolean, which -not negates.
This results in groups (and therfore devices) containing at least 1 older-than-30-days date getting removed from further pipeline processing.
Select-Object then receives only those group objects whose members all have dates that fall within the last 30 days, and uses calculated properties (via hashtable literals (#{...}) with standardized entries) to construct the output objects:
A group object's .Name property contains the value of the grouping property/ies passed to Group-Object, which in this case is the input objects' devicename property; #{ l='devicename'; e='Name' } simply renames the .Name property back to devicename.
#{ l='kbs'; e={ $_.Group.kb } } then constructs a kbs property that contains the array of kb values from the members of each group, retrieved by member-access enumeration via a script block { ... }
Note that Select-Object outputs [pscustomobject] instances containing only the explicitly defined properties; in this case, devicename and kbs.
I propose other solution:
import-csv "C:\temp\test.csv" |
select *, #{N="Date";E={[DateTime]$_.Date}} -ExcludeProperty "Date" |
group devicename |
%{
if (($_.Group | where Date -le (Get-Date).AddDays(-30)).Count -eq 0)
{
$LastUpdate=$_.Group | sort Date, kb -Descending | select -First 1
[pscustomobject]#{
DeviceName=$LastUpdate.DeviceName
DateLastUpdate=$LastUpdate.Date
LastUpdate=$LastUpdate.Kb
UpdateList=$_.Group.Kb -join ', '
Group=$_.Group
}
}
}

Powershell, comparing 2 files to find the amount of unique entries in both files

I have 2 files. The contents of
File 1 is: 4,22,1,2,3,14,12,13.
File 2 is: 1,50,2,12,3,6,9.
Im trying to write a script that outputs the total unique entries in both files and the total unique numbers in file 1 and file 2. I am currently using:
$howmany = compare-object $(get-content C:\test\file1.txt) $(get-content C:\test\file2.txt)
Write-Host "Total unique entries in both files is:" $howmany.Count
This does the total unique entries in both files but I can't figure out how to find the total unique entries in file 1 and file 2.
I want the output to be something like:
Total unique entries in file 1 is: 4
Total unique entries in file 2 is: 3
Unique numbers in file 1 are: 4 22 14 13
Unique numbers in file 2 are: 50 6 9
This uses the AsHashTable and AsString parameters to return the groups in a hash table, that is, as a collection of key-value pairs.
In the resulting hash table, each property value is a key, and the group elements are the values. Because each key is a property of the hash table object, you can use dot notation to display the values.
$unique = $howmany | Group-Object -Property sideindicator -AsHashTable -AsString
File1
since the output is an array the -join operator is used to join each number to form a string
($unique.'<=' | Select-Object -ExpandProperty inputobject) -join ','
File2
($unique.'=>' | Select-Object -ExpandProperty inputobject) -join ','
File1 - Count unique items
($unique.'<=' | Select-Object -ExpandProperty inputobject).count
File2 - Count unique items
($unique.'=>' | Select-Object -ExpandProperty inputobject).count
(I know you have an accepted answer, I just wanted to write an alternative. I can't make it work the way I was trying to approach it, but this is close).
function uniques {param($a,$b) $a|? {$b -notcontains $_}}
$f1 = (gc C:\test\file1.txt) | select -unique
$f2 = (gc C:\test\file2.txt) | select -unique
Write-Host "Total unique entries in both files is: $(($f1+$f2 |select -Unique).Count)"
Write-Host "Total unique entries in file 1 is: $((uniques $f1 $f2).Count)"
Write-Host "Total unique entries in file 2 is: $((uniques $f2 $f1).Count)"
Write-Host "Unique numbers in file 1 are: $(uniques $f1 $f2)"
Write-Host "Unique numbers in file 2 are: $(uniques $f2 $f1)"
NB. Your initial code, and therefore #Kiran's answer, has a bug if one of the files contains a duplicate number. e.g. if file1 contains 4,22,1,2,3,14,12,13,4 with a duplicate 4 in it, you'll get 5 unique numbers - 4,22,14,13,4. That's why this has |select -unique for both files when reading them.
NB. my version might fail if a file has only one number, or there is only one unique number. #() around things to make sure they stay as arrays if that matters.

Powershell counting same values from csv

Using PowerShell, I can import the CSV file and count how many objects are equal to "a". For example,
#(Import-csv location | where-Object{$_.id -eq "a"}).Count
Is there a way to go through every column and row looking for the same String "a" and adding onto count? Or do I have to do the same command over and over for every column, just with a different keyword?
So I made a dummy file that contains 5 columns of people names. Now to show you how the process will work I will show you how often the text "Ann" appears in any field.
$file = "C:\temp\MOCK_DATA (3).csv"
gc $file | %{$_ -split ","} | Group-Object | Where-Object{$_.Name -like "Ann*"}
Don't focus on the code but the output below.
Count Name Group
----- ---- -----
5 Ann {Ann, Ann, Ann, Ann...}
9 Anne {Anne, Anne, Anne, Anne...}
12 Annie {Annie, Annie, Annie, Annie...}
19 Anna {Anna, Anna, Anna, Anna...}
"Ann" appears 5 times on it's own. However it is a part of other names as well. Lets use a simple regex to find all the values that are only "Ann".
(select-string -Path 'C:\temp\MOCK_DATA (3).csv' -Pattern "\bAnn\b" -AllMatches | Select-Object -ExpandProperty Matches).Count
That will return 5 since \b is for a word boundary. In essence it is only looking at what is between commas or beginning or end of each line. This omits results like "Anna" and "Annie" that you might have. Select-Object -ExpandProperty Matches is important to have if you have more than one match on a single line.
Small Caveat
It should not matter but in trying to keep the code simple it is possible that your header could match with the value you are looking for. Not likely which is why I don't account for it. If that is a possibility then we could use Get-Content instead with a Select -Skip 1.
Try cycling through properties like this:
(Import-Csv location | %{$record = $_; $record | Get-Member -MemberType Properties |
?{$record.$($_.Name) -eq 'a';}}).Count