Replace PSObj property value based on a list - powershell

I've the following PSObj with some properties stored in an $array :
ComputerName : MyComputer
Time : 08/11/2022 13:57:53
DetectionFile : MyBadFile.exe
ThreatName : WS.Reputation.1
Action : 12
I'm trying to replace the action ID number by it's corresponding description. I've a hashtable with the possibles reasons behind the Action ID
$ActionId = #{
0 = 'Unknown'
1 = 'Blocked'
2 = 'Allowed'
3 = 'No Action'
4 = 'Logged'
5 = 'Command Script Run'
6 = 'Corrected'
7 = 'Partially Corrected'
8 = 'Uncorrected'
10 = 'Delayed Requires reboot to finish the operation.'
11 = 'Deleted'
12 = 'Quarantined'
13 = 'Restored'
14 = 'Detected'
15 = 'Exonerated No longer suspicious (re-scored).'
16 = 'Tagged Marked with extended attributes.'
}
I'm trying to parse each item of this array, and each value of the reason ID to replace the ID by the reason string
# parse array
foreach ($Item in $array) {
# parse possible values
foreach ($value in $ActionId) {
if ($value -eq $item.Action) {
$Item.Action = $ActionId[$value]
$Item.Action
}
}
From my understanding, I'm missing the correct syntax here
$Item.Action = $ActionId[$value]
I do not get any errors, but from the debugger, I'm replacing the action property by $null with the above...

The immediate fix is to loop over the keys (.Keys) of your $ActionId hashtable:
foreach ($Item in $array) {
# parse possible values
foreach ($value in $ActionId.Keys) {
if ($value -eq $item.Action) {
$Item.Action = $ActionId[$value]
$Item.Action # diagnostic output
}
}
}
Note:
To avoid confusion, consider renaming $value to $key.
Generally, note that hashtables are not enumerated in the pipeline / in looping constructs in PowerShell.
That is, foreach ($value in $ActionId) ... doesn't actually loop over the hashtable's entries, and is the same as $value = $ActionID)
If you want to enumerate a hashtable's entries - as key-value pairs of type System.RuntimeType - you would need to use the .GetEnumerator() method; in your case, however, enumerating the keys is sufficient.
However, the simpler and more efficient solution is to test whether the $Item.Action value exists as a key in your hashtable, using the latter's .Contains() method:[1]
foreach ($Item in $array) {
if ($ActionId.Contains($Item.Action)) {
$Item.Action = $ActionId[$Item.Action]
$Item.Action # diagnostic output
}
}
You can further streamline this as follows, though it is conceptually a bit obscure:
foreach ($Item in $array) {
if ($null -ne ($value = $ActionId[$Item.Action])) {
$Item.Action = $value
$Item.Action # diagnostic output
}
}
= is only ever PowerShell's assignment operator; for equality / non-equality comparison, -eq / -ne is required.
Here, an assignment to $value is indeed being performed and the assigned value then acts as the RHS of the -ne operation; in other words: you can use assignment as expressions in PowerShell.
If hashtable $ActionId has no key with value $Item.Action, $ActionId[$Item.Action] quietly returns $null.
Finally - in PowerShell (Core) 7+ only - an even more concise (though not necessarily faster) solution is possible, using ??, the null-coalescing operator:
foreach ($Item in $array) {
$Item.Action = $ActionId[$Item.Action] ?? $Item.Action
$Item.Action # diagnostic output
}
That is, the value of $ActionId[$Item.Action] is only used if it isn't $null; otherwise, $Item.Action, i.e. the current value, is used (which is effectively a no-op).
[1] .ContainsKey() works too, and while this name is conceptually clearer than .Contains(), it is unfortunately not supported by PowerShell's [ordered] hashtables (System.Collections.Specialized.OrderedDictionary) and, generally speaking, not supported by other dictionary (hashtable-like types), given that the System.Collections.IDictionary interface only has .Contains()

In addition mklement0's helpful answer, I was just thinking outside the box (aka question):
This is typical situation where I would consider to use an enum except for the fact that that the keys do not (easily) accept spaces (as in your question).
Enum ActionTypes {
Unknown
Blocked
Allowed
NoAction
Logged
CommandScriptRun
Corrected
PartiallyCorrected
Uncorrected
Delayed
Deleted
Quarantined
Restored
Detected
Exonerated
Tagged
}
$PSObj = [PSCustomObject]#{
ComputerName = 'MyComputer'
Time = [DateTime]'08/11/2022 13:57:53'
DetectionFile = 'MyBadFile.exe'
ThreatName = 'WS.Reputation.1'
Action = 12
}
$PSObj.Action = [ActionTypes]$PSObj.Action
$PSObj
ComputerName : MyComputer
Time : 8/11/2022 1:57:53 PM
DetectionFile : MyBadFile.exe
ThreatName : WS.Reputation.1
Action : Restored
The advantage is that you won't lose the actual action id, meaning if you e.g. insert the object back into a database, it will automatically type cast to the original integer type:
$PSObj.Action
Restored
[int]$PSObj.Action
12

Related

Check if a condition is met by a line within a TXT but "in an advanced way"

I have a TXT file with 1300 megabytes (huge thing). I want to build code that does two things:
Every line contains a unique ID at the beginning. I want to check for all lines with the same unique ID if the conditions is met for that "group" of IDs. (This answers me: For how many lines with the unique ID X have all conditions been met)
If the script is finished I want to remove all lines from the TXT where the condition was met (see 2). So I can rerun the script with another condition set to "narrow down" the whole document.
After few cycles I finally have a set of conditions that applies to all lines in the document.
It seems that my current approach is very slow.( one cycle needs hours). My final result is a set of conditions that apply to all lines of code.
If you find an easier way to do that, feel free to recommend.
Help is welcome :)
Code so far (does not fullfill everything from 1&2)
foreach ($item in $liste)
{
# Check Conditions
if ( ($item -like "*XXX*") -and ($item -like "*YYY*") -and ($item -notlike "*ZZZ*")) {
# Add a line to a document to see which lines match condition
Add-Content "C:\Desktop\it_seems_to_match.txt" "$item"
# Retrieve the unique ID from the line and feed array.
$array += $item.Split("/")[1]
# Remove the line from final document
$liste = $liste -replace $item, ""
}
}
# Pipe the "new cleaned" list somewhere
$liste | Set-Content -Path "C:\NewListToWorkWith.txt"
# Show me the counts
$array | group | % { $h = #{} } { $h[$_.Name] = $_.Count } { $h } | Out-File "C:\Desktop\count.txt"
Demo Lines:
images/STRINGA/2XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg images/STRINGA/3XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg images/STRINGB/4XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg images/STRINGB/5XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg images/STRINGC/5XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
performance considerations:
Add-Content "C:\Desktop\it_seems_to_match.txt" "$item"
try to avoid wrapping cmdlet pipelines
See also: Mastering the (steppable) pipeline
$array += $item.Split("/")[1]
Try to avoid using the increase assignment operator (+=) to create a collection
See also: Why should I avoid using the increase assignment operator (+=) to create a collection
$liste = $liste -replace $item, ""
This is a very expensive operation considering that you are reassigning (copying) a long list ($liste) with each iteration.
Besides it is a bad practice to change an array that you are currently iterating.
$array | group | ...
Group-Object is a rather slow cmdlet, you better collect (or count) the items on-the-fly (where you do $array += $item.Split("/")[1]) using a hashtable, something like:
$Name = $item.Split("/")[1]
if (!$HashTable.Contains($Name)) { $HashTable[$Name] = [Collections.Generic.List[String]]::new() }
$HashTable[$Name].Add($Item)
To minimize memory usage it may be better to read one line at a time and check if it already exists. Below code I used StringReader and you can replace with StreamReader for reading from a file. I'm checking if the entire string exists, but you may want to split the line. Notice I have duplicaes in the input but not in the dictionary. See code below :
$rows= #"
images/STRINGA/2XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGA/3XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGB/4XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGB/5XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGC/5XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGA/2XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGA/3XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGB/4XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGB/5XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
images/STRINGC/5XXXXXXXX_rTTTTw_GGGG1_Top_MMM1_YY02_ZZZ30_AAAA5.jpg
"#
$dict = [System.Collections.Generic.Dictionary[int, System.Collections.Generic.List[string]]]::new();
$reader = [System.IO.StringReader]::new($rows)
while(($row = $reader.ReadLine()) -ne $null)
{
$hash = $row.GetHashCode()
if($dict.ContainsKey($hash))
{
#check if list contains the string
if($dict[$hash].Contains($row))
{
#string is a duplicate
}
else
{
#add string to dictionary value if it is not in list
$list = $dict[$hash].Value
$list.Add($row)
}
}
else
{
#add new hash value to dictionary
$list = [System.Collections.Generic.List[string]]::new();
$list.Add($row)
$dict.Add($hash, $list)
}
}
$dict

How in general can I find functions that contains bugs due to array output?

Unassigned variables are outputted in PowerShell functions. So for instance in the function below you will get an object array returned instead of a string because [regex] is unassigned and needs to become assigned or nullified or whatever.
These bugs are very hard to detect when scanning codebases. Since some functions DO output an object array willingly (and then correctly handled) while for others it is indeed a bug. In this case the output was used to write somewhere and since an array gets transformed to a string it was undetectable.
function Get-PascalizedString {
param(
[string]$String
)
$rx = "(?:[^a-zA-Z0-9]*)(?<first>[a-zA-Z0-9])(?<reminder>[a-zA-Z0-9]*)(?:[^a-zA-Z0-9]*)"
$result = ""
[regex]::Matches($String, $rx) | ForEach-Object {$_.Groups} {
$TextInfo = (Get-Culture).TextInfo
$part = $TextInfo.ToTitleCase($_.Value.ToLower()).Trim()
$part = $part -replace "[^a-zA-Z0-9]"
$result = $result + $part
}
return $result
}
$a = Get-PascalizedString -String "aaa"
write-host $a
$a.GetType() // but...
So is there a smart way to detect these kind of bugs in larger codebases?

Casting Object to String Array Powershell

I want to create an array of strings instead of a variable object so that I can use the "contains" keyword on each index of the array.
$myArray = Get-ADDomain
The above creates an object, which is not what I want. I also tried
[string[]] $myArray = Get-ADDomain
But after that, $myArray only contains one string and it is the first non-empty property of Get-ADDomain, in my case "ComputersContainer". What should I do to receive an array of strings where each string is a different property, such as
$myArray[0] = "AllowedDNSSuffixes = {}"
PowerShell will always return objects by design of course, and specifying that [string[]], does not really change that.
For what you are trying to use, you have to force the array creation. The below is just one way, but I am sure others will have more elegant ways of doing this as well. Though I am curious why one would want to do this, this way. But, hey, that's just me.
# Create an empty array
$DomainData = #()
# Get all the data points for the utilized cmdlet, split on a common delimiter for the array
[string[]]$DomainData = (Get-ADDomain | Select *) -split ';'
# Display the array count
$DomainData.Count
34
# validate getting a value from the array by using an index number
$Item = $DomainData[17]
NetBIOSName=CONTOSO
[array]::IndexOf($DomainData, $Item)
17
# Use that element number to validate the use of the contains comparison operator
0..($DomainData.Count - 1) | %{ If($DomainData[$_] -contains $item){"Index key is $_ contains a value of $Item"} }
Index key is 17 contains a value of NetBIOSName=CONTOSO
# Use the previous with a partial string for a comparison, -contains cannot be used, like or match has to be used
# From the documentation:
# -Contains
# Description: Containment operator. Tells whether a collection of reference values includes a single test value.
$Item = '*domain*'
0..($DomainData.Count - 1) | %{ If($DomainData[$_] -like $item){"Index key is $_ like a value of $Item"} }
Index key is 1 like a value of *domain*
Index key is 6 like a value of *domain*
Index key is 7 like a value of *domain*
Index key is 8 like a value of *domain*
Index key is 18 like a value of *domain*
Index key is 20 like a value of *domain*
You cannot cast a PSObject directly to a string array like that.
However, this can be accomplished rather easily.
To get an array of string from the object
$myArray = Get-ADDomain
# You can use a standard array #() but these tends to be slower for bigger amount of data
$outArray = New-Object -TypeName System.Collections.Generic.List[String]
#To add just the value
$myArray.psobject.properties | Foreach { $outArray.Add($_.Value) }
# To add Name = {Value} instead
$myArray.psobject.properties | Foreach { $outArray.Add("$($_.Name) = {$($_.Value)}") }
Using an hasthable instead:
$myArray = Get-ADDomain
$hashtable = #{}
$myArray.psobject.properties | Foreach { $hashtable[$_.Name] = $_.Value }
# If you need to do something with the key
Foreach ($key in $hashtable.Keys) {
$Value = $hashtable[$key]
if ($value -like '*prod*') {
Write-Host $key
}
}

Compare objects based on subset of properties

Say I have 2 powershell hashtables one big and one small and, for a specific purpose I want to say they are equal if for the keys in the small one, the keys on the big hastable are the same.
Also I don't know the names of the keys in advance. I can use the following function that uses Invoke-Expression but I am looking for nicer solutions, that don't rely on this.
Function Compare-Subset {
Param(
[hashtable] $big,
[hashtable] $small
)
$keys = $small.keys
Foreach($k in $keys) {
$expression = '$val = $big.' + "$k" + ' -eq ' + '$small.' + "$k"
Invoke-Expression $expression
If(-not $val) {return $False}
}
return $True
}
$big = #{name='Jon'; car='Honda'; age='30'}
$small = #{name = 'Jon'; car='Honda'}
Compare-Subset $big $small
A simple $true/$false can easily be gotten. This will return $true if there are no differences:
[string]::IsNullOrWhiteSpace($($small|Select -Expand Keys|Where{$Small[$_] -ne $big[$_]}))
It checks for all keys in $small to see if the value of that key in $small is the same of the value for that key in $big. It will only output any values that are different. It's wrapped in a IsNullOrWhitespace() method from the [String] type, so if any differences are found it returns false. If you want to list differences just remove that method.
This could be the start of something. Not sure what output you are looking for but this will output the differences between the two groups. Using the same sample data that you provided:
$results = Compare-Object ($big.GetEnumerator() | % { $_.Name }) ($small.GetEnumerator() | % { $_.Name })
$results | ForEach-Object{
$key = $_.InputObject
Switch($_.SideIndicator){
"<="{"Only reference object has the key: '$key'"}
"=>"{"Only difference object has the key: '$key'"}
}
}
In primetime you would want something different but just to show you the above would yield the following output:
Only reference object has the key: 'age'

Why is an empty PowerShell pipeline not the same as null?

I am trying to understand the behavior of the #() array constructor, and I came across this very strange test.
It seems that the value of an empty pipeline is "not quite" the same as $null, even though it is -eq $null
The output of each statement is shown after the ###
$y = 1,2,3,4 | ? { $_ -ge 5 }
$z = $null
if ($y -eq $null) {'y is null'} else {'y NOT null'} ### y is null
if ($z -eq $null) {'z is null'} else {'z NOT null'} ### z is null
$ay = #($y)
$az = #($z)
"ay.length = " + $ay.length ### ay.length = 0
"az.length = " + $az.length ### az.length = 1
$az[0].GetType() ### throws exception because $az[0] is null
So the $az array has length one, and $az[0] is $null.
But the real question is: how is it possible that both $y and $z are both -eq $null, and yet when I construct arrays with #(...) then one array is empty, and the other contains a single $null element?
Expanding on Frode F.'s answer, "nothing" is a mostly magical value in PowerShell - it's called [System.Management.Automation.Internal.AutomationNull]::Value. The following will work similarly:
$y = 1,2,3,4 | ? { $_ -ge 5 }
$y = [System.Management.Automation.Internal.AutomationNull]::Value
PowerShell treats the value AutomationNull.Value like $null in most places, but not everywhere. One notable example is in a pipeline:
$null | % { 'saw $null' }
[System.Management.Automation.Internal.AutomationNull]::Value | % { 'saw AutomationNull.Value' }
This will only print:
saw $null
Note that expressions are themselves pipelines even if you don't have a pipeline character, so the following are roughly equivalent:
#($y)
#($y | Write-Output)
Understanding this, it should be clear that if $y holds the value AutomationNull.Value, nothing is written to the pipeline, and hence the array is empty.
One might ask why $null is written to the pipeline. It's a reasonable question. There are some situations where scripts/cmdlets need to indicate "failed" without using exceptions - so "no result" must be different, $null is the obvious value to use for such situations.
I've never run across a scenario where one needs to know if you have "no value" or $null, but if you did, you could use something like this:
function Test-IsAutomationNull
{
param(
[Parameter(ValueFromPipeline)]
$InputObject)
begin
{
if ($PSBoundParameters.ContainsKey('InputObject'))
{
throw "Test-IsAutomationNull only works with piped input"
}
$isAutomationNull = $true
}
process
{
$isAutomationNull = $false
}
end
{
return $isAutomationNull
}
}
dir nosuchfile* | Test-IsAutomationNull
$null | Test-IsAutomationNull
The reason you're experiencing this behaviour is becuase $null is a value. It's a "nothing value", but it's still a value.
PS P:\> $y = 1,2,3,4 | ? { $_ -ge 5 }
PS P:\> Get-Variable y | fl *
#No value survived the where-test, so y was never saved as a variable, just as a "reference"
Name : y
Description :
Value :
Visibility : Public
Module :
ModuleName :
Options : None
Attributes : {}
PS P:\> $z = $null
PS P:\> Get-Variable z | fl *
#Our $null variable is saved as a variable, with a $null value.
PSPath : Microsoft.PowerShell.Core\Variable::z
PSDrive : Variable
PSProvider : Microsoft.PowerShell.Core\Variable
PSIsContainer : False
Name : z
Description :
Value :
Visibility : Public
Module :
ModuleName :
Options : None
Attributes : {}
The way #() works, is that it guarantees that the result is delievered inside a wrapper(an array). This means that as long as you have one or more objects, it will wrap it inside an array(if it's not already in an array like multiple objects would be).
$y is nothing, it's a reference, but no variable data was stored. So there is nothing to create an array with. $z however, IS a stored variable, with nothing(null-object) as the value. Since this object exists, the array constructor can create an array with that one item.