Add multiple members to powershell hashtable - powershell

I have a CSV file that contains subnet information that I will use to populate a CSV file that has server information in it. I'm starting by importing the subnet information and when processing it, I'm trying to add multiple members to the initial hashtable, but it is not behaving as expected.
The following code processes the first item as expected, creating a new column with the correct information. The code indicates that it processes at least through two more sections, but the members are not added. How do I change the code to allow the creation of multiple members to a single array? The goal is to have each subnet's gateway field (column) be unique to that subnet.
The purpose of the five variables (variableA-E) is to mimic what is occuring in the real code. The real code runs comparisons from the hashtable, but that is not neccessary. I'm willing to change that portion if needed.
CSV file contents:
NetworkName,Subnet,VLANID,Gateway,VLAN
Servers,"192.168.1.0/24","2041","192.168.1.1","ServerVLAN-2041"
Workstations,"192.168.2.0/24","1001","192.168.2.1","WorkstationVLAN-1001"
DMZ,"172.16.0.0/28","340","172.16.0.1","DMZVLAN-340"
Servers,"192.168.3.0/24","2043","192.168.3.1","ServerVLAN-2043"
Workstations,"192.168.4.0/24","1004","192.168.4.1","WorkstationVLAN-1004"
DMZ,,,,
CODE:
$csvfile = "C:\temp\testfile.csv"
$hashArray = Import-CSV $csvfile
$variableA = "192.168.1.0"
$variableB = "192.168.2.0"
$variableC = "192.168.3.0"
$variableD = "172.16.0.1"
$variableE = "192.168.5.0"
$hashArray | % {
if ($_.subnet) { $variable = ($_.subnet).split("/")[0] }
Else { $variable = $null }
if ($variable -eq $variableA -and $variable -ne $null)
{
$_ | add-member "ServerGW1" -NotePropertyValue $_.gateway
Write-Host "Added Server gateway 1: "$_.gateway -ForegroundColor Yellow
}
if ($variable -eq $variableC -and $variable -ne $null)
{
$_ | add-member "ServerGW2" -NotePropertyValue $_.gateway
Write-Host "Added Server gateway 2: "$_.gateway -ForegroundColor Yellow
}
if ($variable -eq $variableB -and $variable -ne $null)
{
$_ | add-member "WorkstationGW1" -NotePropertyValue $_.gateway
Write-Host "Added Workstation gateway 1: "$_.gateway -ForegroundColor Yellow
}
if ($variable -eq $variableD -and $variable -ne $null)
{
$_ | add-member "DMZGW1" -NotePropertyValue $_.gateway
Write-Host "Added DMZ gateway 1: "$_.gateway -ForegroundColor Yellow
}
if ($variable -eq $variableE -and $variable -ne $null)
{
$_ | add-member "WorkstationGW2" -NotePropertyValue $_.gateway
Write-Host "Added Workstation gateway 2: "$_.gateway -ForegroundColor Yellow
}
}
$hashArray | Out-GridView
Out-GridView OUTPUT:
Console OUTPUT:
Expected output:

Out-GridView uses properties from the first object to render columns. All new columns (except ServerGW1) are missing because are not initialized in the first object in your $hashArray. You can initialize with $null value all properties for all rows or provide a list of properties to select before you output your result to Out-GridView
$hashArray | Select-Object NetworkName,Subnet,VLANID,Gateway,VLAN, ServerGW1, ServerGW2,WorkstationGW1,WorkstationGW2,DMZGW1 | Out-GridView
Init all properties:
$hashArray | % {
$variable =if ($_.subnet) { ($_.subnet).split("/")[0] }Else { $null }
$_ | add-member "ServerGW1" -NotePropertyValue $(if ($variable -eq $variableA){ $_.gateway}Else { $null })
$_ | add-member "ServerGW2" -NotePropertyValue $(if ($variable -eq $variableC){ $_.gateway}Else { $null })
$_ | add-member "WorkstationGW1" -NotePropertyValue $(if($variable -eq $variableB){ $_.gateway}Else { $null })
$_ | add-member "DMZGW1" -NotePropertyValue $(if ($variable -eq $variableD ){ $_.gateway}Else { $null })
$_ | add-member "WorkstationGW2" -NotePropertyValue $(if ($variable -eq $variableE){ $_.gateway}Else { $null })
}
$hashArray | Out-GridView

To complement cezarypiatek's helpful answer, which provides the crucial pointer:
All formatting cmdlets, including Out-GridView decide what properties (columns) to show based on the first input object, so to guarantee that the all columns of interest are shown, you must ensure that (at least) the first input object contains all properties of interest.
With that in mind, here's a streamlined version of your approach that does that:
$csvfile = "C:\temp\testfile.csv"
$networks = Import-CSV $csvfile
# Define the subnets and their property names as an ordered hashtable.
$subnets = [ordered] #{
'192.168.1.0' = 'ServerGW1'
'192.168.2.0' = 'ServerGW2'
'192.168.3.0' = 'WorkstationGW1'
'172.16.0.0' = 'DMZGW1'
'192.168.4.0' = 'WorkstationGW2'
}
# Add all properties of interest to the input objects, to ensure
# that Out-GridView (or other formatting cmdlets) show them all.
# Construct an array of property names, where '*' stands for the original properties...
$propNames = #('*') + [string[]] $subnets.Values
# ... and create augmented objects based on them.
$networks = $networks | Select-Object -property $propNames
$networks | % {
# See if the 'subnet' column has a value...
if ($subnet = if ($_.subnet) { ($_.subnet).split("/")[0] } else { $null }) {
# ... and, if so, see if a subnet name is defined for the part before '/' ...
if ($subnets.Contains($subnet)) {
# ... and, if so, fill the subnet-named property with the subnet address.
$_.($subnets.$subnet) = $subnet
}
}
}
Import-CSV doesn't return hashtables it returns custom objects ([pscustomobject] instances).
The code relies on the fact that, in the context of expressions, you can both assign to a variable and use the value of that assignment, such as in a conditional, as is the case here (if ($subnet = ...)).

try this, this example is dynamic, you have just to modify $hashvariable like you want (or load $hashvariable with file if you want)
$csvfile ="C:\temp\testfile.csv"
$hashArray = Import-CSV $csvfile
$hashvariable=[ordered]#{"192.168.1.0"="ServerGW1"; "192.168.2.0"="WorkstationGW1"; "192.168.3.0"="ServerGW2"; "172.16.0.1"="DMZGW1"; "192.168.5.0"="WorkstationGW2" }
$hashArray |
%{
$result=$_
$variable = if ($_.subnet -ne $null) {($_.subnet).split("/")[0]} else {""}
foreach ($key in $hashvariable.Keys)
{
$value=if ($variable -eq $key) {$key} else {""}
$result | add-member $hashvariable[$key] -NotePropertyValue $value
}
}
$hashArray | Out-GridView

Related

Check all lines in a huge CSV file in PowerShell

I want to work with a CSV file of more than 300,000 lines. I need to verify information line by line and then display it in a .txt file in the form of a table to see which file was missing for all servers. For example
Name,Server
File1,Server1
File2,Server1
File3,Server1
File1,Server2
File2,Server2
...
File345,Server76
File346,Server32
I want to display in table form this result which corresponds to the example above:
Name Server1 Server2 ... Server 32 ....Server 76
File1 X X
File2 X X
File3 X
...
File345 X
File346 X
To do this actually, I have a function that creates objects where the members are the Server Name (The number of members object can change) and I use stream reader to split data (I have more than 2 columns in my csv so 0 is for the Server name and 5 for the file name)
$stream = [System.IO.StreamReader]::new($File)
$stream.ReadLine() | Out-Null
while ((-not $stream.EndOfStream)) {
$line = $stream.ReadLine()
$strTempo = $null
$strTempo = $line -split ","
$index = $listOfFile.Name.IndexOf($strTempo[5])
if ($index -ne -1) {
$property = $strTempo[0].Replace("-", "_")
$listOfFile[$index].$property = "X"
}
else {
$obj = CreateEmptyObject ($listOfConfiguration)
$obj.Name = $strTempo[5]
$listOfFile.Add($obj) | Out-Null
}
}
When I export this I have a pretty good result. But the script take so much time (between 20min to 1hour)
I didn't know how optimize actually the script. I'm beginner to PowerShell.
Thanks for the futures tips
You might use HashSets for this:
$Servers = [System.Collections.Generic.HashSet[String]]::New()
$Files = #{}
Import-Csv -Path $Path |ForEach-Object {
$Null = $Servers.Add($_.Server)
if ($Files.Contains($_.Name)) { $Null = $Files[$_.Name].Add($_.Server) }
else { $Files[$_.Name] = [System.Collections.Generic.HashSet[String]]$_.Server }
}
$Table = foreach($Name in $Files.get_Keys()) {
$Properties = [Ordered]#{ Name = $Name }
ForEach ($Server in $Servers) {
$Properties[$Server] = if ($Files[$Name].Contains($Server)) { 'X' }
}
[PSCustomObject]$Properties
}
$Table |Format-Table -Property #{ expression='*' }
Note that in contrast to PowerShell's usual behavior, the .Net HashSet class is case-sensitive by default. To create an case-insensitive HashSet use the following constructor:
[System.Collections.Generic.HashSet[String]]::New([StringComparer]::OrdinalIgnoreCase)
See if this works faster. Change filename as required
$Path = "C:\temp\test1.txt"
$table = Import-Csv -Path $Path
$columnNames = $table | Select-Object -Property Server -Unique| foreach{$_.Server} | Sort-Object
Write-Host "names = " $columnNames
$groups = $table | Group-Object {$_.Name}
$outputTable = [System.Collections.ArrayList]#()
foreach($group in $groups)
{
Write-Host "Group = " $group.Name
$newRow = New-Object -TypeName psobject
$newRow | Add-Member -NotePropertyName Name -NotePropertyValue $group.Name
$servers = $group.Group | Select-Object -Property Server | foreach{$_.Server}
Write-Host "servers = " $servers
foreach($item in $columnNames)
{
if($servers.Contains($item))
{
$newRow | Add-Member -NotePropertyName $item -NotePropertyValue 'X'
}
else
{
#if you comment out next line code doesn't work
$newRow | Add-Member -NotePropertyName $item -NotePropertyValue ''
}
}
$outputTable.Add($newRow) | Out-Null
}
$outputTable | Format-Table

Powershell if condition returning no results when some exist

The code block below executes without the CSV being generated suggesting it has no return. However the conditions given are definitely valid for at least one object.
A Mail Enabled Security group in O365 has the "SecurityEnabled" property set to true, the "MailEnabled" property set to true and the property "GroupTypes" is an empty array? string? "{}" whatever the curly brackets are supposed to represent but they're empty.
$Groups = Get-MgGroup -Property "Members" -All
foreach($group in $Groups){
if ( (($group.SecurityEnabled) -and ($group.MailEnabled)) -and ($group.GroupTypes -ne 'Unified')){
$members = Get-MgGroupMember -GroupId $group.Id
[PSCustomObject]#{
'Display Name' = $group.DisplayName
'Members' = $members.AdditionalProperties.displayName -join ";"
} | Export-Csv -path "$OutputFolder\MailEnabledSecurityGroups.csv" -Append -NoTypeInformation
}
continue
}
The GroupTypes property from the MicrosoftGraphGroup instance is an array, in this case you want to use containment operators, more specifically -notin or -notcontains, the condition would look like:
# with `-notcontains`
if ($group.SecurityEnabled -and $group.MailEnabled -and $group.GroupTypes -notcontains 'Unified') {
# with `-notin`
if ($group.SecurityEnabled -and $group.MailEnabled -and 'Unified' -notin $group.GroupTypes) {
As for why it was not working before, -ne and -eq can act as a filter when the LHS (left hand side) of the comparison is an array and because $group.GroupTypes was an empty array, the comparison returned null and the if condition evaluated to false because of this.
$obj = [pscustomobject]#{
GroupTypes = #()
}
$obj.GroupTypes -ne 'Unified' # => null
$obj = [pscustomobject]#{
GroupTypes = #('something else')
}
$obj.GroupTypes -ne 'Unified' # => something else
As aside, it's probably a better idea to export the output all at once instead of appending to the CSV file on each loop iteration (Disk I/O is expensive and will slow down your script a lot!):
Get-MgGroup -Property "Members" -All | ForEach-Object {
if ($_.SecurityEnabled -and $_.MailEnabled -and 'Unified' -notin $_.GroupTypes) {
$members = Get-MgGroupMember -GroupId $_.Id
[PSCustomObject]#{
'Display Name' = $_.DisplayName
'Members' = $members.AdditionalProperties.displayName -join ";"
}
}
} | Export-Csv -path "$OutputFolder\MailEnabledSecurityGroups.csv" -NoTypeInformation
Filtering on Azure side may increase the speed of your code and also reduce the amount of conditions being evaluated on client side:
Get-MgGroup -Property "Members" -Filter "securityEnabled eq true and mailenabled eq true" | ForEach-Object {
if ('Unified' -notin $_.GroupTypes) {
# code here
}
}

Report confusion. without += Trying to deal with $null

I run an all users command for calendar delegation. I then report. The issue is how do I include someone that has no calendar delegation ? (Code Below)
In this line If ($null -ne $DelegateCal) I make sure someone has calendar delegation then build the object.
If I don't use += I am not sure how to build the object when I add a Else for the $null
<#All Google Calendar delegate report#>
# filename function
Import-Module C:\tasks\Modules\MRNAP\MRNAP.psm1
$AllGoogleUsers = gam print users fields suspended | ConvertFrom-Csv | Where-Object { $_.suspended -eq $False } | Select-Object -ExpandProperty PrimaryEmail
ForEach ($UserEmail in $AllGoogleUsers) {
$DelegateCal = gam calendar $UserEmail print acls | convertfrom-csv | Where-Object { $_.'scope.type' -eq 'user' -and $_.'Scope.value' -ne $UserEmail } -ErrorAction SilentlyContinue
If ($null -ne $DelegateCal) {
$CalendarDelegateList = foreach ($line in $DelegateCal) {
[PSCustomObject]#{
Owner = $line.calendarId
Type = 'Calendar'
Delegate = $line.'scope.value'
Role = $line.role
}
}
}
}
$CalendarDelegateList = $CalendarDelegateList | Sort-Object -Property Owner
$filename = MRNAP -ReportName WhoIsCalendarDelegated -Move
$CalendarDelegateLis | Export-Csv $filename -NoTypeInformation | Format-Table text-align=left -AutoSize
This is how I would do it with +=
$AllGoogleUsers = gam print users fields suspended | ConvertFrom-Csv | Where-Object { $_.suspended -eq $False } | Select-Object -ExpandProperty PrimaryEmail
ForEach ($UserEmail in $AllGoogleUsers) {
$DelegateCal = gam calendar $UserEmail print acls | convertfrom-csv | Where-Object { $_.'scope.type' -eq 'user' -and $_.'Scope.value' -ne $UserEmail } -ErrorAction SilentlyContinue
If ($null -ne $DelegateCal) {
foreach ($line in $DelegateCal) {
$CalendarDelegateList += [PSCustomObject]#{
Owner = $UserEmail
Type = 'Calendar'
Delegate = $line.'scope.value'
Role = $line.role
}
}
}
Else {
$CalendarDelegateList += [PSCustomObject]#{
Owner = $UserEmail
Type = 'Calendar'
Delegate = 'None'
Role = 'None'
}
}
}
It is always preferable to let PowerShell collect statement output in an array for you rather than building a list of outputs manually - both for concision and performance; see this answer for more information.
This even works with nested foreach loops, as in your case.
Applied to your scenario (abridged):
[array] $CalendarDelegateList =
foreach ($UserEmail in $AllGoogleUsers) {
$DelegateCal = gam calendar $UserEmail print acls | ConvertFrom-Csv | Where-Object { $_.'scope.type' -eq 'user' -and $_.'Scope.value' -ne $UserEmail } -ErrorAction SilentlyContinue
If ($null -ne $DelegateCal) {
foreach ($line in $DelegateCal) {
# Construct *and ouput* a [pscustomobject] instance.
[PSCustomObject]#{
Owner = $UserEmail
# ...
}
}
}
Else {
[PSCustomObject]#{
Owner = $UserEmail
# ...
}
}
}
All [pscustomobject] instances (implicitly) output from inside the foreach loop (whether directly or from the nested one) are automatically collected in variable $CalendarDelegateList.
Note:
With two or more output objects from the loop, the $CalendarDelegateList variable receives a regular PowerShell array (of type [object[]]).
The [array] type constraint (short for: [object[]]) additionally ensures that the result is an array even if the loop outputs only one object.

Powershell - If Statement Output 2 values

Iam trying to write a script to notify if a VMware VM Custom attribute has a value or if the value is null.
I need the VM name and the Output Value (either Null or Not Null). Here is what I have but doesn't return the accurate information
$vms = Get-VM
foreach ($vm in $vms) {
$tag = $vm | Get-Annotation -CustomAttribute "Backup"
if ($tag.value -eq '$null'){
Write-Output "$vm Attribute doesnt have a value"
}
else {
Write-Output "$vm Attribute has a value assigned"
}
}
Unless you're specifically looking for the literal string value '$null', you probably want to change the comparison to $null -eq $tag.value
You could create a new object with 2 properties:
$vms = Get-VM
foreach ($vm in $vms) {
$tag = $vm | Get-Annotation -CustomAttribute "Backup"
if ($null -eq $tag.value){
$result = "$vm Attribute doesnt have a value"
}
else {
$result = Write-Output "$vm Attribute has a value assigned"
}
# output object with Name + result
[pscustomobject]#{
Name = $vm.Name
Result = $result
}
}
Another, perhaps more PowerShell-idiomatic approach would be to create a similar object with the Select-Object cmdlet:
Get-VM |Select-Object Name,#{Name='HasBackupAttribute';Expression={ $null -eq ($_ | Get-Annotation -CustomAttribute "Backup").Value }}
When looking at dealing with $null comparisons, see this SO Q&A:
In PowerShell, why is $null -lt 0 = $true? Is that reliable?
I do not have VMware at the moment, but in my small Hyper-V lab, running the following delivers the shown results:
Try
{
(
Get-VM |
Where-Object -Property FloppyDrive -eq $null |
Select Name, FloppyDrive -ErrorAction SilentlyContinue
).Count
}
Catch { Write-Warning -Message 'No no records returned'}
# Results
<#
4
#>
Try
{
(
Get-VM |
Where-Object -Property FloppyDrive -ne $null |
Select Name, FloppyDrive -ErrorAction SilentlyContinue
).Count
}
Catch { Write-Warning -Message 'No no records returned'}
# Results
<#
WARNING: No no records returned
#>
The results are the same using these as well.
Try
{
(
Get-VM |
Where-Object -Property FloppyDrive -Match $null |
Select Name, FloppyDrive -ErrorAction SilentlyContinue
).Count
}
Catch { Write-Warning -Message 'No no records returned'}
Try
{
(
Get-VM |
Where-Object -Property FloppyDrive -NotMatch $null |
Select Name, FloppyDrive -ErrorAction SilentlyContinue
).Count
}
Catch { Write-Warning -Message 'No no records returned'}
In your use case, try this refactor:
(Get-VM).Name |
foreach {
if ((Get-VM -Entity $PSitem | Get-Annotation -CustomAttribute 'Backup') -eq $null)
{ "$PSItem - Attribute doesnt have a value" }
else {"$PSItem - Attribute has a value assigned"}
}
Or...
(Get-VM).Name |
foreach {
if ((Get-VM -Entity $PSitem | Get-Annotation -CustomAttribute 'Backup') -Match $null)
{ "$PSItem - Attribute doesnt have a value" }
else {"$PSItem - Attribute has a value assigned"}
}
Annotation values are strings. When there is no value present, the string is considered empty rather than null. So the $null -eq value test will not yield the desired results. You can simply perform an implicit boolean conversion of the value to determine if it is empty.
$vms = Get-VM
foreach ($vm in $vms) {
$tag = $vm | Get-Annotation -CustomAttribute "Backup"
# Empty string returns false. Nonempty string returns true.
if ($tag.value){
Write-Output "$vm Attribute has a value assigned"
}
else {
Write-Output "$vm Attribute doesn't have a value"
}
}
You will discover that $tag.value | Get-Member returns a type System.String. So when value seemingly has no value, it actually is an empty string. You can perform a variety of tests to determine if the value is empty. An empty string value inside of an if statement evaluates to False. See below for some examples, which can all be used inside of if statements.
$Empty = [pscustomobject]#{value = ''}
$NotEmpty = [pscustomobject]#{value = 'My String'}
# Test 1 Using IsNullOrEmpty static method
[string]::IsNullOrEmpty($Empty.value)
True
[string]::IsNullOrEmpty($NotEmpty.value)
False
# Test 2 Using [bool] type accelerator
[bool]$Empty.value
False
[bool]$NotEmpty.value
True
# Test 3 Using length property. Empty string has length of 0.
$Empty.value.length -eq 0
True
$NotEmpty.value.length -eq 0
False
# Test 4 Using if statement. ! in front of an expression negates its boolean value
if ($Empty.value) { "Value is not empty" } else { "Value is empty" }
Value is empty
if ($NotEmpty.value) { "Value is not empty" } else { "Value is empty" }
Value is not empty
if (!$Empty.value) { "Value is empty" }
Value is empty

PowerShell Hash table updates apply to all entries

I am trying to update an hash table entry (Content, which is an array) where I find values - but when I try to set the value for $_ it applies to all entries in the entire hash ($hash).
$ContentArray = #($null, $null)
$Comparison | Add-Member -MemberType NoteProperty -Name "Content" -value $ContentArray
If($GetAdvancedData -eq "true"){
$Hash| ForEach-Object{
If ($_.VarianceType -ne "Missing")
{
$_.Item
$id = $_.id[0]
$elem = $_.elementName
Write-debug "Checking $elem"
Write-debug "Checking for version $id"
try{
$content = Get-VersionContent -FilteredVersionID $id
Write-debug "Content found"
Write-debug "$content"
# Various tests to try and set the value:
#ForEach ($Key in $Hash.GetEnumerator() | Where-Object {$_.id -eq $id}){$Key.Content[0] = $content}
#$Hash.Content[0] = "$content" | Where-Object {$Hash.id[0] -eq $id}
#$Hash.DiffType = "Content"| Where-Object {$_.id[0] -eq $id}
# $_.content.SetValue("$content","0")
$_.Content[0] = $content
# Reset $content to Null
$content = $null
}
catch
{
Write-debug "No content found"
}
}
}
I have tried setting it via a where clause based on another key value, using SetValue, and simply doing an = statement, but in each case it sets the entire hash tables content to $content - I feel as if I must be missing something obvious, but I can't see why (if I use the PowerShell ISE and debug $_ returns only the single record from the ForEach loop)
A silly mistake-
$Hash.GetEnumerator() | Where-Object {$_.id[0] -eq $id} | foreach{$_.Content = $content,$null}
did the trick for me!