How do you export objects with a varying amount of properties? - powershell

Warning - I've asked a similar question in the past but this is slightly different.
tl;dr; I want to export objects which have a varying number of properties. eg; object 1 may have 3 IP address and 2 NICs but object 2 has 7 IP addresses and 4 NICs (but not limited to this amount - it could be N properties).
I can happily capture and build objects that contain all the information I require. If I simply output my array to the console each object is shown with all its properties. If I want to out-file or export-csv I start hitting a problem surrounding the headings.
Previously JPBlanc recommended sorting the objects based on the amount of properties - ie, the object with the most properties would come first and hence the headings for the most amount of properties would be output.
Say I have built an object of servers which has varying properties based on IP addresses and NIC cards. For example;
ServerName: Mordor
IP1: 10.0.0.1
IP2: 10.0.0.2
NIC1: VMXNET
NIC2: Broadcom
ServerName: Rivendell
IP1: 10.1.1.1
IP2: 10.1.1.2
IP3: 10.1.1.3
IP4: 10.1.1.4
NIC1: VMXNET
Initially, if you were to export-csv an array of these objects the headers would be built upon the first object (aka, you would only get ServerName, IP1, IP2, NIC1 and NIC2) meaning for the second object you would lose any subsequent IPs (eg IP3 and IP4). To correct this, before an export I sort based on the number of IP properties - tada - the first object now has the most IPs in the array and hence none of the subsequent objects IPs are lost.
The downside is when you then have a second varying property - eg NICs. Once my sort is complete based on IP we then have the headings ServerName, IP1 - IP4 and NIC1. This means the subsequent object property of NIC2 is lost.
Is there a scalable way to ensure that you aren't losing data when exporting objects like this?

Try:
$o1 = New-Object psobject -Property #{
ServerName="Mordor"
IP1="10.0.0.1"
IP2="10.0.0.2"
NIC1="VMXNET"
NIC2="Broadcom"
}
$o2 = New-Object psobject -Property #{
ServerName="Rivendell"
IP1="10.1.1.1"
IP2="10.1.1.2"
IP3="10.1.1.3"
IP4="10.1.1.4"
NIC1="VMXNET"
}
$arr = #()
$arr += $o1
$arr += $o2
#Creating output
$prop = $arr | % { Get-Member -InputObject $_ -MemberType NoteProperty | Select -ExpandProperty Name } | Select -Unique | Sort-Object
$headers = #("ServerName")
$headers += $prop -notlike "ServerName"
$arr | ft -Property $headers
Output:
ServerName IP1 IP2 IP3 IP4 NIC1 NIC2
---------- --- --- --- --- ---- ----
Mordor 10.0.0.1 10.0.0.2 VMXNET Broadcom
Rivendell 10.1.1.1 10.1.1.2 10.1.1.3 10.1.1.4 VMXNET
If you know the types(NICS, IPS..), but not the count(ex. how many NICS) you could try:
#Creating output
$headers = $arr | % { Get-Member -InputObject $_ -MemberType NoteProperty | Select -ExpandProperty Name } | Select -Unique
$ipcount = ($headers -like "IP*").Count
$niccount = ($headers -like "NIC*").Count
$format = #("ServerName")
for ($i = 1; $i -le $ipcount; $i++) { $format += "IP$i" }
for ($i = 1; $i -le $niccount; $i++) { $format += "NIC$i" }
$arr | ft -Property $format

What about getting a list of all unique property headers and then doing a select on all the objects? When you do a select on an object for a nonexistent property it will create a blank one.
$allHeaders = $arrayOfObjects | % { Get-Member -inputobject $_ -membertype noteproperty | Select -expand Name } | Select -unique
$arrayOfObjects | Select $allHeaders
Granted you are looping through ever object to get the headers, so for a very large amount of objects it may take awhile.

Here's my attempt at a solution. I'm very tired now so hopefully it makes sense. Basically I'm calculating the largest amount of NIC and IP note properties, creating a place holder object that has those amounts of properties, adding it as the first item in a CSV, and then removing it from the CSV.
# Create example objects
$o1 = New-Object psobject -Property #{
ServerName="Mordor"
IP1="10.0.0.1"
IP2="10.0.0.2"
NIC1="VMXNET"
NIC2="Broadcom"
}
$o2 = New-Object psobject -Property #{
ServerName="Rivendell"
IP1="10.1.1.1"
IP2="10.1.1.2"
IP3="10.1.1.3"
IP4="10.1.1.4"
NIC1="VMXNET"
}
# Add to an array
$servers = #($o1, $o2)
# Calculate how many IP and NIC properties there are
$IPColSize = ($servers | Select IP* | %{($_ | gm -MemberType NoteProperty).Count} | Sort-Object -Descending)[0]
$NICColSize = ($servers | Select NIC* | %{($_ | gm -MemberType NoteProperty).Count} | Sort-Object -Descending)[0]
# Build a place holder object that will contain enough properties to cover all of the objects in the array.
$cmd = '$placeholder = "" | Select ServerName, {0}, {1}' -f (#(1..$IPColSize | %{"IP$_"}) -join ", "), (#(1..$NICColSize | %{"NIC$_"}) -join ", ")
Invoke-Expression $cmd
# Convert to CSV and remove the placeholder
$csv = $placeholder,$servers | %{$_ | Select *} | ConvertTo-Csv -NoTypeInformation
$csv | Select -First 1 -Last ($csv.Count-2) | ConvertFrom-Csv | Export-Csv Solution.csv -NoTypeInformation

Related

matching data across two arrays and combining with additional data in array

The Goal
See if $SP.ip is in $NLIP.IpRanges and if it is, add $NLIP.IpRanges and $NLIP.DisplayName to the $SP array or all into a new array.
The Arrays
Array 1 is $SP, it's a CSV import and has the properties 'name' and 'ip', it looks like this:
name: bob
ip: 1.9.8.2
Array 2 is $NLIP and has the relevant properties 'IpRanges' and 'DisplayName'. It's fetched from: $NLIP = Get-AzureADMSNamedLocationPolicy | where-object {$_.OdataType -eq "#microsoft.graph.ipNamedLocation"}, it looks like this:
DisplayName : Named Location 1
IpRanges : {class IpRange {
CidrAddress: 16.29.28.9/28 #fictitious CIDR
}
, class IpRange {
CidrAddress: 1.9.8.3/28 #fictitious CIDR
}
}
The Code / the problem
I'm using IPInRange.ps1 function from https://github.com/omniomi/PSMailTools to find if the IP is in the range. It works like so:
> IPInRange 1.9.8.2 1.9.8.3/28
True
I also worked out that $NLTP.IpRanges.split() | Where-Object ($_ -like "*/*"} can return all the ranges, but $NLIP | Where-Object {$_.IpRanges.split() -like "*/*"} doesn't. I would naturally use the second to keep the variable in the pipe to return the DisplayName. So I'm struggling on how to pull the individual ranges out in such a way that I can then add the 'IpRange' and 'DisplayName' to an array.
Also, maybe it's because I haven't worked out the above issue, but I'm struggling to think how I would iterate through both arrays and combine them into one. I know I would probably enter into a foreach ($item in $SP) and create a temporary array, but after that it's getting hazy.
The result
What I'm hoping to have in the end is:
name: bob
ip: 1.9.8.2
IpRange: 1.9.8.3/28 #fictitious CIDR
DisplayName: Named Location 1
thanks in advance.
I believe this will work for you if I understood the NLIP construct correctly.
We will loop through all the SP objects and see if we can find any NLIP that match the IP range using the IPinRange function you linked. We will then add the 2 properties you want to the SP object if matched and finally pass thru to the pipeline or you can append | export-csv -path YourPath to the end if you would like to send to a csv file
$SP | ForEach-Object {
$target = $_
$matched = $NLIP | ForEach-Object {
$item = $_
# Using where to single out matching range using IPinRange function
$_.IpRanges.Where({ IPInRange -IPAddress $target.ip -Range $_.CidrAddress }) |
ForEach-Object {
# for matching range output custom object containing the displayname and iprange
[PSCustomObject]#{
DisplayName = $item.DisplayName
IpRange = $_.CidrAddress
}
}
}
# add the 2 properties (DisplayName and IpRange) from the match to the original $SP
# object and then pass thru
$target | Add-Member -NotePropertyName DisplayName -NotePropertyValue $matched.DisplayName
$target | Add-Member -NotePropertyName IpRange -NotePropertyValue $matched.IpRange -PassThru
}
By the way, this is how I envisioned the NLIP objects and what I tested with
$NLIP = #(
[pscustomobject]#{
DisplayName = 'Named location 1'
IpRanges = #(
[pscustomobject]#{
CidrAddress = '16.29.28.9/28'
},
[pscustomobject]#{
CidrAddress = '1.9.8.3/28'
}
)
},
[pscustomobject]#{
DisplayName = 'Named location 2'
IpRanges = #(
[pscustomobject]#{
CidrAddress = '16.29.28.25/28'
},
[pscustomobject]#{
CidrAddress = '1.9.8.25/28'
}
)
}
)
Let's to shed some lights in the hazy darkness by first creating a Minimal, Reproducible Example (mcve):
$SP = ConvertFrom-Csv #'
IP, Name
1.9.8.2, BOB
10.10.10.10, Apple
16.29.28.27, Pear
16.30.29.28, Banana
'#
$NLIP = ConvertFrom-Csv #'
IPRange, SubNet
16.29.28.9/28, NetA
1.9.8.3/28, NetB
'#
To tackle this, you need two loops where the second loop is inside the first loop. For the outer loop you might use the ForEach-Object cmdlet which lets you stream each object and with that actually use less memory (assuming that you import the data from a file and eventually export it to a new file). Within the inner loop you might than cross link each IP address with the IPRange using the function you refer to and in case the condition is true create a new PSCustomObject:
$SP |ForEach-Object { # | Import-Csv .\SP.csv |ForEach-Object { ...
ForEach($SubNet in $NLIP) {
if (IPInRange $_.IP $SubNet.IPRange) {
[PSCustomObject]#{
IP = $_.IP
Name = $_.Name
IPRange = $SubNet.IPRange
SubNet = $SubNet.SubNet
}
}
}
} # | Export-Csv .\Output.csv
Which results in:
IP Name IPRange SubNet
-- ---- ------- ------
1.9.8.2 BOB 1.9.8.3/28 NetB
16.29.28.27 Pear 16.29.28.9/8 NetA
16.30.29.28 Banana 16.29.28.9/8 NetA
But as you are considering 3rd party scripts anyways, you might as well use this Join-Object script/Join-Object Module (see also: In Powershell, what's the best way to join two tables into one?):
$SP |Join $NLIP -Using { IPInRange $Left.IP $Right.IPRange }
Which gives the same results.

Format-Table not taking effect (Exchange - powershell)

first of all sorry if my english is not the best. but ill try to explain my issue with as much detail as i can
Im having an issue where i cant get Format-Table to effect the output i give it.
below is the part im having issues with atm.
cls
$TotalSize = $($mailboxes. #{name = ”TotalItemSize (GB)”; expression = { [math]::Round((($_.TotalItemSize.Value.ToString()).Split(“(“)[1].Split(” “)[0].Replace(“,”, ””) / 1GB), 2) } });
$UserN = $($mailboxes.DisplayName)
$itemCount = $($mailboxes.ItemCount)
$LastLogonTime = $($mailboxes.ItemCount)
$allMailboxinfo = #(
#lager dataen som skal inn i et objekt
#{Username= $UserN; ItemCount = $itemCount; LastLogonTime = $($mailboxes.ItemCount); Size = $TotalSize}) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
$Table = $allMailboxinfo | Format-Table | Out-String
$Table
the output of this gives me what almost looks like json syntax below each title of the table.
Username LastLogonTime ItemCount Size
-------- ------------- --------- ----
{username1, username2,username3,userna...} {$null, $null, $null, $null...} {$null, $null, $null, $null...} {$null, $null, $null, $null...}
running the commands by themselves seem to work tho. like $mailboxes.DisplayName gives the exact data i want for displayname. even in table-format.
the reason im making the table this way instead of just using select-object, is because im going to merge a few tables later. using the logic from the script below.
cls
$someData = #(
#{Name = "Bill"; email = "email#domain.com"; phone = "12345678"; id = "043546" }) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
$moreData = #(
#{Name = "Bill"; company = "company 04"}) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
$Merge = #(
#plots the data into a new table
#{Name = $($someData.Name); e_mail = $($someData.email); phone = $($someData.phone); id = $($someData.id); merged = $($moreData.company) }) | % { New-Object object | Add-Member -NotePropertyMembers $_ -PassThru }
#formatting table
$Table = $Merge | Format-Table | Out-String
#print table
$Table
if you are wondering what im doing with this.
My goal, all in all. is a table with using the info from Exchange;
DisplayName, TotalItemSize(GB), ItemCount, LastLogonTime, E-mail adress, archive + Maxquoata, Quoata for mailbox.
You're creating a single object where each property holds an array of property values from the original array of mailbox objects.
Instead, create 1 new object per mailbox:
# construct output objects with Select-Object
$allMailBoxInfo = $mailboxes |Select #{Name='Username';Expression='DisplayName'},ItemCount,#{Name='LastLogonTime';Expression='ItemCount'},#{Name='Size';Expression={[math]::Round((($_.TotalItemSize.Value.ToString()).Split("(")[1].Split(" ")[0].Replace(",", "") / 1GB), 2) }}
# format table
$Table = $allMailBoxInfo | Format-Table | Out-String
# print table
$Table

Count the number of columns in an array with PowerShell

I have the following code:
$array = Get-Process
Now, I want to count the number of columns. I have tried:
$array.Columns.Count
$array.Columns
And all without luck.
What should I do to get the correct number of columns? That should be 8.
<<Added on January, 27th, 2021: >>
I have modified the code to:
$Processes = #(Get-Process | Select-Object -Property Name, Description, ProductVersion, #{Name="Application";Expression={$_.Description + " " + $_.ProductVersion}},#{Name="Executable";Expression={(($_."Path").split("\"))[-1]}} |Where {$_.Executable})
$Array = #()
$Record = [ordered] #{"Name" = "";
"Application" = ""}
ForEach ($ProcesName in $Processes)
{
$Record."Name" = $ProcesName.Name
$Record."Application" = $ProcesName.Application
$objRecord = New-Object PSObject -Property $Record
$Array += $objRecord
}
Clear-Host
Write-Host "Number of columns: $($Array.Column.Count)"
Write-Host "Number rows: $($Array.Count)"
The result is the same: both the column and row are the same. And that is not correct. I want to use the technique in a new script with much more columns.
<<End part that has been added on January, 27th, 2021 >>
Feedback is appreciated.
With kind regards,
TheStingPilot
$array is an array of objects of type System.Diagnostics.Process. There aren't columns as such. The objects have properties which you may choose to display in columns using $array | ft or as a list $array | fl *. You can count the properties like this:
$array | Get-Member -MemberType Property | measure
If you'd like the answer to be 8 then you can do this:
$array | select Handles, NPM, PM, WS, CPU, Id, SI, Name | Get-Member -MemberType NoteProperty | measure

Select all unique third octet in a list of IP addresses with PowerShell

I have a list of IP addresses. They all start with 10.10. I want all the unique values of the third octet. This way I can count how many of that unique value there are.
10.10.26.251
10.10.27.221
10.10.26.55
10.10.31.12
10.10.12.31
10.10.31.11
10.10.27.15
10.10.26.5
When I am done I want to know that I have 3 .26 network devices, 2 27, and so on so forth. Other than breaking down the octet with a split and looping through each one, I can't think of any single liners. Any suggestions?
here's a small variant. [grin] i already had this before noticing the other answers - and it is a tad different.
what it does ...
creates a collection of IPv4 address objects to work with
groups them by a calculated property [the 3rd octet]
creates a [PSCustomObject] for each resulting group
sends it to the $Octet3_Report variable
shows it on screen
output to a CSV file would be easy at that point. here's the code ...
$IP_List = #(
[ipaddress]'10.10.26.251'
[ipaddress]'10.10.27.221'
[ipaddress]'10.10.26.55'
[ipaddress]'10.10.31.12'
[ipaddress]'10.10.12.31'
[ipaddress]'10.10.31.11'
[ipaddress]'10.10.27.15'
[ipaddress]'10.10.26.5'
)
$Octet3_Report = $IP_List |
Group-Object -Property {$_.ToString().Split('.')[2]} |
ForEach-Object {
[PSCustomObject]#{
Octet_3 = $_.Name
Count = $_.Count
}
}
$Octet3_Report
on screen output ...
Octet_3 Count
------- -----
26 3
27 2
31 2
12 1
It's like me to figure it out after the fact.
The Return contains the dns records. The IP address are stored inside recorddata. I pull the end of the IP address off. Then loop through grabbing only the range and count with a foreach loop to make it cleaner.
$DNSRecordCounts = #()
$Ranges = ($Return | where-object {$_.recorddata -like "10.10.*"}).recorddata -replace "\.\d{1,3}$" | select -Unique
foreach ($range in $Ranges) {
$DNSRecordCounts += [pscustomobject][ordered]#{
IPRange = $range
Count = ($Return | Where-Object {$_.recorddata -like "$($range).*"}).Count
}
}
Based on your question and what I can infer from your own answer, if you are looking for something a little more like "idiomatic" PowerShell you want the following:
$Return `
| Select-Object -ExpandProperty recorddata `
| ForEach-Object {
$_ -match "\d+\.\d+\.(?<octet>\d+)\.\d+" | Out-Null
$Matches.octet
} `
| Group-Object `
| ForEach-Object {
[PSCustomObject]#{
Octet = $_.Name
Count = $_.Count
}
}

Seeking balanced combination of fast, terse, and legible code to add up values from an array of objects

Given the following array of objects:
Email Domain Tally
----- ----- -----
email1#domainA.com domainA.com 4
email1#domainB.com domainB.com 1
email2#domainC.com domainC.com 6
email4#domainA.com domainA.com 1
I'd like to "group by" Domain and add up Tally as I go. The end result would like this:
Domain Tally
------ -----
domainA.com 5
domainB.com 1
domainC.com 6
I have something that works but I feel like it's overly complicated.
$AllTheAddresses = Get-AllTheAddresses
$DomainTally = #()
foreach ($Addy in $AllTheAddresses)
{
if ($DomainTally | Where-Object {$_.RecipientDomain -eq $Addy.RecipientDomain})
{
$DomainTally |
Where-Object {$_.RecipientDomain -eq $Addy.RecipientDomain} |
ForEach-Object {$_.Tally += $Addy.Tally }
}
else
{
$props = #{
RecipientDomain = $Addy.RecipientDomain
Tally = $Addy.Tally
}
$DomainTally += New-Object -TypeName PSObject -Property $props
}
}
In my example, I'm creating the addresses as hashtables, but PowerShell will let you refer to the keys by .Property similar to an object.
If you're truly just summing by the Domain, then it seems like you don't need anything more complicated than a HashTable to create your running total.
The basic summation:
$Tally = #{}
$AllTheAddresses | ForEach-Object {
$Tally[$_.Domain] += $_.Tally
}
Using this sample data...
$AllTheAddresses = #(
#{ Email = "email1#domainA.com"; Domain = "domainA.com"; Tally = 4 };
#{ Email = "email1#domainB.com"; Domain = "domainB.com"; Tally = 1 };
#{ Email = "email1#domainC.com"; Domain = "domainC.com"; Tally = 6 };
#{ Email = "email1#domainA.com"; Domain = "domainA.com"; Tally = 1 }
)
And you get this output:
PS> $tally
Name Value
---- -----
domainC.com 6
domainB.com 1
domainA.com 5
Here is a "PowerShellic" version, notice the piping and flow of the data.
You could of course write this as a one liner (I did originally before I posted the answer here). The 'better' part of this is using the Group-Object and Measure-Object cmdlets. Notice there are no conditionals, again because the example uses the pipeline.
$AllTheAddresses |
Group-Object -Property Domain |
ForEach-Object {
$_ |
Tee-Object -Variable Domain |
Select-Object -Expand Group |
Measure-Object -Sum Tally |
Select-Object -Expand Sum |
ForEach-Object {
New-Object -TypeName PSObject -Property #{
'Domain' = $Domain.Name
'Tally' = $_
}
} |
Select-Object Domain, Tally
}
A more terse version
$AllTheAddresses |
Group Domain |
% {
$_ |
Tee-Object -Variable Domain |
Select -Expand Group |
Measure -Sum Tally |
Select -Expand Sum |
% {
New-Object PSObject -Property #{
'Domain' = $Domain.Name
'Tally' = $_
}
} |
Select Domain, Tally
}
Group-Object is definitely the way to go.
In the interest of terseness:
Get-AllTheAddresses |Group-Object Domain |Select-Object #{N='Domain';E={$_.Name}},#{N='Tally';E={($_.Group.Tally |Measure-Object).Sum}}