matching data across two arrays and combining with additional data in array - powershell

The Goal
See if $SP.ip is in $NLIP.IpRanges and if it is, add $NLIP.IpRanges and $NLIP.DisplayName to the $SP array or all into a new array.
The Arrays
Array 1 is $SP, it's a CSV import and has the properties 'name' and 'ip', it looks like this:
name: bob
ip: 1.9.8.2
Array 2 is $NLIP and has the relevant properties 'IpRanges' and 'DisplayName'. It's fetched from: $NLIP = Get-AzureADMSNamedLocationPolicy | where-object {$_.OdataType -eq "#microsoft.graph.ipNamedLocation"}, it looks like this:
DisplayName : Named Location 1
IpRanges : {class IpRange {
CidrAddress: 16.29.28.9/28 #fictitious CIDR
}
, class IpRange {
CidrAddress: 1.9.8.3/28 #fictitious CIDR
}
}
The Code / the problem
I'm using IPInRange.ps1 function from https://github.com/omniomi/PSMailTools to find if the IP is in the range. It works like so:
> IPInRange 1.9.8.2 1.9.8.3/28
True
I also worked out that $NLTP.IpRanges.split() | Where-Object ($_ -like "*/*"} can return all the ranges, but $NLIP | Where-Object {$_.IpRanges.split() -like "*/*"} doesn't. I would naturally use the second to keep the variable in the pipe to return the DisplayName. So I'm struggling on how to pull the individual ranges out in such a way that I can then add the 'IpRange' and 'DisplayName' to an array.
Also, maybe it's because I haven't worked out the above issue, but I'm struggling to think how I would iterate through both arrays and combine them into one. I know I would probably enter into a foreach ($item in $SP) and create a temporary array, but after that it's getting hazy.
The result
What I'm hoping to have in the end is:
name: bob
ip: 1.9.8.2
IpRange: 1.9.8.3/28 #fictitious CIDR
DisplayName: Named Location 1
thanks in advance.

I believe this will work for you if I understood the NLIP construct correctly.
We will loop through all the SP objects and see if we can find any NLIP that match the IP range using the IPinRange function you linked. We will then add the 2 properties you want to the SP object if matched and finally pass thru to the pipeline or you can append | export-csv -path YourPath to the end if you would like to send to a csv file
$SP | ForEach-Object {
$target = $_
$matched = $NLIP | ForEach-Object {
$item = $_
# Using where to single out matching range using IPinRange function
$_.IpRanges.Where({ IPInRange -IPAddress $target.ip -Range $_.CidrAddress }) |
ForEach-Object {
# for matching range output custom object containing the displayname and iprange
[PSCustomObject]#{
DisplayName = $item.DisplayName
IpRange = $_.CidrAddress
}
}
}
# add the 2 properties (DisplayName and IpRange) from the match to the original $SP
# object and then pass thru
$target | Add-Member -NotePropertyName DisplayName -NotePropertyValue $matched.DisplayName
$target | Add-Member -NotePropertyName IpRange -NotePropertyValue $matched.IpRange -PassThru
}
By the way, this is how I envisioned the NLIP objects and what I tested with
$NLIP = #(
[pscustomobject]#{
DisplayName = 'Named location 1'
IpRanges = #(
[pscustomobject]#{
CidrAddress = '16.29.28.9/28'
},
[pscustomobject]#{
CidrAddress = '1.9.8.3/28'
}
)
},
[pscustomobject]#{
DisplayName = 'Named location 2'
IpRanges = #(
[pscustomobject]#{
CidrAddress = '16.29.28.25/28'
},
[pscustomobject]#{
CidrAddress = '1.9.8.25/28'
}
)
}
)

Let's to shed some lights in the hazy darkness by first creating a Minimal, Reproducible Example (mcve):
$SP = ConvertFrom-Csv #'
IP, Name
1.9.8.2, BOB
10.10.10.10, Apple
16.29.28.27, Pear
16.30.29.28, Banana
'#
$NLIP = ConvertFrom-Csv #'
IPRange, SubNet
16.29.28.9/28, NetA
1.9.8.3/28, NetB
'#
To tackle this, you need two loops where the second loop is inside the first loop. For the outer loop you might use the ForEach-Object cmdlet which lets you stream each object and with that actually use less memory (assuming that you import the data from a file and eventually export it to a new file). Within the inner loop you might than cross link each IP address with the IPRange using the function you refer to and in case the condition is true create a new PSCustomObject:
$SP |ForEach-Object { # | Import-Csv .\SP.csv |ForEach-Object { ...
ForEach($SubNet in $NLIP) {
if (IPInRange $_.IP $SubNet.IPRange) {
[PSCustomObject]#{
IP = $_.IP
Name = $_.Name
IPRange = $SubNet.IPRange
SubNet = $SubNet.SubNet
}
}
}
} # | Export-Csv .\Output.csv
Which results in:
IP Name IPRange SubNet
-- ---- ------- ------
1.9.8.2 BOB 1.9.8.3/28 NetB
16.29.28.27 Pear 16.29.28.9/8 NetA
16.30.29.28 Banana 16.29.28.9/8 NetA
But as you are considering 3rd party scripts anyways, you might as well use this Join-Object script/Join-Object Module (see also: In Powershell, what's the best way to join two tables into one?):
$SP |Join $NLIP -Using { IPInRange $Left.IP $Right.IPRange }
Which gives the same results.

Related

Powershell Pipeline - return a new Object, that was created within pipline

I keep running into the same problem again, and i have my default way of handling it, but it keeps bugging me.
Isn't there any better way?
So basicly i have a pipline running, do stuff within the pipline, and want to return a Key/Value Pair from within the pipline.
I want the whole pipline to return a object of type psobject (or pscustomobject).
Here is the way i do it everytime.
I create a hashtable at the beginning of the pipline and add key/Value Pairs from within the pipline to this hashtable using the .Add() method.
Afterwards i create a psobject by passing the hashtbale to New-Object`s -Property Parameter. This gives me the desired result.
Get-Process | Sort -Unique Name | ForEach-Object -Begin { $ht = #{} } -Process {
# DO STUFF
$key = $_.Name
$val = $_.Id
# Add Entry to Hashtable
$ht.Add($key,$val)
}
# Create PSObject from Hashtable
$myAwesomeNewObject = New-Object psobject -Property $ht
# Done - returns System.Management.Automation.PSCustomObject
$myAwesomeNewObject.GetType().FullName
But this seems a bit cluncky, isn't there a more elegant way of doing it?
Something like this:
[PSObject]$myAwesomeNewObject = Get-Process | Sort -Unique Name | ForEach-Object -Process {
# DO STUFF
$key = $_.Name
$val = $_.Id
# return Key/Val Pair
#{$key=$val}
}
# Failed - returns System.Object[]
$myAwesomeNewObject.GetType().FullName
This unfortunally dosn't work, since the pipe returns an array of hashtables, but i hope you know now what iam trying to achieve.
Thanks
Not sure if this is more elegant but just another way of doing it, this uses an anonymous function so $ht will no longer be available after execution, and casts to [pscustomobject] instead of using New-Object:
[pscustomobject] (Get-Process | Sort -Unique Name | & {
begin { $ht = #{ } }
process {
# DO STUFF
$key = $_.Name
$val = $_.Id
# Add Entry to Hashtable
$ht.Add($key, $val)
}
end { $ht }
})
You can also use the -End parameter to convert the final hash table to a pscustomobject as part of the pipeline, without needing to set the whole thing to a variable
$ht[$key]=$val is also a nice shorthand for $ht.Add($key,$val):
Get-Process |
Sort -Unique Name |
Foreach -Begin { $ht = #{} } -Process {
$ht[$_.Name] = $_.Id
} -End {[pscustomobject]$ht} |
## continue pipeline with pscustomobject
Thanks to #Santiago Squarzon and #Cpt.Whale answers, i were able to combine them to create a solution that pleases me:
$myAwesomeNewObject = `
Get-Process | Sort -Unique Name | & {
begin { $ht = #{} }
process {
# DO STUFF
$key = $_.Name
$val = $_.Id
# Add Entry to Hashtable
$ht[$key]=$val
}
end {[pscustomobject]$ht}
}
# Success - System.Management.Automation.PSCustomObject
$myAwesomeNewObject.Gettype().FullName
# And helper Hashtable is NULL thanks to the
# anonym function
$null -eq $ht
Thanks alot Guys
Alternatively you may create a hashtable using Group-Object -AsHashTable:
# Store the PIDs of all processes into a PSCustomObject, keyed by the process name
$processes = [PSCustomObject] (Get-Process -PV proc |
Select-Object -Expand Id |
Group-Object { $proc.Name } -AsHashtable)
# List all PIDs of given process
$processes.chrome
Notes:
Common parameter -PV (alias of -PipelineVariable) makes sure that we can still access the full process object from within the calculated property of the Group-Object command, despite that we have a Select-Object command in between.
The values of the properties are arrays, which store the process IDs of all instances of each process. E. g. $processes.chrome outputs a list of PIDs of all instances of the chrome process.

Combine (add) value from 2 different hashtable together

I was tasked with created a script to pull DHCP stats from a couple of particular scopes. I was able to get the stats no problem, however the output returned 2 results from the same scope. The problem is, whoever created this scope, created 2 identical scopes on 2 different servers but set the unusable address flipped in order to create "round robin"
I.e.
Server 1 has scope 10.10.92.0/23 (255.255.254.0)
10.10.92.48 as start of addresses
10.10.93.249 as end of addresses
10.10.93.1 to 10.10.93.249 IP **EXCLUSION**
Server 2 has scope 10.10.92.0/23 (255.255.254.0)
10.10.92.48 as start of addresses
10.10.93.249 as end of addresses
10.10.92.48 to 10.10.92.255 IP **EXCLUSION**
So my output in the HTML file would show:
1st result
2nd result
I redid my code and I manage to separate the results into hashtables, but now I want to combine the results and then generate a new html file based on that. The below code works but all attempts to try and combine the results resulted in failure or syntax error. Looking for a little guidance here.
$hashtable1 = #{} #create hash table
$scopes1 = Get-DhcpServerv4Scope -ComputerName NAMEOFSERVER | Select-Object ScopeId, SubnetMask, StartRange, EndRange, LeaseDuration, State | Where-Object -FilterScript {$_.ScopeId -like "10.10*" } #get all scopes
foreach ($scope in $scopes1) { $stats1 = Get-DhcpServerv4ScopeStatistics -ComputerName covdhcp5 -ScopeId $scope.scopeid.IPAddressToString | select Free, InUse, Reserved, percentageInUse
$array1 = #()
foreach ($var1 in $stats1) { $array1 += $var1 }
$hashtable1.add($scope.scopeid.IPAddressToString, $array1)}
$hashtable2 = #{} #create hash table
$scopes2 = Get-DhcpServerv4Scope -ComputerName NAMEOFSERVER | Select-Object ScopeId, SubnetMask, StartRange, EndRange, LeaseDuration, State | Where-Object -FilterScript {$_.ScopeId -like "10.10*" } #get all scopes
foreach ($scope in $scopes2) { $stats2 = Get-DhcpServerv4ScopeStatistics -ComputerName covdhcp6 -ScopeId $scope.scopeid.IPAddressToString | select Free, InUse, Reserved, percentageInUse
$array2 = #()
foreach ($var2 in $stats2) { $array2 += $var2 }
$hashtable2.add($scope.scopeid.IPAddressToString, $array2)}

Split array column and result into extra Colums

I have a table (CSV) which shows all the users that have ever logged on to a bunch of computers. Users can have 2 accounts, one username starts with "a" the other with "b", like a100 and b100 (the user behind is the same person).
Now I need to get the computers that have more then 2 accounts logged on which do not belong the same users. So A64 and B64 are not reported as separate users.
Here is the base list I have:
PC1,A64,B52,B64,A41
PC2,A51,B42,B51,A23
PC3,A42,B51
PC4,A5,B5
PC5,A1,B1,A14,A6
My plan was to split the "User"-column into more columns, so the table would look like this:
Computername,user1,user2,user3,user4,UserX
After this was done, I could iterate through the table and remove the leading letter in the Username, then I would try to get rid of doubles.
Do you think that makes sense?
Now I got stuck in the first task already. I know how to iterate though the second Column but how do I managed to get the result into another array so the output would be like:
Computername,user1,user2,user3,user4,UserX
Can you help me split?
$UserComputers = import-csv -Delimiter ";" "input.csv" -Header
'Computername','user1','user2','user3','user4'
$UserComputers | Select-Object *,
#{n='User1';e={$_.User1.Split(',')[0]}},
#{n='User2';e={$_.User1.Split(',')[1]}}
I get the error: Select-Object : The property cannot be processed because the property "User1" already exists.
It is useful to make "user" an array.
Get-Content "input.csv" | foreach {
$name, $users = $_.Split(",")
[pscustomobject]#{ Name = $name; Users = $users }
} | Where-Object { ($_.Users.Substring(1) | Select-Object -Unique).Count -gt 2 }
The output is below.
Name Users
---- -----
PC1 {A64, B52, B64, A41}
PC2 {A51, B42, B51, A23}
PC3 {A42, B51}
PC5 {A1, B1, A14, A6}
Input File (input.csv)
PC1,A64,B52,B64,A41
PC2,A51,B42,B51,A23
PC3,A42,B51
PC4,A5,B5
PC5,A1,B1,A14,A6
Powershell Script
Get-Content -Path .\input.csv |
Select-Object #{ Name = "Computer"; Expression = { $_.Split(',')[0] } },
#{ Name="Users"; Expression = { $_.Split(',')[1..($_.Split(',').Length-1)] |
Foreach-Object { $_.Substring(1) } | Select-Object -Unique } } |
Where-Object { $_.Users.Count -gt 2 }
Result:
Computer Users
------------- -----
PC1 {64, 52, 41}
PC2 {51, 42, 23}
PC5 {1, 14, 6}
P.S. Bonus: If you want to see more than 4 elements of the array on the screen change the variable
$FormatEnumerationLimit = 20
Explanation of the variable meaning
If your file is like your base list, you can do the following to build a new file with all the columns you need:
$maxColCount = 0
$data = get-content input.csv
foreach ($line in $data) {
$MaxColCount = [math]::Max($maxcolcount,($line -split ",").count)
}
$headers = #("ComputerName")
$MaxUserCount = $MaxColCount - 1
Foreach ($c in (1..$MaxUserCount)) {
$Headers += "User$c"
}
$Headers = $Headers -join ","
$Headers,$data | Set-Content "output.csv"
The code above assumes input.csv has the following format and each column after the first is a user:
PC1,A64,B52,B64,A41
PC2,A51,B42,B51,A23
PC3,A42,B51
PC4,A5,B5
PC5,A1,B1,A14,A6

Select all unique third octet in a list of IP addresses with PowerShell

I have a list of IP addresses. They all start with 10.10. I want all the unique values of the third octet. This way I can count how many of that unique value there are.
10.10.26.251
10.10.27.221
10.10.26.55
10.10.31.12
10.10.12.31
10.10.31.11
10.10.27.15
10.10.26.5
When I am done I want to know that I have 3 .26 network devices, 2 27, and so on so forth. Other than breaking down the octet with a split and looping through each one, I can't think of any single liners. Any suggestions?
here's a small variant. [grin] i already had this before noticing the other answers - and it is a tad different.
what it does ...
creates a collection of IPv4 address objects to work with
groups them by a calculated property [the 3rd octet]
creates a [PSCustomObject] for each resulting group
sends it to the $Octet3_Report variable
shows it on screen
output to a CSV file would be easy at that point. here's the code ...
$IP_List = #(
[ipaddress]'10.10.26.251'
[ipaddress]'10.10.27.221'
[ipaddress]'10.10.26.55'
[ipaddress]'10.10.31.12'
[ipaddress]'10.10.12.31'
[ipaddress]'10.10.31.11'
[ipaddress]'10.10.27.15'
[ipaddress]'10.10.26.5'
)
$Octet3_Report = $IP_List |
Group-Object -Property {$_.ToString().Split('.')[2]} |
ForEach-Object {
[PSCustomObject]#{
Octet_3 = $_.Name
Count = $_.Count
}
}
$Octet3_Report
on screen output ...
Octet_3 Count
------- -----
26 3
27 2
31 2
12 1
It's like me to figure it out after the fact.
The Return contains the dns records. The IP address are stored inside recorddata. I pull the end of the IP address off. Then loop through grabbing only the range and count with a foreach loop to make it cleaner.
$DNSRecordCounts = #()
$Ranges = ($Return | where-object {$_.recorddata -like "10.10.*"}).recorddata -replace "\.\d{1,3}$" | select -Unique
foreach ($range in $Ranges) {
$DNSRecordCounts += [pscustomobject][ordered]#{
IPRange = $range
Count = ($Return | Where-Object {$_.recorddata -like "$($range).*"}).Count
}
}
Based on your question and what I can infer from your own answer, if you are looking for something a little more like "idiomatic" PowerShell you want the following:
$Return `
| Select-Object -ExpandProperty recorddata `
| ForEach-Object {
$_ -match "\d+\.\d+\.(?<octet>\d+)\.\d+" | Out-Null
$Matches.octet
} `
| Group-Object `
| ForEach-Object {
[PSCustomObject]#{
Octet = $_.Name
Count = $_.Count
}
}

powershell: Check if any of a bunch of properties is set

I'm importing a csv-file which looks like this:
id,value1.1,value1.2,value1.3,Value2.1,Value2.2,Value3.1,Value3.2
row1,v1.1,,v1.3
row2,,,,v2.1,v2.2
row3,,,,,,,v3.2
Now I want to check, if any of the value-properties in one group is set.
I can do
Import-Csv .\test.csv | where {$_.Value1.1 -or $_.Value1.2 -or $_.Value1.3}
or
Import-Csv .\test.csv | foreach {
if ($_.Value1 -or $_.Value2 -or $_.Value3) {
Write-Output $_
}
}
But my "real" csv-file contains about 200 columns and I have to check 31 properties x 5 different object types that are mixed up in this csv. So my code will be realy ugly.
Is there anything like
where {$_.Value1.*}
or
where {$ArrayWithPropertyNames}
?
You could easily use the Get-Member cmdlet to get the properties which have the correct prefix (just use * as a wildcard after the prefix).
So to achieve what you want you could just filter the data based on whether any of the properties with the correct prefix contains data.
The script below uses your sample data, with a row4 added, and filters the list to find all items which have a value in any property starting with value1.
$csv = #"
id,value1.1,value1.2,value1.3,Value2.1,Value2.2,Value3.1,Value3.2
row1,v1.1,,v1.3
row2,,,,v2.1,v2.2
row3,,,,,,,v3.2
row4,v1.1,,v1.3
"#
$data = ConvertFrom-csv $csv
$data | Where {
$currentDataItem = $_
$propertyValues = $currentDataItem |
# Get's all the properties with the correct prefix
Get-Member 'value1*' -MemberType NoteProperty |
# Gets the values for each of those properties
Foreach { $currentDataItem.($_.Name) } |
# Only keep the property value if it has a value
Where { $_ }
# Could just return $propertyValues, but this makes the intention clearer
$hasValueOnPrefixedProperty = $propertyValues.Length -gt 0
Write-Output $hasValueOnPrefixedProperty
}
Alternate solution:
$PropsToCheck = 'Value1*'
Import-csv .\test.csv |
Where {
(($_ | Select $PropsToCheck).psobject.properties.value) -contains ''
}