Possible to pull info from AdditionalProperties dictionary with Microsoft Graph PowerShell cmdlets? - powershell

I am trying to use PowerShell Graph cmdlets instead of the Azure AD module cmdlets. With the Azure AD module, I can do this:
# This is what I want:
get-azureadgroupmember -objectid $GroupID | select-object -property displayname, `
mail, userprincipalname, objectid
DisplayName Mail UserPrincipalName ObjectId
----------- ---- ----------------- --------
John Smith John.Smith#example.org jsmith#example.org 4bae8291-6ec3-192b-32ce-dd21869ef784
(...)
# All of these properties are directly accessible in the returned objects:
$res = get-azureadgroupmember -objectid $GroupID
$res[0] | fl -prop *
# Shows long list of directly accessible properties
I'm trying to figure out the equivalent with PowerShell Graph:
$res = get-mggroupmember -groupid $GroupID
$res[0] | fl -prop *
# Only properties are DeletedDateTime, Id, and AdditionalProperties
# Want to do something like this, but it doesn't work:
get-mggroupmember -groupid $GroupID | select-object -property id, `
additionalproperties['displayName'], additionalproperties['mail'], `
additionalproperties['userPrincipalName']
# This works, but is there a better option???
get-mggroupmember -groupid $GroupID | foreach-object { `
"{0},{1},{2},{3}" -f $_.id, $_.additionalproperties['displayName'], `
$_.additionalproperties['mail'], $_.additionalproperties['userPrincipalName']
}
AdditionalProperties is a dictionary (IDictionary) which contains displayname, mail, and userprincipalname. My thought is there is probably a better way to do this or to get at the information.
There are a few interesting parameters in get-mggroupmember that I'm not clear on including "-expandproperty" and "-property". I've tried playing around with these but haven't had any luck. I'm wondering if there's a way to use these to do what I want.
Suggestions?

Given the following $object, 3 properties and one of them AdditionalProperties is a Dictionary<TKey,TValue>:
$dict = [Collections.Generic.Dictionary[object, object]]::new()
$dict.Add('displayName', 'placeholder')
$dict.Add('mail', 'placeholder')
$dict.Add('userPrincipalName', 'placeholder')
$object = [pscustomobject]#{
DeletedDateTime = 'placeholder'
Id = 'placeholder'
AdditionalProperties = $dict
}
Supposing from this object you're interested in Id, displayName and mail, you could use Select-Object with calculated properties:
$object | Select-Object #(
'Id'
#{
Name = 'displayName'
Expression = { $_.additionalProperties['displayName'] }
}
#{
Name = 'mail'
Expression = { $_.additionalProperties['mail'] }
}
)
However this gets messy as soon as you need to pick more property values from the objects, PSCustomObject with a loop comes in handy in this case:
$object | ForEach-Object {
[pscustomobject]#{
Id = $_.Id
displayName = $_.additionalProperties['displayName']
mail = $_.additionalProperties['mail']
}
}
Both alternatives would output the same "flattened" object that can be converted to Csv without any issue:
As Object
Id displayName mail
-- ----------- ----
placeholder placeholder placeholder
As Csv
"Id","displayName","mail"
"placeholder","placeholder","placeholder"
In that sense, you could construct an array of objects using one of the above techniques, for example:
Get-MgGroupMember -GroupId $GroupID | ForEach-Object {
[pscustomobject]#{
Id = $_.id
displayName = $_.additionalproperties['displayName']
mail = $_.additionalproperties['mail']
userPrincipalName = $_.additionalproperties['userPrincipalName']
}
}
If you're looking for a programmatical way to flatten the object, you can start by using this example, however it's important to note that this can only handle an object which's property is nested only once, in other words, it can't handle recursion:
$newObject = [ordered]#{}
foreach($property in $object.PSObject.Properties) {
if($property.Value -is [Collections.IDictionary]) {
foreach($addproperty in $property.Value.GetEnumerator()) {
$newObject[$addproperty.Key] = $addproperty.Value
}
continue
}
$newObject[$property.Name] = $property.Value
}
[pscustomobject] $newObject
The output from this would become a flattened object like this, which also, can be converted to Csv without any issue:
DeletedDateTime : placeholder
Id : placeholder
displayName : placeholder
mail : placeholder
userPrincipalName : placeholder
It's also worth noting that above example is not handling possible key collision, if there are 2 or more properties with the same name, one would override the others.
Bonus function that should work with the objects returned by the cmdlets from Graph, AzureAD and Az Modules. This function can be useful to flatten their Dictionary`2 property. It only looks one level deep if the property value implements IDictionary so don't expect it to flatten any object. For the given example should work well.
function Select-GraphObject {
[CmdletBinding()]
param(
[parameter(ValueFromPipeline, DontShow)]
[object] $InputObject,
[parameter(Position = 0)]
[string[]] $Properties = '*'
)
begin {
$firstObject = $true
$toSelect = [Collections.Generic.List[object]]::new()
}
process {
if($firstObject) {
foreach($property in $InputObject.PSObject.Properties) {
foreach($item in $Properties) {
if($property.Value -is [Collections.IDictionary]) {
foreach($key in $property.Value.PSBase.Keys) {
if($key -like $item -and $key -notin $toSelect.Name) {
$toSelect.Add(#{
$key = { $_.($property.Name)[$key] }
})
}
}
continue
}
if($property.Name -like $item -and $property.Name -notin $toSelect) {
$toSelect.Add($property.Name)
}
}
}
$firstObject = $false
}
$out = [ordered]#{}
foreach($item in $toSelect) {
if($item -isnot [hashtable]) {
$out[$item] = $InputObject.$item
continue
}
$enum = $item.GetEnumerator()
if($enum.MoveNext()) {
$out[$enum.Current.Key] = $InputObject | & $enum.Current.Value
}
}
[pscustomobject] $out
}
}
Using copies of the $object from above examples, if using the default value of -Properties, the example objects would be flattened:
PS /> $object, $object, $object | Select-GraphObject
DeletedDateTime Id displayName mail userPrincipalName
--------------- -- ----------- ---- -----------------
placeholder placeholder placeholder placeholder placeholder
placeholder placeholder placeholder placeholder placeholder
placeholder placeholder placeholder placeholder placeholder
Or we can filter for specific properties, even Keys from the AdditionalProperties Property:
PS /> $object, $object, $object | Select-GraphObject Id, disp*, user*
Id displayName userPrincipalName
-- ----------- -----------------
placeholder placeholder placeholder
placeholder placeholder placeholder
placeholder placeholder placeholder

Related

How to fix System Object value in PowerShell

I'm Importing a CSV file and reading a column that look like this
Exchange Mailboxes
Include:[john.doe#outlook.com]
Include:[david.smith#outlook.com]
Include:[kevin.love#outlook.com]
I use Get-EXOMailbox to get their DisplayName and Id. After that I'm trying to pass it in my New-Object like below so that I can export it. The problem I have is when I look at my Excel file, it showing System.Object[] on every row instead of showing each actual DisplayName and Id.
Any help on how to display it correctly would be really appreciated.
$result = Import-Csv "C:\AuditLogSearch\Dis\Modified-Audit-Log-Records.csv" |
Where-Object { -join $_.psobject.Properties.Value } |
ForEach-Object {
$exoMailbox = ($_.'Exchange Mailboxes' -split '[][]')[1]
$exoUser = Get-EXOMailbox -Filter "PrimarySmtpAddress -eq '$exoMailbox'"
# Construct and output a custom object with the properties of interest.
[pscustomobject] #{
UserName = $exoUser.DisplayName
UserId = $exoUser.Identity
}
}
New-Object PsObject -Property #{
'Searched User' = $result.UserName //I'm trying to pass here
'SharePoint URL' = $spUrl
'Searched User GMID' = $result.UserId //and here
'Site Owner' = $spositeOwner
User = $u.User
"Result Status" = $u."Result Status"
"Date & Time" = $u."Date & Time"
"Search Conditions" = $u."Search Conditions"
"SharePoint Sites" = $u."SharePoint Sites"
"Exchange Public Folders" = $u."Exchange Public Folders"
"Exchange Mailboxes" = $u."Exchange Mailboxes".Split([char[]]#('[', ']'))[1]
"Case Name" = $u."Case Name"
"Search Criteria" = $u."Search Criteria"
"Record Type" = $u."Record Type"
"Hold Name" = $u."Hold Name".Split(('\'))[1]
"Activity" = if ($null -ne ($importData | where-object { $_.Name -eq $u."Activity" }).Value) { ($importData | where-object { $_.Name -eq $u."Activity" }).Value }
else { $u."Activity" }
} | Select-object -Property User, "Date & Time", "Case Name", "Hold Name", "Record Type", "Activity" , "Searched User", "Searched User GMID", "SharePoint URL", "Exchange Mailboxes", "Exchange Public Folders" , "Search Criteria", "Result Status"
}
$xlsx = $result | Export-Excel #params
$ws = $xlsx.Workbook.Worksheets[$params.Worksheetname]
$ws.Dimension.Columns
$ws.Column(1).Width = 20
$ws.Column(2).Width = 20
$ws.Column(3).Width = 15
$ws.Column(4).Width = 15
$ws.Column(5).Width = 15
$ws.Column(6).Width = 160
$ws.View.ShowGridLines = $false
Close-ExcelPackage $xlsx
$result is an array of objects, containing an object for each non-empty row in your input CSV; thus, adding values such as $result.UserName to the properties of the object you're creating with New-Object will be arrays too, which explains your symptom (it seems that Export-Excel, like Export-Csv doesn't meaningfully support array-valued properties and simply uses their type name, System.Object[] during export).
It sounds like the easiest solution is to add the additional properties directly in the ForEach-Object call, to the individual objects being constructed and output via the existing [pscustomobject] literal ([pscustomobject] #{ ... }):
$result =
Import-Csv "C:\AuditLogSearch\Dis\Modified-Audit-Log-Records.csv" |
Where-Object { -join $_.psobject.Properties.Value } | # only non-empty rows
ForEach-Object {
$exoMailbox = ($_.'Exchange Mailboxes' -split '[][]')[1]
$exoUser = Get-EXOMailbox -Filter "PrimarySmtpAddress -eq '$exoMailbox'"
# Construct and output a custom object with the properties of interest.
[pscustomobject] #{
UserName = $exoUser.DisplayName
UserId = $exoUser.Identity
# === Add the additional properties here:
'Searched User' = $exoUser.UserName
'SharePoint URL' = $spUrl
'Searched User GMID' = $exoUser.UserId
'Site Owner' = $spositeOwner
# ...
}
}
Note:
The above shows only some of the properties from your question; add as needed (it is unclear where $u comes from in some of them.
Using a custom-object literal ([pscustomobject] #{ ... }) is not only easier and more efficient than a New-Object PSObject -Property #{ ... }[1] call, unlike the latter it implicitly preserves the definition order of the properties, so that there's no need for an additional Select-Object call that ensures the desired ordering of the properties.
[1] Perhaps surprisingly, PSObject ([psobject]) and PSCustomObject ([pscustomobject]) refer to the same type, namely System.Management.Automation.PSObject, despite the existence of a separate System.Management.Automation.PSCustomObject, which custom-objects instances self-report as (([pscustomobject] #{}).GetType().FullName) - see GitHub issue #4344 for background information.

(PowerShell) How do I filter usernames with Get-EventLog

I'm working on a Powershell script to get all users who have logged in/out of a server in the past 7 days, where their name is not like "*-organization". The below works, but no matter what I try I'm not able to filter names
$logs = get-eventlog system -ComputerName $env:computername -source Microsoft-Windows-Winlogon -After (Get-Date).AddDays(-7)
$res = #()
ForEach ($log in $logs)
{
if($log.instanceid -eq 7001){
$type = "Logon"
}
Elseif ($log.instanceid -eq 7002){
$type = "Logoff"
}
Else { Continue }
$res += New-Object PSObject -Property #{Time = $log.TimeWritten; "Event" = $type; User = (New-Object System.Security.Principal.SecurityIdentifier $Log.ReplacementStrings[1]).Translate([System.Security.Principal.NTAccount])}};
$res
I've tried adding this line in various places and ways, but no matter what I can't get it to filter. It either fails and tells me my operator must have a property and value, or it runs fine and ignores any username filtering.
| Where-Object $_.User -notlike "*-organization"
Is it even possible to filter the login username with this method? If so, what am I doing wrong? If it's not possible, is there another way I can get what I need?
There would have to be a property named 'user' for that to work. Get-eventlog is actually obsolete now, and replaced by get-winevent. Unfortunately, you have to get into the xml to filter by usersid. I've included a time filter.
$a = get-winevent #{logname='system';
providername='Microsoft-Windows-Winlogon'} -MaxEvents 1
$e = $a.ToXml() -as 'xml'
$e.event.EventData
Data
----
{TSId, UserSid}
get-winevent #{logname='system';providername='Microsoft-Windows-Winlogon';
data='S-2-6-31-1528843147-473324174-2919417754-2001';starttime=(Get-Date).AddDays(-7);
id=7001,7002}
In powershell 7 you can refer to the eventdata named data fields directly:
get-winevent #{logname='system';providername='Microsoft-Windows-Winlogon';
usersid='S-2-6-31-1528843147-473324174-2919417754-2001'}
The get-winevent docs say you can use "userid" in the filterhashtable, but I can't get that to work.
EDIT: Actually this works. But without limiting it too much, at least for me.
get-winevent #{logname='system';userid='js2010'}
get-winevent #{providername='Microsoft-Windows-Winlogon';userid='js2010'}
You can do this with the -FilterXPath parameter like below:
$filter = "(*[System/EventID=7001] or *[System/EventID=7002]) and *[System/Provider[#Name='Microsoft-Windows-Winlogon']]"
$result = Get-WinEvent -LogName System -FilterXPath $filter | ForEach-Object {
# convert the event to XML and grab the Event node
$eventXml = ([xml]$_.ToXml()).Event
$eventData = $eventXml.EventData.Data
$userSID = ($eventData | Where-Object { $_.Name -eq 'UserSid' }).'#text'
$userName = [System.Security.Principal.SecurityIdentifier]::new($userSID).Translate([System.Security.Principal.NTAccount])
# you can add username filtering here if you like.
# remember the $userName is in formal DOMAIN\LOGONNAME
# if ($username -notlike "*-organization") {
# output the properties you need
[PSCustomObject]#{
Time = [DateTime]$eventXml.System.TimeCreated.SystemTime
Event = if ($eventXml.System.EventID -eq 7001) { 'LogOn' } else { 'LogOff' }
UserName = $userName
UserSID = $userSID
Computer = $eventXml.System.Computer
}
# }
}
# output on screen
$result
# output to CSV file
$result | Export-Csv -Path 'X:\TheOutputFile.csv' -NoTypeInformation
Note, I have commented out the username filtering in the code. It is just there to give you an idea of where to put it. Of course, you can also filter the $result afterwards:
$result | Where-Object { $_.UserName -notlike "*-organization" }
Adding to #js2010's helpful answer, and with the assumption you're using PowerShell 5.1. I usually identify the property array index and use Select-Object to create a custom property as needed.
$WinEvents =
get-winevent #{logname='system'; providername='Microsoft-Windows-Winlogon'} |
Select-Object #{Name = 'Time'; Expression = {$_.TimeCreated}},
#{Name = 'Event'; Expression = { If($_.ID -eq 7001){'Logon'} ElseIf($_.ID -eq 7002){ 'Logoff' } } },
#{Name = 'User'; Expression = { [System.Security.Principal.SecurityIdentifier]::new( $_.Properties[1].Value ).Translate([System.Security.Principal.NTAccount]) } }
In your case this should add a property called User with a value like DomainName\UserName to the objects. I also added expressions to derive the other properties you were adding to your custom objects. Select-Object emits custom objects as well so this should give the result you're looking for.
Let me know if this helps.
Update
Respectfully, the other 2 answers make the assumption that you are looking for logon/off events for a specific user. That's not how I read the question; in particular:
"get all users who have logged in/out of a server"
While PowerShell 7+ does let you directly cite UserID in the FilterHashtable, it's not very useful here because we're not seeking events for a specific user. Furthermore, it seems unhelpful for the ultimate output as by default it echoes as a SID. It would still need to be translated, not only for display but for further filtering. I'm also not positive that UserID will always be the same as Properties[1], there's certainly some variance when looking at other event IDs.
The XML work is very cool, but I don't think it's called for here.
There were some issues with my answer as well. I overlooked filtering the event IDs & dates up front. I also realized we don't need to instantiate [System.Security.Principal.SecurityIdentifier] class because the property is already typed as such. Along with some readability improvements I corrected those issues below.
# Should be the 1st line!
using NameSpace System.Security.Principal
$ResolveEventType = #{ 7001 = 'Logon'; 7002 = 'Logoff' }
$FilterHashTable =
#{
LogName = 'system'
ProviderName = 'Microsoft-Windows-Winlogon'
ID = 7001,7002
StartTime = (Get-Date).AddDays(-7)
}
[Array]$WinEvents =
Get-WinEvent -FilterHashtable $FilterHashTable |
Select-Object #{ Name = 'Time'; Expression = { $_.TimeCreated } },
#{ Name = 'Event'; Expression = { $ResolveEventType[ $_.ID ] } },
#{ Name = 'User'; Expression = { $_.Properties[1].Value.Translate( [NTAccount] ) } }
$WinEvents |
Where-Object{ $_.UserName -notlike "*-organization" } |
Format-Table -AutoSize
This tested good in PowerShell 5.1 & 7.0. I added Format-Table to display the output, but you can just change that out for an Export-Csv command as needed
Note: The last 2 pipelines can be combined, but I thought this was a
little more readable.
Let me know if this helps.

How can I speed up a query to Win32_UserAccount filtering by FullName property

I'm trying to find a specific user based on the FullName property of the user using a simple WMI command in powershell:
Get-WmiObject win32_useraccount -Filter "fullname='Jack Ryan'"
There are about 50,000 users and this is taking up to 30 seconds to find all the users based on the FullName property.
However when I try to search based on Name instead of FullName I get a response back in under a second. I don't understand why it's taking so long to lookup by FullName.
How can I speed this up? (Unfortunately I need to filter based on FullName and get all the usernames for those identities)
Getting group members via ADSI, as per #rboy comment. Probably easiest to have your own function here if you cannot use Get-LocalGroupMember.
function Get-LocalMembers {
Param(
[parameter(Mandatory=$false)][string]$GroupName
)
$ADSI = [ADSI]"WinNT://$env:COMPUTERNAME"
if ($PSBoundParameters.ContainsKey('GroupName')) {
$Groups = $ADSI.Children.Find($GroupName,'Group')
}
else {
$Groups = $ADSI.Children | Where-Object { $_.SchemaClassName -eq 'Group' }
}
Foreach ($Group in $Groups) {
[PSCustomObject] #{
Group = $($Group | Select-Object -ExpandProperty Name)
Members = $(
$Group.Invoke('members') | ForEach-Object {
$_.GetType().InvokeMember("Name",'GetProperty',$null,$_,$null)
}
)
}
}
}
Usage:
Get-LocalMembers # Get members of all groups
Get-LocalMembers -GroupName Administrators # Get members of specified group
EDIT
Getting both Name and FullName properties. I'm not sure really how much more we could do with ADSI in PowerShell here. I've never really explored it any further than user objects and group/member lists.
function Get-LocalMembers {
Param(
[parameter(Mandatory=$false)][string]$GroupName
)
$ADSI = [ADSI]"WinNT://$env:COMPUTERNAME"
if ($PSBoundParameters.ContainsKey('GroupName')) {
$Groups = $ADSI.Children.Find($GroupName,'Group')
}
else {
$Groups = $ADSI.Children | Where-Object { $_.SchemaClassName -eq 'Group' }
}
Foreach ($Group in $Groups) {
[PSCustomObject] #{
Group = $($Group | Select-Object -ExpandProperty Name)
Members = $(
$Group.Invoke('members') | ForEach-Object {
[PSCustomObject] #{
Name = $_.GetType().InvokeMember("Name",'GetProperty',$null,$_,$null)
FullName = $(
# A Group can be a member of a group and doesn't have a 'FullName property'
Try {
$_.GetType().InvokeMember("Fullname",'GetProperty',$null,$_,$null)
}
Catch {
"Group"
}
)
}
}
)
}
}
}
Disclaimer: I'm running this on a local, non-domain joined laptop none of my accounts have the Fullname property populated.

PowerShell variable definition from a pscustomobject

i've got this piece of code from a script i found on the web (just showing the part that interests me)
ForEach ($Computer in $Computername) {
$adsi = [ADSI]"WinNT://$Computername"
$adsi.Children | where {$_.SchemaClassName -eq 'user'} | ForEach {
[pscustomobject]#{
UserName = $_.Name[0]
SID = ConvertTo-SID -BinarySID $_.ObjectSID[0]
PasswordAge = [math]::Round($_.PasswordAge[0]/86400)
LastLogin = If ($_.LastLogin[0] -is [datetime]){$_.LastLogin[0]}Else{'Never logged on'}
UserFlags = Convert-UserFlag -UserFlag $_.UserFlags[0]
MinPasswordLength = $_.MinPasswordLength[0]
MinPasswordAge = [math]::Round($_.MinPasswordAge[0]/86400)
MaxPasswordAge = [math]::Round($_.MaxPasswordAge[0]/86400)
BadPasswordAttempts = $_.BadPasswordAttempts[0]
MaxBadPasswords = $_.MaxBadPasswordsAllowed[0]
}
}
}
the code displays things on the console, but i would like to define/use these values as variables instead (as i want to use them in a hash table afterwards to send them in a http/POST request afterwards)
is there a way to get all these attributes as variables such as $LastLogin, $MinPasswordAge etc ?
as i don't want to display them, but send them in a POST like this :
$postParams = #{LastLogin=$LastLogin;MinPasswordAge=$MinPasswordAge}
Invoke-WebRequest -Uri http://example.com/foobar -Method POST -Body $postParams
to be honest i'm a complete newbie in PowerShell (i'm a Perl guru) and i don't know what pscustomobject does in there, i just want to define the variables in that loop, and use them at the end.
i've tried a couple of things with no success (can post them if required)
thanks !
Your own solution works, but only if you perform all processing inside the ForEach-Object script block (unless there's only ever 1 iteration, which doesn't appear to be the case here).
If you want to process the results later, you can simply collect them in an array by assigning the entire foreach loop to a variable (code shortened):
$allUsers = foreach ($Computer in $Computername) {
$adsi = [ADSI]"WinNT://$Computername"
$adsi.Children | where {$_.SchemaClassName -eq 'user'} | ForEach {
# Output a custom object for each user.
[pscustomobject]#{
ComputerName = $Computer # also record the computer name
UserName = $_.Name[0]
SID = ConvertTo-SID -BinarySID $_.ObjectSID[0]
# ...
}
}
}
You can then simply enumerate the collected [pscustomobject]s and access their properties rather than using variables:
foreach ($user in $allUsers) {
# Use the properties to define a hashtable for later use in a http/POST request.
$ht = #{
User = $user.UserName
# ...
}
}
nm,
i found the solution a minute ago.
just got rid of that pscustomobject hash completely, and assigning the variables directory
$adsi.Children | where {$_.SchemaClassName -eq 'user'} | ForEach {
$UserName = $_.Name[0]
$SID = ConvertTo-SID -BinarySID $_.ObjectSID[0]
$PasswordAge = [math]::Round($_.PasswordAge[0]/86400)
$LastLogin = If ($_.LastLogin[0] -is [datetime]){$_.LastLogin[0]}Else{'Never logged on'}
$UserFlags = Convert-UserFlag -UserFlag $_.UserFlags[0]
$MinPasswordLength = $_.MinPasswordLength[0]
$MinPasswordAge = [math]::Round($_.MinPasswordAge[0]/86400)
$MaxPasswordAge = [math]::Round($_.MaxPasswordAge[0]/86400)
$BadPasswordAttempts = $_.BadPasswordAttempts[0]
$MaxBadPasswords = $_.MaxBadPasswordsAllowed[0]
Write-Host $UserName
}
}

Handling multiple CSVs in Powershell efficiently

I am retrieving two CSVs from an API, one called students.csv similar to:
StudentNo,PreferredFirstnames,PreferredSurname,UPN
111, john, smith, john#email.com
222, jane, doe, jane#email.com
one called rooms.csv:
roomName, roomNo, students
room1, 1, {#{StudentNo=111; StudentName=john smith; StartDate=2018-01-01T00:00:00; EndDate=2018-07-06T00:00:00},....
room2, 2,{#{StudentNo=222; StudentName=jane doe; StartDate=2018-01-01T00:00:00; EndDate=2018-07-06T00:00:00},...
The third column in rooms.csv is an array as provided by the API
What is the best way to consolidate the two into
StudentNo,PreferredFirstnames,PreferredSurname,UPN, roomName
111, john, smith, john#email.com, room1
222, jane, doe, jane#email.com, room2
Im thinking something like...
$rooms = Import-Csv rooms.csv
$students = Import-Csv students.csv
$combined = $students | select-object StudentNo,PreferredSurname,PreferredFirstnames,UPN,
#{Name="roomName";Expression={ ForEach ($r in $rooms) {
if ($r.Students.StudentNo.Contains($_.StudentNo) -eq "True")
{return $r.roomName}}}}
This works, but is the foreach the right way to go am i mixing things up or is there a more efficient way???
--- Original Post ---
With all of this information I need to compare the student data and update AzureAD and then compile a list of data including first name, last name, upn, room and others that are retrieved from AzureAD.
My issue is "efficiency". I have code that mostly works but it takes hours to run. Currently I am looping through students.csv and then for each student looping through rooms.csv to find the room they're in, and obviously waiting for multiple api calls in-between all this.
What is the most efficient way to find the room for each student? Is importing the CSV as a custom PSObject comparable to using hash tables?
I was able to get your proposed code to work but it requires some tweaks to the code and data:
There must be some additional step where you are deserializing the students column of rooms.csv to a collection of objects. It appears to be a ScriptBlock that evaluates to an array of HashTables, but some changes to the CSV input are still needed:
The StartDate and EndDate properties need to be quoted and cast to [DateTime].
At least for rooms that contain multiple students, the value must be quoted so Import-Csv doesn't interpret the , separating array elements as an additional column.
The downside of using CSV as an intermediate format is the original property types are lost; everything becomes a [String] upon import. Sometimes it's desirable to cast back to the original type for efficiency purposes, and sometimes it's absolutely necessary in order for certain operations to work. You could cast those properties every time you use them, but I prefer to cast them once immediately after import.
With those changes rooms.csv becomes...
roomName, roomNo, students
room1, 1, "{#{StudentNo=111; StudentName='john smith'; StartDate=[DateTime] '2018-01-01T00:00:00'; EndDate=[DateTime] '2018-07-06T00:00:00'}}"
room2, 2, "{#{StudentNo=222; StudentName='jane doe'; StartDate=[DateTime] '2018-01-01T00:00:00'; EndDate=[DateTime] '2018-07-06T00:00:00'}}"
...and the script becomes...
# Replace the [String] property "students" with an array of [HashTable] property "Students"
$rooms = Import-Csv rooms.csv `
| Select-Object `
-ExcludeProperty 'students' `
-Property '*', #{
Name = 'Students'
Expression = {
$studentsText = $_.students
$studentsScriptBlock = Invoke-Expression -Command $studentsText
$studentsArray = #(& $studentsScriptBlock)
return $studentsArray
}
}
# Replace the [String] property "StudentNo" with an [Int32] property of the same name
$students = Import-Csv students.csv `
| Select-Object `
-ExcludeProperty 'StudentNo' `
-Property '*', #{
Name = 'StudentNo'
Expression = { [Int32] $_.StudentNo }
}
$combined = $students `
| Select-Object -Property `
'StudentNo', `
'PreferredSurname', `
'PreferredFirstnames', `
'UPN', `
#{
Name = "roomName";
Expression = {
foreach ($r in $rooms)
{
if ($r.Students.StudentNo -contains $_.StudentNo)
{
return $r.roomName
}
}
#TODO: Return text indicating room not found?
}
}
The reason this can be slow is because you are performing a linear search - two of them, in fact - for every student object; first through the collection of rooms (foreach), then through the collection of students in each room (-contains). This can quickly turn into a lot of iterations and equality comparisons because in every room to which the current student is not assigned you are iterating the entire collection of that room's students, on and on until you do find the room for that student.
One easy optimization you can make when performing a linear search is to sort the items you're searching (in this case, the Students property will be ordered by the StudentNo property of each student)...
# Replace the [String] property "students" with an array of [HashTable] property "Students"
$rooms = Import-Csv rooms.csv `
| Select-Object `
-ExcludeProperty 'students' `
-Property '*', #{
Name = 'Students'
Expression = {
$studentsText = $_.students
$studentsScriptBlock = Invoke-Expression -Command $studentsText
$studentsArray = #(& $studentsScriptBlock) `
| Sort-Object -Property #{ Expression = { $_.StudentNo } }
return $studentsArray
}
}
...and then when you're searching that same collection if you come across an item that is greater than the one you're searching for you know the remainder of the collection can't possibly contain what you're searching for and you can immediately abort the search...
#{
Name = "roomName";
Expression = {
foreach ($r in $rooms)
{
# Requires $room.Students to be sorted by StudentNo
foreach ($roomStudentNo in $r.Students.StudentNo)
{
if ($roomStudentNo -eq $_.StudentNo)
{
# Return the matched room name and stop searching this and further rooms
return $r.roomName
}
elseif ($roomStudentNo -gt $_.StudentNo)
{
# Stop searching this room
break
}
# $roomStudentNo is less than $_.StudentNo; keep searching this room
}
}
#TODO: Return text indicating room not found?
}
}
Better yet, with a sorted collection you can also perform a binary search, which is faster than a linear search*. The Array class already provides a BinarySearch static method, so we can accomplish this in less code, too...
#{
Name = "roomName";
Expression = {
foreach ($r in $rooms)
{
# Requires $room.Students to be sorted by StudentNo
if ([Array]::BinarySearch($r.Students.StudentNo, $_.StudentNo) -ge 0)
{
return $r.roomName
}
}
#TODO: Return text indicating room not found?
}
}
The way I would approach this problem, however, is to use a [HashTable] mapping a StudentNo to a room. There is a little preprocessing required to build the [HashTable] but this will provide constant-time lookups when retrieving the room for a student.
function GetRoomsByStudentNoTable()
{
$table = #{ }
foreach ($room in $rooms)
{
foreach ($student in $room.Students)
{
#NOTE: It is assumed each student belongs to at most one room
$table[$student.StudentNo] = $room
}
}
return $table
}
# Replace the [String] property "students" with an array of [HashTable] property "Students"
$rooms = Import-Csv rooms.csv `
| Select-Object `
-ExcludeProperty 'students' `
-Property '*', #{
Name = 'Students'
Expression = {
$studentsText = $_.students
$studentsScriptBlock = Invoke-Expression -Command $studentsText
$studentsArray = #(& $studentsScriptBlock)
return $studentsArray
}
}
# Replace the [String] property "StudentNo" with an [Int32] property of the same name
$students = Import-Csv students.csv `
| Select-Object `
-ExcludeProperty 'StudentNo' `
-Property '*', #{
Name = 'StudentNo'
Expression = { [Int32] $_.StudentNo }
}
$roomsByStudentNo = GetRoomsByStudentNoTable
$combined = $students `
| Select-Object -Property `
'StudentNo', `
'PreferredSurname', `
'PreferredFirstnames', `
'UPN', `
#{
Name = "roomName";
Expression = {
$room = $roomsByStudentNo[$_.StudentNo]
if ($room -ne $null)
{
return $room.roomName
}
#TODO: Return text indicating room not found?
}
}
You can ameliorate the hit of building $roomsByStudentNo by doing so at the same time as importing rooms.csv...
# Replace the [String] property "students" with an array of [HashTable] property "Students"
$rooms = Import-Csv rooms.csv `
| Select-Object `
-ExcludeProperty 'students' `
-Property '*', #{
Name = 'Students'
Expression = {
$studentsText = $_.students
$studentsScriptBlock = Invoke-Expression -Command $studentsText
$studentsArray = #(& $studentsScriptBlock)
return $studentsArray
}
} `
| ForEach-Object -Begin {
$roomsByStudentNo = #{ }
} -Process {
foreach ($student in $_.Students)
{
#NOTE: It is assumed each student belongs to at most one room
$roomsByStudentNo[$student.StudentNo] = $_
}
return $_
}
*Except for on small arrays