dynamically creating key/value of an object and exporting to CSV - powershell

After getting a search result from an LDAP Server, i need to create a pscustomobject dynamically.
The Problem here is that some of the attributes are not set for all users.
this is why i cannot create the pscustomobject the traditional way.
Name = $($item.Attributes['givenname'].GetValues('string'))
Surname = $($item.Attributes['sn'].GetValues('string'))
The Attribute Name does not exist for all users and doing this throws an error.
How can i create the pscustomobject in this case where i need to add both key and value dynamically.
Here is what i have so far:
$vals="cn","tel","email","sn","givenname","ou"
$c.Bind()
$r = New-Object System.DirectoryServices.Protocols.SearchRequest -ArgumentList $baseDN,$Filter,$scope,$attrlist
$re = $c.SendRequest($r)
foreach ($item in $re.Entries) {
foreach($attr in $vals){
if($item.Attributes.Keys -contains $attr){
$pskeys += $attr
}}
foreach($pskey in $pskeys){
$data += [pscustomobject]#{
$($pskey) = $($item.Attributes[$pskey].GetValues('string'))
}}
$pskeys = #()
}
givenname does not exist for all the users and this is why the pscustombject must be created dynamically.
I cannot use a HashTable or some kind of a List as duplicate values must be allowed. There are cases where the attributes sn and givenname are equal.
After hours of trying and failing i can only hope for the Wizards of Stackoverflow to show me how this can be achieved.
I need a pscustomobject where i can save the available attributes and skip the missing attributes dynamically. Is there a way to do this?
Regards

Try following :
$table = [System.Collections.ArrayList]::new()
foreach ($item in $re.Entries) {
$newRow = New-Object -TypeName psobject
foreach($attr in $vals){
if($item.Attributes.Keys -contains $attr){
$pskeys += $attr
}}
foreach($pskey in $pskeys){
foreach($item in $item[$pskey].Attributes.GetValues('string'))
{
$newRow | Add-Member -NotePropertyName $item.Name -NotePropertyValue $item.Value
}
}
$table.Add($newRow) | Out-Null
}
$table | Format-Table

Finally!
I have gotten it to work!
The Trick was to enclose $pskey and $item.Attributes[$pskey].GetValues('string') in $()
Without $() Add-Member was adding the properties as Arrays and not as Strings.
Here is the working Code:
$table = New-Object System.Collections.ArrayList
$c.Bind()
$r = New-Object System.DirectoryServices.Protocols.SearchRequest -ArgumentList $baseDN,$Filter,$scope,$attrlist
$re = $c.SendRequest($r)
foreach ($item in $re.Entries) {
$newRow = New-Object -TypeName psobject
foreach($attr in $vals){
if($item.Attributes.Keys -contains $attr){
$pskeys += $attr
}}
foreach($pskey in $pskeys){
$newRow | Add-Member -NotePropertyName $($pskey) -NotePropertyValue $($item.Attributes[$pskey].GetValues('string'))
}
$table.Add($newRow) | Out-Null
$pskeys = #()
}
$table | Export-Csv -Path $ExportPath -NoTypeInformation -Encoding UTF8 -Append -Delimiter ";"
Thank You jdweng for pointing me in the right direction.
$table | Format-Table on the console and the resulting CSV after the Export look flawless now.
My Problem is solved.

Related

Array in a foreach loop

What am I doing wrong here?
The mailbox has an active an inactive mailbox so it will return two mailboxes.
However, when trying to capture the output, I am only getting the last account in the array
Note, this is a simplified version of a larger script, but kept it simple for this example.
$guid = import-csv "c:\temp\Mailboxes.csv"
$DAta = New-Object psobject
$Data | Add-Member -MemberType NoteProperty -Name alias -Value $null
$Data | Add-Member -MemberType NoteProperty -Name guid -Value $null
$mbxcol = #()
#$data = $null
foreach ($G in $Guid){
$mbx = Get-mailbox $g.alias -IncludeInactiveMailbox
$data.alias = $mbx.alias
$data.guid = $mbx.guid
$MBXCol += $Data
}
$mbxcol
As explained in comments, every array element is a reference of the same object ($Data), a simple way to demonstrate using Object.ReferenceEquals Mehod with this example:
foreach ($item in 0..10) {
$data.Alias = 'mailbox{0}' -f $item
$data.Guid = [guid]::NewGuid()
$mbxcol += $data
}
[object]::ReferenceEquals($data, $mbxcol[0]) # -> True
As for how to simplify and make your code more efficient, do not add elements (+=) to a fixed collection (#( )):
$result = (Import-Csv "c:\temp\Mailboxes.csv").Alias |
Get-Mailbox -IncludeInactiveMailbox |
Select-Object Alias, Guid
A much more simple example of your code is:
$guid = ("A","B")
$Data = New-Object psobject
$Data | Add-Member -MemberType NoteProperty -Name alias -Value $null
$mbxcol = #()
foreach ($G in $Guid){
$mbx = $g
$data.alias = $mbx
$MBXCol += $Data
}
$mbxcol
As #Santiago mentioned in his comment, $Data is a reference to an object, so each time you update it, you overwrite it, even if it is in an array. To fix this, instantiate the object each loop as follows:
$guid = ("A","B")
$mbxcol = #()
foreach ($G in $Guid){
$Data = New-Object psobject
$Data | Add-Member -MemberType NoteProperty -Name alias -Value $null
$mbx = $g
$data.alias = $mbx
$MBXCol += $Data
}
$mbxcol

Error with huge list of AD groups. Not giving output

I am trying to automate a powerbi report using AD data. I was doing a test on this code and it will give me a good output that I need for a couple of groups. But I was trying to run this with 250K list of groups and I am getting this error. I tried couple of times and it takes a lot of time and it was not giving me any output.
Error
Exception calling "GetSteppablePipeline" with "1" argument(s): "The
parameter is incorrect. "
Note: All attributes are generic and no custom attributes
$groupUsersList = Get-Content $Path
$tableStorage = #()
$groupUsersList | ForEach-Object {
$GroupProperties = get-adgroup $_ -prop *
$table = new-object psobject
$table | Add-Member -NotePropertyName Name -NotePropertyValue $GroupProperties.Name
$table | Add-Member -NotePropertyName GroupCategory -NotePropertyValue $GroupProperties.GroupCategory
$table | Add-Member -NotePropertyName Owner -NotePropertyValue $GroupProperties.extensionAttribute12.Split("=")[1].split(",").split("|")[0]
$table | Add-Member -NotePropertyName OwnerinfoID -NotePropertyValue $GroupProperties.info.Split("=")[1].split(",").split("|")[0]}
$table | Add-Member -NotePropertyName InCloud -NotePropertyValue $GroupProperties.InCloud
$table | Add-Member -NotePropertyName Created -NotePropertyValue $GroupProperties.Created
$table | Add-Member -NotePropertyName Owner2 -NotePropertyValue $GroupProperties.BusinessOwner.Split("=")[1].Split(",")[0]
$table | Add-Member -NotePropertyName Description -NotePropertyValue $GroupProperties.Description
$table | Add-Member -NotePropertyName Owner3 -NotePropertyValue $GroupProperties.info
$table | Add-Member -NotePropertyName atttrib12 -NotePropertyValue $GroupProperties.extensionAttribute12
$tableStorage += $table
}
$tableStorage | Export-Csv -Path $output'.csv' -NoTypeInformation -Force
output:
Exception calling "GetSteppablePipeline" with "1" argument(s): "The parameter is incorrect. "
Try this code instead, it should be faster than what you're doing. In addition, it should help you understand the errors you're getting.
As you said, you were getting the You cannot call a method on a null-valued expression exception. One possible way you're getting this exception is for example, let's assume that the value for $adGroup.info is this:
$string = 'This is the group information without the equal sign'
Now if we try to split the string as you're doing:
$string.Split("=")[1].split(",").split("|")[0]
You would get that exact same error:
You cannot call a method on a null-valued expression.
At line:1 char:1
+ $string.Split("=")[1].split(",").split("|")[0]
Code
$ErrorActionPreference = 'Stop'
$groupUsersList = Get-Content $Path
$destinationPath = "$HOME\Documents\output.csv"
$tableStorage = [system.collections.generic.list[pscustomobject]]::new()
foreach($group in $groupUsersList)
{
if([string]::IsNullOrWhiteSpace($group))
{
continue
}
try
{
$hash = #{
Identity = $group
Properties = #(
'inCloud','BusinessOwner'
'extensionAttribute12','Info'
'Created','Description'
)
}
$adGroup = Get-ADGroup #hash
}
catch
{
Write-Warning $_
continue
}
try
{
$businessOwner = $adGroup.BusinessOwner.Split("=")[1].Split(",")[0]
}
catch
{
'Can't split this: {0}' -f $adGroup.BusinessOwner
$businessOwner = $adGroup.BusinessOwner
}
try
{
$info = $adGroup.info.Split("=")[1].split(",").split("|")[0]
}
catch
{
'Can't split this: {0}' -f $adGroup.info
$info = $adGroup.info
}
try
{
$owner = $adGroup.extensionAttribute12.Split("=")[1].split(",").split("|")[0]
}
catch
{
'Can't split this: {0}' -f $adGroup.extensionAttribute12
$owner = $adGroup.extensionAttribute12
}
$tableStorage.Add(
[pscustomobject]#{
Name = $adGroup.Name
GroupCategory = $adGroup.GroupCategory
Owner = $owner
OwnerinfoID = $info
InCloud = $adGroup.InCloud
Created = $adGroup.Created
Owner2 = $businessOwner
Description = $adGroup.Description
Owner3 = $adGroup.info
atttrib12 = $adGroup.extensionAttribute12
})
}
if($tableStorage)
{
$tableStorage | Export-Csv -Path $destinationPath -NoTypeInformation -Force
"CSV Exported successfully to $destinationPath"
}
A few points to consider
Adding items to System.Array #() is a lot slower than adding items to a collections.generic.list.
Creating objects with new psobject + Add-Member is a lot slower than casting objects with pscustomobject.
Calling ALL properties on Get-ADGroup -Properties * is a lot slower than calling only the properties you need -Properties prop1,prop2,prop3.

Powershell - extract to csv from the loop

when i tried to extact results from this code
$Output = #()
foreach($emailproxy in $emailproxies)
{
if($emailproxy.Mail -like "*com" -or $emailproxy.Mail -like "*org"){ Write-Host $emailproxy.Mail}
$Output = New-Object -TypeName PSObject -Property #{
Mail = $emailproxy.Mail
} | Select-Object Mail
}
$Output | Export-Csv C:\TempDownloads\results.csv
I only get 1 result .. why is that ?
For each iteration in the loop, you are assigning (overwriting) the value directly to the $Output variable, instead of adding the value to the array.
Inside the for loop, you need to replace $Output = with $Output +=.
$Output = <new value> -> Assign <new value> to $Output variable, existing value of $Output is overwritten and lost.
$Output += <new value> is equivalent to $Output = $Output + <new value>, where <new value> and current value of $Output are first combined and then assigned to $Output.
This is because you are overwrighting the variable $output each time. You need to append properties to the psobject using add-member
Output = New-Object -TypeName Psobject
foreach($emailproxy in $emailproxies) { if($emailproxy.Mail -like "*com" -or $emailproxy.Mail -like "*org"){ Write-Host $emailproxy.Mail}
$Output | Add-Member -MemberType NoteProperty -Name "Mail" -Value $_.mail
}
$Output | Export-Csv "filepath"

Improving the speed of Get-FileMetaData

I'm currently using the below script taken from scriptingguys.com (all credit to them, I just added the bottom 2 lines.) That takes a directory and pulls the file path and comments field from the meta data of the files. Currently the script take's a little over 1.5 minutes to fully run. Is there anyway to speed this up or use a different method to get this data?
I am using this script at the start of some software I have written and 1.5+ minutes is too long for the script to complete. Any thoughts/comments?
Function Get-FileMetaData
{
Param([string[]]$folder)
foreach($sFolder in $folder)
{
$a = 0
$objShell = New-Object -ComObject Shell.Application
$objFolder = $objShell.namespace($sFolder)
foreach ($File in $objFolder.items())
{
$FileMetaData = New-Object PSOBJECT
for ($a ; $a -le 266; $a++)
{
if($objFolder.getDetailsOf($File, $a))
{
$hash += #{$($objFolder.getDetailsOf($objFolder.items, $a)) =
$($objFolder.getDetailsOf($File, $a)) }
$FileMetaData | Add-Member $hash
$hash.clear()
} #end if
} #end for
$a=0
$FileMetaData
} #end foreach $file
} #end foreach $sfolder
} #end Get-FileMetaData
$fileMetaData = Get-FileMetaData -folder "C:\Pics" | select 'Name', 'Path', 'Comments' | Sort-Object 'Name'
$fileMetaData | select 'Name', 'Path', 'Comments' | Export-CSV "C:\SCRIPTS\TestDirectory.txt" -encoding Utf8 -NoTypeInformation
Solved by wOxxOm, thanks for your help! Running the below and now working.
Function Get-FileMetaData(
[string[]]$folders,
[string[]]$properties
) {
$shellApp = New-Object -ComObject Shell.Application
$supportsOrdered = $PSVersionTable.PSVersion.Major -ge 3
$hash = if ($supportsOrdered) { [ordered]#{} } else { #{} }
# walk the folders and get the properties by index found above
$folders | ForEach {
$shellFolder = $shellApp.namespace($_)
# get all headers and find their indexes
$allProps = #{}
foreach ($index in 0..266) {
$allProps[$shellFolder.getDetailsOf($shellFolder.items, $index)] = $index
}
$shellFolder.items() | ForEach {
$file = $_
$hash.Clear()
foreach ($prop in $properties) {
if (($index = $allProps[$prop]) -ne $null) {
if ($value = $shellFolder.getDetailsOf($file, $index)) {
$hash[$prop] = $value
}
}
}
if ($supportsOrdered) {
[PSCustomObject]$hash
} else {
Select $properties -inputObject (
New-Object PSObject -Property $hash
)
}
}
}
}
Get-FileMetaData -folders 'C:\PICS' -properties Name, Path, Comments | Sort-Object Name |
select Name, Path, Comments | Export-Csv 'C:\Scripts\test.txt' -encoding UTF8 -NoTypeInformation
getDetailsOf is slow, and your code needlessly invokes it 267 times for each file when you only need it for 3 properties.
Collect the property names just once at the start of the function, don't do it on every file
Add-Member is slow. Don't invoke it on every property. Collect all found properties in a hashtable and pass it once to Add-Member or, since you create an empty object, directly to New-Object. To enforce the order of properties use Select-Object in PowerShell 2. Note, PowerShell 3.0 and newer support [ordered] and [PSCustomObject] typecast (see the code below).
Use pipelining instead of foreach statements so that the results appear immediately
Files are already sorted by name, at least on NTFS file system in Windows, so no need to sort.
Function Get-FileMetaData(
[string[]]$folders,
[string[]]$properties
) {
$shellApp = New-Object -ComObject Shell.Application
# get all headers and find their indexes
$shellFolder = $shellApp.namespace($folders[0])
$allProps = #{}
foreach ($index in 0..266) {
$allProps[$shellFolder.getDetailsOf($shellFolder.items, $index)] = $index
}
$supportsOrdered = $PSVersionTable.PSVersion.Major -ge 3
$hash = if ($supportsOrdered) { [ordered]#{} } else { #{} }
# walk the folders and get the properties by index found above
$folders | ForEach {
$shellFolder = $shellApp.namespace($_)
$shellFolder.items() | ForEach {
$file = $_
$hash.Clear()
foreach ($prop in $properties) {
if (($index = $allProps[$prop]) -ne $null) {
$hash[$prop] = $shellFolder.getDetailsOf($file, $index)
}
}
if ($supportsOrdered) {
[PSCustomObject]$hash
} else {
Select $properties -inputObject (
New-Object PSObject -Property $hash
)
}
}
}
}
Usage example 1:
Get-FileMetaData -folders 'r:\folder1', 'r:\folder2' -properties Name, Path, Comments
Usage example 2:
Get-FileMetaData -folders 'r:\folder1', 'r:\folder2' -properties Name, Path, Comments |
Export-Csv r:\results.csv -encoding UTF8 -NoTypeInformation
Usage example 3 gets all properties, which is slow:
Get-FileMetaData -folders 'r:\folder1', 'r:\folder2'

Powershell how to convert System.Collections.Generic.Dictionary to PSCustomObject?

I have a System.Collections.Generic.Dictionary that I have obtained by running the code snippet below. How do I convert $Obj to a PSCustomObject?
[void][System.Reflection.Assembly]::LoadWithPartialName("System.Web.Extensions")
$jsonserial= New-Object -TypeName System.Web.Script.Serialization.JavaScriptSerializer
$jsonserial.MaxJsonLength = 67108864
$Obj = $jsonserial.DeserializeObject($JsonString)
New-Object should work from PS 2.0 onwards:
$custom = new-object PSCustomObject -Property $obj
I ended up using the following after the last line in the question
$Obj | ForEach-Object {
$props = #{}
$_.GetEnumerator() | ForEach-Object {
$props[$_.Key] = $_.Value
}
[PSCustomObject]$props
}
If there is a faster, shorter or better way, please feel free to answer
This seems to work:
$ht = #{}
$ht += $obj
[PSCustomObject]$ht
Edit:
V3 solution:
$ht = [collections.hashtable]$obj
[PSCustomObject]$ht