Expanding System.Object[] for Export to CSV - powershell

Hello again and sorry!
To keep it really short. What am i doing wrong?
Im attempting to export a list of users filtered by using a customobject to a CSV and it outputs it into the same block. Is there no way to change this? I only ask because, all the other pages ive looked at it keeps telling me to use -join, to join them as strings which does the exact same thing. Is it not possible to output it as multiple rows for each user?
$GPMem = Get-ADGroupMember -Identity security.group | Select-Object -ExpandProperty Name
[array]$TPpl = $GPMem | Where-Object {$_ -like "T*"}
[array]$RPpl = $GPMem | Where-Object {$_ -like "r*"}
[array]$CPpl = $GPMem | Where-Object {$_ -like "c*"}
[pscustomobject]#{
TPeople = (#($TPpl) |Out-String).Trim()
TPCount = $TPpl.Count
RPeople = (#($RPpl) |Out-String).ToString()
RPCount = $TPpl.Count
CPeople = $CPpl
CPCount = $TPpl.Count
} | Export-Csv -Path C:\Users\abraham\Desktop\csv.csv -NoTypeInformation -Force

here is how to insert a ; OR newline into the values of a column ... [grin]
$ThingList = #('One Thing', 'Two Thing', 'Three Thing')
$ThingList -join '; '
'=' * 20
$ThingList -join [System.Environment]::NewLine
output ...
One Thing; Two Thing; Three Thing
====================
One Thing
Two Thing
Three Thing

create 3 more arrays for the count (each array will be exported to a column), then find the array with the most count and generate a psobject for each line.
[array]$TPpl = #("T1" ,"T2", "T3")
[array]$TPpl_count = #($TPpl.Count)
[array]$RPpl = #("R1" ,"R2", "R3", "R4")
[array]$RPpl_count = #($RPpl.Count)
[array]$CPpl = #("C1" ,"C2", "C3", "C4","C5")
[array]$CPpl_count = #($CPpl.Count)
$leng = [array]$TPpl.Count,$RPpl.Count,$CPpl.Count
$max = ($leng | measure -Maximum).Maximum
$csv = for($i=0;$i -lt $max;$i++){
New-Object -TypeName psobject -Property #{
"TPeople" = $(if ($TPpl[$i]) { $TPpl[$i]})
"TPCount" = $(if ($TPpl_count[$i]) { $TPpl_count[$i]})
"RPeople" = $(if ($RPpl[$i]) { $RPpl[$i]})
"RPCount" = $(if ($RPpl_count[$i]) { $RPpl_count[$i]})
"CPeople" = $(if ($CPpl[$i]) { $CPpl[$i]})
"CPCount" = $(if ($CPpl_count[$i]) { $CPpl_count[$i]})
}
}
$csv | Export-Csv C:\Temp\test.csv -NoTypeInformation
result:
your final code should be:
$GPMem = Get-ADGroupMember -Identity security.group | Select-Object -ExpandProperty Name
[array]$TPpl = $GPMem | Where-Object {$_ -like "T*"}
[array]$RPpl = $GPMem | Where-Object {$_ -like "r*"}
[array]$CPpl = $GPMem | Where-Object {$_ -like "c*"}
[array]$TPpl_count = #($TPpl.Count)
[array]$RPpl_count = #($RPpl.Count)
[array]$CPpl_count = #($CPpl.Count)
$leng = [array]$TPpl.Count,$RPpl.Count,$CPpl.Count
$max = ($leng | measure -Maximum).Maximum
$csv = for($i=0;$i -lt $max;$i++){
New-Object -TypeName psobject -Property #{
"TPeople" = $(if ($TPpl[$i]) { $TPpl[$i]})
"TPCount" = $(if ($TPpl_count[$i]) { $TPpl_count[$i]})
"RPeople" = $(if ($RPpl[$i]) { $RPpl[$i]})
"RPCount" = $(if ($RPpl_count[$i]) { $RPpl_count[$i]})
"CPeople" = $(if ($CPpl[$i]) { $CPpl[$i]})
"CPCount" = $(if ($CPpl_count[$i]) { $CPpl_count[$i]})
}
}
$csv | Export-Csv C:\Temp\test.csv -NoTypeInformation

Related

PowerShell Output to CSV Not Placing Columns in Correct Order

How do I code this to output the columns in this order:
unique_sis_group_id, unique_sis_user_id, unique_sis_school_id, mm_admin
My code:
$objs = #();
$output = Import-Csv -Path "c:\users\patrick\desktop\RelayFiles\RelayMemberships\file2concatenate.csv" | ForEach {
$Object = New-Object PSObject -Property #{
unique_sis_group_id = [String]::Concat($_.unique_sis_group_id, $_.unique_sis_user_id)
unique_sis_user_id = $_.unique_sis_school_id
unique_sis_school_id = $_.mm_admin
mm_admin = 0
}
$objs += $Object;
}
$objs
$objs | Export-Csv -NoTypeInformation c:\users\patrick\desktop\RelayFiles\RelayMemberships\memberships.csv
You assign the ForEach-Object to $output, but nothing inside actually outputs something.
Using a [PSCustomObject] is much simpler:
## Q:\Test\2019\01\25\SO_54373962.ps1
$CsvIn = "c:\users\patrick\desktop\RelayFiles\RelayMemberships\file2concatenate.csv"
$CsvOut = "c:\users\patrick\desktop\RelayFiles\RelayMemberships\memberships.csv"
$output = Import-Csv -Path $CsvIn | ForEach-Object {
[PSCustomObject]#{
unique_sis_group_id = [String]::Concat($_.unique_sis_group_id, $_.unique_sis_user_id)
unique_sis_user_id = $_.unique_sis_school_id
unique_sis_school_id = $_.mm_admin
mm_admin = 0
}
}
$output
$output | Export-Csv $CsvOut -NoTypeInformation
This took care of it. Is there a better way?
$objs =#();
$output = Import-Csv -Path "c:\users\patrick\desktop\RelayFiles\RelayMemberships\file2concatenate.csv" | ForEach {
$Object = New-Object PSObject -Property #{
unique_sis_group_id = [String]::Concat($_.unique_sis_group_id, $_.unique_sis_user_id)
unique_sis_user_id = $_.unique_sis_school_id
unique_sis_school_id = $_.mm_admin
mm_admin = 0
}
$objs += $Object;
}
$objs |
Select-Object "unique_sis_group_id", "unique_sis_user_id",
"unique_sis_school_id", "mm_admin" |
Export-CSv -NoTypeInformation c:\users\patrick\desktop\RelayFiles\RelayMemberships\memberships.csv

Converting CSV Column to Rows

I'm trying to create a PowerShell script that transpose a CSV file from column to rows.
I found examples of doing the opposite (converting row based CSV to column) but I found nothing on column to rows. My problem being that I don't know exactly how many column I'll have. I tried adapting the row to column to column to rows but unsuccessfully.
$a = Import-Csv "input.csv"
$a | FT -AutoSize
$b = #()
foreach ($Property in $a.Property | Select -Unique) {
$Props = [ordered]#{ Property = $Property }
foreach ($Server in $a.Server | Select -Unique){
$Value = ($a.where({ $_.Server -eq $Server -and
$_.Property -eq $Property })).Value
$Props += #{ $Server = $Value }
}
$b += New-Object -TypeName PSObject -Property $Props
}
$b | FT -AutoSize
$b | Out-GridView
$b | Export-Csv "output.csv" -NoTypeInformation
For example my CSV can look like this:
"ID","DATA1"
"12345","11111"
"54321","11111"
"23456","44444"
or this (number of column can vary):
"ID","DATA1","DATA2","DATA3"
"12345","11111","22222","33333"
"54321","11111",,
"23456","44444","55555",
and I would like the script to convert it like this:
"ID","DATA"
"12345","11111"
"12345","22222"
"12345","33333"
"54321","11111"
"23456","44444"
"23456","55555"
The trick is to query the members of the table to get the column names. Once you do that then the rest is straightforward:
function Flip-Table ($Table) {
Process {
$Row = $_
# Get all the columns names, excluding the ID field.
$Columns = ($Row | Get-Member -Type NoteProperty | Where-Object Name -ne ID).Name
foreach ($Column in $Columns) {
if ($Row.$Column) {
$Properties = [Ordered] #{
"ID" = $Row.ID
"DATA" = $Row.$Column
}
New-Object PSObject -Property $Properties
}
}
# Garbage collection won't kick in until the end of the script, so
# invoke it every 100 input rows.
$Count++;
if (($Count % 100) -eq 0) {
[System.GC]::GetTotalMemory('forceFullCollection') | out-null
}
}
}
Import-Csv input.csv | Flip-Table | Export-Csv -NoTypeInformation output.csv
Well, here is mine. I'm not as fancy as the rest:
$in = Get-Content input.csv | Select -Skip 1
$out = New-Object System.Collections.ArrayList
foreach($row in $in){
$parts = $row.Split(',')
$id = $parts[0]
foreach($data in $parts[1..$parts.Count]){
if($data -ne '' -AND $data -ne $null){
$temp = New-Object PSCustomObject -Property #{'ID' = $id;
'Data' = $data}
$out.Add($temp) | Out-Null
}
}
}
$out | Export-CSV output.csv -NoTypeInformation
You can do something like this
# Convert csv to object
$csv = ConvertFrom-Csv #"
"ID","DATA1","DATA2","DATA3"
"12345","11111","22222","33333"
"54321","11111",,
"23456","44444","55555"
"#
# Ignore common members and the ID property
$excludedMembers = #(
'GetHashCode',
'GetType',
'ToString',
'Equals',
'ID'
)
$results = #()
# Iterate around each csv row
foreach ($row in $csv) {
$members = $row | Get-Member
# Iterate around each member from the 'row object' apart from our
# exclusions and empty values
foreach ($member in $members |
Where { $excludedMembers -notcontains $_.Name -and $row.($_.Name)}) {
# add to array of objects
$results += #{ ID=$row.ID; DATA=$row.($member.Name)}
}
}
# Write the csv string
$outstring = "ID,DATA"
$results | foreach { $outstring += "`n$($_.ID),$($_.DATA)" }
# New csv object
$csv = $outstring | ConvertFrom-Csv
Probably not the most elegant solution, but should do what you need
I left some comments explaining what it does
If you only want to accept a limited number DATA columns (e.g. 5), you could do:
ForEach ($i in 1..5) {$CSV | ? {$_."Data$i"} | Select ID, #{N='Data'; E={$_."Data$i"}}}
And if you have a potential unlimited number of DATA columns:
ForEach ($Data in ($CSV | Select "Data*" -First 1).PSObject.Properties.Name) {
$CSV | ? {$_.$Data} | Select ID, #{N='Data'; E={$_.$Data}}
}

Why is my output from Get-EC2VPC exporting as an empty CSV?

I have this codes below all working OK - however when i try to export to xls it does not export anything - i am getting blind now... any one can help?
$StoppedInstances = (Get-EC2Instance).instances | Where-Object {$_.State.Name -eq "stopped" -or $_.State.Name -eq "running"}
$VPCS = Get-EC2Vpc
foreach ($VPC in $VPCS) {
$StoppedInstances | Where-Object {$_.VpcId -eq $VPC.VpcId} | foreach {
New-Object -TypeName PSObject -Property #{
'InstanceId' = $_.InstanceId
'InstanceName' = ($_.Tags | Where-Object {$_.Key -eq 'Name'}).Value
'LaunchTime' = $_.LaunchTime
'State' = $_.State.Name
#'State1' = $_.State.GetType()
'Private IP' = $_.PrivateIpAddress
'Public IP' = $_.PublicIpAddress
'Public Dns' = $_.PublicDnsName
'loadbalace' = $_.AmiLaunchIndex
'vpcID' = $_.VpcId
'instancetype' = $_.InstanceType
'EBSDISK' = $_.BlockDeviceMappings.Count
'ELB' = $_.NetworkInterfaces
} | Format-Table -GroupBy date -Wrap | Export-Csv C:\temp\test4.csv
}
}
Because you're piping into Format-Table. Format-Table is used ONLY when you send data to the screen, like when you want to view something in the console. Remove your Format-Table statement and this will work as is.
In this example I use Tee-Object to snap out a variable containing the output of your command, and then send the main stream on to Format-Table for viewing.
Then, in the next step, we pipe that variable into the CSV file you want to export.
$StoppedInstances = (Get-EC2Instance).instances | Where-Object {$_.State.Name -eq "stopped" -or $_.State.Name -eq "running"}
$VPCS = Get-EC2Vpc
$export = foreach ($VPC in $VPCS) {
$StoppedInstances | Where-Object {$_.VpcId -eq $VPC.VpcId} | foreach {
New-Object -TypeName PSObject -Property #{
'InstanceId' = $_.InstanceId
'InstanceName' = ($_.Tags | Where-Object {$_.Key -eq 'Name'}).Value
'LaunchTime' = $_.LaunchTime
'State' = $_.State.Name
#'State1' = $_.State.GetType()
'Private IP' = $_.PrivateIpAddress
'Public IP' = $_.PublicIpAddress
'Public Dns' = $_.PublicDnsName
'loadbalace' = $_.AmiLaunchIndex
'vpcID' = $_.VpcId
'instancetype' = $_.InstanceType
'EBSDISK' = $_.BlockDeviceMappings.Count
'ELB' = $_.NetworkInterfaces
}
}
}
$export | Format-Table -GroupBy date -Wrap
$export | Export-Csv C:\temp\test4.csv

Change the script to export Groups and nested objects differently

I created a similiar script like that:
$Groups = Get-QADGroup
$Result = #()
$Groups | ForEach-Object {
$Group = $_
$Members = Get-QADGroupMember $Group -Indirect | ? objectClass -eq "user"
$Obj = '' | Select-Object -Property Name, Members
$Obj.Name = $Group.Name
$Obj.Members = ($Members | % {$_.SamAccountName + "_" + 'Test'})
$Result += $Obj
}
$Result | Export-Csv -Path C:\Temp\groups.csv -NoTypeInformation -Encoding Unicode -Delimiter ";"
The output looks something like this (example data):
"Name";"Members"
"RootGroup01";"Subuser01_Test";"Subuser02_Test";"Subuser03_Test"
"RootGroup02";"Subuser02_Test"
"RootGroup03";"Subuser01_Test";"Subuser02_Test";"Subuser04_Test";
Is it possible to change the script, that I get something like this?:
"RootGroup01";"RootGroup02";"RootGroup03"
"Subuser01_Test";"Subuser02_Test";"Subuser01_Test"
"Subuser02_Test";;"Subuser02_Test"
"Subuser03_Test";;"Subuser04_Test"
The group names should be the header and the belonging users are in the right column. If there is no user, the column cell just stays empty.
You can write such function to transpose your data:
function Transpose-Data{
param(
[String[]]$Names,
[Object[][]]$Data
)
for($i = 0;; ++$i){
$Props = [ordered]#{}
for($j = 0; $j -lt $Data.Length; ++$j){
if($i -lt $Data[$j].Length){
$Props.Add($Names[$j], $Data[$j][$i])
}
}
if(!$Props.get_Count()){
break
}
[PSCustomObject]$Props
}
}
Then you invoke it in the following way:
$Groups = #(Get-QADGroup)
$Members = #(
$Groups | ForEach-Object {
,#(
Get-QADGroupMember $_ -Indirect |
? objectClass -eq user |
% {$_.SamAccountName + "_" + 'Test'}
)
}
)
Transpose-Data $Groups.Name $Members |
Export-Csv -Path C:\Temp\groups.csv -NoTypeInformation -Encoding Unicode -Delimiter ";"

Filter csv file by array of hashtables

What I am trying todo is filter a csv file (happens to be a weblog) with an array of hashtables (user information from a database).
#$data is from a database. (about 500 items) Type: System.Data.DataTable
$users = #()
foreach($row in $data)
{
$userItem = #{
LoginId = $row[0]
LastName = $row[3]
FirstName = $row[4]
LastAccess = $null
}
$users += $userItem
}
#Log files are about 14,000 lines long
$logfiles = Get-ChildItem $logFolder -Recurse | where {$_.Extension -eq ".log"} | Sort-Object BaseName -Descending
foreach($log in $logfiles)
{
$csvLog = Import-Csv $log.FullName -Header ("Blank","LoginId","Date")
$u = $users | Select {&_.LoginId}
$filteredcsvLog = $cvsLog | Where-Object { $u -contains $_.LoginId}
#This returns null
....
}
This does not seem to work, what am I missing. My guess is that I need to flatten the array into [string[]], however I can't seem todo that either.
Rather than do an array of hashtables, I would do a hashtable of custom objects e.g.:
$users = #{}
foreach($row in $data)
{
$userItem = new-object psobject -property #{
LoginId = $row[0]
LastName = $row[3]
FirstName = $row[4]
LastAccess = $null
}
$users[$userItem.LoginId] = $userItem
}
Then the filtering is easier and faster:
foreach($log in $logfiles)
{
$csvLog = Import-Csv $log.FullName -Header ("Blank","LoginId","Date")
$filteredcsvLog = $cvsLog | Where-Object { $users[$_.LoginId} }
....
}