PowerShell - Expand array column in array with repeating rows - powershell

Question

I have an System.Array object called $Data, the first ([0]) item looks like this:
RecordDate : {43739, 43740, 43741, 43742...}
MAX_LAST_UPDATE_DATE : 30/10/2019 14:08:33
EMPLOYEE_NUMBER : 1000522
EFFECTIVE_START_DATE : 01/10/2019 00:00:00
EFFECTIVE_END_DATE : 31/12/4712 00:00:00
CC : 0726
REGION_NAME : Head Office
LOCATION_NAME : Inventory
FIRST_NAME : Name
MIDDLE_NAMES : Mid
LAST_NAME : Last
KNOWN_AS : NickName
JOB_TITLE : Inventory Manager
WORK_NUMBER :
Employment Category : Full Time
NORMAL_HOURS : 40
GROUP_NAME : Indirect
Manager Employee Number : 1034422
PERSON_TYPE : Employee
HIRE_DATE : 16/11/1983 00:00:00
TERMINATION_DATE :
DATE_OF_BIRTH : 23/05/1966 00:00:00
NATIONAL_IDENTIFIER : 111
I'm trying to kind of unpivot the first column "RecordDate" on the entire array like this:
RecordDate : 43739
MAX_LAST_UPDATE_DATE : 30/10/2019 14:08:33
EMPLOYEE_NUMBER : 1000522
EFFECTIVE_START_DATE : 01/10/2019 00:00:00
EFFECTIVE_END_DATE : 31/12/4712 00:00:00
CC : 0726
REGION_NAME : Head Office
LOCATION_NAME : Inventory
FIRST_NAME : Name
MIDDLE_NAMES : Mid
LAST_NAME : Last
KNOWN_AS : NickName
JOB_TITLE : Inventory Manager
WORK_NUMBER :
Employment Category : Full Time
NORMAL_HOURS : 40
GROUP_NAME : Indirect
Manager Employee Number : 1034422
PERSON_TYPE : Employee
HIRE_DATE : 16/11/1983 00:00:00
TERMINATION_DATE :
DATE_OF_BIRTH : 23/05/1966 00:00:00
NATIONAL_IDENTIFIER : 111
RecordDate : 43740
MAX_LAST_UPDATE_DATE : 30/10/2019 14:08:33
EMPLOYEE_NUMBER : 1000522
EFFECTIVE_START_DATE : 01/10/2019 00:00:00
EFFECTIVE_END_DATE : 31/12/4712 00:00:00
CC : 0726
REGION_NAME : Head Office
LOCATION_NAME : Inventory
FIRST_NAME : Name
MIDDLE_NAMES : Mid
LAST_NAME : Last
KNOWN_AS : NickName
JOB_TITLE : Inventory Manager
WORK_NUMBER :
Employment Category : Full Time
NORMAL_HOURS : 40
GROUP_NAME : Indirect
Manager Employee Number : 1034422
PERSON_TYPE : Employee
HIRE_DATE : 16/11/1983 00:00:00
TERMINATION_DATE :
DATE_OF_BIRTH : 23/05/1966 00:00:00
NATIONAL_IDENTIFIER : 111
RecordDate : 43741
MAX_LAST_UPDATE_DATE : 30/10/2019 14:08:33
...
Is there a way to do that with some sneaky Select -expandproperty or do the opposite of what Group-Object is capable of; without doing combination of for($i) and for($j) loops?
It's quite simple on a Table in in Excel PowerQuery, as you just click Expand and voilà.
Regards, Jarek

Answer 1

You can combine Select-Object -ExpandProperty with the common -PipelineVariable parameter and cloning (PSv3+ syntax):
For input collections of [pscustomobject] or [hashtable] instances:
# Sample input array of custom objects to expand by .RecordDate
$array =
[pscustomobject] #{ RecordDate = 1, 2; OtherProp1 = 'one'; OtherProp2 = 'two' },
[pscustomobject] #{ RecordDate = 3, 4; OtherProp1 = 'three'; OtherProp2 = 'four' }
# Write the array elements to the pipeline, and store each in variable
# $objectOrHashtable for use in a later pipeline segment.
Write-Output $array -PipelineVariable objectOrHashtable |
# Expand the input object's .RecordData property, i.e. send its
# elements one by one to the next pipeline segment.
Select-Object -ExpandProperty RecordDate |
ForEach-Object {
# Clone the original input object.
$clone = if ($objectOrHashtable -is [Management.Automation.PSCustomObject]) {
$objectOrHashtable.psobject.Copy()
} else { # assume [hashtable] or a type that implements [System.ICloneable]
$objectOrHashtable.Clone()
}
# Assign the record date at hand to the clone...
$clone.RecordDate = $_
# ... and output it.
$clone
}
The above yields the following; note that 4 objects were output, based on enumerating the elements of the input objects' .RecordDate array while retaining all other properties:
RecordDate OtherProp1 OtherProp2
---------- ---------- ----------
1 one two
2 one two
3 three four
4 three four
Note:
The above works with two types of input object:
custom objects ([pscustomobject] instances, such as created by Import-Csv)
Note: For technical reasons you cannot use -is [pscustomobject] and must instead use the full type name, System.Management.Automation.PSCustomObject (the System. prefix can be omitted); [pscustomobject], for historical reasons, is the same as [psobject] (System.Management.Automation.PSObject), and -is [psobject] is also true for objects that aren't custom objects.
hashtables (System.Collections.Hashtable instances - but not [ordered] hashtables); more generally, any type that implements System.ICloneable.
The cloning that is performed on custom objects and hashtable is shallow (member-wise), but with scalar string and numeric values that is sufficient.
Generally, the ICloneable interface doesn't prescribe the specifics of the cloning behavior, which is why its use is generally discouraged.
For input collections of [System.Data.DataRow] instances:
Cloning a collection of System.Data.DataRow instances - the rows of data table, System.Data.DataTable - requires custom cloning logic, but the approach and structure of the output are fundamentally the same:
# Create a sample DataTable...
$dt = [System.Data.DataTable]::new('sample')
# ... define the columns ...
$dt.Columns.AddRange([System.Data.DataColumn[]] (
#{ ColumnName = 'RecordDate'; DataType = [object[]] },
#{ ColumnName = 'OtherProp1'; DataType = [string] },
#{ ColumnName = 'OtherProp2'; DataType = [string] }
))
# ...and add sample rows.
#{ RecordDate = 1, 2; OtherProp1 = 'one'; OtherProp2 = 'two' },
#{ RecordDate = 3, 4; OtherProp1 = 'three'; OtherProp2 = 'four' } | % {
$dt.Rows.Add(($dr = $dt.NewRow()))
foreach ($entry in $_.GetEnumerator()) {
$dr[$entry.Key] = $entry.Value
}
}
# Create an auxiliary, empty clone of the input data table
# to facilitate cloning of individual rows.
$dtAux = $dt.Clone()
# Write the data rows to the pipeline, and store each in variable
# $obj for use in a later pipeline segment.
Write-Output $dt.Rows -PipelineVariable row |
# Expand the input object's .RecordData property, i.e. send its
# elements one by one to the next pipeline segment.
Select-Object -ExpandProperty RecordDate |
ForEach-Object {
# Clone the data row at hand.
$dtAux.Clear(); $dtAux.ImportRow($row)
$clone = $dtAux.Rows[0]
# Assign the record date at hand to the clone...
$clone.RecordDate = #($_)
# ... and output it.
$clone
}