Suppose I have object $foo with many (500+) properties.
$foo.q1_sales = "1000"
$foo.q1_expense = "800"
$foo.q2_sales = "1325"
$foo.q2_expense = "1168"
$foo.q3_sales = "895"
$foo.q3_expense = "980"
$foo.q4_sales = "900"
$foo.q4_expense = "875"
...
I want to loop through all properties in $foo and get each value and process it in some way.
$quarters = #("1","2","3","4")
foreach($quarter in $quarters) {
if($foo.q$quarter_sales -gt $foo.q$quarter_expense) {
#process data
}
}
How do I accomplish this? Get-Variable? Get-Member? some combination? Some other way?
Changing the structure of $foo is not an option, unless we can do it programmatically. Sorry.
You can use a subexpression to evaluate the property name, such as:
$quarters = #("1","2","3","4")
foreach($quarter in $quarters) {
if($foo.$("q"+$quarter+"_sales") -gt $foo.$("q"+$quarter+"_expense")) {
#process data
}
}
That will evaluate the sub-expressions first, so it figures out "q"+$quarter+"_sales" and then just evaluates $foo.q1_sales as a result.
Get-Member is a good thought. Here's a generalized attempt to expand on that, so that you can see how it could be done. For my example, my object will be a DateTime:
# Define our object
$object = (Get-Date)
# Get the property names
$properties = $object | Get-Member -MemberType "Property" | % { $_.Name }
# Get our collection of values by iterating the collection of properties
# and for each property getting the value.
$values = $properties | % { $object."$_" }
And then the output would just be the values of each property of DateTime:
$values
Tuesday, October 14, 2014 12:00:00 AM
14
Tuesday
287
17
Local
972
44
10
45
635489054859729996
Ticks : 638859729996
Days : 0
Hours : 17
Milliseconds : 972
Minutes : 44
Seconds : 45
TotalDays : 0.739420983791667
TotalHours : 17.746103611
TotalMilliseconds : 63885972.9996
TotalMinutes : 1064.76621666
TotalSeconds : 63885.9729996
2014
This assumes that you only want MemberTypes of "Property", so this may not go far enough if you're also after NoteProperty membertypes. Maybe "*Property" for that?
Otherwise, this should work for any arbitrary type with properties. Just swap out my $object with yours.
#TheMadTechnician has already provided the simplest answer. Here's an alternative way:
#Sampledata
$foo = New-Object psobject -Property #{
q1_sales = "1000"
q1_expense = "800"
q2_sales = "1325"
q2_expense = "1168"
q3_sales = "895"
q3_expense = "980"
q4_sales = "900"
q4_expense = "875"
}
#Convert to array
$array = $foo.psobject.Properties |
#Group by quarter id(number) so we can create an object per quarter
Group-Object { $_.Name -replace 'q(\d+)_.*', '$1' } |
ForEach-Object {
New-Object psobject -Property #{
Quarter = $_.Name
Sales = [int]($_.Group | Where-Object { $_.Name -match 'sales' } | Select-Object -ExpandProperty Value)
Expense = [int]($_.Group | Where-Object { $_.Name -match 'expense' } | Select-Object -ExpandProperty Value)
}
}
#Get quarters with positive result
$array | Where-Object { $_.Sales -gt $_.Expense } | ForEach-Object {
#process data (sends object through in this sample)
$_
}
Output:
Sales Expense Quarter
----- ------- -------
1325 1168 2
1000 800 1
900 875 4
Related
I have a JSON file that looks like this:
{
"id": 10011,
"title": "Test procedure",
"slug": "slug",
"url": "http://test.test",
"email": "test#test.com",
"link": "http://test.er",
"subject": "testing",
"level": 1,
"disciplines": [
"discipline_a",
"discipline_b",
"discipline_c"
],
"areas": [
"area_a",
"area_b"
]
},
I was trying to use the following command to convert that into the CSV file:
(Get-Content "PATH_TO\test.json" -Raw | ConvertFrom-Json)| Convertto-CSV -NoTypeInformation | Set-Content "PATH_TO\test.csv"
However, for disciplines and areas I am getting System.Object[] in the resulting CSV file.
Is there a way to put all those nested values as a separate columns in CSV file like area_1, area_2 etc. And the same for disciplines.
2017-11-20, Completely rewrote function to improve performance and add features as -ArrayBase and support for PSStandardMembers and grouped objects.
Flatten-Object
Recursively flattens objects containing arrays, hash tables and (custom) objects. All added properties of the supplied objects will be aligned with the rest of the objects.
Requires PowerShell version 2 or higher.
Cmdlet
Function Flatten-Object { # Version 00.02.12, by iRon
[CmdletBinding()]Param (
[Parameter(ValueFromPipeLine = $True)][Object[]]$Objects,
[String]$Separator = ".", [ValidateSet("", 0, 1)]$Base = 1, [Int]$Depth = 5, [Int]$Uncut = 1,
[String[]]$ToString = ([String], [DateTime], [TimeSpan]), [String[]]$Path = #()
)
$PipeLine = $Input | ForEach {$_}; If ($PipeLine) {$Objects = $PipeLine}
If (#(Get-PSCallStack)[1].Command -eq $MyInvocation.MyCommand.Name -or #(Get-PSCallStack)[1].Command -eq "<position>") {
$Object = #($Objects)[0]; $Iterate = New-Object System.Collections.Specialized.OrderedDictionary
If ($ToString | Where {$Object -is $_}) {$Object = $Object.ToString()}
ElseIf ($Depth) {$Depth--
If ($Object.GetEnumerator.OverloadDefinitions -match "[\W]IDictionaryEnumerator[\W]") {
$Iterate = $Object
} ElseIf ($Object.GetEnumerator.OverloadDefinitions -match "[\W]IEnumerator[\W]") {
$Object.GetEnumerator() | ForEach -Begin {$i = $Base} {$Iterate.($i) = $_; $i += 1}
} Else {
$Names = If ($Uncut) {$Uncut--} Else {$Object.PSStandardMembers.DefaultDisplayPropertySet.ReferencedPropertyNames}
If (!$Names) {$Names = $Object.PSObject.Properties | Where {$_.IsGettable} | Select -Expand Name}
If ($Names) {$Names | ForEach {$Iterate.$_ = $Object.$_}}
}
}
If (#($Iterate.Keys).Count) {
$Iterate.Keys | ForEach {
Flatten-Object #(,$Iterate.$_) $Separator $Base $Depth $Uncut $ToString ($Path + $_)
}
} Else {$Property.(($Path | Where {$_}) -Join $Separator) = $Object}
} ElseIf ($Objects -ne $Null) {
#($Objects) | ForEach -Begin {$Output = #(); $Names = #()} {
New-Variable -Force -Option AllScope -Name Property -Value (New-Object System.Collections.Specialized.OrderedDictionary)
Flatten-Object #(,$_) $Separator $Base $Depth $Uncut $ToString $Path
$Output += New-Object PSObject -Property $Property
$Names += $Output[-1].PSObject.Properties | Select -Expand Name
}
$Output | Select ([String[]]($Names | Select -Unique))
}
}; Set-Alias Flatten Flatten-Object
Syntax
<Object[]> Flatten-Object [-Separator <String>] [-Base "" | 0 | 1] [-Depth <Int>] [-Uncut<Int>] [ToString <Type[]>]
or:
Flatten-Object <Object[]> [[-Separator] <String>] [[-Base] "" | 0 | 1] [[-Depth] <Int>] [[-Uncut] <Int>] [[ToString] <Type[]>]
Parameters
-Object[] <Object[]>
The object (or objects) to be flatten.
-Separator <String> (Default: .)
The separator used between the recursive property names. .
-Depth <Int> (Default: 5)
The maximal depth of flattening a recursive property. Any negative value will result in an unlimited depth and could cause a infinitive loop.
-Uncut <Int> (Default: 1)
The number of object iterations that will left uncut further object properties will be limited to just the DefaultDisplayPropertySet. Any negative value will reveal all properties of all objects.
-Base "" | 0 | 1 (Default: 1)
The first index name of an embedded array:
1, arrays will be 1 based: <Parent>.1, <Parent>.2, <Parent>.3, ...
0, arrays will be 0 based: <Parent>.0, <Parent>.1, <Parent>.2, ...
"", the first item in an array will be unnamed and than followed with 1: <Parent>, <Parent>.1, <Parent>.2, ...
-ToString <Type[]= [String], [DateTime], [TimeSpan]>
A list of value types (default [String], [DateTime], [TimeSpan]) that will be converted to string rather the further flattened. E.g. a [DateTime] could be flattened with additional properties like Date, Day, DayOfWeek etc. but will be converted to a single (String) property instead.
Note:
The parameter -Path is for internal use but could but used to prefix property names.
Examples
Answering the specific question:
(Get-Content "PATH_TO\test.json" -Raw | ConvertFrom-Json) | Flatten-Object | Convertto-CSV -NoTypeInformation | Set-Content "PATH_TO\test.csv"
Result:
{
"url": "http://test.test",
"slug": "slug",
"id": 10011,
"link": "http://test.er",
"level": 1,
"areas.2": "area_b",
"areas.1": "area_a",
"disciplines.3": "discipline_c",
"disciplines.2": "discipline_b",
"disciplines.1": "discipline_a",
"subject": "testing",
"title": "Test procedure",
"email": "test#test.com"
}
Stress testing a more complex custom object:
New-Object PSObject #{
String = [String]"Text"
Char = [Char]65
Byte = [Byte]66
Int = [Int]67
Long = [Long]68
Null = $Null
Booleans = $False, $True
Decimal = [Decimal]69
Single = [Single]70
Double = [Double]71
Array = #("One", "Two", #("Three", "Four"), "Five")
HashTable = #{city="New York"; currency="Dollar"; postalCode=10021; Etc = #("Three", "Four", "Five")}
Object = New-Object PSObject -Property #{Name = "One"; Value = 1; Text = #("First", "1st")}
} | Flatten
Result:
Double : 71
Decimal : 69
Long : 68
Array.1 : One
Array.2 : Two
Array.3.1 : Three
Array.3.2 : Four
Array.4 : Five
Object.Name : One
Object.Value : 1
Object.Text.1 : First
Object.Text.2 : 1st
Int : 67
Byte : 66
HashTable.postalCode : 10021
HashTable.currency : Dollar
HashTable.Etc.1 : Three
HashTable.Etc.2 : Four
HashTable.Etc.3 : Five
HashTable.city : New York
Booleans.1 : False
Booleans.2 : True
String : Text
Char : A
Single : 70
Null :
Flatting grouped objects:
$csv | Group Name | Flatten | Format-Table # https://stackoverflow.com/a/47409634/1701026
Flatting common objects:
(Get-Process)[0] | Flatten-Object
Or a list (array) of objects:
Get-Service | Flatten-Object -Depth 3 | Export-CSV Service.csv
Note that a command as below takes hours to compute:
Get-Process | Flatten-Object | Export-CSV Process.csv
Why? because it results in a table with a few hundred rows and several thousand columns. So if you if would like to use this for flatting process, you beter limit the number of rows (using the Where-Object cmdlet) or the number of columns (using the Select-Object cmdlet).
For the latest Flatten-Object version, see: https://powersnippets.com/flatten-object/
The CSV conversion/export cmdlets have no way of "flattening" an object, and I may be missing something, but I know of no way to do this with a built-in cmdlet or feature.
If you can guarantee that disciplines and areas will always have the same number of elements, you can trivialize it by using Select-Object with derived properties to do this:
$properties=#('id','title','slug','url','email','link','subject','level',
#{Name='discipline_1';Expression={$_.disciplines[0]}}
#{Name='discipline_2';Expression={$_.disciplines[1]}}
#{Name='discipline_3';Expression={$_.disciplines[2]}}
#{Name='area_1';Expression={$_.areas[0]}}
#{Name='area_2';Expression={$_.areas[1]}}
)
(Get-Content 'PATH_TO\test.json' -Raw | ConvertFrom-Json)| Select-Object -Property $properties | Export-CSV -NoTypeInformation -Path 'PATH_TO\test.csv'
However, I am assuming that disciplines and areas will be variable length for each record. In that case, you will have to loop over the input and pull the highest count value for both disciplines and areas, then build the properties array dynamically:
$inputData = Get-Content 'PATH_TO\test.json' -Raw | ConvertFrom-Json
$counts = $inputData | Select-Object -Property #{Name='disciplineCount';Expression={$_.disciplines.Count}},#{Name='areaCount';Expression={$_.areas.count}}
$maxDisciplines = $counts | Measure-Object -Maximum -Property disciplineCount | Select-Object -ExpandProperty Maximum
$maxAreas = $counts | Measure-Object -Maximum -Property areaCount | Select-Object -ExpandProperty Maximum
$properties=#('id','title','slug','url','email','link','subject','level')
1..$maxDisciplines | % {
$properties += #{Name="discipline_$_";Expression=[scriptblock]::create("`$_.disciplines[$($_ - 1)]")}
}
1..$maxAreas | % {
$properties += #{Name="area_$_";Expression=[scriptblock]::create("`$_.areas[$($_ - 1)]")}
}
$inputData | Select-Object -Property $properties | Export-CSV -NoTypeInformation -Path 'PATH_TO\test.csv'
This code hasn't been fully tested, so it may need some tweaking to work 100%, but I believe the ideas are solid =)
I have data in this shape:
externalName,day,workingHours,hoursAndMinutes
PRJF,1,11,11:00
PRJF,2,11,11:00
PRJF,3,0,0:00
PRJF,4,0,0:00
CFAW,1,11,11:00
CFAW,2,11,11:00
CFAW,3,11,11:00
CFAW,4,11,11:00
CFAW,5,0,0:00
CFAW,6,0,0:00
and so far code is
$gdata = Import-csv $filepath\$filename | Group-Object -Property Externalname;
$test = #()
$test += foreach($rostername in $gdata) {
$rostername.Group | Select -Unique externalName,
#{Name = 'AllDays';Expression = {(($rostername.Group) | measure -Property day).count}},
}
$test;
What I can't work out is how to do a conditional count of the lines where day is non-zero.
The aim is to produce two lines:
PRJF, 4, 2, 11
CFAW, 6, 4, 11
i.e. Roster name, roster length, days on, average hours worked per day on.
You need a where-object to filter for non zero workinghours
I'd use a [PSCustomObject] to generate a new table
EDIT a bit more efficient with only one Measure-Object
## Q:\Test\2018\08\06\SO_51700660.ps1
$filepath = 'Q:\Test\2018\08\06'
$filename = 'SO_S1700660.csv'
$gdata = Import-Csv (Join-Path $filepath $filename) | Group-Object -Property Externalname
$test = ForEach($Roster in $gdata) {
$WH = ($Roster.Group.Workinghours|Where-Object {$_ -ne 0}|Measure-Object -Ave -Sum)
[PSCustomObject]#{
RosterName = $Roster.Name
RosterLength = $Roster.Count
DaysOn = $WH.count
AvgHours = $WH.Average
TotalHours = $WH.Sum
}
}
$test | Format-Table
Sample output:
> .\SO_51700660.ps1
RosterName RosterLength DaysOn AvgHours TotalHours
---------- ------------ ------ -------- ----------
PRJF 4 2 11 22
CFAW 6 4 11 44
I have a hash table as below:
$Hash = #{
Team1=$Team1.count
Team2=$Team2.count
Team3=$Team3.count
}
$GroupByTeam = New-Object psobject -Property $Hash |
Select 'Team1','Team2','Team3' | ConvertTo-Html -Fragment
This is fine and each "team" returns their own value. However, teams may have a null value and I wish to substitute this for "0".
In an attempt to work this out, I have tried to select the null value first but can't seem to do this:
$Hash.values | select -property Values
Values
------
{1, 2}
But
$Hash.values | select -property Values | where {$_.Values is $null}
doesn't pull back anything. Also tried:
$Hash.values | select -expandproperty Values | where {$_.Values is $null}
Any ideas?
thanks
Your best option is to cast the values to int when creating the hashtable:
$Hash = #{
Team1 = [int]$Team1.Count
Team2 = [int]$Team2.Count
Team3 = [int]$Team3.Count
}
If that's not possible for some reason you could go with an enumerator:
($Hash.GetEnumerator()) | ForEach-Object {
if ($_.Value -eq $null) { $Hash[$_.Name] = 0 }
}
or (as Mathias suggested) use the Keys property to the same end:
($Hash.Keys) | ForEach-Object {
if ($Hash[$_] -eq $null) { $Hash[$_] = 0 }
}
Note that either way you need to use a subexpression (or assign the enumerated objects/keys to a variable) otherwise you'll get an error because you're modifying a data structure while it's being enumerated.
What you'll want to do is collect the keys that refer to null values, and then populate those with 0s:
# Create and populate hashtable
$HashTable = #{
Team1 = 123
Team2 = $null
Team3 = 456
}
# Find keys of `$null` values
$nullKeys = $HashTable.Keys |Where-Object { $HashTable[$_] -eq $null }
# Populate appropriate indices with 0
$nullKeys |ForEach-Object { $HashTable[$_] = 0 }
I am working on query user command in PowerShell to filter the content to get the users who wer disconnected for more than 2 days on the server.
This is my result:
USERNAME SESSIONNAME ID STATE IDLE TIME LOGON TIME
a_admin 2 Disc 20+16:56 19.08.2015
b_admin 3 Disc . 10.12.2015
c_admin 4 Disc 5+22:33 24.08.2015
d_admin 5 Disc 17:47 17.12.2015
e_admin 6 Disc 101+18:58 02.09.2015
f_admin 7 Disc 1+01:27 14.12.2015
The problem is the query user don't retrieve the data as an object format, so I can't select any column from these data, can any one help me to find a way to filter this content? Also, I am having a problem in the content of the idle time. It seems weird!?
I tried to put the output in a text file then get the content back and do some filtration, but the result is the same (USERNAME with empty records).
query user produces string output. You can't convert that to objects by piping it into Format-Table. And Select-Object won't do with the output of Format-Table what you seem to expect anyway.
Use a regular expression match to transform the string output into a list of objects:
$server = 'servername'
$re = '(\w+)\s+?(\S*)\s+?(\d+)\s+Disc\s+(\S+)\s+(\d+\.\d+\.\d+)'
query user /server:$server | Where-Object { $_ -match $re } | ForEach-Object {
New-Object -Type PSCustomObject -Property #{
'Username' = $matches[1]
'SessionID' = $matches[3]
'IdleTime' = $matches[4]
'LogonTime' = $matches[5]
}
} | Select-Object Username, IdleTime
This will give you everything as string values, though. Since you want to filter on the idle time you may want to convert the values to appropriate types. Using a more elaborate regular expression (with named groups) will help with that.
$server = 'servername'
$re = '(?<username>\w+)\s+?' +
'(\S*)\s+?' +
'(?<session>\d+)\s+' +
'Disc\s+' +
'(?:(?:(?<days>\d+)\+)?(?<hours>\d+):)?(?<minutes>\d+)\s+' +
'(?<logon>\d+\.\d+\.\d+)'
query user /server:$server | Where-Object { $_ -match $re } | ForEach-Object {
New-Object -Type PSCustomObject -Property #{
'Username' = $matches['username']
'SessionID' = [int]$matches['session']
'IdleTime' = if ($matches['days']) {
New-TimeSpan -Days $matches['days'] -Hours $matches['hours'] -Minutes $matches['minutes']
} elseif ($matches['hours']) {
New-TimeSpan -Hours $matches['hours'] -Minutes $matches['minutes']
} else {
New-TimeSpan -Minutes $matches['minutes']
}
'LogonTime' = [DateTime]::ParseExact($matches['logon'], 'dd\.MM\.yyyy', [Globalization.CultureInfo]::InvariantCulture)
}
} | Where-Object {
$_.IdleTime.TotalDays -gt 2
} | Select-Object Username, IdleTime
I have:
$report.gettype().name
Object[]
echo $report
Item Average
-- -------
orange 0.294117647058824
orange -0.901960784313726
orange -0.901960784313726
grape 9.91335740072202
grape 0
pear 3.48736462093863
pear -0.0324909747292419
pear -0.0324909747292419
apple 12.1261261261261
apple -0.0045045045045045
I want to create a variable, $total, (such as a hash table) which contains the sum of the 'Average' column for each item, for example,
echo $total
orange -1.5097
grape 9.913
pear 3.423
apple 12.116
Right now I'm thinking of looping through the $report, but it's hell ugly, and I am looking for something more elegant than the following starting point (incomplete):
$tmpPrev = ""
foreach($r in $report){
$tmp = $r.item
$subtotal = 0
if($tmp <> $tmpPrev){
$subtotal += $r.average
}
How could I do this?
Cmdlets Group-Object and Measure-Object help to solve the task in a PowerShell-ish way:
Code:
# Demo input
$report = #(
New-Object psobject -Property #{ Item = 'orange'; Average = 1 }
New-Object psobject -Property #{ Item = 'orange'; Average = 2 }
New-Object psobject -Property #{ Item = 'grape'; Average = 3 }
New-Object psobject -Property #{ Item = 'grape'; Average = 4 }
)
# Process: group by 'Item' then sum 'Average' for each group
# and create output objects on the fly
$report | Group-Object Item | %{
New-Object psobject -Property #{
Item = $_.Name
Sum = ($_.Group | Measure-Object Average -Sum).Sum
}
}
Output:
Sum Item
--- ----
3 orange
7 grape
I've got a more command-line solution.
Given $report
$groupreport = $report | Group-Object -Property item -AsHashTable
is
Name Value
---- -----
grape {#{Item=grape; Average=9.91335740072202}, #{Item=grape; Average=0}}
orange {#{Item=orange; Average=0.294117647058824}, #{Item=orange; Average=-0.901960784313726...
apple {#{Item=apple; Average=12.1261261261261}, #{Item=apple; Average=-0.0045045045045045}}
pear {#{Item=pear; Average=3.48736462093863}, #{Item=pear; Average=-0.0324909747292419}, #...
then
$tab=#{}
$groupreport.keys | % {$tab += #{$_ = ($groupreport[$_] | measure-object -Property average -sum)}}
gives
PS> $tab["grape"]
Count : 2
Average :
Sum : 9,91335740072202
Maximum :
Minimum :
Property : Average
PS> $tab["grape"].sum
9,91335740072202
It seems short and usable.
Summary
$groupreport = $report | Group-Object -Property item -AsHashTable
$tab = #{}
$groupreport.keys | % {$tab += #{$_ = ($groupreport[$_] | measure-object -Property average -sum)}}
$tab.keys | % {write-host $_ `t $tab[$_].sum}
I don't know if you can get rid of looping. What about:
$report | % {$averages = #{}} {
if ($averages[$_.item]) {
$averages[$_.item] += $_.average
}
else {
$averages[$_.item] = $_.average
}
} {$averages}