Powershell - Creating Wide Tables - powershell

Good afternoon!
I am a Powershell novice trying to understand how data output works. I am trying to find the differences between two users in Active Directory. I found a solution that worked (Compare-Object on two AD user accounts), but the data some of in the relevant fields was truncated with an ... which didn't help.
I found a solution which seems very elegant at the bottom of the page here: http://poshoholic.com/2010/11/11/powershell-quick-tip-creating-wide-tables-with-powershell/
I attempted to combine these two into a single script. This is what I have:
$user1 = get-aduser jdoe -Properties *
$user2 = get-aduser jsmith -Properties *
$Usercomparison = #()
$user1.GetEnumerator() | ForEach-Object {
If ($User2.($_.Key) -eq $_.Value)
{
$Comparison = 'Equal'
}
else
{
$Comparison = 'Different'
}
$UserObj = New-Object PSObject -Property ([ordered]#{
Property = $_.Key
User1 = $_.Value
User2 = $User2.($_.Key)
Comparison = $Comparison
})
$UserComparison += $UserObj
}
$UserComparison
| Format-Table -Property * -AutoSize `
| Out-String -Width 4096 `
| Out-File C:\Users\USER\Desktop\differences.txt
This produces an error that "an empty pipe element is not allowed". If I delete a line return to put the first pipe line after the $UserComparison variable...
$UserComparison | Format-Table -Property * -AutoSize `
| Out-String -Width 4096 `
| Out-File C:\aliases.txt
...then the text file is created, but it's badly formatted. Only the first two columns appear in the output and there is a ton of wasted whitespace to the right and several blank line returns after each line... nothing like the example on the website.
Is this because the script I found writes the data to a variable and then just prints the variable on screen instead of using a command that can be output properly? I feel like I have all the pieces that I need, just not in the right configuration to get the output I want.
Thanks!

So, #1 the line:
$UserComparison
| Format-Table -Property * -AutoSize `
| Out-String -Width 4096 `
| Out-File C:\Users\USER\Desktop\differences.txt
Doesn't work because you are first executing
$UserComparison
Which outputs the contents of $UserComparison. Next, you execute
| Format-Table -Property * -AutoSize `
Which errors out because nothing is being piped into Format-Table. The "ticks" ( ` ) at the end of the Format-Table statement is a continue line statement i.e. the second version:
$UserComparison | Format-Table -Property * -AutoSize `
| Out-String -Width 4096 `
| Out-File C:\aliases.txt
Is correct because it will be interpreted as one giant line.
Second question, the reason why you are having issues is because 4096 characters is not enough space to hold everything, and so is truncated. Remember, -AutoSize will calculate the width of the longest item, and make that the width of the column. There are some items that are too long. For ex. For me, the thumbnailPhoto (which happened to be item 140 in my array):
$UserComparison[140]
Gives something like this (truncated depending on width):
Property User1
-------- -----
thumbnailPhoto {255 216 255 224 0 16 74 70 73 70 0 1 1 1 0 96 0...
When I calculate the width of this, it gives me:
#Calculate width of User1
($UserComparison[140].User1 | Out-String).Length
7555
#Calculate width of full field
($UserComparison[140] | Out-String).Length
12297
Yes, User1 is 7,555 characters long. That means that Format-Table -Autosize will make the User1 column at least 7,555 characters wide, which obviously is truncated by the 4,096 width limit that you specified on the Out-String, and then won't display the User2 or Comparison columns. In this case, your Out-String needs to have a width of at least 12,297 wide in order to display the full field.
The workaround is to specify an even bigger width on the Out-String that is guaranteed to be wider, like, say, 50,000 so your code would be:
$UserComparison | Format-Table -Property * -AutoSize `
| Out-String -Width 50000 `
| Out-File C:\Users\USER\Desktop\differences.txt
Now, the downside to doing things this way is that every line in the text file will be the full width of the longest item, and so (in my case) every line will be 12,297 characters long. This makes things harder to read.
Other ways to output things would be to:
Limit things to just displaying the Property and Comparison columns:
$UserComparison | Select Property, Comparison `
| Format-Table -Property * -AutoSize `
| Out-String -Width 4096 `
| Out-File SimpleCompare.txt
Or if you need to see what the full values are, chop each property up into a separate table with a ForEach-Object, and then pass that through so that would be easier to read, and each property is limited to it's specific width:
$UserComparison | Select Property, Comparison, User1, User2 `
| ForEach-Object { $_ | Format-Table -Property * -AutoSize `
| Out-String -Width 50000 `
| Out-File EasyToRead.txt -Append }

Related

PowerShell Export-CSV - Missing Columns [duplicate]

This question already has an answer here:
Not all properties displayed
(1 answer)
Closed 1 year ago.
This is a follow-up question from PowerShell | EVTX | Compare Message with Array (Like)
I changed the tactic slightly, now I am collecting all the services installed,
$7045 = Get-WinEvent -FilterHashtable #{ Path="1system.evtx"; Id = 7045 } | select
#{N=’Timestamp’; E={$_.TimeCreated.ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')}},
Id,
#{N=’Machine Name’; E={$_.MachineName}},
#{N=’Service Name’; E={$_.Properties[0].Value}},#{N=’Image Path’;E=$_.Properties[1].Value}},
#{N=’RunAsUser’; E={$_.Properties[4].Value}},#{N=’Installed By’; E={$_.UserId}}
Now I match each object for any suspicious traits and if found, I add a column 'Suspicious' with the value 'Yes'. This is because I want to leave the decision upto the analyst and pretty sure the bad guys might use something we've not seen before.
foreach ($Evt in $7045)
{
if ($Evt.'Image Path' -match $sus)
{
$Evt | Add-Member -MemberType NoteProperty -Name 'Suspicious' -Value 'Yes'
}
}
Now, I'm unable to get PowerShell to display all columns unless I specifically Select them
$7045 | Format-Table
Same goes for CSV Export. The first two don't include the Suspicious Column but the third one does but that's because I'm explicitly asking it to.
$7045 | select * | Export-Csv -Path test.csv -NoTypeInformation
$7045 | Export-Csv -Path test.csv -NoTypeInformation
$7045 | Select-Object Timestamp, Id, 'Machine Name', 'Service Name', 'Image Path', 'RunAsUser', 'Installed By', Suspicious | Export-Csv -Path test.csv -NoTypeInformation
I read the Export-CSV documentation on MS. Searched StackOverFlow for some tips, I think it has something to do with PS checking the first Row and then compares if the property exists for the second row and so on.
Thank you
The issue you're experiencing is partially because of how objects are displayed to the console, the first object's Properties determines the displayed Properties (Columns) to the console.
The bigger problem though, is that Export-Csv will not export those properties that do not match with first object's properties unless they're explicitly added to the remaining objects or the objects are reconstructed, for this one easy way is to use Select-Object as you have pointed out in the question.
Given the following example:
$test = #(
[pscustomobject]#{
A = 'ValA'
}
[pscustomobject]#{
A = 'ValA'
B = 'ValB'
}
[pscustomobject]#{
C = 'ValC'
D = 'ValD'
E = 'ValE'
}
)
Format-Table will not display the properties B to E:
$test | Format-Table
A
-
ValA
ValA
Format-List can display the objects properly, this is because each property with it's corresponding value has it's own console line in the display:
PS /> $test | Format-List
A : ValA
A : ValA
B : ValB
C : ValC
D : ValD
E : ValE
Export-Csv and ConvertTo-Csv will also miss properties B to E:
$test | ConvertTo-Csv
"A"
"ValA"
"ValA"
You have different options as a workaround for this, you could either add the Suspicious property to all objects and for those events that are not suspicious you could add $null as Value.
Another workaround is to use Select-Object explicitly calling the Suspicious property (this works because you know the property is there and you know it's Name).
If you did not know how many properties your objects had, a dynamic way to solve this would be to discover their properties using the PSObject intrinsic member.
using namespace System.Collections.Generic
function ConvertTo-NormalizedObject {
[CmdletBinding()]
param(
[Parameter(ValueFromPipeline, Mandatory)]
[object[]] $InputObject
)
begin {
$list = [List[object]]::new()
$props = [HashSet[string]]::new([StringComparer]::InvariantCultureIgnoreCase)
}
process {
foreach($object in $InputObject) {
$list.Add($object)
foreach($property in $object.PSObject.Properties) {
$null = $props.Add($property.Name)
}
}
}
end {
$list | Select-Object ([object[]] $props)
}
}
Usage:
# From Pipeline
$test | ConvertTo-NormalizedObject | Format-Table
# From Positional / Named parameter binding
ConvertTo-NormalizedObject $test | Format-Table
Lastly, a pretty easy way of doing it thanks to Select-Object -Unique:
$prop = $test.ForEach{ $_.PSObject.Properties.Name } | Select-Object -Unique
$test | Select-Object $prop
Using $test for this example, the result would become:
A B C D E
- - - - -
ValA
ValA ValB
ValC ValD ValE
Continuing from my previous answer, you can add a column Suspicious straight away if you take out the Where-Object filter and simply add another calculated property to the Select-Object cmdlet:
# create a regex for the suspicious executables:
$sus = '(powershell|cmd|psexesvc)\.exe'
# alternatively you can join the array items like this:
# $sus = ('powershell.exe','cmd.exe','psexesvc.exe' | ForEach-Object {[regex]::Escape($_)}) -join '|'
$7045 = Get-WinEvent -FilterHashtable #{ LogName = 'System';Id = 7045 } |
Select-Object Id,
#{N='Timestamp';E={$_.TimeCreated.ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')}},
#{N='Machine Name';E={$_.MachineName}},
#{N='Service Name'; E={$_.Properties[0].Value}},
#{N='Image Path'; E={$_.Properties[1].Value}},
#{N='RunAsUser'; E={$_.Properties[4].Value}},
#{N='Installed By'; E={$_.UserId}},
#{N='Suspicious'; E={
if ($_.Properties[1].Value -match $sus) { 'Yes' } else {'No'}
}}
$7045 | Export-Csv -Path 'X:\Services.csv' -UseCulture -NoTypeInformation
Because you have many columns, this will not fit the console width anymore if you do $7045 | Format-Table, but the CSV file will hold all columns you wanted.
I added switch -UseCulture to the Export-Csv cmdlet, which makes sure you can simply double-click the csv file so it opens correctly in your Excel.
As sidenote: Please do not use those curly so-called 'smart-quotes' in code as they may lead to unforeseen errors. Straighten these ’ thingies and use normal double or single quotes (" and ')

Running Powershell command Out-File in Azure Pipeline cuts lines

As part of a CI pipeline, I need to generate a list of all the files in the repository, including certain properties of them, and to Out-File it into a file. The command I use:
Get-ChildItem $Path -File -Recurse | Select-Object -Property LastWriteTime, #
{
label = "Size(KB)"
expr = { [string]::Format("{0:0.00}", $_.Length/1KB) }
}, FullName, <some_other_property> | Out-File $OutputFile
My problem is, that running this script from the command line gives the desired result.
However, running this during a Azure Pipeline build does 2 bad things:
cuts the lines in the FullName column when they are long:
LastWriteTime Size(KB) Name
------------ ------- ----
<some_date> <some size> ASomeWhatLong...
Doesn't display the rest of the properties such as <some_other_property>
If I turn FullName into Name it all goes OK, but I really need the FullName property.
Because I'm working in a Air-gapped environment I can't copy all the outputs and everything.
I've tried using the -Width flag for Out-File with no success.
I believe what's happening under the hood, ps uses the ToString() method of the object created which outputs it as the Format-Table cmdlet does. You get truncated properties because of the Window's size. To look at it you could use:
(Get-Host).ui.RawUI.WindowSize
Probably this is too small.
What I would suggest is the following:
Pipe the object into Format-Table
Get-ChildItem $Path -File -Recurse | Select-Object -Property LastWriteTime, #
{
label = "Size(KB)"
expr = { [string]::Format("{0:0.00}", $_.Length/1KB) }
}, FullName, <some_other_property> | Format-Table | Out-String | Out-File $OutputFile
This probably won't work as it is, but you could play with the Format-Table's properties like: -Wrap. By default it will allocate enough space for the first properties, and the last one it would try to 'fit' it, which might look as:
LastWriteTime Size(KB) Name
------------ ------- ----
<some_date> <some size> ASomeWhatLong
foobarfoobarfo
foobarfoobarfo
To solve this, you can use the -Property argument, which needs to be as:
$propertWidth = [int]((Get-Host).ui.RawUI.WindowSize.Width / 3)
$property = #(
#{ Expression = 'LastWriteTime'; Width = $propertWidth; },
#{ Expression = 'Size(KB)'; Width = $propertWidth; },
#{ Expression = 'FullName'; Width = $propertWidth; }
)
... | Format-Table -Property $property -Wrap | ...
If you don't mind having a JSON into your file, you could use:
Get-ChildItem $Path -File -Recurse | Select-Object -Property LastWriteTime, #
{
label = "Size(KB)"
expr = { [string]::Format("{0:0.00}", $_.Length/1KB) }
}, FullName, <some_other_property> | ConvertTo-Json | Out-File $OutputFile
But take into account that ConvertTo-Json has a default Depth of 2. This will also truncate your objects if you have nested objects. But as far as property lengths, it will do fine.

Compare objects and export the difference

I would like to make a script that compares O365 tenant settings. Reading them is fine, now I would like to make some kind of difference object.
A related question is here, but no answer.
Powershell Compare-Object and getting the Differences into a File
I already have a json file from both tenants created like:
$srcTenant | Select-Object -Property * | ConvertTo-Json | Out-File "$targetfolder\$targetfile"
Now, I would like a file that only contains the properties that are collected with the script below:
I am so far:
$properties = ($srcTenant | Get-Member -MemberType Property | Select-Object -ExpandProperty Name)
$selectedproperties = #{}
$i = 0
foreach ($property in $properties) {
if (Compare-Object $srcTenant $trgTenant -Property "$property") {
$selectedproperties.Add($i, "$property")
$i++
}
}
The $selectedproperties variable contains 9 properties and I would like to export only this 9 in the same format as the other two.
Name Value
---- -----
8 StorageQuotaAllocated
7 StorageQuota
6 ResourceQuotaAllocated
5 ResourceQuota
4 OwnerAnonymousNotification
3 OneDriveStorageQuota
2 DefaultLinkPermission
1 ConditionalAccessPolicy
0 AllowDownloadingNonWebViewableFiles
So, I am looking for something like:
$srcTenant | Select-Object -Property (that 9 property above) | ConvertTo-Json | Out-File "$targetfolder\$targetfile
Other options achieving the same result are welcome too :)
Select-Object -Property can take an array of property names.
see the first example here

Powershell to reformat the file content

I have a script which gets content from one file and checks for its ip. Then that is added to some other text file.
[System.Collections.ArrayList]$hlist1 = Get-Content -Path "C:\Timezone\Update\host.txt"
$hiplist = New-Object System.Collections.ArrayList
$hlist2 = New-Object System.Collections.ArrayList
ForEach ($h in $hlist1)
{
$hip = Resolve-DnsName $h
$hiplist.Add($hip)
}
$hiplist | Out-File "C:\Timezone\Update\hiplist.txt"
The file which is getting created is as shown below:
---- ---- --- ------- --------
WIN-JB2A2FS84MQ.domain.com A 1200 Answer 10.3.0.4
8
WIN-QP0BH4SD2H9.domain.com A 1200 Answer 10.3.1.1
9
I need to:
get rid of the first -------- lines.
get the entire ip in the same line (10.3.0.10)
Have tried Format-Table -Autosize, then Select -Skip 1 etc, but no luck.
How can this be achieved.? Please note that the code works fine as expected when it is ran manually, but throws this issue when executed using task scheduler.
Edit Based on Matt's answer
Now the text file contains:
"Address","IPAddress","QueryType","IP4Address","Name","Type","CharacterSet","Section","DataLength","TTL"
"10.3.0.48","10.3.0.48","A","10.3.0.48","WIN-JB2A2FS84MQ.domain.com","A","Unicode","Answer","4","1200"
"10.3.1.19","10.3.1.19","A","10.3.1.19","WIN-QP0BH4SD2H9.domain.com","A","Unicode","Answer","4","1200"
Peter-sal's reply output:
Name Type TTL Section IPAddress
---- ---- --- ------- ---------
WIN-JB2A2FS84MQ.domain.com A 1200 Answer 10.3.0.48
WIN-QP0BH4SD2H9.domain.com A 1200 Answer 10.3.1.19
But again on top of Name there's one space. I need to delete everything present before WIN-JB2.....
I cannot test perfectly but I would like to come back to an earlier comment of mine. Resolve-DNSName returns objects so their output is better destined for something object aware. Export-CSV should be preferable here.
$hlist1 = Get-Content -Path "C:\Timezone\Update\host.txt"
$hlist1 | ForEach-Object{Resolve-DnsName $_} |
Export-Csv "C:\Timezone\Update\hiplist.txt" -NoTypeInformation
I normally don't like this but if you prefer you should be able to use the Format-table output now. This seems to be more inline with what you are looking for.
$hlist1 = Get-Content -Path "C:\Timezone\Update\host.txt"
$hlist1 | ForEach-Object{Resolve-DnsName $_} |
Format-Table -HideTableHeaders | Select-Object -Skip 1 |
Out-File "C:\Timezone\Update\hiplist.txt" -Width 200
Perhaps you prefer that output. The header should be removed now as well as a blank line in the beginning.
That creates some white-space before and after the output. Simple solution is to wrap that up in a Trim()
$hlist1 = Get-Content -Path "C:\Timezone\Update\host.txt"
$results = ($hlist1 | ForEach-Object{Resolve-DnsName $_} |
Format-Table -HideTableHeaders |
Out-string).Trim()`
$results | Out-File "C:\Timezone\Update\hiplist.txt" -Width 200`

Powershell: align right for a column value from Select-Object in Format-Table format

I have the following array value $outData with several columns. I am not sure how I align some columns right?
$outData | Select-Object `
Name `
#{Name="Freespace(byte)"; Expression={"{0:N0}" -f $_.FreeSpace}}, '
.... # other colums `
| Format-Table -AutoSize
It works fine. However, when I tried to use align for the freespace column to right:
#{Name="Freespace(byte)"; Expression={"{0:N0}" -f $_.FreeSpace}; align="right"}, '
I got error message "Specified method is not supported". Not sure if there is any way to align the value to right?
The align directive goes in a hashtable that is specified to the Format-Table cmdlet. IOW, align is not a supported hashtable entry for Select-Object. So make sure to do your formatting via hashtables in the hashtable passed to Format-Table e.g.:
gps | select name,pm | format-table #{n='Name';e={$_.Name};align='right'},PM
or in your case:
$outData | Format-Table Name,
#{n="Freespace(byte)";e={"{0:N0}" -f $_.FreeSpace};a="right"}
Given the updates to Powershell in the last 8 years, this answer may not have existed in '10.
The trick is to assign a number of columns within the calculated expression's format block {0:N0}, once assigned, it will align the column to the right.
In the original example, include ,15 as part of the number formatting:
#{Name="Freespace(byte)"; Expression={"{0,15:N0}" -f $_.FreeSpace}}
I generally use the character count of the Name= value to ensure the entire name is visible.
here is an ugly looking one liner that builds from several other threads:
Get-WmiObject win32_LogicalDisk | where { $_.DriveType -eq 3 } | Format-Table DeviceID,VolumeName,#{N="Size";E={'{0:N0}' -f $_.Size};a="right"},#{N="FreeSpace";E={'{0:N0}' -f $_.FreeSpace};a="right"},#{N="Used";E={'{0:N0}' -f ($_.Size - $_.FreeSpace)};a="right"}