How do you sort a formatted table that uses a hash-table to create it?

How do you sort a formatted table that uses a hash-table to create it? - powershell

In a nutshell I'm trying to get what would be the output of this command to run.
Get-Process | Format-Table | sort-object Handles
Yes, I know you usually sort BEFORE Format-Table, but in this case I've created a hash-table within the Format-Table command that needs to be sorted. The problem is, there is no way to sort with Format-Table that I can figure out.
I also considered trying to output the FT to CSV and then massaging it but that didn't work for me either.
I'm expecting to get a sorted table.

I've created a hash-table within the Format-Table command that needs to be sorted
By definition, you can not output a hashtable via any of the -Format-* cmdlets:
Format-* cmdlets emit output objects whose sole purpose is to provide formatting instructions to PowerShell's for-display output-formatting system. In short: only ever use Format-* cmdlets to format data for display, never for subsequent programmatic processing - see this answer for more information.
Assuming you've used calculated properties, with Format-Table, use them with Select-Object instead, which produces data output, namely in the form of [pscustomobject] instances whose properties you can sort by, via Sort-Object.
For instance, the following creates custom objects with .Name and .MemUse properties and sorts by the latter, then outputs the top 10 results:
Get-Process |
Select-Object Name, #{ Name = 'MemUse'; Expression = 'WorkingSet64' } |
Sort-Object -Descending MemUse |
Select-Object -First 10
Inverting the logic and letting Sort-Object operate on the original objects output by Get-Process is more efficient:
Get-Process |
Sort-Object -Descending WorkingSet64 |
Select-Object -First 10 |
Select-Object Name, #{ Name = 'MemUse'; Expression = 'WorkingSet64' }
If you only care about for-display output, you can replace Select-Object Name, ... with Format-Table -Name, ... or Format-List -Name, ..., which illustrates an important point: Format-* calls should generally only come last in a pipeline.

Related

Create Csv with loop and output

This basically works
foreach ($cprev in $CopyPreventeds) {
Write-Host ("prevented copy $(($cprev)."Name")")
$cprev | Select-Object Path, Name, Length, LastWrite, DestinationNewer | Export-Csv '.\prevented.csv' -NoTypeInformation
}
But only the last output is written to the csv. How could I write all contents to a new csv with an output at the same time for the user in PowerShell.

Maybe I'm missing something?
While I appreciate a solution has already been proposed in the comments, I have to ask, given the narrow scope of the question why are we using an obscure, albeit clever technique? And/or, repeatedly invoking Export-Csv...
The question doesn't mention sparing a variable. Moreover, There doesn't appear to be a need for the ForEach loop.
$CopyPreventeds |
Select-Object Path, Name, Length, LastWrite, DestinationNewer |
Export-Csv '.\prevented.csv' -NoTypeInformation
In the above $CopyPreventeds already exists and remains so, unmolested after the export. You would need only to output it again for the benefit of an interactive user. All taking advantage of PowerShell's intuitive pipeline and features.
Moreover, since the iteration variable $cprev isn't needed you are still less one variable.
Note: You don't need -Append because you are streaming into a single Export-Csv command, as opposed to repeatedly invoking it.
There are at least 2 ways (probably many more) you could conveniently output to an interactive user.
1: Echo a header, something like "The following copies were prevented:" then echo the variable $CopyPreventeds, presumably to a table.
Note: That given multiple points at which you seem only interested in a subset of properties. You may think about trimming those objects beforehand:
$CopyPreventeds =
$CopyPreventeds |
Select-Object Path, Name, Length, LastWrite, DestinationNewer
$CopyPreventeds | Export-Csv '.\prevented.csv' -NoTypeInformation
Write-Host "The following copies were prevented:"
$CopyPreventeds | Format-Table -AutoSize | Out-Host
Note: More than 4 Properties in a [PSCustomObject] (resulting from Select-Object) where custom formatting hasn't been defined will by default output as a list, so use Format-Table to overcome that. Out-Host is then used to prevent pipeline pollution.
2: Return to using a ForEach-Object Loop for the output between the Select-Object and the Export-Csv command.
$CopyPreventeds |
Select-Object Path, Name, Length, LastWrite, DestinationNewer
ForEach-Object{
"Prevented Copy : {0}, {1}, {2}, {3}, {4}" -f $_.Path, $_.Name, $_.Length, $_.LastWrite, $_.DestinationNewer |
Write-Host
$_
} |
Export-Csv '.\prevented.csv' -NoTypeInformation
In this example, when you are done outputting to the screen (admittedly a little messy), you emit $_ from the loop, thus piping it to Export-Csv just the same.
Note: there are a number of ways to construct strings, I choose to use the -f operator here because it's a little cleaning than imbedding numerous $() sub expressions. And, of course this assume you want to prefix on every line Which I personally think is gratuitous, so I'd choose something more like #1..

Powershell Performance tuning for aggregation operation on big delimited files

I have a delimited file with 350 columns. The delimiter is \034(Field separator).
I have to extract a particular column value and find out the count of each distinct value of that column in the file. If the count of distinct value is greater or equal to 2, I need to output it to a file.
The source file is 1GB. I have written the following command. It is very slow.
Get-Content E:\Test\test.txt | Foreach {($_ -split '\034')[117]} | Group-Object -Property { $_ } | %{ if($_.Count -ge 2) { Select-Object -InputObject $_ -Property Name,Count} } | Export-csv -Path "E:\Test\test2.csv" -NoTypeInformation
Please help!

I suggest using a switch statement to process the input file quickly (by PowerShell standards):
# Get an array of all the column values of interest.
$allColValues = switch -File E:\Test\test.txt {
default { # each input line
# For better performance with *literal* separators,
# use the .Split() *method*.
# Generally, however, use of the *regex*-based -split *operator* is preferable.
$_.Split([char] 0x1c)[117] # hex 0x1c is octal 034
}
}
# Group the column values, and only output those that occur at least
# twice.
$allColValues | Group-Object -NoElement | Where-Object Count -ge 2 |
Select-Object Name, Count | Export-Csv E:\Test\test2.csv -NoTypeInformation
Tip of the hat to Mathias R. Jessen for suggesting the -NoElement switch, which streamlines the Group-Object call by only maintaining abstract group information; that is, only the grouping criteria (as reflected in .Name, not also the individual objects that make up the group (as normally reflected in .Group) are returned via the output objects.
As for what you tried:
Get-Content with line-by-line streaming in the pipeline is slow, both generally (the object-by-object passing introduces overhead) and, specifically, because Get-Content decorates each line it outputs with ETS (Extended Type System) metadata.
GitHub issue #7537 proposes adding a way to opt-out of this decoration.
At the expense of memory consumption and potentially additional work for line-splitting, the -Raw switch reads the entire file as a single, multi-line string, which is much faster.
Passing -Property { $_ } to Group-Object isn't necessary - just omit it. Without a -Property argument, the input objects are grouped as a whole.
Chaining Where-Object and Select-Object - rather than filtering via an if statement in a ForEach-Object call combined with multiple Select-Object calls - is not only conceptually clearer, but performs better.

Powershell, Using the results of a Invoke-SQLcmd as a variable in a file path [duplicate]

I have a script that creates several jobs and stores two simple values in the jobs.
Start-Job -ScriptBlock {param ([string]$compip) tnc $compip | select RemoteAddress,PingSucceeded -WarningAction SilentlyContinue} -ArgumentList $compip
This works fine. What I would like to know is how can I store the following code into a variable?
Get-Job | Receive-Job | sort RemoteAddress | FT
I have tried this, but it does not work as I thought it would:
$pcs = Get-Job | Receive-Job | sort RemoteAddress | FT
$pcs.RemoteAddress
Am I going at this the wrong way? I would like to store the data from the get-job command above to use the values later in the script. I assumed it would work because the output looks correct:
Command:
Get-Job | Receive-Job | sort RemoteAddress | FT
Output:
RemoteAddress PingSucceeded
------------- -------------
192.168.0.163 True
192.168.0.101 False
192.168.0.2 False
192.168.0.251 True

Problem with Format-cmdlets
The issue here is your use of FT which is an alias for Format-Table. These Format- cmdlets are designed for console/screen output only. There are many things you can do with them to tailor that output but in every case PowerShell needs to massage the data in order to be able to do so. This includes breaking down to the passed objects into groups of different objects...
Microsoft.PowerShell.Commands.Internal.Format.FormatEndData
Microsoft.PowerShell.Commands.Internal.Format.FormatEntryData
Microsoft.PowerShell.Commands.Internal.Format.FormatStartData
Microsoft.PowerShell.Commands.Internal.Format.GroupEndData
Microsoft.PowerShell.Commands.Internal.Format.GroupStartData
The above data types were extract from running this code.
Get-ChildItem c:\temp | Format-Table | Get-Member
So you no longer have the System.IO.FileInfo and System.IO.DirectoryInfo objects that you would normally get from Get-ChildItem
Another large issue comes from Format-cmdlets nature to truncated data, like arrays with a large numbers of elements or long strings, to make as much fit on screen. In the case of arrays, this is due to the preference variable $FormatEnumerationLimit which is commonly defaulted to 4.
Ten Numbers
-----------
{1, 2, 3, 4...}
These, and other, limitations can all be mitigated with cmdlet switches like -AutoSize and -HideTableHeaders, out-string -width, etc. That does not matter however because...
Solution
Good news is the solution is very simple. Stop using them for anything other than console output. Using my earlier example:
Saving results in a variable?: $result = Get-ChildItem C:\temp
Exporting Data: Get-ChildItem C:\temp | Export-CSV $path -NoTypeInformation. Other Export-cmdlets could be preferred here like Export-CLIXml for complex objects when you want to store them for use elsewhere. If you are just looking for something pretty to include in your output then consider ConvertTo-HTML instead.
Extracting individual properties?: Just use Select-Object. $result | Select prop1, prop2. You can also expand your property selection to just get the strings or string array with -ExpandProperty: $result | Select -ExpandProperty prop1
Performing inline calculations with said properties?: Use calculated expression just as you would with the Format-Cmdlets. $result | Select prop1, #{Name="prop2";Expression={$_.prop2 * 3}
Potential Acceptable Use
Some prefer the output for use in emails and for recording statistics. While it is integral to keep data in its more easily used format for later use. However if you really need that data keep in mind that you are not working with the object your originally had anymore.
So if you needed your data in a table format but stored as a string then consider Out-String
$body = Get-ChildItem c:\temp | Format-Table | Out-String
but remember that Format-Table will play with object output in order to get it to display on screen (truncated array properties and long strings). Really.. if you wanted it nice and formatted then you should just use ConvertTo-HTML.
Point is you almost never need to keep the data from Format-Table. There is almost always a better way.

How can I store output from Format-Table for later use

I have a script that creates several jobs and stores two simple values in the jobs.
Start-Job -ScriptBlock {param ([string]$compip) tnc $compip | select RemoteAddress,PingSucceeded -WarningAction SilentlyContinue} -ArgumentList $compip
This works fine. What I would like to know is how can I store the following code into a variable?
Get-Job | Receive-Job | sort RemoteAddress | FT
I have tried this, but it does not work as I thought it would:
$pcs = Get-Job | Receive-Job | sort RemoteAddress | FT
$pcs.RemoteAddress
Am I going at this the wrong way? I would like to store the data from the get-job command above to use the values later in the script. I assumed it would work because the output looks correct:
Command:
Get-Job | Receive-Job | sort RemoteAddress | FT
Output:
RemoteAddress PingSucceeded
------------- -------------
192.168.0.163 True
192.168.0.101 False
192.168.0.2 False
192.168.0.251 True

using powershell and pipeing output od Select-Object to access selected columns

I have the power shell below that selectes certain fields
dir -Path E:\scripts\br\test | Get-FileMetaData | Select-Object name, Comments, Path, Rating
what i want to do is utilize Name,Comments,Path,Rating in further Pipes $_.name etc dosnt work

If I understand your question correctly, you want to do something with the output of Select-Object, but you want to do it in a pipeline.
To do this, you need to pass the output down the pipeline into a Cmdlet that accepts pipeline input (such as ForEach-Object). If the next operation in the pipeline does not accept pipeline input, you will have to set the output to a variable and access the information through the variable,
Using ForEach-Object
In this method, you will be processing each object individually. This will be similar to the first option in Method 1 (that is, dealing with individual items in the collection of items returned by Select-Object).
dir | Get-FileMetaData | Select-Object Name,Comments,Path,Rating | ForEach-Object {
# Do stuff with $_
# Note that $_ is a single item in the collection returned by Select-Object
}
The variable method is included in case your next Cmdlet does not accept pipeline input.
Using Variable
In this method, you will treat $tempVariable as an array and you can operate on each item. If need be, you can actually access each column individually, getting everything at once.
$tempVariable = dir | Get-FileMetaData | Select-Object Name,Comments,Path,Rating
# Do stuff with each Name by using $tempVariable[i].Name, etc.
# Or do stuff with all Names by using $tempVariable.Name, etc.

We Keep Coding

iphone swift flutter scala powershell matlab mongodb postgresql perl eclipse