I'm fairly new to powershell scripting after years of unix/linux scripting and I'm still trying to get the handle of object output. I am trying to run the Get-Alias command to return just the column "Name." When I try to select just the column "name" I either get the whole get-alias output, or errors.
The script/ command line I am trying to use is: get-alias | format-table
When I add select-object logic, it either cuts off the name of the alias and leaves off the command name or the select-object logic is just ignored. I'm sure the solution is simple. Thanks
tl;dr:
PS> Get-Alias | Select-Object -ExpandProperty DisplayName
? -> Where-Object
cd -> Set-Location
...
That is, the true property name underlying the Name display column is DisplayName.
PowerShell's default output formatting sometimes uses column names that differ from or the underlying objects' property names (which is the case here) and/or use values that are calculated.
What shows as column Name by default is actually the underlying objects' .DisplayName property, whereas the true .Name property contains just the alias' own name.
You can inspect the members (properties and methods) of an object's type via Get-Member or, to see both the property (but not method) names and their concrete values for a given object, pipe to Format-List *:
PS> Get-Alias | Select-Object -First 1 | Format-List *
HelpUri : https://go.microsoft.com/fwlink/?LinkID=113423
ResolvedCommandName : Where-Object
DisplayName : ? -> Where-Object
ReferencedCommand : Where-Object
ResolvedCommand : Where-Object
Definition : Where-Object
Options : ReadOnly, AllScope
Description :
OutputType : {}
Name : ?
CommandType : Alias
Source :
Version :
Visibility : Public
ModuleName :
Module :
RemotingCapability : None
Parameters : {[InputObject, System.Management.Automation.ParameterMetadata], [FilterScript, System.Management.Automation.ParameterMetadata], [Property,
System.Management.Automation.ParameterMetadata], [Value, System.Management.Automation.ParameterMetadata]...}
ParameterSets :
Optional reading: How to inspect a given type's formatting data:
For background information, see Get-Help about_Format.ps1xml.
Get-Alias outputs [System.Management.Automation.AliasInfo] instances (as Get-Member will tell you).
Using the Get-FormatData cmdlet you can inspect a type's formatting data:
Caveat: Not all formatting data in effect is by default reported by Get-FormatData for backward-compatibility reasons - see this GitHub issue; notable omissions are [System.IO.FileInfo] and [System.IO.DirectoryInfo], as returned by Get-ChildItem / Get-Item ; in short, to see all formatting data in effect for the PowerShell version at hand, use:
Get-FormatData -PowerShellVersion $PSVersionTable.PSVersion [...]
Somewhat ironically, there is no custom formatting data defined for the System.Management.Automation.ExtendedTypeDefinition instances that Get-Format outputs, and the default formatting isn't helpful.
Currently, the simplest way to inspect formatting data is to pipe to the Format-Custom cmdlet, which provides an informal, human-readable representation of the internal structure of objects:
# Best to send this to a file and open that file in a text editor for perusal.
$prev, $FormatEnumerationLimit = $FormatEnumerationLimit, [int]::MaxValue
Get-FormatData System.Management.Automation.AliasInfo | Format-Custom -Depth 10
$FormatEnumerationLimit = $prev
Note:
$FormatEnumerationLimit is temporarily set to the highest possible value to ensure that all column / property definitions are shown in the output (by default, only up to 4 would be shown, and omitted ones would be indicated with ... (Windows PowerShell, 3 chars.) / … (PowerShell Core, 1 char.).
-Depth 10 ensures that the internal structure of the output objects is fully represented; by default, only up to 5 levels would be.
The output will look vaguely like JSON, and is suited to visual inspection only, not programmatic processing.
You'll see one or more of the following blocks:
class ExtendedTypeDefinition
{
TypeNames =
[
<typeName>
]
FormatViewDefinition =
[
class FormatViewDefinition
{
Name = <formatName>
Control =
class <viewType>Control
...
}
class FormatViewDefinition
{
Name = <formatName>
Control =
class <viewType>Control
...
}
}
<typeName> is the full type name to which the format applies (e.g., System.Management.Automation.AliasInfo)
<formatName> is an internally used, non-standardized name for the format-definition at hand
<viewType>Control is one of the following classes, defining the views for use with the corresponding Format-Table,Format-List, Format-Wide, and Format-Custom cmdlets: TableControl, ListControl, WideControl, CustomControl, all derived from the System.Management.Automation.PSControl base class.
Note that the output may contain duplicate information, as it does in the case of System.Management.Automation.AliasInfo (because the formatting information is repeated for all other command types that share the same formatting), but it is sufficient to examine the first class <viewType>Control instance of interest.
Specifically, here's an excerpt from the first class TableControl block:
class TableControl # Defines the format for Format-Table
{
Headers = # The table headers (column names and column properties)
...
class TableControlColumnHeader # The "Name" column *header*
{
Label = Name
Alignment = Undefined
Width = 50
}
...
]
Rows = # The table rows (underlying or calculated property values)
[
class TableControlRow
{
Columns =
[
...
class TableControlColumn # The "Name" column *value*
{
Alignment = Undefined
DisplayEntry =
class DisplayEntry
{
ValueType = ScriptBlock # A *calculated* property
Value =
if ($_.CommandType -eq "Alias")
{
$_.DisplayName
}
else
{
$_.Name
}
}
FormatString =
}
...
]
...
Note that the header and column-value definitions are matched positionally; e.g., the 2nd TableControlColumn instance defines the value for the 2nd TableControlColumnHeader instance.
The above explains why, even though the column is entitled "Name", it is an alias' .DisplayName property that is displayed, whereas all the other command types are represented by their .Name property, by way of the embedded script block (the piece of PowerShell code in which $_ represents the input object at hand).
Presumably, the decision to show the .DisplayName for aliases was made to provide more helpful information by default, but the discrepancy between the column name and underlying property name can certainly lead to confusion.
Note that there is also a dedicated table view for type [System.Management.Automation.AliasInfo] that (a) defines only 2 columns, and (b) calls the .DisplayProperty column values by their true name.
However, as of the PowerShell versions listed above, this view is preempted by the multi-type definition discussed above and effectively ignored.
Related
function Main {
$result1 = DoWork1
$result1.GetType()
$result2 = DoWork2
$result2.GetType()
}
function DoWork1 {
$result1 = Invoke-Sqlcmd -Query "select top 1 * from customer" -ServerInstance "(localdb)\MSSQLLocalDB" -Database "Database1" -OutputAs DataTables
#assign to variable then return
return $result1
}
function DoWork2 {
#return results without assigning to variable
return Invoke-Sqlcmd -Query "select top 1 * from customer" -ServerInstance "(localdb)\MSSQLLocalDB" -Database "Database1" -OutputAs DataTables
}
Main
Here is the unexpected output:
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True False DataRow System.Object
True True DataTable System.ComponentModel.MarshalByValueComponent
Using a similar example from the previous Q&A, to reproduce the same behavior:
function Invoke-SqlExample {
$dtt = [System.Data.DataTable]::new()
[void] $dtt.Columns.Add('Name')
$row = $dtt.NewRow()
$row.Name = "Hello"
$dtt.Rows.Add($row)
, $dtt
}
function Main {
$result1 = DoWork1
$result1.GetType()
$result2 = DoWork2
$result2.GetType()
}
function DoWork1 {
$result1 = Invoke-SqlExample
[pscustomobject]#{
Function = $MyInvocation.MyCommand.Name
Type = $result1.GetType().Name
} | Out-Host
return $result1
# Immediate fixes:
# return , $result1
# Write-Output $result1 -NoEnumerate
# $PSCmdlet.WriteObject($result1, $false) !! Only if Advanced Function
}
function DoWork2 {
return Invoke-SqlExample
}
Main
The output you would get from this is:
Function Type
-------- ----
DoWork1 DataTable
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True False DataRow System.Object
True True DataTable System.ComponentModel.MarshalByValueComponent
We can see that the unrolling of the DataTable is only done when previously assigned to a variable, even though the variable ($result1 in DoWork1 is still of the type DataTable).
This could be explained as, DoWork2 happens in a single pipeline as opposed to DoWork1 which happens in two pipelines, first the output from Invoke-SqlExample is collected in a variable, and then emitted as output (this is where the unrolling is triggered). This is based on assumptions, and may not be entirely correct.
As iRon suggested in his helpful comment from the prior answer, an immediate fix to have DoWork1 return the DataTable instance untouched (unrolled), we can use the comma operator , which will wrap the DataTable instance in an array which is then lost during enumeration (output from the function) since it is an array of one element. The other alternative would be using Write-Output -NeEnumerate. As last alternative we can also use $PSCmdlet.WriteObject(..., $false) only if the function is an advanced one.
Adding a similar example that demonstrates the same behavior, this one was provided by mclayton in his helpful comment:
function test {
, [Collections.Generic.List[string]]#(
'hello'
'world'
)
}
(& { test }).GetType() # => List`1
(& { $a = test; $a }).GetType() # => Object[]
To complement Santiago Squarzon's helpful answer and mclayton's helpful comments:
Note:
The use of return is incidental to the behavior, given that return ... is merely syntactic sugar for ...; return, i.e. outputting ..., followed by exiting the scope; at the end of a script or function, return is never required, as the function / script is exited anyway. Any statement in a script or function can produce output, due to PowerShell's implicit output behavior - see this answer.
The behavioral difference comes down this:
Invoke-Sqlcmd -Query ... is a command, so whatever it outputs is output as-is to the success output stream of the pipeline in which the enclosing function runs.
A command in PowerShell is a unit of functionality you invoke by name, which comprises cmdlets, scripts, functions, external programs, and script blocks.
Commands invariably involve a pipeline whose success output stream they directly write to - whether or not multiple commands are connected with |.
By contrast, $result1 is an expression, and if an expression evaluates to a value of a .NET type that PowerShell considers an enumerable,[1] it is enumerated when it is sent to the success output stream (of a pipeline), which in the case of a a collection means sending its elements one by one; note that the original collection type is therefore lost in the process.
As for the rationale for performing such enumeration: It is in line with the fundamental nature of PowerShell pipelines as streams of individual objects of indeterminate length - see this answer.
An expression in PowerShell is a construct that involves any combination of variable references, literals, most operators, and .NET method calls. In an of itself, an expression does not involve a pipeline, and collections and enumerables are treated as themselves. (An expression may have nested pipelines, however, if commands participate in it.)
A pipeline - and therefore enumeration - does get involved when an expression is used in the following contexts:
When you enclose the expression in $(...), the subexpression operator or #(...), the array-subexpression operator; e.g. $(1+2) or #(1+2)
Note: (...), the grouping operator does not do that when applied to an expression: in the context of a larger expression, it merely clarifies precedence. However, when applied to a command instead (only one supported, e.g., (Get-ChildItem).Count), it runs the command to completion in a nested pipeline, collects its output object(s) and either returns them as-is, if there's only one, otherwise as an [object[]] array.
When you pipe the expression to a command, using |, the pipeline operator; e.g. 1..3 | ForEach-Object { 1 + $_ }. Note that expressions are only allowed as the first segment of pipeline.
When you output the expression - whether implicitly or explicitly (with the rarely necessary Write-Output), whether in combination with return or not - from inside a command, such as in the case of outputting $result1 from your DoWork1 function.
The upshot:
If you store a command's output in a variable ($result1 = Invoke-Sqlcmd -Query ...), and later output that variable's value to the success output stream (return $result1 or just $result1 or Write-Output $result1), you potentially introduce an additional layer of enumeration.
To prevent enumeration, you can use the techniques mentioned in Santiago's answer (transitory single-element array wrapper trick constructed with unary , - , $return1 - or Write-Output's -NoEnumerate switch - Write-Output -NoEnumerate $return1)
However, this only matters for commands that output collections as a whole, as a single object, instead of streaming collections, i.e enumerating them and outputting their elements, one by one, which is the typical case.
Commands that output collections as single objects are rare, not least because the general expectation of PowerShell commands is that they indeed stream their output.
Invoke-SqlCmd -OutputAs DataTables is such a command, only because PowerShell - perhaps surprisingly - considers System.Data.DataTable instances enumerable - it conceives of it as a collection of rows, which it enumerates in the pipeline by default.[1]
[1] See the bottom section of this answer for an overview of what types PowerShell considers enumerable. In short: PowerShell has several exceptions where it considers types that declare themselves to be enumerable not enumerable, and one opposite exception: System.Data.DataTable does not declare itself enumerable, yet PowerShell enumerates it, namely as the collection of its rows stored in the .Rows property; Santiago provided the relevant source-code link.
I'm doing a script in powershell where i'm modifying the Chrome Bookmarks file.
This is what i want to do.
Read the file and parse the Json (Done)
Check if a certain folder is added, if it isn't, add it.
Parse again the object to Json, and save a new bookmark file. (i know how to do it)
This is how i convert it to Object:
$BkmrkJson = Get-Content $GoogleBkmrk | ConvertFrom-Json
And this is how i'm adding a new Object to the current "Childrens(Urls or bookmarks)".
$BkmrkJson.roots.bookmark_bar.children += New-Object -TypeName psobject -Property #{children=#();date_added="";date_modified="";guid="";id="";name="HV2";type="folder"}
My main problem, it's that when i add it, it isn't respecting the order of the properties. The usual order it's "children, date_added, date_modified, guid, id, name, type".
I add some values in blank, because Chrome adds new values automatically, after i add that value, or children, i parse again the psobject to Json.
$MyFinalJson = ConvertTo-Json $BkmrkJson -Depth 9
But when i create the file, it wasn't made correctly. So my principal question it's, how i can add correctly a new object to the parsed one, so when i parse it again, can recognize correctly the new ones.
hashtables (#{ ... }) are inherently unordered in PowerShell / .NET, i.e the order in which their entries are later enumerated isn't guaranteed, which means that by the time your New-Object call receives its -Property argument, the definition order of the entries in the hashtable is already lost.
However, PowerShell v3+ offers syntactic sugar for constructing custom objects ([pscustomobject] (aka [psobject])), in which case the order of entries, i.e. the order of the resulting properties is guaranteed to reflect the definition order, namely if you cast a hashtable to [pscustomobject]:
$BkmrkJson.roots.bookmark_bar.children +=
[pscustomobject] #{
children=#(); # Note: ";" only strictly needed in *single-line* defs.
date_added="";
date_modified="";
guid="";
id="";
name="Humach V2";
type="folder"
}
Note that in cases where you do want to stick with a hashtable (dictionary), you can "cast" a hashtable literal to [ordered], which also guarantees preserving the input order; specifically, this syntactic sugar creates an ordered hashtable, i.e. a System.Collections.Specialized.OrderedDictionary instance, which also implements the IDictionary interface, but (a) enumerates its entries in definition order and (b) allows accessing entries by positional index, as an alternative to using a key as the index; e.g.:
$orderedHashTable = [ordered] #{ zebra = 26; quebec = 17; alpha = 1}
$orderedHashTable.Keys # -> 'zebra', 'quebec', 'alpha'
# Access by key.
$orderedHashTable['quebec'] # -> 17
# Access by positional index
$orderedHashTable[0] # -> 26 (key 'zebra')
I have a inventory database from my company and I'm wanting to sort some entries based on pricing. I was thinking originally I would have to do everything by hand but I figured Sort-Object should work... until I remembered Sort-Object and its infamous string sorting. Easy, i'll sort by converting it to an integer except of course a currency value has symbol such as $ at the start.
The original code I used which caused the string sorting is below. The classic 200 is higher than 1000 etc:
$Result | Sort-Object -Property Price | Format-Table -Property Price
The int code I tried is:
$Result | Sort-Object -Property { [int]$_.Price } | Format-Table -Property Price
This results in output like "Cannot convert value "$414.50" to type "System.Int32". | Error: "Input string was not in a correct format." Makes sense, cant convert a $ to an int.
So is there any way around this without me having to sort by hand?
Thanks
To add to mclayton's helpful answer:
It is simpler to use a predefined [cultureinfo] instance that uses the your currency format, such as en-US (US-English) in the [decimal]::Parse() call, in combination with C, the currency format specifier.
#(
[pscustomobject] #{ Price='$414.50' },
[pscustomobject] #{ Price='99.02$' }
[pscustomobject] #{ Price='999.03' }
[pscustomobject] #{ Price='$5.04' }
) |
Sort-Object { [decimal]::Parse($_.Price, 'C', [cultureinfo] 'en-US') }
Output (correctly numerically sorted):
Price
-----
$5.04
99.02$
$414.50
999.03
Note:
As the sample input values show, there's some flexibility with respect to what input formats are accepted, such as a trailing $, and a value without $_.
If the current culture can be assumed to be en-US (or a different culture that uses the same currency symbol and formatting, notably also the same decimal separator, .), you can omit the [cultureinfo] 'en-US' argument in the [decimal]::Parse() call above - though for robustness I suggest keeping it.
As an aside: PowerShell's casts (which don't support currency values) always use the invariant culture with string operands, irrespective of the current culture. Thus, something like [decimal] '3.14' is recognized even while a culture that uses , as the decimal separator is in effect.
While the invariant culture - whose purpose is to provide representations that aren't culture-dependent and remain stable over time - is based on the US-English culture, it can not be used here, because its currency symbol is ¤; e.g., (9.99).ToString('C', [cultureinfo]::InvariantCulture) yields ¤9.99.
An input value that cannot be parsed as a currency causes an (effectively) non-terminating error,[1] and such values sort before the currency values.
If you simply want to ignore non-conforming values, use try { [decimal]::Parse(...) } catch { }
If you want to abort processing on encountering non-confirming values pass -ErrorAction Stop to the Sort-Object call.
[1] A .NET method call that fails causes a statement-terminating error, but since the error occurs in a script block (in the context of a calculated property), only the statement inside the script block is terminated, not the enclosing Sort-Object call
Firstly, you probably want [decimal] instead of [int] because [int] "414.50" is 414, not 414.50 so you'll be losing precision.
That aside, I'm adapting this answer for C#: https://stackoverflow.com/a/56603818/3156906
$fi = new-object System.Globalization.NumberFormatInfo;
$fi.CurrencySymbol = "`$";
#("`$10.00", "`$2.00") | Sort-Object -Property #{
"Expression" = { [decimal]::Parse($_, "Currency", $fi) }
};
# $2.00
# $10.00
The advantage of this is that invalid database values like - e.g. $1.$10 - that might have crept in will throw an exception, as will different currencies like £1.00 so you're getting a bit of extra data validation for free.
Note that the results remain as strings, but they're sorted as currency amounts (decimals). If you want the actual numeric value you'll need to convert the values separately...
In response to my previous question, I was given a working script that is based on a known-in-advance data type literally specified as [int].
Now I would like to change the data type dynamically. Is it possible?
The answer to your previous question uses type literals ([...]), which require that all types be specified verbatim (by their literal names).
Variable references are not supported in type literals; e.g. [Func[Data.DataRow, $columnType]] does not work - it causes a syntax error.
To generalize the linked answer based on dynamically determined (indirectly specified) types, two modifications are needed:
You must construct (instantiate) the closed generic types involved in the LINQ method call via the .MakeArrayType() and .MakeGenericType() methods.
You must use -as, the conditional type conversion operator to cast the input objects / the transformation script block ({ ... }) to those types.
# Create a sample data table...
[Data.DataTable] $dt = New-Object System.Data.DataTable
[Data.DataColumn] $column = New-Object System.Data.DataColumn "Id", ([int])
$dt.Columns.Add($column)
# ... and add data.
[Data.DataRow]$row = $dt.NewRow()
$row["Id"] = 1
$dt.Rows.Add($row)
$row = $dt.NewRow()
$row["Id"] = 2
$dt.Rows.Add($row)
# Using reflection, get the open definition of the relevant overload of the
# static [Linq.Enumerable]::Select() method.
# ("Open" means: its generic type parameters aren't yet bound, i.e. aren't
# yet instantiated with concrete types.)
$selectMethod = [Linq.Enumerable].GetMethods().Where({
$_.Name -eq 'Select' -and $_.GetParameters()[-1].ParameterType.Name -eq 'Func`2'
}, 'First')
# Dynamically set the name of the column to use in the projection.
$colName = 'Id'
# Dynamically set the in- and output types to use in the LINQ
# .Select() (projection) operation.
$inType = [Data.DataRow]
$outType = [int]
# Now derive the generic types required for the LINQ .Select() method
# from the types above:
# The array type to serve as the input enumerable.
# Note: As explained in the linked answer, the proper - but more cumbersome -
# solution would be to use reflection to obtain a closed instance of
# the generic .AsEnumerable() method.
$inArrayType = $inType.MakeArrayType()
# The type for the 'selector' argument, i.e. the delegate performing
# the transformation of each input object.
$closedFuncType = [Func`2].MakeGenericType($inType, $outType)
# Close the generic .Select() method with the given types
# and invoke it.
[int[]] $results = $selectMethod.MakeGenericMethod($inType, $outType).Invoke(
# No instance to operate on - the method is static.
$null,
# The arguments for the method, as an array.
# Note the use of the -as operator with the dynamically constructed types.
(
($dt.Rows -as $inArrayType),
({ $args[0].$colName } -as $closedFuncType)
)
)
# Output the result.
$results
Taking a step back:
As shown in the linked answer, PowerShell's member-access enumeration can provide the same functionality, with greatly simplified syntax and without needing to deal with types explicitly:
# Create a sample data table...
[Data.DataTable] $dt = New-Object System.Data.DataTable
[Data.DataColumn] $column = New-Object System.Data.DataColumn "Id", ([int])
$dt.Columns.Add($column)
# ... and add data.
[Data.DataRow]$row = $dt.NewRow()
$row["Id"] = 1
$dt.Rows.Add($row)
$row = $dt.NewRow()
$row["Id"] = 2
$dt.Rows.Add($row)
# Dynamically set the name of the column to use in the projection.
$colName = 'Id'
# Use member-access enumeration to extract the value of the $colName column
# from all rows.
$dt.$colName # Same as: $dt.Rows.$colName
You may want to back up and ask yourself why you are trying to use LINQ in PowerShell. A tip is that if it looks like C#, there is likely a better way to do it.
I assume that you are new to PowerShell so I will give you a quick heads up as to why LINQ is actually "easier" in PowerShell (technically it is not LINQ anymore but I think it looks like it is) when combined with the pipeline.
Try Get-Help *-Object on a PowerShell prompt sometime. Notice the cmdlets that show up? Select-Object, Where-Object, Group-Object, and Sort-Object do the same things as LINQ and their names match up to what you expect. Plus, there is no strongly-typed requirement strictly speaking.
$data | Where-Object -Property Greeting -Like *howdy* | Select-Object Name,Year,Greeting
in a text something like this, I need to be able to read the project code which is unique per text file.
devices :
meta : #{Projectcode=rvmf99999}
public_keys : #{Key=ssh-
select-string -pattern "rvmf" picks up the whole line, I just need rvmf and the digits after that.
# Sample input
$txt = #'
devices :
meta : #{Projectcode=rvmf99999}
public_keys : #{Key=ssh-
'#
$txt | Select-String 'rvmf\d+' | foreach { $_.Matches[0].Value } # -> 'rvmf99999'
Regex 'rvmf\d+' captures substring 'rvmf' followed by 1 or more (+) digits (\d).
The object output by Select-String has a .Matches property whose first entry's .Value property contains what the regex captured.
Specifically, the output objects are of type Microsoft.PowerShell.Commands.MatchInfo, which contains the input line (property .Line) as well as metadata about the source of the line and details about the regex-matching operation in the .Matches property.
Specifically, the .Matches property contains a collection of match-information objects; unless -AllMatches was passed to Select-Object, there will only be one element, however.
Each element of the .Matches collection is a System.Text.RegularExpressions.Match instance, whose .Value property contains what the regex captured as a whole.
Note: There is an upcoming feature - green-lighted, but not yet implemented as of PowerShell Core 7.0.0-preview.5 - that will greatly simplify the command:
# NOT YET IMPLEMENTED as of PowerShell Core 7.0.0-preview.5
$txt | Select-String 'rvmf\d+' -OnlyMatching # -> 'rvmf99999'
-OnlyMatching will only output the part of the line that was matched.