$res = Invoke-Sqlcmd -Query "select * from customer" -ServerInstance "(localdb)\MSSQLLocalDB" -Database "Database1" -OutputAs DataTables
$res | Where-Object FirstName -eq "John"
$res.Where({$_.FirstName -eq "John"})
This is the output:
Id FirstName City
-- --------- ----
1 John Augusta
1 John Augusta
I was expecting this:
Id FirstName City
-- --------- ----
1 John Augusta
Id FirstName City
-- --------- ----
1 John Augusta
Building on the helpful comments:
The behavior is by design:
Objects output by a given script or a single interactively submitted command line are all sent to the success output stream of a single pipeline.
When output from a pipeline is neither captured nor redirected, PowerShell applies for-display output formatting to all output objects in the success output stream of a given pipeline, and if the first output object implicitly selects tabular formatting, all subsequent objects of the same type are formatted as part of the same table that is printed to the host (display, terminal).
Things get tricky if subsequent objects are of a different type as well as if they're also [pscustomobject] instances, but with different property sets - see this answer for more information.
If you want to format a given command's output individually, you have three basic options, all of which are suboptimal in case you also want to option to later programmatically process the output, not just format it for display:
Send the output directly to the host, using Out-Host:
$res | Where-Object FirstName -eq "John" | Out-Host
This bypasses the success output stream, meaning that this output cannot be captured or redirected.
Use a Format-* cmdlet such as Format-Table explicitly:
$res | Where-Object FirstName -eq "John" | Format-Table
This sends objects representing formatting instructions rather than the original objects to the success output stream, which the host (as the default output target) renders correctly, but these objects are meaningless for further programmatic processing.
Use Out-String (possibly preceded by a call to a Format-* cmdlet to select the kind of formatting and/or control the specifics of the formatting):
$res | Where-Object FirstName -eq "John" | Out-String
This sends a single, multi-line string to the success output stream that contains the same representation you would see in the host with Out-Host[1] and since strings always render as-is, you'll see the same host output; in programmatic processing, these strings are relatively more meaningful than the formatting-instruction objects output by Format-* cmdlets, but still amount to a loss of information compared to the original output objects.
[1] Unfortunately, Out-String always appends a trailing newline to this representation; this problematic behavior is the subject of GitHub issue #14444. As zett42 points out, using Out-String -Stream avoids this problem, albeit at the expense of sending the lines of the multi-line string representation individually to the output stream; To avoid that, you can use (... | Out-String -Stream) -join "`n"
Related
I am trying to query multiple servers with WMI, but I don't always have access to the servers.
The code is below. Alas, it returns "access is denied" to the console, but I can't seem to get rid of it. Oh well.
However, I am trapping the servers that I can't connect to, so that I can tell someone else to look at them, or request access.
But when I run the code, it only returns the first list of servers; even if $failed_servers has values, nothing is returned. If I tell both to pipe to ogv, then two windows pop up.
Why won't both "$variable|select" work? If I remove the select on $failed_servers, then it shows up, albeit just sitting immediately underneath the successful ones. Which is okay-but-not-great.
$list = ("servera","serverb","serverc")
$failed_servers = #()
$final = foreach ($server_instance in $list)
{
$errors=#()
gwmi -query "select * from win32_service where name like '%SQLSERVER%'" -cn $server_instance -ErrorVariable +errors -ErrorAction SilentlyContinue
if ($errors.Count -gt 0) {$failed_servers += $server_instance
}
}
$final|select pscomputername, name, startmode, state |where {$_.pscomputername -ne $null}
$failed_servers |select #{N='Failed Servers'; E={$_}}
What you're experiencing is merely a display problem:
Both your Select-Object calls produce output objects with 4 or fewer properties whose types do not have explicit formatting data associated with them (as reported by Get-FormatData).
This causes PowerShell's for-display output formatting system to implicitly render them via the Format-Table cmdlet.
The display columns that Format-Table uses are locked in based on the properties of the very first object that Format-Table receives.
Therefore, your second Select-Object call, whose output objects share no properties with the objects output by the first one, effectively produces no visible output - however, the objects are sent to the success output stream and are available for programmatic processing.
A simple demonstration:
& {
# This locks in Month and Year as the display columns of the output table.
Get-Date | Select-Object Month, Year
# This command's output will effectively be invisible,
# because the property set Name, Attributes does not overlap with
# Month, Year
Get-Item \ | Select-Object Name, Attributes
}
The output will look something like this - note how the second statement's output is effectively invisible (save for an extra blank line):
Month Year
----- ----
9 2021
Note the problem can even affect a single statement that outputs objects of disparate types (whose types don't have associated formatting data); e.g.:
(Get-Date | Select-Object Year), (Get-Item \ | Select-Object Name)
Workarounds:
Applying | Format-List to the command above makes all objects visible, though obviously changes the display format.
Intra-script you could pipe each Select-Object pipeline to Out-Host to force instant, pipeline-specific formatting, but - given that the results are sent directly to the host rather than to the success output stream - this technique precludes further programmatic processing.
Potential future improvements:
GitHub issue #7871 proposes at least issuing a warning if output objects effectively become invisible.
I’ve got a few functions that return an array of PsCustomObjects, and returning other stuff would be bad.
Yet, at times, I’d like to pass in -Verbose and spit out some information to the host (using Write-Host). I can easily implement Test-Verbose, but when I want to spit out a table (with Format-Table), it’s not as easy as capturing the output and piping it to Write-Host.
What is the right way to show the output of Format-Table to the host, without sending it to the stdout, where it will be tallied as part of the function’s return values?
For example:
$objects | Format-Table | Write-Host
Thanks!!
You should be able to do this with Out-String. For example, change you example to:
$objects | Format-Table | Out-String | Write-Host
Does that work?
I am doing an API request which returns a bunch of data. In attempted to search through it with Select-String, it just spits out the entire value stored in the variable. This is an internet server which I am calling an api.
$return = Invoke-RestMethod -Method GET -Uri $uri -Headers #{"authorization" = $token} -ContentType "application/json"
$file = $return.data
$file | Out-String -Stream | Select-String -Pattern "word"
this returns the entire value of $file. printing $file looks like same as the pipe output. Why is this not working?
$file.Gettype says it is a system.object, another answer said to use Out-String, but something is not working.
$file.Gettype
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True String System.Object
To complement iRon7's helpful answer with the precise logic of Out-String's -Stream switch, as of PowerShell 7.1:
Out-String, like the other Out-* cmdlets such as Out-File, uses PowerShell's rich output-formatting system to generate human-friendly representations of its input objects.
Without -Stream, Out-String only ever produces a single, (typically) multiline string.
With -Stream, line-by-line output behavior typically occurs - except for input objects that happen to be multiline strings, which are output as-is.
Because this exception is both obscure and unhelpful, GitHub proposal #14638 suggests removing it.
For so-called in-band data types, -Stream works as follows, which truly results in line-by-line output:
Input objects are formatted by PowerShell's rich formatting system, and the lines that make up the resulting representation are then output one by one.
Out-of-band data types are individually formatted outside of the formatting system, by simply calling their .NET .ToString() method.
In short: data types that represent a single value are out-of-band, and in addition to [string] out-of-band data types also comprise [char] and the various (standard) numeric types, such as [int], [long], [double], ...
[string] is the only out-of-band type that itself can result in a multiline representation, because calling .ToString() on a string is effective no-op that returns the string itself - whether it is single- or multiline.
Therefore:
Any string - notably also a multiline string - is output as-is, as a whole, and splitting it into individual lines requires an explicit operation; e.g. (note that regex \r?\n matches both Windows-style CRLF and Unix-style LF-only newlines):
"line 1`nline 2`nline 3" -split '\r?\n' # -> 'line 1', 'line 2', 'line 3'
If your input objects are a mix of in-band objects and (invariably out-of-band) multiline strings, you can combine Out-String -Stream with -split; e.g.:
((Get-Date), "line 1`nline 2`nline 3" | Out-String -Stream) -split '\r?\n'
On closer inspection, I suspect that your issue comes from an ambiguity in the Out-String documentation:
-Stream
Indicates that the cmdlet sends a separate string for each line of an
input object. By default, the strings for each object are accumulated
and sent as a single string.
Where the word line should be read as item.
To split you raw string into separate lines, you will need to split your string using the following command:
$Lines = $return.data -split [Environment]::NewLine
Note that this assumes that your data uses the same characters for a new line as the system you working on. If this is not the case, you might want to split the lines using an regular expression, e.g.:
$Lines = $return.data -split "`r*`n"
So what does the-Stream parameter do?
It sends a separate string for each item of an input object.
Where in this definition, it is also a known common best PowerShell practice to use a singular name for possible plural input objectS.
Meaning if you use the above defined $Lines variable (or something like $Lines = Get-Content .\File.json), the input object "$Lines" is a collection of strings:
$Lines.GetType().Name
String[]
if you stream this to Out-String it will (by default) join all the items and return a single string:
($Lines | Out-String).GetType().Name
String
In comparison, if you use the -Stream parameter, it will pass each separated item from the $Lines collection directly to the next cmdlet:
($Lines | Out-String -Stream).GetType().Name
Object[]
I have created a document issue for this: #7133 "line" should be "item"
Note:
In general, it is a bad practice to peek and poke directly into a
serialized string
(including Json) using string
methods and/or cmdlets (like Select-String). Instead you should use
the related parser (e.g.
ConvertFrom-Json)
for searching and replacing which will result in an easier syntax
and usually takes care of known issues and pitfalls.
Select-String outputs Microsoft.PowerShell.Commands.MatchInfo objects. It seems to me that the output is somehow fancified via the PS engine or something to highlight your match, but ultimately it does print the entire matched string.
You should check out the members of the object Select-String provides, like this:
$file | Out-String -Stream | Select-String -Pattern "word" | Get-Member
TypeName: Microsoft.PowerShell.Commands.MatchInfo
Name MemberType Definition
---- ---------- ----------
...
Matches Property System.Text.RegularExpressions.Match[] Matches {get;set;}
...
What you're interested in is the Matches property. It contains a bunch of information about the match. To extract exactly what you want, look at the Value property of Matches:
($file | Out-String -Stream | Select-String -Pattern "word").Matches.Value
word
Another way:
$file | Out-String -Stream | Select-String -Pattern "word" | ForEach-Object {$_.Matches} | Select-Object -Property Value
Value
-----
word
Or
$file | Out-String -Stream | Select-String -Pattern "word" | ForEach-Object {$_.Matches} | Select-Object -ExpandProperty Value
word
I'm trying to do the following:
Display a list from a multidimensional array:
$AssetList | Where-Object {$_.agenttype -eq "N"} | Select-Object number, hostname, agentstatus | Write-Output
Then read in a number from the user:
$Victim = Read-Host -Prompt 'Enter the number of the host you want rid of...'
What happens in reality is the Read-Host is displayed before the Write-Output. How can I flip it and reverse it?
I think the issue here is that you're using Write-Output (which writes to the output stream for your script/function). If you just want the data to be displayed on the console before the Read-Host cmdlet, try using Out-String | Write-Host. You can probably get away with just Out-String on its own.
Out-String
The Out-String cmdlet converts the objects that Windows PowerShell
manages into an array of strings. By default, Out-String accumulates
the strings and returns them as a single string, but you can use the
stream parameter to direct Out-String to return one string at a time.
This cmdlet lets you search and manipulate string output as you would
in traditional shells when object manipulation is less convenient.
https://technet.microsoft.com/en-us/library/hh849952.aspx
I'm trying to do some processing logic - running some commands in parallel based on the tree configuration CSV file:
Operation;Parent;Enabled;Propagated;Job_ID;Status;Started;Finished
CA1;n/a;Y;N;;;;
PROD1;n/a;Y;N;;;Y;
CON1;CA1;N;N;;;Y;
CON2;CON1;N;N;;;Y;
I load the file into the variable and then I'm trying to find the next step which needs to be processed:
$Data = Import-Csv -delimiter ";" .\config.csv
$NextStep = $Data | Select-Object -first 1 | Where-Object {$_.Started -eq ""}
$NextStepText = $NextStep.Operation | ft -autosize | out-string
The problem is that it seems like $NextStep.Operation contains new line character. When I display it I get:
PS C:\temp\SalesForce> $NextStep.operation
CA1
PS C:\temp\SalesForce> $NextStep.Operation.Contains("`n")
False
Do you know what I'm doing wrong? I would like to display the content without the "dummy" new line character which is there even if contains method is saying it is not there.
Or please advise how to do it better. I'm still learning PowerShell; so far I just google the commands, and I'm trying to put it together.
The newline isn't in your data, it's being added by Out-String. Observe the output of the following (in particular, where you do and don't get the newline after CA1):
$Data = import-csv -delimiter ";" .\config.csv
$NextStep = $Data | select-object -first 1 | where-object {$_.Started -eq ""}
$NextStepText = $NextStep.Operation | ft -autosize | out-string
"hi"
$NextStepText
"hi"
$NextStep.Operation;
"hi"
$NextStep.Operation | ft -autosize
"hi"
You shouldn't be using Format-Table at that step (and Out-String is unnecessary in this script) if you intend to use $NextStepText for anything other than direct output later on. Consider Format-Table (or any of the Format-* cmdlets) the end of the line for usable data.
Why do you think that there is a new line character of some sort in there? If you are using the ISE then what you posted doesn't look like there is. It is normal to have a blank line between commands (in the v2/v3 ISE, not sure about v4), so what you posted would not indicate that it contains any new line characters.
You can always check the $NextStep.Operation.Length to see if it says 3 or 4. If there is a `n in there it'll show up in the length. For example (copied and pasted out of my v3 PS ISE):
PS C:\> $test = "Test`nTest2"
PS C:\> $test
Test
Test2
PS C:\> $test.Length
10
PS C:\>
That was to show that there is a new line character injected by following it with text, without any text following the new line character it looks like this:
PS C:\> $test = "Test`n"
PS C:\> $test
Test
PS C:\> $test.Length
5
PS C:\>
You'll notice that there are 2 blank lines after the text "Test" on the second command. The first is the line injected into the variable, and the second is the obligatory line that PS puts in to show separation between commands.
Out-String unexpectedly appends a trailing newline to the string it outputs.
This problematic behavior is discussed in GitHub issue #14444.
A simple demonstration:
# -> '42<newline>'
(42 | Out-String) -replace '\r?\n', '<newline>'
However, you neither need Format-Table nor Out-String in your code:
Format-* cmdlets output objects whose sole purpose is to provide formatting instructions to PowerShell's for-display output-formatting system. In short: only ever use Format-* cmdlets to format data for display, never for subsequent programmatic processing - see this answer for more information.
Out-String is capable of interpreting these formatting instructions, i.e. it does produce data - in the form of a single, multi-line string by default - that is the string representation of what would print to the display.
As such, the resulting string contains a representation for the human observer, not a structured text format suitable for programmatic processing.
In your case, Format-Table is applied to a string, which is pointless, because strings always render as themselves, in full (-AutoSize has no effect); piping to Out-String then in effect returns the original string with an (undesired) newline appended.
Therefore, use a simple variable assignment to store the property value of interest in a separate variable:
$NextStepText = $NextStep.Operation