I'm trying to do some processing logic - running some commands in parallel based on the tree configuration CSV file:
Operation;Parent;Enabled;Propagated;Job_ID;Status;Started;Finished
CA1;n/a;Y;N;;;;
PROD1;n/a;Y;N;;;Y;
CON1;CA1;N;N;;;Y;
CON2;CON1;N;N;;;Y;
I load the file into the variable and then I'm trying to find the next step which needs to be processed:
$Data = Import-Csv -delimiter ";" .\config.csv
$NextStep = $Data | Select-Object -first 1 | Where-Object {$_.Started -eq ""}
$NextStepText = $NextStep.Operation | ft -autosize | out-string
The problem is that it seems like $NextStep.Operation contains new line character. When I display it I get:
PS C:\temp\SalesForce> $NextStep.operation
CA1
PS C:\temp\SalesForce> $NextStep.Operation.Contains("`n")
False
Do you know what I'm doing wrong? I would like to display the content without the "dummy" new line character which is there even if contains method is saying it is not there.
Or please advise how to do it better. I'm still learning PowerShell; so far I just google the commands, and I'm trying to put it together.
The newline isn't in your data, it's being added by Out-String. Observe the output of the following (in particular, where you do and don't get the newline after CA1):
$Data = import-csv -delimiter ";" .\config.csv
$NextStep = $Data | select-object -first 1 | where-object {$_.Started -eq ""}
$NextStepText = $NextStep.Operation | ft -autosize | out-string
"hi"
$NextStepText
"hi"
$NextStep.Operation;
"hi"
$NextStep.Operation | ft -autosize
"hi"
You shouldn't be using Format-Table at that step (and Out-String is unnecessary in this script) if you intend to use $NextStepText for anything other than direct output later on. Consider Format-Table (or any of the Format-* cmdlets) the end of the line for usable data.
Why do you think that there is a new line character of some sort in there? If you are using the ISE then what you posted doesn't look like there is. It is normal to have a blank line between commands (in the v2/v3 ISE, not sure about v4), so what you posted would not indicate that it contains any new line characters.
You can always check the $NextStep.Operation.Length to see if it says 3 or 4. If there is a `n in there it'll show up in the length. For example (copied and pasted out of my v3 PS ISE):
PS C:\> $test = "Test`nTest2"
PS C:\> $test
Test
Test2
PS C:\> $test.Length
10
PS C:\>
That was to show that there is a new line character injected by following it with text, without any text following the new line character it looks like this:
PS C:\> $test = "Test`n"
PS C:\> $test
Test
PS C:\> $test.Length
5
PS C:\>
You'll notice that there are 2 blank lines after the text "Test" on the second command. The first is the line injected into the variable, and the second is the obligatory line that PS puts in to show separation between commands.
Out-String unexpectedly appends a trailing newline to the string it outputs.
This problematic behavior is discussed in GitHub issue #14444.
A simple demonstration:
# -> '42<newline>'
(42 | Out-String) -replace '\r?\n', '<newline>'
However, you neither need Format-Table nor Out-String in your code:
Format-* cmdlets output objects whose sole purpose is to provide formatting instructions to PowerShell's for-display output-formatting system. In short: only ever use Format-* cmdlets to format data for display, never for subsequent programmatic processing - see this answer for more information.
Out-String is capable of interpreting these formatting instructions, i.e. it does produce data - in the form of a single, multi-line string by default - that is the string representation of what would print to the display.
As such, the resulting string contains a representation for the human observer, not a structured text format suitable for programmatic processing.
In your case, Format-Table is applied to a string, which is pointless, because strings always render as themselves, in full (-AutoSize has no effect); piping to Out-String then in effect returns the original string with an (undesired) newline appended.
Therefore, use a simple variable assignment to store the property value of interest in a separate variable:
$NextStepText = $NextStep.Operation
Related
I am doing an API request which returns a bunch of data. In attempted to search through it with Select-String, it just spits out the entire value stored in the variable. This is an internet server which I am calling an api.
$return = Invoke-RestMethod -Method GET -Uri $uri -Headers #{"authorization" = $token} -ContentType "application/json"
$file = $return.data
$file | Out-String -Stream | Select-String -Pattern "word"
this returns the entire value of $file. printing $file looks like same as the pipe output. Why is this not working?
$file.Gettype says it is a system.object, another answer said to use Out-String, but something is not working.
$file.Gettype
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True String System.Object
To complement iRon7's helpful answer with the precise logic of Out-String's -Stream switch, as of PowerShell 7.1:
Out-String, like the other Out-* cmdlets such as Out-File, uses PowerShell's rich output-formatting system to generate human-friendly representations of its input objects.
Without -Stream, Out-String only ever produces a single, (typically) multiline string.
With -Stream, line-by-line output behavior typically occurs - except for input objects that happen to be multiline strings, which are output as-is.
Because this exception is both obscure and unhelpful, GitHub proposal #14638 suggests removing it.
For so-called in-band data types, -Stream works as follows, which truly results in line-by-line output:
Input objects are formatted by PowerShell's rich formatting system, and the lines that make up the resulting representation are then output one by one.
Out-of-band data types are individually formatted outside of the formatting system, by simply calling their .NET .ToString() method.
In short: data types that represent a single value are out-of-band, and in addition to [string] out-of-band data types also comprise [char] and the various (standard) numeric types, such as [int], [long], [double], ...
[string] is the only out-of-band type that itself can result in a multiline representation, because calling .ToString() on a string is effective no-op that returns the string itself - whether it is single- or multiline.
Therefore:
Any string - notably also a multiline string - is output as-is, as a whole, and splitting it into individual lines requires an explicit operation; e.g. (note that regex \r?\n matches both Windows-style CRLF and Unix-style LF-only newlines):
"line 1`nline 2`nline 3" -split '\r?\n' # -> 'line 1', 'line 2', 'line 3'
If your input objects are a mix of in-band objects and (invariably out-of-band) multiline strings, you can combine Out-String -Stream with -split; e.g.:
((Get-Date), "line 1`nline 2`nline 3" | Out-String -Stream) -split '\r?\n'
On closer inspection, I suspect that your issue comes from an ambiguity in the Out-String documentation:
-Stream
Indicates that the cmdlet sends a separate string for each line of an
input object. By default, the strings for each object are accumulated
and sent as a single string.
Where the word line should be read as item.
To split you raw string into separate lines, you will need to split your string using the following command:
$Lines = $return.data -split [Environment]::NewLine
Note that this assumes that your data uses the same characters for a new line as the system you working on. If this is not the case, you might want to split the lines using an regular expression, e.g.:
$Lines = $return.data -split "`r*`n"
So what does the-Stream parameter do?
It sends a separate string for each item of an input object.
Where in this definition, it is also a known common best PowerShell practice to use a singular name for possible plural input objectS.
Meaning if you use the above defined $Lines variable (or something like $Lines = Get-Content .\File.json), the input object "$Lines" is a collection of strings:
$Lines.GetType().Name
String[]
if you stream this to Out-String it will (by default) join all the items and return a single string:
($Lines | Out-String).GetType().Name
String
In comparison, if you use the -Stream parameter, it will pass each separated item from the $Lines collection directly to the next cmdlet:
($Lines | Out-String -Stream).GetType().Name
Object[]
I have created a document issue for this: #7133 "line" should be "item"
Note:
In general, it is a bad practice to peek and poke directly into a
serialized string
(including Json) using string
methods and/or cmdlets (like Select-String). Instead you should use
the related parser (e.g.
ConvertFrom-Json)
for searching and replacing which will result in an easier syntax
and usually takes care of known issues and pitfalls.
Select-String outputs Microsoft.PowerShell.Commands.MatchInfo objects. It seems to me that the output is somehow fancified via the PS engine or something to highlight your match, but ultimately it does print the entire matched string.
You should check out the members of the object Select-String provides, like this:
$file | Out-String -Stream | Select-String -Pattern "word" | Get-Member
TypeName: Microsoft.PowerShell.Commands.MatchInfo
Name MemberType Definition
---- ---------- ----------
...
Matches Property System.Text.RegularExpressions.Match[] Matches {get;set;}
...
What you're interested in is the Matches property. It contains a bunch of information about the match. To extract exactly what you want, look at the Value property of Matches:
($file | Out-String -Stream | Select-String -Pattern "word").Matches.Value
word
Another way:
$file | Out-String -Stream | Select-String -Pattern "word" | ForEach-Object {$_.Matches} | Select-Object -Property Value
Value
-----
word
Or
$file | Out-String -Stream | Select-String -Pattern "word" | ForEach-Object {$_.Matches} | Select-Object -ExpandProperty Value
word
I am new to PowerShell, and I have the following example code and output to illustrate my problem:
select-string "$env:appdata\..\Local\test\*.ini" -pattern "example_adjustment=" | select filename, line | sort-object -property line -Descending >> file.txt
Filename Line
-------- ----
test1.ini example_adjustment="4.2"
test4.ini example_adjustment="11.0000000"
test2.ini example_adjustment="1.20"
test5.ini example_adjustment="0.90"
test3.ini example_adjustment="0.90"
I want to be able to modify the output so that the "Line" values appear as their numbers only and in float format for the purpose of the sort performing correctly. The end result is I'd be appending that information to a text file.
How would I go about modifying the Line property? I saw a post about regex, but I cannot edit directly from the pipe using regex it seems.
I cannot edit directly from the pipe using regex it seems.
You most certainly can! :-)
Use the -replace regex operator inside a calculated property:
... |Select filename,#{Name='Line';Expression={$_.Line -replace 'example_adjustment="([^"]*)"','$1'}}
I am trying to write a simple PS script to check large .txt log files for a short string: "SRVE0242I:"
$lines = Select-String -Path $logDir -Pattern "SRVE0242I:" | Select-Object line | Out-String
On output though, it only displays the following:
Line
[28/06/17 13:48:27:839] 00000020 ServletWrappe I SRVE0242I: [User] [User] [com_xxxxxxx_...
And not the full line. Is there a limit to how many characters this pulls? I can't find any info on any restrictions for the Select-String cmdlet. Is there a better way to do this so that I don't a) pull the heading "Line" in my list of lines (Don't really want to create table formatting for such a simple output) and b) get the whole line when I pull the info?
You are seeing it like this because it's displaying the Line property using the default Format-Table view and shortening it to the width of the console.
Do this instead:
$lines = Select-String -Path $logDir -Pattern "SRVE0242I:" | Select-Object -ExpandProperty line
This returns the value of the Line property as a string to the $lines variable. You don't need to use Out-String.
There is! Long story short, Select-Object is doing the truncating here. Here's one way to get the first untruncated line in a Select-String output
$(Select-String -Path $logDir -Pattern "SRVE0242I:")[0].Line
When you run into something like this, you can break down the individual steps to determine what's happening by piping things to Get-Member. Here's what's happening in the code above:
Select-String <# args #> | Get-Member
Select-String gives us a MatchInfo object, which (as you've correctly determined) has a 'Line' property. When run on it's own, Select-String will actually spit out all the information you're looking for, and will not truncate it by default (at least, on v6.0.0-beta). It does give you an array of MatchInfo objects if it finds multiple matches, so you have to index into that array if you just want the first one (like I did above).
Select-String <# args #> | Select-Object Line | Get-Member
Select-Object applies PowerShell's default formatting for objects which, in most cases, will truncate your output for easier viewing. For objects with a bunch of members (like a MatchInfo object), it will try to do one per line by default.
Select-String <# args #> | Select-Object Line | Out-String | Get-Member
Out-String directly translates it's input to a string. That is, rather than trying to cast something to a string or pull a string Property out of an object that's passed to it, it just changes whatever it receives into an object. In this case, it turns the already-formatted MatchInfo output into a string. Nothing happens to the output on the terminal, but Get-Member will reveal a String rather than a MatchInfo object.
It's not directly relevant here, but if you're interested in modifying the default formatting, it's governed by the types.ps1xml file.
I'm trying to figure out why Select-Object
adds a lot of whitespace at the start of its output; and
truncates long properties with ellipsis.
Here's a repro of what I mean. Suppose you run these commands on C:\:
New-Item "MyTest" -Type Directory
cd MyTest
"Some very long lorem ipsum like text going into a certain file, bla bla bla and some more bla." | Out-File test.txt
Get-ChildItem | Select-String "text" | Select-Object LineNumber,Line
This will show output like this:
The ellipsis I can understand, that would be just the way the command ends up getting formatted when the result is written to the console host. However, the whitespace at the start still confuses me in this case.
Things get weirder for me though when I pipe the result to either clip or Out-File output.txt. I get similarly formatted output, with a lot of whitespace at the start and truncated Line properties.
Which command is causing this behavior, and how can I properly solve this? Most importantly: how can I get the full results into a file or onto my clipboard?
The default behavior of outputting the data is to use Format-Table without any modifiers, and the default behavior of Format-Table is to split the viewport into columns of equal width. This makes no assumption on the output width, and is faster in that the cmdlet doesn't need to process any string data from the pipeline prior to output.
To reduce the whitespace, you should use Format-Table -AutoSize as the output method. The -AutoSize switch first measures the widths of data, then outputs with regard to calculated width. If you need to not receive ellipsis and always display the full data set, add -Wrap switch to Format-Table, this way the value will be wrapped into more than a single line, but you can copy it via selecting a square area in Powershell window, just strip newlines off the clipped contents.
Get-ChildItem | Select-String "text" | Select-Object LineNumber,Line | Format-Table -AutoSize -Wrap
I'd say the best way to get the full output into a file would be to export the result as a CSV:
Get-ChildItem |
Select-String "text" |
Select-Object LineNumber,Line |
Export-Csv 'out.csv'
You could also build a string from the selected properties, which might be better for copying the data to the clipboard:
Get-ChildItem |
Select-String "text" |
ForEach-Object { '{0}:{1}' -f $_.LineNumber, $_.Line } |
Tee-Object 'out.txt' | clip
The behavior you observed is caused by the way PowerShell displays output. Basically, it looks at the first object and counts the properties. Objects with less than 5 properties are sent to Format-Table, otherwise to Format-List. The columns of tabular output are spread evenly across the available space. As #Vesper already mentioned you can enforce proportional column width by using the -AutoSize parameter, and wrapping of long lines by using the -Wrap parameter. Format-List wraps long strings by default.
See this blog post from Jeffrey Snover for more information.
Is there any way to retain the Powershell command coloring scheme after execution of the command in the Console Pane.
And is there a way to color the output column-headers separately.
None of the themes I browsed or could find online achieve this.
Try the following:
Get-Disk | Select -property * | ft -AutoSize -Wrap | Out-String -stream | %{if($_ | Select-String 'Style' -NotMatch){Write-Host $_ -f Yellow}else{Write-Host $_}}
This passes your output to a string. The -stream operator makes each line into a separate string, and you then check each string for something in your header ('Style'). If it checks, the string gets written out with the color formatting.
The Output Colorizer VSC extension does it for the Output stream. I wish it could colorize the terminal output too.
https://marketplace.visualstudio.com/items?itemName=IBM.output-colorizer
#uruloki