I am trying to output to a text file the results of the powershell cmdlet Compare-Object The problem is I cannot eliminate the ellipse truncation.
The code below provides a table formatting definition variable which specifies a width of 1000 for the Path column. Yet the output file always truncates the Path column at 122 characters.
The Compare-Object cmdlet is comparing two ArrayLists which are just lists of file path strings from common folder paths between two servers.
What I am attempting to do is put the SideIndicator as the first column and the full path in the second. I do not want truncating of the file path.
$tableFormat = #{Expression={$_.SideIndicator};Label="Side Indicator";width=15}, #{Expression={$_.InputObject};Label="Path";width=1000}
$outputFilename = ($server1 + "_" + $server2 + "_FileCompare" + ".txt");
Compare-Object $Hive1FileArray $Hive2FileArray -IncludeEqual | Format-Table $tableFormat | Out-String | Out-File $outputFilename
I also tried removing Out-String from the pipe makes no difference.
What is going wrong here?
Thanks
Compare-Object $Hive1FileArray $Hive2FileArray -IncludeEqual |`
Format-Table $tableFormat -AutoSize |`
Out-String -Width 1000 |`
Out-File $outputFilename
Read
Get-Help 'Format-Table' -ShowWindow or its Online Version:
-AutoSize
Adjusts the column size and number of columns based on the width of
the data. By default, the column size and number are determined by the
view.
Get-Help 'Out-String' -ShowWindow or its Online Version:
-Width <Int32>
Specifies the number of characters in each line of output. Any
additional characters are truncated, not wrapped. If you omit this
parameter, the width is determined by the characteristics of the host
program. The default value for the Windows PowerShell console is 80
(characters).
Not much more to say not knowing Compare-Object cmdlet input objects…
I know this is a year old but another useful parameter of Format-Table is -wrap.
-Wrap []
Indicates that the cmdlet displays text that exceeds the column width on the next line. By default, text that exceeds the column width is truncated.
Required? false
Position? named
Default value False
Accept pipeline input? False
Accept wildcard characters? false
Related
I have the following silly PowerShell script:
$username = 'rny'
$null = mkdir "c:\Users\$username\YYY"
$null = mkdir "c:\Users\$username\YYY\TODO"
$null = mkdir "c:\Users\$username\YYY\TODO\2021-12-22_Foo-bar-baz-etc"
$files = "C:\Users\$username\one-two-three-four.sql.wxyz",
"C:\Users\$username\YYY\TODO\2021-11_29_abcdefghijklmnop.wxyz",
"C:\Users\$username\YYY\TODO\2021-12-22_Foo-bar-baz-etc\another-filename.wxyz"
foreach ($file in $files) {
$null = new-item $file
}
Get-ChildItem . -errorAction silentlyContinue -recurse -filter *.wxyz | select-object fullName
foreach ($file in $files) {
remove-item -literalPath $file
}
rmdir "c:\Users\$username\YYY\TODO\2021-12-22_Foo-bar-baz-etc"
rmdir "c:\Users\$username\YYY\TODO"
rmdir "c:\Users\$username\YYY"
When I execute it, the output of the get-childItem ... | select-object pipeline is truncated:
FullName
--------
C:\Users\rny\one-two-three-four.sql.wxyz
C:\Users\rny\YYY\TODO\2021-11_29_abcdefghijklmnop.wxyz
C:\Users\rny\YYY\TODO\2021-12-22_Foo-bar-baz-etc\an...
Note especially the last line. This behaviour was noted elsewhere on SuperUser and the accepted answer is to pipe the output into format-table with -autoSize. So far, so good.
However, If I comment the second file in the assignment of the $files array like so
$files = "C:\Users\$username\one-two-three-four.sql.wxyz",
# "C:\Users\$username\YYY\TODO\2021-11_29_abcdefghijklmnop.wxyz",
"C:\Users\$username\YYY\TODO\2021-12-22_Foo-bar-baz-etc\another-filename.wxyz"
the output is not truncated anymore:
FullName
--------
C:\Users\rny\one-two-three-four.sql.wxyz
C:\Users\rny\YYY\TODO\2021-12-22_Foo-bar-baz-etc\another-filename.wxyz
This puzzles me because the name of the file that was truncated is now fully visible and I have no explanation for this.
So, what exactly causes the truncation of the file in one case and not in the other case?
This isn't so much to do with Select-Object per se - it's more to do with how PowerShell converts values into string representations, and specifically in this case how it does that when it displays uncaptured output from cmdlets on the console.
PowerShell (Windows and Core) has a bunch of preconfigured "views" that define how some built-in types are rendered - e.g. whether they use Format-List or Format-Table, what properties to display, and in the case of tables, how wide to display each column - see about_Format.ps1xml.
For other types, PowerShell tries to make a best-guess on the fly. To do that it waits for the first N items in arrive from the input to make a decision on the formatting rules to apply. I can't find any definitive documentation that says how many items PowerShell waits for, so that might be a good follow-up question :-).
And you can obviously override these defaults by passing formatting parameters for Format-Table and Format-List.
In your case the top-level script has received pipeline output containing an array of PSCustomObject objects (i.e. the output from Select-Object) and it's decided to show them in a table with a column for the FullName property.
Example 1
In your first example, it's looked at the first two PSCustomObject items and decided to make the FullName column 54 characters wide since that's the length of C:\Users\rny\YYY\TODO\2021-11_29_abcdefghijklmnop.wxyz, and the third item gets truncated to that same width (if you include the ...) because it wasn't included in the decision-making process for column widths.
FullName
--------
C:\Users\rny\one-two-three-four.sql.wxyz
C:\Users\rny\YYY\TODO\2021-11_29_abcdefghijklmnop.wxyz
C:\Users\rny\YYY\TODO\2021-12-22_Foo-bar-baz-etc\an...
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^| 54 characters
Example 2
In your second example, PowerShell sees the longest FullName property in the first couple of PSCustomObjects is C:\Users\rny\YYY\TODO\2021-12-22_Foo-bar-baz-etc\another-filename.wxyz and so uses a column width of 70.
FullName
--------
C:\Users\rny\one-two-three-four.sql.wxyz
C:\Users\rny\YYY\TODO\2021-12-22_Foo-bar-baz-etc\another-filename.wxyz
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^| 70 characters
Example 3
Finally, if you do what #notjustme suggests in the comments and add -ExpandProperty FullName onto Select-Object you get an array of string values instead of an array of PSCustomObjects which is why you might see PowerShell apply different formatting rules - you don't get a FullName header for example because the values are strings not objects with properties, and it's using Format-List instead of Format-Table.
C:\Users\rny\one-two-three-four.sql.wxyz
C:\Users\rny\YYY\TODO\2021-12-22_Foo-bar-baz-etc\another-filename.wxyz
To add some background to mcclayton's helpful answer:
Specifically, you're seeing the effects of the infamous 300-msec. delay built into Format-Table formatting, which PowerShell implicitly applies to instances of .NET types that have 4 or fewer properties and do not have explicit formatting data associated with them.
See this answer for details (which is given in the context of a different symptom of the same problem, namely unexpected output ordering), but the short of it is: The delay is used to infer suitable column widths from the specific property values received within the delay period.
This means that objects with property values received after the 300-msec. delay may be truncated in their column display if their values happen to be wider than the widest among the values received during the delay period.
Specifically, your symptom implies that only the first two objects were received within the delay period, and that the longer among the two property values then locked in the column width; when the third object was received later, the column width was already locked in, and the longer value was truncated (indicated with trailing ... in Windows PowerShell (3 . chars.) and … in PowerShell (Core) 7+ (single char))
The only way to avoid truncating is to know the max. column width ahead of time and pass it to an explicit Format-Table call -
notably, this prevents using the output as data. See below.
Here's a simple way to provoke the problem:
Note: The Select-Object calls below aren't strictly needed, but are provided for symmetry with the question.
# Create blocks of two objects with strings of different length in their
# .Prop value: 10 chars. vs. 100 chars.
$count = 10000 # How often to repeat each object in a row.
$objs =
(, [pscustomobject] #{ Prop = ('x' * 10) } * $count) +
(, [pscustomobject] #{ Prop = ('y' * 100) } * $count)
# Depending on the value of $count - which translates into how
# long it takes until the second block of objects starts emitting -
# truncation will occur or not.
$objs | Select-Object Prop
With blocks of 10,000 objects, I do see the truncation: it takes long enough for the first block - with the short property value - to lock in the width of the display column, causing the objects in the second block to be truncated:
Prop
----
xxxxxxxxxx
...
yyyyyyyyy… # <- truncated, because width 10 was locked in during the delay
...
To prevent truncation, pass a calculated property to Format-Table specifying the max. width:
$objs | Select-Object Prop | Format-Table #{ n='Prop'; e='Prop'; width = 100 }
I'm trying this command in Windows 10 PowerShell:
Get-ChildItem -Recurse | select FullName,Length,LastAccessTime
The result only contains FullName , while LastAccessTime and Length is missing from output.
PowerShell ScreenShot
What am I doing wrong?
The problem is merely a display problem:
Because the paths among the output objects are long and FullName is the first property selected, the remaining properties (columns) don't print, because they cannot fit on the screen. However, the properties are present, and can be used programmatically.
Note: If the intent is to save to a file for later programmatic processing, you shouldn't use > / Out-File, which result in the same for-display formatting that you see in the console (terminal), because this formatting is meant only for the human observer.
For programmatic processing, use a structured data format such as CSV, as shown in Ashok Prasath's answer
Workarounds:
A quick-and-dirty workaround is to put the FullName property last, as Doug Maurer advises, which will make all properties (columns) show, though the FullName property values will be truncated (symbolized with …), and notably from the start of the paths:
# FullName is placed *last*
# Caveat: Paths will be truncated at the *end*.
Get-ChildItem -Recurse | select Length, LastAccessTime, FullName
If you don't mind switching to a list-based format, where each property value is represented on its own line prefixed by the property name, pipe to Format-List; note that overly long values will line-wrap:
Get-ChildItem -Recurse | select FullName,Length,LastAccessTime | Format-List
If you do want tabular output and don't mind line-wrapping in your output, you can pipe to Out-String with a -Width value large enough to fit all columns (note that Out-File also supports -Width):
Get-ChildItem -Recurse | select FullName,Length,LastAccessTime |
Out-String -Width 300
If you prefer horizontal scrolling to line wrapping, you could save the above to a file and open in it a text editor or, with a text editor such as Visual Studio Code, you can directly pipe the output to it:
# Opens the output directly in a new Visual Studio Code editor tab.
Get-ChildItem -Recurse | select FullName,Length,LastAccessTime |
Out-String -Width 300 | code -
Otherwise - if you do want to see tabular display in the console, in the exact column order specified and without line-wrapping - your only option is to truncate the FullName values so that all columns can fit; note that, for brevity, I am omitting the select (Select-Object) call in favor of direct use of Format-Table:
Get-ChildItem -Recurse |
Format-Table #{ e='FullName'; w=[Console]::WindowWidth-45; a='right'},
Length, LastAccessTime
Note how the column width (w entry, short for width) is based on the console window's with ([Console]::WindowWidth) minus 45 characters, to show as much of the FullName values as possible while still leaving room for the other two columns; also, to ensure that truncation (symbolized with … is applied to the start of the path - so that file/directory name is always displayed - the column is right-aligned (via the a entry, short for alignment); if you prefer truncation of the end of the path, omit the a entry (which then defaults to left).
For more information on this hashtable-based (#{ ... }) way of specifying so-called calculated properties for Format-Table, including a way to truncate from the start while also maintaining left alignment, see this answer.
This will work:
Get-ChildItem -Recurse | select FullName,Length,LastAccessTime | Export-Csv list.csv
I am trying to write a simple PS script to check large .txt log files for a short string: "SRVE0242I:"
$lines = Select-String -Path $logDir -Pattern "SRVE0242I:" | Select-Object line | Out-String
On output though, it only displays the following:
Line
[28/06/17 13:48:27:839] 00000020 ServletWrappe I SRVE0242I: [User] [User] [com_xxxxxxx_...
And not the full line. Is there a limit to how many characters this pulls? I can't find any info on any restrictions for the Select-String cmdlet. Is there a better way to do this so that I don't a) pull the heading "Line" in my list of lines (Don't really want to create table formatting for such a simple output) and b) get the whole line when I pull the info?
You are seeing it like this because it's displaying the Line property using the default Format-Table view and shortening it to the width of the console.
Do this instead:
$lines = Select-String -Path $logDir -Pattern "SRVE0242I:" | Select-Object -ExpandProperty line
This returns the value of the Line property as a string to the $lines variable. You don't need to use Out-String.
There is! Long story short, Select-Object is doing the truncating here. Here's one way to get the first untruncated line in a Select-String output
$(Select-String -Path $logDir -Pattern "SRVE0242I:")[0].Line
When you run into something like this, you can break down the individual steps to determine what's happening by piping things to Get-Member. Here's what's happening in the code above:
Select-String <# args #> | Get-Member
Select-String gives us a MatchInfo object, which (as you've correctly determined) has a 'Line' property. When run on it's own, Select-String will actually spit out all the information you're looking for, and will not truncate it by default (at least, on v6.0.0-beta). It does give you an array of MatchInfo objects if it finds multiple matches, so you have to index into that array if you just want the first one (like I did above).
Select-String <# args #> | Select-Object Line | Get-Member
Select-Object applies PowerShell's default formatting for objects which, in most cases, will truncate your output for easier viewing. For objects with a bunch of members (like a MatchInfo object), it will try to do one per line by default.
Select-String <# args #> | Select-Object Line | Out-String | Get-Member
Out-String directly translates it's input to a string. That is, rather than trying to cast something to a string or pull a string Property out of an object that's passed to it, it just changes whatever it receives into an object. In this case, it turns the already-formatted MatchInfo output into a string. Nothing happens to the output on the terminal, but Get-Member will reveal a String rather than a MatchInfo object.
It's not directly relevant here, but if you're interested in modifying the default formatting, it's governed by the types.ps1xml file.
I'm trying to figure out why Select-Object
adds a lot of whitespace at the start of its output; and
truncates long properties with ellipsis.
Here's a repro of what I mean. Suppose you run these commands on C:\:
New-Item "MyTest" -Type Directory
cd MyTest
"Some very long lorem ipsum like text going into a certain file, bla bla bla and some more bla." | Out-File test.txt
Get-ChildItem | Select-String "text" | Select-Object LineNumber,Line
This will show output like this:
The ellipsis I can understand, that would be just the way the command ends up getting formatted when the result is written to the console host. However, the whitespace at the start still confuses me in this case.
Things get weirder for me though when I pipe the result to either clip or Out-File output.txt. I get similarly formatted output, with a lot of whitespace at the start and truncated Line properties.
Which command is causing this behavior, and how can I properly solve this? Most importantly: how can I get the full results into a file or onto my clipboard?
The default behavior of outputting the data is to use Format-Table without any modifiers, and the default behavior of Format-Table is to split the viewport into columns of equal width. This makes no assumption on the output width, and is faster in that the cmdlet doesn't need to process any string data from the pipeline prior to output.
To reduce the whitespace, you should use Format-Table -AutoSize as the output method. The -AutoSize switch first measures the widths of data, then outputs with regard to calculated width. If you need to not receive ellipsis and always display the full data set, add -Wrap switch to Format-Table, this way the value will be wrapped into more than a single line, but you can copy it via selecting a square area in Powershell window, just strip newlines off the clipped contents.
Get-ChildItem | Select-String "text" | Select-Object LineNumber,Line | Format-Table -AutoSize -Wrap
I'd say the best way to get the full output into a file would be to export the result as a CSV:
Get-ChildItem |
Select-String "text" |
Select-Object LineNumber,Line |
Export-Csv 'out.csv'
You could also build a string from the selected properties, which might be better for copying the data to the clipboard:
Get-ChildItem |
Select-String "text" |
ForEach-Object { '{0}:{1}' -f $_.LineNumber, $_.Line } |
Tee-Object 'out.txt' | clip
The behavior you observed is caused by the way PowerShell displays output. Basically, it looks at the first object and counts the properties. Objects with less than 5 properties are sent to Format-Table, otherwise to Format-List. The columns of tabular output are spread evenly across the available space. As #Vesper already mentioned you can enforce proportional column width by using the -AutoSize parameter, and wrapping of long lines by using the -Wrap parameter. Format-List wraps long strings by default.
See this blog post from Jeffrey Snover for more information.
I am using a PowerShell script to find all occurrences of a regex expression and output it to file. I have two objectives for this question.
Remove leading white space from a column's value
Specify a width for an extra field (LineNumbers)
This is what I have so far:
gci -recurse -include *.* | Select-String -pattern $regexPattern |`
Format-Table -GroupBy Name -Property Path, Line -AutoSize
This outputs the following:
Path Line
---- ----
C:\myRandomFile.txt This is line 1 and it is random text.
C:\myRandomFile.txt This is line 2 and it has leading white space.
C:\myNextRandomFile.txt This is line 3 and it has leading white space.
C:\myLastRandomFile.txt This is line 4.
This is because the files have leading white spaces (actually indents/tab spaces, but outputted as white spaces). I can't change the original files and remove the leading white space as they are our production files/SQL scripts.
I want to trim the leading white space for the Line column so that the output looks like this:
Path Line
---- ----
C:\myRandomFile.txt This is line 1 and it is random text.
C:\myRandomFile.txt This is line 2 and it has no leading white space.
C:\myNextRandomFile.txt This is line 3 and it has no leading white space.
C:\myLastRandomFile.txt This is line 4 and this is how it should look.
And, if I add the LineNumbers column by using
-property LineNumbers
then the LineNumbers column take up about half the space in the row. Can I specify the width of the LineNumbers? I've tried the -AutoSize flag, but this doesn't seem to work well. I've tried
LineNumber;width=50
LineNumber width=50
LineNumber -width 50
and all variations of this, but I get errors to the likes of "Format-Table: A parameter cannot be found that matches parameter name width=50"
I can't test it right now, but I think this should do the trick or at least get you going in the right direction:
gci -recurse -include *.* | Select-String -pattern $regexPattern |`
Format-Table Path, #{Name='Line'; Expression={$_.Line -replace '^\s+', ''}; Width=50}
You can use the TrimStart() method to remove leading spaces. There's also TrimEnd() to remove characters from the end, or Trim() to remove characters from both sides of the string.
I would not use Format-Table for output to a file.
I'd rather use Export-Csv
gci -recurse -include *.* | Select-String -pattern $regexPattern |`
select-object linenumber, path, line | Export-Csv c:\mycsv.csv -Delimiter "`t"
If you still want to use Format-Table I reccomend reading this aricle
http://www.computerperformance.co.uk/powershell/powershell_-f_format.htm
Quote:
"{0,28} {1, 20} {2,-8}" -f ` creates:
A column for the 1st item of 28 characters, right-aligned and adds a
space A column for the 2nd item of 20 characters right-aligned and
adds a space A column for the 3rd item of 8 characters left-aligned.
In case someone from this decade comes looking here, there is an alternative using Trim():
gci -recurse -include *.* | Select-String -pattern $regexPattern |`
Format-Table Path, #{Name='Line'; Expression={$_.Line.Trim()}; Width=50}