I try to get PowerShell to spit out a Semantic Versioning Variable but get stuck in it displaying just the command I entered (doing that in the ISE) or either of 2 errors ('missing argument' or 'doesn't accept piped input'), which, if I try to resolve them, end in the command simply being displayed again.
How do I get this:
(Invoke-WebRequest -Uri http://someplace).Links.href | Out-String -Stream |
Select-String -Pattern [regex]$someGoodRegex -OutVariable $NodeVersion_target
assuming the regex and web request point to solid things to simply stick the searched term in the -OutVariable defined?
On a more general note, is there a good way to display the properties of the objects in the pipe? After a great deal of digging, I stumbled upon {$_} but can't get it to display anything but the command again if the command gets a little more complex that just a simple cmdlet.
Remove the [regex]. Select-String already treats the argument to the parameter -Pattern as a regular expression.
Remove the $ from the variable name. You need it to use a variable directly, but the -OutVariable parameter expects the bare variable name without the leading $.
You can also remove the Out-String -Stream.
Something like this should work:
$uri = 'http://www.example.com/'
$re = 'v\d+\.\d+\.\d+/s'
(Invoke-WebRequest -Uri $uri).Links.href |
Select-String -Pattern $re -OutVariable NodeVersion_target
Alternatively you can assign the output of the pipeline to a variable instead of using -OutVariable:
$uri = 'http://www.example.com/'
$re = 'v\d+\.\d+\.\d+/s'
$NodeVersion_target = (Invoke-WebRequest -Uri $uri).Links.href |
Select-String -Pattern $re
The latter is actually more PoSh.
As for inspecting the current object in a pipeline: pipe into Get-Member to get a list of the properties/methods of the pipelined objects, and pipe into Format-List * to get a list of the values of the pipelined objects.
Related
I am doing an API request which returns a bunch of data. In attempted to search through it with Select-String, it just spits out the entire value stored in the variable. This is an internet server which I am calling an api.
$return = Invoke-RestMethod -Method GET -Uri $uri -Headers #{"authorization" = $token} -ContentType "application/json"
$file = $return.data
$file | Out-String -Stream | Select-String -Pattern "word"
this returns the entire value of $file. printing $file looks like same as the pipe output. Why is this not working?
$file.Gettype says it is a system.object, another answer said to use Out-String, but something is not working.
$file.Gettype
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True String System.Object
To complement iRon7's helpful answer with the precise logic of Out-String's -Stream switch, as of PowerShell 7.1:
Out-String, like the other Out-* cmdlets such as Out-File, uses PowerShell's rich output-formatting system to generate human-friendly representations of its input objects.
Without -Stream, Out-String only ever produces a single, (typically) multiline string.
With -Stream, line-by-line output behavior typically occurs - except for input objects that happen to be multiline strings, which are output as-is.
Because this exception is both obscure and unhelpful, GitHub proposal #14638 suggests removing it.
For so-called in-band data types, -Stream works as follows, which truly results in line-by-line output:
Input objects are formatted by PowerShell's rich formatting system, and the lines that make up the resulting representation are then output one by one.
Out-of-band data types are individually formatted outside of the formatting system, by simply calling their .NET .ToString() method.
In short: data types that represent a single value are out-of-band, and in addition to [string] out-of-band data types also comprise [char] and the various (standard) numeric types, such as [int], [long], [double], ...
[string] is the only out-of-band type that itself can result in a multiline representation, because calling .ToString() on a string is effective no-op that returns the string itself - whether it is single- or multiline.
Therefore:
Any string - notably also a multiline string - is output as-is, as a whole, and splitting it into individual lines requires an explicit operation; e.g. (note that regex \r?\n matches both Windows-style CRLF and Unix-style LF-only newlines):
"line 1`nline 2`nline 3" -split '\r?\n' # -> 'line 1', 'line 2', 'line 3'
If your input objects are a mix of in-band objects and (invariably out-of-band) multiline strings, you can combine Out-String -Stream with -split; e.g.:
((Get-Date), "line 1`nline 2`nline 3" | Out-String -Stream) -split '\r?\n'
On closer inspection, I suspect that your issue comes from an ambiguity in the Out-String documentation:
-Stream
Indicates that the cmdlet sends a separate string for each line of an
input object. By default, the strings for each object are accumulated
and sent as a single string.
Where the word line should be read as item.
To split you raw string into separate lines, you will need to split your string using the following command:
$Lines = $return.data -split [Environment]::NewLine
Note that this assumes that your data uses the same characters for a new line as the system you working on. If this is not the case, you might want to split the lines using an regular expression, e.g.:
$Lines = $return.data -split "`r*`n"
So what does the-Stream parameter do?
It sends a separate string for each item of an input object.
Where in this definition, it is also a known common best PowerShell practice to use a singular name for possible plural input objectS.
Meaning if you use the above defined $Lines variable (or something like $Lines = Get-Content .\File.json), the input object "$Lines" is a collection of strings:
$Lines.GetType().Name
String[]
if you stream this to Out-String it will (by default) join all the items and return a single string:
($Lines | Out-String).GetType().Name
String
In comparison, if you use the -Stream parameter, it will pass each separated item from the $Lines collection directly to the next cmdlet:
($Lines | Out-String -Stream).GetType().Name
Object[]
I have created a document issue for this: #7133 "line" should be "item"
Note:
In general, it is a bad practice to peek and poke directly into a
serialized string
(including Json) using string
methods and/or cmdlets (like Select-String). Instead you should use
the related parser (e.g.
ConvertFrom-Json)
for searching and replacing which will result in an easier syntax
and usually takes care of known issues and pitfalls.
Select-String outputs Microsoft.PowerShell.Commands.MatchInfo objects. It seems to me that the output is somehow fancified via the PS engine or something to highlight your match, but ultimately it does print the entire matched string.
You should check out the members of the object Select-String provides, like this:
$file | Out-String -Stream | Select-String -Pattern "word" | Get-Member
TypeName: Microsoft.PowerShell.Commands.MatchInfo
Name MemberType Definition
---- ---------- ----------
...
Matches Property System.Text.RegularExpressions.Match[] Matches {get;set;}
...
What you're interested in is the Matches property. It contains a bunch of information about the match. To extract exactly what you want, look at the Value property of Matches:
($file | Out-String -Stream | Select-String -Pattern "word").Matches.Value
word
Another way:
$file | Out-String -Stream | Select-String -Pattern "word" | ForEach-Object {$_.Matches} | Select-Object -Property Value
Value
-----
word
Or
$file | Out-String -Stream | Select-String -Pattern "word" | ForEach-Object {$_.Matches} | Select-Object -ExpandProperty Value
word
I am trying to write a simple PS script to check large .txt log files for a short string: "SRVE0242I:"
$lines = Select-String -Path $logDir -Pattern "SRVE0242I:" | Select-Object line | Out-String
On output though, it only displays the following:
Line
[28/06/17 13:48:27:839] 00000020 ServletWrappe I SRVE0242I: [User] [User] [com_xxxxxxx_...
And not the full line. Is there a limit to how many characters this pulls? I can't find any info on any restrictions for the Select-String cmdlet. Is there a better way to do this so that I don't a) pull the heading "Line" in my list of lines (Don't really want to create table formatting for such a simple output) and b) get the whole line when I pull the info?
You are seeing it like this because it's displaying the Line property using the default Format-Table view and shortening it to the width of the console.
Do this instead:
$lines = Select-String -Path $logDir -Pattern "SRVE0242I:" | Select-Object -ExpandProperty line
This returns the value of the Line property as a string to the $lines variable. You don't need to use Out-String.
There is! Long story short, Select-Object is doing the truncating here. Here's one way to get the first untruncated line in a Select-String output
$(Select-String -Path $logDir -Pattern "SRVE0242I:")[0].Line
When you run into something like this, you can break down the individual steps to determine what's happening by piping things to Get-Member. Here's what's happening in the code above:
Select-String <# args #> | Get-Member
Select-String gives us a MatchInfo object, which (as you've correctly determined) has a 'Line' property. When run on it's own, Select-String will actually spit out all the information you're looking for, and will not truncate it by default (at least, on v6.0.0-beta). It does give you an array of MatchInfo objects if it finds multiple matches, so you have to index into that array if you just want the first one (like I did above).
Select-String <# args #> | Select-Object Line | Get-Member
Select-Object applies PowerShell's default formatting for objects which, in most cases, will truncate your output for easier viewing. For objects with a bunch of members (like a MatchInfo object), it will try to do one per line by default.
Select-String <# args #> | Select-Object Line | Out-String | Get-Member
Out-String directly translates it's input to a string. That is, rather than trying to cast something to a string or pull a string Property out of an object that's passed to it, it just changes whatever it receives into an object. In this case, it turns the already-formatted MatchInfo output into a string. Nothing happens to the output on the terminal, but Get-Member will reveal a String rather than a MatchInfo object.
It's not directly relevant here, but if you're interested in modifying the default formatting, it's governed by the types.ps1xml file.
As a C# developer, I'm still learning the basics of PowerShell and often getting confused.
Why does the $_. give me the intellisense list of vaild property names in the first example, but not the second?
Get-Service | Where {$_.name -Match "host" }
Get-Service | Write-Host $_.name
What is the basic difference in these two examples?
When I run the second one, it gives this error on each iteration of Get-Service:
Write-Host : The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the input and its properties do not match any of the parameters
that take pipeline input.
At line:3 char:15
+ Get-Service | Write-Host $_.name
+ ~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (wuauserv:PSObject) [Write-Host], ParameterBindingException
+ FullyQualifiedErrorId : InputObjectNotBound,Microsoft.PowerShell.Commands.WriteHostCommand
My coworker and I were first using a foreach loop to iterate a Get-commandlet, and we were stuck getting the property names to appear. We tried to simplify till we go to the core basics above.
Just figured out sometimes it's a typo in the commandlet, below first one fails because the commandlet should be Get-Service instead of Get-Services.
foreach ($item in Get-Services)
{
Write-Host $item.xxx #why no intellisense here?
}
foreach ($item in Get-Service)
{
Write-Host $item.Name
}
First part: you can't use $_ like that, it represents current pipeline object only within script blocks. These script blocks are usually used with any *-Object cmdlet (but there are other use cases too). Not all parameters/ cmdlet support it. Write-Host is one of those that don't.
Second: It looks like you are using own function (GetServices). PowerShell v3 intellisense is depending on command metadata (OutputType). If any cmdlet/ function produces object but is silent about OutputType, intellisense won't work. It's pretty simple to get it, and you can lie and still get proper intellisense for any existing type:
function Get-ServiceEx {
[OutputType('System.ServiceProcess.ServiceController')]
param ()
'This is not really a service!'
}
(Get-ServiceEx).<list of properties of ServiceController object>
You can read more about it on my blog.
Intellisense will work if you put $_ inside a scriptblock.
The following will trigger intellisense:
Get-Service | Foreach-Object {$_.Name} # Intellisense works
Get-Service | Where-Object {$_.Name} # Intellisense works
Get-Service | Write-Host {$_.Name} # Intellisense works
Note that your command need not be a valid command: the third example will not work but intellisense will display auto-complete for $_ anyway because it is inside a scriptblock.
I suspect it is because $_ is only usable inside a scriptblock (e.g. switch, %, ?) but I have no hard-evidence for it.
$_ is the current object in the pipeline. It's the same as $item in a foreach ($item in $array).
Get-Service | Where {$_.name -Match "host" }
Get-Service | Write-Host $_.name
The difference between these two lines is a fundamental part of the PowerShell design. Pipelines are supposed to be easy. You should be able to ex. search for and delete files as simply as:
Get-ChildItem *.txt | Remove-Item
This is clean, simple, and everyone can use it. The cmdlets are built to identify the type of the incoming object, adapt to support the specific type, and process the object.
However, sometimes you need more advanced object manipulation in the pipeline, and that's where Where-Object and Foreach-Object comes in. Both cmdlets lets you write your own processing logic without having to create an function or cmdlet first. To be able to access the object in your code(processing logic), you need an identifier for it, and that is $_. $_ is also supported in some other special cmdlets, but it's not used in most cmdlets(including Write-Host).
Also, Intellisense in foreach ($item in Get-Service) works. You had a typo.
I'm trying to do some processing logic - running some commands in parallel based on the tree configuration CSV file:
Operation;Parent;Enabled;Propagated;Job_ID;Status;Started;Finished
CA1;n/a;Y;N;;;;
PROD1;n/a;Y;N;;;Y;
CON1;CA1;N;N;;;Y;
CON2;CON1;N;N;;;Y;
I load the file into the variable and then I'm trying to find the next step which needs to be processed:
$Data = Import-Csv -delimiter ";" .\config.csv
$NextStep = $Data | Select-Object -first 1 | Where-Object {$_.Started -eq ""}
$NextStepText = $NextStep.Operation | ft -autosize | out-string
The problem is that it seems like $NextStep.Operation contains new line character. When I display it I get:
PS C:\temp\SalesForce> $NextStep.operation
CA1
PS C:\temp\SalesForce> $NextStep.Operation.Contains("`n")
False
Do you know what I'm doing wrong? I would like to display the content without the "dummy" new line character which is there even if contains method is saying it is not there.
Or please advise how to do it better. I'm still learning PowerShell; so far I just google the commands, and I'm trying to put it together.
The newline isn't in your data, it's being added by Out-String. Observe the output of the following (in particular, where you do and don't get the newline after CA1):
$Data = import-csv -delimiter ";" .\config.csv
$NextStep = $Data | select-object -first 1 | where-object {$_.Started -eq ""}
$NextStepText = $NextStep.Operation | ft -autosize | out-string
"hi"
$NextStepText
"hi"
$NextStep.Operation;
"hi"
$NextStep.Operation | ft -autosize
"hi"
You shouldn't be using Format-Table at that step (and Out-String is unnecessary in this script) if you intend to use $NextStepText for anything other than direct output later on. Consider Format-Table (or any of the Format-* cmdlets) the end of the line for usable data.
Why do you think that there is a new line character of some sort in there? If you are using the ISE then what you posted doesn't look like there is. It is normal to have a blank line between commands (in the v2/v3 ISE, not sure about v4), so what you posted would not indicate that it contains any new line characters.
You can always check the $NextStep.Operation.Length to see if it says 3 or 4. If there is a `n in there it'll show up in the length. For example (copied and pasted out of my v3 PS ISE):
PS C:\> $test = "Test`nTest2"
PS C:\> $test
Test
Test2
PS C:\> $test.Length
10
PS C:\>
That was to show that there is a new line character injected by following it with text, without any text following the new line character it looks like this:
PS C:\> $test = "Test`n"
PS C:\> $test
Test
PS C:\> $test.Length
5
PS C:\>
You'll notice that there are 2 blank lines after the text "Test" on the second command. The first is the line injected into the variable, and the second is the obligatory line that PS puts in to show separation between commands.
Out-String unexpectedly appends a trailing newline to the string it outputs.
This problematic behavior is discussed in GitHub issue #14444.
A simple demonstration:
# -> '42<newline>'
(42 | Out-String) -replace '\r?\n', '<newline>'
However, you neither need Format-Table nor Out-String in your code:
Format-* cmdlets output objects whose sole purpose is to provide formatting instructions to PowerShell's for-display output-formatting system. In short: only ever use Format-* cmdlets to format data for display, never for subsequent programmatic processing - see this answer for more information.
Out-String is capable of interpreting these formatting instructions, i.e. it does produce data - in the form of a single, multi-line string by default - that is the string representation of what would print to the display.
As such, the resulting string contains a representation for the human observer, not a structured text format suitable for programmatic processing.
In your case, Format-Table is applied to a string, which is pointless, because strings always render as themselves, in full (-AutoSize has no effect); piping to Out-String then in effect returns the original string with an (undesired) newline appended.
Therefore, use a simple variable assignment to store the property value of interest in a separate variable:
$NextStepText = $NextStep.Operation
I do a lot of regex matching using Select-String in Powershell.
For example, the simplest and maybe the most common match, an IPv4 address:
$regex = \b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b
Now, if I was to match this in a line that said:
$output = `
"Blah blah blah, 202.100.100.9, you're going to match the IP in the middle of this line,
but not on this line, because '2.a.3.one' is not a valid IPv4 address"
and then I do:
$output | Select-String $regex
It will give me the entire line.
I can't really use that in it's raw form.
However if I use this:
$output | Select-String $regex | %{$_.Matches} | %{$_.Value}
It will give me JUST the IP address, which is great.
My question is:
Is there a simpler way to do this?
I'd rather not type out | %{$.Matches} | %{$.Value} every time I want to grab just one particular string.
If you prefer Select-String over the -replace operator (which is just syntactic sugar for calling [Regex]::Replace), PowerShell V3 has a few shortcuts that can save some typing.
First, there is an alias for Select-String => sls.
Second, with simple foreach-object script blocks, you can replace the script block with just the property.
Combining these, you can use
$output | sls $regex | % Matches | % Value
To save even more typing, PowerShell can tab complete Matches but not Value.
Another option that is even less typing is to use property syntax:
($output | sls $regex).Matches.Value
As a bonus, tab completion can complete both Matches and Value in this example. Note that this second example works in V2 but only if there is a single matching line. If there are multiple matching lines, only in V3 will you see all the results, V2 would show nothing or an error if strict mode is enabled.
Try this:
$output -replace ".*($regex).*",'$1'
First I want to thank everyone for their help and their efforts in trying to figure this out for me. All of your suggestions have been good, and useful.
I had a think, and I realized that I am looking for a command that doesn't really exist, so I just wrote a function for it instead:
function Regex-Match {
[cmdletbinding()]
param (
[parameter(Position=0,Mandatory=$true,ValueFromPipeline=$false)]$RegexString,
[parameter(Position=1,Mandatory=$true,ValueFromPipeline=$true)]$SearchString
)
$Results = #($SearchString | Select-String -Pattern $RegexString | % {$_.Matches} | % {$_.Value})
Return $Results
}
To further shorten this, I can use
New-Alias -Name regx -Value Regex-match
Example command:
"test string" | regx "\w\s\w"
or
Regex-Match -SearchString (gc .\Stuff.txt) -RegexString "\w+"