Shorter Syntax for grabbing a string? [Select-String] - powershell

I do a lot of regex matching using Select-String in Powershell.
For example, the simplest and maybe the most common match, an IPv4 address:
$regex = \b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b
Now, if I was to match this in a line that said:
$output = `
"Blah blah blah, 202.100.100.9, you're going to match the IP in the middle of this line,
but not on this line, because '2.a.3.one' is not a valid IPv4 address"
and then I do:
$output | Select-String $regex
It will give me the entire line.
I can't really use that in it's raw form.
However if I use this:
$output | Select-String $regex | %{$_.Matches} | %{$_.Value}
It will give me JUST the IP address, which is great.
My question is:
Is there a simpler way to do this?
I'd rather not type out | %{$.Matches} | %{$.Value} every time I want to grab just one particular string.

If you prefer Select-String over the -replace operator (which is just syntactic sugar for calling [Regex]::Replace), PowerShell V3 has a few shortcuts that can save some typing.
First, there is an alias for Select-String => sls.
Second, with simple foreach-object script blocks, you can replace the script block with just the property.
Combining these, you can use
$output | sls $regex | % Matches | % Value
To save even more typing, PowerShell can tab complete Matches but not Value.
Another option that is even less typing is to use property syntax:
($output | sls $regex).Matches.Value
As a bonus, tab completion can complete both Matches and Value in this example. Note that this second example works in V2 but only if there is a single matching line. If there are multiple matching lines, only in V3 will you see all the results, V2 would show nothing or an error if strict mode is enabled.

Try this:
$output -replace ".*($regex).*",'$1'

First I want to thank everyone for their help and their efforts in trying to figure this out for me. All of your suggestions have been good, and useful.
I had a think, and I realized that I am looking for a command that doesn't really exist, so I just wrote a function for it instead:
function Regex-Match {
[cmdletbinding()]
param (
[parameter(Position=0,Mandatory=$true,ValueFromPipeline=$false)]$RegexString,
[parameter(Position=1,Mandatory=$true,ValueFromPipeline=$true)]$SearchString
)
$Results = #($SearchString | Select-String -Pattern $RegexString | % {$_.Matches} | % {$_.Value})
Return $Results
}
To further shorten this, I can use
New-Alias -Name regx -Value Regex-match
Example command:
"test string" | regx "\w\s\w"
or
Regex-Match -SearchString (gc .\Stuff.txt) -RegexString "\w+"

Related

Powershell, how to capture argument(s) of Select-String and include with matched output

Thanks to #mklement0 for the help with getting this far with answer given in Powershell search directory for code files with text matching input a txt file.
The below Powershell works well for finding the occurrences of a long list of database field names in a source code folder.
$inputFile = 'C:\DataColumnsNames.txt'
$outputFile = 'C:\DataColumnsUsages.txt'
Get-ChildItem C:\ProjectFolder -Filter *.cs -Recurse -Force -ea SilentlyContinue |
Select-String -Pattern (Get-Content $inputFile) |
Select-Object Path, LineNumber, line |
Export-csv $outputfile
However, many lines of source code have multiple matches, especially ADO.NET SQL statements with a lot of field names on one line. If the field name argument was included with the matching output the results will be more directly useful with less additional massaging such as lining up everything with the original field name list. For example if there is a source line "BatchId = NewId" it will match field name list item "BatchId". Is there an easy way to include in the output both "BatchId" and "BatchId = NewId"?
Played with the matches object but it doesn't seem to have the information. Also tried Pipeline variable like here but X is null.
$inputFile = 'C:\DataColumnsNames.txt'
$outputFile = 'C:\DataColumnsUsages.txt'
Get-ChildItem C:\ProjectFolder -Filter *.cs -Recurse -Force -ea SilentlyContinue |
Select-String -Pattern (Get-Content $inputFile -PipelineVariable x) |
Select-Object $x, Path, LineNumber, line |
Export-csv $outputile
Thanks.
The Microsoft.PowerShell.Commands.MatchInfo instances that Select-String outputs have a Pattern property that reflects the specific pattern among the (potential) array of patterns passed to -Pattern that matched on a given line.
The caveat is that if multiple patterns match, .Pattern only reports the pattern among those that matched that is listed first among them in the -Pattern argument.
Here's a simple example, using an array of strings to simulate lines from files as input:
'A fool and',
'his barn',
'are soon parted.',
'foo and bar on the same line' |
Select-String -Pattern ('bar', 'foo') |
Select-Object Line, LineNumber, Pattern
The above yields:
Line LineNumber Pattern
---- ---------- -------
A fool and 1 foo
his barn 2 bar
foo and bar on the same line 4 bar
Note how 'bar' is listed as the Pattern value for the last line, even though 'foo' appeared first in the input line, because 'bar' comes before 'foo' in the pattern array.
To reflect the actual pattern that appears first on the input line in a Pattern property, more work is needed:
Formulate your array of patterns as a single regex using alternation (|), wrapped as a whole in a capture group ((...)) - e.g., '(bar|foo)')
Note: The expression used below, '({0})' -f ('bar', 'foo' -join '|'), constructs this regex dynamically, from an array (the array literal 'bar', 'foo' here, but you can substitute any array variable or even (Get-Content $inputFile)); if you want to treat the input patterns as literals and they happen to contain regex metacharacters (such as .), you'll need to escape them with [regex]::Escape() first.
Use a calculated property to define a custom Pattern property that reports the capture group's value, which is the first among the values encountered on each input line:
'A fool and',
'his barn',
'are soon parted.',
'foo and bar on the same line' |
Select-String -AllMatches -Pattern ('({0})' -f ('bar', 'foo' -join '|')) |
Select-Object Line, LineNumber,
#{ n='Pattern'; e={ $_.Matches[0].Groups[1].Value } }
This yields (abbreviated to show only the last match):
Line LineNumber Pattern
---- ---------- -------
...
foo and bar on the same line 4 foo
Now, 'foo' is properly reported as the matching pattern.
To report all patterns found on each line:
Switch -AllMatches is required to tell Select-String to find all matches on each line, represented in the .Matches collection of the MatchInfo output objects.
The .Matches collection must then be enumerated (via the .ForEach() collection method) to extract the capture-group value from each match.
'A fool and',
'his barn',
'are soon parted.',
'foo and bar on the same line' |
Select-String -AllMatches -Pattern ('({0})' -f ('bar', 'foo' -join '|')) |
Select-Object Line, LineNumber,
#{ n='Pattern'; e={ $_.Matches.ForEach({ $_.Groups[1].Value }) } }
This yields (abbreviated to show only the last match):
Line LineNumber Pattern
---- ---------- -------
...
foo and bar on the same line 4 {foo, bar}
Note how both 'foo' and 'bar' are now reported in Pattern, in the order encountered on the line.
The solid information and examples from #mklement0 were enough to point me in the right direction for researching and understanding more about Powershell and the object pipeline and calculated properties.
I was able to finally achieve my goals of a cross referencing a list of table and field names to the C# code base.The input file is simply table and field names, pipe delimited. (one of the glitches I had was not using pipe in the split, it was a visual error that took awhile to finally see, so check for that). The output is the table name, field name, code file name, line number and actual line. It's not perfect but much better than manual effort for a few hundred fields! And now there are possibilities for further automation in the data mapping and conversion project. Thought about using C# utility programming but that might have taken just as long to figure out and implement and much more cumbersome that a working Powershell.
The key for me at this point is "working"! My first deeper dive into the abstruse world of Powershell. The key points of my solution are the use of the calculated property to get the table and field names in the output, realization that expressions can be used in certain places like to build a Pattern and that the pipeline is passing only certain specific objects after each command (maybe that is too restricted of a view but it's better than what I had before).
Hope this helps someone in future. I could not find any examples close enough to get over the hump and so asked my first ever stackoverflow questions.
$inputFile = "C:\input.txt"
$outputFile = "C:\output.csv"
$results = Get-Content $inputfile
foreach ($i in $results) {
Get-ChildItem -Path "C:\ProjectFolder" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force |
Select-String -Pattern $i.Split('|')[1] |
Select-Object #{ n='Pattern'; e={ $i.Split('|')[0], $i.Split('|')[1] -join '|'} }, Filename, LineNumber, line |
Export-Csv $outputFile -Append}

PowerShell - Condensing a line property into into from the pipe

I am new to PowerShell, and I have the following example code and output to illustrate my problem:
select-string "$env:appdata\..\Local\test\*.ini" -pattern "example_adjustment=" | select filename, line | sort-object -property line -Descending >> file.txt
Filename Line
-------- ----
test1.ini example_adjustment="4.2"
test4.ini example_adjustment="11.0000000"
test2.ini example_adjustment="1.20"
test5.ini example_adjustment="0.90"
test3.ini example_adjustment="0.90"
I want to be able to modify the output so that the "Line" values appear as their numbers only and in float format for the purpose of the sort performing correctly. The end result is I'd be appending that information to a text file.
How would I go about modifying the Line property? I saw a post about regex, but I cannot edit directly from the pipe using regex it seems.
I cannot edit directly from the pipe using regex it seems.
You most certainly can! :-)
Use the -replace regex operator inside a calculated property:
... |Select filename,#{Name='Line';Expression={$_.Line -replace 'example_adjustment="([^"]*)"','$1'}}

Extract specific data

Please help. I am trying to extract multiple filenames from the following .xml file. I then need to copy the list of files from one folder to another. A part of the XML I have posted below:
<component>
<altname>HP Broadcom Online Firmware Upgrade Utility for VMware 5.x</altname>
<filename>CP021404.scexe</filename>
<name>HP Broadcom Online Firmware Upgrade Utility for VMware 5.x</name>
<description>This package contains vSphere 5.1 and VMware </description>
<component>
<component>
<altname>Online ROM Flash - Power Management Controller </altname>
<filename>CP021615.scexe</filename>
I used Windows PowerShell as below and got the output, but the output contains filenames (CP021404.scexe, CP021614.scexe below), line# and symbol still in it. What am I doing wrong on my first PS attempt?
PowerShell
$input_path = ‘C:\PowerShell\hpsum_inventory.xml’
$output_file = ‘C:\powershell\hpsum_inventory-o.xml’
$regex = ".exe"
select-string -Path $input_path -Pattern $regex -AllMatches > $output_file
Output
PowerShell\hpsum_inventory.xml:8: <filename>CP021404.scexe</filename>
PowerShell\hpsum_inventory.xml:18: <filename>CP021614.scexe</filename>
The problem is that you're using a RegEx match and the period character in RegEx matches any character except Line Feed/New Line characters, so it's matching any character followed by 'exe'. Really what you want to do is read the file as XML, and just output the <filename> nodes.
$input_path = ‘C:\PowerShell\hpsum_inventory.xml’
$output_file = ‘C:\powershell\hpsum_inventory-o.xml’
$regex = "exe$"
(Select-Xml -Path $input_path -XPath //filename).node.InnerText | ?{$_ -match $regex} | out-file $output_file
Edit: Ok, you need to incorporate that into a string, that's easy enough. We'll add a ForEach loop (I use the alias % for that) to the last line to insert the file name into a string.
(Select-Xml -Path $input_path -XPath //filename).node.InnerText | ?{$_ -match $regex} | %{"copy c:\powershell\$_ x:\firmware\"} | out-file $output_file
Edit2: Ok, so you want the knowledge in general of how to match text in a file. Can do! Select string will do what you want actually, it just wasn't the best method in general for the example you gave earlier. This gets a bit more interesting, since you need to be familiar with RegEx matching patterns, but other than that it's fairly straight forward. You want to use the -Pattern match again, but let me suggest a better pattern:
"filename>(.*?)<"
That looks for the filename tag, including closing > on it, and grabs everything up to the next < character. The () denote a capturing group, so the rest is ignored as far as the capture goes. Then we pipe to a ForEach loop, and for each line that it finds that matches we select the Matches property, and the second Group property of that (the first contains the whole text, including the filename> and < bits). So it looks like this:
$input_path = 'C:\PowerShell\hpsum_inventory.xml'
$output_file = 'C:\powershell\hpsum_inventory-o.xml'
$regex = "filename>(.*?)<"
select-string -Path $input_path -Pattern "filename>(.*?)<"|%{$_.matches.groups[1].value}
Now that only gets the file names. If we want to incorporate the rest of your thing about inserting it into text you enclose the part in the ForEach loop inside a sub-expression $() and then put that into your double quoted string like such:
select-string -Path $input_path -Pattern "filename>(.*?)<"|%{"copy c:\powershell\$($_.matches.groups[1].value) x:\firmware"}|Out-File $output_file
Personally I would suggest not doing that directly as it limits you. I'd collect the data in an array, then pipe that array into a process that does what you want, but then at least you have the collection so you can do with it what you want.
$input_path = 'C:\PowerShell\hpsum_inventory.xml'
$output_file = 'C:\powershell\hpsum_inventory-o.xml'
$regex = "filename>(.*?)<"
$Filenames = select-string -Path $input_path -Pattern "filename>(.*?)<"|%{$_.matches.groups[1].value}
$Filenames|%{"copy c:\powershell\$_ x:\firmware"}|Out-File $output_file
Why do it that way? What if you don't want to over-write something? Then you can do something like:
$Filenames|?{$_ -notin (GCI X:\firmware -file|select -expand name)}|%{"copy c:\powershell\$_ x:\firmware"}|Out-File $output_file
For your collection of serial numbers, try the regex pattern of:
"Serial Number: (\S*)"
In RegEx there are a few escaped characters that have special meaning, and capitalizing them inverts that meaning. \s means whitespace, so spaces, tabs, what not. Doing it as a capital means something that is NOT whitespace. The asterisk means however many of the previous thing (not whitespace) it can find. So this looks for 'Serial Number: ' and then captures everything after that until it reaches the end of the line or encounters whitespace. Check out this link to see how it works.

How do I remove newline from a PowerShell variable

I'm trying to do some processing logic - running some commands in parallel based on the tree configuration CSV file:
Operation;Parent;Enabled;Propagated;Job_ID;Status;Started;Finished
CA1;n/a;Y;N;;;;
PROD1;n/a;Y;N;;;Y;
CON1;CA1;N;N;;;Y;
CON2;CON1;N;N;;;Y;
I load the file into the variable and then I'm trying to find the next step which needs to be processed:
$Data = Import-Csv -delimiter ";" .\config.csv
$NextStep = $Data | Select-Object -first 1 | Where-Object {$_.Started -eq ""}
$NextStepText = $NextStep.Operation | ft -autosize | out-string
The problem is that it seems like $NextStep.Operation contains new line character. When I display it I get:
PS C:\temp\SalesForce> $NextStep.operation
CA1
PS C:\temp\SalesForce> $NextStep.Operation.Contains("`n")
False
Do you know what I'm doing wrong? I would like to display the content without the "dummy" new line character which is there even if contains method is saying it is not there.
Or please advise how to do it better. I'm still learning PowerShell; so far I just google the commands, and I'm trying to put it together.
The newline isn't in your data, it's being added by Out-String. Observe the output of the following (in particular, where you do and don't get the newline after CA1):
$Data = import-csv -delimiter ";" .\config.csv
$NextStep = $Data | select-object -first 1 | where-object {$_.Started -eq ""}
$NextStepText = $NextStep.Operation | ft -autosize | out-string
"hi"
$NextStepText
"hi"
$NextStep.Operation;
"hi"
$NextStep.Operation | ft -autosize
"hi"
You shouldn't be using Format-Table at that step (and Out-String is unnecessary in this script) if you intend to use $NextStepText for anything other than direct output later on. Consider Format-Table (or any of the Format-* cmdlets) the end of the line for usable data.
Why do you think that there is a new line character of some sort in there? If you are using the ISE then what you posted doesn't look like there is. It is normal to have a blank line between commands (in the v2/v3 ISE, not sure about v4), so what you posted would not indicate that it contains any new line characters.
You can always check the $NextStep.Operation.Length to see if it says 3 or 4. If there is a `n in there it'll show up in the length. For example (copied and pasted out of my v3 PS ISE):
PS C:\> $test = "Test`nTest2"
PS C:\> $test
Test
Test2
PS C:\> $test.Length
10
PS C:\>
That was to show that there is a new line character injected by following it with text, without any text following the new line character it looks like this:
PS C:\> $test = "Test`n"
PS C:\> $test
Test
PS C:\> $test.Length
5
PS C:\>
You'll notice that there are 2 blank lines after the text "Test" on the second command. The first is the line injected into the variable, and the second is the obligatory line that PS puts in to show separation between commands.
Out-String unexpectedly appends a trailing newline to the string it outputs.
This problematic behavior is discussed in GitHub issue #14444.
A simple demonstration:
# -> '42<newline>'
(42 | Out-String) -replace '\r?\n', '<newline>'
However, you neither need Format-Table nor Out-String in your code:
Format-* cmdlets output objects whose sole purpose is to provide formatting instructions to PowerShell's for-display output-formatting system. In short: only ever use Format-* cmdlets to format data for display, never for subsequent programmatic processing - see this answer for more information.
Out-String is capable of interpreting these formatting instructions, i.e. it does produce data - in the form of a single, multi-line string by default - that is the string representation of what would print to the display.
As such, the resulting string contains a representation for the human observer, not a structured text format suitable for programmatic processing.
In your case, Format-Table is applied to a string, which is pointless, because strings always render as themselves, in full (-AutoSize has no effect); piping to Out-String then in effect returns the original string with an (undesired) newline appended.
Therefore, use a simple variable assignment to store the property value of interest in a separate variable:
$NextStepText = $NextStep.Operation

Filter the output of a command as if it was text

I have a simple question, but I am also a beginner in PowerShell. I think it has to do with the fact that the output of the Get-Process command (alias ps) is objects and not text.
I want to get a list of the services running that have the name "sql" in them.
This is what I tried so far, but every attempt returns nothing:
Get-Service | where {$_ -match 'sql'}
Get-Service | where {$_ -like 'sql'}
Get-Service | Select-String sql
I am looking for a pattern that lets me treat the output of every command as searchable text.
Just forget it :o)
Outputs are objects. You are right, and you are going to use this.
So mjolinor has the shortest answer, but for your knowledge just test:
Get-Service | Get-Member
So you will understand that
Get-Service | Where-Object {$_.name -match ".*sql.*" }
also works, and there you've got your text as a property of the object.
Most answers here focus on finding the service name with "sql" in the name, not on filtering the entire output as if it was text. Also, the accepted answer uses a non-PowerShell function, "findstr".
So, granted, what follows is not the most elegant solution, but for sake of completeness I would like to provide the 100% PowerShell solution that takes the question of the OP literally:
(get-Service | Out-String) -split "`r`n" | Select-String sql
We need Out-String, because using the solutions provided in other answers doesn't provide us the full text output of the Get-Service command, only the Name parameter.
We need to split on newlines, because Select-String seems to treat the entire text as one long string, and returns it as a whole, if "sql" is found in it.
I use Select-String instead of findstr, because findstr is not a PowerShell function.
This is a purist answer, and in practice, for this specific use-case, I would not recommend it. But for people coming here through Google Search based on the question title, this is a more accurate answer...
Get-Service | Select-String -Pattern "sql"
This works just like grep. And you can even sort:
Get-Service | Select-String -Pattern "sql" | sort
The other answers are right of course about your specific question of starting services that have "sql" in their name, but to answer the generic question:
You can do Get-Service | Out-String, and you will get the output as string, much like how Unix commands work.
Also when the output is piped to non-PowerShell commands, it does get converted to text, so for example: Get-Service | grep sql would work the way you wanted.
But again, like #JPBlanc says, it is good embrace the way PowerShell works, which is that the outputs are objects. It gives you way more control and keeps things simple and readable (the Unix commands with sed, awk and what not operating on text output of other command outputs can get very cryptic!).
You're working way too hard at it:
Get-Service *sql*
If anyone wants more information on logical operations, please see Using the Where-Object Cmdlet:
• -lt -- Less than
• -le -- Less than or equal to
• -gt -- Greater than
• -ge -- Greater than or equal to
• -eq -- Equal to
• -ne -- Not equal to
• -like - Like; uses wildcards for pattern matching
Get-Service | where {$_ -match 'sql'} would be Get-Service | where {$_ -eq "sql"}
Get-Service | where {$_ -like 'sql'} would be Get-Service | where {$_ -like "sql"}
And now an actual example.
PS C:\> Get-Service | where {$_.name -like "net*"}
Status Name DisplayName
------ ---- -----------
Running Net Driver HPZ12 Net Driver HPZ12
Running Netlogon Netlogon
That the text of the name is a property of the object is important to get your head around, and how to use the property values in a filter.
Another aspect of PowerShell you can leverage to solve this is selecting properties out of objects with Select-Object (alias select):
Get-Service | select -expand name
will get you a string array with the names of the servers, and two of your original three filters would work on that. The -like isn't going to work, because there's no wildcards in the test string. The only thing it will ever match is just 'sql'.
I still believe the first solution I posted is best. It's important to know how to do late filtering, but also how to use early filtering when you can.
If you want to list all services with "sql" in the service name, just use:
get-service -name *sql*
You probably want this:
Function Select-ObjectPropertyValues {
param(
[Parameter(Mandatory=$true, Position=0)]
[String]
$Pattern,
[Parameter(ValueFromPipeline)]
$input
)
$input | Where-Object {($_.PSObject.Properties | Where-Object {$_.Value -match $Pattern} | Measure-Object).count -gt 0} | Write-Output
}
Here we are going though each property of an object to see if it matches the given pattern. If the object contains one or more such properties, we write it out. End result: grep by all properties of an object.
Put it in your configuration files and grep to your heart's content.
how about:
Get-Service| Out-String -stream | Select-String sql
where the key point is that -stream option converts the Out-String output in separate lines of text.