Get-Content with Wait and processing each line - powershell

Using -Match gives me True/False values instead of the lines of text.
Get-Content ("\\path”) -tail 10 -wait | % {
foreach ($data in ($_ -match "Execute")) {
$First = $data.Substring(26,37).Trim()
Write-Host $First
}
}
I used below code without -tail -wait to do what I, but I can't change to parsing the file using -tail with Get-Content.
$DB = Get-Content ("\\path”) -tail -ReadCount 5000000 | foreach { $_ -match "string to match" } | foreach{ $_.Trim()}
foreach ($Data in $DB) {
$First = $Data.Substring(26,37).Trim()
$Second = $Data
Write-Host $First
Write-Host $Second
}

As a workaround, you can array-cast your $_, like this:
foreach ($data in (,$_ -match "Execute")) {
Here's the output difference:
$data=#("bla bla","foo bla","foo bar","bla bar")
PS > $data | % { foreach ($a in ($_ -match "bla")){$a}}
True
True
False
True
PS > $data | % { foreach ($a in (,$_ -match "bla")){$a}}
bla bla
foo bla
bla bar

The -match expression returns a boolean result, but also updates a special variable, $matches.
You can filter lines by using -match in a where-object expression to return only those lines which match.
$myFilteredArray = $myArray | where-object{$_ -match 'execute'}
If using it for something that simple though, you may as well use -like:
$myFilteredArray = $myArray | where-object{$_ -like '*execute*'}
If you want to be clever, you can also use regular expressions; that way $matches will hold all captured groups (including a special group named 0 which holds the original line).
Simple Example / plain text search
#(
'line 1 - please '
,'line 2 - execute'
,'line 3 - this'
,'line 4 - script'
) | where-object {$_ -match 'execute'} | foreach-object{
"-Match found the following matches:"
$matches
"The line giving this result was"
$_
}
Example using Regex / capturing group
#(
'line 1 - please '
,'line 2 - execute'
,'line 3 - this'
,'line 4 - script'
) | where-object {$_ -match '(?<capturedData>.*?)(?: - )execute'} | foreach-object{
"-Match found the following matches:"
$matches
"The line giving this result was"
$_
}
-match: https://ss64.com/ps/syntax-regex.html
-like, -match, and other comparisons: https://ss64.com/ps/syntax-compare.html

Well, -match when applied to a scalar returns a boolean result. When applied to an array it returns the matching elements. In your second example you use ReadCount to force Get-Content to send more than a line through the pipeline at a time. Of course you're then getting an array of lines every time instead of just a line, thus changing the semantics of -match.
To make your first example work, simply change foreach into if.

Related

How do I loop through a string read from a file against a fixed set of characters individually and if they match print the string in powershell

I currently have a foreach loop, that gets content from a small dictionary file (only strings over 3 characters). I am looking to compare each character in the $line against my other characters, in this case "b" "i" "n" "g" "o" so that if all the characters in $line are in bingo, then it prints the word. If not it loops to the next word.
So far I have:
foreach($line in Get-Content Desktop/dict.txt | Sort-Object Length, { $_ })
The bit I can't get (not too familiar with powershell) is this:
if($line.length -gt 3){
if( i in $line == 'b')
if( i in $line == 'i')
if( i in $line == 'n')
if( i in $line == 'g')
if( i in $line == 'o')
write-output $line
}
}
If I understood correctly, if you want to check if $line is contained in bingo you could use the -match for case insensitive and -cmatch for case sensitive operators. See Comparison Operators.
For example:
PS /> 'bingo' -match 'ing'
True
PS /> 'bingo' -match 'bin'
True
PS /> 'bingo' -match 'ngo'
True
The code could look like this:
foreach($line in Get-Content Desktop/dict.txt | Sort-Object Length, { $_ })
{
if($line.length -gt 3 -and 'bingo' -match $line)
{
$line
# you can add break here to stop this loop if the word is found
}
}
Edit
If you want to check if 3 or more characters in bingo (in any order) are contained in $line, there are many ways to do this, this is the approach I would take:
# Insert magic word here
$magicWord = 'bingo'.ToCharArray() -join '|'
foreach($line in Get-Content Desktop/dict.txt | Sort-Object Length, { $_ })
{
# Remove [Text.RegularExpressions.RegexOptions]::IgnoreCase if you want it to be Case Sensitive
$check = [regex]::Matches($line,$magicWord,[Text.RegularExpressions.RegexOptions]::IgnoreCase)
# If 3 or more unique characters were matched
if(($check.Value | Select-Object -Unique).count -ge 3)
{
'Line is: {0} || Characters Matched: {1}' -f $line,-join $check.Value
}
}
Demo
Given the following words:
$test = 'ngob','ibgo','gn','foo','var','boing','ingob','oubingo','asdBINGasdO!'
It would yield:
Line is: ngob || Characters Matched: ngob
Line is: ibgo || Characters Matched: ibgo
Line is: boing || Characters Matched: boing
Line is: ingob || Characters Matched: ingob
Line is: oubingo || Characters Matched: obingo
Line is: asdBINGasdO! || Characters Matched: BINGO
So you want to get back any words that are the same length and have the same characters no matter the order?
$dict = #(
'bingo'
'rambo'
'big'
'gobin'
'bee'
'ebe'
'been'
'ginbo'
)
$word = 'bingo'
$dict |
Where-Object { $_.length -eq $word.Length } |
ForEach-Object {
$dictwordLetters = [System.Collections.Generic.List[char]]::new($_.ToCharArray())
$word.ToCharArray() | ForEach-Object {
$dictwordLetters.Remove($_) | Out-Null
}
if (-not $dictwordLetters.Count) {
$_
}
}
The following will be the output
bingo
gobin
ginbo
By taking parts of both answers I was able to get the result I was after. As I am new to this, not sure how to thank #martin and #santiago for their work.
This was the code that was put together, which was pretty much taking the dictionary file and then rather than a fixed string size made it greater than 3:
$dict = #(Get-Content Desktop/dict.txt | Sort-Object Length, { $_ })
$word = 'bingo'
$dict |
Where-Object { $_.length -gt 2 } |
ForEach-Object {
$dictwordLetters = [System.Collections.Generic.List[char]]::new($_.ToCharArray())
$word.ToCharArray() | ForEach-Object {
$dictwordLetters.Remove($_) | Out-Null
}
if (-not $dictwordLetters.Count) {
$_
}
}
Your assistance was greatly appreciated.
Here's my two cents:
$dict = 'apple', 'brown', 'green', 'cake', 'bin', 'pear', 'big', 'milk', 'bio', 'bong', 'bingo', 'bodings', 'gibson'
# the search term as string
$term = 'bingo'
# merge the unique characters into a regex like '[bingo]+'
$chars = '[{0}]+' -f (($term.ToCharArray() | Select-Object -Unique) -join '')
# loop over the array (lines in the text file)
$dict | ForEach-Object {
# get all matched characters, uniqify and join if there are more matches.
$found = (($_ | Select-String -Pattern $chars -AllMatches).Matches.Value | Select-Object -Unique ) -join '' | Where-Object { $_.Length -ge 3 }
if ($found) {
# outputting an object better explains what is matched in which line
[PsCustomObject]#{
Line = $_
CharactersMatched = $found
}
# of course, you can also simply output the found matching characters
# $found
}
}
Output:
Line CharactersMatched
---- -----------------
brown bon
bin bin
big big
bio bio
bong bong
bingo bingo
bodings boing
gibson gibon
The previous answers all seem overly complicated. If you are trying to match strings then that sounds like a problem that requires a regular expression, and if that is the case then Select-String would be a better option than Get-Content. Below is an example, I am not sure if it is exactly right for your needs but should point you in the right direction:
Select-String 'Desktop/dict.txt' -pattern '^[bingo]{3,}$'

Cycling through multiple variables in for loop

I just started working with Powershell and this is my first script.
I am checking for 3 strings in last 50 lines of a log file. I need to find all three strings and print error message if any one of those is missing. I have written following script but it does not give me the expected results.
(Get-Content C:\foo\bar.log )[-1..-50] | Out-File C:\boom\shiva\log.txt
$PO1 = Get-Content C:\boom\shiva\log.txt | where {$_ -match "<Ping:AD_P01_RCV> ok"}
$PO2 = Get-Content C:\boom\shiva\log.txt | where {$_ -match "<Ping:AD_P02_SND> ok"}
$PO3 = Get-Content C:\boom\shiva\log.txt | where {$_ -match "<Ping:AD_P03_RCV> ok"}
I am satisfied with above piece of code. The problem is with the below. I dont want to use if-else thrice. I am struggling to draft a for loop which can save space and still give me the same result.
if (!$PO1)
{
"PO1 is critical"
}
else
{
"PO1 is OK"
}
if (!$PO2)
{
"PO2 is critical"
}
else
{
"PO2 is OK"
}
if (!$PO3)
{
"PO3 is critical"
}
else
{
"PO3 is OK"
}
Can someone gave me small example of how i can fit these 3 if-else in one for loop.
If you only want to find out that all 3 strings are present this script will also show which one is missing.
(binary encoded in the variable $Cnt)
## Q:\Test\2018\07\13\SO_51323760.ps1
##
$Last50 = Get-Content 'C:\foo\bar.log' | Select-Object -Last 50
$Cnt = 0
if ($Last50 -match "<Ping:AD_P01_RCV> ok"){$Cnt++}
if ($Last50 -match "<Ping:AD_P02_SND> ok"){$Cnt+=2}
if ($Last50 -match "<Ping:AD_P03_RCV> ok"){$Cnt+=4}
if ($cnt -eq 7){
"did find all 3 strings "
} else {
"didn't find all 3 strings ({0})" -f $cnt
}
Variant immediately complaining missing P0(1..3)
$Last50 = Get-Content 'C:\foo\bar.log' | Select-Object -Last 50
if (!($Last50 -match "<Ping:AD_P01_RCV> ok")) {"PO1 is critical"}
if (!($Last50 -match "<Ping:AD_P02_SND> ok")) {"PO2 is critical"}
if (!($Last50 -match "<Ping:AD_P03_RCV> ok")) {"PO3 is critical"}
Sorry I'm a bit slow this monday.
To check in a loop different variables by building the variable name:
1..3| ForEach-Object {
If (!(Get-Variable -name "P0$_").Value){"`$P0$_ is critical"}
}
What you're trying to do is better addressed with a hashtable than with individually named variables.
$data = Get-Content 'C:\boom\shiva\log.txt'
$ht = #{}
1..3 | ForEach-Object {
$key = 'P{0:d2}' -f $_
$str = if ($_ -eq 2) {"${key}_SND"} else {"${key}_RCV"}
$ht[$key] = $data -match "<ing:AD_${str}> ok"
}
$ht.Keys | ForEach-Object {
if ($ht[$_]) {
"${key} found in log."
} else {
"${key} not found in log."
}
}
You can check if all lines were present at least once with something like this:
if (($ht.Values | Where-Object { $_ }).Count -lt 3) {
'Line missing from log.'
}
PSv3 introduced the -Tail (-Last) parameter to Get-Content, which is the most efficient way to extract a fixe number of lines from the end of a file.
You can pipe its output to Select-String, which accepts an array of regex patterns, any of which produces a match (implicit OR logic).
$matchingLines = Get-Content -Tail 50 C:\foo\bar.log |
Select-String '<Ping:AD_P01_RCV> ok', '<Ping:AD_P02_SND> ok', '<Ping:AD_P03_RCV> ok'
if ($matchingLines) { # at least 1 of the regexes matched
$matchingLines.Line # output the matching lines
} else { # nothing matched
Write-Warning "Nothing matched."
}
I finally got below draft that resolved my query to cycle variables through a for loop. I finally had to convert those individual variables to a array. But htis gives me expected result. Basically i need this script to provide input to my Nagios plugin which needs minor modification but its done.
(Get-Content C:\foo\bar.log )[-1..-50] | Out-File C:\boom\shiva\log.txt
$j = 1
$PO = new-object object[] 3
$PO[0] = Get-Content C:\boom\shiva\log.txt | where {$_ -match "<Ping:AD_P01_RCV> ok"}
$PO[1] = Get-Content C:\boom\shiva\log.txt | where {$_ -match "<Ping:AD_P02_SND> ok"}
$PO[2] = Get-Content C:\boom\shiva\log.txt | where {$_ -match "<Ping:AD_P03_RCV> ok"}
foreach( $i in $PO){
if (!$i){
"PO "+$j+" is CRITICAL"}
else{
"PO "+$j+" is OK"}
$j+=1
}
Thank you LotPings, Ansgar and mklement0 for your support and responses. I picked up a few things from your answers.

Splitting in Powershell

I want to be able to split some text out of a txtfile:
For example:
Brackets#Release 1.11.6#Path-to-Brackets
Atom#v1.4#Path-to-Atom
I just want to have the "Release 1.11.6" part. I am doing a where-object starts with Brackets but I don't know the full syntax. Here is my code:
"Get-Content -Path thisfile.txt | Where-Object{$_ < IM STUCK HERE > !
You could do this:
((Get-Content thisfile.txt | Where-Object { $_ -match '^Brackets' }) -Split '#')[1]
This uses the -match operator to filter out any lines that don't start with Brackets (the ^ special regex character indicates that what follows must be at the beginning of the line). Then it uses the -Split operator to split those lines on # and then it uses the array index [1] to get the second element of the split (arrays start at 0).
Note that this will throw an error if the split on # doesn't return at least two elements and it assumes that the text you want is always the second of those elements.
$bracketsRelease = Get-Content -path thisfile.txt | foreach-object {
if ( $_ -match 'Brackets#(Release [^#]+)#' )
{
$Matches[1]
}
}
or
(select-string -Path file.txt -Pattern 'Brackets#(Release [^#]+)#').Matches[0].Groups[1].value

Retrieving second part of a line when first part matches exactly

I used the below steps to retrieve a string from file
$variable = 'abc#yahoo.com'
$test = $variable.split('#')[0];
$file = Get-Content C:\Temp\file1.txt | Where-Object { $_.Contains($test) }
$postPipePortion = $file | Foreach-Object {$_.Substring($_.IndexOf("|") + 1)}
This results in all lines that contain $test as a substring. I just want the result to contain only the lines that exactly matches $test.
For example, If a file contains
abc_def|hf#23$
abc|ohgvtre
I just want the text ohgvtre
If I understand the question correctly you probably want to use Import-Csv instead of Get-Content:
Import-Csv 'C:\Temp\file1.txt' -Delimiter '|' -Header 'foo', 'bar' |
Where-Object { $_.foo -eq $test } |
Select-Object -Expand bar
To address the exact matching, you should be testing for equality (-eq) rather than substring (.Contains()). Also, there is no need to parse the data multiple times. Here is your code, rewritten to to operate in one pass over the data using the -split operator.
$variable = 'abc#yahoo.com'
$test = $variable.split('#')[0];
$postPipePortion = (
# Iterate once over the lines in file1.txt
Get-Content C:\Temp\file1.txt | foreach {
# Split the string, keeping both parts in separate variables.
# Note the backslash - the argument to the -split operator is a regex
$first, $second = ($_ -split '\|')
# When the first half matches, output the second half.
if ($first -eq $test) {
$second
}
}
)

How can I search the first line and the last line in a text file?

I need to only search the 1st line and last line in a text file to find a "-" and remove it.
How can I do it?
I tried select-string, but I don't know to find the 1st and last line and only remove "-" from there.
Here is what the text file looks like:
% 01-A247M15 G70
N0001 G30 G17 X-100 Y-100 Z0
N0002 G31 G90 X100 Y100 Z45
N0003 ; --PART NO.: NC-HON.PHX01.COVER-SHOE.DET-1000.050
N0004 ; --TOOL: 8.55 X .3937
N0005 ;
N0006 % 01-A247M15 G70
Something like this?
$1 = Get-Content C:\work\test\01.I
$1 | select-object -index 0, ($1.count-1)
Ok, so after looking at this for a while, I decided there had to be a way to do this with a one liner. Here it is:
(gc "c:\myfile.txt") | % -Begin {$test = (gc "c:\myfile.txt" | select -first 1 -last 1)} -Process {if ( $_ -eq $test[0] -or $_ -eq $test[-1] ) { $_ -replace "-" } else { $_ }} | Set-Content "c:\myfile.txt"
Here is a breakdown of what this is doing:
First, the aliases for those now familiar. I only put them in because the command is long enough as it is, so this helps keep things manageable:
gc means Get-Content
% means Foreach
$_ is for the current pipeline value (this isn't an alias, but I thought I would define it since you said you were new)
Ok, now here is what is happening in this:
(gc "c:\myfile.txt") | --> Gets the content of c:\myfile.txt and sends it down the line
% --> Does a foreach loop (goes through each item in the pipeline individually)
-Begin {$test = (gc "c:\myfile.txt" | select -first 1 -last 1)} --> This is a begin block, it runs everything here before it goes onto the pipeline stuff. It is loading the first and last line of c:\myfile.txt into an array so we can check for first and last items
-Process {if ( $_ -eq $test[0] -or $_ -eq $test[-1] ) --> This runs a check on each item in the pipeline, checking if it's the first or the last item in the file
{ $_ -replace "-" } else { $_ } --> if it's the first or last, it does the replacement, if it's not, it just leaves it alone
| Set-Content "c:\myfile.txt" --> This puts the new values back into the file.
Please see the following sites for more information on each of these items:
Get-Content uses
Get-Content definition
Foreach
The Pipeline
Begin and Process part of the Foreach (this are usually for custom function, but they work in the foreach loop as well)
If ... else statements
Set-Content
So I was thinking about what if you wanted to do this to many files, or wanted to do this often. I decided to make a function that does what you are asking. Here is the function:
function Replace-FirstLast {
[CmdletBinding()]
param(
[Parameter( `
Position=0, `
Mandatory=$true)]
[String]$File,
[Parameter( `
Position=1, `
Mandatory=$true)]
[ValidateNotNull()]
[regex]$Regex,
[Parameter( `
position=2, `
Mandatory=$false)]
[string]$ReplaceWith=""
)
Begin {
$lines = Get-Content $File
} #end begin
Process {
foreach ($line in $lines) {
if ( $line -eq $lines[0] ) {
$lines[0] = $line -replace $Regex,$ReplaceWith
} #end if
if ( $line -eq $lines[-1] ) {
$lines[-1] = $line -replace $Regex,$ReplaceWith
}
} #end foreach
}#End process
end {
$lines | Set-Content $File
}#end end
} #end function
This will create a command called Replace-FirstLast. It would be called like this:
Replace-FirstLast -File "C:\myfiles.txt" -Regex "-" -ReplaceWith "NewText"
The -Replacewith is optional, if it is blank it will just remove (default value of ""). The -Regex is looking for a regular expression to match your command. For information on placing this into your profile check this article
Please note: If you file is very large (several GBs), this isn't the best solution. This would cause the whole file to live in memory, which could potentially cause other issues.
try:
$txt = get-content c:\myfile.txt
$txt[0] = $txt[0] -replace '-'
$txt[$txt.length - 1 ] = $txt[$txt.length - 1 ] -replace '-'
$txt | set-content c:\myfile.txt
You can use the select-object cmdlet to help you with this, since get-content basically spits out a text file as one huge array.
Thus, you can do something like this
get-content "path_to_my_awesome_file" | select -first 1 -last 1
To remove the dash after that, you can use the -Replace switch to find the dash and remove it. This is better than using System.String.Replace(...) method because it can match regex statements and replace whole arrays of strings too!
That would look like:
# gc = Get-Content. The parens tell Powershell to do whatever's inside of it
# then treat it like a variable.
(gc "path_to_my_awesome_file" | select -first 1 -last 1) -Replace '-',''
If your file is very large you might not want to read the whole file to get the last line. gc -Tail will get the last line very quickly for you.
function GetFirstAndLastLine($path){
return New-Object PSObject -Property #{
First = Get-Content $path -TotalCount 1
Last = Get-Content $path -Tail 1
}
}
GetFirstAndLastLine "u_ex150417.log"
I tried this on a 20 gb log file and it returned immediately. Reading the file takes hours.
You will still need to read the file if you want to keep all excising content and you want only to remove from the end. Using the -Tail is a quick way to check if it is there.
I hope it helps.
A cleaner answer to the above:
$Line_number_were_on = 0
$Awesome_file = Get-Content "path_to_ridiculously_excellent_file" | %{
$Line = $_
if ($Line_number_were_on -eq $Awesome_file.Length)
{ $Line -Replace '-','' }
else
{ $Line } ;
$Line_number_were_on++
}
I like one-liners, but I find that readability tends to suffer sometimes when I put terseness over function. If what you're doing is going to be part of a script that other people will be reading/maintaining, readability might be something to consider.
Following Nick's answer: I do need to do this on all text files in the directory tree and this is what I'm using now:
Get-ChildItem -Path "c:\work\test" -Filter *.i | where { !$_.PSIsContainer } | % {
$txt = Get-Content $_.FullName;
$txt[0] = $txt[0] -replace '-';
$txt[$txt.length - 1 ] = $txt[$txt.length - 1 ] -replace '-';
$txt | Set-Content $_.FullName
}
and it looks like it's working well now.
Simple process:
Replace $file.txt with your filename
Get-Content $file_txt | Select-Object -last 1
I was recently searching for comments in the last line of .bat files. It seems to mess up the error code of previous commands. I found this useful for searching for a pattern in the last line of files. Pspath is a hidden property that get-content outputs. If I used select-string, I would lose the filename. *.bat gets passed as -filter for speed.
get-childitem -recurse . *.bat | get-content -tail 1 | where { $_ -match 'rem' } |
select pspath
PSPath
------
Microsoft.PowerShell.Core\FileSystem::C:\users\js\foo\file.bat