Get-Content - Get all Content, starting from a specific linenumber - powershell

My first question here, and just want to say thanks for all the input I've gotten over the years from this site.
I'm also new to powershell so the answar might be very simple.
I'm working on a Script that ment check a log file every 5 mins. (schedulded from ActiveBatch).
At the moment the script is searching for ERROR in a logfile. And it works fine.
But my problem is that the script searches the entire file throgh every time. So when an ERROR do occur, the check "fails" every 5 minutes the rest of the day. Untill a new logfile is generated.
My script:
Write-Host Opretter variabler...
$file = "${file}"
$errorString = "${errorString}"
Write-Host file variable is: $file
Write-Host errorString variable is: $errorString
Write-Host
Write-Host Select String Results:
$ssResult = Get-Content $file | Select-String $errorString -SimpleMatch
Write-Host
Write-Host There was $ssResult.Count `"$errorString`" statements found...
Write-Host
IF ($ssResult.Count -gt 0) {Exit $ssResult.Count}
So what i would like, is to Find the ERROR, and then Remeber the Linenumber (Perhaps in a file). Then in the next run (5minutes later) i want to start the search from that line.
for example. And error is found on line 142, the Script exits with error code 142. five minutes later the script is run again, and it should start from line 143, and go through the rest of the file.

You can remember number of error strings found in file:
$ssResult.Count > C:\path\to\file.txt
Then number of new erros is:
$errorCount = $ssResult.Count - (Get-Content C:\path\to\file.txt)
Remember to set the value in file to zero on first run of script and every time a new logfile is generated.

You basically gave a pretty good description of how it will work:
Read the last line number
$if (Test-Path $Env:TEMP\last-line-number.txt) {
[int]$LastLineNumber = #(Get-Content $Env:TEMP\last-line-number.txt)[0]
} else {
$LastLineNumber = 0
}
Read the file
$contents = Get-Content $file
Find the first error starting at $LastLineNumber (one of the rare cases where for is appropriate in PowerShell, lest we want to create nicer objects)
for ($i = $LastLineNumber; $i -lt $contents.Count; $i++) {
if ($contents[$i] -like "*$errorString*") {
$i + 1 > $Env:TEMP\last-line-number.txt
exit ($i + 1)
}
}

Select-String returns matchinfo objects, which have the line number, so you can should be able to do something like this:
$lasterror = Get-Content $lasterrorfile
$newerrors = select-string -Path $file -Pattern $errorstring -SimpleMatch |
where $_.LineNumber -gt $lasterror
Write-Host "$($newerrors.count) found."
if ($newerrors.count)
{$newerrors[-1].LineNumber | Set-Content $lasterrorfile}

So this is my final Script, Thanks Dano. I'm sure the Day-Reset thing can be done smarter, but this seems to work :)
#logic for Day-Reset
Write-Host checking if its a new day...
$today = Get-Date -format dddd
$yesterday = Get-Content $ENV:TEMP\${adapterName}_yesterday.txt
Write-Host today variable is: $today
Write-Host yesterday variable is: $yesterday
Write-Host
IF ($today.CompareTo($yesterday))
{
Get-Date -format dddd > $ENV:TEMP\${adapterName}_yesterday.txt
0 > $ENV:TEMP\${adapterName}_numberofErrors.txt
}
Write-Host Setting variables...
$file = "${file}"
$errorString = "${errorString}"
Write-Host file variable is: $file
Write-Host errorString variable is: $errorString
Write-Host
Write-Host Select String Results:
$ssResult = Get-Content $file | Select-String $errorString -SimpleMatch
Write-Host There was $ssResult.Count `"$errorString`" statements found...
$errorCount = $ssResult.Count - (Get-Content $ENV:TEMP\${adapterName}_numberofErrors.txt)
Write-Host There was $errorCount new `"$errorString`" statements found...
Write-Host
$ssResult.Count > $Env:TEMP\FXAll_numberofErrors.txt
Exit $errorCount

Related

Powershell: exclusive lock for a file during multiple set-content and get-content operations

I am trying to write a script which runs on multiple client machines and writes to a single text file on a network share.
I want to ensure that only one machine can maniputale the file at any one time, whilst the other machines run a loop to check if the file is available.
the script runs this first:
Set-Content -Path $PathToHost -Value (get-content -Path $PathToHost | Select-String -Pattern "$HostName " -NotMatch) -ErrorAction Stop
Which removes some lines if they are matching the criteria. Then I want to append a new line with this:
Add-Content $PathToHost "$heartbeat$_" -ErrorAction Stop
The problem is that between the execution of those two commands another client has access to the file and begins to write to the file as well.
I have explored the solution here: Locking the file while writing in PowerShell
$PathToHost = "C:\file.txt"
$mode = "Open"
$access = "ReadWrite"
$share = "None"
$file = [System.IO.File]::Open($path, $mode, $access, $share)
$file.close()
Which can definitely lock the file, but I am not sure how to proceed to then read and write to the file.
Any help is much appreciated.
EDIT: Solution as below thanks to twinlakes' answer
$path = "C:\Users\daniel_mladenov\hostsTEST.txt"
$mode = "Open"
$access = "ReadWrite"
$share = "none"
$file = [System.IO.File]::Open($path, $mode, $access, $share)
$fileread = [System.IO.StreamReader]::new($file, [Text.Encoding]::UTF8)
# Counts number of lines in file
$imax=0
while ($fileread.ReadLine() -ne $null){
$imax++
}
echo $imax
#resets read position to beginning
$fileread.basestream.position = 0
#reads content of whole file and discards mathching lines
$content=#()
for ($i=0; $i -lt $imax; $i++){
$ContentLine = $fileread.ReadLine()
If($ContentLine -notmatch "$HostIP\s" -and $ContentLine -notmatch "$HostName\s"){
$content += $ContentLine
}
}
echo $content
#Writes remaining lines back to file
$filewrite = [System.IO.StreamWriter]::new($file)
$filewrite.basestream.position = 0
for ($i=0; $i -lt $content.length; $i++){
$filewrite.WriteLine($content[$i])
}
$filewrite.WriteLine($heartbeat)
$filewrite.Flush()
$file.SetLength($file.Position) #trims file to the content which has been written, discarding any content past that point
$file.close()
$file is a System.IO.FileStream object. You will need to call the write method on that object, which requires a byte array.
$string = # the string to write to the file
$bytes = [Text.Encoding]::UTF8.GetBytes($string)
$file.Write($bytes, 0, $bytes.Length)

Powershell Host File edit

Guys i'm having some issues converting my Perl script to powershell, I need some help. In the host file of our machines, we have all of the URL's to our test environments blocked. In my PERL script, based on which environment is selected, it will comment out the line of the environment selected to allow access and block others so the testers can't mistakenly do things in the wrong environment.
I need help converting to powershell
Below is what I have in PERL:
sub editHosts {
print "Editing hosts file...\n";
my $file = 'C:\\Windows\\System32\\Drivers\\etc\\hosts';
my $data = readFile($file);
my #lines = split /\n/, $data;
my $row = '1';
open (FILE, ">$file") or die "Cannot open $file\n";
foreach my $line (#lines) {
if ($line =~ m/$web/) {
print FILE '#'."$line\n"; }
else {
if ($row > '21') {
$line =~ s/^\#*127\.0\.0\.1/127\.0\.0\.1/;
$line =~ s/[#;].*$//s; }
print FILE "$line\n"; }
$row++;
}
close(FILE);
}
Here is what i've tried in Powershell:
foreach ($line in get-content "C:\windows\system32\drivers\etc\hosts") {
if ($line -contains $web) {
$line + "#"
}
I've tried variation including set-content with what used to be in the host file, etc.
Any help would be appreciated!
Thanks,
Grant
-contains is a "set" operator, not a substring operator. Try .Contains() or -like.
This will comment out lines matching the variable $word, while removing # from non-matches (except the header):
function Edit-Hosts ([string]$Web, $File = "C:\windows\system32\drivers\etc\hosts") {
#If file exists and $web is not empty/whitespace
if((Test-Path -Path $file -PathType Leaf) -and $web.Trim()) {
$row = 1
(Get-Content -Path $file) | ForEach-Object {
if($_ -like "*$web*") {
#Matched PROD, comment out line
"#$($_)"
} else {
#No match. If past header = remove comment
if($row -gt 21) { $_ -replace '^#' } else { $_ }
}
$row++
} | Set-Content -Path $file
} else {
Write-Error -Category InvalidArgument -Message "'$file' doesn't exist or Web-parameter is empty"
}
}
Usage:
Edit-Hosts -Web "PROD"
This is a similar answer to Frode F.'s answer, but I'm not yet able to comment to add my 2c worth, so have to provide an alternative answer instead.
It looks like one of the gotchas moving from perl to PowerShell, in this example, is that when we get the content of the file using Get-Content it is an "offline" copy, i.e. any edits are not made directly to the file itself. One approach is to compile the new content to the whole file and then write that back to disk.
I suppose that the print FILE "some text\n"; construct in perl might be similar to "some text" | Out-File $filename -Encoding ascii -Append in PowerShell, albeit you would use the latter either (1) to write line-by-line to a new/empty file or (2) accept that you are appending to existing content.
Two other things about editing the hosts file:
Be sure to make sure that your hosts file is ASCII encoded; I have caused a major outage for a key enterprise application (50k+ users) in learning that...
You may need to remember to run your PowerShell / PowerShell ISE by right-clicking and choosing Run as Administrator else you might not be able to modify the file.
Anyway, here's a version of the previous answer using Out-File:
$FileName = "C:\windows\system32\drivers\etc\hosts"
$web = "PROD"
# Get "offline" copy of file contents
$FileContent = Get-Content $FileName
# The following creates an empty file and returns a file
# object (type [System.IO.FileInfo])
$EmptyFile = New-Item -Path $FileName -ItemType File -Force
foreach($Line in $FileContent) {
if($Line -match "$web") {
"# $Line" | Out-File $EmptyFile -Append -Encoding ascii
} else {
"$Line" | Out-File $EmptyFile -Append -Encoding ascii
}
}
Edit
The ($Line -match "$web") takes whatever is in the $web variable and treats it as a regular expression. In my example I was assuming that you were just wanting to match a simple text string, but you might well be trying to match an IP address, etc. You have a couple of options:
Use ($Line -like "*$web*") instead.
Convert what is in $web to be an escaped regex, i.e. one that will match literally. Do this with ($Line -match [Regex]::Escape($web)).
You also wanted to strip off comments from any line past row 21 of the hosts file, should that line not match $web. In perl you have used the s substitution operator; the PowerShell equivalent is -replace.
So... here is an updated version of that foreach loop:
$LineCount = 1
foreach($Line in $FileContent) {
if($Line -match [Regex]::Escape($web) {
# ADD comment to any matched line
$Line = "#" + $Line
} elseif($LineCount -gt 21) {
# Uncomment the other lines
$Line = $Line -replace '^[# ]+',''
}
# Remove 'stacked up' comment characters, if any
$Line = $Line -replace '[#]+','#'
$Line | Out-File $EmptyFile -Append -Encoding ascii
$LineCount++
}
More Information
Are there good references for moving from Perl to Powershell?
How to use operator '-replace' in PowerShell to replace strings of texts with special characters and replace successfully
about_Comparison_Operators
http://www.comp.leeds.ac.uk/Perl/sandtr.html
If you wanted to verify what was in there and then add entries, you could use the below which is designed to be ran interactively and returns any existing entries you specify in the varibles:
Note: the `t is powershell's in script method for 'Tab' command.
$hostscontent
# Script to Verify and Add Host File Entries
$hostfile = gc 'C:\Windows\System32\drivers\etc\hosts'
$hostscontent1 = $hostfile | select-string "autodiscover.XXX.co.uk"
$hostscontent2 = $hostfile | select-string "webmail.XXX.co.uk"
$1 = "XX.XX.XXX.XX`tautodiscover.XXX.co.uk"
$2 = "webmail.XXX.co.uk"
# Replace this machines path with a path to your list of machines e.g. $machines = gc \\machine\machines.txt
$machines = gc 'c:\mytestmachine.txt'
ForEach ($machine in $machines) {
If ($hostscontent1 -ne $null) {
Start-Sleep -Seconds 1
Write-Host "$machine Already has Entry $1" -ForegroundColor Green
} Else {
Write-Host "Adding Entry $1 for $machine" -ForegroundColor Green
Start-Sleep -Seconds 1
Add-Content -Path C:\Windows\System32\drivers\etc\hosts -Value "XX.XX.XXX.XX`tautodiscover.XXX.co.uk" -Force
}
If ($hostscontent2 -ne $null) {
Start-Sleep -Seconds 1
Write-Host "$machine Already has Entry $2" -ForegroundColor Green
} Else {
Write-Host "Adding Entry $2 for $machine" -ForegroundColor Green
Start-Sleep -Seconds 1
Add-Content -Path C:\Windows\System32\drivers\etc\hosts -Value "XX.XX.XXX.XX`twebmail.XXX.co.uk" -Force
}
}

Why Am I geting an Empty Pipeline Error in this Script

I'm an extremely novice Powershell student who was given the task of getting the following code to work and I keep getting an Empty Pipeline Error at the line remarked 'Gives Empty Pipeline Error'. After quite a few hours of researching this I am still stumped as to what is causing this. The script is supposed to search the Application.evtx log and return any errors from the last 24 hours. I would greatly appreciate any help that could get me pointed in the right direction.
Here's the code:
#look for Errors script
#Creates Function named CheckLogs
Function CheckLogs()
{
# Defines a named parameter $logfile as a string
param ([string]$logfile)
if(!$logfile) {write-host "Usage: ""C:\Windows\System32\winevt\Logs\Application.evtx"""; exit}
# Accesses the file stored in $logfile variable and looks for the string "ERROR"
cat $logfile | Select-string "ERROR" -SimpleMatch |select -expand line |
foreach
{
$_ -match '(.+)\s\[(ERROR)\]\S(.+)'| Out-Null
new-object psobject -Property#{Timestamp=[datetime]$matches[1];Error=$matches[2]}
| #Gives Empty Pipeline Error
where {$_.timestamp -gt (get-date).AddDays(-1)}
$error_time=[datetime]($matches[1])
if ($error_time -gt (Get-Date).AddDays(-1))
{
write-output "CRITICAL: There is an error in the log file $logfile around
$($error_time.ToShortTimeString( ))"; exit(2)
}
}
write-output "OK: There were no errors in the past 24 hours."
}
CheckLogs "C:\Windows\System32\winevt\Logs\Application.evtx" #Function Call
You can't put the pipe | character on a line by itself. You can end a line with | and then continue the pipeline on the next line though.
This should work:
new-object psobject -Property#{Timestamp=[datetime]$matches[1];Error=$matches[2]} |
where {$_.timestamp -gt (get-date).AddDays(-1)}

How to dump the foreach loop output into a file in PowerShell?

I have wrote the following script to read the CSV file to perform the custom format of output.
Script is below:
$Content = Import-Csv Alert.csv
foreach ($Data in $Content) {
$First = $Data.DisplayName
$Second = $Data.ComputerName
$Third = $Data.Description
$Four = $Data.Name
$Five = $Data.ModifiedBy
$Six = $Data.State
$Seven = $Data.Sev
$Eight = $Data.Id
$Nine = $Data.Time
Write-Host "START;"
Write-Host "my_object="`'$First`'`;
Write-Host "my_host="`'$Second`'`;
Write-Host "my_long_msg="`'$Third`'`;
Write-Host "my_tool_id="`'$Four`'`;
Write-Host "my_owner="`'$Five`'`;
Write-Host "my_parameter="`'$Four`'`;
Write-Host "my_parameter_value="`'$Six`'`;
Write-Host "my_tool_sev="`'$Seven`'`;
Write-Host "my_tool_key="`'$Eight`'`;
Write-Host "msg="`'$Four`'`;
Write-Host "END"
}
The above script executing without any error.
Tried with Out-File and redirection operator in PowerShell to dump the output into a file, but I'm not finding any solution.
Write-Host writes to the console. That output cannot be redirected unless you run the code in another process. Either remove Write-Host entirely or replace it with Write-Output, so that the messages are written to the Success output stream.
Using a foreach loop also requires additional measures, because that loop type doesn't support pipelining. Either run it in a subexpression:
(foreach ($Data in $Content) { ... }) | Out-File ...
or assign its output to a variable:
$output = foreach ($Data in $Content) { ... }
$output | Out-File ...
Another option would be replacing the foreach loop with a ForEach-Object loop, which supports pipelining:
$Content | ForEach-Object {
$First = $_.DisplayName
$Second = $_.ComputerName
...
} | Out-File ...
Don't use Out-File inside the loop, because repeatedly opening the file will perform poorly.

Printing the whole line Powershell on word finder

$check = $args[1]
$numArgs = $($args.count)
$totMatch = 0
#reset variables for counting
for ( $i = 2; $i -lt $numArgs; $i++ )
{
$file = $args[$i]
if ( Test-Path $file ) {
#echo "The input file was named $file"
$match = #(Select-String $check $file -AllMatches | Select -Expand Matches | Select -Expand Value).count
echo "There were $match Matches in $file"
echo "There were $match Matches in $file" >> Output.txt
$totMatch = $totMatch + $match
}
else {
echo "File $file does not exist"
echo "File $file does not exist" >> Output.txt
}
}
echo "Total Matches Found: $totMatch"
Esentially i created a quick app to find the word searched and check the instances in the file, would anyone know how to edit this to send the whole Line that the word was found in to the Ouput.txt file, So rather on top of instances add the whole line itself? Thanks in advance
I can't see that your code works properly; even though you don't say how it's supposed to work (why is $check taken from args[1] instead of args[0]?).
Your Select-String line is getting the matching lines, then doing some selecting which throws away the line data you want, but doesn't seem to be necessary.
I've reworked it as:
$check = $args[0]
$totalMatches = 0
foreach ( $file in $args[1..$args.Length] )
{
if ( Test-Path $file ) {
$matches = Select-String $check $file -AllMatches -SimpleMatch
Write-Output "There were $($matches.Count) Matches in $file" | Tee-Object -FilePath "output.txt" -Append
foreach ($match in $matches) {
Write-Output $match.Line | Tee-Object -FilePath "output.txt" -Append
}
Write-Host
$totalMatches = $totalMatches + $matches.Count
}
else {
Write-Output "File $file does not exist" | Tee-Object -FilePath "output.txt" -Append
}
}
echo "Total Matches Found: $totalMatches"
Changes:
Take the $check as the first argument
Iterate over the arguments directly instead of counting through them
Added -SimpleMatch so it doesn't work with regexes, since you didn't mention them
Removed the select-object -expand bits, just grab the select-string results
Loop through the results and get the line from $match.line
Added Tee-Object which both writes to screen and to file in one line