Powershell remove two line before a match of string - powershell

I have file.txt that has content like this with empty line in between:
TESTING CONNECTION TO SERVER 1.1.1.1
COULD NOT CONNECT TO SERVER
TESTING CONNECTION TO SERVER 1.2.111.11
CONNECTION TO SERVER SUCCESSFUL
TESTING CONNECTION TO SERVER 12.1.1211.1
COULD NOT CONNECT TO SERVER
The main aim is to remove the IP address or its line from the file which are not able to connect. If the string matches like 'COULD NOT CONNECT' then the ip address or its line before has to be removed using PowerShell script.
$data = foreach($line in Get-Content $env:TEMP\file.txt)
{
if($line -like '*COULD NOT CONNECT*')
{
$linenumber = Get-Content $env:TEMP\file.txt | select-string 'COULD NOT CONNECT' | Select-Object LineNumber
$b = 2
foreach($lines in $linenumber
$lineno=$lines.LineNumber - $b
write-host $lineno
#logic 1
$content = Get-Content $env:TEMP\file.txt
$content | Skip -Index $lineno > $env:TEMP\OUTPUT_refined.txt
#logic 2
Get-Content $env:TEMP\file.txt |
Where-Object ReadCount -ne $lineno |
Set-Content -Encoding Utf8 $env:TEMP\file.txt -Force
#logic 3
$file = "$env:TEMP\file.txt"
$outFile = [IO.File]::CreateText("$env:TEMP\OUTPUT_refined.txt")
$lineNo = 0
try {
foreach ($line in [IO.File]::ReadLines("$env:TEMP\file.txt")) {
if (++$lineNo -eq $lineno) { continue }
$outFile.WriteLine($line)
}
} finally {
$outFile.Dispose()
}
}
}
else
{
$line
}
}
$data | Set-Content $env:TEMP\file.txt -Force
Try attempts with various logics based on calculating line numbers and creating new file with refined content.
After removing the contents either I need it can be same file or we can use Set-Content and create a new file anything is fine. Experts help needed. Thanks.

this code removes the 3 lines: i am using directly the regex of C#
$data = Get-Content -Path file.txt - raw
#select multiline option for regex ->
# so you can work with `r`n (line of feed, new line)
$option = [System.Text.RegularExpressions.RegexOptions]::Multiline
$pattern = [regex]::new("TESTING.+?`r?`n`r?`nCOULD NOT CONNECT.+?`r?`n", $option)
$data = $pattern.Replace($data, "")
$data | Set-Content -Path newfile.txt -Force
TESTING.+?`r?`n`r?`nCOULD NOT CONNECT.+?`r?`n
i am searching all lines beginning by TESTING and finishing by 2 Endoflines (following the text editor endofline is 'r'n or 'n so 'r? means 0 or 1 occurence) and following by COULD NOT CONNECT then end of this line.
that is the selection of 3 lines

Related

Requirement is to add 4 line if the matching pattern followed by the next pattern is unmatched along with count number in power shell

Hello my input file will be like below,my requiremnet is to add 4 line if the macthing pattern folled by the next pattern is unmacthed along with count number.
i will check look for the socket and if matches will incrremnt the line count to +1 toi get the next line and look for the word "address",if the address is not present i need to insert a set of line "communication.manageraddress_9,communication.manageraddress_10,communication.manageraddress_11" netx to the line.
communication.manageraddress_7=xxx.com
communication.managerid_7=xxx
communication.managerport_7=xxx
communication.socket_7=xx
communication.manageraddress_8=xxx.com
communication.managerid_8=xxx
communication.managerport_8=xxx
communication.socket_8=plain
Added by Manager
communication.managerhealthmon_4=true
communication.protocolrev_4=3
communication.managerhealthmon_1=true
communication.protocolrev_1=2
output will be like this
communication.manageraddress_7=xxx.com
communication.managerid_7=xxx
communication.managerport_7=xxx
communication.socket_7=xx
communication.manageraddress_8=xxx.com
communication.managerid_8=xxx
communication.managerport_8=xxx
communication.socket_8=plain
communication.manageraddress_9=xxx.com
communication.managerid_9=xxx
communication.managerport_9=xxx
communication.socket_9=plain
communication.manageraddress_10=xxx.com
communication.managerid_10=xxx
communication.managerport_1o=xxx
communication.socket_1o=plain
Added by Manager
communication.managerhealthmon_4=true
communication.protocolrev_4=3
communication.managerhealthmon_1=true
communication.protocolrev_1=2
this my script and i am struck with insert into text file along with increment number,can some one help in power shell.
$files = $File = 'C:\Users\rseerala\Desktop\ARUN\in.txt'
#$NewContent = Get-Content -Path $File
foreach($file in $files){
$content = Get-Content $file
for($i = 0; $i -lt $content.Count; $i++){
$line = $content[$i]
if ($line.Contains("socket"))
{
$line = $content[$i+2]
if ($line.Contains("address"))
{
Write-Host "This line starts with 6"
}}}}
Ok, so if I understand correctly, this is what you want:
#read the file as a single multiline string
$txt = Get-Content -Path 'C:\Users\rseerala\Desktop\ARUN\in.txt' -Raw
# if it contains the magic word '.socket_' followed by a number
if ($txt -match '\.socket_\d+') {
# first split off the 'Added by Manager' stuff
$content, $managerAdded = ($txt -split 'Added by Manager').Trim()
# split the content part into separate blocks of 4 lines
$blocks = $content -split '(\r?\n){2}' | Where-Object { $_ -match '\S' }
# get the index value from the last block
$index = [int]([regex] '(?i)\.socket_(\d+)').Match($blocks[-1]).Groups[1].Value
# now repeat the blocks you already have and output copies with incremented indices
$newBlocks = ($blocks | ForEach-Object {
$_ -replace '_\d+=', ('_{0}=' -f ++$index)
}) -join "`r`n`r`n"
# finally, combine the content part with the new blocks
# and the 'Added by Manager' lines with double newlines
$result = $content, $newBlocks, 'Added by Manager', $managerAdded -join "`r`n`r`n"
# output on screen
$result
# write to a new file
$result | Set-Content -Path 'C:\Users\rseerala\Desktop\ARUN\out.txt'
}
else {
Write-Warning "The file does not contain the word '.socket_' followed by a number.."
}
Output:
communication.manageraddress_7=xxx.com
communication.managerid_7=xxx
communication.managerport_7=xxx
communication.socket_7=xx
communication.manageraddress_8=xxx.com
communication.managerid_8=xxx
communication.managerport_8=xxx
communication.socket_8=plain
communication.manageraddress_9=xxx.com
communication.managerid_9=xxx
communication.managerport_9=xxx
communication.socket_9=plain
communication.manageraddress_10=xxx.com
communication.managerid_10=xxx
communication.managerport_10=xxx
communication.socket_10=plain
Added by Manager
communication.managerhealthmon_4=true
communication.protocolrev_4=3
communication.managerhealthmon_1=true
communication.protocolrev_1=2

powershell: delete specific line from x to x

I'm new in powershell and I absolutely dont get it ...
Just want to delete line 7 to 2500 of a text file. First 6 lines should be untouched.
With linux bash everything is so easy, just:
sed -i '7,2500d' $file
Did not find any solution for mighty powershell :-(
Thank you.
Use Get-Content to read the contents of the file into a variable. The variable can be indexed like a regular PowerShell array. Get the parts of the array you need then pipe the variable into Set-Content to write back to the file.
$file = Get-Content test.log
$keep = $file[0..1] + $file[7..($file.Count - 1)]
$keep | Set-Content test.log
Using this as the contents of the file test.log:
One
Two
Three
Four
Five
Six
Seven
Eight
Nine
This script will output the following into test.log (overwriting the contents):
One
Two
Eight
Nine
In your case, you will want to use $file[0..5] + $file[2500..($file.Count - 1)].
To remove a series of lines in a text file, you could do something like this:
$fileIn = 'D:\Test\File1.txt'
$fileOut = 'D:\Test\File2.txt'
$startRemove = 7
$endRemove = 2500
$currentLine = 1
# needs .NET 4
$newText = foreach ($line in [System.IO.File]::ReadLines($fileIn)) {
if ($currentLine -lt $startRemove -or $currentLine -gt $endRemove) { $line}
$currentLine++
}
$newText | Set-Content -Path $fileOut -Force
Or, if your version of .NET is below 4.0
$reader = [System.IO.File]::OpenText($fileIn)
$newText = while($null -ne ($line = $reader.ReadLine())) {
if ($currentLine -lt $startRemove -or $currentLine -gt $endRemove) { $line }
$currentLine++
}
$reader.Dispose()
$newText | Set-Content -Path $fileOut -Force
Select-object -index takes an array, so:
1..10 > file
(get-content file) | select -index (0..5) | set-content file
get-content file
1
2
3
4
5
6
Or:
(cat file)[0..5] | set-content file

Replace first duplicate without regex and increment

I have a text file and I have 3 of the same numbers somewhere in the file. I need to add incrementally to each using PowerShell.
Below is my current code.
$duped = Get-Content $file | sort | Get-Unique
while ($duped -ne $null) {
$duped = Get-Content $file | sort | Get-Unique | Select -Index $dupecount
$dupefix = $duped + $dupecount
echo $duped
echo $dupefix
(Get-Content $file) | ForEach-Object {
$_ -replace "$duped", "$dupefix"
} | Set-Content $file
echo $dupecount
$dupecount = [int]$dupecount + [int]"1"
}
Original:
12345678
12345678
12345678
Intended Result:
123456781
123456782
123456783
$filecontent = (get-content C:\temp\pos\bart.txt )
$output = $null
[int]$increment = 1
foreach($line in $filecontent){
if($line -match '12345679'){
$line = [int]$line + $increment
$line
$output += "$line`n"
$increment++
}else{
$output += "$line`n"
}
}
$output | Set-Content -Path C:\temp\pos\bart.txt -Force
This works in my test of 5 lines being
a word
12345679
a second word
12345679
a third word
the output would be :
a word
12345680
a second word
12345681
a third word
Let's see if i understand the question correctly:
You have a file with X-amount of lines:
a word
12345678
a second word
12345678
a third word
You want to catch each instance of 12345678 and add 1 increment to it so that it would become:
a word
12345679
a second word
12345679
a third word
Is that what you are trying to do?

Powershell Host File edit

Guys i'm having some issues converting my Perl script to powershell, I need some help. In the host file of our machines, we have all of the URL's to our test environments blocked. In my PERL script, based on which environment is selected, it will comment out the line of the environment selected to allow access and block others so the testers can't mistakenly do things in the wrong environment.
I need help converting to powershell
Below is what I have in PERL:
sub editHosts {
print "Editing hosts file...\n";
my $file = 'C:\\Windows\\System32\\Drivers\\etc\\hosts';
my $data = readFile($file);
my #lines = split /\n/, $data;
my $row = '1';
open (FILE, ">$file") or die "Cannot open $file\n";
foreach my $line (#lines) {
if ($line =~ m/$web/) {
print FILE '#'."$line\n"; }
else {
if ($row > '21') {
$line =~ s/^\#*127\.0\.0\.1/127\.0\.0\.1/;
$line =~ s/[#;].*$//s; }
print FILE "$line\n"; }
$row++;
}
close(FILE);
}
Here is what i've tried in Powershell:
foreach ($line in get-content "C:\windows\system32\drivers\etc\hosts") {
if ($line -contains $web) {
$line + "#"
}
I've tried variation including set-content with what used to be in the host file, etc.
Any help would be appreciated!
Thanks,
Grant
-contains is a "set" operator, not a substring operator. Try .Contains() or -like.
This will comment out lines matching the variable $word, while removing # from non-matches (except the header):
function Edit-Hosts ([string]$Web, $File = "C:\windows\system32\drivers\etc\hosts") {
#If file exists and $web is not empty/whitespace
if((Test-Path -Path $file -PathType Leaf) -and $web.Trim()) {
$row = 1
(Get-Content -Path $file) | ForEach-Object {
if($_ -like "*$web*") {
#Matched PROD, comment out line
"#$($_)"
} else {
#No match. If past header = remove comment
if($row -gt 21) { $_ -replace '^#' } else { $_ }
}
$row++
} | Set-Content -Path $file
} else {
Write-Error -Category InvalidArgument -Message "'$file' doesn't exist or Web-parameter is empty"
}
}
Usage:
Edit-Hosts -Web "PROD"
This is a similar answer to Frode F.'s answer, but I'm not yet able to comment to add my 2c worth, so have to provide an alternative answer instead.
It looks like one of the gotchas moving from perl to PowerShell, in this example, is that when we get the content of the file using Get-Content it is an "offline" copy, i.e. any edits are not made directly to the file itself. One approach is to compile the new content to the whole file and then write that back to disk.
I suppose that the print FILE "some text\n"; construct in perl might be similar to "some text" | Out-File $filename -Encoding ascii -Append in PowerShell, albeit you would use the latter either (1) to write line-by-line to a new/empty file or (2) accept that you are appending to existing content.
Two other things about editing the hosts file:
Be sure to make sure that your hosts file is ASCII encoded; I have caused a major outage for a key enterprise application (50k+ users) in learning that...
You may need to remember to run your PowerShell / PowerShell ISE by right-clicking and choosing Run as Administrator else you might not be able to modify the file.
Anyway, here's a version of the previous answer using Out-File:
$FileName = "C:\windows\system32\drivers\etc\hosts"
$web = "PROD"
# Get "offline" copy of file contents
$FileContent = Get-Content $FileName
# The following creates an empty file and returns a file
# object (type [System.IO.FileInfo])
$EmptyFile = New-Item -Path $FileName -ItemType File -Force
foreach($Line in $FileContent) {
if($Line -match "$web") {
"# $Line" | Out-File $EmptyFile -Append -Encoding ascii
} else {
"$Line" | Out-File $EmptyFile -Append -Encoding ascii
}
}
Edit
The ($Line -match "$web") takes whatever is in the $web variable and treats it as a regular expression. In my example I was assuming that you were just wanting to match a simple text string, but you might well be trying to match an IP address, etc. You have a couple of options:
Use ($Line -like "*$web*") instead.
Convert what is in $web to be an escaped regex, i.e. one that will match literally. Do this with ($Line -match [Regex]::Escape($web)).
You also wanted to strip off comments from any line past row 21 of the hosts file, should that line not match $web. In perl you have used the s substitution operator; the PowerShell equivalent is -replace.
So... here is an updated version of that foreach loop:
$LineCount = 1
foreach($Line in $FileContent) {
if($Line -match [Regex]::Escape($web) {
# ADD comment to any matched line
$Line = "#" + $Line
} elseif($LineCount -gt 21) {
# Uncomment the other lines
$Line = $Line -replace '^[# ]+',''
}
# Remove 'stacked up' comment characters, if any
$Line = $Line -replace '[#]+','#'
$Line | Out-File $EmptyFile -Append -Encoding ascii
$LineCount++
}
More Information
Are there good references for moving from Perl to Powershell?
How to use operator '-replace' in PowerShell to replace strings of texts with special characters and replace successfully
about_Comparison_Operators
http://www.comp.leeds.ac.uk/Perl/sandtr.html
If you wanted to verify what was in there and then add entries, you could use the below which is designed to be ran interactively and returns any existing entries you specify in the varibles:
Note: the `t is powershell's in script method for 'Tab' command.
$hostscontent
# Script to Verify and Add Host File Entries
$hostfile = gc 'C:\Windows\System32\drivers\etc\hosts'
$hostscontent1 = $hostfile | select-string "autodiscover.XXX.co.uk"
$hostscontent2 = $hostfile | select-string "webmail.XXX.co.uk"
$1 = "XX.XX.XXX.XX`tautodiscover.XXX.co.uk"
$2 = "webmail.XXX.co.uk"
# Replace this machines path with a path to your list of machines e.g. $machines = gc \\machine\machines.txt
$machines = gc 'c:\mytestmachine.txt'
ForEach ($machine in $machines) {
If ($hostscontent1 -ne $null) {
Start-Sleep -Seconds 1
Write-Host "$machine Already has Entry $1" -ForegroundColor Green
} Else {
Write-Host "Adding Entry $1 for $machine" -ForegroundColor Green
Start-Sleep -Seconds 1
Add-Content -Path C:\Windows\System32\drivers\etc\hosts -Value "XX.XX.XXX.XX`tautodiscover.XXX.co.uk" -Force
}
If ($hostscontent2 -ne $null) {
Start-Sleep -Seconds 1
Write-Host "$machine Already has Entry $2" -ForegroundColor Green
} Else {
Write-Host "Adding Entry $2 for $machine" -ForegroundColor Green
Start-Sleep -Seconds 1
Add-Content -Path C:\Windows\System32\drivers\etc\hosts -Value "XX.XX.XXX.XX`twebmail.XXX.co.uk" -Force
}
}

Powershell to count columns in a file

I need to test the integrity of file before importing to SQL.
Each row of the file should have the exact same amount of columns.
These are "|" delimited files.
I also need to ignore the first line as it is garbage.
If every row does not have the same number of columns, then I need to write an error message.
I have tried using something like the following with no luck:
$colCnt = "c:\datafeeds\filetoimport.txt"
$file = (Get-Content $colCnt -Delimiter "|")
$file = $file[1..($file.count - 1)]
Foreach($row in $file){
$row.Count
}
Counting rows is easy. Columns is not.
Any suggestions?
Yep, read the file skipping the first line. For each line split it on the pipe, and count the results. If it isn't the same as the previous throw an error and stops.
$colCnt = "c:\datafeeds\filetoimport.txt"
[int]$LastSplitCount = $Null
Get-Content $colCnt | ?{$_} | Select -Skip 1 | %{if($LastSplitCount -and !($_.split("|").Count -eq $LastSplitCount)){"Process stopped at line number $($_.psobject.Properties.value[5]) for column count mis-match.";break}elseif(!$LastSplitCount){$LastSplitCount = $_.split("|").Count}}
That should do it, and if it finds a bad column count it will stop and output something like:
Process stopped at line number 5 for column count mis-match.
Edit: Added a Where catch to skip blank lines ( ?{$_} )
Edit2: Ok, if you know what the column count should be then this is even easier.
Get-Content $colCnt | ?{$_} | Select -Skip 1 | %{if(!($_.split("|").Count -eq 210)){"Process stopped at line number $($_.psobject.Properties.value[5]), incorrect column count of: $($_.split("|").Count).";break}}
If you want it to return all lines that don't have 210 columns just remove the ;break and let it run.
A more generic approach, including a RegEx filter:
$path = "path\to\folder"
$regex = "regex"
$expValue = 450
$files= Get-ChildItem $path | Where-Object {$_.Name -match $regex}
Foreach( $f in $files) {
$filename = $f.Name
echo $filename
$a = Get-Content $f.FullName;
$i = 1;
$e = 0;
echo "Starting...";
foreach($line in $a)
{
if ($line.length -ne $expValue){
echo $filename
$a | Measure-Object -Line
echo "Long:"
echo $line.Length;
echo "Line NÂș: "
echo $i;
$e = $e + 1;
}
$i = $i+1;
}
echo "Finished";
if ($e -ne 0){
echo $e "errors found";
}else{
echo "No errors"
echo ""
}
}
echo "All files examined"
Another possibility:
$colCnt = "c:\datafeeds\filetoimport.txt"
$DataLine = (Get-Content $colCnt -TotalCount 2)[1]
$DelimCount = ([char[]]$DataLine -eq '|').count
$MatchString = '.*' + ('|.*' * $DelimCount )
$test = Select-String -Path $colCnt -Pattern $MatchString -NotMatch |
where { $_.linenumber -ne 1 }
That will find the number of delimiter characters in the second line, and build a regex pattern that can be used with Select-String.
The -NotMatch switch will make it return any lines that don't match that pattern as MatchInfo objects that will have the filename, line number and content of the problem lines.
Edit: Since the first line is "garbage" you probably don't care if it didn't match so I added a filter to the result to drop that out.