Check if the last line in a file is empty in PowerShell - powershell

I want to add some content to a file in a new line.
But add-content appends a string to last line if there is no new line symbol at the end.
E.g. if I want to add AAA string and if I have a file file1.txt
my last line(last cursor position here)
the result will be
my last lineAAA
On the other hand, if I use file2.txt
my last line
(last cursor position here)
the command will result in
my last line
AAA
So I need to check if the last line is empty or not. If it's not empty I will just add `n symbol to the string.
But if I run the commands
$lastLine = get-content $filename -Tail 1
if($lastLine.Length -ne 0) { ... }
it will always return me a length of not empty string even if my last line contains no symbols.
How can I check if my last line is empty ?

You could opt to start adding newlines to the file and for the first line to add do
$file = 'D:\Test\Blah.txt'
$firstLine = 'AAA'
Add-Content -Path $file -Value ("{0}`r`n{1}" -f (Get-Content -Path $file -Raw).TrimEnd(), $firstLine)
After that first line, you can simply keep using Add-Content which always appends an newline (unless you tell it not to do that with switch -NoNewline).
Seeing your comment, you can test the length of the last line like this:
$file = 'D:\Test\Blah.txt'
$lastLine = ((Get-Content -Path $file -Raw) -split '\r?\n')[-1]
# $lastLine.Length --> 0
if($lastLine.Length -ne 0) { ... }
The -Raw switch tells Get-Content to read the file as a whole in a single string. Split this string into separate lines with -split '\r?\n' and you'll get an array, including the last empty line

When you use "Get-Content -Tail 1", it will always recover the last "non empty" line.
# -----------------
# Your method returns the same line even if the file contains an empty line at the end of the file
# -----------------
$lastEmptyLine = Get-Content "test_EmptyLine.txt" -Tail 1
$lastNonEmptyLine = Get-Content "test_NonEmptyLine.txt" -Tail 1
($lastEmptyLine -match '(?<=\r\n)\z')
#False
($lastNonEmptyLine -match '(?<=\r\n)\z')
#False
So if you want to keep the "Test" method (and not simply use Add-Content) you could use the following method :
# -----------------
# This method can tell you if a file finishes by an empty line or not
# -----------------
$contentWithEmptyLine = [IO.File]::ReadAllText("test_EmptyLine.txt")
$contentWithoutEmptyLine = [IO.File]::ReadAllText("test_NonEmptyLine.txt")
($contentWithEmptyLine -match '(?<=\r\n)\z')
#True
($contentWithoutEmptyLine -match '(?<=\r\n)\z')
#False
# -----------------
# You can also use Get-Content with Raw option
# -----------------
$rawContentWithEmptyLine = Get-Content "test_EmptyLine.txt" -Raw
$rawContentWithoutEmptyLine = Get-Content "test_NonEmptyLine.txt" -Raw
($rawContentWithEmptyLine -match '(?<=\r\n)\z')
#True
($rawContentWithoutEmptyLine -match '(?<=\r\n)\z')
#False
-Raw Ignores newline characters and returns the entire contents of a file in one string with the newlines preserved. By default, newline
characters in a file are used as delimiters to separate the input into
an array of strings. This parameter was introduced in PowerShell 3.0.
References :
Get-Content (Microsoft.PowerShell.Management)
Check CRLF at the end of every file
about_Comparison_Operators - PowerShell
Regular expression - Wikipedia

Related

Using Powershell to copy and replace content from one file to another

I have two files: FileA and FileB, they are nearly identical.
Both files have a section which starts with ////////// MAIN \\\\\\\\\\. I need to replace the whole content from this point until the end of the file.
So the process high level looks like:
find content (starting with ////////// MAIN \\\\\\\\\\) until the end of the file in FileA and copy it to clipboard
find content (starting with ////////// MAIN \\\\\\\\\\) until the end of the file in FileB and replace it with the content from the clipboard
How do I do this?
I understand that it would look like this (found it online) but I'm missing the pattern and logic I can use for selecting the text until the end of the file:
# FileA
$inputFileA = "C:\fileA.txt"
# Text to be inserted
$inputFileB = "C:\fileB.txt"
# Output file
$outputFile = "C:\fileC.txt"
# Find where the last </location> tag is
if ((Select-String -Pattern "\</location\>" -Path $inputFileA |
select -last 1) -match ":(\d+):")
{
$insertPoint = $Matches[1]
# Build up the output from the various parts
Get-Content -Path $inputFileA | select -First $insertPoint | Out-File $outputFile
Get-Content -Path $inputFileB | Out-File $outputFile -Append
Get-Content -Path $inputFileA | select -Skip $insertPoint | Out-File $outputFile -Append
}
You could do that in two lines of code:
# first write the top part including the '////////// MAIN \\\\\\\\\\' from FileB to the new file
((Get-Content -Path "D:\Test\fileB.txt" -Raw) -split '(?<=/+ MAIN \\+\r?\n)', 2)[0] | Set-Content -Path "D:\Test\fileC.txt" -NoNewline
# then append the bottom part excluding the '////////// MAIN \\\\\\\\\\' from FileA to the new file
((Get-Content -Path "D:\Test\fileA.txt" -Raw) -split '/+ MAIN \\+\r?\n', 2)[-1] | Add-Content -Path "D:\Test\fileC.txt"
Regex details:
(?<= # Assert that the regex below can be matched, with the match ending at this position (positive lookbehind)
/ # Match the character “/” literally
+ # Between one and unlimited times, as many times as possible, giving back as needed (greedy)
\ MAIN\ # Match the characters “ MAIN ” literally
\\ # Match the character “\” literally
+ # Between one and unlimited times, as many times as possible, giving back as needed (greedy)
\r # Match a carriage return character
? # Between zero and one times, as many times as possible, giving back as needed (greedy)
\n # Match a line feed character
)
Or, if the files are quite large:
# first write the top part including the '////////// MAIN \\\\\\\\\\' from FileB to the new file
$copyThis = $true
$content = switch -Regex -File "D:\Test\fileB.txt" {
'/+ MAIN \\+' { $copyThis = $false; $_ ; break}
default { if ($copyThis) { $_ } }
}
$content | Set-Content -Path "D:\Test\fileC.txt"
# then append the bottom part excluding the '////////// MAIN \\\\\\\\\\' from FileA to the new file
$copyThis = $false
$content = switch -Regex -File "D:\Test\fileA.txt" {
'/+ MAIN \\+' { $copyThis = $true }
default { if ($copyThis) { $_ } }
}
$content | Add-Content -Path "D:\Test\fileC.txt"

Re-assembling split file names with Powershell

I'm having trouble re-assembling certain filenames (and discarding the rest) from a text file. The filenames are split up (usually on three lines) and there is always a blank line after each filename. I only want to keep filenames that begin with OPEN or FOUR. An example is:
OPEN.492820.EXTR
A.STANDARD.38383
333
FOUR.383838.282.
STAND.848484.NOR
MAL.3939
CLOSE.3480384.ST
ANDARD.39393939.
838383
The output I'd like would be:
OPEN.492820.EXTRA.STANDARD.38383333
FOUR.383838.282.STAND.848484.NORMAL.3939
Thanks for any suggestions!
The following worked for me, you can give it a try.
See https://regex101.com/r/JuzXOb/1 for the Regex explanation.
$source = 'fullpath/to/inputfile.txt'
$destination = 'fullpath/to/resultfile.txt'
[regex]::Matches(
(Get-Content $source -Raw),
'(?msi)^(OPEN|FOUR)(.*?|\s*?)+([\r\n]$|\z)'
).Value.ForEach({ -join($_ -split '\r?\n').ForEach('Trim') }) |
Out-File $destination
For testing:
$txt = #'
OPEN.492820.EXTR
A.STANDARD.38383
333
FOUR.383838.282.
STAND.848484.NOR
MAL.3939
CLOSE.3480384.ST
ANDARD.39393939.
838383
OPEN.492820.EXTR
A.EXAMPLE123
FOUR.383838.282.
STAND.848484.123
ZXC
'#
[regex]::Matches(
$txt,
'(?msi)^(OPEN|FOUR)(.*?|\s*?)+([\r\n]$|\z)'
).Value.ForEach({ -join($_ -split '\r?\n').ForEach('Trim') })
Output:
OPEN.492820.EXTRA.STANDARD.38383333
FOUR.383838.282.STAND.848484.NORMAL.3939
OPEN.492820.EXTRA.EXAMPLE123
FOUR.383838.282.STAND.848484.123ZXC
Read the file one line at a time and keep concatenating them until you encounter a blank line, at which point you output the concatenated string and repeat until you reach the end of the file:
# this variable will keep track of the partial file names
$fileName = ''
# use a switch to read the file and process each line
switch -Regex -File ('path\to\file.txt') {
# when we see a blank line...
'^\s*$' {
# ... we output it if it starts with the right word
if($s -cmatch '^(OPEN|FOUR)'){ $fileName }
# and then start over
$fileName = ''
}
default {
# must be a non-blank line, concatenate it to the previous ones
$s += $_
}
}
# remember to check and output the last one
if($s -cmatch '^(OPEN|FOUR)'){
$fileName
}

Powershell - Count number of carriage returns line feed in .txt file

I have a large text file (output from SQL db) and I need to determine the row count. However, since the source SQL data itself contains carriage returns \r and line feeds \n (NEVER appearing together), the data for some rows spans multiple lines in the output .txt file. The Powershell I'm using below gives me the file line count which is greater than the actual SQL row count. So I need to modify the script to ignore the additional lines - one way of doing it might be just counting the number of times CRLF or \r\n occurs (TOGETHER) in the file and that should be the actual number of rows but I'm not sure how to do it.
Get-ChildItem "." |% {$n = $_; $c = 0; Get-Content -Path $_ -ReadCount 1000 |% { $c += $_.Count }; "$n; $c"} > row_count.txt
I just learned myself that the Get-Content splits and streams each lines in a file by CR, CRLF, and LF sothat it can read data between operating systems interchangeably:
"1`r2`n3`r`n4" | Out-File .\Test.txt
(Get-Content .\Test.txt).Count
4
Reading the question again, I might have misunderstood your question.
In any case, if you want to split (count) on only a specific character combination:
CR
((Get-Content -Raw .\Test.txt).Trim() -Split '\r').Count
3
LF
((Get-Content -Raw .\Test.txt).Trim() -Split '\n').Count
3
CRLF
((Get-Content -Raw .\Test.txt).Trim() -Split '\r\n').Count # or: -Split [Environment]::NewLine
2
Note .Trim() method which removes the extra newline (white spaces) at the end of the file added by the Get-Content -Raw parameter.
Addendum
(Update based on the comment on the memory exception)
I am afraid that there is currently no other option then building your own StreamReader using the ReadBlock method and specifically split lines on a CRLF. I have opened a feature request for this issue: -NewLine Parameter to customize line separator for Get-Content
Get-Lines
A possible way to workaround the memory exception errors:
function Get-Lines {
[CmdletBinding()][OutputType([string])] param(
[Parameter(ValueFromPipeLine = $True)][string] $Filename,
[String] $NewLine = [Environment]::NewLine
)
Begin {
[Char[]] $Buffer = new-object Char[] 10
$Reader = New-Object -TypeName System.IO.StreamReader -ArgumentList (Get-Item($Filename))
$Rest = '' # Note that a multiple character newline (as CRLF) could be split at the end of the buffer
}
Process {
While ($True) {
$Length = $Reader.ReadBlock($Buffer, 0, $Buffer.Length)
if (!$length) { Break }
$Split = ($Rest + [string]::new($Buffer[0..($Length - 1)])) -Split $NewLine
If ($Split.Count -gt 1) { $Split[0..($Split.Count - 2)] }
$Rest = $Split[-1]
}
}
End {
$Rest
}
}
Usage
To prevent the memory exceptions it is important that you do not assign the results to a variable or use brackets as this will stall the PowerShell PowerShell pipeline and store everything in memory.
$Count = 0
Get-Lines .\Test.txt | ForEach-Object { $Count++ }
$Count
The System.IO.StreamReader.ReadBlock solution that reads the file in fixed-size blocks and performs custom splitting into lines in iRon's helpful answer is the best choice, because it both avoids out-of-memory problems and performs well (by PowerShell standards).
If performance in terms of execution speed isn't paramount, you can take advantage of
Get-Content's -Delimiter parameter, which accepts a custom string to split the file content by:
# Outputs the count of CRLF-terminated lines.
(Get-Content largeFile.txt -Delimiter "`r`n" | Measure-Object).Count
Note that -Delimiter employs optional-terminator logic when splitting: that is, if the file content ends in the given delimiter string, no extra, empty element is reported at the end.
This is consistent with the default behavior, where a trailing newline in a file is considered an optional terminator that does not resulting in an additional, empty line getting reported.
However, in case a -Delimiter string that is unrelated to newline characters is used, a trailing newline is considered a final "line" (element).
A quick example:
# Create a test file without a trailing newline.
# Note the CR-only newline (`r) after 'line 1'
"line1`rrest of line1`r`nline2" | Set-Content -NoNewLine test1.txt
# Create another test file with the same content plus
# a trailing CRLF newline.
"line1`rrest of line1`r`nline2`r`n" | Set-Content -NoNewLine test2.txt
'test1.txt', 'test2.txt' | ForEach-Object {
"--- $_"
# Split by CRLF only and enclose the resulting lines in [...]
Get-Content $_ -Delimiter "`r`n" |
ForEach-Object { "[{0}]" -f ($_ -replace "`r", '`r') }
}
This yields:
--- test1.txt
[line1`rrest of line1]
[line2]
--- test2.txt
[line1`rrest of line1]
[line2]
As you can see, the two test files were processed identically, because the trailing CRLF newline was considered an optional terminator for the last line.

Replace a non-unique line of text under a unique line of text in a text file using powershell

I have the following txt file.
[AppRemover]
Enable=0
[CleanWipe]
Enable=0
[RerunSetup]
Enable=0
How do I change the Enable=0 to Enable=1 under [CleanWipe] only?
Below is how I plan on using the code with my file.
$Path = C:\temp\file.txt
$File = Get-Content -Path $Path
# Code to update file
$File | Out-File $Path
You can use -replace to update the value if it is 0.
$Path = C:\temp\file.txt
(Get-Content $Path -Raw) -replace "(?<text>\[CleanWipe\]\r?\nEnable=)0",'${text}1' |
Set-Content $Path
Using a module that parses INI files will be the best solution though. I'd recommend trying PsIni.
Explanation:
The -Raw switch reads the file contents as a single string. This makes it easier to work with newline characters.
-replace performs a regex match and then replace. Below is the regex match breakdown.
(?<text>) is a named capture group. Anything matched within that capture group can be recalled in the replace string as '${text}'.
\[CleanWipe\] is a literal match of [CleanWipe] while escaping the [] characters with \.
\r? is optional carriage return
\n is the newline character
Enable= is a literal match
0 is a literal match
The replace string is the capture group contents and 1 when a match exists. Technically, a capture group is not needed if you want to use a positive lookbehind instead. The positive lookbehind assertion is (?<=). That solution would look like the following:
$Path = C:\temp\file.txt
(Get-Content $Path -Raw) -replace "(?<=\[CleanWipe\]\r?\nEnable=)0",'1' |
Set-Content $Path
The problem with the -replace solutions as they written is they will update the file regardless of a change actually being made to the contents. You would need to add an extra comparison to prevent that. Other issues could be extra white space on any of these lines. You can account for that by adding \s* where you think those possibilities may exist.
Alternative With More Steps:
$file = Get-Content $Path
$TargetIndex = $file.IndexOf('[CleanWipe]') + 1
if ($file[$TargetIndex] -match 'Enable=0') {
$file[$TargetIndex] = 'Enable=1'
$file | Set-Content $Path
}
This solution will only update the file if it meets the match condition. It uses the array method IndexOf() to determine where [CleanWipe] is. Then assumes the line you want to change is in the next index.
IndexOf() is not the only way to find an index. The method requires that your line match the string exactly. You can use Select-String (case-insensitive by default) to return a line number. Since it will be a line number and not an index (indexes start at 0 while line numbers start at 1), it will invariably be the index number you want.
$file = Get-Content $Path
$TargetIndex = ($file | Select-String -Pattern '[CleanWipe]' -SimpleMatch).LineNumber
if ($file[$TargetIndex] -match 'Enable=0') {
$file[$TargetIndex] = 'Enable=1'
$file | Set-Content $Path
}

how to add a new line in a config file after the last variable data set

I want to add a line to a series of cfg files in subfolders after a variable line.
some text ...
light.0 = some text
light.1 = some text
...
light.n = some text
... some text
Each text file has a varied nth data line.
All I want to add is the (n+1)th data line after those nth lines in each cfg files in subfolders.
light.(n+1) = some text
I want to carry out this task in PowerShell.
# Get all the config files, and loop over them
Get-ChildItem "d:\test" -Recurse -Include *.cfg | ForEach-Object {
# Output a progress message
Write-Host "Processing file: $_"
# Make a backup copy of the file, forcibly overwriting one if it's there
Copy-Item -LiteralPath $_ -Destination "$_+.bak" -Force
# Read the lines in the file
$Content = Get-Content -LiteralPath $_ -Raw
# A regex which matches the last "light..." line
# - line beginning with light.
# - with a number next (capture the number)
# - then equals, text up to the end of the line
# - newline characters
# - not followed by another line beginning with light
$Regex = '^light.(?<num>\d+) =.*?$(?![\r\n]+^light)'
# A scriptblock to calculate the regex replacement
# needs to output the line which was captured
# and calculat the increased number
# and output the new line as well
$ReplacementCalculator = {
param($RegexMatches)
$LastLine = $RegexMatches[0].Value
$Number = [int]$RegexMatches.groups['num'].value
$NewNumber = $Number + 1
"$LastLine`nlight.$NewNumber = some new text"
}
# Do the replacement and insert the new line
$Content = [regex]::Replace($Content, $Regex, $ReplacementCalculator, 'Multiline')
# Update the file with the new content
$Content | Set-Content -Path $_
}
(* I'm sure I read that somewhere)
Assumes the 'light' lines are contiguous with no blocks of other text in the middle, and that they are ordered, with the highest number being last. You might have to play with the line endings \r\n in the regex and `n in the replacement text, to make them match up.
(Sorry about the regex)