Add Content to a specific line in powershell - powershell

I have seen this post:
Add-Content - append to specific line
But I cannot add these lines because "Array index is out of range".
What my script is doing:
Find the line
Loop through the array that contains the data i want to add
$file[$line]+=$data
$line+=1
Write to file
Should I create a new file content and then add each line of the original file to it?
IF so, do you know how to do that and how to stop and add my data in between?
Here is the part of my code where I try to add:
$f=Get-Content $path
$ct=$begin+1 #$begin is the line where I want to place content under
foreach($add in $add_to_yaml)
{
$f[$ct]+=$add
$ct+=1
}
$f | Out-File -FilePath $file

Let's break down your script and try to analyze what's going on:
$f = Get-Content $path
Get-Content, by default, reads text files and spits out 1 string per individual line in the file. If the file found at $path has 10 lines, the resulting value stored in $f will be an array of 10 string values.
Worth noting is that array indices in PowerShell (and .NET in general) are zero-based - to get the 10th line from the file, we'd reference index 9 in the array ($f[9]).
That means that if you want to concatenate stuff to the end of (or "under") line 10, you need to specify index 9. For this reason, you'll want to change the following line:
$ct = $begin + 1 #$begin is the line where i want to place content under
to
$ct = $begin
Now that we have the correct starting offset, let's look at the loop:
foreach($add in $add_to_yaml)
{
$f[$ct] += $add
$ct += 1
}
Assuming $add_to_yaml contains multiple strings, the loop body will execute more than once. Let's take a look at the first statement:
$f[$ct] += $add
We know that $f[$ct] resolves to a string - and strings have the += operator overloaded to mean "string concatenation". That means that the string value stored in $f[$ct] will be modified (eg. the string will become longer), but the array $f itself does not change its size - it still contains the same number of strings, just one of them is a little longer.
Which brings us to the crux of your issue, this line right here:
$ct += 1
By incrementing the index counter, you effectively "skip" to the next string for every value in $add_to_yaml - so if the number of elements you want to add exceeds the number of lines after $begin, you naturally reach a point "beyond the bounds" of the array before you're finished.
Instead of incrementing $ct, make sure you concatenate your new string values with a newline sequence:
$f[$ct] = $f[$ct],$add -join [Environment]::Newline
Putting it all back together, you end up with something like this (notice we can discard $ct completely, since its value is constant an equal to $begin anyway):
$f = Get-Content $path
foreach($add in $add_to_yaml)
{
$f[$begin] = $f[$begin],$add -join [Environment]::Newline
}
But wait a minute - all the strings in $add_to_yaml are simply going to be joined by newlines - we can do that in a single -join operation and get rid of the loop too!
$f = Get-Content $path
$f[$begin] = #($f[$begin];$add_to_yaml) -join [Environment]::Newline
$f | Out-File -FilePath $file
Much simpler :)

Related

Powershell Script needed to redact items in between separators in a row according to defined list

Hello and thanks for reading. It's my first post and really need some help. The hardest part is getting my question across in a way that people will understand. I will try my best.
I have some huge csv files (some in excess of 8 millions rows so Excel not an option really) where I need to modify the contents of the 3rd 'field' in each row according to sets of words defined in a reference file
So an example csv might be something like:
AB12|TEST|CAT DOG MOUSE|TEST1|TEST2|TEST3||TEST4
CD34|TEST|HORSE CART TRAIN|TEST1|TEST2|TEST3||TEST4
etc etc.
In my reference file I have a list eg:
CAT
HORSE CART
These are contained in a CSV
What I need is to modify the files so that the 3rd 'field' (everything after the 2nd'|' and before the 3rd '|' is compared to the reference list and modified to match. ie in the first line, everything after CAT would be deleted and in the second line, everything after HORSE CART would be deleted within this 3rd field. So the resultant file outputted would look like:
AB12|TEST|CAT|TEST1|TEST2|TEST3||TEST4
CD34|TEST|HORSE CART|TEST1|TEST2|TEST3||TEST4
I normally use F.A.R.T to modify large files, but this needs to be a bit more clever than FART is able to offer.
I really hope this makes sense to someone out there and appreciate any help you might offer.
So far I have been experimenting with this, but it's a long way off doing what I want:
cls
$content = ""
write-output "** Original String **"
write-output ""
$content = Get-Content "~\Desktop\Test\*.dat"
$content
$separator1 = " "
$separator2 = "|"
$parts = $content.split($separator1)
write-output ""
write-output "** Revised String **"
write-output ""
$part1 = echo $parts[0]
$part3 = $part2.split($separator2)
$part4 = $part3[1]
$revised = $part1, $part4 -join "|"
$revised
write-output ""
So in summary then: This is really a modified 'Find and Replace Text' function that concentrates on a single field in each line, looks for matching sets of words, then deletes everything in that field other than the matched words, which are defined in a separate csv file.
Ok, as comparing arrays in PowerShell doesn't support wild cards we have to do this the old fashioned (costly) way. Compare every field with every reference.
I've not provided an example of reading the file as that can be done in different ways with regards to speed or memory consumption (your choice).
Also, I've provided the reference as an array instead as a file input to keep the example to the point (and easily testable).
The output should then of course be done to a new file instead of written to the host.
$file = #"
F1|F2|F3|F4|F5|F6|F7|F8
AB12|TEST|CAT DOG MOUSE|TEST1|TEST2|TEST3||TEST4
CD34|TEST|HORSE CART TRAIN|TEST1|TEST2|TEST3||TEST4
CD34|TEST|HORSE CART|TEST1|TEST2|TEST3||TEST4
"#
$ref = #("CAT*","HORSE CART*")
$file.split("`n") | foreach {# line in file
$outline = $nul
$_.split('|') | foreach {# field in the line
$field = $_
$refhit = $false
$ref | foreach {# item in the ref array
if ($field -like $_) {# replace field with ref
$refhit = $true
$outline += $_.TrimEnd('*') + '|'
}# end match
}# end ref
if (!$refhit){#pass on the field as is
$outline += "$field|"
}
}#end field
# Output filtered line
write-host $outline.TrimEnd('|')
}#end line

stop script when the target (find and replace text) is reached 1 or 2 or 3 times depending on config

What if some file contains strings like
11111111111111
22222222222222
33333333333333
44444444444444
22222222222222
33333333333333
22222222222222
11111111111111
and I need to find and replace 22222222222222 to 777777777777777 just 2 times while processing the file string by string using foreach and save file with changes after that.
$file = 'path_to\somefile.txt'
foreach($string in $file)
{
$new_string = $string -replace '22222222222222', '777777777777777'
}
$string | Out-file $file
I understand that the script above will replace all strings and will be saved just with one string instead of my requirements.
As per the comment, one needs to keep track how many replacements are done.
To read a file line by line, use .Net's StreamReader.ReadLine(). In a while loop, keep reading until you are at end of file.
While reading lines, there are a few things. Keep track how many times the replaceable string is encountered, and, if need be, replace it. The results must be saved in both cases: a line that was replaced and all the other lines too.
It's slow to add content into a file on line by line cases, even on the days of SSDs. A bulk operation is much more efficient. Save the modified data into a StringBuilder. After the whole file is processed, write the contents on a single operation. If the file's going to be large (one gigabyte or more), consider writing 10 000 rows a time. Like so,
$sb = New-Object Text.StringBuilder
$reader = [IO.File]::OpenText("MyFile.csv")
$found = 0
$oldVal = '22222222222222'
$newVal = '777777777777777'
# Read the file line by line
while($null -ne ($line = $reader.ReadLine())) {
# Only change two first occurrances
if( ($found -lt 2) -and ($line -eq $oldVal) ) {
++$found
$line = $newVal
}
# Add data into a buffer
[void]$sb.AppendLine($line)
}
add-content "SomeOtherFile.csv" $sb.ToString()
$reader.close()

Powershell; Trying to get .replace or -replace to loop through a single file and replace the same single line with something different each time

Every time I try to run this it stops at the first position and only replaces one time when I need it to replace the same line each time.
$kp3 = get-content -path .\Desktop\name.txt
$kp4 = get-content -path .\Desktop\source1.txt
$wk =#("wk","fa","fav")
for ($i=0;$i -le 3;$i++){
$te = $kp3[1] #this is the position im trying to replace.
$kp2 = ".\Desktop\name.txt"
$name = $wk[$i]
(Get-Content $kp2).Replace($te,$name)|set-Content $kp2
}
My text file should have the "fav" in it because it is last in the array but instead it is stopping at "fa"
What you are trying to replace is never updated in your loop. We can't see what's in your file but I'm surprised it's getting to 'fa'.
You are changing the content of $kp2, but reading what to replace from $kp3, which is never updated.

How can I loop through each record of a text file to replace a string of characters

I have a large .txt file containing records where a date string in each record needs to be incremented by 2 days which will then update the field to the right of it which contains dashes --------- with that date. For example, a record contains the following record data:
1440149049845_20191121000000 11/22/2019 -------- 0.000 0.013
I am replacing the -------- dashes with 11/24/2019 (2 days added to the date 11/22/2019) so that it shows as:
1440149049845_20191121000000 11/22/2019 11/24/2019 0.000 0.013
I have the replace working on a single record but need to loop through the entire .txt file to update all of the records. Here is what I tried:
$inputRecords = get-content '\\10.12.7.13\vipsvr\Rancho\MRDF_Report\_Report.txt'
foreach ($line in $inputRecords)
{
$item -match '\d{2}/\d{2}/\d{4}'
$inputRecords -replace '-{2,}',([datetime]$matches.0).adddays(2).tostring('MM/dd/yyyy') -replace '\b0\.000\b','0.412'
}
I get an PS error stating: "Cannot convert null to type "System.DateTime"
I'm sorry but why are we using RegEx for something this simple?
I can see it if there are differently formatted lines in the file, you'd want to make sure you aren't manipulating unintended lines, but that's not indicated in the question. Even still, it doesn't seem like you need to match anything within the line itself. It seems like it's delimited on spaces which would make a simple split a lot easier.
Example:
$File = "C:\temp\Test.txt"
$Output =
ForEach( $Line in Get-Content $File)
{
$TmpArray = $Line.Split(' ')
$TmpArray[2] = (Get-Date $TmpArray[1]).AddDays(2).ToString('M/dd/yyyy')
$TmpArray -join ' '
}
The 3rd element in the array do the calculation and reassign the value...
Notice there's no use of the += operator which is very slow compared to simply assigning the output to a variable. I wouldn't make a thing out of it but considering we don't know how big the file is... Also the String format given before 'mm/dd/yyyy' will result in 00 for the month like for example '00/22/2019', so I changed that to 'M/dd/yyyy'
You can still add logic to skip unnecessary lines if it's needed...
You can send $Output to a file with something like $Output | Out-File <FilePath>
Or this can be converted to a single pipeline that outputs directly to a file using | ForEach{...} instead of ForEach(.. in ..) If the file is truly huge and holding $Output in memory is an issue this is a good alternative.
Let me know if that helps.
You mostly had the right idea, but here are a few suggested changes, but not exactly in this order:
Use a new file instead of trying to replace the old file.
Iterate a line at a time, replace the ------, write to the new file.
Use '-match' instead of '-replace', because as you will see below that you need to manipulate the capture more than a simple '-replace' allows.
Use [datetime]::parseexact instead of trying to just force cast the captured text.
[string[]]$inputRecords = get-content ".\linesource.txt"
[string]$outputRecords
foreach ($line in $inputRecords) {
[string]$newLine = ""
[regex]$logPattern = "^([\d_]+) ([\d/]+) (-+) (.*)$"
if ($line -match $logPattern) {
$origDate = [datetime]::parseexact($Matches[2], 'mm/dd/yyyy', $null)
$replacementDate = $origDate.adddays(2)
$newLine = $Matches[1]
$newLine += " " + $origDate.toString('mm/dd/yyyy')
$newLine += " " + $replacementDate.toString('mm/dd/yyyy')
$newLine += " " + $Matches[4]
} else {
$newLine = $line
}
$outputRecords += "$newLine`n"
}
$outputRecords.ToString()
Even if you don't use the whole solution, hopefully at least parts of it will be helpful to you.
Using the suggested code from adamt8 and Steven, I added to 2 echo statements to show what gets displayed in the variables $logpattern and $line since it is not recognizing the pattern of characters to be updated. This is what displays from the echo:
Options MatchTimeout RightToLeft
CalNOD01 1440151020208_20191205000000 12/06/2019 12/10/2019
None -00:00:00.0010000 False
CalNOD01 1440151020314_20191205000000 12/06/2019 --------
None -00:00:00.0010000 False
this is the rendered output:
CalNOD01 1440151020208_20191205000000 12/06/2019 12/10/2019
CalNOD01 1440151020314_20191205000000 12/06/2019 --------
This is the code that was used:
enter image description here

Question regarding incrementing a string value in a text file using Powershell

Just beginning with Powershell. I have a text file that contains the string "CloseYear/2019" and looking for a way to increment the "2019" to "2020". Any advice would be appreciated. Thank you.
If the question is how to update text within a file, you can do the following, which will replace specified text with more specified text. The file (t.txt) is read with Get-Content, the targeted text is updated with the String class Replace method, and the file is rewritten using Set-Content.
(Get-Content t.txt).Replace('CloseYear/2019','CloseYear/2020') | Set-Content t.txt
Additional Considerations:
General incrementing would require a object type that supports incrementing. You can isolate the numeric data using -split, increment it, and create a new, joined string. This solution assumes working with 32-bit integers but can be updated to other numeric types.
$str = 'CloseYear/2019'
-join ($str -split "(\d+)" | Foreach-Object {
if ($_ -as [int]) {
[int]$_ + 1
}
else {
$_
}
})
Putting it all together, the following would result in incrementing all complete numbers (123 as opposed to 1 and 2 and 3 individually) in a text file. Again, this can be tailored to target more specific numbers.
$contents = Get-Content t.txt -Raw # Raw to prevent an array output
-join ($contents -split "(\d+)" | Foreach-Object {
if ($_ -as [int]) {
[int]$_ + 1
}
else {
$_
}
}) | Set-Content t.txt
Explanation:
-split uses regex matching to split on the matched result resulting in an array. By default, -split removes the matched text. Creating a capture group using (), ensures the matched text displays as is and is not removed. \d+ is a regex mechanism matching a digit (\d) one or more (+) successive times.
Using the -as operator, we can test that each item in the split array can be cast to [int]. If successful, the if statement will evaluate to true, the text will be cast to [int], and the integer will be incremented by 1. If the -as operator is not successful, the pipeline object will remain as a string and just be output.
The -join operator just joins the resulting array (from the Foreach-Object) into a single string.
AdminOfThings' answer is very detailed and the correct answer.
I wanted to provide another answer for options.
Depending on what your end goal is, you might need to convert the date to a datetime object for future use.
Example:
$yearString = 'CloseYear/2019'
#convert to datetime
[datetime]$dateConvert = [datetime]::new((($yearString -split "/")[-1]),1,1)
#add year
$yearAdded = $dateConvert.AddYears(1)
#if you want to display "CloseYear" with the new date and write-host
$out = "CloseYear/{0}" -f $yearAdded.Year
Write-Host $out
This approach would allow you to use $dateConvert and $yearAdded as a datetime allowing you to accurately manipulate dates and cultures, for example.