Replacing bunch of string characters in a text file using Powershell - powershell

Working on a PowerShell code which will replace a set of characters from a text file in a folder (Contain lot of Text files). Is there a way where it can do it for all the files in the folder?
The issue is it creates a new file when I run the code (New_NOV_1995.txt) but it doesn't change any characters in the new file.
$lookupTable = #{
'¿' = '|'
'Ù' = '|'
'À' = '|'
'Ú' = '|'
'³' = '|'
'Ä' = '-'
}
$original_file = 'C:\FilePath\NOV_1995.txt'
$destination_file = 'C:\FilePath\NOV_1995_NEW.txt'
Get-Content -Path $original_file | ForEach-Object {
$line = $_
$lookupTable.GetEnumerator() | ForEach-Object {
if ($line -match $_.Key)
{
$line = $line -replace $_.Key, $_.Value
}
}
$line
} | Set-Content -Path $destination_file

While something like this would work, performance might be a problem. My only testing was on a tiny file containing the $lookupTable.
$lookupTable = #{
'¿' = '|'
'Ù' = '|'
'À' = '|'
'Ú' = '|'
'³' = '|'
'Ä' = '-'
}
$original_file = 'C:\FilePath\NOV_1995.txt'
$destination_file = 'C:\FilePath\NOV_1995_NEW.txt'
$originalContent = Get-Content -Path $original_file
$lookupTable.GetEnumerator() | % {
$originalContent = $originalContent -replace $_.Key,$_.Value
}
$originalContent | Out-File -FilePath $destination_file

Your code as you have it there is actually working for me. There is still a possible encoding issue maybe with your files. Does your file look right when you just read it into the console with Get-Content $path? If the file does not look right you might need to play with the -Encoding switches of the
Set-Content and Get-Contentcmdlets.
Improving on your current logic.
I changed your $lookuptable to a pair of psobjects. Since you are making the same replacement for the most part I combined them into a single regex.
The next part I hummed and hawed about but since, after my proposed change, you are only doing two replacements I figure you could just chain the two into a single replacement line. Otherwise you could have a foreach-object in there but I think this is simpler and faster.
This way we don't need to test for a match. -replace is doing the testing for us.
$toPipe = [pscustomobject]#{
Pattern = '¿|Ù|À|Ú|³'
Replacement = "|"
}
$toHypen = [pscustomobject]#{
Pattern = 'Ä'
Replacement = "-"
}
$path = "c:\temp\test\test"
Get-ChildItem -Path $path | ForEach-Object{
(Get-Content $_.FullName) -replace $toPipe.Pattern,$toPipe.Replacement -replace $toHypen.Pattern,$toHypen.Replacement |
Set-Content $_.FullName
}
Note that this will change the original files. Testing is encouraged.
Set-Content and Get-Content are not the best when it comes to performance so you might need to consider using [IO.File]::ReadAllLines($file) and its partner static method [IO.File]::WriteAllLines($file)

Related

Replacing a keyword in a .txt file. Poweshell Scripting

I'm trying to replace a keyword in a .txt file, but nothing happens and the .txt file remains the same.
The keyword im trying to replace is "dataSource.password".
I'm trying to use -replace and Foreach-Object on replacing certain keywords
Here's the code:
$text = Get-Clipboard
$data = "dataSource.password="
$dt = $data + $text
$line1 = Get-content C:\Users\asd\Desktop\sample.txt | Select-String datasource.password | Select-Object -ExpandProperty Line
$content1 = Get-Content C:\Users\asd\Desktop\sample.txt
$content1 | ForEach-Object {$_ -replace $line1, $dt} | Set-Content C:\Users\asd\Desktop\sample.txt

Function to replace text in a file and save it

I am new programming, I am trying to improve the following code of a script, I was thinking of making a function to improve it but I do not know where to start or if it is the best option.
$replaceText01 = (Get-Content -path $copyFileLocation -Raw) -replace '"INSTANCENAME="TEST""',$NUEVAINESTANCIA
Set-Content $copyFileLocation $replaceText01
$replaceText02 = (Get-Content -path $copyFileLocation -Raw) -replace '"INSTANCEID="TEST""',$NUEVAINESTANCIAID
Set-Content $copyFileLocation $replaceText02
$replaceText03 = (Get-Content -path $copyFileLocation -Raw) -replace "NT Service\SQLAgent#TEST", $CUENTAAGTN
Set-Content $copyFileLocation $replaceText03
$replaceText04 = (Get-Content -path $copyFileLocation -Raw) -replace "NT Service\MSSQL#TEST", $CUENTASQLSER
Set-Content $copyFileLocation $replaceText04
$user = "$env:UserDomain\$env:USERNAME"
write-host $user
$replaceText = (Get-Content -path $copyFileLocation -Raw) -replace "##MyUser##", $user
Set-Content $copyFileLocation $replaceText
First off, I would probably try to read the file only once. Then since you are doing many similar operations, I would put all the data about the operations in an array, and then iterate over those data.
In this code, I first read the file. Then I define all the strings that should be replaced together with the strings that should replace them. Then I use a loop to iterate over the data so that we don't repeat the same code all the time.
$data = Get-Content -Path $copyFileLocation -Raw
$replacements = #(
#('"INSTANCENAME="TEST""', $NUEVAINESTANCIA),
#('"INSTANCEID="TEST""', $NUEVAINESTANCIAID),
#("NT Service\SQLAgent#TEST", $CUENTAAGTN),
#("NT Service\MSSQL#TEST", $CUENTASQLSER),
#("##MyUser##", "$env:UserDomain\$env:USERNAME")
)
$replacements | ForEach-Object {
$data = $data.Replace($_[0], $_[1])
}
Set-Content -Path $copyFileLocation -Value $data
It's also possible to get this even shorter if you use the pipeline instead of assigning the data to a variable
$data = Get-Content -Path $copyFileLocation -Raw
#(
#('"INSTANCENAME="TEST""', $NUEVAINESTANCIA),
#('"INSTANCEID="TEST""', $NUEVAINESTANCIAID),
#("NT Service\SQLAgent#TEST", $CUENTAAGTN),
#("NT Service\MSSQL#TEST", $CUENTASQLSER),
#("##MyUser##", "$env:UserDomain\$env:USERNAME")
) | ForEach-Object {
$data = $data.Replace($_[0], $_[1])
}
Set-Content -Path $copyFileLocation -Value $data
Edit: Missed that you were asking on how to make it into a function.
By looking at what your doing I assume you are modifying an SQL unattended install file, and have named it as such.
A good idea here is to make most of the parameters mandatory, so you are sure the user at least specifies all the required parameters. Maybe you want to specify MyUser later, so this is a good candidate for being a parameter with a default value.
Function Set-SQLInstallFileVariables {
Param(
[Parameter(Mandatory)][string]$FilePath,
[Parameter(Mandatory)][string]$NUEVAINESTANCIA,
[Parameter(Mandatory)][string]$NUEVAINESTANCIAID,
[Parameter(Mandatory)][string]$CUENTAAGTN,
[Parameter(Mandatory)][string]$CUENTASQLSER,
[string]$MyUser = "$env:UserDomain\$env:USERNAME"
)
$data = Get-Content -Path $FilePath -Raw
#(
#('"INSTANCENAME="TEST""', $NUEVAINESTANCIA),
#('"INSTANCEID="TEST""', $NUEVAINESTANCIAID),
#("NT Service\SQLAgent#TEST", $CUENTAAGTN),
#("NT Service\MSSQL#TEST", $CUENTASQLSER),
#("##MyUser##", $MyUser)
) | ForEach-Object {
$data = $data.Replace($_[0], $_[1])
}
Set-Content -Path $copyFileLocation -Value $data
}
The first thing to notice is that you read and write the file over and over again on every replacement, which is not very efficient.
This is not needed; once read the content is in a string variable and can get manipulated in memory multiple times before writing back to file.
One approach is to use string arrays that hold the search and replacement strings.
For this to work properly, both arrays must have the same number of elements.
$inputFile = 'D:\Test\TheFile.txt' # your input file path here ($copyFileLocation)
$outputFile = 'D:\Test\TheReplacedFile.txt' # for safety create a new file instead of overwriting the original
$searchStrings = '"INSTANCENAME="TEST""','"INSTANCEID="TEST""',"NT Service\SQLAgent#TEST","NT Service\MSSQL#TEST","##MyUser##"
$replaceStrings = $NUEVAINESTANCIA, $NUEVAINESTANCIAID, $CUENTAAGTN, $CUENTASQLSER, "$env:UserDomain\$env:USERNAME"
# get the current content of the file
$content = Get-Content -path $copyFileLocation -Raw
# loop over the search and replace strings to do all replacements
for ($i = 0; $i -lt $searchStrings.Count; $i++) {
$content = $content -replace [regex]::Escape($searchStrings[$i]), $replaceString[$i]
}
# finally, write the updated content to a (new) file
$content | Set-Content -Path $copyFileLocation
Another approach would be to use a Hashtable that stores both the search strings and the replacement strings:
$inputFile = 'D:\Test\TheFile.txt' # your input file path here ($copyFileLocation)
$outputFile = 'D:\Test\TheReplacedFile.txt' # for safety create a new file instead of overwriting the original
$hash = #{
'"INSTANCENAME="TEST""' = $NUEVAINESTANCIA
'"INSTANCEID="TEST""' = $NUEVAINESTANCIAID
"NT Service\SQLAgent#TEST" = $CUENTAAGTN
"NT Service\MSSQL#TEST" = $CUENTASQLSER
"##MyUser##" = "$env:UserDomain\$env:USERNAME"
}
# get the current content of the file
$content = Get-Content -path $copyFileLocation -Raw
# loop over the items in the hashtable to do all replacements
$hash.GetEnumerator() | ForEach-Object {
# the `$_` is an automatic variable you get within a ForEach-Object{}
# It represents a single item on each iteration.
$content = $content -replace [regex]::Escape($_.Key), $_.Value
}
# finally, write the updated content to a (new) file
$content | Set-Content -Path $copyFileLocation
In both cases, we're using -replace, which is a case-insensitive regex replacement. Because your search strings hold characters that have special meaning in regex (# and \) we need to escape these with [regex]::Escape()
Hope that helps
Here is mine attempt at it. Its not meant to be lean. Rather to be really clear about what it is doing while being full of functions (too many) and stopping you from getting the content multiple times.
After seeing what everyone else answered, their's is better code. Hopefully mine is readable and give you a better idea about using the pipeline and functions :)
Function Replace1 {Process{$_ -replace '"INSTANCENAME="TEST""',$NUEVAINESTANCIA}}
Function Replace2 {Process{$_ -replace '"INSTANCEID="TEST""',$NUEVAINESTANCIAID}}
Function Replace3 {Process{$_ -replace "NT Service\SQLAgent#TEST", $CUENTAAGTN}}
Function Replace4 {Process{$_ -replace "NT Service\MSSQL#TEST", $CUENTASQLSER}}
Function Replace6 {Process{$_ -replace "##MyUser##", $user}}
$user = "$env:UserDomain\$env:USERNAME"
Write-Host $user
Get-Content -path $copyFileLocation | Replace1 | Replace2 | Replace3 | Replace4 | Replace5 | Replace6 | Set-Content -path $copyFileLocation

Powershell: Read Text file line by line and split on "|"

I am having trouble splitting a line into an array using the "|" in a text file and reassembling it in a certain order. There are multiple lines like the original line in the text file.
This is the original line:
80055555|Lastname|Firstname|AidYear|DCDOCS|D:\BDMS_UPLOAD\800123456_11-13-2018 14-35-53 PM_1.pdf
I need it to look this way:
80055555|DCDOCS|Lastname|Firstname|AidYear|D:\BDMS_UPLOAD\800123456_11-13-2018 14-35-53 PM_1.pdf
Here is the code I am working with:
$File = 'c:\Names\Complete\complete.txt'
$Arr = $File -split '|'
foreach ($line in Get-Content $File)
{
$outputline = $Arr[0] + "|" + $Arr[4] + "|" + $Arr[1] + "|" + $Arr[2] + "|" +
"##" + $Arr[5] |
Out-File -filepath "C:\Names\Complete\index.txt" -Encoding "ascii" -append
}
You need to process every line of the file on its own and then split them.
$File = get-content "D:\test\1234.txt"
foreach ($line in $File){
$Arr = $line.Split('|')
[array]$OutputFile += $Arr[0] + "|" + $Arr[4] + "|" + $Arr[1] + "|" + $Arr[2] + "|" + "##" + $Arr[5]
}
$OutputFile | out-file -filepath "D:\test\4321.txt" -Encoding "ascii" -append
edit: Thx to LotPings for this alternate suggestion based on -join and the avoidance of += to build the array (which is inefficient, because it rebuilds the array on every iteration):
$File = get-content "D:\test\1234.txt"
$OutputFile = foreach($line in $File){($line.split('|'))[0,4,1,2,3,5] -Join '|'}
$OutputFile | out-file -filepath "D:\test\4321.txt" -Encoding "ascii"
To offer a more PowerShell-idiomatic solution:
# Sample input line.
$line = '80055555|Lastname|Firstname|AidYear|DCDOCS|D:\BDMS_UPLOAD\800123456_11-13-2018 14-35-53 PM_1.pdf'
# Split by '|', rearrange, then re-join with '|'
($line -split '\|')[0,4,1,2,3,5] -join '|'
Note how PowerShell's indexing syntax (inside [...]) is flexible enough to accept an arbitrary array (list) of indices to extract.
Also note how -split's RHS operand is \|, i.e., an escaped | char., given that | has special meaning there, because it is interpreted as a regex.
To put it all together:
$File = 'c:\Names\Complete\complete.txt'
Get-Content $File | ForEach-Object {
($_ -split '\|')[0,4,1,2,3,5] -join '|'
} | Out-File -LiteralPath C:\Names\Complete\index.txt -Encoding ascii
As for what you tried:
$Arr = $File -split '|'
Primarily, the problem is that the -split operation is applied to the input file path, not to the file's content.
Secondarily, as noted above, to split by a literal | char., \| must be passed to -split, because it expects a regex (regular expression).
Also, instead of using Out-File inside a loop with -Append, it is more efficient to use a single pipeline with ForEach-Object, as shown above.
Since your input file is actually a CSV file without headers and where the fields are separated by the pipe symbol |, why not use Import-Csv like this:
$fileIn = 'C:\Names\Complete\complete.txt'
$fileOut = 'C:\Names\Complete\index.txt'
(Import-Csv -Path $File -Delimiter '|' -Header 'Item','LastName','FirstName','AidYear','Type','FileName' |
ForEach-Object {
"{0}|{1}|{2}|{3}|{4}|{5}" -f $_.Item, $_.Type, $_.LastName, $_.FirstName, $_.AidYear, $_.FileName
}
) | Add-Content -Path $fileOut -Encoding Ascii

Windows Powershell to replace a set of characters from a text files in a folder

Working on a code which will replace a set of characters from a text files in a folder. IS there a way where it can do it for all the files in the folder. I am using a Windows 7 OS and Powershell Version 3.Attaching the code which I have. The issue is it creates a new file when I run the code (New_NOV_1995.txt) but it doesn't change any character in the new file as mentioned in the code. Help very much Appreciated.
$lookupTable = #{
'¿' = '|'
'Ù' = '|'
'À' = '|'
'Ú' = '|'
'³' = '|'
'Ä' = '-'
}
$original_file = 'C:\FilePath\NOV_1995.txt'
$destination_file = 'C:\FilePath\NOV_1995_NEW.txt'
Get-Content -Path $original_file | ForEach-Object {
$line = $_
$lookupTable.GetEnumerator() | ForEach-Object {
if ($line -match $_.Key)
{
$line = $line -replace $_.Key, $_.Value
}
}
$line
} | Set-Content -Path $destination_file
In the following example, I'm assuming that H:\Replace_String is a directory. In your code above, you don't have a backslash so it would only select files in the root of H:.
$configFiles = Get-ChildItem -path H:\Replace_String\*.txt
foreach ($file in $configFiles)
{
(Get-Content $file) |
Foreach-Object { $_ -replace "Cat", "New_Cat" } |
Foreach-Object { $_ -replace "Dog", "New_Dog" } |
Set-Content $file
}
The (original) answer proposed by Tony Hinkle needs another loop. The reason for this is that Get-Content produces an array. Each line represents an element of the array.
$configFiles = Get-ChildItem -path 'H:\Replace_String\*.txt'
foreach ($file in $configFiles){
$output = #()
$content = Get-Content $file
foreach ($line in $content) {
$line = $content.Replace("Cat", "New_Cat")
$line = $content.Replace("Dog", "New_Dog")
$output += $line
}
$output | Set-Content -Path $file
}
Edit: I noticed that Tony Hinkle's answer was modified as I posted this. He's sending everything through a pipeline where I'm storing the array in a variable then looping through. The pipeline method is probably more memory efficient. The variable with second loop for each element of the array is more easily modified to do more than just the two replacments.

powershell: replacing string using hash table

I have files which need to be modified according to mapping provided in CSV. I want to read each line of my txt file and depending if specified value exist I want to replace other strings in that line according to my CSV file (mapping). For that purpose I have used HashTable. Here is my ps script:
$file ="path\map.csv"
$mapping = Import-CSV $file -Encoding UTF8 -Delimiter ";"
$table = $mapping | Group-Object -AsHashTable -AsString -Property Name
$original_file = "path\input.txt"
$destination_file = "path\output.txt"
$content = Get-Content $original_file
foreach ($line in $content){
foreach ($e in $table.GetEnumerator()) {
if ($line -like "$($e.Name)") {
$line = $line -replace $e.Values.old_category, $e.Values.new_category
$line = $line -replace $e.Values.old_type, $e.Values.new_type
}
}
}
Set-Content -Path $destination_file -Value $content
My map.csv looks as follows:
Name;new_category;new_type;old_category;old_type
alfa;new_category1;new_type1;old_category1;old_type1
beta;new_category2;new_type2;old_category2;old_type2
gamma;new_category3;new_type3;old_category3;old_type3
And my input.txt content is:
bla bla "bla"
buuu buuu 123456 "test"
"gamma" "old_category3" "old_type3"
alfa
When I run this script it creates exactly the same output as initial file. Can someone tell me why it didn't change the line where "gamma" appears according to my mapping ?
Thanks in advance
Couple of things to change.
Firstly there is no need to change $mapping to a hash, Import-Csv already gives you an object array to work with.
Secondly, if you want to update the elements of $content, you need to use a for loop such that you can directly access modify them. Using a foreach creates a new variable in the pipeline and you were previously modifying it but then never writing it back to $content
Below should work:
$file ="map.csv"
$mapping = Import-CSV $file -Encoding UTF8 -Delimiter ";"
$original_file = "input.txt"
$destination_file = "output.txt"
$content = Get-Content $original_file
for($i=0; $i -lt $content.length; $i++) {
foreach($map in $mapping) {
if ($content[$i] -like "*$($map.Name)*") {
$content[$i] = $content[$i] -replace $map.old_category, $map.new_category
$content[$i] = $content[$i] -replace $map.old_type, $map.new_type
}
}
}
Set-Content -Path $destination_file -Value $content