How can I use PowerShell to import a file, and replace all existing placeholder variables with ones defined in the script? [duplicate] - powershell

This question already has answers here:
Expanding variables in file contents
(3 answers)
Closed 3 years ago.
I have a number of .txt files that I use as standardized templates for Arista switch deployments, but I want to update them rapidly using PowerShell.
I replaced all the necessary values in these templates with $variables and I now am attempting to write a script to replace them.
I found a solution that I liked, but it is not working for me. I am not sure what I am doing wrong here.
https://stackoverflow.com/a/9326779/
This is a snippet from the source file :
router bgp $asn
router-id 10.1.1.1
bgp listen range 192.168.$id.0/25 peer-group cluster$id remote-as $asn
neighbor cluster$id peer-group
neighbor cluster$id update-source Loopback0
neighbor cluster$id description cluster$id-BGP
neighbor cluster$id ebgp-multihop 3
neighbor cluster$id maximum-routes 12000
network 10.1.1.1/32
exit
Here is a snippet from the powershell script :
$newvars = #{
'$id' = '101'
'$asn' = '12345'
}
$template = '.\Arista\arista.txt'
$destination_file = '.\switchconfig' + $id + '.txt'
Get-Content -Path $template | ForEach-Object {
$line = $_
$newvars.GetEnumerator() | ForEach-Object {
if ($line -match $_.Key)
{
$line = $line -replace $_.Key, $_.Value
}
}
$line
} | Set-Content -Path $destination_file
What I want is to have a group of variables defined (upwards of 30), and then replace each instance of that variable in the text file with the value contained in the script.
This solution seemed good, since it would avoid doing a "replace" over and over, but it just prints the file as it originally was.

Since powershell uses $ as an identifier for variables (reserved), you have to properly escape that when running your method. Following is a little off but does what you are looking for. update your dictionary with \ before your $ sign to replace text including $.
$newvars = #{
'\$id' = '101'
'\$asn' = '12345'
}
$template = "C:\temp\new.txt"
$destination_file = "C:\temp\replaced.txt"
$data = #()
foreach($line in Get-Content $template) {
foreach($key in $newvars.Keys) {
if ($line -match $key) {
$line = $line -replace $key, $newvars[$key]
}
}
$data += $line
}
$data | Out-File $destination_file
Another thing to note.. in your file you are defining $id as the name of the file. I am not sure where but that variable would always be null as its not defined yet (unless your snippet here is different from your actual code.
If you want to use Invoke-Expression, you can use it in the following way,
$id = '101'
$asn = '12345'
$template = (Get-Content "C:\temp\new.txt") | out-string
$data = Invoke-Expression "`"$template`""
$data | Out-File "C:\Temp\test.txt"
You'll have to make sure your variables ($id, $asn) have a value to replace when evaluating the variables within your text file.

Related

Changing multiple lines in a text file based on a psobject

I'm working on a script which will add some additional informations to a txt file. These informations are stored in a CSV file which looks like this (the data will differs each time the script will launch):
Number;A;B;ValueOfB
FP01340/05/20;0;1;GTU_01,GTU_03
FP01342/05/20;1;0;GTU01
The txt file looks like this (data inside will of course differ each time):
1|1|FP01340/05/20|2020-05-02|2020-05-02|2020-05-02|166,91|203,23|36,32|nothing interesting 18|33333|63-111 somewhere|||||
2|zwol|9,00|9,00|0,00
2|23|157,91|194,23|36,32
1|1|FP01341/05/20|2020-05-02|2020-05-02|2020-05-02|12,19|14,99|2,80|Some info |2222222|blabla|11-111 something||||
2|23|12,19|14,99|2,80
1|1|FP01342/05/20|2020-05-02|2020-05-02|2020-05-02|525,36|589,64|64,28|bla|222222|blba 36||62030|something||
2|5|213,93|224,63|10,70
2|8|120,34|129,97|9,63
2|23|191,09|235,04|43,95
What I need to do is to find a line which contains 'Number' and then add value 'A' and 'B' from a CSV in a form: |0|1 and then on the first line below, at the end, add 'ValueofB' in a form |AAA_01,AAA_03
So the first two lines should look like this at the end:
1|1|FP01340/05/20|2020-05-02|2020-05-02|2020-05-02|166,91|203,23|36,32|nothing interesting 18|33333|63-111 somewhere||||||0|1
2|zwol|9,00|9,00|0,00|AAA_01,AAA_03
2|23|157,91|194,23|36,32
Rest of lines should not be touched.
I made a script which uses select-string method with context to find what I need to - put that into an object and then add to previously found strings what I need to and put that in to an another object.
My script is as follws:
$csvFile = Import-Csv -Path Somepath\file.csv -Delimiter ";"
$file = "Somepath2\SomeName.txt"
$LinesToChange = #()
$script:LinesToChange = $LinesToChange
$LinesOriginal = #()
$script:LinesOriginal = $LinesOriginal
foreach ($line in $csvFile) {
Select-String -Path $file -Pattern "$($Line.number)" -Encoding default -Context 0, 1 | ForEach-Object {
$1 = $_.Line
$2 = $_.Context.PostContext
}
$ListOrg = [pscustomobject]#{
Line_org = $1
Line_GTU_org = $2
}
$LinesOriginal = $LinesOriginal + $ListOrg
$lineNew = $ListOrg.Line_org | foreach { $_ + "|$($line.A)|$($line.B)" }
$GTUNew = $ListOrg.Line_GTU_org | foreach { $_ + "|$($line.ValueofB)" }
$ListNew = [pscustomobject]#{
Line_new = $lineNew
Line_GTU_new = $GTUNew
Line_org = $ListOrg.Line_org
Line_GTU_org = $ListOrg.Line_GTU_org
}
$LinesToChange = $LinesToChange + $ListNew
}
The output is an object $LinesToChange which have original lines and lines after the change. The issue is I have no idea how to use that to change the txt file. I tried few methods and ended up with file which contains updated lines but all others are doubbled (I tried foreach) or PS is using whole RAM and couldn't finish the job :)
My latest idea is to use something like that:
(Get-Content -Path $file) | ForEach-Object {
$line = $_
$LinesToChange.GetEnumerator() | ForEach-Object {
if ($line -match "$($LinesToChange.Line_org)") {
$line = $line -replace "$($LinesToChange.Line_org)", "$($LinesToChange.Line_new)"
}
if ($line -match "$($LinesToChange.Line_GTU_org)") {
$line = $line -replace "$($LinesToChange.Line_GTU_org)", "$($LinesToChange.Line_GTU_new)"
}
}
} | Set-Content -Path Somehere\newfile.txt
It seemed promising at first, but the variable $line contains all lines and as such it can't find the match.
Also I need to be sure that the second line will be directly below the first one (it is unlikely but it can be a case that there will be two or more lines with the same data while the "number" from CSV file is unique) so preferably while changing the txt file it would be needed to find a match for a two-liner; in short:
find this two lines:
1|1|FP01340/05/20|2020-05-02|2020-05-02|2020-05-02|166,91|203,23|36,32|nothing interesting 18|33333|63-111 somewhere|||||
2|zwol|9,00|9,00|0,00
change them to:
1|1|FP01340/05/20|2020-05-02|2020-05-02|2020-05-02|166,91|203,23|36,32|nothing interesting 18|33333|63-111 somewhere||||||0|1
2|zwol|9,00|9,00|0,00|AAA_01,AAA_03
Do that for all lines in a $LinesToChange
Any help will be much appreciated!
Greetings!
Some strange text file you have there, but anyway, this should do it:
# read in the text file as string array
$txt = Get-Content -Path '<PathToTheTextFile>'
$csv = Import-Csv -Path '<PathToTheCSVFile>' -Delimiter ';'
# loop through the items (rows) in the CSV and find matching lines in the text array
foreach ($item in $csv) {
$match = $txt | Select-String -Pattern ('|{0}|' -f $item.Number) -SimpleMatch
if ($match) {
# update the matching text line (array indices count from 0, so we do -1)
$txt[$match.LineNumber -1] += ('|{0}|{1}' -f $item.A, $item.B)
# update the line following
$txt[$match.LineNumber] += ('|{0}' -f $item.ValueOfB)
}
}
# show updated text on screen
$txt
# save updated text to file
$txt | Set-Content -Path 'Somehere\newfile.txt'

Parse a list line by line, create a new list in Powershell

I need to read in a file that contains lines of source/destination IPs and ports as well as a tag. I'm using Get-Content:
Get-Content $logFile -ReadCount 1 | % {
} | sort | get-unique | Out-File "C:\Log\logout.txt"
This is an example of the input file:
|10.0.0.99|345|195.168.4.82|58164|spam|
|10.0.0.99|345|195.168.4.82|58164|robot|
|10.0.0.99|231|195.168.4.82|58162|spam|
|195.168.4.82|58162|10.0.0.99|231|robot|
|10.0.0.99|345|195.168.4.82|58168|spam|
|10.0.0.99|345|195.168.4.82|58169|spam|
What I need to do is output a new list, but if the same source/destination IPs/ports are both 'spam' and 'robot' I just need to output that line as 'robot' (lines 1 and 2 above).
I need to do the same if the reverse direction of an existing connection is either 'spam' or 'robot', I just need one or the other and it would be 'robot' (lines 3 and 4 above). There will be plenty of 'spam' lines without a duplicate or reverse connection (the last couple lines above), they need to just stay the same.
This is what i've been using to create the reverse direction of the connection, but I haven't been able to figure out how to properly create the new list:
$reverse = '|' + ($_.Split("|")[3,4,1,2,5] -join '|') + '|'
Output of the above would be:
|10.0.0.99|345|195.168.4.82|58164|robot|
|195.168.4.82|58162|10.0.0.99|231|robot|
|10.0.0.99|345|195.168.4.82|58168|spam|
|10.0.0.99|345|195.168.4.82|58169|spam|
(except that second line didn't have to be the reversed direction)
Thanks for any help!
Since both direct and reverse connections are checked and their line order may not be sequential, I would use a hashtable to store the type of both directions and do everything algorithmically:
$checkPoints = #{}
$output = [ordered]#{}
$reader = [IO.StreamReader]'R:\1.txt'
while (!$reader.EndOfStream) {
$line = $reader.ReadLine()
$s = $line.split('|')
$direct = [string]::Join('|', $s[1..4])
$reverse = [string]::Join('|', ($s[3,4,1,2]))
$type = $s[5]
$known = $checkPoints[$direct]
if (!$known -or ($type -eq 'robot' -and $known -eq 'spam')) {
$checkPoints[$direct] = $checkPoints[$reverse] = $type
$output[$direct] = $line
$output.Remove($reverse)
} elseif ($type -eq 'spam' -and $known -eq 'robot') {
$output.Remove($reverse)
}
}
$reader.Close()
Set-Content r:\2.txt -Encoding utf8 -value #($output.Values)

Retrieving second part of a line when first part matches exactly

I used the below steps to retrieve a string from file
$variable = 'abc#yahoo.com'
$test = $variable.split('#')[0];
$file = Get-Content C:\Temp\file1.txt | Where-Object { $_.Contains($test) }
$postPipePortion = $file | Foreach-Object {$_.Substring($_.IndexOf("|") + 1)}
This results in all lines that contain $test as a substring. I just want the result to contain only the lines that exactly matches $test.
For example, If a file contains
abc_def|hf#23$
abc|ohgvtre
I just want the text ohgvtre
If I understand the question correctly you probably want to use Import-Csv instead of Get-Content:
Import-Csv 'C:\Temp\file1.txt' -Delimiter '|' -Header 'foo', 'bar' |
Where-Object { $_.foo -eq $test } |
Select-Object -Expand bar
To address the exact matching, you should be testing for equality (-eq) rather than substring (.Contains()). Also, there is no need to parse the data multiple times. Here is your code, rewritten to to operate in one pass over the data using the -split operator.
$variable = 'abc#yahoo.com'
$test = $variable.split('#')[0];
$postPipePortion = (
# Iterate once over the lines in file1.txt
Get-Content C:\Temp\file1.txt | foreach {
# Split the string, keeping both parts in separate variables.
# Note the backslash - the argument to the -split operator is a regex
$first, $second = ($_ -split '\|')
# When the first half matches, output the second half.
if ($first -eq $test) {
$second
}
}
)

Multiple global find-replace in CSV files

I've been struggling with what I think is a really simple problem but I can't see it. I have a stack of 30-odd csv files of varying contents generated daily by different applications that I need to normalize before importing into a single reporting db. An Extract, Transform and Load (ETL) type of thing - global find and replace.
Looping through the files is no problem - not sure whether using ForEach-Object Fullname is the best way to go as outputting to an 'OUT' folder messes it up but using -Name means I have to include the path.
Basically, all 'True'/'False' text is to be replaced with 1/0, same with 'yes'/'no', poweredon/poweredoff, etc. Also we have 4 sites - each needs replacing with a ref. id, loads of stuff like that. I've tried modifying loads of scripts I've found on line - many in here. Tried using the replacement text in an array, pulling the CSV into a string, just can't see it. I've been doing the same thing for years with VBScript and it's easy. But I need to learn PowerShell so I'm going to persevere with it.
Ok, here is a quick search and replace function for you. It can read multiple CSV files and match\replace multiple values.
function Replace-CsvValue
{
[CmdletBinding()] # Enable pipeline support
Param
(
# Filename, mandatory, takes pipeline input
[Parameter(Mandatory = $true, ValueFromPipeline = $true)]
# Alias, allows to directly pipe Get-ChildItem output to this function
[Alias('FullName')]
[string]$File,
# Scriptblock, mandatory, does actual search and replace
[Parameter(Mandatory = $true, ValueFromPipelineByPropertyName = $true)]
[scriptblock]$ScriptBlock
)
Process
{
# Import CSV
$CsvFile = $File | Import-Csv
# Generate new filename
$NewFileName = Join-Path -Path (Split-Path -Path $File -Parent) -ChildPath ('Processed_' + (Split-Path -Path $File -Leaf))
# Iterate over each line in CSV
$CsvFile | ForEach-Object {
# Execute scritblock against record
& $ScriptBlock
}
# Export CSV
$CsvFile | Export-Csv -Path $NewFileName -NoTypeInformation
}
}
Usage:
Write scriptblock with required replace logic
Pipe filenames or Get-ChildItem output to the function and pass scriptblock
Example:
Original CSV file:
State, Active, Available
PoweredOn, True, Yes
Function call:
# Scriptblock with replace logic
$ReplaceRule = {
# Iterate over each item in CSV line
$Item = $_
$_.PSObject.Properties.Name | ForEach-Object {
# If item name matches...
switch ($_)
{
'State' {
# If item value matches...
if($Item.$_ -eq 'PoweredOn')
{
$Item.$_ = 'Online'
}
# Or if item value matches...
elseif($Item.$_ -eq 'PoweredOff')
{
$Item.$_ = 'Offline'
}
break
}
# More replace rules, you can add your own here...
'Active' {
if($Item.$_ -eq 'True')
{
$Item.$_ = '1'
}
elseif($Item.$_ -eq 'False')
{
$Item.$_ = '0'
}
break
}
'Available' {
if($Item.$_ -eq 'Yes')
{
$Item.$_ = '1'
}
elseif($Item.$_ -eq 'No')
{
$Item.$_ = '0'
}
break
}
}
}
}
# Get all CSV files that match wildcard and
# feed them to the Replace-CsvValue function
Get-ChildItem -Path '.\' -Filter '*Report*.csv' | Replace-CsvValue -ScriptBlock $ReplaceRule
Processed CSV file:
"State","Active","Available"
"Online","1","1"
I made this csv for testing with the help of Mockaroo. Notice someones first name is True. I have that in there as a check to be sure my logic is working.
Present Name Lunch State
------- ---- ----- -----
TRUE Jesse Daniels No Powered Off
FALSE Debra Cunningham Yes Powered Off
TRUE True Jones Yes Powered Off
TRUE George Fernandez Yes Powered Off
FALSE Lisa Cox No Powered On
For the purpose of this I think it would be simple to just ignore the fact that it is a CSV and just replace the text outright. The caveat we have to be careful for is partial matches. Using regex we should be able to account for that possibility.
From comments you already know that you can chain -replace. Lets add some regex magic in there to make the process easier.
$filename = "C:\temp\MOCK_DATA.csv"
$oneKeywordPattern = "Powered On","Yes","True" -join "|"
$zeroKeywordPattern = "Powered Off","No","False" -join "|"
(Get-Content $filename) -replace "(?<=^|,)$oneKeywordPattern(?=$|,)","1" -replace "(?<=^|,)$zeroKeywordPattern(?=$|,)","0" | Set-Content $filename
To make sure that the csv structure is accounted for we only replace if the element is at the start of the line or a comma followed the end of the line or comma (This is using a lookahead and lookbehind.). This also ensures that we only change full elements and True Jones is not affected.
We used $oneKeywordPattern so that you can add elements to the array that need to be changed to a 1. We join them with a pipe so that it is treated as a alternative regex pattern. Its counterpart $zeroKeywordPattern functions just the same.
Output
Present Name Lunch State
------- ---- ----- -----
1 Jesse Daniels 0 0
0 Debra Cunningham 1 0
1 True Jones 1 0
1 George Fernandez 1 0
0 Lisa Cox 0 1
You could likely have other patterns that do not need to be changed with this logic. Just chain another -replace and remember that it supports regex so watch out for special characters.
The two caveats here is that if the files are large it could take a while to load the file and process the regexes (especially if you add more.) Also if your text is enclosed in quotes we don't currently account for that but it would be easy.
Basically, all 'True'/'False' text is to be replaced with 1/0, same with 'yes'/'no', poweredon/poweredoff, etc. Also we have 4 sites - each needs replacing with a ref. id, loads of stuff like that. I've tried modifying loads of scripts I've found on line - many in here. Tried using the replacement text in an array, pulling the csv into a string, just can't see it. I've been doing the same thing for years with vbscript and it's easy. But I need to learn PShell so I'm going to persevere with it. I'd really appreciate some help here.
If it's that static, you can probably get away with:
$changes = #{
'true' = '1';
'false' = '0';
'poweredon' = '1';
'poweredoff' = '0'
}
$folder = "" # your folder here
$csvFiles = ls $folder *.csv
foreach ($file in $csvFiles) {
$csvData = import-csv $file
foreach ($row in $csvData) {
$cells = $row | `
gm | `
?{$_.MemberType -eq 'NoteProperty'} | `
select -exp Name
foreach ( $cell in $cells ) {
$val = $row."$cell"
$valueNeedsChanging = $changes.ContainsKey($val)
if ( $valueNeedsChanging ) {
$newValue = $changes[$val]
$row."$cell" = $newValue
}
}
}
cp $file.FullName "$($file.FullName).bak" # back it up before saving
$csvData | export-csv -Path $file.FullName -NoTypeInformation
}
I chose to use Import- and Export-CSV to preserve the structure of the CSV file for files that have a lot of advanced formatting.

What is an equivalent of *Nix 'cut' command in Powershell?

I have following content in a configuration file (sample.cfg),
Time_Zone_Variance(Mins):300
Alert_Interval(Mins):2
Server:10.0.0.9
Port:1840
I'm trying to store an each values after the : by using split in PowerShell. but i'm not able to produce require output.
Can someone tell me how to use PowerShell split for the above problem ?
You can read the contents of the file using Get-Content, then pipe each line through ForEach-Object, then use the split command on each line, taking the second item in the array as follows:
$filename = "sample.cfg"
Get-Content $filename | ForEach-Object {
$_.split(":")[1]
}
Output
300
2
10.0.0.9
1840
Update
I prefer the approach by #AnsgarWiechers, but if you really need specifically named values you could create a hashtable and replace the name with the value:
$configValues = #{
hour = "Time_Zone_Variance(Mins)"
min = "Alert_Interval(Mins)"
server = "Server"
port = "Port"
}
Get-Content $filename | ForEach-Object {
# Courtesy of Ansgar Wiechers
$key, $value = $_ -split ':', 2
foreach($configValuesKey in $($configValues.keys)) {
if ($configValues[$configValuesKey] -eq $key)
{
$configValues[$configValuesKey] = $value
}
}
}
write-host "`nAll Values:"
$configValues
write-host "`nIndividual value:"
$configValues.port
Output
All Values:
Name Value
---- -----
port 1840
min 2
server 10.0.0.9
hour 300
Individual value:
1840
How's this?
function cut {
param(
[Parameter(ValueFromPipeline=$True)] [string]$inputobject,
[string]$delimiter='\s+',
[string[]]$field
)
process {
if ($field -eq $null) { $inputobject -split $delimiter } else {
($inputobject -split $delimiter)[$field] }
}
}
PS C:\> 'hi:there' | cut -f 0 -d :
hi
PS C:\> 'hi:there' | cut -f 1 -d :
there
PS C:\> 'hi:there' | cut -f 0,1 -d :
hi
there
PS C:\> 'hi:::there' | cut -f 0 -d :+
hi
PS C:\> 'hi there' | cut
hi
there
For a more succint syntax, this will also do the trick:
((Get-Content "your-file.txt") -Split ":")[1]
So the trick to use the -Split method is to have a String object returned by Get-Content (alias cat can also be used, actually), and from the resulting String[] object you can use the brackets to extract the nth item.
Note: Using -Split without parenthesis around Get-Content won't work since -Split is not a parameter name for that command... 🤷‍♂️
I suppose you don't want to just split the lines, but actually create key/value pairs. That could be achieved like this:
$config = #{}
Get-Content 'C:\path\to\sample.cfg' | % {
$key, $value = $_ -split ':', 2
$config[$key] = $value
}
You could also use the ConvertFrom-StringData cmdlet:
Get-Content 'C:\path\to\sample.cfg' | % {
ConvertFrom-StringData ($_ -replace ':','=')
}
The -replace operation is necessary, because ConvertFrom-StringData expects key and value to be separated by =. If you could change the delimiter in the config file from : to =, you could use ConvertFrom-StringData $_ without replacement.