I have a text file "list.txt" with a list of hundreds of URL's that I want to parse, along with some common-to-all config data, into individual xml files (config files) using each value in "list.txt", like so:
list.txt contains:
line_1
line_2
line_3
The boilerplate config data looks like (using line_1 as an example):
<?xml version="1.0"?>
<Website xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Url>line_1.mydomain.com</Url>
<Title>line_1</Title>
<Enabled>true</Enabled>
<PluginInName>Tumblr</PluginInName>
</Website>
So if "list.txt" contains 100 items, I want 100 config files written with the URL and Title elements individualized.
I have fumbled with several posts on reading the array and on creating text files, but I haven't been able to make any of it work.
What I tried, although it's munged at this point. I'm not sure where I started or how I got to here:
$FileName = "C:\temp\list.txt"
$FileOriginal = Get-Content $FileName
# create an empty array
Foreach ($Line in $FileOriginal)
{
$FileModified += $Line
if ($Line -match $pattern)
{
# Add Lines after the selected pattern
$FileModified += 'add text'
$FileModified += 'add second line text'
}
}
Set-Content $fileName $FileModified
This is way beyond my neophyte Powershell skills. Can anyone help out?
You're looking for a string-templating approach, where a string template that references a variable is instantiated on demand with the then-current variable value:
# Define the XML file content as a *template* string literal
# with - unexpanded - references to variable ${line}
# (The {...}, though not strictly necessary here,
# clearly delineates the variable name.)
$template = #'
<code>
<?xml version="1.0"?>
<Website xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Url>${line}.mydomain.com</Url>
<Title>${line}</Title>
<Enabled>true</Enabled>
<PluginInName>Tumblr</PluginInName>
</Website>
'#
# Loop over all input lines.
Get-Content C:\temp\list.txt | ForEach-Object {
$line = $_ # store the line at hand in $line.
# Expand the template based on the current $line value.
$configFileContent = $ExecutionContext.InvokeCommand.ExpandString($template)
# Save the expanded template to an XML file.
$configFileContent | Set-Content -Encoding Utf8 "$line.xml"
}
Notes:
I've chosen UTF-8 encoding for the output XML files, and to name them "$line.xml", i.e. to name them for each input line and to store them in the current location; adjust as needed.
The template expansion (interpolation) is performed via automatic variable $ExecutionContext, whose .InvokeCommand property provides access to the .ExpandString() method, which allows performing string expansion (interpolation) on demand, as if the input string were a double-quoted string - see this answer for a detailed example.
Surfacing the functionality of the $ExecutionContext.InvokeCommand.ExpandString() method in a more discoverable way via an Expand-String cmdlet is the subject of this GitHub feature request.
Ansgar Wiechers points out that a simpler alternative in this simple case - given that only a single piece of information is passed during template expansion - is to use PowerShell's string-formatting operator, -f to fill in the template:
# Define the XML file content as a *template* string literal
# with '{0}' as the placeholder for the line variable, to
# be instantiated via -f later.
$template = #'
<code>
<?xml version="1.0"?>
<Website xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Url>{0}.mydomain.com</Url>
<Title>{0}</Title>
<Enabled>true</Enabled>
<PluginInName>Tumblr</PluginInName>
</Website>
'#
# Loop over all input lines.
Get-Content C:\temp\list.txt | ForEach-Object {
# Expand the template based on the current $line value.
$configFileContent = $template -f $_
# Save the expanded template to an XML file.
$configFileContent | Set-Content -Encoding Utf8 "$line.xml"
}
Optional reading: choosing between -f and $ExecutionContext.InvokeCommand.ExpandString() for template expansion:
Tip of the hat to Ansgar for his help.
Using -f:
Advantages:
It is made explicit on invocation what values will be filled in.
Additionally, it's easier to include formatting instructions in placeholders (e.g., {0:N2} to format numbers with 2 decimal places).
Passing the values explicitly allows easy reuse of a template in different scopes.
An error will occur by default if you accidentally pass too few or too many values.
Disadvantages:
-f placeholders are invariably positional and abstract; e.g., {2} simply tells you that you're dealing with the 3rd placeholder, but tells you nothing about its purpose; in larger templates with multiple placeholders, this can become an issue.
Even if you pass the right number of values, they may be in the wrong order, which can lead to subtle bugs.
Using $ExecutionContext.InvokeCommand.ExpandString():
Advantages:
If your variables have descriptive names, your template will be more readable, because the placeholders - the variable names - will indicate their purpose.
No need to pass values explicitly on invocation - the expansion simply relies on the variables available in the current scope.
Disadvantages:
If you use a template in multiple functions (scopes), you need to make sure that the variables referenced in the template are set in each.
At least by default, $ExecutionContext.InvokeCommand.ExpandString() will quietly ignore nonexistent variables referenced in the template - which may or may not be desired.
However, you can use Set-StrictMode -Version 2 or higher to report an error instead; using Set-StrictMode is good practice in general, though note that its effect isn't lexically scoped and it can disable convenient functionality.
Generally, you manually need to keep your template in sync with the code that sets the variables referenced in the template, to ensure that the right values will be filled in (e.g., if the name of a referenced variable changes, the template string must be updated too).
Related
I'm trying to find an efficient way to read the value of a string variable in a PowerShell .ps1 file and then update the same variable/value in another .ps1 file. In my specific case, I would update a variable for the version # on script one and then I would want to run a script to update it on multiple other .ps1 files. For example:
1_script.ps1 - Script I want to read variable from
$global:scriptVersion = "v1.1"
2_script.ps1 - script I would want to update variable on (Should update to v1.1)
$global:scriptVersion = "v1.0"
I would want to update 2_script.ps1 to set the variable to "v1.1" as read from 1_script.ps1. My current method is using get-content with a regex to find a line starting with my variable, then doing a bunch of replaces to get the portion of the string I want. This does work, but it seems like there is probably a better way I am missing or didn't get working correctly in my tests.
My Modified Regex Solution Based on Answer by #mklement0 :
I slightly modified #mklement0 's solution because dot-sourcing the first script was causing it to run
$file1 = ".\1_script.ps1"
$file2 = ".\2_script.ps1"
$fileversion = (Get-Content $file1 | Where-Object {$_ -match '(?m)(?<=^\s*\$global:scriptVersion\s*=\s*")[^"]+'}).Split("=")[1].Trim().Replace('"','')
(Get-Content -Raw $file2) -replace '(?m)(?<=^\s*\$global:scriptVersion\s*=\s*")[^"]+',$fileversion | Set-Content $file2 -NoNewLine
Generally, the most robust way to parse PowerShell code is to use the language parser. However, reconstructing source code, with modifications after parsing, may situationally be hampered by the parser not reporting the details of intra-line whitespace - see this answer for an example and a discussion.[1]
Pragmatically speaking, using a regex-based -replace solution is probably good enough in your simple case (note that the value to update is assumed to be enclosed in "..." - but matching could be made more flexible to support '...' quoting too):
# Dot-source the first script in order to obtain the new value.
# Note: This invariably executes *all* top-level code in the script.
. .\1_script.ps1
# Outputs to the display.
# Append
# | Set-Content -Encoding utf8 2_script.ps1
# to save back to the input file.
(Get-Content -Raw 2_script.ps1) -replace '(?m)(?<=^\s*\$global:scriptVersion\s*=\s*")[^"]+', $global:scriptVersion
For an explanation of the regex and the ability to experiment with it, see this regex101.com page.
[1] Syntactic elements are reported in terms of line and column position, and columns are character-based, meaning that spaces and tabs are treated the same, so that a difference of, say, 3 character positions can represent 3 spaces, 3 tabs, or any mix of it - the parser won't tell you. However, if your approach allows keeping the source code as a whole while only removing and splicing in certain elements, that won't be a problem, as shown in iRon's helpful answer.
To compliment the helpful answer from #mklement0. In case your do go for the PowerShell abstract syntax tree (AST) class, you might use the Extent.StartOffset/Extent.EndOffset properties to reconstruct your script:
Using NameSpace System.Management.Automation.Language
$global:scriptVersion = 'v1.1' # . .\Script1.ps1
$Script2 = { # = Get-Content -Raw .\Script2.ps1
[CmdletBinding()]param()
begin {
$global:scriptVersion = "v1.0"
}
process {
$_
}
end {}
}.ToString()
$Ast = [Parser]::ParseInput($Script2, [ref]$null, [ref]$null)
$Extent = $Ast.Find(
{
$args[0] -is [AssignmentStatementAst] -and
$args[0].Left.VariablePath.UserPath -eq 'global:scriptVersion' -and
$args[0].Operator -eq 'Equals'
}, $true
).Right.Extent
-Join (
$Script2.SubString(0, $Extent.StartOffset),
$global:scriptVersion,
$Script2.SubString($Extent.EndOffset)
) # |Set-Content .\Script2.ps1
I have a file1 with several variables defined:
'$GateWay_HostName'='Biscuits'
'$DomainName'='AND'
'$DC_Name'='Gravy'
I have another file (File2) with line by line commands to send to Cisco devices. I do not want to read file, replace variables, then save file, because the passwords will be entered in cleartext. I can't seem to figure out how to import the variables and replace any matching string with the value.
I can pull in the variables and call them in the script and reference $GateWay_HostName for example with:
$ReplaceVars = Get-Content "C:\folder\file1.csv" | ConvertFrom-StringData
But I can't seem to find anything about going through the imported string to replace all of the variables (some appear once, some appear many, some don't exist).
$CommandstoSend = (Get-Content -Path "C:\folder\File2" -raw)
Because the code to execute the commands are passed as raw, it won't read the variables on the fly. I need to import the raw data otherwise plink won't pass the commands, but I don't care when it gets converted to "raw". Also, if I'm going to end up using a search and replace, I know I don't need to have $VARIABLEname format.
Thanks
You can do the following:
$ReplaceVars = Get-Content "C:\folder\file1.csv" -Raw | ConvertFrom-StringData
$replaceString = ($ReplaceVars.Keys |% {[regex]::Escape($_)}) -join '|'
[regex]::Replace((Get-Content file2 -raw),$replaceString,{param($m) $ReplaceVars[$m.Value]})
Using -Raw on Get-Content creates a single string. This is important for returning a single hash table after piping into ConvertFrom-StringData. Without -Raw, ConvertFrom-StringData outputs an array of hash tables, making lookups more complex.
[regex]::Escape() is to escape special regex characters like $ that could be in your variable names. Joining on | creates a regex alternation (equivalent to logical OR).
The [regex]::Replace() method allows for a script block to be used in the replacement string parameter. Using a parameter $m, we can reference the object that contains the matched text (stored in the Value property) and manipulate it. Here we use it as a key name ($m.Value) for the $replaceVars hash table.
Effectively, this solution looks for text that matches a hash table key name and then replaces that name with the corresponding hash table value.
IMO, the better solution is to configure file2 to be a template file. You can use string format placeholders where the substitutions need to happen. Then just use the string format operator -f. See below for an example:
# file2 contents
$file2 = #'
My DC Name = {0}
Domain Name = {1}
I don't want to edit this line.
Gateway Host = {2}
'#
$file2 -f 'Gravy','AND','Biscuits'
Output:
My DC Name = Gravy
Domain Name = AND
I don't want to edit this line.
Gateway Host = Biscuits
How to read machine.config file to check whether it contains a key?
Example: oracle.managedaccess.client.
Below is the code:
$configFilePath = [System.Runtime.InteropServices.RuntimeEnvironment]::SystemConfigurationFile
Write-Host $configFilePath
$cnf = . $configFilePath
echo name
Write-Host $cnf
#check machine.config contains oracle.managedaccess.client
The . operator is for running PowerShell scripts into the context of the calling script (as opposed to running them in a child context via the & operator). It's not for reading arbitrary files.
For reading plain text files use Get-Content. For reading XML files (which your machine.config most likely is) create an XML object and use its Load() method:
$xml = New-Object Xml
$xml.Load('C:\path\to\your\machne.config')
You can check for the presence of a node or attribute by using the SelectNodes() method with an XPath expression. An empty result means the node doesn't exist.
Ansgar Wiechers' helpful answer explains the problem with your code well and offers helpful background information.
Here's a concrete solution that shows some related techniques:
$file = [System.Runtime.InteropServices.RuntimeEnvironment]::SystemConfigurationFile
[xml] $xml = Get-Content -Raw $file
# Get the <oracle.manageddataaccess.client> element
# that, in this example, is assumed to be a child element of the
# top-level <configuration> element.
$elem = $xml.configuration.'oracle.manageddataaccess.client'
# Determine if the element exists.
$exists = $null -ne $elem
Explanation:
[xml] $xml = Get-Content -Raw $file is a more PowerShell-idiomatic way of reading an XML document from a file; using the [xml] type accelerator automatically turns the file's content, obtained with Get-Content, into an XML document (System.Xml.XmlDocument).
$xml.configuration.'oracle.manageddataaccess.client' uses PowerShell's adaptation of the XML DOM to provide convenient access to the elements in the document hierarchy with dot notation (.)
Note that this assumes that you know the precise location (path to) the target element; if you don't, use .SelectNodes() or .SelectSingleNode() as recommended in Ansgar's answer; see example at the bottom.
Note the need to enclose oracle.manageddataaccess.client in '...' (quotes), because the element name itself contains . characters.
For more information about how PowerShell adapts the XML DOM, see this answer.
If the targeted element exists, a System.Xml.XmlElement instances is returned, if the target element has child elements of its own (otherwise, its .InnerText property would be returned).
If no such element exists, the return value is $null.
$exists = $null -eq $elem creates a Boolean variable that indicates whether the element of interest exists or not.
Example uses of .SelectSingleNode() with XPath queries:
The following returns a <oracle.manageddataaccess.client> element found anywhere in the document, if present, and $null otherwise:
$xml.SelectSingleNode('//oracle.manageddataaccess.client')
The following locates a <section> element that is a child element of elements <configuration> and <configSections> and that contains a name attribute containing the string appSettings
$xml.SelectSingleNode('/configuration/configSections/section[#name="appSettings"]')
So my challenge today.
I have a config file (really just a txt document) that stores variables to store information passed between scripts or to be used after restarts.
I am looking for a more efficient way to read and update the file. Currently I read the file with:
Get-Content $current\Install.cfg | ForEach-Object {
Set-Variable -Name line -Value $_
$a, $b = $line.Split('=')
Set-Variable -name $a -Value $b
}
But to overwrite the contents, I recreate the file with:
ECHO OSV=$OSV >>"$ConfigLoc\tool.cfg"
ECHO OSb=$OSb >>"$ConfigLoc\tool.cfg"
ECHO cNum=$cNum >>"$ConfigLoc\tool.cfg"
ECHO cCode=$cCode >>"$ConfigLoc\tool.cfg"
ECHO Comp=$Comp >>"$ConfigLoc\tool.cfg"
Each time I have added a new saved variable, I have just hardcoded the new variable into both the original config file and the config updater.
As my next updates require an additional 30 variables to my current 15. I would like something like:
Get-Content $current\Install.cfg | ForEach-Object {
Set-Variable -Name line -Value $_
$a, $b = $line.Split('=')
ECHO $a=$$a
}
Where $$a uses the variable $a in the loop as the variable name to load the value.
Best example i can show to clarify is:
ECHO $a=$$a (in current loop)
Echo OSV=$OSV (actually appears in code as)
Not sure how to clarify this anymore, or how to achieve it with the variable title also being a variable.
If you want to create a file that has name=value parameters, here's an alternate suggestion. This is a snippet of a real script I use every day. You might modify it so it reads your .csv input and uses it instead of the hard coded values.
$Sites = ("RawSiteName|RoleName|DevUrl|SiteID|HttpPort|HttpsPort", `
"SiteName|Name of role|foo.com|1|80|443" `
) | ConvertFrom-CSV -Delimiter "|"
$site = $sites[0]
Write-Host "RawSiteName =$($site.RawSiteName)"
You might be able to use something similar to $text = Get-Content MyParameters.csv and pipe that to the ConvertFrom-CSV cmdlet. I realize it's not a direct answer to what you are doing but it will let you programmatically create a file to pass across scripts.
Thanks for the help everyone. This is the solution I am going with. Importing and exporting couldn't be simpler. If I have to manually update the XML install default I can with ease which is also amazing. I also love the fact that even if you import as $Test you can still use $original to access variables. I will be creating multiple hashtables to organize the different data I will be using going forward and just import/export it in a $config variable as the master.
$original = #{
OSV='0'
OSb='0'
cNum='00000'
cCode='0000'
Client='Unknown'
Comp='Unknown'
}
$original | Export-Clixml $Home\Desktop\sample.cfg
$Test = Import-Clixml $Home\Desktop\sample.cfg
Write $Test
Write $original.Client
In essence, you're looking for variable indirection: accessing a variable indirectly, via its name stored in another variable.
In PowerShell, Get-Variable allows you to do that, as demonstrated in the following:
# Sample variables.
$foo='fooVal'
$bar='barVal'
# List of variables to append to the config file -
# the *names* of the variables above.
$varsToAdd =
'foo',
'bar'
# Loop over the variable names and use string expansion to create <name>=<value> lines.
# Note how Get-Variable is used to retrieve each variable's value via its *name*.
$(foreach ($varName in $varsToAdd) {
"$varName=$(Get-Variable $varName -ValueOnly)"
}) >> "$ConfigLoc/tool.cfg"
With the above, the following lines are appended to the output *.cfg file:
foo=fooVal
bar=barVal
Note that you can read such a file more easily with the ConvertFrom-StringData, which outputs a hashtable with the name-value pairs from the file:
$htSettings = Get-Content -Raw "$ConfigLoc/tool.cfg" | ConvertFrom-StringData
Accessing $htSettings.foo would then return fooVal, for instance.
With a hashtable as the settings container, updating the config file becomes easier, as you can simply recreate the file with all settings and their current values:
$htSettings.GetEnumerator() |
ForEach-Object { "$($_.Key)=$($_.Value)" } > "$ConfigLoc/tool.cfg"
Note: PowerShell by default doesn't enumerate the entries of a hashtable in the pipeline, which is why .GetEnumerator() is needed.
Generally, though, this kind of manual serialization is fraught, as others have pointed out, and there are more robust - though typically less friendly - alternatives.
With your string- and line-based serialization approach, there are two things to watch out for:
All values are saved as a strings, so you have to manually reconvert to the desired data type, if necessary - and even possible, given that not all objects provide meaningful string representations.
Generally, the most robust serialization format is Export-CliXml, but note that it is not a friendly format - be careful with manual edits.
ConvertFrom-StringData will fail with duplicate names in the config file, which means you have to manually ensure that you create no duplicate entries when you append to the file - if you use the above approach of recreating the file from a hashtable every time, however, you're safe.
I am trying to automate Active Directory installation on Windows Server 2008 using windows powershell. I created a text file with .tmpl extension and added:
[DCINSTALL]
ReplicaOrNewDomain=_ReplicaOrNewDomain__
Then I created an answer file in a text format:
[DCINSTALL]
ReplicaOrNewDomain=$env:ReplicaOrNewDomain
Now I want to be able to write a script in PowerShell which will use the template file to get the value of variable ReplicaOrNewDomain from environment and replace $env:ReplicaOrNewDomain by that value in the text file so that I can use that answer file for AD installation.
You have a few options to do this. One is Environment.ExpandEnvironmentVariables. This uses a %variable% syntax (instead of $env:variable), so it would be simpler if you only want to substitute environment variables:
gc input.tmpl | foreach { [Environment]::ExpandEnvironmentVariables($_) } | sc out.ini
A more complete expansion of PowerShell expressions can be achieve via ExpandString. This is more useful if you want to insert actual PowerShell expressions into the template:
gc input.tmpl | foreach { $ExecutionContext.InvokeCommand.ExpandString($_) } | sc out.ini
A third option would be something like a customized templating scheme that uses Invoke-Expression, which I implemented here.
You can do that with a simple replacement like this:
$f = 'C:\path\to\your.txt'
(Get-Content $f -Raw) -replace '\$env:ReplicaOrNewDomain', $env:ReplicaOrNewDomain |
Set-Content $f
or like this:
$f = 'C:\path\to\your.txt'
(Get-Content $f -Raw).Replace('$env:ReplicaOrNewDomain', $env:ReplicaOrNewDomain) |
Set-Content $f
Note that when using the -replace operator you need to escape the $ (because otherwise it'd have the special meaning "end of string"). When using the Replace() method you just need to use single quotes to prevent expansion of the variable in the search string.
However, why the intermediate step of replacing the template parameter _ReplicaOrNewDomain__ with a different template parameter $env:ReplicaOrNewDomain? You would make your life easier if you just kept the former and replaced that with the value of the environment variable ReplicaOrNewDomain.
One thing that I like to do with my template files is something like this.
[DCINSTALL]
ReplicaOrNewDomain={0}
OtherVariable={1}
Then in my code I can use the format operator -f to make the changes.
$pathtofile = "C:\temp\test.txt"
(Get-Content $pathtofile -Raw) -f $env:ReplicaOrNewDomain, "FooBar" | Set-Content $pathtofile
It can help if you have multiple things that you need to update at once. Update your file with as many place holders as you need. You can use the same one multiple times if need be in the file.
[DCINSTALL]
ReplicaOrNewDomain={0}
SimilarVariable={0}
Caveat
If your actual file is supposed to contain curly braces you need to double them up to the are escaped.
You can use the ExpandString function, like this:
$ExecutionContext.InvokeCommand.ExpandString($TemplVal)
(assuming $TemplVal has the template string).