PowerShell - API data into XML - powershell

I am new to powershell and trying to write my first script.I am using PowerShell v2.0. I have the following script that makes an API call and gets data into $data variable.
$FullURL = $url1+$url2+$Url3
$client = New-Object System.Net.WebClient
$data = $client.DownloadString($FullURL)
Set-Content -Value $data -Path 'c:\API.txt'
$data outputs the below (example). Note - gettype() results are string. -
<attribute name="Business Unit">Platform</attribute>
<attribute name="Department">Channels Technology</attribute>
<attribute name="Team">Stackexchange</attribute>
<attribute name="Environment">World</attribute>
<attribute name="ServerModel">Synology</attribute>
<attribute name="datacentre">New York</attribute>
<attribute name="Application">PowerShell Teacher</attribute>
<attribute name="Description">Learn How To Use PowerShell</attribute>
I need to get the above into the below sample of the XML file, in between the attributes tags -
<selfAnnounce>
<enabled>true</enabled>
<retryInterval>60</retryInterval>
<requireReverseConnection>false</requireReverseConnection>
<probeName>
<hostname/>
<data>_</data>
<port/>
<data>-SA</data>
</probeName>
<managedEntity>
<name></name>
<attributes>
</attributes>
I am not sure where to begin with this one. I thought it would be something like the below but the results are all in one tag, maybe because they are not pscustom objects -
[xml]$XML = Get-Content $SelfannounceXMLEdit
$data | ForEach-Object {
$tempchild = $XML.CreateElement("Attributename")
$tempchild.set_InnerText($_)
$newType = $XML.netprobe.selfAnnounce.managedEntity.attributes.AppendChild($tempchild)
}
$XML.Save($SelfannounceXMLEdit)
This gives the following results which is obviously wrong for an XML file-
<attributes>
<Attributename><attribute name="Business Unit">Platform</attribute> <attribute name="Department">Channels Technology</attribute> <attribute name="Team">Stackexchange</attribute> <attribute name="Environment">World</attribute> <attribute name="ServerModel">Synology</attribute> <attribute name="datacentre">New York</attribute> <attribute name="Application">PowerShell Teacher</attribute> <attribute name="Description">Learn How To User PowerShell</attribute></Attributename>
</attributes>
Results should look like the following -
<?xml version="1.0" encoding="ISO-8859-1"?>
<netprobe compatibility="1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://google.com/netprobe.xsd">
<selfAnnounce>
<enabled>true</enabled>
<retryInterval>10</retryInterval>
<requireReverseConnection>false</requireReverseConnection>
<probeName>
<hostname />
<data>_</data>
<port />
<data>-SA</data>
</probeName>
<managedEntity>
<name></name>
<attributes>
<attribute name="Business Unit">Platform</attribute>
<attribute name="Department">Channels Technology</attribute>
<attribute name="Team">Stackexchange</attribute>
</attributes>
<types>
<type>Core</type>
<type>Core Windows</type>
<!--Autogenerated types-->
<!--End of Autogenerated types-->
</types>
</managedEntity>
<gateways>
<gateway>
<hostname>MFT556</hostname>
<port>1234</port>
</gateway>
</gateways>
</selfAnnounce>
</netprobe>
Please help me resolve this issue. I have tried converting $data into xml but keep getting errors. I have tried exporting the API as XML but get errors. any help is appreciated.

You can't modify an element type AFAIK, so the AttributeName-node is useless.
...managedEntity.attributes is empty, which means dot-accesing it will return an empty string which doesn't have a AppendChild()
I would create an xml-document for each string from the API and import the attribute node to the "real" xml-file and append it. Remember to use ex SelectSingleNode() to actually get the attributes-node, especially the first time when it's empty. Try:
[xml]$XML = Get-Content $SelfannounceXMLEdit
($data -split "`n") | Where-Object { $_.Trim() } | ForEach-Object {
#Create XMLdocument for <attribute name="foo">bar</attribute>
$tempchild = [xml]$_.Trim()
#Import the attribute-node in the temp xmldocument to the "real" document context
$attribute = $xml.ImportNode($tempchild.attribute, $true)
#Append attribute-node
$newType = $XML.netprobe.selfAnnounce.managedEntity.selectsinglenode("attributes").AppendChild($attribute)
}
$XML.Save($SelfannounceXMLEdit)

Related

How in Powershell see all XML Levels

I have a test xml and I want to get the value from this line ATTRIBUTE NAME="News- offers_OPT_EMAIL">F
so I can check for the value F or T
if I do below I can get the title but how do I get the above line value.
[xml]$xml = Get-Content testFile.xml
$xml
$xml.CUSTOMERS.CUSTOMER.NAME.TITLE
sample XML code
<?xml version="1.0" encoding="UTF-8"?>
<CUSTOMERS xml:lang="en">
<CUSTOMER CREATED_DATE="2018-01-01 05:18:53.0" GROUP_ID="" ID="95656565">
<NAME>
<TITLE>M</TITLE>
<FIRST_NAME>Joe</FIRST_NAME>
<LAST_NAME>Smith</LAST_NAME>
</NAME>
<GENDER/>
<DATE_OF_BIRTH/>
<ADDRESS>
<ADDRESS_LINE_1>1 White House</ADDRESS_LINE_1>
<ADDRESS_LINE_2>Old Ave</ADDRESS_LINE_2>
<ADDRESS_LINE_3/>
<TOWNCITY>LONDON</TOWNCITY>
<COUNTY/>
<POSTCODE>18659</POSTCODE>
<COUNTRY>France</COUNTRY>
</ADDRESS>
<ADDRESS>
<ADDRESS_LINE_1>175 avenue de la division Leclerc</ADDRESS_LINE_1>
<ADDRESS_LINE_2/>
<ADDRESS_LINE_3/>
<TOWNCITY>Antony</TOWNCITY>
<COUNTY/>
<POSTCODE>92160</POSTCODE>
<COUNTRY>France</COUNTRY>
</ADDRESS>
<CONTACT_DETAILS>
<TELEPHONE MARKETING_OPT_IN="F" TYPE="MOBILE">0123456789</TELEPHONE>
<EMAIL MARKETING_OPT_IN="F">johnsmith#gmail.com</EMAIL>
</CONTACT_DETAILS>
<ATTRIBUTE NAME="News- offers_OPT_EMAIL">F</ATTRIBUTE>
<NOTE>NA</NOTE>
</CUSTOMER>
You could use SelectSingleNode or SelectNodes with an XPath expression. There are several options to achieve what you want, depending on your intention, but this would be one way to do it:
# finde the nodes
$nodes = $xml.SelectNodes("//*[local-name()='ATTRIBUTE'][#NAME='News- offers_OPT_EMAIL']")
# get value
$nodes.InnerText
Or if the value of the attribute doesn't matter, simply do:
$xml.customers.customer.attribute.'#text'

web.config changes for KB3159706

i spent a few hours trying to code this by myself, but i don't know much about editing the web.config and all the examples i found don't come close to what i need. CHANGE#1 is unique because it does include the typical key=value.
I want to be able to script (PowerShell) the required modifications of Web.Config, only if the values do not already exist.
CHANGE#1:
Onsert this
(if not already there and "true"): multipleSiteBindingsEnabled="true"
<serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true"/>
CHANGE#2:
Insert this if not already there:
<endpoint address=""
binding="basicHttpBinding"
bindingConfiguration="SSL"
contract="Microsoft.UpdateServices.Internal.IClientWebService" />
<endpoint address="secured"
binding="basicHttpBinding"
bindingConfiguration="SSL"
contract="Microsoft.UpdateServices.Internal.IClientWebService" />
It goes between here:
<services>
<service
name="Microsoft.UpdateServices.Internal.Client"
behaviorConfiguration="ClientWebServiceBehaviour">
<!-- ... CODE FROM CHANGE#2 GOES HERE ... -->
</service>
</services>
This is the code so far for change#1 (not working):
$sWSUSwebConfig = "C:\Program Files\Update Services\WebServices\ClientWebService\Web.Config"
$xFileContent = [Xml](Get-Content $sWSUSwebConfig)
$root = $xFileContent.get_DocumentElement()
foreach ($item in $root."system.serviceModel"."serviceHostingEnvironment") {
if ($item."multipleSiteBindingsEnabled" -ine "true") {
$activeConnection = $root.serviceHostingEnvironment
$activeConnection.SetAttribute("multipleSiteBindingsEnabled", "true")
#$item.add."multipleSiteBindingsEnabled" = "true"
$iKeyFound = $true
}
}
$xFileContent.Save("c:\temp\web.config")
Reference for modifications: step 3 from kb3159706.

Adding suffix to a list with foreach

I am including a .properties file, which has a list of properties:
configuration.files = file1, file2
configuration.files.file1.source = config/filename1
configuration.files.file2.source = config/filename2
Now I need the paths for each file changed to something like this:
vendor/project/config/filename1
vendor/project/config/filename2
To achieve that, I tried to foreach this list and prepend that suffix and overriding the existing property:
<foreach list="${configuration.files}" target="_prepend-vendor-path" param="file" >
<property name="configuration.files.${file}.source" value="/vendor/project/${configuration.files.${file}.source}" override="true"/>
</foreach>
<target name="_prepend-vendor-path" >
<echo msg="${configuration.files.${file}.source}" />
</target>
This doesn't work and I can't figure out why. Is it even possible to use target names like ${suffix}.name ? If not, how could I achive my goal here?
I just did some workaround for this, writing out the properties and their values to a file and readin them after the loop has finished with override = true:
<target name="_prepend-vendor-path" >
<exec dir="${project.basedir}" command="echo configuration.files.${file}.source = /vendor/project/${configuration.files.${file}.source} >> ${project.temp.config}" passthru="true" checkreturn="true" />
</target>
and after the foreach simply:
<property file="${project.temp.config}" override="true"/>
For some reason the properties won't be overridden in the foreach and I just can't figgure out why, but this little trick made it for me.
You can suffix your property values from your file with the property task using a filterchain and a regular expression replacement:
<?xml version="1.0"?>
<project name="Phing Build Tests" default="append-custom-path" basedir=".">
<target name="append-custom-path">
<property file="prop.properties">
<filterchain>
<replaceregexp>
<regexp pattern="^(.*)" replace="vendor/project/$1"/>
</replaceregexp>
</filterchain>
</property>
<echo>${configuration.files.file1.source}</echo>
<echo>${configuration.files.file2.source}</echo>
</target>
</project>

BIML GetDropAndCreateDdl creating incorrect lengths for datatypes

When I use GetDropAndCreateDdl to generate CREATE scripts for tables, I get datatypes for columns that are different than what the datatypes actually are.
This results in a package validation error that "the error output has properties that do not match the properties of its corresponding data source column" and a validation status of "VS_NEEDSNEWMETADATA".
If I right-click the connection source, select Show Advanced Editor, and take a look at the Column Mappings, I can see that [Column1] in the Available External Columns list has a different Length than the datatype that was generated in the GetDropAndCreateDdl. I can delete and re-create the metadata mappings, but that is not a viable solution since there are many dataflow tasks.
How do I get GetDropAndCreateDdl to create the correct datatypes with correct lengths?
I am using ImportDB to get the list of tables, metadata, etc.
Environment.biml
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Connections>
<OdbcConnection Name="OdbcSrc_DV" ConnectionString="Dsn=Source-32bit-test;" />
<OleDbConnection Name="OleDbDst_Staging" ConnectionString="Provider=SQLNCLI11;Server=SQL-DEV;Initial Catalog=Source_Staging;Integrated Security=SSPI;" />
</Connections>
<Databases>
<Database Name="Source" ConnectionName="OdbcSrc_DV" />
<Database Name="Source_Staging" ConnectionName="OleDbDst_Staging" />
</Databases>
<Schemas>
<Schema Name="dbo" DatabaseName="Source" />
<Schema Name="dbo" DatabaseName="Source_Staging" />
</Schemas>
</Biml>
CreateTableMetadata.biml
<## import namespace="System.Data" #>
<## import namespace="Varigence.Biml.CoreLowerer.SchemaManagement" #>
<#
var sourceConnection = RootNode.DbConnections["OdbcSrc_DV"];
var importResult = sourceConnection.ImportDB("", "", ImportOptions.ExcludeForeignKey | ImportOptions.ExcludeColumnDefault | ImportOptions.ExcludeViews);
var tableNamesToImport = new List<string>() { "Test_Table" };
#>
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Tables>
<# foreach (var table in importResult.TableNodes.Where(item => tableNamesToImport.Contains(item.Name)).OrderBy(item => item.Name)) { #>
<Table Name="<#=table.Name#>" SchemaName="Source.dbo">
<Columns>
<#=table.Columns.GetBiml()#>
</Columns>
<Annotations>
<Annotation AnnotationType="Tag" Tag="SourceSchemaQualifiedName"><#=table.SchemaQualifiedName#></Annotation>
</Annotations>
</Table>
<# } #>
</Tables>
</Biml>
DeployTargetTables.biml
<## template tier="2" #>
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="MasterTableDeploy" ConstraintMode="Parallel">
<Tasks>
<# foreach (var table in RootNode.Tables) { #>
<ExecuteSQL Name="SQL CREATE <#=table.Name#>" ConnectionName="OleDbDst_Staging">
<DirectInput><#=table.GetDropAndCreateDdl()#></DirectInput>
</ExecuteSQL>
<# } #>
</Tasks>
</Package>
</Packages>
</Biml>
CreateLoadPackages.biml
<## template tier="2" #>
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="Copy Data" ConstraintMode="Parallel">
<Tasks>
<# foreach (var table in RootNode.Tables) { #>
<ExecuteSQL Name="SQL TRUNCATE <#=table.Name#>" ConnectionName="OleDbDst_Staging">
<DirectInput>TRUNCATE TABLE <#=table.Name#></DirectInput>
</ExecuteSQL>
<Dataflow Name="DFT LOAD <#=table.Schema.Name#>_<#=table.Name#>">
<PrecedenceConstraints>
<Inputs>
<Input OutputPathName="SQL TRUNCATE <#=table.Name#>.Output" />
</Inputs>
</PrecedenceConstraints>
<Transformations>
<OdbcSource Name="ODBC_SRC <#=table.Name#>" Connection="OdbcSrc_DV">
<DirectInput>SELECT <#=table.GetColumnList()#> FROM <#=table.GetTag("SourceSchemaQualifiedName")#></DirectInput>
</OdbcSource>
<OleDbDestination Name="ODBC_DST <#=table.Name#>" ConnectionName="OleDbDst_Staging">
<TableOutput TableName="<#=table.ScopedName#>" />
</OleDbDestination>
</Transformations>
</Dataflow>
<# } #>
</Tasks>
</Package>
</Packages>
</Biml>
Here is the script that gets created from the ODBC source -
And here are the results from INFORMATION_SCHEMA.COLUMNS for the same table -
Have you tried using the newer method: GetDatabaseSchema? I've found it to be much more reliable across different connection types.
http://www.cathrinewilhelmsen.net/2015/07/12/biml-extension-methods-getdatabaseschema/

How do I read/write App.config settings with PowerShell?

I'd like to use PowerShell as part of our automated build process to update an App.config file while deploying into our test environment. How can I do this?
Given this sample App.config: C:\Sample\App.config:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<connectionStrings>
<add name="dbConnectionString"
connectionString="Data Source=(local);Initial Catalog=Northwind;Integrated Security=True"/>
</connectionStrings>
</configuration>
The following script, C:\Sample\Script.ps1, will read and write a setting:
# get the directory of this script file
$currentDirectory = [IO.Path]::GetDirectoryName($MyInvocation.MyCommand.Path)
# get the full path and file name of the App.config file in the same directory as this script
$appConfigFile = [IO.Path]::Combine($currentDirectory, 'App.config')
# initialize the xml object
$appConfig = New-Object XML
# load the config file as an xml object
$appConfig.Load($appConfigFile)
# iterate over the settings
foreach($connectionString in $appConfig.configuration.connectionStrings.add)
{
# write the name to the console
'name: ' + $connectionString.name
# write the connection string to the console
'connectionString: ' + $connectionString.connectionString
# change the connection string
$connectionString.connectionString = 'Data Source=(local);Initial Catalog=MyDB;Integrated Security=True'
}
# save the updated config file
$appConfig.Save($appConfigFile)
Execute the script:
PS C:\Sample> .\Script.ps1
Output:
name: dbConnectionString
connectionString: Data Source=(local);Initial Catalog=Northwind;Integrated Security=True
Updated C:\Sample\App.config:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<connectionStrings>
<add name="dbConnectionString"
connectionString="Data Source=(local);Initial Catalog=MyDB;Integrated Security=True" />
</connectionStrings>
</configuration>
The code can be much more shorter (based on Robin's app.config):
$appConfig = [xml](cat D:\temp\App.config)
$appConfig.configuration.connectionStrings.add | foreach {
$_.connectionString = "your connection string"
}
$appConfig.Save("D:\temp\App.config")