Octopus Web.config transforms for client endpoint address - web-config

I have a web project that has uses two service endpoints located in the Web.config file under the client --> endpoint --> address portion
I have found the following per Octopus Variables section but cannot seem to find any reference of how to address the changes using and actual variable like you normally would
I am using the webui for Octopus which would be
http://{server-name}/app#/projects/{project-name}/variables
variable-substitution-syntax
I attempted assigning the variables like so, but the values never updated
the original entries look like the following
<endpoint address="http://services-test.example.com/test.svc/soap" binding="basicHttpBinding" bindingConfiguration="soap" contract="test.service" name="soap" />
Name Address Instance
Endpoint[A].Address test-service-a.example.com 1
Endpoint[B].Address test-service-b.example.com 2
Is this something that is ever possible using Octopus Variables? (I know it can be done using regular Web.config Transforms, as we are doing that already).
If it is possible what is would the correct replacement value for the
endpoint address
be and how would I accomplish this for multiple different endpoint addresses?

Sounds like you are most of the way there. If it's already working with your Web.config transforms then all you need to do is replace the value IN the transform with the variable replacement token.
For example: Web.Release.Config
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
<system.serviceModel>
<client>
<endpoint address="http://#{Server1}/test.svc/soap" name="x1"
xdt:Locator="Match(name)" xdt:Transform="SetAttributes(address)" />
<endpoint address="#{Endpoint2}" name="x2"
xdt:Locator="Match(name)" xdt:Transform="SetAttributes(address)" />
</client>
</system.serviceModel>
</configuration>
Of course there are lots of options here.
For filenames you could stick with default conventions 'Web.Release.config' or go with 'Web.[Environment].config' or go with something custom. We use 'Web.Octopus.Config' so that it wont get picked up by any other process.
More on naming transforms here: https://octopus.com/docs/deploying-applications/configuration-files#Configurationfiles-Namingconfigurationtransformfiles
More on custom transforms (Web.Octopus.com) here: https://octopus.com/docs/deploying-applications/configuration-files#Configurationfiles-AdditionalConfigurationTransforms
For variables, you could define a variable for just the server (name=x1) which is simpler or just put the whole address in a variable with gives Octopus a lot more control (name=x2).
The key part is getting the variable replacement tokens into the config. Octopus runs the variable substitution on config files first, and then runs the transforms. What that means is the first pass will replace the tokens in your Web.Release.Config, and then Octopus will run the transforms against Web.config
Hope that helps.

Related

Web.config variable update not working with Azure DevOps pipeline (File Transform)

I'm trying to use the task "File Transform" within my pipeline to modify a few values inside my web.config file during deployment.
I created the tasks as the following:
And in the pipeline, inside the variables, I set up the key & value that I want to replace in the XML file.
Everything seems to go fine, however, when I check the log, I see this
So, the file was simple skipped and the transformation didn't work.
The variable that I'm trying to modify in the web.config file has the following path inside the XML file
<aspNetCore>
<environmentVariables>
<environmentVariable name="x" value="y" />
</environmentVariables>
</aspNetCore>
</configuration>
I read the documentation multiples times, but it's not clear if I'm doing something wrong.
Please, could you provide some suggestion of what I need to check to make it work?
Thank you
According to the document about XML variable substitution:Variable substitution takes effect only on the applicationSettings, appSettings, connectionStrings, and configSections elements of configuration files. It does not apply to your environmentVariable element. It worked well on my side when I change value in appSettings element.
Please refer to the samples in the document about file transform.
I'm answering my own question.
Based on the comments that I received, the solution that I found was the following:
1- Use Replace Token Task (https://github.com/qetza/vsts-replacetokens-task#readme) in the pipeline AFTER the deployment. I also removed that "File Transform", since it wouldn't work for what I needed.
2- In the configuration, I pointed the root folder to the deployed application, no the Zip file, as instructed by this other question (Azure Dev ops replace tokens per environment in release pipeline)
3- In the web.config file, I used the token prefix and suffix, as suggested "#{token}#
4- Added the variables to the variable groups
After deploying, I checked that it worked fine.

Automate deploy of Machine Key in a shared config IIS

I'm using Azure DevOps to deploy an ASP.NET application to an IIS servers on-prem. The IIS servers are using a shared configuration so they need a custom Machine Key setup.
I can use XML transform to add add the machineKey entry in the Web.Config
<system.web>
<machineKey decryptionKey="{hex-key value}" validationKey="{hex-key value}"/>
</system.web>
but I don't want to have the actual keys in source control so I'll need to replace those values at deploy time. Substitution is easy enough for appsettings and connection strings but how can I substitute values in the System.Web section of the Web.Config?
How can I substitute values in the System.Web section of the
Web.Config?
What about consider to use one extension Replace token? For why I recommend it is because it can achieve the demand that it can only be replaced during the pipeline running.
Also, its usage is very convenient. Just need to specify the prefix and suffix in the task, and then make apply them in to your web.config file.
Then specify the corresponding variables with same name in Variables tab.
Only this, during the pipeline running, the task could find the corresponding token correctly and replace the value into it.
For detailed steps, you could refer to my previous answer for details: Use replace token task.

Different Endpoint Certificates Per Environment in Service Fabric

I have a service fabric application that exposes an SSL endpoint. I would like to use a different certificate based on the environment. I'm trying to do this with parameters in the ApplicationMainfest.xml file in the same way that I specify other things, such as instance counts. However, parameters appear not to be working for this. I'm wondering if this is actually true and if there are certain things that you cannot parameterize. Also, is there any way to specify a different certificate based on the environment?
Here are the relevant pieces from my application manifest:
<Parameter Name="CERTNAME" DefaultValue="MyCert" />
...
<Certificates>
<EndpointCertificate X509FindValue="..." Name="MyCert" />
<EndpointCertificate X509FindValue="..." Name="SVSSL" />
</Certificates>
<Policies>
<EndpointBindingPolicy EndpointRef="ServiceEndpointHttps" CertificateRef="[CERTNAME]" />
</Policies>
On deployment, I get the following error:
Register-ServiceFabricApplicationType : The CertificateRef '[CERTNAME]' in EndpointBindingPolicy is invalid. There is no matching Certificate in the corresponding ApplicationManifest.
Today the certificate value itself is parameterizable but not the Ref. So instead of changing the reference or the name, you would parameterize the X509FindValue and keep the endpointbindingpolicy the same.
As a note, just any time you run into something you want to parameterize but can't figure out how to do it, there are a few options. Consider for example most things in the Service Manifest, like the port that the service listens on (if you have it statically configured). There are some other ways around this:
Create different manifests (service manifests or application manifests) and replacing them when creating the application package for a given environment
Using something during your build/deployment stages (such as the Tokenizer Task in VSTS) to replace a stub value with the actual value given the environment that the package is being crafted for
Move most of the endpoint configuration stuff to settings.xml and replace those values via the normal application parameter/override behavior. This would mean taking on the work of configuring your endpoints yourself, however.

AppSettings values not being picked up in web.config in subdirectory

We have a set of WCF Services being hosted in IIS. In our architecture, we have different instances of the service that vary in configuration hosted in subdirectories under the root.
I have a Web.Config in the root that has some configuration information in the AppSettings section. Specifically, it looks like this:
<appSettings>
<add key="Environment" value="Local" />
</appSettings>
In the different subdirectories, I add another Web.config which adds other settings that are specific to the services in that subdirectory.
<appSettings>
<add key="Subdir" value="ABC" />
</appSettings>
However, when I invoke a service (.svc) file in the subdirectory, there is only one value in the ConfigurationManager.AppSettings collection ("Environment"). The "Subdir" key is nowhere to be found. (BTW, the code is written using ConfigurationManager, but I've also tried using WebConfigurationManager, and the result is the same.)
The documentation on MSDN clearly says that the Web.config files in nested directories are supposed to be cumulative. So why isn't my "Subdir" key showing up in that collection?
Thanks in advance for any help you can give me.
For the benefit of anyone else that encounters this, I found the answer.
Even though this app is a service running under IIS, and is being accessed through IIS, the point where this config element is needed is deep down in a derived class of System.ServiceModel.ServiceHostFactory. At that point, HttpContext.Current is null, so the ConfigurationManager has no access to Web directory structure information, and has to depend simply on the application configuration, which is at the top level.
So our solution is to make each Subdirectory an IIS application, and give it all of its own config information in one place.

How does the web server locate a file on server through URL?

Has anyone ever tried to implement a web server? Or know something about the underhood of a working web server program? I am wondering what happens exactly from when a URL is received by the web server to a file on the web server is located and sent back as response.
Does the server just keep an internal table to remember the mapping between the URLs it supports and the corresponding local paths? Or is there anything more tricky?
Thanks!
Update
Thanks for your replies. Here's my understanding for now.
I checked with the Microsoft IIS (Internet Information Service), I noticed that IIS can host multiple sites, and foreach site IIS memorize its root path on the local file system. Different sites on the same host share the same host name or IP, and they are differentiated by separate ports. For example:
http://www.myServer.com:1111/folderA/pageA.htm
The web server will use www.myServer.com:1111 part of the URL string to locate which path on its local file system will be used, and then in that local path, it searches for subfolder folderA and then the file pageA.htm.
The web server only need to memorize the following mapping between 2 plain strings:
"http://www.myServer.com:1111/" <---> "D:\myWebRoot"
I don't know where this kind of mapping info is stored, maybe some config files for the Web Server Program in question.
But the result of this mapping granularity is that we could only access content within that mapped local folder. We couldn't do arbitray mapping.
Update - 2 -
I found where the IIS keep the mapping, here's some quotes from applicationHost.config:
<sites>
<site name="Default Web Site" id="1" serverAutoStart="false">
<application path="/">
<virtualDirectory path="/" physicalPath="%SystemDrive%\inetpub\wwwroot" />
</application>
<bindings>
<binding protocol="http" bindingInformation="*:80:" />
<binding protocol="net.tcp" bindingInformation="808:*" />
<binding protocol="net.pipe" bindingInformation="*" />
<binding protocol="net.msmq" bindingInformation="localhost" />
<binding protocol="msmq.formatname" bindingInformation="localhost" />
</bindings>
</site>
<site name="myIISService" id="2" serverAutoStart="true">
<application path="/" applicationPool="myIISService">
<virtualDirectory path="/" physicalPath="D:\MySites\MyIISService" />
</application>
<bindings>
<binding protocol="http" bindingInformation="*:8022:" />
</bindings>
</site>
<siteDefaults>
<logFile logFormat="W3C" directory="%SystemDrive%\inetpub\logs\LogFiles" />
<traceFailedRequestsLogging directory="%SystemDrive%\inetpub\logs\FailedReqLogFiles" />
</siteDefaults>
<applicationDefaults applicationPool="DefaultAppPool" />
<virtualDirectoryDefaults allowSubDirConfig="true" />
</sites>
Update - 3 -
After I read foo's reply, my undersanding of a "server" is enlarged. I want to make some comment based on my recent learning of WCF.
No matter what kind of server it is, we could always send messages to them by specifying the protocol, URL, port. For example:
[http://www.myserver.com:1111/]page.htm
[net.tcp://www.myserver.com/]someService.svc/someMethod
[net.msmq://www.myserver.com/]someService.svc
[net.pipe://localhost/]
After the messages arrives at the server program using the parts in square bracket of above URLs, the rest part of the url will send to the server program as input for further processing. And the following behaviour could be as simple as static content feeding or as complex as dynamic content generating.
Depends on the webserver and what its focus is.
(For all items, checking access rights, remapping and such steps apply of course.)
General-purpose webservers like Apache start out with files and directories, so they split up the URL into a hierarchical path description, try to find a file at the given location, and serve it if it exists. (This gets more complex with modules and filetypes; some filetypes imply processing the file as a script and returning the script output rather than just piping out the file contents, and so on).
Application servers like Tomcat do a mapping to servlets; if they have found a servlet that will handle the URL, they call it and pass any leftover URL parts/parameters to it for further handling.
Embedded webservers may even use hardcoded lookup tables for available URL patterns, directly mapping to functions to be called.
Special-purpose webservers will do whatever is required; some won't even parse the URL but just the other headers (like some streaming servers do).
It all depends on what you want to achieve. In most cases, you will be best off with nginx or Apache and maybe some modules and/or finetuning.
Be aware that any HTTP header can be used for mapping the request to whatever means of producing output you have. Hostname, port and URL are used most often, but you may as well take language or client IP or other header data and use them in the mapping.
So for your question: Yes, it can be as simple as that; and yes, it can be substantially more tricky (with mapping, rewriting, and complex processing).
For servers that serve "files", a typical approach is to treat the path portion of the URL as a relative path starting at a "web root" directory defined in the server's configuration. However, a URL doesn't have to correspond to a file on disk at all; it could correspond to an object or method in a running web application, or a database record, or anything else.
For static files there's usually no means of a mapping. The only what the webserver need to know is the absolute disk file system path to the public web document root which is usually definied somewhere in some deployment configuration file (httpd.conf for Apache HTTPD, server.xml and/or context.xml for Apache Tomcat, etc). The webserver extracts the relevant part from the URL, converts it to an absolute disk file system path based on the path to the web document root, locates the file on disk and streams it.