I am setting up Continuous Deployment for a web app and I'm having trouble doing config transformations. The build and release both work without config transforms, but I would like to add a config transform step to ensure the correct connection strings, storage accounts, etc are in the web config.
I am using the config transform task in VSTS. The artifact that is getting built during the build process is a zip file. I am then extracting those files, and it gives an extremely deep file structure. The config transform works if I hardcode the file structure, but I worry that this could change, so I'd like to generalize it. If I do $(mypath)/**/Web.config it does not error and the logs show that it is doing transforms, but once it finishes, I look at the file in Azure and can see that the transforms weren't actually done or saved.
My requirements for this are that I can't use xslt transforms or the XML transforms(web.environment.config) because we would like to keep the important values out of source control and in VSTS. We are also doing config transforms on NLog.config, and those transforms are setup the same way and seeing the same issues. Any help on getting this more general path to work, or a different way of solving this would be greatly appreciated.
Other potentially helpful information:
MSBuild Arguments in Build Solution step:
/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactstagingdirectory)\\"
Path Structure
$(My.Paths.InstallDirectory)\Content\d_C\a\1\s\Apps\ProjectName\ProjectName\obj\Release\Package\PackageTmp\Web.config
There is File Transforms & Variable Substituion Options in Azure App Service Deploy task, so you can use XML variable substitution or JSON variable substitution to do it without extracting the package file (zip file):
For example:
Add a new variable in Release definition (Scope: select an environment), for example: Name: DefaultConnection; Value: Data source=xxx…), clicking lock icon to change variable type to secret.
Check XML variable substitution option in Azure App Service Deploy task
Then during the deploying, any config and parameters.xml files will be updated with that variable value if the key or name entries in appSettings, applicationSettings and connectionStrings sections are matched the variable name
Related article: XML variable substitution
Another way is that you can change the value in web deploy parameters file (SetParameters.xml) through Token replace task (You also can specify the different file in Azure App Service Deploy task per to Environment of release, such as SetParameters.Dev.xml, SetParameters.QA.xml)
An article about Configuring Parameters for web package deployment
BTW: You can remove the source structure from web deploy package by specifying /P:PackageTempRootDir="" msbuild argument
Related
Im setting up a new build and deploy pipeline in Azure Devops. It is an older Web application with some transformation files for the web.config. In the old days we would build the same code x times depending on how many environments. This this is no longer necesary as I read from here https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/transforms-variable-substitution?view=vsts#xmltransform
it looks like the deploy pipeline can pick up the changes from my transform file.
But the problem is that my other transform files does not get included in the package so I get these warning message:
[warning]Unable to apply transformation for the given package. Verify the following.
[warning]1. Whether the Transformation is already applied for the MSBuild generated package during build. If yes, remove the <DependentUpon> tag for each config in the csproj file and rebuild.
[warning]2. Ensure that the config file and transformation files are present in the same folder inside the package.
And yes when i download the artifact the Web.[stage].config files are not there as suggested.
Is there some setting somewhere that let me include these files? Or stop them from being transformed?
For Web applications
MSBuild knows how to transform the web.config files based on the following settings/properties/parameters (in order)
Build Configuration dotnet publish --configuration Release
Publish profile dotnet publish --configuration Release /p:PublishProfile=FolderProfile
Environment dotnet publish --configuration Release /p:EnvironmentName=Production
Custom file transform dotnet publish --configuration Release /p:CustomTransformFileName=custom.transform
I think it's typical for developers to make this happen based on build configuration only, and I believe MSBuild (and dotnet) know how to do this based on the <DependentUpon>Web.config</DependentUpon> element in the Web.[configuration].config item in the project or build script file.
Azure DevOps Release Pipelines is a little different.
The pipeline wants to transform your web.config after the project has been built/published and doesn't know how to do that if MSBuild (or dotnet) has already made an attempt at it. Thus:
[warning]Unable to apply transformation for the given package. Verify the following.
[warning]1. Whether the Transformation is already applied for the MSBuild generated package during build. If yes, remove the tag for each config in the csproj file and rebuild.
[warning]2. Ensure that the config file and transformation files are present in the same folder inside the package.
The warning text states:
Remove the <DependentUpon> tag for each config in the csproj
Thus: you need to remove the tag from the csproj to prevent MSBuild from transforming the files
Or: you need to use the /p:TransformWebConfigEnabled=False argument to MSBuild. (note: I believe it is correct that this can be used w/o removing the dependent upon tag, but I could be wrong)
Make sure the transform source and target files are in the same folder inside the package.
There may be several ways to do this. I've chosen to mark the transform source config files as content to force MSBuild to include them in the published package.
Now you need to organize your release pipeline in accordance with the File Transforms and Value Substitutions documentation.
[section]Starting: IIS Web App Deploy
====================================
Task : IIS Web App Deploy
Description : Deploy a website or web application using Web Deploy
Version : 0.0.51
Author : Microsoft Corporation
Help : More information
====================================
...[command]C:...\ctt\ctt.exe s:C:...\Web.config t:C:...\Web.Release.config d:C:...\Web.config pw i
[command]C:...\ctt\ctt.exe s:C:...\Web.config t:C:...\Web.Development.config d:C:...\Web.config pw i
XML Transformations applied successfully
...
For Non-Web Applications Needing .config Transformation
Getting your .config files to the release pipeline can happen several ways. Here are two.
Your release should have "access" to the repository as a part of the artifact, which will ensure that the deploy agent downloads the source (not desirable IMHO).
You will need to include the web.[stage].config files as part of your build artifact with a copy task, or a minimatch that picks them up.
Once you have the .config files available to the release pipeline
You can use the File Transform Task or XDT Transform Task to perform the transformation operations.
Option 2 is the route I've gone.
Here is an image of what that task looks like for me.
That task puts the config in the artifact from the build that I can then use in the release pipeline without rebuilding xx times.
Cleanup
If you're in a position where you care to not have the transform files persisting on the agent after the release is complete, then you'll need to add that to your pipeline.
I have a webjob and a webapp (both separate projects), I would like to build one artifact for all environments and do the transformations during the release step rather than build, as that way I have to create an artifact per environment. So I am creating two separate artifacts (one for webapp and one for webjob) per environment and applying the xml transforms during publish to the app service, now everything works fine, except that the transformed file for the webjob is placed in the root directory of the webapp, which is not what I intend to do. I would like the file to be placed as app_data\jobs\continous\myjob\myjob.exe.config
I've seen the slow cheeta and CTT transforms, but those are out of the scope of this question as they do transform only on build.
You can use Kudu api to run the commands to copy or delete files.
More information about how to call kudu api during build/relase, you can refer to: How to access Kudu in Azure using power shell script
You should transform during build and your transformation should happen as tokens in the build rather than actual target values. Then in deployment you can replace the token values with actual target environment values. That is the proper approach for packaging and deployment of any application type.
Steps should be
In build tokenize the configurations with transformation (use this extension task)
Package tokenized configurations as build output
At deployment apply target values to tokens (use replace tokens task comes with marketplace extension, which replaces values of defined configs with release variable values by mapping names automatically. in other words token should be the parameter name)
Here is an example done on a windows service. But it is applicable for webjobs (have tested this) as well and it is the proper solution.
I am configuring VSTS build and release process.
I have a scenario where I have DEV and QA environments. I have different config files for each of the environments, think of the naming convention Test.Dev.Config
Test.QA.Config
In the "Copy files to" step of the build process, in the "contents" field, are you able to use the BuildConfiguration variable in order to tell the build process to copy the configuration files relevant to the BuildConfiguration:
so that the output of the package contains the Test.DEV.config files for the Dev build step and the Test.QA.config file for the QA build step.
I am basically testing out a options for config transforms in VSTS
Using a VSO variable combined with a string literal often doesn't work as expected within a task like File Copy. Also, you don't want to do that in every task that requires build configuration specific config file. What you can do instead, is to have a separate variable in your build definition that derives from BuildConfiguration variable like so:
Variable Value
BuildConfigFile *.$(BuildConfiguration).config
And then, within the File Copy task, you can directly use this variable as:
_PublishedWebsites\App_Config\$(BuildConfigFile)
For more details on defining one variable in terms of the other, refer this SO post.
Web.config file is transformed locally with different environments like(DEV, QA, Prod). How these environment specific config files will be used while release the application on different environment like(DEV, QA, Prod) on VSTS.
There is XDT Transform extension that can apply transform for config file, so the easy way is that you can include/add the related configure files in release artifact, then configure transform per to release environments.
Another way is that you can replace the value in configure file through Replace Tokens or Tokenization task.
On the other hand, if you are deploying the app through Azure App Service Deploy task, there is File Transforms & Variable Substitution Options that can do transform.
Both the Azure and IIS deployment tasks now support file transforms and variable substitution.
The File Transformation will look for and apply a *.<environment>.config transform file where <environment> matches the Azure Pipeline environment for which the task is executing.
Variable Substitution will apply the value of any pipeline variables to the corresponding connectionStrings or appSettings entries where the name matches the variable name.
Yesterday I published a blog on how to do this in VSTS: https://www.4dotnet.nl/kennis/blog-deploying-your-asp-net-web-app-and-database-to-azure-using-vsts
In other scenario's, I've successfully used the XDT Transformation Tool: https://ctt.codeplex.com/
I have a project which I want to deploy via team city but when it builds I want to generate a number of different artifacts.
In my app.config file I have an app setting called "platform" which I want to change the value of for each artifact. For example on build I would get three artifacts, dev, test, staging which would have had the "platform" setting changed to the corresponding value.
I have created the powershell script and run it successfully outside of team city, but I have not been able to work out how to set the file directory where the app.config exists within team city.
Is there a system variable I can use to target the current builds app.config file? Or is there a different / better way that I can accomplish what I want to do?
Thanks
Or is there a different / better way that I can accomplish what I want to do?
While this sort of question can lead to an argument over which is better... Based on what you have described, I believe that Octopus Deploy would be a good fit. Octopus Deploy integrates very nicely with TeamCity, and it handles the transformation configurations that are required to allow deployment to each of your environments, i.e. Dev, Test, and Staging. You would simply configure these as Environments within your Octopus Configuration, and during installation, Octopus can modify the configuration files as required, based on variables that you define.
I would recommand to create 3 build steps inside your TeamCity configuration :
Inside the Parameters tab, you can define configuration parameters. Just define here the 3 values : ie: PlatformDev, PlatformTest, PlatformStaging
Using Visual Studio Runner
You might be using Visual Studio runner type. So, here you can add specific values from MSBuild properties:
Dev Step:
Passing the parameter Dev, and executing your code
/p:CustomPlatform="%PlatformDev%"
Test Step:
Passing the parameter Test, and executing your code.
/p:CustomPlatform="%PlatformTest%"
Staging Step:
Passing the parameter Staging, and executing your code.
/p:CustomPlatform="%PlatformStaging%"
Then, the value CustomPlatform would be accessible inside your MSBuild scripts.
Using Powershell hand-made scripts
If you are using a powershell script to run your compilation/artifact creation, you can just add this CustomPlatform as a parameter of your script, and use it directly.
Using XSLT
Another way to do this might be to use XSLT to transform you app.config file, using a specific value.