I cloned an existing Build pipeline, where everything is the same except the variable values, however the XML transformation is not happening correctly, it's transforming using the incorrect file.
The QA pipeline works successfully, and looks like the below:
Pipeline variables:
Build solution step:
Log file showing transformation with "QA", which is correct:
The Dev pipeline is the exact same as above, but with different variable values:
Issue is that it's picking up "debug" file instead of "dev" file:
All config files exist in the branch I am building:
Any idea why the Dev pipeline is using Web.Debug.config instead of Web.Dev.config file during build process?
I figured out the issue. The transformation is happening as part of the build solution step - in the solution file. The issue is that someone had changed the configuration in the solution file so that every config defaulted to "Debug", except for QA, which was set correctly. Once I reverted that change, everything worked as expected.
Related
We have 3 different release environments DEV, UAT, PROD.
I made a small change in a script (just added a comment), just so that I can test the deployment process, as the previous people working on this project have left and I will be taking it over.
I managed to deploy to DEV, but when I try to deploy to UAT and PROD I get the following error:
2023-01-16T12:08:55.5352211Z Config file: D:\a\r1\a_Azure Data Factory CI\drop\deployment\config-uat.csv
2023-01-16T12:08:55.5519094Z Validation of config file completed.
##[error]ADFT0010: Wrong path defined in config for object(path): pipeline.Cube 15 mins(properties.activities.typeProperties.url)
Do I just need to make a small change in a config file or would I need to get my head around this particular pipeline?
I went into the pipeline within the UAT ADF, but could not see anything that would cause it to fail.
I can manually trigger that particular pipeline in UAT and it works fine.
I have looked at the setting for the activity and it contains the following url: https://northeurope.asazure.windows.net/servers/asvfmmymiuatne/models/mymi/refreshes
The URL looks to be exactly the same url as defined in the config-uat.csv:
https://northeurope.asazure.windows.net/servers/asvfmmymiuatne/models/mymi/refreshes
I'm completely new to Azure DevOps Pipelines so if I'm doing something incorrectly I'd appreciate a nod in the right direction... I setup a build pipeline and that seems to be working, now I'm trying to setup a release pipeline in order to run tests, it's mostly based on Microsoft's documentation:
https://learn.microsoft.com/en-us/azure/devops/test/run-automated-tests-from-test-hub?view=azure-devops
Before running tests I need to transform a config file to replace some variables like access keys, usernames, etc. What I setup is what I have below but for the life of me I can't figure out what text box Package or folder refers to. The documentation is super helpful as you can imagine:
File path to the package or a folder
but what package or what folder is this referring to??? I've tried several different things but everything errors with
##[error]Error: Nopackagefoundwithspecifiedpattern D:\a\r1\a\**\*.zip
or pretty much whatever I specify for a value.
The File Transform task supports the .zip files.
Test with the default File Transform task settings, I could reproduce this issue.
In Release pipeline, the file path could has one more node for the build artifacts .zip file.
The format example:
$(System.DefaultWorkingDirectory)\{Source alias name}\{Artifacts name}\*.zip
So you could try to set the $(System.DefaultWorkingDirectory)/**/**/*.zip in Package Or folder field
For example:
On the other hand, you can check the specific path in the Release log -> Download Artifacts Step.
$(System.DefaultWorkingDirectory): D:\a\r1\a
You could aslo use this specific path in the task.
Update:
If your file is Project Folder, you refer to the following sample:
File structure：
Task Settings:
Note:You only need to assign to the folder node.
You could also select the folder path via ... option.
I created a release pipeline it is working very well but the XML transformation part is not working as I wanted.
I gave a name for my stage as "Development" and I have "Web.Development.config" file as I showed below, I checked the "XML transformation option" but not is changed. When I add "Web.Release.config" it works but I want to use "Web.Development.config" for the Development environment, I want to use "Web.Production.config" for the Production environment.
When I removed the "Web.Release.config" nothing happens.
When I added the "Web.Release.config" nothing happens. It is skipping or not using "Web.Development.config".
What am I missing? What is the wrong part?
After trying a lot of things, finally, I found a solution to my problem. These are the steps:
Go to your build definition, choose "Build Solution" task and add "/p:autoparameterizationwebconfigconnectionstrings=false" as an argument. If you don't do that, the xml transformation process will skip replacing "connectionString" attribute.
Make your all config files as content and mark them as copy always. After your edit, they should be like below in your project file. You should remove Web.config tag.
Go to your release definition and check the XML transformation option.
Make sure that you gave the same name to your stage with your *..config file.
You can check this link for more details: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/transforms-variable-substitution?view=azure-devops
By default Azure DevOps uses your web.config and web.release.config and then applies an environment specific config. There is no way to exclude web.release.config from your transformations except from renaming/deleting the file.
See the docs on XML Transformation for all the details.
Im setting up a new build and deploy pipeline in Azure Devops. It is an older Web application with some transformation files for the web.config. In the old days we would build the same code x times depending on how many environments. This this is no longer necesary as I read from here https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/transforms-variable-substitution?view=vsts#xmltransform
it looks like the deploy pipeline can pick up the changes from my transform file.
But the problem is that my other transform files does not get included in the package so I get these warning message:
[warning]Unable to apply transformation for the given package. Verify the following.
[warning]1. Whether the Transformation is already applied for the MSBuild generated package during build. If yes, remove the <DependentUpon> tag for each config in the csproj file and rebuild.
[warning]2. Ensure that the config file and transformation files are present in the same folder inside the package.
And yes when i download the artifact the Web.[stage].config files are not there as suggested.
Is there some setting somewhere that let me include these files? Or stop them from being transformed?
For Web applications
MSBuild knows how to transform the web.config files based on the following settings/properties/parameters (in order)
Build Configuration dotnet publish --configuration Release
Publish profile dotnet publish --configuration Release /p:PublishProfile=FolderProfile
Environment dotnet publish --configuration Release /p:EnvironmentName=Production
Custom file transform dotnet publish --configuration Release /p:CustomTransformFileName=custom.transform
I think it's typical for developers to make this happen based on build configuration only, and I believe MSBuild (and dotnet) know how to do this based on the <DependentUpon>Web.config</DependentUpon> element in the Web.[configuration].config item in the project or build script file.
Azure DevOps Release Pipelines is a little different.
The pipeline wants to transform your web.config after the project has been built/published and doesn't know how to do that if MSBuild (or dotnet) has already made an attempt at it. Thus:
[warning]Unable to apply transformation for the given package. Verify the following.
[warning]1. Whether the Transformation is already applied for the MSBuild generated package during build. If yes, remove the tag for each config in the csproj file and rebuild.
[warning]2. Ensure that the config file and transformation files are present in the same folder inside the package.
The warning text states:
Remove the <DependentUpon> tag for each config in the csproj
Thus: you need to remove the tag from the csproj to prevent MSBuild from transforming the files
Or: you need to use the /p:TransformWebConfigEnabled=False argument to MSBuild. (note: I believe it is correct that this can be used w/o removing the dependent upon tag, but I could be wrong)
Make sure the transform source and target files are in the same folder inside the package.
There may be several ways to do this. I've chosen to mark the transform source config files as content to force MSBuild to include them in the published package.
Now you need to organize your release pipeline in accordance with the File Transforms and Value Substitutions documentation.
[section]Starting: IIS Web App Deploy
====================================
Task : IIS Web App Deploy
Description : Deploy a website or web application using Web Deploy
Version : 0.0.51
Author : Microsoft Corporation
Help : More information
====================================
...[command]C:...\ctt\ctt.exe s:C:...\Web.config t:C:...\Web.Release.config d:C:...\Web.config pw i
[command]C:...\ctt\ctt.exe s:C:...\Web.config t:C:...\Web.Development.config d:C:...\Web.config pw i
XML Transformations applied successfully
...
For Non-Web Applications Needing .config Transformation
Getting your .config files to the release pipeline can happen several ways. Here are two.
Your release should have "access" to the repository as a part of the artifact, which will ensure that the deploy agent downloads the source (not desirable IMHO).
You will need to include the web.[stage].config files as part of your build artifact with a copy task, or a minimatch that picks them up.
Once you have the .config files available to the release pipeline
You can use the File Transform Task or XDT Transform Task to perform the transformation operations.
Option 2 is the route I've gone.
Here is an image of what that task looks like for me.
That task puts the config in the artifact from the build that I can then use in the release pipeline without rebuilding xx times.
Cleanup
If you're in a position where you care to not have the transform files persisting on the agent after the release is complete, then you'll need to add that to your pipeline.
I am setting up Continuous Deployment for a web app and I'm having trouble doing config transformations. The build and release both work without config transforms, but I would like to add a config transform step to ensure the correct connection strings, storage accounts, etc are in the web config.
I am using the config transform task in VSTS. The artifact that is getting built during the build process is a zip file. I am then extracting those files, and it gives an extremely deep file structure. The config transform works if I hardcode the file structure, but I worry that this could change, so I'd like to generalize it. If I do $(mypath)/**/Web.config it does not error and the logs show that it is doing transforms, but once it finishes, I look at the file in Azure and can see that the transforms weren't actually done or saved.
My requirements for this are that I can't use xslt transforms or the XML transforms(web.environment.config) because we would like to keep the important values out of source control and in VSTS. We are also doing config transforms on NLog.config, and those transforms are setup the same way and seeing the same issues. Any help on getting this more general path to work, or a different way of solving this would be greatly appreciated.
Other potentially helpful information:
MSBuild Arguments in Build Solution step:
/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactstagingdirectory)\\"
Path Structure
$(My.Paths.InstallDirectory)\Content\d_C\a\1\s\Apps\ProjectName\ProjectName\obj\Release\Package\PackageTmp\Web.config
There is File Transforms & Variable Substituion Options in Azure App Service Deploy task, so you can use XML variable substitution or JSON variable substitution to do it without extracting the package file (zip file):
For example:
Add a new variable in Release definition (Scope: select an environment), for example: Name: DefaultConnection; Value: Data source=xxx…), clicking lock icon to change variable type to secret.
Check XML variable substitution option in Azure App Service Deploy task
Then during the deploying, any config and parameters.xml files will be updated with that variable value if the key or name entries in appSettings, applicationSettings and connectionStrings sections are matched the variable name
Related article: XML variable substitution
Another way is that you can change the value in web deploy parameters file (SetParameters.xml) through Token replace task (You also can specify the different file in Azure App Service Deploy task per to Environment of release, such as SetParameters.Dev.xml, SetParameters.QA.xml)
An article about Configuring Parameters for web package deployment
BTW: You can remove the source structure from web deploy package by specifying /P:PackageTempRootDir="" msbuild argument