My build:
Build with msbuild (/t: build)
Publish with msbuild (/t: publish)
Package with nuget
Deploy with octopus
produces an artifact (nuget package) that contains the following files:
Azure.ccproj.cspkg
ServiceConfiguration.Production.csfg
Web.config
(which will be transformed during deployment via Octopus)
The cspkg is a valid, deployable package. The problem is that it contains a web.config that is pre-transform because the transform occurs at deployment time. See the Octopus documentation for transforms and variable substitution for reference.
How do I overwrite the web.config inside the cspkg with the transformed web.config that resides in the deployment package?
I have powershell and the full .net framework at my disposal.
Alternatively, if it makes more sense to unpack the cspkg, overwrite the file and then re-package, I consider that acceptable. I am not sure how to do that either.
I know that Save-AzureServiceProjectPackage exists but I cannot get it to run and the documentation is lacking.
I have an Octopus project with 2 steps: first for Dev hosted in IIS, second for Prod hosted in Azure. TeamCity procuded 2 nuget packages: one by OctoPack for Dev, another by NuGet Pack for Prod with cspkg.
I have this target in my Azure.ccproj:
<UsingTask TaskName="TransformXml" AssemblyFile="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v12.0\Web\Microsoft.Web.Publishing.Tasks.dll" />
<Target Name="Transform" BeforeTargets="BeforeBuild" Condition="'$(TEAMCITY_VERSION)' != ''">
<PropertyGroup>
<SourceTransformFile>..\Api\Web.config</SourceTransformFile>
<TransformFile>..\Api\Web.Prod.config</TransformFile>
<DestinationTransformFile>..\Api\Web.config</DestinationTransformFile>
</PropertyGroup>
<TransformXml
Source="$(SourceTransformFile)"
Transform="$(TransformFile)"
Destination="$(DestinationTransformFile)" />
</Target>
The condition allows to run in only on the build server and not locally.
I have solved this by sending the solution as it is to Octopus Deploy which allows me to run .config transformations in Octopus. After running the transformations I create the .cspkg package with custom powershell script.
I have written a thorough post of the Octopus deploying part. You can find it here: http://henrihietala.github.io/
I have figured out how to do what I need based upon the information contained in Brad Webber's post in the Octopus Support forum.
I have posted a public git repo containing a simple sample solution and documentation here.
Related
I need to create nuget package for a .net core 2.1 as part of build and release process in Azure Devops
What I would like to do
1) In the build section build the project and then add the compiled code to artifact
2) In Release definition will have 2 deployments one for Beta release where the version will be like 1.2.3-Beat.2 and push to azure artifact nuget and another deployment to release where the version will be like 1.2.3.2 and push to azure artifact nuget.
Currently I have only one build definition which will build (nuget package gets created during build process) and push to azure artifact nuget.
Pipeline I would like to create
Use the dotnet pack task with the --no-build option, and in your pre-release stage set the VersionSuffix value.
Note: My current team uses a set of Powershell scripts to append build number data to the Major.Minor data found in the .csproj (or AssemblyInfo.cs for netFramework), but that doesn't change the answer to your question. Once you figure out what the Major.Minor.Patch[.Build] data is going to be, you can use the VersionSuffix property in a dotnet pack task with --no-build to communicate quality of the package as it moves through your pipeline.
Given a .csproj file that looks like this:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
</PropertyGroup>
<PropertyGroup>
<PackageVersion>1.0.0.1</PackageVersion>
<AssemblyVersion>1.0.1812.201</AssemblyVersion>
<FileVersion>1.0.1812.201</FileVersion>
</PropertyGroup>
<ItemGroup>
<Content Include="Assemblies\*">
<Pack>true</Pack>
<PackagePath>lib\$(TargetFramework)</PackagePath>
</Content>
</ItemGroup>
</Project>
Again, if we ignore the versioning step that we use, the dotnet pack task in the pipeline image above will produce a package with the version 1.0.0.1-Beta.
Then in your stable release stage, don't set the suffix value and let the package get its version from the .csproj file like normal (eg. 1.0.0.1).
The elements and values in the sample .csproj file above can be written as direct edits to the .csproj or they can be set using the Properties >> Package tab.
If the values are not changed in the properties menu or added explicitly, the elements and values do not appear in the .csproj and are assumed by dotnet build|test|pack commands.
Finding the right combination of these properties and values can be daunting if you aren't familiar with how they're fitted together. I found this article useful when trying to decipher the version properties.
Also, you should understand 1.0.1-b2 < 1.0.1, so your pre-release version might be 1.2.3.2-beta1 and your stable version would be 1.2.3.2.
As the package version is included the moment you perform 'nuget pack', which you would generally perform during build, it might become a bit complicated to change that version afterwards.
What may be interesting in your case is using the concept of views in Azure Artifacts, this would allow you to promote a package to the release view at a later state without having to rebuild the package.
There is a nice extension on the marketplace that would allow you to do this from within a release: https://marketplace.visualstudio.com/items?itemName=rvo.vsts-promotepackage-task
using this flow you can have the packages in the pre-release view/feed as long as you like and make them available in the release view whenever you see fit.
downside of this is that the packages are not identified as pre-release packages by Nuget as you would not be packaging them with the proper semver for that
Im setting up a new build and deploy pipeline in Azure Devops. It is an older Web application with some transformation files for the web.config. In the old days we would build the same code x times depending on how many environments. This this is no longer necesary as I read from here https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/transforms-variable-substitution?view=vsts#xmltransform
it looks like the deploy pipeline can pick up the changes from my transform file.
But the problem is that my other transform files does not get included in the package so I get these warning message:
[warning]Unable to apply transformation for the given package. Verify the following.
[warning]1. Whether the Transformation is already applied for the MSBuild generated package during build. If yes, remove the <DependentUpon> tag for each config in the csproj file and rebuild.
[warning]2. Ensure that the config file and transformation files are present in the same folder inside the package.
And yes when i download the artifact the Web.[stage].config files are not there as suggested.
Is there some setting somewhere that let me include these files? Or stop them from being transformed?
For Web applications
MSBuild knows how to transform the web.config files based on the following settings/properties/parameters (in order)
Build Configuration dotnet publish --configuration Release
Publish profile dotnet publish --configuration Release /p:PublishProfile=FolderProfile
Environment dotnet publish --configuration Release /p:EnvironmentName=Production
Custom file transform dotnet publish --configuration Release /p:CustomTransformFileName=custom.transform
I think it's typical for developers to make this happen based on build configuration only, and I believe MSBuild (and dotnet) know how to do this based on the <DependentUpon>Web.config</DependentUpon> element in the Web.[configuration].config item in the project or build script file.
Azure DevOps Release Pipelines is a little different.
The pipeline wants to transform your web.config after the project has been built/published and doesn't know how to do that if MSBuild (or dotnet) has already made an attempt at it. Thus:
[warning]Unable to apply transformation for the given package. Verify the following.
[warning]1. Whether the Transformation is already applied for the MSBuild generated package during build. If yes, remove the tag for each config in the csproj file and rebuild.
[warning]2. Ensure that the config file and transformation files are present in the same folder inside the package.
The warning text states:
Remove the <DependentUpon> tag for each config in the csproj
Thus: you need to remove the tag from the csproj to prevent MSBuild from transforming the files
Or: you need to use the /p:TransformWebConfigEnabled=False argument to MSBuild. (note: I believe it is correct that this can be used w/o removing the dependent upon tag, but I could be wrong)
Make sure the transform source and target files are in the same folder inside the package.
There may be several ways to do this. I've chosen to mark the transform source config files as content to force MSBuild to include them in the published package.
Now you need to organize your release pipeline in accordance with the File Transforms and Value Substitutions documentation.
[section]Starting: IIS Web App Deploy
====================================
Task : IIS Web App Deploy
Description : Deploy a website or web application using Web Deploy
Version : 0.0.51
Author : Microsoft Corporation
Help : More information
====================================
...[command]C:...\ctt\ctt.exe s:C:...\Web.config t:C:...\Web.Release.config d:C:...\Web.config pw i
[command]C:...\ctt\ctt.exe s:C:...\Web.config t:C:...\Web.Development.config d:C:...\Web.config pw i
XML Transformations applied successfully
...
For Non-Web Applications Needing .config Transformation
Getting your .config files to the release pipeline can happen several ways. Here are two.
Your release should have "access" to the repository as a part of the artifact, which will ensure that the deploy agent downloads the source (not desirable IMHO).
You will need to include the web.[stage].config files as part of your build artifact with a copy task, or a minimatch that picks them up.
Once you have the .config files available to the release pipeline
You can use the File Transform Task or XDT Transform Task to perform the transformation operations.
Option 2 is the route I've gone.
Here is an image of what that task looks like for me.
That task puts the config in the artifact from the build that I can then use in the release pipeline without rebuilding xx times.
Cleanup
If you're in a position where you care to not have the transform files persisting on the agent after the release is complete, then you'll need to add that to your pipeline.
I was using publish to folder option through Visual Studio by right-clicking on the project -> publish -> publish to folder. Result was always ready-to-copy project with applied transformations.
I wanted to automate this process using VSTS and have setup build on VSTS.
I used next steps:
- NuGet restore
- Build solution
- Publish Build Artifacts to $(build.artifactstagingdirectory)
- Windows machine file copy from $(build.artifactstagingdirectory) to remote machine using admin login and password
And finally I'm getting zip package on remote machine with complicated folder structure without applied transformations inside at all.
What is wrong? How I can setup same "publish to folder" as in Visual Studio but using VSTS?
Add below Target to your .csproj to enable transforming config files
<Target Name="TransformConfigFiles" AfterTargets="AfterBuild" Condition="'$(TransformConfigFiles)'=='true'">
<ItemGroup>
<DeleteAfterBuild Include="$(WebProjectOutputDir)\Web.*.config" />
</ItemGroup>
<TransformXml Source="Web.config" Transform="$(ProjectConfigTransformFileName)" Destination="$(WebProjectOutputDir)\Web.config" />
<Delete Files="#(DeleteAfterBuild)" /></Target>
In your build solution step add the following build arguments "/p:TransformConfigFiles=true" will make the config transformation using the above added target to .csproj
/p:TransformConfigFiles=true /p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:OutDir="$(build.stagingDirectory)"
Then you can use a publish step to publish your $(build.stagingDirectory) contents. You can use $(build.stagingDirectory)_PublishedWebsites as path to publish if you only need the website output.
This will allow you to get the ms deploy package as well as xcopy deploy published website files.
You can use copy files task before the publish task to copy any additional files if you have any to $(build.stagingDirectory) and get them published as build artifacts.
Use VSTS release management with deployment groups to deploy your application to target server. You can use IIS deploy task to deploy to IIS using ms deploy package. If you are using web deploy package you can use a parameters.xml in your web app to get the web config parameters assigned to .setparameters.xml so that you can change values in the deployment time using IIS deployment task.
You are publishing web application through File System method, it is based on the specified configuration (e.g. Debug, Release) to transform web.config. So you need to check which configuration you specified in build solution task (e.g. Visual Studio Build task)
Simple tasks:
NuGet Tool Installer task
NuGet restore task
Visual Studio Build task (MSBuild Arguments: /p:SkipInvalidConfigurations=true /p:DeployOnBuild=true /p:WebPublishMethod=FileSystem /p:publishUrl="$(build.artifactstagingdirectory)\\" /p:DeployDefaultTarget=WebPublish; Platform: $(BuildPlatform); Configuration: $(BuildConfiguration)) Note: BuildPlatform and BuildConfiguration are build variables. It will publish web app to artifacts directory ([agent working folder]/1/a)
Publish Build Artifacts (Path to publish: $(build.artifactstagingdirectory))
We are in process of creating architecture for VSTS CI/CD to deploy our web app to our Azure App Services.
We want to exclude the web.config while deploying it to the Azure server as we are directly modifying the web.config on the different environment.
CI Tasks looks like this:
CI Taks
CD Task:
Deploy Azure App Service
I am aware of other ways of updating the web.config https://learn.microsoft.com/en-us/vsts/build-release/tasks/transforms-variable-substitution, but in our case we want to skip the web.config file.
I couldn’t find the option to skip file in during release in VSTS as mentioned in this thread
How do I exclude the .cs files within an artifact from a vs-team-services CI build?
Is there a way to exclude certain files while building and deploying the release?
Added -skip:objectName=filePath,absolutePath=web\.config in additional arguments. This skips updating the web.config file during deployment.
You can exclude the web.config before publishing artifacts in your build definition: copy the web packages files to a directory (such as $(build.binariesdirectory)), then copy the files exclude web.config to another folder (such as $(Build.ArtifactStagingDirectory)/package), and zip the files under $(Build.ArtifactStagingDirectory)/package. And finally publish the zip file as build artifacts.
Details changes in the build definition as below:
Change the MSbuild arguments as /p:OutDir="$(build.binariesdirectory)\\" in Visual Studio Build task.
Add a Copy Files task after Visual Studio Build task. Settings for this task as below:
Add Archive Files task after Copy Files task. And settings as below:
Change the Publish Artifacts task as below:
Now the build artifacts are exclude web.config file.
Additional arguments
-skip:objectName=filePath,absolutePath=\\Configuration\\AppSettings\\Base.config
you can add
-skip:objectName=filePath,absolutePath='.*\PackageTmp\Web.config$'
in Additional Arguments in "Deploy IIS WebSite/App" deployment VSTS task, this will not deploy your root web.config file.
I am trying to learn how to use Go CD to implement continuous integration into my company's workflow. I'm using GitHub for my source control and have my pipeline set up to monitor the repository for changes and attempt to rebuild whenever there's a change. All that is working fine. Now I'm trying to add unit tests to the pipeline and can't get the unit test project to build.
This is the config xml for my build command:
<exec command="%Windir%\Microsoft.NET\Framework64\v4.0.30319\MsBuild.exe" workingdir="UnitTest">
<arg>/T:rebuild</arg>
<arg>/P:Configuration=Release</arg>
<arg>/m</arg>
<runif status="passed" />
</exec>
This is the relevant part of the error that I'm getting:
The missing file is ..\packages\MSTest.TestAdapter.1.1.18\build\net45\MSTest.TestAdapter.props
The unit test project uses some NuGet packages, so it needs these files to build properly, but they are not in the working directory for GoCD. I added a job to my pipeline before trying to build the unit tests that restores the NuGet packages to the location specified in the error message. Here is the config xml for my restore nugget command:
<exec command="C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\ReadyRoll\Octopack\build\NuGet.exe" workingdir="UnitTest">
<arg>restore</arg>
<arg>packages.config</arg>
<arg>-PackagesDirectory</arg>
<arg>..\packages</arg>
<runif status="passed" />
</exec>
When I re-ran the pipeline and monitored the working directory, I see the NuGet files get created when the "restore" step is run, however building the unit test project deletes the files and I still get the error.
I thought this was because I was using /T:rebuild in my MSBuild command, but it still deleted the files when I changed it to /T:build.
Can anyone give me a tip on how add a job to my pipeline that builds a project using NuGet packages? I thought about adding the "NuGet.exe restore" command to the project file, but I couldn't figure out how to do it (or if it can be done?).
Thanks in advance!
I figured out what I was doing wrong. I set up the build and restore commands in separate jobs, I needed to make them both tasks in the same job. I did that, and it worked.
I think this has something to do with how GoCD checks out the material for each job? But since I'm so new to this (and may not be right about the reasons), I'm not going to speculate about why this changed caused it to work.
If anyone knows the reason, I'd be happy to know.