We are using Visual Studio Team Services.
We have a Prod-Branch which is builded by our Prod-Build-Definition and deployed by our Prod-Release-Definition to our Test / Integration and Production Environments.
With each Prod-Release deployed to the customer, we create a Prod-Rel-Version-x.x.x Branch from the Prod-Branch (in Case we need that for a Hotfix).
During the Sprint we are developping on a Dev-Branch which is builded by our Dev-Build-Definition and deployed by our Dev-Release-Definition to our DEV Environment for Developer Tests.
After the Sprint (or from time to time) the Dev-Branch is merged to the Main-Branch and then to the Prod-Branch. From there it is deployed to the different Stages for Testing by the customer.
When there is a Hotfix-Case, we fix the bug on the Prod-Rel-Version-x.x.x Branch and would like to reuse our existing Prod-Build-Definition to build this Hotfix-Version and deploy to the different stages by the existing Prod-Release-Definition for testing and going live with this version.
How can we reuse our Prod-Build-Definition with this different Branch (Prod-Rel-Version-x.x.x Branch instead of the Prod-Branch)?
When I look at the build definition, I think i would be possible, just be editing the Server Path (Repository > Mappings) from $/NameOfOurApp/Prod to $/NameOfOurApp/Prod-Rel-Version-x.x.x)...that should do the trick or not? But from what I read, it's not possible to use Build-Variables in Server Mappings, so I cannot change this variable for example in the Queue new Build Dialog...
What's the best way to accomplish my scenario?
The only way to do this is to create a single build definition which downloads all the branches. Then use variables in the tasks to select the version to build. This will become very messy (and slow) very fast.
Instead it's much easier to clone the build definition. Alternatively you can create a Build Definition Template from an existing build definition and use that to create a new Build Definition.
A much, much better solution however, is not to rely on so many branches.You only need the branch when you really need to make a hotfix, and you only need the stages branches when you have a lot of findings in higher stages. By improving teh way you work, you'll be able to get rid of the branches, simplifying the work for all.
Update
VSTS and TFS 2018 now support the use of variables in the workspace definition.
Related
I am new to using Azure DevOps builds/pipelines, as the source code for the solutions I need to build are in TFVC I am limited to using the Classic (i.e. UI) builds rather than YAML.
When I want to test changes to a build definition I sometimes want to run a clean build, i.e. ensure that sources and artifacts from earlier builds are removed before the new build run, yet leave normal builds (i.e. ones triggered by changes in TFVC) to be incremental so to make builds faster.
I am used to TeamCity which has a plethora of options with regards to managing source and artifacts retention between builds, including a simple "clean" check box when triggering a manual build.
ADO Builds seems very limited in this regards and if I want to have a clean build it seems the only option is to change the build definition, select clean, run the build, then change the build definition again to remove the clean option.
Are there better ways to manage "ad-hoc" clean sources and artifacts in ADO Builds?
Those settings are either on/off. They wouldn't accept conditional run-time variables.
That being said, you might try leveraging the "Save as draft" option. It seems to create a DRAFT pipeline definition you could execute for your changes.
You could probably just flip it back to no clean before publishing. I don't really use that feature, but I am going to guess on the back-end it is using a different temporary definition id. That will probably mean on the build agent a new folder gets created under _work. If that is the case, you probably wouldn't even need to flip the clean sources since it will not exist the first run. It also probably means if this a self-hosted agent you will have doubled the work folder size and you might have to manage that clean-up after you are done.
If it does create the second work folder, this is probably preferable as it means you won't break the incremental build on the build directly following your test with clean. Whether you did that ad-hoc or through editing the build definition.
Build.Clean variable is deprecated, you can only use Clean option to clean local repo on the agent currently.
I'd like suggest you submit a user voice at website below, product team will evaluate it carefully:
https://developercommunity.visualstudio.com/content/idea/post.html?space=21
One workaround is adding a Post Build Cleanup task in the end of your pipeline, when you want to run builds incrementally, you can disable this task.
First attempt at automated build and continuous deployment so any process suggestions / improvements are welcome.
I have a repository with different build definitions. One for each of the following: database project, api, and web. (Will add more later for etl / reports) Each build has a filter so it only builds if code in a specific path has been changed.
Currently I have separate releases using continuous deployment for each build. So when the code changes, it builds that auto deploys. This works, but really isn't practical because of dependencies.
What I am looking to do is have one release definition that includes all build artifacts. Then have deployment phases that only run conditionally if a specific build artifact was created (something in that project changed). This way all builds / releases don't run every time, but are tied together when there are related changes.
I am going down the path of trying to created a custom condition on the deployment phase, but can't seem to figure out a way to make this work. I appreciate any help with this.
I have a repository with different build definitions. One for each of
the following: database project, api, and web. (Will add more later
for etl / reports) Each build has a filter so it only builds if code
in a specific path has been changed
Path filters are not to be used in your situation.
If you see Microsoft's git repo,
They have all their codebase from the Windows and Devices Group (WDG) in one big repo. Each root folder is a separate product and completely unrelated to the rest. (eg. Xbox, HoloLens, Windows OS, etc).
Path filters makes sense here because if I git push code to Xbox, I don't want Hololens code also to be built.
Web / DB / API projects all need to be built together, packaged together and deployed together.
I am assuming the project uses .NET stack.
Keep the DB, Web and API projects are in the same solution. Create a single build definition that builds the solution and create multiple artifacts(dacpac, webdeploy package etc.) by adding multiple publish artifacts step.
See screenshot of a build with multiple artifacts.
Link the artifacts from this build to the Release Definition and you should be able to deploy.
I'm using Visual Studio Team Services to build my project which is stored in GitHub (here). The master branch contains multiple projects which make up the solution. Amongst those are a WebAPI project and a Cordova project. I need to build those using two separate build definitions in VSTS.
Previously I had set-up my build definition and used the branch filters to filter on what had been pushed to the repo. For instance:
master/src/API
This worked, but it doesn't any more. It seems as if the underlying code has changed. A filter of 'master' still works and I understand how this feature is probably meant to filter specifically on branches and maybe not on folders within the branch?
It's not a huge problem, but at this time all of my builds will trigger with every check-in, even if nothing changed in the meantime for that source code. So I'm not wondering what a good solution for this issue would be:
Put every project in it's own branch. Seems like a workaround
Some other filter option or maybe another syntax or something?
Leave it as it and don't worry about the extra builds (but that itches, you know...)
Anyone running a similar set-up?
Path filters is not supported for VSTS GitHub CI Build, it is available for Git CI Build on VSTS. You can vote this user voice: https://visualstudio.uservoice.com/forums/330519-team-services/suggestions/15140571-enable-continuous-integration-path-filters-for-git
The workaround is as you said that put every project in its own branch.
I have been researching and am trying to figure out the best branching and deployment strategy to accomplish the requirements below. Maybe I’m missing something but it is more complicated than it seems. Ideally, we’d just have one permanent branch, ‘master’, that could have specific commits tagged to mark releases to production.
Our current strategy is based on Git Flow and has permanent branches ‘master’ (only has releases to production) and ‘develop’. The primary thing that complicates using a multiple permanent-branches model is the concept of “promoting” the same build from the staging environment to production. Currently, this needs to be done in a separate source code branch (deployments to staging come from ‘develop’, deployments to prod come from ‘master’).
Tools: Git (VSTS), TeamCity, Octopus Deploy
Requirements (feature and hotfix lifecycles):
All code is reviewed via pull requests (enforced via branch policies)
All code gets deployed to a staging environment for testing
We can quickly go back to any snapshot of code that was deployed previously
If testing is successful, then the same build can be “promoted” from our staging environment to production (no need to build again)
Features accumulate over time before pushing out to production as a single release. Hotfixes have to be able to go through without getting caught up in the "all or nothing" next regular release.
I like the idea of having one permanent branch with tags (re: The master/develop split is redundant, http://endoflineblog.com/gitflow-considered-harmful), but having additional permanent branches may better facilitate deploying to different lifecycles/versions (feature and hotfix) to Octopus.
I have been wrestling with how best to pull this off and I may be over complicating things. Any feedback is appreciated.
It seems you have a number of questions and they are quite broad... I'll add some comments to each of your requirements as a conversation starter, but this whole thread might get blocked by moderators as it is definitely not the style of questions SO was made for.
All code is reviewed via pull requests (enforced via branch policies)
I haven't looked at VSTS for ages, but I'd expect they already support branch policies and pull-requests, so not sure if there's anything you need here other than configure settings in your repositories.
In case VSTS does not support that, you might consider moving to a tool that does e.g. BitBucket, GitHub, etc. Both of these have an on-premises version in case you can't (or don't want to) use the cloud hosted version.
All code gets deployed to a staging environment for testing
You achieve that with setting up lifecycles in Octopus Deploy, to make sure deployments/promotions follow the the sequence you want.
We can quickly go back to any snapshot of code that was deployed previously
You already have source control, so all you need now is traceability from the code that is deployed in an environment, to the deployment version in Octopus Deploy, the build job in TeamCity, the branch and exact commit in your source control.
There's a few things that you can do, to achieve that:
Define a versioning scheme that works for you. I like to use semantic versioning. "Major" and "Minor" versions are defined by the developers, and the "Patch" is the auto-incremented number from TeamCity (%build.number%). Every git push build the code and generates a unique build version (%major%.%minor%.%build.number%)
As part of the build steps in TeamCity, before you compile the code, make sure your source files are patched with the version number assigned by each build, the commit hash from your source control, and the branch name. e.g. if you are using .NET, make sure all the AssemblyInfo.cs files are updated with that version, so that the version is embedded in the binaries. This allows anyone to query the version looking at the properties of the binary files, and also allows you to display the app version on the app itself (e.g. status bar, footer, caption, about box, etc.)
Have TeamCity tag your source control with the version number of every build, so you can quickly see on your source control history. You probably only want to do that for the master branch, though which is what you care about.
Have Octopus tag your source control with the deployment version number and the environment name, so that you can quickly see (from your source control) what got deployed where.
Steps 1 and 2 are the most important ones, really. 3 and 4 are just nice-to-have. Most of the time you'll just open the app in the environment, check the commit hash in the "About", and do a git checkout to that commit hash...
If testing is successful, then the same build can be "promoted" from our staging environment to production (no need to build again)
Again, Octopus Deploy lifecycles, and make sure anything different in each environment is defined in the configuration file of the application, which is updated during the Octopus deployment, using environment-specific variables.
In terms of branch workflow, this last requirement makes it mandatory to merge changes into master (or whatever your "production" branch is) before the deployment lifecycle can begin.
I have a NSIS installer that we previously built using nAnt scripts that copy some files around and run makensis.exe via a exec task to build the installer exe. After the nant script completes, I have the compelte structure for our CD and also our download.
I was just doing a get from sourcesafe onto an unused desktop and using it as a build box, compiling there. Sometimes we would have a couple of files checked in that fix something critical. In those cases I would go to the build box, and very selectively get only those files, to avoid getting other changed files that we aren't ready to release yet. Basically I am able to allow development to continue and selectively include certain changed files into the installer for release.
Now we no longer have a free box, and need to build from our server. So I am setting up CI Factory so that the developer can kick the build off without remoting into the server. The one issue I am struggling with, is the best way to continue to allow this selective change control to occur. The default concept of CI that CI Factory implements is fine for internal development "head". However, I also want to setup a CCNet project that is run only on demand via a Force Build for this "public release" type of build.
This is what I've brain stormed so far, without being sure how well this will work, if at all(still figuring out what CCNet and CI Factory are all about). The "public release" CCNet project config/build would be setup such that it would not get latest. Modifications would not trigger a build. Since the other CCNet project that is using the default CI methodology(we'll call it the "CI project") of getting latest when changes are detected, then these two projects can't share the same working directory. So the "public release" would need a different working copy, so that its files won't get updated when the CI project's build is triggered. The developer would need to remote into the server, one VSS, selectively do a get into the "public release"'s working copy, and then force a build through CI Factory.
The disadvantage's I see with this is
1) Having to remote in to selectively do gets.
2) I have no idea how to allow a single CI Factory project to have two different working copies of the Product folder, so that each project configuration block has it's own.
3) I'm afraid of what kind of strangeness this might cause. I'm not quite sure yet how to specify a source control block in CCNet project config block, but prevent it from doing a get latest when it builds. I'm still gradually figuring out what things are in scripts and can be easily taken out without breaking other things, versus what is not meant to be mucked around with and/or is not configurable.
I would really like to hear about how others deal with this issue of selectively releasing changes, if you have a similar situation. I am constrained to VSS, so my immediate need is to solve this with that in mind, but at the same time I'd be interested in hearing how you manage this with other source control systems. I guess you would probably have a branch that is your latest developments branch, and then merge changes into the trunk whenever you want to release them? I really don't trust VSS for branching/merging, and I think the branching concepts might be a little too much overhead and learning curve for this shop. Like I said though, stories with other source control systems would be useful future knowledge for me.
Thanks in advance.
You need a branching structure in your repository to facilitate this. Something like the release branch method. Only select individuals can commit to this branch (or have a release/stable for that). Set up your manual CI launches to pull from the release branch as release nightly promote to milestone or final from there. I don't like the idea of manually modifying things on your build machine. Set up the changes in version control, in a safe place to prepare your release and let CI build from there, but manually triggered.
Check out these branching patterns. I suggested C3, codeline-per-release, often called release branching.
Heres an article on VSS branching that includes a link to merging.
This question looks similar.
Maybe you could move to another source control system with better support for this kind of thing. Any suggestions from MS people out there?