We have many TFS builds in place. I have same set of steps used in many builds. I want to combine them into a single step. So that I can reuse it across the builds instead of repeating each step in each build. Because it is difficult when I have to make changes to the power-shell scripts which need to be changed everywhere.
If you are use TFS 2017, you can use Task Groups. Press ctrl key -> select the tasks you want to encapsulate -> right click -> create task group. After that, if you need these tasks in another build definition, you can select the task group instead.
If you use other older TFS versions, task groups is not available. But you can clone a build definition, then the new build definition will have the tasks and setting with the one you cloned.
Related
Our repositories has folders, the code in the folder are sometimes dependent on code in other folders, but only in one direction. For way of explanation:
C depends on B
B depends on A
We have 3 builds required on our Pull Request policy for master:
We have a build (BuildC) that builds ONLY folder C
We have a build (BuildB) that builds B and C
We have a build (BuildA) that builds A, B, and C
The policy specifies:
Changes in folder C require BuildC
Changes in Folder B require BuildB
Changes in Folder A require BuildA
Desired effect:
Depending on the case, I want the Pull Request to require ONLY ONE of the three builds. Here are the cases:
BuildA - Should run when there are changes in folder A (even if there are changes elsewhere)
BuildB - Should run when there are changes in B (and/or C) but NOT IN A. If there are changes in folder A, this build should NOT run
BuildC - Should run when the only changes are in folder C... if changes exist in folder A and/or B in addition to C... this build should not run.
What actually happens is that if you change something in folder A and C, two builds run: BuildA and BuildC... and if the changes in folder C depend on folder A, then BuildC build fails. In any case, the run of buildC is a waste.
Is there a way to have Azure DevOps queue only 1 build... but the best one. So in our example case, BuildA will run but not BuildC... but if the changes were only in Folder C, it would run Build C?
There is no way to accomplish what you want using build triggers or policies. There is no "Don't build when there are changes in folder X". There are a few options though, but they require a bit of rethinking:
Option 1: Use jobs & conditions
Create a single Pipeline with a build stage and 4 jobs.
The first job uses a commandline tool to detect which projects need to be rebuilt and sets an output variable
The other 3 jobs depend on the first job and have a condition set on them to only trigger when a variable (set in the first job) has a certain value.
That way you can take complete control over the build order of all 3 projects.
Option 2: Use an orchestration pipeline
There are extensions in the marketplace to trigger another build and wait for its result.
Perform a similar calculation as in option 1 and trigger the appropriate build, waiting for its result.
Option 3: Use Pipeline Artifacts
Instead of building A+B+C in build C, download the results from A+B, then build C. This will require uploading pipeline artefacts at the end of each job and for each subsequent job to do an incremental build by downloading these artifacts and thereby skipping the build process.
You could even download the "last successful" results in case you want to skip building the code.
Option 4: Use NuGet
Instead of pipeline artifacts, use nuget packages to publish the output from Build A and consume them in Build B. Or even, publish A in job A and consume it from job B in the same build definition.
Option 5: Rely on incremental builds
If you're running on a self-hosted agent, you can turn off the "Clean" option for your pipeline, in care the same agent has built your build before, if will simply re-use the build output of the previous run, in case none of the input files have changed (and you haven't made any incorrect msbuild customizations). It will essentially skip building A if msbuild can calculate it won't need to build A.
The advantage of a single build with multiple jobs is that you can specify the order of the jobs A, B, C and can control what happens in each job. The big disadvantage is that each job adds the overhead of fetching sources or downloading artifacts. You can optimize that a bit by clearly setting the wildcards for what pieces you want to publish and to restore.
If you don't need the sources in subsequent stages (and aren't using YAML pipelines), you can use my Don't Sync Sources task (even with Git) to skip the sync step, allowing you to take control over exactly what happens in each job.
Many of these options rely on you figuring out which projects contain changed files since the last successful build. You can use the git or tfvc commandline utilities to tell you which files were changed, but creating the perfect script may be a bit harder when you have build batching turned on, in which case multiple changes will trigger your build at once, so you can't just rely on the "latest changes". In that case you may need to ure the REST API to ask Azure DevOps al the commitIds or all changeset numbers associated with this build to do the proper diff to calculate which projects contain changes.
Long-term, relying on a single build with multiple jobs or nuget packages is likely going to be easier to maintain.
I am new to using Azure DevOps builds/pipelines, as the source code for the solutions I need to build are in TFVC I am limited to using the Classic (i.e. UI) builds rather than YAML.
When I want to test changes to a build definition I sometimes want to run a clean build, i.e. ensure that sources and artifacts from earlier builds are removed before the new build run, yet leave normal builds (i.e. ones triggered by changes in TFVC) to be incremental so to make builds faster.
I am used to TeamCity which has a plethora of options with regards to managing source and artifacts retention between builds, including a simple "clean" check box when triggering a manual build.
ADO Builds seems very limited in this regards and if I want to have a clean build it seems the only option is to change the build definition, select clean, run the build, then change the build definition again to remove the clean option.
Are there better ways to manage "ad-hoc" clean sources and artifacts in ADO Builds?
Those settings are either on/off. They wouldn't accept conditional run-time variables.
That being said, you might try leveraging the "Save as draft" option. It seems to create a DRAFT pipeline definition you could execute for your changes.
You could probably just flip it back to no clean before publishing. I don't really use that feature, but I am going to guess on the back-end it is using a different temporary definition id. That will probably mean on the build agent a new folder gets created under _work. If that is the case, you probably wouldn't even need to flip the clean sources since it will not exist the first run. It also probably means if this a self-hosted agent you will have doubled the work folder size and you might have to manage that clean-up after you are done.
If it does create the second work folder, this is probably preferable as it means you won't break the incremental build on the build directly following your test with clean. Whether you did that ad-hoc or through editing the build definition.
Build.Clean variable is deprecated, you can only use Clean option to clean local repo on the agent currently.
I'd like suggest you submit a user voice at website below, product team will evaluate it carefully:
https://developercommunity.visualstudio.com/content/idea/post.html?space=21
One workaround is adding a Post Build Cleanup task in the end of your pipeline, when you want to run builds incrementally, you can disable this task.
Task Groups are a great way to build a library of commonly executed Build and/or Deployment Actions in Visual Studio Online. Does anyone know of a way to make a copy of (or clone) an existing Task group? I don't see that as an option in the UI anywhere.
I think there is no way to do this yet. It's not even possible on TFS On-premise. Though our TFS professional told us, it should come with TFS2017 Update 1.
Only thing you can do, is select all tasks in your task group and create a new TaskGroup with them. Which is basically the same as cloning the task group itself - it only takes you some clicks more.
Or you could export your task group, change its name and Id in the exported Json, and import it.
Currently there isn't the copy Task Group feature in VSTS and TFS.
You can create TaskGroups according to your requirement and manage them from TaskGroups tab, then use them directly in different scenario. Please follow below steps.
Create a Task Group from Build/Release definition
Navigate to Build&Release > Task Groups
Select the Task Group you need to modify, change the Properties/Tasks
Save the Task Group
If you just want to copy the Task Groups, you can install the "Copy Build Steps" extension. it can copy build/release steps from one definition to another, also support copy the Task Group.
After installing the extension, navigate to the Builds hub > All definitions. A new menu item, ‘Copy build steps’ will be available when you click the ‘’…’’ behind the definition.
To Use a task group:
In a build/release definition, choose Add steps/tasks to open in the Task catalog. Select the category you specified for your task group and find the task group (it displays the name and description you specified when you created it), then choose Add - just as you would with any other task.
I am trying understand how to really use PowerShell in TFS. I would like to add a release step to create a work item task, to tell our release management team they require to approve a release.
Our company has many team projects and I want this to be available as a module that can be used by any project. The process I am using to use this is I am adding an artifact for the TFS project that we store our generic item.
It is under $\BuildSupport, for this example, I create a build for my project under $\WEB. I am also creating the release Under $\Web as well. I create my new release definition, I add my Build artifact.
I then add a Team foundation Version Control Artifact to $\BuildSupport Source alias is "BuildSupport".
I then add a environment step called TEST PRODUCTION and add PowerShell task where the script path is: $(System.DefaultWorkingDirectory)/BuildSupport/Main/BuildTools/TFSBuildScripts/CreateWorkItem.ps1
My issue is that when creating a new release it asks every time for me to choose the artifact version based the changeset for BuildSupport.
I don't want to have everyone do this every time is there a better way?
There isn’t the way to queue a new release without specify the version of TFVC artifacts. I submit a user voice here: Default version of artifacts when queue new release.
The workaround is that:
You can create a task group for each team projects, then add this task group to related release definitions. (you need to update the task group if there are updates)
Create or open a release definition
Add PowerShell task and configure it (e.g. Type: Inline Script, Arguments: [necessary arguments], Inline Script: [powershell script] etc…). You also can specify the PowerShell file in the Shared folder (Map BuildSupport to shared folder, then current version is the "default" version when queue release)
Right click this task > Create Task Group
After that you can add this Task Group to other release definitions directly.
Another way is that you can create a build/release task extension, then install it for necessary team project, after that you can add that custom build/release task to build/release definition. For this way, you just need to update the extension if there is the update.
I am working on creating custom vsts build task using Hosted agent and powershell script. I just want to add reference of existing task available out of the box in vsts (Publish Build Artifacts). Is there any way to reference this task in our custom task? Or I just have to implement the functionality provided by PublishBuildArtifacts manually?
The way to reference another task us to grab the sources from the Task Repository on GitHub and package them with your own task (in a subfolder). You'll need to copy the inputs from their task.json and merge them into yours if you want to allow other users to configure the fields exactly the same way as the other task is doing.
You can find the task implementations here: https://github.com/Microsoft/vsts-tasks make sure you select the right branch, the master branch is the bleeding edge, and it may contain a version of the tasks that is not fully battle tested or may not be compatible with the latest version of the agent that has been released (or the minimal agent version you are targeting).
Or you can grab the implementation from a build agent's tasks directory.
Remember that for certain functionality, the VSTS Task SDK has built-in methods to upload artefact, which may make your life easier if you decide to implement the functionality on your own.
The team that built the agent has been pretty specific about making sure that tasks are self-contained and need to package their own dependencies or flag a demand. This is to ensure that each task can evolve and change independently.