Anybody know if it is possible to pass in a repo name / base the build on a dynamic repo name? This would allow us to share the same build definition across different branches, cutting down on definitions when creating a feature branch, etc.
When using a TFVC repo we would store the different releases in the same repo but different paths. We could reuse the same build definition across different releases/FB's by altering the source path such as $/product/$(release)/......
It appears Git likes to have the repo hard-coded into the build (hence the dropdown - no way to plug in a variable.
While the question is targeted to On-prem Azure DevOps, if it is possible in the hosted environment it would be helpful to know.
I recommend using YAML build templates. By default these check out "self" and are stored in the repo. That way they work on forks, branches etc. Each branch can contain tweaks to the build process as well.
With the 'old' UI based builds this isn't possible.
What you are looking for is actually two things:
templates - this allows you reuse definition accross different pipelines
triggers - this allows you to trigger pipeline when commit happens on different branches
Looks like Task Groups solved the need (mostly). I was hoping to have one build definition that could be shared across multiple branches; while this appears to be possible on the hosted model, on prem is different.
I am able to clone a build (or use templates) to have an entry point into the repo/branch to get the sources, then pass off the work to a common task group. If I need to modify the build process for multiple branches, just modify the task group.
Related
I am trying to find a solution for a rather complicated Azure Devops build pipeline.
I have 3 repositories (lets call them RepoA, RepoB and RepoC) which each contain templates for jobs that need to run for builds within those repositories.
I also have a Build repository which contains templates for tasks that are used by the jobs in all 3 of the main repositories.
I have run into a major problem when trying to build a unified pipeline that can run jobs across the different repositories however. While I can implement a pipeline with a repository resource pointing to RepoA, RepoB and RepoC, when I add a template job from any of those repos using the templateJob.yaml#RepoA syntax, I receive an error due to the commonTaskTemplate.yaml#build reference within the job template in RepoA.
Templates in other repositories may not specify a repository source
when referencing another template.
Is there any clean work around for this issue? I could implement the task templates as custom tasks in an Azure Devops Extension, but that will make it much harder to make any changes to those shared tasks. I could also re-implement the shared tasks in every branch of every repo, but that makes a maintenance nightmare as the tasks would be near impossible to update.
It seems like every alternative I can come up with results in it being quite difficult to update the common tasks and it seems like there should be some way to do this more easily since I can't imagine that needing common tasks across repositories and a common pipeline that builds across repositories can be that uncommon of a thing.
Any other ideas on how to work around this limitation?
I have a TFVC project with about 4 branches. I need to somehow setup build and release pipelines that build an artifact for each branch. Because I will eventually need to repeat this process elsewhere, I would like to prevent having to duplicate the same build pipelines for each branch. I am able to configure a single build pipeline that works on whatever branch I need using a user-defined variable when the pipeline is kicked off, but now I need to enable continuous integration on the build.
My current build pipeline trigger configuration
I need this to work such that whenever someone checks into one of the TFVC branches, the build is kicked off and can correctly identify which branch was updated. From what I have found, this means that my initial idea of a user-defined variable is not going to work any longer. Is there a predefined pipeline variable that I can use to tell which branch was checked into, so that that branch is the one that is checked out and built? If not, is there some other way to do this in one pipeline, or do I ultimately need to duplicate this build pipeline for each branch?
Sorry, it's not available with TFVC in Azure DevOps/TFS build pipeline.
For CI trigger, you could select the version control paths you want to include and exclude. In most cases, you should make sure that these filters are consistent with your TFVC mappings on the Repository tab. It's not able to dynamically set workspace mapping path based on the branch which continuous integration trigger your build pipeline.
You could also take a look at Daniel's explanation in this question: When my TFS build is triggered by a branch-specific check-in, why doesn't it set that branch as its source?
TFVC relies on workspace mappings to know what to download. The workspace mappings can encompass multiple TFVC repos across different
team projects, multiple branches within a single repository......
As a result, there's no way for it to understand how to dynamically change workspace mappings to be for a specific branch.
Conclusion: You may need one build for every branch, duplicate the pipeline simply change the path filters in trigger and workspace mappings.
I want to create automated deployment pipeline for azure datafactory.
For one stream of development we can configure it using doc
https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment
But when it comes to deploying to two diff test datafactories for parrallel features development (in two different branches), it is not working because the adb_publish which gets generated is only specific to the one datafactory.
Currently we are doing deployement using powershell scripts and passing object list which needs to be deployed.
Our repo is in Azure devops.
I tried
linking the repo to multiple df but then it is causing issue, perhaps when finding deltas to publish.
Creating forks of repo instead of branches so that adb_publish can be seperate for the every datafactory - but this approach will not work when there is a conflict, which needs manual merge, so the testing will be required again instead of moving to prod.
Adf_publish get generated whenever you publish. Publishing takes whatever you have in your repo and updates data factory with it.
To develop multiple features in parallel, you need to just use "Save". Save will commit your changes to the branch you are actually working on. Other branches will do the same. Whenever you want to publish, you need to first make a pull request from your branch to master, then publish. Any merge conflict should be solved when merging everything in the master branch. Then just publish and there shouldn't be any conflicts, and adf_publish will get generated after that.
Hope this helped!
Since a GitHub repository can be associated with only one data factory. And you are only allowed to publish to the Data Factory service from your collaboration branch. Check this
It seems there is not a direct and easy way to accomplish this. If forking repo as workaround, you may have to solve the conflicts before merging as #Martin suggested.
I have created a project 'Company 1' and I want to create subproject inside that project like 'Deploy VM', 'Deploy App' etc. so I can put all 'Company 1' related code inside one project. And I can create multiple builds inside the same project.
But I am not seeing the option to create a sub-project. How can I achieve this in Azure DevOps? Do I have to create multiple repositories in the same project or there is another way to do this?
Thanks.
The term Project in Azure DevOps is a bit misleading. It would make more sense if it were called "workspace" or something like that. Each project can contain a very large number of repositories, build/release piplelines, team spaces, area paths, etc.
In fact, my entire organization lives primarily in one Project space with hundreds of repos and build definitions + dozens of separate team spaces.
Creating separate project will fully segregate concerns, so it's not a good idea unless you want true, full separation with no relationships to the others.
In your scenario, I recommend creating separate repositories and build definitions in the same Project space.
There is no sub-project in Azure Devops. Why don't you create a branch for your repo? You can pull/push codes for the new branch.
Git branches aren't much more than a small reference that keeps an exact history of commits, so they are very cheap to create. Committing changes to a branch will not affect other branches, and you can share branches with others without having to merge the changes into the main project. Create new branches to isolate changes for a feature or a bug fix from your master branch and other work. You can add some new features in one branch.
You can also build target or multiple branches using Azure Pipelines or TFS. Please see this document.
We are using Visual Studio Team Services.
We have a Prod-Branch which is builded by our Prod-Build-Definition and deployed by our Prod-Release-Definition to our Test / Integration and Production Environments.
With each Prod-Release deployed to the customer, we create a Prod-Rel-Version-x.x.x Branch from the Prod-Branch (in Case we need that for a Hotfix).
During the Sprint we are developping on a Dev-Branch which is builded by our Dev-Build-Definition and deployed by our Dev-Release-Definition to our DEV Environment for Developer Tests.
After the Sprint (or from time to time) the Dev-Branch is merged to the Main-Branch and then to the Prod-Branch. From there it is deployed to the different Stages for Testing by the customer.
When there is a Hotfix-Case, we fix the bug on the Prod-Rel-Version-x.x.x Branch and would like to reuse our existing Prod-Build-Definition to build this Hotfix-Version and deploy to the different stages by the existing Prod-Release-Definition for testing and going live with this version.
How can we reuse our Prod-Build-Definition with this different Branch (Prod-Rel-Version-x.x.x Branch instead of the Prod-Branch)?
When I look at the build definition, I think i would be possible, just be editing the Server Path (Repository > Mappings) from $/NameOfOurApp/Prod to $/NameOfOurApp/Prod-Rel-Version-x.x.x)...that should do the trick or not? But from what I read, it's not possible to use Build-Variables in Server Mappings, so I cannot change this variable for example in the Queue new Build Dialog...
What's the best way to accomplish my scenario?
The only way to do this is to create a single build definition which downloads all the branches. Then use variables in the tasks to select the version to build. This will become very messy (and slow) very fast.
Instead it's much easier to clone the build definition. Alternatively you can create a Build Definition Template from an existing build definition and use that to create a new Build Definition.
A much, much better solution however, is not to rely on so many branches.You only need the branch when you really need to make a hotfix, and you only need the stages branches when you have a lot of findings in higher stages. By improving teh way you work, you'll be able to get rid of the branches, simplifying the work for all.
Update
VSTS and TFS 2018 now support the use of variables in the workspace definition.