I'm new to the yaml build config game and as such I do a lot of trial and error. So I want to be able to test run my pipeline and see if it is working as I intended. But I don't want to have to check in the yaml file into my master branch for every incremental change to be able to test run it.
Is it possible to run a pipeline from a separate branch or without checking in code all together?
Thanks!
Agree with Shayki, you can create a test branch based on the master branch. Then create a new pipeline. Switch to the test branch in yaml.
The changes in this yaml will be committed to the test branch.
Related
I have a ADO git repo with a YAML-based build pipeline that builds a docker image, runs some tests using Docker Compose, and then pushes the image and a helm chart to an ACR.
I'd like the have the build/test part run on PRs too. I have created a second pipeline that's just the first half of the normal build, and assigned it as a Build Validation pipeline for a branch.
However, I don't seem to be able to get the triggers right. If I have trigger: none in the test pipeline, it never triggers. If I have branch names, it is also run on merge alongside the normal build pipeline. How is this supposed to work? The docs define all the individual parts, but not really how they are expected to interact!
Am I supposed to have a multistage pipeline and use that somehow for the validation? (it's just 4 Steps in one Job in one Stage currently).
I am hoping to avoid building the same image too many times, or storing untested images anywhere outside the build agent.
I make it work with the following configuration:
In my YAML pipeline, I set the trigger: none.
In branch policies of a branch, I create a build validation with automatic trigger:
Then I create a pull request to that branch, and the pipeline runs automatically:
There are two possible mistakes:
The "Manual" trigger is selected in build validation, so that the pipeline needs to be run manually rather than triggered automatically.
The branch with branch policy set should be the same branch as the target branch of pull request.
What I'm thinking about is to have a step in the pipeline to generate a full-blown pipeline to run afterwards.
Apparently this particular thing is not there yet (feature requests here, here). But maybe somebody has some fresh thoughts on workarounds?
Not really. It's a pain that's just a fact of life when working with YAML pipelines. It's especially annoying when trying to work through runtime vs compile time variable resolution issues.
Commit, run, commit, run, commit, run, over and over.
For the dynamic work you could set up a repo with one dummy yaml file and a pipeline registration that targets this yaml file.
From a static pipeline that is responsible for kicking off a dynamic pipeline you then execute two steps:
Create a fresh branch of that "dynamic" yaml file and commit the required dynamic workload
Not sure about branch limits though. You could also decide to reuse a branch.
Kick off this "dynamic" pipeline through az devops cli using the static pipeline's access token
Also see the following documentation:
https://learn.microsoft.com/en-us/azure/devops/cli/azure-devops-cli-in-yaml?view=azure-devops
We are following gitflow model in our project using Azure DevOps Services. I have a classic editor based pipeline which builds the Dev and Release/R1.0 branch.
I am going to setup a classic editor based pipeline which will build my Release R.10 from the master branch after merging Release/R1.0 branch at the end of the release. Let us say this classic editor based build pipeline is MyProduct-R1.0
After the release, I will be tagging the master branch and deleting the Release/R1.0 branch as per GitFlow model. However, I will be retaining the build pipeline MyProduct-R1.0
My question is this: Suppose after the Release R1.0 once master branch has moved ahead and I want to do a build of master branch at R1.0 tag, how do I do using the MyProduct-R1.0 pipeline which was used to originally build the R1.0 release?
I know this is possibly a confusing question, but I have tried my best to give it a shot.
Thanks,
Update 2: I think my question is more about the branch specification for my MyProduct-R1.0 release pipeline. I can't give master since master will evolve after the Release R1.0. I can't give Release/R1.0 since this branch will be deleted after the Release as per gitflow model. So what branch specifications should I provide for my pipeline?
Use YAML builds. There's no mechanism for this with JSON ("classic editor") builds, since JSON builds are versioned separately from source code.
Like Daniel said YAML is the best path. Handles this gracefully as the Build definition lives with the branch.
In the past for GUI builds, when I dealt with build definition drift, instead of creating new build definitions, I would use conditional steps tied to the branch that was being built, but it can be painful to go back and update\maintain.
I think here is what will achieve what I am after. Once I do my release and tag it as R1.0, then I can always use my pipeline at a later date to build the same source code in the master branch for R1.0 using either it's tag or the commit it.
Note that this shows Dev is my default branch for this pipeline as it is not yet the Release pipeline which I depict below. I am yet to create it.
Am I missing something?
We often want to test the first part of our release pipeline using a code from the branch. While I can easily trigger the build from any branch, not so with the release pipeline - I have no idea how to trigger it from anything else other than the branch specified in it.
So far my solution was to temporary clone the pipeline, redirect to another branch and use it while testing. Then delete it.
But I wonder if there is a better way. We use TFS 2018 (on-premises), so no YAML to show here.
This is how our pipeline looks like:
When I click the Source I get:
And the Continuous Deployment Trigger on the Source:
Now clicking the Build Artifact:
And the CD trigger:
The four smoketest5x are the same. They have two tasks (a ps1 script and to publish the test results) and their triggers are:
Pre-Deployment:
Post-Deployment:
Finally, the Production stage. Its pre-deployment trigger is the most interesting:
And there is manual approval.
It does not seem possible to provide the branch at release creation time, but maybe I am missing something here?
You can't specify a branch at release time. If you need to do that, create a build pipeline that publishes an artifact, then release a build. You can choose builds versions at deployment time.
I am modifying a concourse script I inherited.
We have a pipeline that looks like this:
[1]build - [2]test A - [3]test B - [4]publish - [5]deploy
Our team worked only with the master branch before but now we are using feature branches.
I made a modification to build step to run for every branch - not only master. But I want publish step to run only for the master branch builds.
How can I make step 5 to run only when the step 1 was run for the master branch? Is there a way of doing this on Concourse?
If the same pipeline builds all the branches, then either you are not using the git resource or you are using a modified git resource that supports multiple branches.
I assume that what you call "steps" 1...5 are Concourse jobs.
I can see two options.
Option 1. Simple but dirty/misleading.
At the beginning of steps 4 and 5, check for the git branch. If not master, return success immediately. The pipeline will stay green and the publish and deploy jobs will look like they have ran, while actually they did nothing.
Option 2. Slightly more complicated but the proper way.
Use two pipelines. One pipeline for all branches except master. This pipeline will contain only steps 1 ... 3. The second pipeline will track the master branch and have all the steps 1 ... 5.
The question now is how to manage these 2 pipelines in a DRY fashion (no code duplication, no possibility for the pipelines to drift away one from the other)?
The answer is to use a YAML merge tool like https://github.com/geofffranks/spruce. The README mentions BOSH but it is a completely generic YAML merge tool.
The last part is how to stay DRY and at the same time specify different branches. This can be obtained with https://concourse-ci.org/setting-pipelines.html#pipeline-vars.