I am setting up CI/CD at work and there is one step I’m not sure how to do and furthermore, if it is a right thing do.
For background, I am used to develop in C# with Visual Studio, source code in TFS and deploying with basic script that copies files on the intranet.
Now, I’m requested to setup Build and Release pipelines on Dot Net Core projects in GitHub.
I have three branches on this project: DEV, RELEASE and MASTER
I created one pipeline that triggers on DEV’s commits, creates an artefact and deploy to DEV server.
Those are the pipelines that deploy all developers work to a DEV server where they run their own tests.
Next step, when we want to deploy to staging servers, we click a button in Azure, this merge the DEV branch to the RELEASE branch but I know close to nothing in GitHub, not even sure those are the appropriate words.
When the merge is done, this will trigger a build pipeline that will create a different artefact, when this artefact is updated, deploy to staging server.
Once this release is validated on Staging and Quality, we would merge RELEASE to MASTER and do the same until PROD servers. It is all on intranet and self-hosted agents.
Is that a good way of doing things? Can it be done this way? I need a PowerShell task or is there something that exists?
If you are using Azure DevOps pipeline, the pipeline should select GitHub for repository type, then we can configure the CI trigger.
a. Configure CI trigger:
Classic steps:
1.Open project setting->Service connections->select GitHub-> create a new GitHub service connection
2.Create a new build pipeline via classic editor-> Select GitHub as the source.
3.Open pipeline->select the tab Triggers-> enable the option Enable continuous integration and configure the Branch filters
b.YAML steps:
1.Open project setting->Service connections->select GitHub-> create a new GitHub service connection
2.Create a new build pipeline and select GitHub(YAML)
c.The sample of Check out GitHub repositories in your pipeline
resources:
repositories:
- repository: MyGitHubRepo # The name used to reference this repository in the checkout step
type: github
endpoint: MyGitHubServiceConnection
name: MyGitHubOrgOrUser/MyGitHubRepo
trigger:
- {branch name}
Configure CD trigger:
Please refer to this doc to configure the release trigger.
If you are using GitHub action.
Please select the correct workflow to configure the CI/CD, Please refer this doc for more details
CI sample:
on:
push:
branches:
- ' DEV'
Update1
When Dev branch is updated, it is built and deployed to Dev server by pipeline.
Create build A and release B, configure the CI build trigger, when the branch Dev is updated, it triggers the build pipeline A, and when build pipeline A is completed, it triggers the release pipeline B.
click a button to synchronize Dev branch to Release branch.
We cannot see the button, as a workaround, we can add task powershell and call the API to create pull request and complete the pull request. We also can add task cmd and publish the code via git cmd
Related
I'm trying to implement azure automatic pipeline build with azure-pipelines.yaml
i have tried the below methods in order to get it work.
azure-pipelines.yaml(yml)
.vsts-ci.yml(yaml)
.azure-pipelines.yaml(yml)
But when I'm trying with UI it's detecting.
Is it possible to build auto build pipeline in Azure-devops?
If you're building a GitHub repo in an Azure DevOps pipeline, there's some stuff you need to setup.
Azure Pipelines can automatically build and validate every pull request and commit to your GitHub repository. This article describes how to configure the integration between GitHub and Azure Pipelines.
If you're new to Azure Pipelines integration with GitHub, follow the steps in Create your first pipeline to get your first pipeline working with a GitHub repository, and then come back to this article to learn more about configuring and customizing the integration between GitHub and Azure Pipelines.
Source: Build GitHub repositories.
If you explicitly need a DevOps Pipeline to build your GitHub repo, have a look at the article linked to above. Otherwise think about either working with GitHub Actionshttps://github.com/features/actions on your GitHub repo, or hosting your repo in Azure Repos.
Depending on where your repository is located you should place your .yml file inside this repository in order to work with the trigger. You should also use the trigger keyword.
Then when you create your pipeline, you will choose Github, Azure Repos Git, whatever you use and select existing pipeline.
If you use the starter pipeline of Azure Devops, the pipeline will trigger out of the box when a commit is pushed on the main branch.
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- script: echo Hello, world!
displayName: 'Run a one-line script'
- script: |
echo Add other tasks to build, test, and deploy your project.
echo See https://aka.ms/yaml
displayName: 'Run a multi-line script'
You can change main to which ever branch you need to trigger automatically.
I would like to run a yaml pipeline from one project. I have a task in my yaml to scan all the source code. Using this Yaml I would like to scan all the source code in master branch for all the project and all the repository inside the same Org.
How can I get all the repo for all the project and iterate? Can someone help me ?
test.yaml
repositories:
- repository: justAnotherName
type: github
name: myGitRepo
endpoint: myGitServiceConnection
trigger:
branches:
include:
- master
steps:
- task: CredScan#2
inputs:
toolMajorVersion: 'V2'
outputFormat: 'tsv'
scanFolder: '$(Build.SourcesDirectory)'
If you're looking to pull every repo within a project, you have one of two options (see below). However, I'd advise caution before attempting this on a Microsoft-hosted agent, they have a 60-minute timeout by default. If you're using a self-hosted agent, you need not worry. I'd still advise breaking this up to avoid creating a long-running release that also consumes a large amount of disk space with each run.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml#timeouts
That being said, here are the options you have:
Option 1 (Not the best)
Manually add a repository: dependency for every project and a checkout: task for every repo within the projects.
This is heavily manual and would require maintenance every time a report is added.
Option 2
You can write a custom PowerShell/bash script that uses the Azure DevOps API and git to automatically scan all projects and repos within the org and pull them onto the machine.
Start by issuing a request to get all of the projects within the org:
https://learn.microsoft.com/en-us/rest/api/azure/devops/core/projects/list?view=azure-devops-rest-6.0
Then, iterate through every project and get all repos:
https://learn.microsoft.com/en-us/rest/api/azure/devops/git/repositories/list?view=azure-devops-rest-6.0
Finally, iterate through each repo and run git clone [repository URL] to clone it onto the build agent.
NOTE: You will want to ensure to have a lot of free disk space on the agent machine and that you clean up the build space after this operation.
I am working on an Azure pipeline for a dotnet core project that runs on Windows Self hosted agent.
I want my pipeline to use Git commands to check out the release branch and merge the develop branch into it. Next steps will be to build the release branch and deploy to intranet servers
I don’t know Git wording very good, I was using TFS for years. I use the commands below and got the logs here:
- task: CmdLine#2
displayName: Checkout Release branch
inputs:
script: |
#echo off
git checkout release
git pull develop
git status
From the logs, I understand:
It downloads the content of the develop branch because it is the default branch in GitHub, I’d rather want the release branch but I believe Azure is like that
I manage to switch to release but I have these errors that I don’t understand:
##[error]Previous HEAD position was bc94bd2 Update Staging Build Pipeline.yml
##[error]Switched to branch 'release'
I understood that pull can be used with local or remote branch so I use it to fetch and merge the develop branch to the release branch but I get: [error]fatal: 'develop' does not appear to be a git repository
Do I have to specify credentials on every calls to git?
On the last step, it fetches again the code from the develop branch and I understand why
If you could help me improve my script, that would be great,
Many thanks.
You can use git merge commands to merge branches. To merge develop branch into release branch you can use git merge origin/develop. Check the document for more information. See below example:
steps:
- checkout: self
persistCredentials: true
- task: cmdLine#2
inputs:
script: |
#echo off
git checkout release
git merge origin/develop
git status
However, it is not recommended to deploy release branch in above way. You can change the default branch of your azure pipeline to release branch and enabled the Continuous Integration trigger for release branch.
So that you can create a pull request to merge develop into release from the github UI or by using commands. After develop is merged into release, the azure pipeline will be automatically triggered to deploy from release branch. Note: the azure pipeline yaml file must exist in release branch too. See below steps:
1, To change azure pipeline branch from develop to release:
On your azure devops pipeline Edit page, Click the 3dots and click Triggers
Go to YAML tab--> Get Sources-->Click the 3dots to change the default branch.
2, Set CI trigger for release branch
In the azure pipeline yaml file, set the trigger to include release branch(You can also set PR trigger):
(Actually you do not need to follow above steps to change the default branch. You just need to include the azure pipeline yaml file in release branch and set the CI trigger to include release branch as below)
trigger:
branches:
include:
- release
exclude:
- develop #if you want disable CI trigger for develop branch
By adding the CI trigger to include the release branch in the azure pipeline yaml file. Azure pipeline will automatically be triggered on release branch when merging from develop into release branch.
I have four environments that I deploy to.
I also have four different code branches that we use to deploy code from.
We constantly switch the branches we use to deploy on these environments.
One time I want to build and deploy a daily branch on my test environment.
Later I want to build and deploy a enhancements branch on the same test environment.
Next I want to build and deploy the daily branch on my test2 environment.
I think you get the picture
We are currently using a manual process to pull from the branch we want deployed, then zip it up and push it to AWS code deploy.
Using Azure DevOps pipeline and release what is the easiest method to allow me to switch to use different branchs on different environments.
I currently have a successful setup in Azure DevOps that performs a gradle build, creates the artifact and then lets me push it over to AWS CodeDeploy on one of my environments. I just can't seem to figure out a way to eastily swtich the branch without creating tons of Azure pipelines and releases.
Thanks all!
Where you manually trigger a build pipeline by clicking Queue or Run Pipeline, A new windows shown as below will be prompted which allows you to switch the branches.
If you want to automatically deploy different branch to different environment. You can push the build artifacts over to AWS CodeDeploy in a release pipeline and set the branch filters. Please refer to below steps:
1, set branch filter in the build pipeline as shown in below screenshot which will build the selected branched. Check here for more information about triggers.
2, create a release pipline to push build artifacts over to AWS CodeDeploy.
And Set the Artifact filters which will only allow the artifacts built from the specified branch to be deployed to this tage.
You could use a queue time variable to specify the branch name you would like to use on your build pipeline. You would need to:
Edit your build pipeline and create the variable on the "variables" tab. Make sure to mark the "Settable at queue time" check
variable creation
Update the source of your build pipeline, to specify the new variable under the "Default branch" option. It would look something like this:
pipeline source
RUN your pipeline. Before finally clicking on RUN, you will be able to specify the desire branch:
set variable value
Hope this works
I have my git repository hosted on Azure DevOps. I created a new yaml based build pipeline in the master branch and have set the trigger section to two existing branches. Other branches don't have a azure-pipeline.yml file nor any kind of branch policies are set for this DevOps project.
trigger:
batch: 'true'
branches:
include:
- master
- develop
The trigger gets invoked for every change in the master branch as expected. But is ignoring any pushes to the develop branch.
If I configure a build pipeline with the visual editor and define the exact two branches there, for every push a build will be triggered.
Any idea how Azure Pipeline respects the build definition also for other branches without copy and pasting the whole definition for every possible branch?
Build trigger is not invoked on push for Azure Git repositories
I have created a sample with the syntax:
trigger:
batch: 'true'
branches:
include:
- master
- Dev
And it works fine on my side. Then I check the new project you provided, but I found that the .yml file is incomplete and does not contain a trigger: node.
So, to resolve this issue, we need to double check the .yml file you modified in under the master branch, and you build .yml file is you modified.
Besides, when we edit the build pipeline, there is a extended button, we could select the option Triggers to set the build trigger with visual editor:
If above not help you, you can try to create a new build pipeline, set the trigger only with Develop branch, check if it works fine, then return to the previous with master
and develop branch.
If all of the above methods are not worked, you may need share a detailed sample and some steps, the reason for this problem may be hidden in the corner we ignore.
Hope this helps.