I have a monorepo where a lot of applications and libs are build.
I tried to make the CI/CD process the more readable and performant as possible.
Problem is I'm not so familiar with all the possibilites azure pipeline give me.
Should I have a pipeline for each environement? for each app? or can I create only one and using tags or something like that?
thanks
For monorepo, you can create one pipeline for all applications, or create individual CI/CD for each one. It depends on how you define in the pipeline.
One pipeline for all: you can use different stages, file path filter, conditions...etc to isolate the builds&testing of application, add stage/job name to display which application it's building.
Individual pipeline: define more pipelines for each application, but could be more readable.
There is a monorepo pipeline sample which has one pipeline and seperate pipelines, you can choose the best way for your project.
Related
We have an Azure DevOps Pipeline defined as a template in a .yml file, and would like to use this to build 100+ projects.
There is zero parameterization to do, and the template essentially just runs against the project repo as-is.
Normally, we would just go into each of those project repos and create an azure-pipelines.yml file and reference the template pipeline, easy.
However, I would like to know if we can avoid creating these azure-pipelines.yml in each of the 100+ project repos. Is there some way to just run the template against 100+ repos, but as separate pipelines?
No.
YAML pipelines are based upon the YAML being stored in the application's repo, as code, alongside the application code. You can pull in templates from other repos, but you can't have one pipeline that is automatically valid for any repo.
Trying to use one template to build 100+ applications without the possibility of pinning the template to a specific version on an application by application basis is setting yourself up for a nightmare at some point in the future, when the template changes in such a way that it's no longer valid for a subset of your 100 applications.
I am using npm package based CI-CD approach for ADF. I want to selectively deploy some pipelines and datasets on prod, instead of deploying everything in repository.
Is there any powershell script where I can send list of ADF objects which I want to deploy using my CI-CD pipeline?
Instead of powershell, if there is any other way, please let me know that as well.
As per official documentation, Data factory entities depend on each other.
For example, triggers depend on pipelines, and pipelines depend on datasets and other pipelines. Selective publishing of a subset of resources could lead to unexpected behaviors and errors.
On rare occasions when you need selective publishing, consider using a hotfix.
Steps to deploy a hotfix
Need your inputs as I am new to Azure DevOps. Is it possible to have multiple release pipeline & single build pipeline in Azure? As per the best practices, if within a single solution, there are multiple projects, it's best to have single build pipeline per project & then create corresponding release pipelines. Now information I have received from developers, their application is so integrated that it is very difficult to seperate out projects to their individual solutions. They are suggesting to create multiple release pipelines for windows services & web services for various projects.
is it still possible?
Is it possible to have multiple release pipeline & single build pipeline in Azure?
The answer is yes.
There is no one-to-one correspondence between the build pipeline and the release pipeline. We can have multiple release pipeline & single build pipeline, or have multiple build pipeline & single release pipeline. It all depends on your needs.
But when we are dealing with these special situations, we need to be very clear about the output of the build pipeline and the generation of artifacts or the input of the release pipeline and the deployment of artifacts, otherwise it is easy to cause confusion.
For your situation, I have create a simple sample to explain the process.
In the build pipeline, I have two projects in my build pipeline. Build the solution (or all projects) by Visual Studio build task, then we need create two artifacts for each projects:
Then publish the build artifacts with different artifact name:
So we get two artifact for one build pipeline:
Last but not least, we create multiple release pipeline (or we could create multiple stages) for windows services & web services. Release pipeline will download two artifacts from the build pipeline, We only need to deploy the artifacts of the corresponding project to the corresponding server instead of deploying all the artifacts to the server.
For example, if we only deploy the artifact from the project1, we just need to select the artifact, like:
The artifact source should be:
$(System.ArtifactsDirectory)\_TestBuild\drop1
Ì would like to deploy Storage, SQL Logical Server and SQL database with ARM. Currently templates are all located under own folder in repo /Storage /SQLLogicalServer /SQLDB
I can deploy they one by one with dedicated pipeline currently.
What is best practice to deploy several Azure resources with single pipeline?
Is it good idea to use one master ARM template that call each resource ARM?
Any tutorial or article about this scenario?
What is best practice to deploy several Azure resources with single
pipeline? Is it good idea to use one master ARM template that call
each resource ARM? Any tutorial or article about this scenario?
You can follow Deploy nested Azure Resource Manager templates for testing environments to combine your three templates into one, then you would easily use one single pipeline to deploy them together. Here's a similar scenario like yours.
Note:
If you have separate pipelines for your three ARM templates, you can easily customize them. E.g: You can set the pipeline to be triggered only when the corresponding ARM template repo is changed. If the ARM templates are in one pipeline, they would be deployed even when only one of them is changed. (In that scenario, you can use this as workaround.)
Whether it's good idea totally depends on your needs/scenarios. Hope it helps :)
I use azure pipeline for build my solution.
Aftere build I need to generate and upload multiple packages with different assets. Packing implemented as a number of additional tasks in my agent job.
But I need ability to generate only selected packages or all packages based on specified arguments.
What is the best way to achieve this?
Ideally, it would have 2 pipelines. The first is automatic project build. And the second should use the result of the first and be able to repeatedly start manually with the desired parameters to exclude project rebuild. But I do not know how this can be implemented.
Not sure if I understand the question correctly, but two possible answers would be:
Classic pipelines, with build pipeline for project build and release pipeline for uploading the artifact(s) that build generates, if using release pipeline is applicable. Release pipeline can have a cd-trigger for the first run, and redeployed manually after that. If need to change release variables for subsequent deployments, you can create a new release with the same build artifact.
Multi-staged pipeline, with build and upload as different stages, manually redeploy/rerun the upload stage when needed. Build phase generates deployable pipeline artifact(s).
Somehow I think you're looking for more elaborate solution, as you state that you're already using pipelines. So how about creative use of conditional tasks (https://learn.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml) using pre-defined variables like Build.Reason (with value 'Manual') to exclude the execution of certain tasks in some runs. Then group tasks you want to re-use into either build templates or task groups.