Scenario
My code is one repository and all my dependencies is in another repo (another project) that I need to build my code.
Following is my azure-pipelines.yml
# File: azure-pipelines.yml
pool:
vmImage: 'ubuntu-latest'
variables:
phpVersion: 7.3
resources:
repositories:
- repository: myLibraries
type: git
name: myProject/libraries
steps:
- checkout: self
- checkout: myLibraries
path: libraries
- script: |
sudo update-alternatives --set php /usr/bin/php$(phpVersion)
sudo update-alternatives --set phar /usr/bin/phar$(phpVersion)
sudo update-alternatives --set phpdbg /usr/bin/phpdbg$(phpVersion)
sudo update-alternatives --set php-cgi /usr/bin/php-cgi$(phpVersion)
sudo update-alternatives --set phar.phar /usr/bin/phar.phar$(phpVersion)
php -version
displayName: 'Use PHP version $(phpVersion)'
When I run my pipeline, I got the following error:
Checkout of repository 'myLibraries' is not supported. Only 'self' and 'none' are supported.,Checkout of multiple repositories is not supported.
references:
https://github.com/microsoft/azure-pipelines-yaml/blob/master/design/multicheckout.md
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#using-other-repositories
Multiple repository checkout is now supported - https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#multi-repo-checkout
resources:
repositories:
- repository: MyGitHubRepo # The name used to reference this repository in the checkout step
type: github
endpoint: MyGitHubServiceConnection
name: MyGitHubOrgOrUser/MyGitHubRepo
- repository: MyBitBucketRepo
type: bitbucket
endpoint: MyBitBucketServiceConnection
name: MyBitBucketOrgOrUser/MyBitBucketRepo
- repository: MyAzureReposGitRepository
type: git
name: MyProject/MyAzureReposGitRepo
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
- checkout: MyGitHubRepo
- checkout: MyBitBucketRepo
- checkout: MyAzureReposGitRepository
- script: dir $(Build.SourcesDirectory)
The text you posted provides the answer:
Checkout of multiple repositories is not supported.
If you want to use multiple repos in a build, you'll need to do it yourself.
Some options:
Use submodules or subtrees
Have a git clone step that clones the second repository
Publish the relevant contents from the second repository to an artifacts feed and restore them as necessary
All of these have upsides and downsides.
Related
In an Azure DevOps pipeline, I had
resources:
repositories:
- repository: self
type: git
name: MyProject.Web
trigger:
- master
- repository: UiRepo
type: git
name: MyProject.Web.UI
trigger:
- main
[other stuff]
steps:
- checkout: self
- checkout: UiRepo
[other stuff]
This has worked fine: The pipeline ran when triggered by completion of a pull request to either the master branch of the "self" repo or the main branch of the UiRepo. The checkouts pulled from the master branch of "self" and the main branch of UiRepo.
Now I'm adding a branch called release/1.0 to each of the repos, and will introduce other release/x.x branches in the future. So now I have:
resources:
repositories:
- repository: self
type: git
name: MyProject.Web
trigger:
- master
- release/*
- repository: UiRepo
type: git
name: MyProject.Web.UI
trigger:
- main
- release/*
[other stuff]
steps: # ??????
- checkout: self
- checkout: UiRepo
[other stuff]
Now the following is required:
If the triggering branch's name ($Build.SourceBranchName) is either 'master' or 'main', check out master from "self" and main from UiRepo.
Otherwise, check out a branch whose value is $Build.SourceBranch from both repos.
At https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#checking-out-a-specific-ref, I've read that I can use inline syntax in the checkout steps to include specific refs. In my case, I think this comes out as follows:
- ${{ if in(variables['Build.SourceBranchName''], 'master', 'main') }}:
- checkout: git://MyProject/MyProject.Web#refs/heads/master
- checkout: git://MyProject/MyProject.Web.UI#refs/heads/main
- ${{ else }}
- checkout: git://MyProject/MyProject.Web#$(Build.SourceBranch)
- checkout: git://MyProject/MyProject.Web.UI#$(Build.SourceBranch)
(or do I need to be using [ variables['Build.SourceBranch'] ] here instead of $(Build.SourceBranch)?)
But, by dispensing with the references in the checkout steps to "self" and UiRepo, this seems to be divorcing the connection between the checkout steps and the repos as I've already defined them. Does that matter? Is this correct anyway? If it isn't, how can I accomplish my goal?
The if/else you are using is named Conditional Insertion in DevOps concept.
Need to clarify is in this concept, it indeed can accept predefined variables， but only which is 'Available in templates'.
I notice you are using Build.Repository.SourceBranchName, this variable not even in the list of predefined variables, let alone the concept of 'Available in templates'. So the first part of your Conditional Insertion will not be process. Only the second part will be able to process.
And please do not use 'self' as the alias of the repository, this will causes ambiguity.(Your current situation use this is no problem, this is just a suggestion.)
So basically, your pipeline definition should be like this:
trigger:
- none
resources:
repositories:
- repository: self
type: git
name: Repo1
trigger:
- master
- release/*
- repository: UiRepo
type: git
name: Repo2
trigger:
- main
- release/*
pool:
vmImage: ubuntu-latest
steps:
- ${{ if in(variables['Build.SourceBranchName'], 'master', 'main') }}: #This place needs to change
- checkout: git://BowmanCP/Repo1#refs/heads/master
- checkout: git://BowmanCP/Repo2#refs/heads/main
- ${{ else }}:
- checkout: git://BowmanCP/Repo1#$(Build.SourceBranch)
- checkout: git://BowmanCP/Repo2#$(Build.SourceBranch)
Works prefect on my side:
I am trying to deploy to AWS using Github actions. The only problem is, that I have a main repo and frontend and backend submodules inside it.
This is the script I am using for deploy:
name: Deploy
on:
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout source code
uses: actions/checkout#v2
- name: Generate deployment package
run: git submodule update --init --recursive
run: zip -r deploy.zip . -x '*.git*'
- name: Get timestamp
uses: gerred/actions/current-time#master
id: current-time
- name: Run string replace
uses: frabert/replace-string-action#master
id: format-time
with:
pattern: '[:\.]+'
string: "${{ steps.current-time.outputs.time }}"
replace-with: '-'
flags: 'g'
- name: Deploy to EB
uses: einaregilsson/beanstalk-deploy#v18
with:
aws_access_key: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws_secret_key: ${{ secrets.AWS_SECRET_KEY }}
application_name: test-stage
environment_name: Testenv-env
version_label: "${{ steps.format-time.outputs.replaced }}"
region: eu-center-1
deployment_package: deploy.zip
The problem is while it is creating a zip. It does not include submodules. Without submodules the project almost contains nothing. Is it possible somehow to iclude them? Or do you have any better solutions for this?
Consulting the actions/checkout documentation, there is a submodules argument (default value false) that controls whether the checkout includes submodules. So, you likely want
steps:
- name: Checkout source code
uses: actions/checkout#v2
with:
submodules: true
I have pipeline.yml which is stored in repo - buildresources and it also have resources.json file
buildresources repo
pipeline.yml
resources.json
now I want to pass resources.json in following pipeline.yml which further extends template in other repo,how do I pass the resources.json path in buildresources repo to deploy-iac.yml in AzureIacPoC/AzureIacPoc repo
resources:
repositories:
- repository: cloud-iac # The name used to reference this repository in the checkout step
type: git
name: AzureIacPoC/AzureIacPoc
ref: refs/heads/features/azure
trigger:
branches:
include: [features/*, master]
extends:
template: deploy-iac.yml#cloud-iac
parameters:
resourceGroupName: 'anuj-test-automation'
location: 'Canada Central'
csmfile: resources.json
environment: 'sandbox'
resources:
repositories:
- repository: AnotherRepoName
type: git
name: YourProjectName/AnotherRepoName
pool:
vmImage: ubuntu-latest
steps:
- checkout: AnotherRepoName
- script: dir $(Build.SourcesDirectory)
displayName: 'Run a one-line script'
I am using the new Azure DevOps Yaml multi stage pipeline functionality
I've got an Azure DevOps yaml pipeline file for which I want to use templates. I would like the pipeline to checkout self and another repository.
For some reason, self repo has been checked out when this runs, but the repo: pipelines is not being checked out and therefore the job fails (because some of the file dependencies it requires are not there.
Here is an excerpt from my template:
resources:
repositories:
- repository: self
- repository: pipelines
name: vstsproject/pipelines
type: git
source: pipelines
variables:
# Container registry service connection established during pipeline creation
imageRepository: 'vstsprojectweb'
dockerfilePath: '$(Build.SourcesDirectory)/src/Dockerfile.CI'
BuildConfiguration: 'Release'
tag: '$(Build.BuildId)'
stages:
- stage: 'PRD'
jobs:
- template: update-connection-string-db.yml#pipelines
parameters:
resourceGroup: 'application-DEV'
DBSearchString: '###dbservername###'
What is it that I am doing wrong?
I have referred to this microsoft documentation.
You don't need to reference the linked repo in the resources (i.e. self), and if it is the only repo, then it is checked out by default in jobs (not deployment jobs), but if you have additional repos, then you need to check them out manually (with -checkout: <name_of_repo>).
So just do (PS: Cleaned up a little, assumed that the repo is in the same project):
resources:
repositories:
- repository: pipelines
source: pipelines
variables:
# Container registry service connection established during pipeline creation
imageRepository: 'vstsprojectweb'
dockerfilePath: '$(Build.SourcesDirectory)/src/Dockerfile.CI'
BuildConfiguration: 'Release'
tag: '$(Build.BuildId)'
stages:
- stage: 'PRD'
jobs:
- checkout: self
- checkout: pipelines
- template: update-connection-string-db.yml#pipelines
parameters:
resourceGroup: 'application-DEV'
DBSearchString: '###dbservername###'
I ended up putting everything into the same repo and then checking out self in a job.
That worked for me.
jobs:
- job: dbconnectionstring
displayName: 'db connection string'
pool: Windows
steps:
- checkout: self
- template: templates/update-connection-string-db.yml
I'm trying to use a git-resource to get, modify and push a file but it isn't work, can someone help me?
The two resources point to the same git repository and the goal is to add a file in the repository.
I can't understand where I'm wrong, concourse output is green but the repository doesn't have the new file
This is the job:
jobs:
- name: myjob
plan:
- get: input-repo
- get: output-repo
- task: simpletask
config:
platform: linux
image_resource:
type: docker-image
source:
repository: ubuntu
run:
path: sh
args:
- -exc
- |
cat a_file > output-repo/another_file
inputs:
- name: input-repo
- name: output-repo
- put: input-repo
params: { repository: output-repo }
you should not need two different resources for this. What you want to do is to get (git clone) the repo, modify it, git commit it and then put (git push) it.
So you want something like this.
jobs:
- name: myjob
plan:
- get: repo
- task: simpletask
config:
platform: linux
image_resource:
type: docker-image
source:
repository: ubuntu
run:
path: sh
args:
- -exc
- |
cp -r repo changed-repo
cat a_file > changed-repo/another_file
git add .
git commit -m "message"
inputs:
- name: repo
- put: repo
params: { repository: changed-repo }