Concourse CI, get and put a git-resource - concourse

I'm trying to use a git-resource to get, modify and push a file but it isn't work, can someone help me?
The two resources point to the same git repository and the goal is to add a file in the repository.
I can't understand where I'm wrong, concourse output is green but the repository doesn't have the new file
This is the job:
jobs:
- name: myjob
plan:
- get: input-repo
- get: output-repo
- task: simpletask
config:
platform: linux
image_resource:
type: docker-image
source:
repository: ubuntu
run:
path: sh
args:
- -exc
- |
cat a_file > output-repo/another_file
inputs:
- name: input-repo
- name: output-repo
- put: input-repo
params: { repository: output-repo }

you should not need two different resources for this. What you want to do is to get (git clone) the repo, modify it, git commit it and then put (git push) it.
So you want something like this.
jobs:
- name: myjob
plan:
- get: repo
- task: simpletask
config:
platform: linux
image_resource:
type: docker-image
source:
repository: ubuntu
run:
path: sh
args:
- -exc
- |
cp -r repo changed-repo
cat a_file > changed-repo/another_file
git add .
git commit -m "message"
inputs:
- name: repo
- put: repo
params: { repository: changed-repo }

Related

Github actions: Re-usable workflows

So what I am trying to do is to organize my ci workflow. I have a self-hosted runner that is my personal computer. Since the workflow files is getting too large I am thinking about reducing the size by having different workflow files instead.
so this is my files so far:
name: "Pipeline"
run-name: ${{ github.actor }} just pushed.
on:
schedule:
- cron: "0 2 * * *"
push:
branches-ignore:
- "docs/*"
tags:
- "v*"
workflow_dispatch:
inputs:
cc_platform:
description: "Input of branch name to build metalayers."
required: true
default: "dev"
jobs:
build:
runs-on: self-hosted
name: Build job.
steps:
- name: Checking out the repository.
uses: actions/checkout#v3
- name: Settings Credentials.
uses: webfactory/ssh-agent#v0.6.0
with:
ssh-private-key: ${{ secrets.SSH_KEY_CC }}
- name: Creating config file.
run:
touch conf
- name: Removing old folders & running build script.
run: |
if [[ -d "my_folder" ]]; then rm -rf my_folder; fi
./build
- name: Generate a zip file of artifacts.
uses: actions/upload-artifact#v3
with:
name: art_${{ github.sha }}
path: |
.*.rootfs.wic.xz
.*.rootfs.manifest
if-no-files-found: error
so what I want to do it to have the artifacts and the build step in their own workflow file. But it does give me som concersn.
Will each workflow file get run in their own runner?
Do they share memory?

Concourse CI git clone multiple repos to same directory

Can someone help me to achieve my request. I would like to git clone multiple repos into the same directory where the first git repo is downloaded. Below is my pipeline.yml file. Any help is much appreciated.
resources:
- name: workspace-repo1
type: git
source:
uri: <git-repo1-url>
branch: master
private_key: ((publishing-outputs-private-key))
- name: workspace-repo2
type: git
source:
uri: <git-repo2-url>
branch: master
private_key: ((publishing-outputs-private-key))
- name: workspace-repo3
type: git
source:
uri: <git-repo3-url>
branch: master
private_key: ((publishing-outputs-private-key))
jobs:
- name: job-test
public: true
plan:
- get: workspace-repo1
- get: workspace-repo2
- get: workspace-repo3
- task: app-tests
config:
platform: linux
image_resource:
type: docker-image
source: {repository: golang, tag: 1.14.15-stretch}
inputs:
- name: workspace-repo1 (git repo1, repo2,repo3 should be downloaded here)
outputs:
- name: workspace-repo-deb
run:
path: workspace-repo1/local_build.sh
You could potentially do something like
- task: prep
file: scripts/merge.yml
input_mapping: {repo1: workspace-1, repo1: workspace-2, ...}
---
platform: linux
image_resource:
type: docker-image
source:
repository: alpine
params:
#this could whatever path you want
SRC_PATH: src
inputs:
- name: ci-scripts
- name: repo1
- name: repo2
run:
path: scripts/merge.sh
#this output could be use in the next step, it contains the directories you copied in the script below
outputs:
- name: build
#!/bin/sh
set -e
mkdir -p build
#copy around anything you need
cp -rf repo1/../../. build/
cp -rf repo2/../../. build/
cp -rf repo3/../../. build/
#listing content of build folder just checking
ls -la build/

concourse pipeline - throwing error for sub folders "not a valid repository name"

I have repository which has two frontend application and one server folder. i need to create pipeline for two frontend(angular) and one server(nodejs) folder. if I create a pipeline for main folder(concourse-pipeline) its working fine. but when I try to create to pipeline for subfolders(frontend) its throwing an error as "not a valid repository name". I'm not sure whats going wrong here.
- name: repo
type: git
source:
uri: git#github.com:test-repo/concourse-pipeline.git
branch: master
private_key: ((repo.private-key))
- name: frontend
type: git
source:
uri: git#github.com:test-repo/concourse-pipeline/frontend.git
branch: master
private_key: ((repo.private-key))
- name: version
type: semver
source:
driver: git
initial_version: 0.0.1
uri: git#github.com:test-repo/concourse-pipeline.git
private_key: ((repo.private-key))
branch: master
file: version
- name: run-server
type: git
source:
uri: git#github.com:test-repo/concourse-pipeline.git
branch: master
private_key: ((repo.private-key))
jobs:
- name: run-server
build_logs_to_retain: 20
max_in_flight: 1
plan:
- get: run-server
trigger: true
- task: run-tests
config:
platform: linux
image_resource:
type: registry-image
source:
repository: node
inputs:
- name: run-server
run:
path: /bin/sh
args:
- -c
- |
echo "Node Version: $(node --version)"
echo "NPM Version: $(npm --version)"
cd run-server
npm install
npm test
- name: run-frontend
build_logs_to_retain: 20
max_in_flight: 1
plan:
- get: frontend
trigger: true
- task: run-tests
config:
platform: linux
image_resource:
type: registry-image
source:
repository: node
inputs:
- name: frontend
run:
path: /bin/sh
args:
- -c
- |
echo "Node Version: $(node --version)"
echo "NPM Version: $(npm --version)"
cd frontend
npm install
ng test
- name: bump-version
plan:
- get: repo
trigger: true
- put: version
params:
bump: patch
- name: build-repo
plan:
- get: repo
trigger: true
- get: version
params:
build: repo
tag_file: version/version
tag_as_latest: true
Any help would be appreciated
The uri of the frontend resource seems invalid.
uri: git#github.com:test-repo/concourse-pipeline/frontend.git
The Github address should be only git#github.com:(user):(repository).git

Azure DevOps - multiple repo checkout

Scenario
My code is one repository and all my dependencies is in another repo (another project) that I need to build my code.
Following is my azure-pipelines.yml
# File: azure-pipelines.yml
pool:
vmImage: 'ubuntu-latest'
variables:
phpVersion: 7.3
resources:
repositories:
- repository: myLibraries
type: git
name: myProject/libraries
steps:
- checkout: self
- checkout: myLibraries
path: libraries
- script: |
sudo update-alternatives --set php /usr/bin/php$(phpVersion)
sudo update-alternatives --set phar /usr/bin/phar$(phpVersion)
sudo update-alternatives --set phpdbg /usr/bin/phpdbg$(phpVersion)
sudo update-alternatives --set php-cgi /usr/bin/php-cgi$(phpVersion)
sudo update-alternatives --set phar.phar /usr/bin/phar.phar$(phpVersion)
php -version
displayName: 'Use PHP version $(phpVersion)'
When I run my pipeline, I got the following error:
Checkout of repository 'myLibraries' is not supported. Only 'self' and 'none' are supported.,Checkout of multiple repositories is not supported.
references:
https://github.com/microsoft/azure-pipelines-yaml/blob/master/design/multicheckout.md
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#using-other-repositories
Multiple repository checkout is now supported - https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#multi-repo-checkout
resources:
repositories:
- repository: MyGitHubRepo # The name used to reference this repository in the checkout step
type: github
endpoint: MyGitHubServiceConnection
name: MyGitHubOrgOrUser/MyGitHubRepo
- repository: MyBitBucketRepo
type: bitbucket
endpoint: MyBitBucketServiceConnection
name: MyBitBucketOrgOrUser/MyBitBucketRepo
- repository: MyAzureReposGitRepository
type: git
name: MyProject/MyAzureReposGitRepo
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
- checkout: MyGitHubRepo
- checkout: MyBitBucketRepo
- checkout: MyAzureReposGitRepository
- script: dir $(Build.SourcesDirectory)
The text you posted provides the answer:
Checkout of multiple repositories is not supported.
If you want to use multiple repos in a build, you'll need to do it yourself.
Some options:
Use submodules or subtrees
Have a git clone step that clones the second repository
Publish the relevant contents from the second repository to an artifacts feed and restore them as necessary
All of these have upsides and downsides.

Concourse CI: static_buildpack issue

I am deploying a simple angular4 application to cloud foundry using static_buildpack. While accessing the application I am always getting nginx 403 issue.
jobs:
- name: app
serial: true
plan:
- get: develop-repo
- task: npm-build
config:
platform: linux
image_resource:
type: docker-image
source:
repository: node
run:
path: sh
args:
- -exec
- |
cd develop-repo
npm install
npm run dist
inputs:
- name: develop-repo
outputs:
- name:
- put: develop
params:
manifest: develop-repo/manifest.yml
current_app_name: app
path: develop-repo
resources:
- name: develop-repo
type: git
- name: develop
type: cf
manifest.yml:
---
applications:
- name: app
instances: 1
memory: 512M
disk_quota: 512M
buildpack: staticfile_buildpack
stack: cflinuxfs2
All I am doing is git clone -> npm build -> cf deploy
Note: All resource variables are rightly set. Just ignored for better readability
After trying out couple of options, I found that by publishing the artifacts to the output folder we can push the app from the output folder
---
jobs:
- name: app
serial: true
plan:
- get: develop
- task: npm-build
config:
platform: linux
image_resource:
type: docker-image
source:
repository: node
inputs:
- name: develop
outputs:
- name: artifacts
run:
path: sh
args:
- -exec
- |
cd develop
npm install
npm run dist
ls
cp -R dist ../artifacts/
- put: deploy-cf
params:
manifest: develop/ci/manifests/manifest-int.yml
path: artifacts/dist
resources:
- name: develop
type: git
source:
uri: <<GITHUB-URI>>
branch:<<GITHUB-BRANCH>>
username:<<GITHUB-USERNAME>>
password: <<GITHUB-PASSWORD>>
- name: deploy-cf
type: cf
source:
api: <<CF-API>>
username: <<CF-USERNAME>>
password: <<CF-PASSWORD>>
organization: <<CF-ORG>>
space: <<CF-SPACE>>