I'm automating the testing and deployment of a web app. I have the following stages that need to run:
Test
Deploy (dev)
Deploy (prod)
When a PR is opened, I'd like only stages 1. (test) and 2. (deploy, dev) to run. Then, when the PR is merged to master, I'd like only stage 3. (deploy, prod) to run. I don't want any CI/CD on regular commits/pushes.
Here's how I'm trying to define my .travis.yml:
# install and other things omitted for brevity
stages:
- name: dev-test
if: type = pull_request
branch:
except: master
- name: dev-deploy
if: type = pull_request
branch:
except: master
- name: prod-deploy
if: type = pull_request
branch:
only: master
jobs:
include:
- stage: dev-test
script: python scripts/dev_test.py
- stage: dev-deploy
script: python scripts/dev_deploy.py
- stage: prod-deploy
script: python scripts/prod_deploy.py
Right now all three stages run when a PR is opened (not just dev-test and dev-deploy). How should I set up my stages to accomplish what I want?
I solved this by not using build stages altogether. Instead, I just verify attributes of the GitHub PR in my script directive in .travis.yml, via Travis environment variables.
# install and other things omitted for brevity
script:
- if [[ $TRAVIS_EVENT_TYPE == "pull_request" ]] ; then python scripts/dev_test.py ; fi
- if [[ $TRAVIS_EVENT_TYPE == "pull_request" && $TRAVIS_PULL_REQUEST_BRANCH != "master" ]] ; then python scripts/dev_deploy.py ; fi
- if [[ $TRAVIS_EVENT_TYPE == "push" && $TRAVIS_BRANCH == "master" ]] ; then python scripts/prod_deploy.py ; fi
Related
The environment which I'm deploying to has 3 stages
dev
uat
production
The git repository for the project has dev and main branches.
On push to dev, the CI pipeline runs, creates an image and pushes the image to a registry.
Next the CD release pipeline is triggered and auto deploys to the dev environment.
I want the next stage to be triggered by the merge from dev to main,
so I can't see how I can use the same CD pipeline - I'd love to be shown that this is actually possible.
So for now, I'm going with 2 CI pipelines and 2 cd pipelines as follows:
dev-branch
CI (auto triggered by push to dev)
CD
dev-stage (auto triggered by CI above)
main-branch
CI (auto triggered by push to main)
CD
uat-stage (auto triggered by CI above)
prod-stage
The problem that I'd like to solve with this setup is :
can this be done with a single CI pipeline, suppressing the CD pipeline runs based on the source branch in git which triggered the build?
My CI pipeline yaml is as follows, I'd like to use this one file for both main and dev,
rather than having to duplicate it with the only difference being the individual triggers for dev and main.
trigger:
branches:
include:
- dev
- main
variables:
- group: devops-project-group
- template: app-variables.yml
pool:
vmImage: ubuntu-latest
steps:
- checkout: self
- bash: |
echo 'starting build ...'
you can use conditions:
- bash: |
echo 'starting MASTER build ...'
condition: and(succeeded(), variables['Build.SourceBranch'], 'refs/heads/master')
alternatively you can do ifs:
steps:
- ${{ if eq(variables['Build.SourceBranch'], 'refs/heads/master') }}:
- bash: master step1
- bash: master step2
etc
${{ else }}:
- bash: dev step1
- bash: dev step2
etc
-
or you can determine branch in the bash step and do something accordingly
I have a github workflow, that runs automatically on push to branch 'dev' and I would like to have a specific step that only runs in dev and manually triggered.
I know that this is posible using GitLab, by adding "when: manual" to the step but I can't find a way of doing this on GitHub.
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [ dev ]
#pull_request:
# branches: [ dev ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
jobs:
triage:
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/dev'
steps:
- uses: actions/labeler#v4
test-workflow2:
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/dev'
steps:
# Runs a set of commands using the runners shell
- name: Deploy latest
run: |
echo the test worked
test-workflow3-manual:
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/dev'
steps:
# Runs a set of commands using the runners shell
- name: Deploy latest
run: |
echo the test worked
# when:
# manual ???
Thank you all in advance and have a nice week
Checking the current branch and event should work with an if:
if: ${{ github.ref_name == 'dev' && github.event_name == 'workflow_dispatch' }}
See github context for more details.
I have the problem, that I want to trigger another pipeline (B) in antoher project (B), only when the deploy job in pipeline (A) is finished. But my configuration starts the second pipeline as soon as the deploy job in pipeline (A) starts. How can I do it, that the second pipeline is triggered, only when the deploy job in pipeline (A) in projet (A) is finished?
Here is my gitlab-ci.yml
workflow:
rules:
- if: '$CI_COMMIT_BRANCH'
before_script:
- gem install bundler
- bundle install
pages:
stage: deploy
script:
- bundle exec jekyll build -d public
artifacts:
paths:
- public
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
staging:
variables:
ENVIRONMENT: staging
stage: build
trigger: example/example
test:
stage: test
script:
- bundle exec jekyll build -d test
artifacts:
paths:
- test
rules:
- if: '$CI_COMMIT_BRANCH != "master"'
You don't declare stages order, so gitlab pipeline don't know what order are expected.
At the beginning of .gitlab-ci.yaml file add something like this (or whatever order you want):
stages:
- deploy
- test
- build
# rest of you file...
Alternatively you can use needs to build jobs relation.
I currently have two GitHub actions workflow files that are pretty much identical, only one is configured to react to push/pull_requests on branch master the other on production.
The staging workflow starts like this:
env:
GCLOUD_PROJECT: my-project-stg
VERCEL_TARGET: staging
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
The production workflow starts like:
env:
GCLOUD_PROJECT: my-project-prd
VERCEL_TARGET: production
on:
push:
branches: [ production ]
pull_request:
branches: [ production ]
The rest of the workflow files is the same, so this is clearly not very DRY.
I would like to have 1 single workflow file and somehow switch between two sets of variables based on the branch name. Is there a way to achieve this or am I maybe approach this from the wrong angle?
If it were possible to extend both workflow files on a shared base definition that would also solve it I guess.
It is not possible to set workflow-level environment variables from a job. Each job runs in its own machine and setting an env variable there only affects all of the later steps in that job.
There are currently two ways to share data between jobs; either you create an artifact and use that, or you declare and set job outputs. The latter works for string values.
Here's an example:
name: "Main"
on:
push:
branches:
- master
- production
pull_request:
branches:
- master
- production
jobs:
init:
runs-on: ubuntu-latest
outputs:
gcloud_project: ${{ steps.setvars.outputs.gcloud_project }}
phase: ${{ steps.setvars.outputs.phase }}
steps:
- name: Cancel previous workflow
uses: styfle/cancel-workflow-action#0.4.0
with:
access_token: ${{ github.token }}
- name: Set variables
id: setvars
run: |
if [[ "${{github.base_ref}}" == "master" || "${{github.ref}}" == "refs/heads/master" ]]; then
echo "::set-output name=gcloud_project::my-project-dev"
echo "::set-output name=phase::staging"
fi
if [[ "${{github.base_ref}}" == "production" || "${{github.ref}}" == "refs/heads/production" ]]; then
echo "::set-output name=gcloud_project::my-project-prd"
echo "::set-output name=phase::production"
fi
print:
runs-on: ubuntu-latest
needs: init
steps:
- name: Print
run: echo "gcloud_project=${{needs.init.outputs.gcloud_project}}"
You can drop the global env statement, combine the event triggers to
on:
push:
branches:
- master
- production
pull_request:
branches:
- master
- production
and then add a first step that checks which branch the workflow is running on and set the environment there:
- name: Set environment for branch
run: |
if [[ $GITHUB_REF == 'refs/heads/master' ]]; then
echo "GLCOUD_PROJECT=my-project-stg" >> "$GITHUB_ENV"
echo "VERCEL_TARGET=staging" >> "$GITHUB_ENV"
else
echo "GLCOUD_PROJECT=my-project-prd" >> "$GITHUB_ENV"
echo "VERCEL_TARGET=production" >> "$GITHUB_ENV"
fi
Things could have changed but for anyone still searching for an answer, there is a way to set a variabe visible for the entire workflow.
jobs:
weekday_job:
runs-on: ubuntu-latest
env:
DAY_OF_WEEK: Mon
Then access by jobs.<job_id>.steps[*].env
More info in the docs
There's now a better way of achieving this using reusable workflows:
https://docs.github.com/en/actions/learn-github-actions/reusing-workflows
I currently have two GitHub actions workflow files that are pretty much identical, only one is configured to react to push/pull_requests on branch master the other on production.
The staging workflow starts like this:
env:
GCLOUD_PROJECT: my-project-stg
VERCEL_TARGET: staging
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
The production workflow starts like:
env:
GCLOUD_PROJECT: my-project-prd
VERCEL_TARGET: production
on:
push:
branches: [ production ]
pull_request:
branches: [ production ]
The rest of the workflow files is the same, so this is clearly not very DRY.
I would like to have 1 single workflow file and somehow switch between two sets of variables based on the branch name. Is there a way to achieve this or am I maybe approach this from the wrong angle?
If it were possible to extend both workflow files on a shared base definition that would also solve it I guess.
It is not possible to set workflow-level environment variables from a job. Each job runs in its own machine and setting an env variable there only affects all of the later steps in that job.
There are currently two ways to share data between jobs; either you create an artifact and use that, or you declare and set job outputs. The latter works for string values.
Here's an example:
name: "Main"
on:
push:
branches:
- master
- production
pull_request:
branches:
- master
- production
jobs:
init:
runs-on: ubuntu-latest
outputs:
gcloud_project: ${{ steps.setvars.outputs.gcloud_project }}
phase: ${{ steps.setvars.outputs.phase }}
steps:
- name: Cancel previous workflow
uses: styfle/cancel-workflow-action#0.4.0
with:
access_token: ${{ github.token }}
- name: Set variables
id: setvars
run: |
if [[ "${{github.base_ref}}" == "master" || "${{github.ref}}" == "refs/heads/master" ]]; then
echo "::set-output name=gcloud_project::my-project-dev"
echo "::set-output name=phase::staging"
fi
if [[ "${{github.base_ref}}" == "production" || "${{github.ref}}" == "refs/heads/production" ]]; then
echo "::set-output name=gcloud_project::my-project-prd"
echo "::set-output name=phase::production"
fi
print:
runs-on: ubuntu-latest
needs: init
steps:
- name: Print
run: echo "gcloud_project=${{needs.init.outputs.gcloud_project}}"
You can drop the global env statement, combine the event triggers to
on:
push:
branches:
- master
- production
pull_request:
branches:
- master
- production
and then add a first step that checks which branch the workflow is running on and set the environment there:
- name: Set environment for branch
run: |
if [[ $GITHUB_REF == 'refs/heads/master' ]]; then
echo "GLCOUD_PROJECT=my-project-stg" >> "$GITHUB_ENV"
echo "VERCEL_TARGET=staging" >> "$GITHUB_ENV"
else
echo "GLCOUD_PROJECT=my-project-prd" >> "$GITHUB_ENV"
echo "VERCEL_TARGET=production" >> "$GITHUB_ENV"
fi
Things could have changed but for anyone still searching for an answer, there is a way to set a variabe visible for the entire workflow.
jobs:
weekday_job:
runs-on: ubuntu-latest
env:
DAY_OF_WEEK: Mon
Then access by jobs.<job_id>.steps[*].env
More info in the docs
There's now a better way of achieving this using reusable workflows:
https://docs.github.com/en/actions/learn-github-actions/reusing-workflows