there are some workflows in a repository and triggered by push behavior.(sometime a push will trigger 2 workflows, sometimes will trigger 5 workflows)
but there are some common works like 'build docker image' for integration test.
I'm not sure how to avoid the duplicated work, I tried reuable workflow, but that will build the image 2 or more times。
is there anyway like the needs keywork to make all triggered action depends on the same job result ？
Check if your use case is similar to "Avoid re-running wokflow for the same commit", with Samuel Ryan's workaround:
Yes, you can use the Check Runs API to identify Workflow runs for a ref.
As luck would have it, someone has already built a comprehensive Action for this use-case: fkirc/skip-duplicate-actions.
Add a new “pre” job to your Workflow, this job uses fkirc/skip-duplicate-actions to determine if your main job should be skipped
Add a condition to your main job using the should_skip output of the “pre” job.
For example, adapted from the fkirc/skip-duplicate-actions README:
jobs:
pre_job:
runs-on: ubuntu-latest
outputs:
should_skip: ${{ steps.skip_check.outputs.should_skip }}
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions#v3.4.0
with:
skip_after_successful_duplicate: 'true'
main_job:
needs: pre_job
if: ${{ needs.pre_job.outputs.should_skip != 'true' }}
runs-on: ubuntu-latest
steps:
- run: echo "Running slow tests..." && sleep 30
Related
I need help figuring something, I am trying to trigger 2 different workflows based on 2 different release tags. I want prod-* to trigger the production workflow and dev-* for the development workflow.
The problem is both tags trigger both workflows and I have no idea how to fix this
(I've canceled both actions but they triggered as you can see)
The tags element is not valid for release events. In consequence, the workflow is triggered for every release event of type published no matter the tag. There is no direct filter for tags with the release event as there is for push and pull_request events.
So you can leverage the if conditional on jobs in combination with the github.ref in the context which contains the tag of the release.
name: Deploy
on:
release:
types: [published]
jobs:
deploy-dev:
name: Deploy to development
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/dev-')
steps:
# [...]
deploy-prod:
name: Deploy to production
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/prod-')
steps:
# [...]
The published trigger is used in both cases here (independently of the tag), because your workflow will start for published OR tags (with the informed pattern) events.
To perform an operation only for a specific tag, you would have to extract the tag version from the $GITHUB_REF (env-var), for example using a step as below with an output in a first job:
- name: Get the version
id: get_tag_version
run: echo ::set-output name=version::${GITHUB_REF/refs\/tags\//}
And then use an if condition on 2 others jobs to check if the tag version contains prod- or dev- (needing the first job) to perform the operation you want for each scenario.
Here is an complete example of what could be used:
name: Example
on:
release:
types: [published]
jobs:
job1:
runs-on: ubuntu-latest
outputs:
tag_version: ${{ steps.get_tag_version.outputs.version }}
steps:
- name: Get the version
id: get_tag_version
run: echo ::set-output name=version::${GITHUB_REF/refs\/tags\//}
job2: # will be executed on for dev- tag
runs-on: ubuntu-latest
needs: [job1]
if: contains( needs.job1.outputs.tag_version , 'dev-')
steps:
[...]
job3: # will be executed on for prod- tag
runs-on: ubuntu-latest
needs: [job1]
if: contains( needs.job1.outputs.tag_version , 'prod-')
steps:
[...]
I coded a workflow to test the implementation above and it worked as expected creating a prod-2 release tag:
workflow file implementation
workflow run
EDIT: Note that you could also use a startWith instead of a contains function for the if expression.
When I create a new release on Github via its UI, I want to trigger the release.yml workflow.
Within the release.yml workflow, I'd like to first run the ci.yml workflow and only if it passes, go ahead and create a release. If the ci.yml worflow fails, remove the newly created release on the Github UI as well.
I have 2 YAML files ci.yml and release.yml
In my release.yml file
on:
release:
types: [created]
jobs:
# I want to run the ci.yml workflow here and then run the jobs below if it passes.
# If the ci.yml workflow fails, revert back and remove the created release.
job1:
.........
job2:
.........
If there is a better way to do achieve this, please let me know.
You can make use of workflow_run event with a condition.
For example, in release.yaml, you can add something like this to run a step only if ci workflow has successfully completed:
on:
workflow_run:
workflows: [ci]
types: [completed]
jobs:
on-success:
runs-on: ubuntu-latest
if: ${{ github.event.workflow_run.conclusion == 'success' }}
steps:
- run: echo 'This is where your release steps can continue'
on-failure:
runs-on: ubuntu-latest
if: ${{ github.event.workflow_run.conclusion == 'failure' }}
steps:
- run: echo 'you can remove this or do something with it like sending an email about why you are not releasing'
There are few other ways to trigger a workflow from another workflow. Some are here:
Using Workflow Dispatch
Repository Dispatch
In your case, if you prefer to use Workflow Dispatch, you can simply invoke the release workflow as last step of ci workflow which will satisfy your needs of running release workflow only when ci is successful.
If you prefer Repository Dispatch, you can dispatch the event at the last step of ci workflow. In this approach you can pass additional inputs to release workflow so that you can have additional flexibility in release workflow.
There are many other way to trigger a workflow which are well documented here. Few of the ways you can consider are: creating a tag in your ci workflow and then use that event in release workflow.
I'm new to GitHub Actions, playing with various options to work out good approaches to CI/CD pipelines.
Initially I had all my CI steps under one job, doing the following:
checkout code from repo
lint
scan source for vulnerabilities
build
test
create image
scan image for vulnerabilities
push to AWS ECR
Some of those steps don't need to be done in sequence though; e.g. we could run linting and source code vulnerability scanning in parallel with the build; saving time (if we assume that those steps are going to pass).
i.e. essentially I'd like my pipeline to do something like this:
job1 = {
- checkout code from repo #required per job, since each job runs on a different runner
- lint
}
job2 = {
- checkout code from repo
- scan source for vulnerabilities
}
job3 = {
- checkout code from repo
- build
- test
- create image
- scan image for vulnerabilities
- await job1 & job2
- push to AWS ECR
}
I have a couple of questions:
Is it possible to setup some await jobN rule within a job; i.e. to view the status of one job from another?
(only relevant if the answer to 1 is Yes): Is there any way to have the failure of one job immediately impact other jobs in the same workflow? i.e. If my linting job detects issues then I can immediately call this a fail, so would want the failure in job1 to immediately stop jobs 2 and 3 from consuming additional time, since they're no longer adding value.
Ideally, some of your jobs should be encapsulated in their own workflows, for example:
Workflow for testing the source by whatever means.
Workflow for (building and-) deploying.
and then, have these workflows depend on each other, or be triggered using different triggers.
Unfortunately, at least for the time being, workflow dependency is not an existing feature (reference).
Edit: Dependencies between workflows is now also possible, as discussed in this StackOverflow question.
Although I feel that including all of your mentioned jobs in a single workflow would create a long and hard to maintain file, I believe you can still achieve your goal by using some of the conditionals provided by the GitHub actions syntax.
Possible options:
jobs.<job_id>.if
jobs.<job_id>.needs
Using the latter, a sample syntax may look like this:
jobs:
job1:
job2:
needs: job1
job3:
needs: [job1, job2]
And here is a workflow ready to be used for testing of the above approach. In this example, job 2 will run only after job 1 completes, and job 3 will not run, since it depends on a job that failed.
name: Experiment
on: [push]
jobs:
job1:
name: Job 1
runs-on: ubuntu-latest
steps:
- name: Sleep and Run
run: |
echo "Sleeping for 10"
sleep 10
job2:
name: Job 2
needs: job1
runs-on: ubuntu-latest
steps:
- name: Dependant is Running
run: |
echo "Completed job 2, but triggering failure"
exit 1
job3:
name: Job 3
needs: job2
runs-on: ubuntu-latest
steps:
- name: Will never run
run: |
echo "If you can read this, the experiment failed"
Relevant docs:
Workflow syntax for GitHub Actions
Context and expression syntax for GitHub Actions
I'm trying to move my CI workflow from CircleCI to GitHub Actions. The last major struggle I'm facing is with deployment.
Currently my workflow is such that when I push a tag to my GitHub repo, it will run the tests, then run the deployment. Only thing is CircleCI filters tags to only run the job if the tag matches the regex: /v[0-9]+(\.[0-9]+)*/.
How can I check to ensure the tag I pushed matches the regex pattern above before running the deployment?
I currently have the following GitHub Actions yml file:
name: CI
on: [create]
jobs:
# ...
deploy:
runs-on: ubuntu-latest
if: github.event.ref_type == 'tag' && github.event.ref == SOMETHING HERE
steps:
- uses: actions/checkout#v1
# ...
Under the if block, I need to change github.event.ref == SOMETHING HERE to be something else. I have looked in the Contexts and expression syntax for GitHub Actions documentation page. But due to how flexible and powerful GitHub Actions is, it seems like there should be a method or way to do this, or at least some type of workaround.
How can I ensure the tag (github.event.ref) matches the regex pattern (/v[0-9]+(\.[0-9]+)*/)?
Unfortunately, I don't think there is any way to do regex matching on if conditional expressions yet.
One option is to use filtering on push events.
on:
push:
tags:
- 'v*.*.*'
Another option is to do the regex check in a separate step where it creates a step output. This can then be used in an if conditional.
- name: Check Tag
id: check-tag
run: |
if [[ ${{ github.event.ref }} =~ ^refs/tags/v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "match=true" >> $GITHUB_OUTPUT
fi
- name: Build
if: steps.check-tag.outputs.match == 'true'
run: |
echo "Tag is a match"
I have managed to achieve this with a two part approach. The first part consists of filtering the tags that you want to run on. The second part is to create a condition on your deploy job.
A cut down version of my workflow looks like the following:
name: CI-CD
on:
push:
branches:
- stable
tags:
- '[0-9]+.[0-9]+.[0-9]+'
- '[0-9]+.[0-9]+.[0-9]+rc[0-9]+'
pull_request:
branches:
- stable
jobs:
test:
steps:
...
deploy:
needs: test
if: startsWith(github.ref, 'refs/tags')
steps:
...
I run my workflows on pushes, tags, and pull request to specific branches. You can get creative with how exactly you want to filter your tags. Check out the filter patterns.
The part I was struggling with a bit was how to set the right condition to run the deploy job. I settled on the following:
needs: test
which ensures that it will only run when the test job has succeeded.
Then I abuse the fact that github.ref will be 'refs/heads/stable' on a push to the branch and 'refs/tags/<your tag>' on pushing a tag. So I simply test for that difference with the following condition.
if: startsWith(github.ref, 'refs/tags')
Works like a charm for me but may depend on your setup.
You can also create separate workflows, of course, but in my case I always wanted to run the test suite first.
EDIT: As #alex-povel comments below, a more explicit check is:
if: github.ref_type == "tag"
As per docs, you can do this:
on:
create:
tags:
- "v[0-9]+.[0-9]+"
I tried the above and can confirm it works. This is not full regex capability but should suffice for your needs.
I'm relatively new to GitHub Actions and I have 2 jobs–one that runs my tests, and one that deploys my project onto a server.
Obviously I want the tests to run on every branch, but deploying should only happen when something gets pushed to master.
I'm struggling to find a way to run a job on a specific branch. I know it's possible to only run entire workflows on a specific branch, however that would mean I would have a "test" workflow and a "deploy" workflow.
This sounds like a solution, however they would run parallel. In an ideal world, the tests would run first, and only if they succeed, then the deploy job would start. This isn't the case when using 2 separate workflows.
How would I be able to achieve this? Is it possible to run jobs on a specific branch?
In a recent update you can now put if conditionals at job level. See the documentation here. https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idif
I tested this workflow which runs the job test on every push, but only runs deploy on the master branch.
name: my workflow
on: push
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Execute tests
run: exit 0
deploy:
runs-on: ubuntu-latest
needs: test
if: github.ref == 'refs/heads/master'
steps:
- name: Deploy app
run: exit 0
What follows is my original answer, and an alternative solution if you prefer to have separate workflows.
The first workflow runs for every branch except master. In this workflow you run tests only.
on:
push:
branches:
- '*'
- '!master'
The second workflow runs for just master and runs both your tests and deploys if the tests were successfully passed.
on:
push:
branches:
- master
Most answers provide a solution for one single branch. To restrict the job to run on any specific set of branches, you can do it using the if conditional with multiple disjunction (||) operators; but this is too verbose and doesn't respect the DRY principle.
The same can be archived with less repetition using the contains function.
Using contains:
contains('
refs/heads/dev
refs/heads/staging
refs/heads/production
', github.ref)
compared to using multiple ||:
github.ref == 'refs/heads/dev' || github.ref == 'refs/heads/staging' || github.ref == 'refs/heads/production' || …
Full example:
---
on: push
jobs:
test:
name: Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Run tests
run: …
deployment:
name: Deployment
runs-on: ubuntu-latest
needs: [test]
if:
contains('
refs/heads/dev
refs/heads/staging
refs/heads/production
', github.ref)
steps:
- uses: actions/checkout#v2
- name: Deploy
run: …
Here is what I've done for steps that should only run on a specific branch.
- name: Publish Image
# Develop branch only
if: github.ref == 'refs/heads/develop'
run: |
... publish commands ...
2021 update
I know it's possible to only run entire workflows on a specific
branch, however that would mean I would have a "test" workflow and a
"deploy" workflow.
This sounds like a solution, however they would run parallel. In an
ideal world, the tests would run first, and only if they succeed, then
the deploy job would start. This isn't the case when using 2 separate
workflows.
You can now use the event workflow_run to achieve the part the tests would run first, and only if they succeed, then the deploy job would start (read on to see how):
Documentation page of workflow_run
https://docs.github.com/en/actions/learn-github-actions/events-that-trigger-workflows#workflow_run
This event occurs when a workflow run is requested or completed, and
allows you to execute a workflow based on the finished result of
another workflow. A workflow run is triggered regardless of the result
of the previous workflow.
For example, if your pull_request workflow generates build artifacts,
you can create a new workflow that uses workflow_run to analyze the
results and add a comment to the original pull request.
Now, considering OP's initial issue:
I want the tests to run on every branch, but deploying should only happen when something gets pushed to master
This now could be solved like this:
The following setup is working, a few minutes ago I just implemented the same logic in one of my repository
Workflow <your_repo>/.github/workflows/tests.yml
name: My tests workflow
on:
push:
branches:
- master
pull_request: {}
jobs:
test:
# ... your implementation to run your tests
Workflow <your_repo>/.github/workflows/deploy.yml
name: My deploy workflow
on:
workflow_run:
workflows: My tests workflow # Reuse the name of your tests workflow
branches: master
types: completed
jobs:
deploy:
# `if` required because a workflow run is triggered regardless of
# the result of the previous workflow (see the documentation page)
if: ${{ github.event.workflow_run.conclusion == 'success' }}
# ... your implementation to deploy your project
Although this discussion is very old but recently I came across the same problem with a slight addition. if condition to check whether the branch is main works but what if someone pushes their branch and updates the workflow yml file to remove the if condition? The deploy job will get triggered without their branch even being reviewed or merged in main and could potentially break the production environment. This could be a concern in open source projects.
I couldn't find an answer for this anywhere so wanted to share my findings. I hope this is the right thread for it.
To ensure that there's no way a job is triggered except for in specific branches, environments can be used. A deploy job will most likely have some api keys to connect to the destination server that might be stored in secrets. Instead of storing them in repository secrets which are accessible globally within the repository, we should store them in respective environments.
The official docs for environments contain detailed explanation with sample scripts but sharing a simple example here. Let's say we want to run production deployment only when main is updated
From repository Settings, create a production environment
Choose Selected Branches in Deployment Branches dropdown and add main in the pattern
Add the api keys in production environment secrets
In the workflow yml we just need to add environment information environment: production as follows (using script from #peterevans's answer)
name: my workflow
on: push
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Execute tests
run: exit 0
deploy:
runs-on: ubuntu-latest
needs: test
if: github.ref == 'refs/heads/main'
environment: production
steps:
- name: Deploy app
run: exit 0
The environment information indicates where the secrets have to be read from. If the current branch name doesn't match the pattern provided in Selected Branches then the job will immediately fail with an error. Since we have a condition to only run this on main, normally that won't bother us because this job will be skipped on other branches anyways. But if someone, mistakenly or with ill intentions, modifies the yml file and removes the condition before pushing their branch, they'll get an error. So, our system remains secure at least from the threat here.
Hope this helps anyone wondering the same.
The following worked for me inside a job:
Where a PR target branch is one of the following:
if: contains(github.base_ref, 'staging')
|| contains(github.base_ref, 'production')
When the source branch of the pull request is one of them below:
if: contains(github.head_ref, 'feature')
|| contains(github.head_ref, 'release')
name: CI
on: push
jobs:
prod-check:
if: ${{ github.ref == 'refs/heads/main' }}
runs-on: ubuntu-latest
steps:
- run: echo "Deploying to production server on branch $GITHUB_REF"
According to the documentation, the if condition is wrapped like so
if: ${{ github.ref == 'refs/heads/main' }}
While you can't have conditions at job level at this moment, you can have conditions at step level - see Contexts and expression syntax for GitHub Actions.
For getting a branch name, current solution is to check GITHUB_REF environment variable - see Default environment variables and this question for details.
Putting it all together - if you decide to go with accepted answer in last link, your workflow might look like this:
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Run tests
run: ./my-tests.sh
deploy:
runs-on: ubuntu-latest
needs: test
steps:
- name: Extract branch name
shell: bash
run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF##*/})"
id: extract_branch
- name: Deploy
run: ./deploy.sh
if: steps.extract_branch.outputs.branch == 'master'
If you'd rather keep everything in workflow file instead of separated script, you can always add if to each step in given job.
I hope it's just a temporary solution/workaround, and job conditions are going to be added before end of beta.
For steps or jobs you can also use github.ref_name which is the branch or tag name that triggered the workflow run.
name: my workflow
on: push
jobs:
if: github.ref_name == 'main'
test:
runs-on: ubuntu-latest
steps:
- name: Execute tests
run: exit 0
For more information about the github context check here.
You need to add these two script to the workflows
//main.yml
name: main
on:
push:
branches:
- main
//size.yml
name: size
on:
pull_request:
branches:
- main
There are already some pretty good answers. Here's a way to specify multiple branches in your conditional:
name: my workflow
on: push
jobs:
deploy:
if: contains(fromJson('["refs/heads/master", "refs/heads/main"]'), github.ref)
runs-on: ubuntu-latest
steps:
- name: Execute tests
run: exit 0
2022
Below workflow runs only for push in main branch:
name: Distribute to Firebase
on:
push:
branches: [ main ]
jobs:
...