GitHub Actions CI Conditional Regex - github

I'm trying to move my CI workflow from CircleCI to GitHub Actions. The last major struggle I'm facing is with deployment.
Currently my workflow is such that when I push a tag to my GitHub repo, it will run the tests, then run the deployment. Only thing is CircleCI filters tags to only run the job if the tag matches the regex: /v[0-9]+(\.[0-9]+)*/.
How can I check to ensure the tag I pushed matches the regex pattern above before running the deployment?
I currently have the following GitHub Actions yml file:
name: CI
on: [create]
jobs:
# ...
deploy:
runs-on: ubuntu-latest
if: github.event.ref_type == 'tag' && github.event.ref == SOMETHING HERE
steps:
- uses: actions/checkout#v1
# ...
Under the if block, I need to change github.event.ref == SOMETHING HERE to be something else. I have looked in the Contexts and expression syntax for GitHub Actions documentation page. But due to how flexible and powerful GitHub Actions is, it seems like there should be a method or way to do this, or at least some type of workaround.
How can I ensure the tag (github.event.ref) matches the regex pattern (/v[0-9]+(\.[0-9]+)*/)?

Unfortunately, I don't think there is any way to do regex matching on if conditional expressions yet.
One option is to use filtering on push events.
on:
push:
tags:
- 'v*.*.*'
Another option is to do the regex check in a separate step where it creates a step output. This can then be used in an if conditional.
- name: Check Tag
id: check-tag
run: |
if [[ ${{ github.event.ref }} =~ ^refs/tags/v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "match=true" >> $GITHUB_OUTPUT
fi
- name: Build
if: steps.check-tag.outputs.match == 'true'
run: |
echo "Tag is a match"

I have managed to achieve this with a two part approach. The first part consists of filtering the tags that you want to run on. The second part is to create a condition on your deploy job.
A cut down version of my workflow looks like the following:
name: CI-CD
on:
push:
branches:
- stable
tags:
- '[0-9]+.[0-9]+.[0-9]+'
- '[0-9]+.[0-9]+.[0-9]+rc[0-9]+'
pull_request:
branches:
- stable
jobs:
test:
steps:
...
deploy:
needs: test
if: startsWith(github.ref, 'refs/tags')
steps:
...
I run my workflows on pushes, tags, and pull request to specific branches. You can get creative with how exactly you want to filter your tags. Check out the filter patterns.
The part I was struggling with a bit was how to set the right condition to run the deploy job. I settled on the following:
needs: test
which ensures that it will only run when the test job has succeeded.
Then I abuse the fact that github.ref will be 'refs/heads/stable' on a push to the branch and 'refs/tags/<your tag>' on pushing a tag. So I simply test for that difference with the following condition.
if: startsWith(github.ref, 'refs/tags')
Works like a charm for me but may depend on your setup.
You can also create separate workflows, of course, but in my case I always wanted to run the test suite first.
EDIT: As #alex-povel comments below, a more explicit check is:
if: github.ref_type == "tag"

As per docs, you can do this:
on:
create:
tags:
- "v[0-9]+.[0-9]+"
I tried the above and can confirm it works. This is not full regex capability but should suffice for your needs.

Related

how to avoid the duplicated job in github action like build image?

there are some workflows in a repository and triggered by push behavior.(sometime a push will trigger 2 workflows, sometimes will trigger 5 workflows)
but there are some common works like 'build docker image' for integration test.
I'm not sure how to avoid the duplicated work, I tried reuable workflow, but that will build the image 2 or more times。
is there anyway like the needs keywork to make all triggered action depends on the same job result ?
Check if your use case is similar to "Avoid re-running wokflow for the same commit", with Samuel Ryan's workaround:
Yes, you can use the Check Runs API to identify Workflow runs for a ref.
As luck would have it, someone has already built a comprehensive Action for this use-case: fkirc/skip-duplicate-actions.
Add a new “pre” job to your Workflow, this job uses fkirc/skip-duplicate-actions to determine if your main job should be skipped
Add a condition to your main job using the should_skip output of the “pre” job.
For example, adapted from the fkirc/skip-duplicate-actions README:
jobs:
pre_job:
runs-on: ubuntu-latest
outputs:
should_skip: ${{ steps.skip_check.outputs.should_skip }}
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions#v3.4.0
with:
skip_after_successful_duplicate: 'true'
main_job:
needs: pre_job
if: ${{ needs.pre_job.outputs.should_skip != 'true' }}
runs-on: ubuntu-latest
steps:
- run: echo "Running slow tests..." && sleep 30

How to exclude certain branches for a path pattern within Github Actions?

How to apply branches-ignore and paths-ignore on GitHub Actions? For example, if I have the following source tree:
|-src\
|----reports\
|-------<foo>DailyReports.ts
I want to only deploy that to production (master branch) but NOT staging (staging branch).
Any advice and insight is appreciated.
You won't be able to achieve what you want with ON trigger configuration alone. Therefore the implementation below wouldn't work as it would trigger for push to the specified path OR for push to the master branch, but not only for both:
on:
push:
path:
- 'src/reports/*DailyReports.ts'
branches:
- master
In that case, you could do it by using only one trigger at the workflow level, and check the other condition with an IF expression.
For example by doing this:
on:
push:
path:
- 'src/reports/*DailyReports.ts'
jobs:
job1:
runs-on: ...
if: github.ref == 'refs/heads/master' # run this job only for the master branch
steps:
...
Or by checking the branch at the workflow level, and then use something like this path-filter action to check if the file or directory has been updated or not before performing some operation.
I personally suggest the first option (cf example), as it's less verbose.
Note: If the src/reports/*DailyReports.ts path isn't working as expected, you can use the filter pattern cheat sheet to find the right expression.

Circleci: How to deploy depending on git release(release a tag)

Is there a way to restrict circleci deployment on checkings that have a git release(release a tag)
I think there are several ways to achieve what you want.
With this example you can run a specific deployment only when the tag matches some regex.
workflows:
my-deploy-workflow:
jobs:
- deploy-my-app:
filters:
tags:
only: <put some regex here>
Another option is to use the actual tag, which you can access through pipeline parameters: << pipeline.git.tag >> (link to docs: https://circleci.com/docs/2.0/pipeline-variables/)
Here, you can use the following construct to test for equality:
workflows:
my-deploy-workflow:
when:
and:
- equal: [something, << pipeline.git.tag >>]
jobs:
- deploy-my-app
Additionally, you could also use the filtering options at the job level, instead of the workflow level.

How to get list of files in a pull request in a github action

I'm trying to set up a github action that will automatically request reviewers based on the names of the files that are in the change. For example, if the diff contains a *.sql file, I'd like to request a review from a specific person, and likewise for other file extensions.
This action on the marketplace is what I'm starting with: https://github.com/marketplace/actions/auto-assign-action. I thought the best way to do this would be to use a conditional, something like:
name: 'DB Review'
on: pull_request
jobs:
add-reviews:
runs-on: ubuntu-latest
steps:
- uses: kentaro-m/auto-assign-action#v1.0.1
if: "{{ contains(github.files, '.sql') }}"
with:
repo-token: "${{ secrets.GITHUB_TOKEN }}"
Unfortunately, it doesn't seem like this magical diff list exists: https://help.github.com/en/actions/automating-your-workflow-with-github-actions/contexts-and-expression-syntax-for-github-actions#github-context, so I was hoping for some other suggestions.
One option could be to use the pull_requests.paths filter and create a new workflow for each of the file types when a pull request is opened you want to handle, along with the people who can handle them.
For example:
on:
pull_request:
types: [opened]
paths:
- '**.sql'
jobs:
add-sql-reviews:
runs-on: ubuntu-latest
steps:
- uses: kentaro-m/auto-assign-action#v1.0.1
with:
repo-token: "${{ secrets.GITHUB_TOKEN }}"
configuration-path: ".github/auto_assign_sql.yml"
Now you create this workflow for each file pattern group you want to support and configure who the reviewers are in each of the tasks.
The pull_requests.paths filter is one option, but I think you'll end up with a workflow for each file pattern if you do it that way? Assuming you are going to create more of these...
You could write a custom step in the job which uses the pull request files API, and then you might have a job for each pattern? And an auto-assign config for each job too?
If you're just looking to assign reviewers based on paths, you might consider using GitHub's CODEOWNERS feature instead of a custom workflow. Or if you're looking for more control than that but don't want to write it yourself, PullApprove lets you create assignment rules and groups using almost any data in the GitHub API.
- name: List files in the repository
run: |
ls ${{ github.workspace }}
`

Only run job on specific branch with GitHub Actions

I'm relatively new to GitHub Actions and I have 2 jobs–one that runs my tests, and one that deploys my project onto a server.
Obviously I want the tests to run on every branch, but deploying should only happen when something gets pushed to master.
I'm struggling to find a way to run a job on a specific branch. I know it's possible to only run entire workflows on a specific branch, however that would mean I would have a "test" workflow and a "deploy" workflow.
This sounds like a solution, however they would run parallel. In an ideal world, the tests would run first, and only if they succeed, then the deploy job would start. This isn't the case when using 2 separate workflows.
How would I be able to achieve this? Is it possible to run jobs on a specific branch?
In a recent update you can now put if conditionals at job level. See the documentation here. https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idif
I tested this workflow which runs the job test on every push, but only runs deploy on the master branch.
name: my workflow
on: push
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Execute tests
run: exit 0
deploy:
runs-on: ubuntu-latest
needs: test
if: github.ref == 'refs/heads/master'
steps:
- name: Deploy app
run: exit 0
What follows is my original answer, and an alternative solution if you prefer to have separate workflows.
The first workflow runs for every branch except master. In this workflow you run tests only.
on:
push:
branches:
- '*'
- '!master'
The second workflow runs for just master and runs both your tests and deploys if the tests were successfully passed.
on:
push:
branches:
- master
Most answers provide a solution for one single branch. To restrict the job to run on any specific set of branches, you can do it using the if conditional with multiple disjunction (||) operators; but this is too verbose and doesn't respect the DRY principle.
The same can be archived with less repetition using the contains function.
Using contains:
contains('
refs/heads/dev
refs/heads/staging
refs/heads/production
', github.ref)
compared to using multiple ||:
github.ref == 'refs/heads/dev' || github.ref == 'refs/heads/staging' || github.ref == 'refs/heads/production' || …
Full example:
---
on: push
jobs:
test:
name: Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Run tests
run: …
deployment:
name: Deployment
runs-on: ubuntu-latest
needs: [test]
if:
contains('
refs/heads/dev
refs/heads/staging
refs/heads/production
', github.ref)
steps:
- uses: actions/checkout#v2
- name: Deploy
run: …
Here is what I've done for steps that should only run on a specific branch.
- name: Publish Image
# Develop branch only
if: github.ref == 'refs/heads/develop'
run: |
... publish commands ...
2021 update
I know it's possible to only run entire workflows on a specific
branch, however that would mean I would have a "test" workflow and a
"deploy" workflow.
This sounds like a solution, however they would run parallel. In an
ideal world, the tests would run first, and only if they succeed, then
the deploy job would start. This isn't the case when using 2 separate
workflows.
You can now use the event workflow_run to achieve the part the tests would run first, and only if they succeed, then the deploy job would start (read on to see how):
Documentation page of workflow_run
https://docs.github.com/en/actions/learn-github-actions/events-that-trigger-workflows#workflow_run
This event occurs when a workflow run is requested or completed, and
allows you to execute a workflow based on the finished result of
another workflow. A workflow run is triggered regardless of the result
of the previous workflow.
For example, if your pull_request workflow generates build artifacts,
you can create a new workflow that uses workflow_run to analyze the
results and add a comment to the original pull request.
Now, considering OP's initial issue:
I want the tests to run on every branch, but deploying should only happen when something gets pushed to master
This now could be solved like this:
The following setup is working, a few minutes ago I just implemented the same logic in one of my repository
Workflow <your_repo>/.github/workflows/tests.yml
name: My tests workflow
on:
push:
branches:
- master
pull_request: {}
jobs:
test:
# ... your implementation to run your tests
Workflow <your_repo>/.github/workflows/deploy.yml
name: My deploy workflow
on:
workflow_run:
workflows: My tests workflow # Reuse the name of your tests workflow
branches: master
types: completed
jobs:
deploy:
# `if` required because a workflow run is triggered regardless of
# the result of the previous workflow (see the documentation page)
if: ${{ github.event.workflow_run.conclusion == 'success' }}
# ... your implementation to deploy your project
Although this discussion is very old but recently I came across the same problem with a slight addition. if condition to check whether the branch is main works but what if someone pushes their branch and updates the workflow yml file to remove the if condition? The deploy job will get triggered without their branch even being reviewed or merged in main and could potentially break the production environment. This could be a concern in open source projects.
I couldn't find an answer for this anywhere so wanted to share my findings. I hope this is the right thread for it.
To ensure that there's no way a job is triggered except for in specific branches, environments can be used. A deploy job will most likely have some api keys to connect to the destination server that might be stored in secrets. Instead of storing them in repository secrets which are accessible globally within the repository, we should store them in respective environments.
The official docs for environments contain detailed explanation with sample scripts but sharing a simple example here. Let's say we want to run production deployment only when main is updated
From repository Settings, create a production environment
Choose Selected Branches in Deployment Branches dropdown and add main in the pattern
Add the api keys in production environment secrets
In the workflow yml we just need to add environment information environment: production as follows (using script from #peterevans's answer)
name: my workflow
on: push
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Execute tests
run: exit 0
deploy:
runs-on: ubuntu-latest
needs: test
if: github.ref == 'refs/heads/main'
environment: production
steps:
- name: Deploy app
run: exit 0
The environment information indicates where the secrets have to be read from. If the current branch name doesn't match the pattern provided in Selected Branches then the job will immediately fail with an error. Since we have a condition to only run this on main, normally that won't bother us because this job will be skipped on other branches anyways. But if someone, mistakenly or with ill intentions, modifies the yml file and removes the condition before pushing their branch, they'll get an error. So, our system remains secure at least from the threat here.
Hope this helps anyone wondering the same.
The following worked for me inside a job:
Where a PR target branch is one of the following:
if: contains(github.base_ref, 'staging')
|| contains(github.base_ref, 'production')
When the source branch of the pull request is one of them below:
if: contains(github.head_ref, 'feature')
|| contains(github.head_ref, 'release')
name: CI
on: push
jobs:
prod-check:
if: ${{ github.ref == 'refs/heads/main' }}
runs-on: ubuntu-latest
steps:
- run: echo "Deploying to production server on branch $GITHUB_REF"
According to the documentation, the if condition is wrapped like so
if: ${{ github.ref == 'refs/heads/main' }}
While you can't have conditions at job level at this moment, you can have conditions at step level - see Contexts and expression syntax for GitHub Actions.
For getting a branch name, current solution is to check GITHUB_REF environment variable - see Default environment variables and this question for details.
Putting it all together - if you decide to go with accepted answer in last link, your workflow might look like this:
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Run tests
run: ./my-tests.sh
deploy:
runs-on: ubuntu-latest
needs: test
steps:
- name: Extract branch name
shell: bash
run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF##*/})"
id: extract_branch
- name: Deploy
run: ./deploy.sh
if: steps.extract_branch.outputs.branch == 'master'
If you'd rather keep everything in workflow file instead of separated script, you can always add if to each step in given job.
I hope it's just a temporary solution/workaround, and job conditions are going to be added before end of beta.
For steps or jobs you can also use github.ref_name which is the branch or tag name that triggered the workflow run.
name: my workflow
on: push
jobs:
if: github.ref_name == 'main'
test:
runs-on: ubuntu-latest
steps:
- name: Execute tests
run: exit 0
For more information about the github context check here.
You need to add these two script to the workflows
//main.yml
name: main
on:
push:
branches:
- main
//size.yml
name: size
on:
pull_request:
branches:
- main
There are already some pretty good answers. Here's a way to specify multiple branches in your conditional:
name: my workflow
on: push
jobs:
deploy:
if: contains(fromJson('["refs/heads/master", "refs/heads/main"]'), github.ref)
runs-on: ubuntu-latest
steps:
- name: Execute tests
run: exit 0
2022
Below workflow runs only for push in main branch:
name: Distribute to Firebase
on:
push:
branches: [ main ]
jobs:
...