I have a repository which contains deploy keys. I want a workflow job which periodically checks whether any of the deploy keys are reaching their maximum allowed age before they must be rotated. I tried writing a workflow like this, using the GITHUB_TOKEN, but it looks like it doesn't have the necessary privileges. My repository belongs to a GitHub Organization.
name: Check age of repository deploy key
# This workflow is triggered on pushes to the repository.
on:
push:
schedule:
# Runs 06:00 every day
- cron: '0 6 */1 * *'
jobs:
expiry_check:
env:
DEPLOY_KEY_METADATA_URL: https://api.github.com/repos/my_org/my_repo/keys
DEPLOY_KEY_MAX_AGE: 3600*24*365 # 1 year
# This job runs on Linux
runs-on: ubuntu-latest
steps:
# GitHub repository checkout
- name: GitHub repository checkout
uses: actions/checkout#v1
- name: Check if any deploy keys are approaching their expiry data
run: |
python3 -c "import requests;import sys;url=sys.argv[1];token=sys.argv[2];r=requests.get(url, headers={'Authorization': f'Bearer {token}'});print(r.text)" $DEPLOY_KEY_METADATA_URL ${{ secrets.GITHUB_TOKEN }}
The response to my API request has this error: {"message":"Resource not accessible by integration","documentation_url":"https://docs.github.com/rest/reference/repos#list-deploy-keys"}
Is there some other solution to this problem, besides personal access tokens and GitHub Apps? The first option is not feasible; business logic can't break when an employee leaves the GitHub Organization. I suppose I could make a GitHub App, but I'd rather avoid that too, if I can. I'm not an admin in my GitHub Organization.
Related
Can I trigger a new workflow from another workflow?
I'm trying to run a workflow after the first workflow has pushed a new release and it seems to ignore it.
Found the answer here:
An action in a workflow run can't trigger a new workflow run. For example, if an action pushes code using the repository's GITHUB_TOKEN, a new workflow will not run even when the repository contains a workflow configured to run when push events occur.
EDIT:
The quote above might be confusing. When I add a Personal Access Token (PAT) to the checkout action with repo permissions granted (and not repository's GITHUB_TOKEN), the following commands DO trigger other workflows:
- name: Checkout Repo
uses: actions/checkout#v2
with:
token: ${{ secrets.PAT_TOKEN }}
(In my case, running semnatic-release after this checkout, which creates a new release with a new tag - did trigger another workflow that runs only if a tag was created)
As described here, you can trigger another workflow using the workflow_run event.
For example we could think of two workflow definitions like this (the only prerequisite is, that both reside in the same repository - but I'am sure, there's also an event for other repos as well):
release.yml
name: CI release
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Release artifact
run: ...
do-something-different.yml
name: Do anything after the release of the first workflow
on:
workflow_run:
workflows: ["CI release"]
types:
- completed
jobs:
notify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Do something
run: ...
A crucial point here is that the name: CI release definition of the first yaml file must exactly match the workflow_run: workflows: ["CI release"] definition in the second yaml file. Another point is that this approach needs to be done on the default branch (which is mostly main or master) as the docs state:
Note: This event will only trigger a workflow run if the workflow file
is on the default branch.
If you don't want to use a general Personal Access Token (which has access to all of your repos), you can generate a dedicated SSH keypair for this purpose and add it to the repository as a Deploy Key. This is done as follows:
Generate an SSH keypair:
ssh-keygen -N "" -f deploy_key -C "github-actions"
Add the private key (generated file deploy_key) as an encryped secret, e.g. COMMIT_KEY to the GitHub project.
Add the public key (generated file deploy_key.pub) as a deploy key with write access to the GitHub project. Tick the Allow write access checkbox.
When checking out the source code in your workflow, add the SSH key:
- name: Checkout
uses: actions/checkout#v3
with:
ssh-key: "${{secrets.COMMIT_KEY}}"
Subsequent push actions in the same workflow will then trigger any configured GitHub workflow as if they were pushed manually.
Another frequently-requested feature for Actions is a way to trigger one workflow based on the completion of another workflow. For example, you may want to take the results of a CI workflow and run some further analysis.
The new workflow_run event enables you to trigger a new workflow when one or more workflows are requested or completed. Runs triggered by the workflow_run event always use the default branch for the repository, and have access to a read/write token as well as secrets. As an example, as a maintainer you could set up a workflow that takes the artifacts generated by the pull request workflow, do some analysis, and post comments back to the pull request. This event is also available as a webhook and works all repos.
This is quoted from Github's blog.
Could anybody tell me how to implement the example proposed using the new event workflow_run? The documentation only provide a very simple example:
on:
workflow_run:
workflows: ["Run Tests"]
branches: [main]
types:
- completed
- requested
I would be very glad if someone can teach me how to achieve the example.
To get the example to work (i.e. to have one workflow wait for another to complete) you need two files. Both files live in the .github/workflows folder of a repository.
The first file would be set up as usual. This file will be triggered by whatever event(s) are set in the on section:
---
name: Preflight
on:
- pull_request
- push
jobs:
preflight-job:
name: Preflight Step
runs-on: ubuntu-latest
steps:
- run: env
The second file states that it should only trigger on the workflow_run event for any workflows with the name Preflight:
---
name: Test
on:
workflow_run:
workflows:
- Preflight
types:
- completed
jobs:
test-job:
name: Test Step
runs-on: ubuntu-latest
steps:
- run: env
This more-or-less the same as the example from the GitHub Actions manual.
As you can see on the actions page
of my example repo, the Preflight workflow will run first. After it has completed, the Test workflow will be triggered:
As you can also see, the branch does not appear for the "Test" workflow.
This is because, (quoting from the manual):
This event will only trigger a workflow run if the workflow file is on the default branch.
This means that the "Test" workflow will run on/with the code from the default branch (usually main or master).
There is a workaround for this...
Every actions is run with a set of contexts. The github context holds information about the event that triggered the workflow. This includes the branch that the event was originally triggered from/for: github.event.workflow_run.head_branch.
This can be used to check out the origination branch in the action, using the actions/checkout action provided by GitHub.
To do this, the Yaml would be:
---
name: Test
on:
workflow_run:
workflows:
- Preflight
types:
- completed
jobs:
test-job:
name: Test Step
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
with:
ref: ${{ github.event.workflow_run.head_branch }}
- run: git branch
- run: env
Currently I have the following code:
name: Build-All
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build-linux-64:
name: ${{ matrix.config.name }} Build
runs-on: ${{ matrix.config.os }}
strategy:
fail-fast: true
matrix:
config:
- os: ubuntu-latest
name: Ubuntu 64
other_linker_flags: '-m64'
arch: x86_64
output: myLib.so
steps:
- name: Make fake file
run: |
echo "hello" > ${{ github.workspace }}/test.txt
- name: Uploading Release
uses: ollydev/upload-release-action#master
with:
repo_token: XXXXXXXXX
file: '${{ github.workspace }}/test.txt'
asset_name: "test"
tag: autobuild
owner: '${{ github.repo.owner }}'
repo: 'B'
overwrite: true
and two repos: A and B.
Repo A has the above yml jobs and it is a private repo. It has all the code, compiles it, and wants to push the release to repo B which is public.
To do this, I created a new github account My-CI and I added it to both the private repo and the public repo. On that new account, I then created a Personal access token with scope: public_repo
and that's it. The code works.. but is there a way to NOT have to create a separate account just to give it access as a CI to both repos? IE: Is there a way that I can create a token on my real account that is read-only for one repo and read-write for another? OR maybe create a github app token or something that can only upload releases for the one repo (B)?
As you've implied, you can't limit the scope of a personal access token to different scopes for different repos. Theres a few ways of doing this.
Intermediate, public storage
The first is to upload the artifacts to an intermediate place, accessible from anywhere, e.g. Dropbox, Docker Hub, etc. Then you can manually trigger a github action in your public repo to pull this artifact back down and create a release from it. To manually trigger this action you could use the repository_dispatch event either using cURL / postman locally (with an access token auth bearer) or using something like https://www.actionspanel.app/ which is a github app which allows you to manually trigger github actions using repository_dispatch, with parameters so your download link would be a parameter.
Personal access token
The simplest option is still a personal access token though. Your workflow above has repo_token: XXXXXXXXX which makes me wonder if you know about github secrets? Ideally this token would be stored in a secret then accessed using ${{ secrets.BRANDONS_TOKEN }}. I would ask why you are worried about a personal access token. If you use github secrets and are careful about the 3rd party code you pass the token to (you may not want to simply pass your token to #master, for example), it should be fine.
GitHub Apps & Webhooks
GitHub apps or webhooks would be another way, you can authenticate those on a per-person basis and per-repo basis but you'd need an application running online to receive and parse the messages and its quite a big piece of work.
(Probably not) GitHub Deploy Keys
Another thing to be aware of is Github Deploy Keys, you can use these to obtain read/write access to a single repository without an account attached. You would then store this deploy key in a secret in the settings of the other repo. However, I'm not sure you can trigger releases with deploy keys - they are not bound to an account so I'm unsure who's username would be visible on the release history.
Problem: We use github actions workflow for CI and we have many github repositories. I need to be able change everything repeatable for every repository at once.
Is it possible to use in github action workflow yml file some snippet that located mb in different repository.
You can include other public and local actions in your workflow, which lets you reuse common steps. Using versioned actions with {owner}/{repo}#{ref}:
steps:
- uses: actions/setup-node#74bc508 # Reference a specific commit
- uses: actions/setup-node#v1 # Reference the major version of a release
- uses: actions/setup-node#v1.2 # Reference a minor version of a release
- uses: actions/setup-node#master # Reference a branch
..or local actions with ./path/to/dir:
jobs:
my_first_job:
steps:
- name: Check out repository
uses: actions/checkout#v2
- name: Use local my-action
uses: ./.github/actions/my-action
https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstepsuses
One way of doing this is having a central CICD / GitHub actions repository with shared workflows which are triggered on repository_dispatch events.
on:
repository_dispatch:
types:
- your_event
jobs:
job1:
name: Do something
runs-on: ubuntu-latest
env:
SOURCE_BRANCH: ${{ github.event.client_payload.source_branch }}
SOURCE_REPO: ${{ github.event.client_payload.source_repo }}
# do all your stuff
Then in each github repo you write a small workflow file which outlines the triggers for the local repo, pushing to master / opening a PR etc. That action simply dispatches a repository_dispatch event to your central CICD repo with the repo and branchname it came from.
name: Trigger external CICD
on:
push:
branches:
- master
jobs:
trigger_cicd:
name: Trigger external CICD
runs-on: ubuntu-latest
steps:
- name: Send repository_dispatch event
uses: peter-evans/repository-dispatch#v1
with:
token: ${{ secrets.CICD_GITHUB_TOKEN }}
repository: yourorg/centralcicdrepo
event-type: ${{ env.EVENT_TYPE }}
client-payload: '{"source_branch": "${{ github.ref }}", "source_repo": "${{ github.repository }}" }'
One gotcha is that you need an access token to talk between repos, in the above example it's added as a secret called CICD_GITHUB_TOKEN. The easiest is to just use your own account but this will label all your central CICD runs as 'triggered by you'. You can also create a bot account or you can have each developer add their access tokens as secrets then map the right author to the right access token.
There is currently (Feb. 3, 2021) no supported method for reusing workflows or snippets from a centralized repository. There are hacks, as Michael Parker has cleverly demonstrated, but these come with significant downsides (eg. observability, opacity, etc.).
I've written this blog post that describes the problem you have in more detail, along with an open-source solution.
––
Similar topics:
DRYing GH Actions workflows
External workflow configuration
Bringing this issue to GH's attention:
Raise this issue with GH
GH Roadmap item
I have a file in a GitHub repository that needs updating occasionally by running a command.
As part of a GitHub Workflows, I want to have a bot running a command, and seeing if it creates a diff on the repo, and if so, make a pull request to the repository automatically.
I have a suspicion that the GitHub Workflows can help me do that as GitHub now lets people run arbitrary containers ("Actions") that do stuff like builds in a repository. I see some official automation workflows that let you "label" and "comment" issues etc here: https://github.com/actions/starter-workflows/tree/master/automation
If I wanted to run an arbitrary command and make a PR to the repository, which GitHub Actions should I be looking at instead of reinventing my own Actions? Any pointers are appreciated.
I made a GitHub Action that I think will help you with this use case.
https://github.com/peter-evans/create-pull-request
create-pull-request action needs to be run in conjunction with other actions or steps that modify or add files to your repository. The changes will be automatically committed to a new branch and a pull request created.
Here is an example that sets most of the main inputs.
on:
repository_dispatch:
types: [create-pull-request]
name: Create Pull Request
jobs:
createPullRequest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Create report file
run: date +%s > report.txt
- name: Create Pull Request
uses: peter-evans/create-pull-request#v3
with:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: Add report file
committer: Peter Evans <peter-evans#users.noreply.github.com>
body: |
New report
- Contains *today's* date
- Auto-generated by [create-pull-request][1]
[1]: https://github.com/peter-evans/create-pull-request
title: '[Example] Add report file'
labels: report, automated pr
assignees: peter-evans
reviewers: peter-evans
milestone: 1
branch: example-patches
To make it bot-like you can trigger the workflow periodically.
on:
schedule:
- cron: '*/5 * * * *'
Alternatively, you can set the workflow to trigger via webhook, as in the example above.
on:
repository_dispatch:
types: [create-pull-request]
To trigger the workflow call the following. [username] is a GitHub username. [token] is a repo scoped token. [repository] is the name of the repository the workflow resides in.
curl -XPOST -u "[username]:[token]" -H "Accept: application/vnd.github.everest-preview+json" -H "Content-Type: application/json" https://api.github.com/repos/[username]/[repository]/dispatches --data '{"event_type": "create-pull-request"}'
For further examples check out the documentation here.