I am trying to deploy to AWS using Github actions. The only problem is, that I have a main repo and frontend and backend submodules inside it.
This is the script I am using for deploy:
name: Deploy
on:
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout source code
uses: actions/checkout#v2
- name: Generate deployment package
run: git submodule update --init --recursive
run: zip -r deploy.zip . -x '*.git*'
- name: Get timestamp
uses: gerred/actions/current-time#master
id: current-time
- name: Run string replace
uses: frabert/replace-string-action#master
id: format-time
with:
pattern: '[:\.]+'
string: "${{ steps.current-time.outputs.time }}"
replace-with: '-'
flags: 'g'
- name: Deploy to EB
uses: einaregilsson/beanstalk-deploy#v18
with:
aws_access_key: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws_secret_key: ${{ secrets.AWS_SECRET_KEY }}
application_name: test-stage
environment_name: Testenv-env
version_label: "${{ steps.format-time.outputs.replaced }}"
region: eu-center-1
deployment_package: deploy.zip
The problem is while it is creating a zip. It does not include submodules. Without submodules the project almost contains nothing. Is it possible somehow to iclude them? Or do you have any better solutions for this?
Consulting the actions/checkout documentation, there is a submodules argument (default value false) that controls whether the checkout includes submodules. So, you likely want
steps:
- name: Checkout source code
uses: actions/checkout#v2
with:
submodules: true
Related
Hello everyone I am currently writing a workflow to auto merge when a pull request is made but I am stuck at an error telling me my token is not set more specifically: 2023-02-19T02:09:08.581Z ERROR environment variable GITHUB_TOKEN not set!. I have set all my tokens in my repo and settings tab. Any help would be appreciated.
name: CI/CD
on:
pull_request:
branches: [ master ]
jobs:
super-linter:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v2
- name: Super-Linter
uses: github/super-linter#v4.10.1
with:
files: ${{ join(github.event.pull_request.changed_files, ',') }}
Merge:
runs-on: ubuntu-latest
needs: super-linter
steps:
- name: Checkout Code
uses: actions/checkout#v2
- name: Merge pull requests
uses: pascalgn/automerge-action#v0.14.1
with:
GITHUB_TOKEN: ${{ secrets.TOKEN }}
deploy:
runs-on: self-hosted
needs: Merge
steps:
#- uses: actions/checkout#v2 #this is used for if you want to push all source code into runner
- name: update code base
working-directory: /test_pipe/www/html
run: sudo git pull origin master
- name: restart
working-directory: /test_pipe/www/html
run: sudo systemctl restart nginx
image of error
pascalgn/automerge-action accepts GITHUB_TOKEN as an env variable, not as an argument. So it should be:
- name: Merge pull requests
uses: pascalgn/automerge-action#v0.14.1
env:
GITHUB_TOKEN: ${{ secrets.TOKEN }}
Refer to the documentation: https://github.com/pascalgn/automerge-action#usage
I am trying to implement a github actions workflow with a job which will plan and apply my terraform code changes only for directory where changes were made. The problem I am currently facing is that I can't figure out how to switch directories so that terraform plan is executed from a directory where code has been updated/changed.
I have a monorepo setup which is as follow:
repo
tf-folder-1
tf-folder-2
tf-folder-3
Each folder contains an independent terraform configuration. So, for example I would like run a workflow only when files change inside tf-folder-1. Such workflow needs to switch to working directory which is tf-folder-1 and then run terraform plan/apply.
jobs:
terraform:
name: "Terraform"
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./tf-folder-1
steps:
- name: Checkout
uses: actions/checkout#v3
- name: Configure AWS credentials from Test account
uses: aws-actions/configure-aws-credentials#v1
with:
role-to-assume: arn:aws:iam::000000000000000:role/deploy-role
aws-region: eu-west-2
- name: Setup Terraform
uses: hashicorp/setup-terraform#v2
...
So far, I have the above terraform job but it only runs for statically defined working-directory. It doesn't work with a use case where it should run the workflow when changes happen within specific folder. Can someone advise how to fix this pipeline?
Thanks
GitHub Actions has path filtering you can take advantage of when you are working with workflows that are triggered off a push or push_request event.
For example say you have a monorepo with the directories, tf_1, tf_2, and tf_3. You can do something like below for when changes occur to the directory tf_1.
name: Demonstrate GitHub Actions on Monorepo
on:
push:
branches:
- master
paths:
- 'tf_1/**'
defaults:
run:
working-directory: tf_1
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
For more details on path filtering, please refer to the GitHub Actions syntax documentation.
You can use a GitHub action that outputs the directories where the files have changed/modified, for example, this one: Changed-files or even perform the calculation with a shell step using git diff.
If you use the GHA suggested you can set the input dir_names to true, which would output unique changed directories instead of filenames, based on the results of that you can change the directory to run your Terraform operations.
Here is the solution to run multiple jobs based on number of directories that have been update.
In the below snippet you can see directories job which will check which directories have been updated, later it output an array or switch which is then used in matrix strategy for terraform job.
jobs:
directories:
name: "Directory-changes"
runs-on: ubuntu-latest
steps:
- uses: theappnest/terraform-monorepo-action#master
id: directories
with:
ignore: |
aws/**/policies
aws/**/templates
aws/**/scripts
- run: echo ${{ steps.directories.outputs.modules }}
outputs:
dirs: ${{ steps.directories.outputs.modules }}
terraform:
name: "Terraform"
runs-on: ubuntu-latest
needs: directories
strategy:
matrix:
directories: ${{ fromJson(needs.directories.outputs.dirs) }}
defaults:
run:
working-directory: ${{ matrix.directories }}
steps:
- name: Checkout
uses: actions/checkout#v3
- name: Setup Terraform
uses: hashicorp/setup-terraform#v2
with:
cli_config_credentials_token: ${{ secrets.TF_CLOUD_TEAM_API_TOKEN_PREPROD }}
- name: Terraform Format
id: fmt
run: terraform fmt -check
- name: Terraform Init
id: init
run: terraform init
- name: Terraform Validate
id: validate
run: terraform validate -no-color
- name: Terraform Plan
id: plan
if: github.event_name == 'pull_request'
run: terraform plan -no-color -input=false
continue-on-error: true
I moved some common ci work into its own repo for linting, static checks etc. Multiple repos will then use this to avoid duplication. Issue I am having is, obviously the checks need to be carried out on the repo that invokes the workflow. How is this made possible? When the common workflow is executed it has no access to the contents of the initial repo. It only checks out itself.
Example source repo:
name: Perform Pre Build Check
on:
push:
workflow_dispatch:
jobs:
checks:
uses: <org>/<common-repo>/.github/workflows/checks.yml#main
Common workflow:
name: Perform Pre-Build Checks
on:
workflow_call:
jobs:
formatting-check:
name: Formatting Check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Run linting check
run: xxxxxx
- name: Install cppcheck
run: sudo apt-get -y install cppcheck
- name: Run cppcheck
run: xxxxx
continue-on-error: true
This is what I ended up doing. Not sure if this was right approach but it worked. Basically just passing in the repo & ref name from current repo that triggered the workflow,
Example source repo:
name: Perform Pre Build Check
on:
push:
workflow_dispatch:
jobs:
checks:
uses: <org>/<common-repo>/.github/workflows/checks.yml#main
with:
repo-name: ${{ github.GITHUB_REPOSITORY }}
ref-name: ${{ github.GITHUB_REF_NAME }}
Common workflow:
name: Perform Pre-Build Checks
on:
workflow_call:
inputs:
repo-name:
required: true
type: string
ref-name:
required: true
type: string
jobs:
formatting-check:
name: Formatting Check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
with:
repository: ${{inputs.repo-name}}
ref: ${{inputs.ref-name}}
- name: Run linting check
run: xxxxxx
- name: Install cppcheck
run: sudo apt-get -y install cppcheck
- name: Run cppcheck
run: xxxxx
continue-on-error: true
I'm trying to mirror a public repo that I don't own, and more importantly mirror their pushes (to trigger another action).
Right now, the sync actions I have seen seem to copy paste a repo into a repo I own, but that repo's pushes don't trigger actions.
Is there a way to do this ?
I don't know the foreign repo's owner. I'm aware that the owner could send a dispatch event, but I want a solution that doesn't rely on the goodwill of someone.
Basically, I want this to happen:
My repo synchronizes with a foreign one every hour, and if there was an update in the last hour then another action gets triggered.
Is there a way to do this ?
For people with the same problem, I found a way through a cron schedule:
Create an empty text file in your repo
Check the original repo's commit id
If that id is different from the one in the text file, then trigger a sync Action (and optionally any other Action), and then add the commit id in a small text file
Otherwise, do nothing
Repeat from 2 on a cron schedule
Updates on the foreign repo will trigger on the cron schedule.
It's not exactly what I wanted, but close enough.
Edit:
Here is a complete .yml file example from my repo, decommented to make it shorter
name: Sync+build+push TTRSS
on:
schedule:
- cron: '0 0 * * *'
workflow_dispatch:
jobs:
get_commits:
runs-on: ubuntu-latest
outputs:
LOCAL: ${{ steps.commits.outputs.SETLOCAL }}
REMOTE: ${{ steps.commits.outputs.SETREMOTE }}
steps:
- name: Checkout
uses: actions/checkout#v2
with:
ref: 'TTRSS-docker'
- name: set local and remote latest commit as environment variables
id: commits
run: |
echo "::set-output name=SETREMOTE::$(git ls-remote https://git.tt-rss.org/fox/ttrss-docker-compose.git HEAD | awk '{ print $1 }')"
echo "::set-output name=SETLOCAL::$(cat last_sync_with_original_repo_commit_id)"
repo_sync:
needs: [get_commits]
runs-on: ubuntu-latest
if: needs.get_commits.outputs.LOCAL != needs.get_commits.outputs.REMOTE
steps:
- name: repo-sync
uses: wei/git-sync#v3
with:
source_repo: "https://git.tt-rss.org/fox/ttrss-docker-compose.git"
source_branch: "master"
destination_repo: "git#github.com:schklom/Mirror-workflows.git"
destination_branch: "TTRSS-docker"
ssh_private_key: ${{ secrets.GITSYNCACTION }}
- name: Checkout
uses: actions/checkout#v2
with:
ref: 'TTRSS-docker'
- name: get most recent commit id on original repo, for next comparison on sync
run: git ls-remote https://git.tt-rss.org/fox/ttrss-docker-compose.git HEAD | awk '{ print $1 }' > last_sync_with_original_repo_commit_id
- name: Commit and push the change
uses: stefanzweifel/git-auto-commit-action#v4
with:
commit_message: Add last_sync_with_original_repo_commit_id
build_push:
needs: [repo_sync]
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
with:
ref: 'TTRSS-docker'
- name: Set up QEMU
uses: docker/setup-qemu-action#v1
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action#v1
- name: Login to Docker Hub
uses: docker/login-action#v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push TTRSS app
uses: docker/build-push-action#v2
with:
context: ./app
file: ./app/Dockerfile
platforms: linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64
pull: true
push: true
tags: |
schklom/ttrss-app:latest
- name: Build and push TTRSS web-nginx
uses: docker/build-push-action#v2
with:
context: ./web-nginx
file: ./web-nginx/Dockerfile
platforms: linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64
pull: true
push: true
tags: |
schklom/ttrss-web-nginx:latest
build_push_filelogging:
needs: [build_push]
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
with:
ref: 'TTRSS-docker-with-filelogging'
- name: Set up QEMU
uses: docker/setup-qemu-action#v1
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action#v1
- name: Login to Docker Hub
uses: docker/login-action#v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push TTRSS app (with file logging)
uses: docker/build-push-action#v2
with:
context: ./
file: ./Dockerfile
platforms: linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64
pull: true
push: true
tags: |
schklom/ttrss-app:with-filelogging-latest
I have a custom Jekyll site which is working fine on local.
I would like to deploy my builded site to my hosting environment. Via FTP with github actions is working fine with this: https://github.com/SamKirkland/FTP-Deploy-Action
This is the FTP workflow:
on: push
name: Publish Website
jobs:
FTP-Deploy-Action:
name: FTP-Deploy-Action
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2.1.0
with:
fetch-depth: 2
- name: FTP-Deploy-Action
uses: SamKirkland/FTP-Deploy-Action#3.1.1
with:
ftp-server: ${{ secrets.FTP_HOST }}
ftp-username: ${{ secrets.FTP_USER }}
ftp-password: ${{ secrets.FTP_PASSWORD }}
local-dir: _site
git-ftp-args: --changed-only
I tried with the _site folder and and action is run with every commit when the _site is not ignored.
So the best, If I am not comitting the _site page, that will do the GitHub server. I found this action: https://github.com/marketplace/actions/jekyll-actions
My test workflow:
on: push
name: Testing the GitHub Pages building
jobs:
jekyll:
runs-on: ubuntu-16.04
steps:
- uses: actions/checkout#v2
# Use GitHub Actions' cache to shorten build times and decrease load on servers
- uses: actions/cache#v1
with:
path: vendor/bundle
key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
restore-keys: |
${{ runner.os }}-gems-
# Standard usage
- uses: humarci/jekyll-action#1.0.0
# FTP maybe from here
This is what I currently use, which I found from The World According to Mike Blog.
This uses ncftp which allows you to upload files via ftp easily.
name: Build & Upload Site
# Run on pushes to the master branch
on:
push:
branches:
- master
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-ruby#v1
with:
ruby-version: '2.7'
# Install the gems in the gemfile & install ncftp
- name: Setup Environment.
run: |
bundle install
sudo apt-get install -y ncftp
# Build the site
- name: Build Site with Jekyll.
run: JEKYLL_ENV=production bundle exec jekyll build
# Looks kind of complicated but just uploads the content of _site folder to the ftp server. It does not upload the _site folder itself.
- name: Upload site to FTP.
env:
ftp_location: ${{ secrets.FTP_LOCATION }} # Pass in required secrets.
ftp_username: ${{ secrets.FTP_USERNAME }}
ftp_password: ${{ secrets.FTP_PASSWORD }}
run: |
ncftpput -R -v -u "$ftp_username" -p "$ftp_password" $ftp_location / _site/*