I am working on a GitHub Actions pipeline where I am creating a terraform plan and then after downloading and reviewing the plan in a file authentication the apply stage. Everything is working smoothly, I get a plan that I am then saving as a tt file using the 'out' flag, but I am not able to figure out how to download the plan file from the runner to my local machine or even save it as an artifact. Please help me out if there is a workaround.
name: 'Terraform PR'
on:
push:
branches:
- main
pull_request:
jobs:
terraform:
name: 'Terraform'
runs-on: ubuntu-latest
defaults:
run:
working-directory: infrastructure/env/dev-slb-alpha/dev
permissions:
id-token: write
contents: write
steps:
- name: Clone Repository (Latest)
uses: actions/checkout#v2
if: github.event.inputs.git-ref != ''
- name: Clone Repository (Custom Ref)
uses: actions/checkout#master
if: github.event.inputs.git-ref == ''
with:
ref: ${{ github.event.inputs.git-ref }}
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials#master
with:
role-to-assume: arn:aws:iam::262267462662:role/slb-dev-github-actions
aws-region: us-east-1
# role-session-name: GithubActionsSession
- name: Setup Terraform
uses: hashicorp/setup-terraform#v1
with:
terraform_version: 1.1.2
- name: Terraform Format
id: fmt
run: terraform fmt -check
- name: Terraform Init
id: init
run: |
# cd infrastructure/env/dev-slb-alpha/dev
terraform init
- name: Terraform Validate
id: validate
run: terraform validate -no-color
- name: Terraform Plan
id: plan
if: github.event_name == 'pull_request'
continue-on-error: true
run: |
# cd infrastructure/env/dev-slb-alpha/dev
touch tfplan.txt
# terraform force-unlock -force d5f2d86a-e0f6-222f-db3f-2c1d792ed528
# terraform force-unlock -force QOCDA86JVO02CCFV3SB010RGP3VV4KQNSO5AEMVJF66Q9ASUAAJG
terraform plan -lock=false -input=false -out=tfplan.txt
readlink -f tfplan.txt
- name: terraform plan upload
uses: actions/upload-artifact#v2
with:
name: plan
path: tfplan.txt
retention-days: 5
- uses: actions/download-artifact#v3
with:
name: my-plan
path: tfplan.txt
- name: Terraform Apply
id: apply
if: github.event_name == 'pull_request'
run: |
cd infrastructure/env/dev-slb-alpha/dev
terraform force-unlock -force 8de3f689-282e-12fd-72b2-cdd27f94e4da
terraform apply
Related
I have a GitHub action code with terraform and ECR, ECS now I have two branch master and feature and when I created Pull-request for feature to master
then only my terraform plan code will run but when i create a Pull-request and merge to master then my GitHub action running but that part is skipped i am not sure why it is happing please find the below attached code
---
name: "workflow"
on:
# Triggers the workflow on push or pull request events but only for the "master" branch
push:
branches: [ "master" ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
jobs:
cd:
name: "Deployment"
runs-on: "ubuntu-latest"
#if: startsWith(github.ref, 'refs/tags/')
steps:
- name: "Checkout Code"
uses: "actions/checkout#v2"
- name: Set tag
id: vars
run: echo "::set-output name=tag::${GITHUB_REF#refs/*/}"
- name: Configure AWS credential
uses: aws-actions/configure-aws-credentials#v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_REGION }}
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login#v1
- name: Build, tag, and push image to Amazon ECR
id: build-image
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: my_ecr_repi
IMAGE_TAG: ${{ github.event.head_commit.message }}
run: |
docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"
- name: Setup Terraform
uses: hashicorp/setup-terraform#v1
with:
cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}
- name: Terraform Init
run: |
cd terraform_with_ALB
terraform init
- name: Terraform Format
id: fmt
run: |
cd terraform_with_ALB
terraform fmt -check
- name: Terraform Validate
id: validate
run: |
cd terraform_with_ALB
terraform validate -no-color
- name: Terraform Plan
id: plan
if: github.event_name == 'pull_request'
run: |
cd terraform_with_ALB
terraform plan -no-color -input=false
continue-on-error: true
till terraform valiate it wokring fine after that it skip terraform plan part
you are missing the pull_request element in the on section.
on:
# Triggers the workflow on push or pull request events but only for the "master" branch
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
I am using a reusable workflow, and the calling workflow does not report a failure if the reused workflow fails.
If I were to call the workflow directly, a failure would occur and get reported which is the expected behaviour.
What needs to be done to be able to report a failure in the workflow?
Calling Workflow
name: Deploy to dev
on:
push:
branches:
- 'main'
permissions:
id-token: write
contents: read
pull-requests: write
jobs:
call-workflow-passing-data:
uses: ./.github/workflows/deploy-to-env.yml
with:
environment: dev
secrets: inherit
continue-on-error: false
Reused Workflow
name: Deploy to Environment
on:
workflow_call:
inputs:
environment:
required: true
type: string
permissions:
id-token: write
contents: read
pull-requests: write
jobs:
validate:
runs-on:
- my-custom-runner
name: Apply Terraform
environment:
name: ${{ inputs.environment }}
env:
TF_VAR_environment: dev
steps:
- name: Checkout this repo
uses: actions/checkout#v3
- uses: actions/setup-node#v3
with:
node-version: 14
- uses: hashicorp/setup-terraform#v2
with:
terraform_version: 1.2.3
- name: Terraform fmt
id: fmt
run: terraform fmt -check
continue-on-error: false
- name: Terraform Init
id: init
run: terraform -chdir=terraform/src init
- name: Terraform Plan
id: plan
run: terraform plan
continue-on-error: false
- name: Terraform Apply
id: apply
run: terraform apply
continue-on-error: false
- name: Terraform outputs
id: outputs
run: terraform output -json
continue-on-error: true
I am trying to implement a github actions workflow with a job which will plan and apply my terraform code changes only for directory where changes were made. The problem I am currently facing is that I can't figure out how to switch directories so that terraform plan is executed from a directory where code has been updated/changed.
I have a monorepo setup which is as follow:
repo
tf-folder-1
tf-folder-2
tf-folder-3
Each folder contains an independent terraform configuration. So, for example I would like run a workflow only when files change inside tf-folder-1. Such workflow needs to switch to working directory which is tf-folder-1 and then run terraform plan/apply.
jobs:
terraform:
name: "Terraform"
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./tf-folder-1
steps:
- name: Checkout
uses: actions/checkout#v3
- name: Configure AWS credentials from Test account
uses: aws-actions/configure-aws-credentials#v1
with:
role-to-assume: arn:aws:iam::000000000000000:role/deploy-role
aws-region: eu-west-2
- name: Setup Terraform
uses: hashicorp/setup-terraform#v2
...
So far, I have the above terraform job but it only runs for statically defined working-directory. It doesn't work with a use case where it should run the workflow when changes happen within specific folder. Can someone advise how to fix this pipeline?
Thanks
GitHub Actions has path filtering you can take advantage of when you are working with workflows that are triggered off a push or push_request event.
For example say you have a monorepo with the directories, tf_1, tf_2, and tf_3. You can do something like below for when changes occur to the directory tf_1.
name: Demonstrate GitHub Actions on Monorepo
on:
push:
branches:
- master
paths:
- 'tf_1/**'
defaults:
run:
working-directory: tf_1
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
For more details on path filtering, please refer to the GitHub Actions syntax documentation.
You can use a GitHub action that outputs the directories where the files have changed/modified, for example, this one: Changed-files or even perform the calculation with a shell step using git diff.
If you use the GHA suggested you can set the input dir_names to true, which would output unique changed directories instead of filenames, based on the results of that you can change the directory to run your Terraform operations.
Here is the solution to run multiple jobs based on number of directories that have been update.
In the below snippet you can see directories job which will check which directories have been updated, later it output an array or switch which is then used in matrix strategy for terraform job.
jobs:
directories:
name: "Directory-changes"
runs-on: ubuntu-latest
steps:
- uses: theappnest/terraform-monorepo-action#master
id: directories
with:
ignore: |
aws/**/policies
aws/**/templates
aws/**/scripts
- run: echo ${{ steps.directories.outputs.modules }}
outputs:
dirs: ${{ steps.directories.outputs.modules }}
terraform:
name: "Terraform"
runs-on: ubuntu-latest
needs: directories
strategy:
matrix:
directories: ${{ fromJson(needs.directories.outputs.dirs) }}
defaults:
run:
working-directory: ${{ matrix.directories }}
steps:
- name: Checkout
uses: actions/checkout#v3
- name: Setup Terraform
uses: hashicorp/setup-terraform#v2
with:
cli_config_credentials_token: ${{ secrets.TF_CLOUD_TEAM_API_TOKEN_PREPROD }}
- name: Terraform Format
id: fmt
run: terraform fmt -check
- name: Terraform Init
id: init
run: terraform init
- name: Terraform Validate
id: validate
run: terraform validate -no-color
- name: Terraform Plan
id: plan
if: github.event_name == 'pull_request'
run: terraform plan -no-color -input=false
continue-on-error: true
I'm using GitHub Environment to deploy into my testing account before merging to my master.I have specified the environment keyword in my workflow as "testing". My workflow will be triggered on a push to test branch which will then run plan and apply to testing account. I would like to have a manual approval after plan runs so I can see the output before approving to deploy into testing account. Please how can I configure manual approval so that after plan runs i can check the plan output before approving to deploy into my test account.
name: Testing Environment
on:
push:
branches:
- test
jobs:
plan&apply:
name: "Run Terragrunt Init,Plan and Apply"
runs-on: ubuntu-20.04
environment: testing
defaults:
run:
working-directory: ${{ env.TERRAFORM_WORKING_DIR }}
steps:
- name: 'Checkout'
uses: actions/checkout#v2
- name: Setup Terraform
uses: hashicorp/setup-terraform#v1.3.2
with:
terraform_version: ${{ env.TERRAFORM_VERSION }}
terraform_wrapper: true
- name: Setup Terragrunt
uses: autero1/action-terragrunt#v1.1.0
with:
terragrunt_version: ${{ env.TERRAGRUNT_VERSION }}
- name: configure AWS credentials
uses: aws-actions/configure-aws-credentials#v1.6.1
with:
aws-region: us-east-1
role-to-assume: ${{ env.ORCHESTRATION_ROLE_ARN }}
- name: Terragrunt Init
id: init
run: terragrunt run-all init -no-color --terragrunt-non-interactive
- name: Terragrunt Plan
id: plan
run: |
terragrunt run-all plan -no-color --terragrunt-non-interactive >/dev/null -out=tfplan
- name: terragrunt Apply
id: apply
run : terragrunt run-all apply -no-color --terragrunt-non-interactive
continue-on-error:true
There are two ways to do this.
Approach 1:
In your GH actions environment settings, add reviewers. Create two jobs - Plan and Apply. Then, add "needs" in apply job. This approach also requires to upload plan output as an artifact as plan and apply are two separate jobs.
name: Testing Environment
on:
push:
branches:
- test
jobs:
plan:
name: "Run Terragrunt Plan"
runs-on: ubuntu-20.04
defaults:
run:
working-directory: ${{ env.TERRAFORM_WORKING_DIR }}
steps:
- name: 'Checkout'
uses: actions/checkout#v2
- name: Setup Terraform
uses: hashicorp/setup-terraform#v1.3.2
with:
terraform_version: ${{ env.TERRAFORM_VERSION }}
terraform_wrapper: true
- name: Setup Terragrunt
uses: autero1/action-terragrunt#v1.1.0
with:
terragrunt_version: ${{ env.TERRAGRUNT_VERSION }}
- name: configure AWS credentials
uses: aws-actions/configure-aws-credentials#v1.6.1
with:
aws-region: us-east-1
role-to-assume: ${{ env.ORCHESTRATION_ROLE_ARN }}
- name: Terragrunt Init
id: init
run: terragrunt run-all init -no-color --terragrunt-non-interactive
- name: Create Artifact Folder
shell: bash
run: |
sudo mkdir -p -m777 ${{ github.workspace }}/tfplanoutput
- name: Terragrunt Plan
id: plan
run: |
terragrunt run-all plan -no-color --terragrunt-non-interactive >/dev/null -out=${{ github.workspace }}/tfplanoutput/tf.plan
- name: Upload Artifact
uses: actions/upload-artifact#v3
with:
name: artifact
path: ${{ github.workspace }}/tfplanoutput/
if-no-files-found: error
apply:
name: "Run Terragrunt Apply"
needs: plan
runs-on: ubuntu-20.04
environment: testing
- name: 'Checkout'
uses: actions/checkout#v2
- name: Setup Terraform
uses: hashicorp/setup-terraform#v1.3.2
with:
terraform_version: ${{ env.TERRAFORM_VERSION }}
terraform_wrapper: true
- name: Setup Terragrunt
uses: autero1/action-terragrunt#v1.1.0
with:
terragrunt_version: ${{ env.TERRAGRUNT_VERSION }}
- name: configure AWS credentials
uses: aws-actions/configure-aws-credentials#v1.6.1
with:
aws-region: us-east-1
role-to-assume: ${{ env.ORCHESTRATION_ROLE_ARN }}
- name: Terragrunt Init
id: init
run: terragrunt run-all init -no-color --terragrunt-non-interactive
- name: Download Build Artifact
uses: actions/download-artifact#v3
with:
name: artifact
path: ${{ github.workspace }}/tfplanoutput
- name: terragrunt Apply
run : terragrunt run-all apply tf.plan -no-color --terragrunt-non-interactive
continue-on-error:true
Approach 2:
You can create composite actions - plan and apply same as above.
Hope this helps!!!
I would like to have checkov scan terraform plan output but I am not getting any success with that.Below is my code in terragrunt.hcl,GitHub Actions workflow and the message I got when my workflow completed.I have tried few methods to have it work but I am still unable to configure it correctly so that checkov can analyse the Json output of terraform plan.I would appreciate any help that I can get on this.Thank you for your assistance inadvance
terragrunt.hcl
terraform {
after_hook "after_hook_plan" {
commands = ["plan"]
execute = ["sh", "-c", "terraform show -json tfplan.binary > ${get_parent_terragrunt_dir()}/plan.json"]
}
}
GitHubActions Workflow
name: 'Checkov Security Scan'
on:
push:
branches:
- test
jobs:
Terraform:
name: 'Terraform'
runs-on: ubuntu-latest
defaults:
run:
working-directory: ${{ env.tf_working_dir }}
steps:
- name: 'checkout'
uses: actions/checkout#v2
- name: configure AWS credentials
uses: aws-actions/configure-aws-credentials#master
with:
aws-region: us-east-1
role-to-assume: ${{ env.dev_role_arn }}
- name: Setup Terraform
uses: hashicorp/setup-terraform#v1.3.2
with:
terraform_version: ${{ env.tf_version }}
terraform_wrapper: true
- name: Setup Terragrunt
uses: autero1/action-terragrunt#v1.1.0
with:
terragrunt_version: ${{ env.tg_version }}
- name: Init
id: init
run: |
terragrunt run-all init --terragrunt-non-interactive
- name: Plan
id: plan
run: |
terragrunt run-all plan -out=tfplan.binary -no-color --terragrunt-non-interactive
- name: 'Test Plan (Checkov)'
uses: bridgecrewio/checkov-action#master
with:
directory: ./applied/test/
quiet: false # optional: display only failed checks
framework: terraform # optional: run only on a specific infrastructure {cloudformation,terraform,kubernetes,all}
output_format: json # optional: the output format, one of: cli, json, junitxml, github_failed_only
checkov output message
{
"passed": 0,
"failed": 0,
"skipped": 0,
"parsing_errors": 0,
"resource_count": 0,
"checkov_version": "2.0.706"
I guess it doesn't support however you can try this
- name: Terraform Plan
id: plan
if: github.event_name == 'pull_request'
run: terraform plan --out tfplan.binary -no-color
continue-on-error: true
- name: Terraform Show
id: show
run: terraform show -json tfplan.binary | jq '.' > tfplan.json
- name: Set up Python 3.8
uses: actions/setup-python#v1
with:
python-version: 3.8
id: setup_py
- name: Install Checkov
id: checkov
run: |
python3 -m pip3 install --upgrade pip3
pip3 install checkov
continue-on-error: true
- name: Run Checkov
id: run_checkov
run: checkov -f tfplan.json -o sarif -s
continue-on-error: true
- name: Upload SARIF file
id: upload_sarif
uses: github/codeql-action/upload-sarif#v1
with:
sarif_file: results.sarif
category: checkov
continue-on-error: true