I have a simple string test in my GCP CloudBuild step, but it never works. The step looks like this
steps:
- id: 'branch name'
name: 'alpine'
entrypoint: 'sh'
args:
- '-c'
- |
export ENV=$BRANCH_NAME
if [ $ENV = "master" ]; then
export ENV="test-dev"
fi
echo "***********************"
echo "$BRANCH_NAME"
echo "$ENV"
echo "***********************"
CloudBuild always reports this as sh: master: unknown operand. It's a literal, obviously.
I put the same code into a little sh script and it ran fine as long as I set a value for BRANCH_NAME. CloudBuild definitely supplies a value for BRANCH_NAME and it shows up in the echo "$BRANCH_NAME" while the echo "$ENV" is always empty.
Is there a way to make this string compare work?
When you use linux env var and not substitution variables (or predefined variables), you have to escape the $ with another one
steps:
- id: 'branch name'
name: 'alpine'
entrypoint: 'sh'
args:
- '-c'
- |
export ENV=$BRANCH_NAME
if [ $$ENV = "master" ]; then
export ENV="test-dev"
fi
echo "***********************"
echo "$BRANCH_NAME"
echo "$$ENV"
echo "***********************"
Related
I have below workflow with workflow dispatcher
jobs:
deploy_infra:
name: Deploy Infra
env:
INFRA_ACCOUNT: >
${{ fromJson('{
"dev": "12345",
"preprd": "678234",
"prd": "91056"
}')[github.event.inputs.stage] }}
steps:
- name: Creating ARN
shell: bash
run: |
echo "ARN is arn:aws:iam::${{ env.INFRA_ACCOUNT }}:role/aws-github-role"
Now this give me output as for stage as "dev"
ARN is arn:aws:iam::12345
:role/aws-github-role
How to fix the line break after the account number?
You are specifying the environment variable as follows:
INFRA_ACCOUNT: >
${{ fromJson('{
"dev": "12345",
"preprd": "678234",
"prd": "91056"
}')[github.event.inputs.stage] }}
With the > operator, it adds a line break at the end.
However, this can be configured using the Block Chomping Indicator.
The default behavior ("Clipping") preserves the last line break without any trailing empty lines.
You can change this to Stripping by using >- instead of >. This results in the final line break being removed:
INFRA_ACCOUNT: >-
${{ fromJson('{
"dev": "12345",
"preprd": "678234",
"prd": "91056"
}')[github.event.inputs.stage] }}
I have CircleCI setup and running fine normally, it will helps with creating deployment for me. Today I have suddenly had an issue with the step in creating the deployment due to an error related to kubernetes.
I have the config.yml followed the doc from https://circleci.com/developer/orbs/orb/circleci/kubernetes
Here is my version of setup in the config file:
version: 2.1
orbs:
kube-orb: circleci/kubernetes#1.3.0
commands:
docker-check:
steps:
- docker/check:
docker-username: MY_USERNAME
docker-password: MY_PASS
registry: $DOCKER_REGISTRY
jobs:
create-deployment:
executor: aws-eks/python3
parameters:
cluster-name:
description: Name of the EKS cluster
type: string
steps:
- checkout
# It failed on this step
- kube-orb/delete-resource:
now: true
resource-names: my-frontend-deployment
resource-types: deployments
wait: true
Below is a copy of the error log
#!/bin/bash -eo pipefail
#!/bin/bash
RESOURCE_FILE_PATH=$(eval echo "$PARAM_RESOURCE_FILE_PATH")
RESOURCE_TYPES=$(eval echo "$PARAM_RESOURCE_TYPES")
RESOURCE_NAMES=$(eval echo "$PARAM_RESOURCE_NAMES")
LABEL_SELECTOR=$(eval echo "$PARAM_LABEL_SELECTOR")
ALL=$(eval echo "$PARAM_ALL")
CASCADE=$(eval echo "$PARAM_CASCADE")
FORCE=$(eval echo "$PARAM_FORCE")
GRACE_PERIOD=$(eval echo "$PARAM_GRACE_PERIOD")
IGNORE_NOT_FOUND=$(eval echo "$PARAM_IGNORE_NOT_FOUND")
NOW=$(eval echo "$PARAM_NOW")
WAIT=$(eval echo "$PARAM_WAIT")
NAMESPACE=$(eval echo "$PARAM_NAMESPACE")
DRY_RUN=$(eval echo "$PARAM_DRY_RUN")
KUSTOMIZE=$(eval echo "$PARAM_KUSTOMIZE")
if [ -n "${RESOURCE_FILE_PATH}" ]; then
if [ "${KUSTOMIZE}" == "1" ]; then
set -- "$#" -k
else
set -- "$#" -f
fi
set -- "$#" "${RESOURCE_FILE_PATH}"
elif [ -n "${RESOURCE_TYPES}" ]; then
set -- "$#" "${RESOURCE_TYPES}"
if [ -n "${RESOURCE_NAMES}" ]; then
set -- "$#" "${RESOURCE_NAMES}"
elif [ -n "${LABEL_SELECTOR}" ]; then
set -- "$#" -l
set -- "$#" "${LABEL_SELECTOR}"
fi
fi
if [ "${ALL}" == "true" ]; then
set -- "$#" --all=true
fi
if [ "${FORCE}" == "true" ]; then
set -- "$#" --force=true
fi
if [ "${GRACE_PERIOD}" != "-1" ]; then
set -- "$#" --grace-period="${GRACE_PERIOD}"
fi
if [ "${IGNORE_NOT_FOUND}" == "true" ]; then
set -- "$#" --ignore-not-found=true
fi
if [ "${NOW}" == "true" ]; then
set -- "$#" --now=true
fi
if [ -n "${NAMESPACE}" ]; then
set -- "$#" --namespace="${NAMESPACE}"
fi
if [ -n "${DRY_RUN}" ]; then
set -- "$#" --dry-run="${DRY_RUN}"
fi
set -- "$#" --wait="${WAIT}"
set -- "$#" --cascade="${CASCADE}"
if [ "$SHOW_EKSCTL_COMMAND" == "1" ]; then
set -x
fi
kubectl delete "$#"
if [ "$SHOW_EKSCTL_COMMAND" == "1" ]; then
set +x
fi
error: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1alpha1"
Exited with code exit status 1
CircleCI received exit code 1
Does anyone have idea what is wrong with it? Im not sure whether the issue is happening on Circle CI side or Kubernetes side.
I was facing the exact issue since yesterday morning (16 hours ago). Then taking #Gavy's advice, I simply added this in my config.yml:
steps:
- checkout
# !!! HERE !!!
- kubernetes/install-kubectl:
kubectl-version: v1.23.5
- run:
And now it works. Hope it helps.
I am following HashiCorp's learning guide on how to set up GitHub Actions and terraform. All is running great besides the step to update the PR with the Terraform Plan.
I am hitting the following error:
An error occurred trying to start process '/home/runner/runners/2.287.1/externals/node12/bin/node' with working directory '/home/runner/work/ccoe-aws-ou-scp-manage/ccoe-aws-ou-scp-manage'. Argument list too long
The code I am using is:
- uses: actions/github-script#0.9.0
if: github.event_name == 'pull_request'
env:
PLAN: "terraform\n${{ steps.plan.outputs.stdout }}"
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const output = `#### Terraform Format and Style 🖌\`${{ steps.fmt.outcome }}\`
#### Terraform Initialization ⚙️\`${{ steps.init.outcome }}\`
#### Terraform Plan 📖\`${{ steps.plan.outcome }}\`
<details><summary>Show Plan</summary>
\`\`\`${process.env.PLAN}\`\`\`
</details>
*Pusher: #${{ github.actor }}, Action: \`${{ github.event_name }}\`*`;
github.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output
})
A clear COPY/Paste from the docs: https://learn.hashicorp.com/tutorials/terraform/github-actions
I have tried with
actions/github-script version 5 and 6 and still the same problem, But when I copy paste the plan all is great. If I do not use the output variable and use some place holder text for the body all is working great. I can see that the step.plan.outputs.stdout is Ok if I print only that.
I will be happy to share more details if needed.
I also encountered a similar issue.
I seem github-script can't give to argument for too long script.
reference:
https://github.com/robburger/terraform-pr-commenter/issues/6#issuecomment-826966670
https://github.community/t/maximum-length-for-the-comment-body-in-issues-and-pr/148867
my answer:
- name: truncate terraform plan result
run: |
plan=$(cat <<'EOF'
${{ format('{0}{1}', steps.plan.outputs.stdout, steps.plan.outputs.stderr) }}
EOF
)
echo "${plan}" | grep -v 'Refreshing state' >> $GITHUB_ENV
echo "EOF" >> $GITHUB_ENV
- name: create comment from plan result
uses: actions/github-script#0.9.0
if: github.event_name == 'pull_request'
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const output = `#### Terraform Initialization ⚙️\`${{ steps.init.outcome }}\`
#### Terraform Plan 📖\`${{ steps.plan.outcome }}\`
<details><summary>Show Plan</summary>
\`\`\`\n
${ process.env.PLAN }
\`\`\`
</details>
*Pusher: #${{ github.actor }}, Action: \`${{ github.event_name }}\`, Working Directory: \`${{ inputs.TF_WORK_DIR }}\`, Workflow: \`${{ github.workflow }}\`*`;
github.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output
})```
Based on #Zambozo's hint in the comments, this worked for me great:
- name: Terraform Plan
id: plan
run: terraform plan -no-color -input=false
- name: generate random delimiter
run: echo "DELIMITER=$(uuidgen)" >> $GITHUB_ENV
- name: truncate terraform plan result
run: |
echo "PLAN<<${{ env.DELIMITER }}" >> $GITHUB_ENV
echo '[maybe truncated]' >> $GITHUB_ENV
tail --bytes=10000 <<'${{ env.DELIMITER }}' >> $GITHUB_ENV
${{ format('{0}{1}', steps.plan.outputs.stderr, steps.plan.outputs.stdout) }}
${{ env.DELIMITER }}
echo >> $GITHUB_ENV
echo "${{ env.DELIMITER }}" >> $GITHUB_ENV
- name: post plan as sticky comment
uses: marocchino/sticky-pull-request-comment#v2
with:
header: plan
message: |
#### Terraform Plan 📖\`${{ steps.plan.outcome }}\`
<details><summary>Show Plan</summary>
```
${{ env.PLAN }}
```
</details>
Notably GitHub does have an upper limit on comment size, so this only displays the last 10kB of the plan (showing the summary and warnings).
This also implements secure heredoc delimiting to avoid malicious output escaping.
Also note that the empty lines before and after the triplebacktics in the message are significant to avoid destroying the formatting.
I have the following scenario, it is simplified for the sake of brevity, but outlines my problem.
I have a 2 Job pipeline.
BreakMatrix Job: A job that runs on an AdminPool and outputs 2 variables with the following names ENV1 and ENV2. The names are important because each of them matches the name of an Environment running in a separate MachinePool VM deployment pool.
Upgrade Job: A deployment job that depends on the BreakMatrix job and that runs on a VM MachinePool, with a tag that will select ENV1 and ENV2 environments.
I am trying to pass in one of the variables to each of the corresponding Environments:
- job: BreakMatrix
pool: AdminPool
steps:
- checkout: none
- powershell: |
$result1 = #{"Hostname" = "Env1Value"}
$result2 = #{"Hostname" = "Env2Value"}
Write-Host "##vso[task.setvariable variable=ENV1;isOutput=true;]$result1"
Write-Host "##vso[task.setvariable variable=ENV2;isOutput=true;]$result2"
name: outputter
- deployment: Upgrade
dependsOn: BreakMatrix
variables:
agentName: $(Environment.ResourceName)
agentName2: $(Agent.Name)
formatted: $[ format('outputter.{0}', variables['agentName']) ]
result1: $[ dependencies.BreakMatrix.outputs[format('outputter.{0}', variables['agentName'])] ]
result2: $[ dependencies.BreakMatrix.outputs[format('outputter.{0}', variables['agentName2'])] ]
result3: $[ dependencies.BreakMatrix.outputs[format('outputter.{0}', variables['Agent.Name'])] ]
hardcode: $[ dependencies.BreakMatrix.outputs['outputter.ENV2'] ]
json: $[ convertToJson(dependencies.BreakMatrix.outputs) ]
environment:
name: MachinePool
resourceType: VirtualMachine
tags: deploy-dynamic
strategy:
rolling:
preDeploy:
steps:
- powershell: |
echo 'Predeploy'
echo "agentName: $(agentName)"
echo "agentName2: $(agentName2)"
echo "env: $(Environment.ResourceName)"
echo "formatted: $(formatted)"
echo "harcode: $(harcode)"
echo "result1: $(result1)"
echo "result2: $(result2)"
echo "result3: $(result3)"
echo "json: $(json)"
deploy:
steps:
- powershell: |
echo 'Deploy'
Output for ENV2 pre-deploy step:
Predeploy
agentName: ENV2
agentName2: ENV2
env: ENV2
formatted: outputter.ENV2
hardcode: {
Hostname:Env2Value}
result1:
result2:
result3:
json: {
outputter.ENV2:
\"Hostname\":\"Env2Value\"
}
If i try to use a predefined variable in a dependency expression they don't seem to properly resolve, but if i just simply map them to a variable / format them they work.
Note: The environment names are actually dynamically calculated before these jobs run so I cannot use parameters or static/compile time variables.
Any suggestions on how to pass in only the relevant variable to each of the environments ?
I got confirmation on the developer community that this is not possible.
Relevant part:
But I am afraid that the requirement couldn’t be achieved.
The reason is that the dependencies.xx.outputs expression can’t read
the variable(variables['Agent.Name']) and format expression value
defined in the YAML pipeline.
It now only supports hard-coding the name of the variable to get the
corresponding job variable value.
I want to create pipeline for building Flutter(Android/iOS) application on Bitbucket easily. Unfortunately, I have not found any article or guide. Is it possible?
Web
image: cirrusci/flutter
pipelines:
branches:
dev:
- step:
name: Run unit tests
script:
- flutter test
- step:
name: Build web
script:
- flutter build web
- step:
name: Deploy to Firebase
deployment: production
script:
- pipe: atlassian/firebase-deploy:1.1.0
variables:
PROJECT_ID: $PROJECT_ID
KEY_FILE: $FIREBASE_KEY_FILE
DEBUG: 'true'
BUILD_DIR: $BUILD_DIR/web/
main:
- step:
name: Run unit tests
script:
- flutter pub get
- flutter test
Android
workflows:
internal-release:
name: Internal Release
environment:
vars:
databaseURL: https://my-app.firebaseio.com
apiKey: asdf
googleAppID: 1:123:android:456
AD_MOB_AD_ID: ca-app-pub-123/456
AD_MOB_APP_ID: ca-app-pub-123~456
FCI_KEYSTORE_PATH: $FCI_BUILD_DIR/android/app/key.jks
FCI_KEYSTORE: Encrypted(xxx)
FCI_KEYSTORE_PASSWORD: Encrypted(xxx==)
FCI_KEY_PASSWORD: Encrypted(xxx==)
FCI_KEY_ALIAS: Encrypted(xxx==)
GCLOUD_SERVICE_ACCOUNT_CREDENTIALS: Encrypted(xxx=)
flutter: stable
xcode: latest
cocoapods: default
triggering:
events:
- push
branch_patterns:
- pattern: '*'
include: true
source: true
cancel_previous_builds: true
when:
changeset:
includes:
- '.'
excludes:
- '**/*.yaml'
- '**/*.yml'
- '**/*.json'
- '**/*.md'
- '**/*.db'
scripts:
- |
# set up key.properties
echo $FCI_KEYSTORE | base64 --decode > $FCI_KEYSTORE_PATH
cat >> "$FCI_BUILD_DIR/android/key.properties" <<EOF
storePassword=$FCI_KEYSTORE_PASSWORD
keyPassword=$FCI_KEY_PASSWORD
keyAlias=$FCI_KEY_ALIAS
storeFile=$FCI_KEYSTORE_PATH
EOF
- |
# set up local properties
echo "flutter.sdk=$HOME/programs/flutter" > "$FCI_BUILD_DIR/android/local.properties"
- cd . && flutter packages pub get
# - cd . && flutter analyze
# - cd . && flutter test
- |
#!/usr/bin/env sh
if ! command -v yq &> /dev/null
then
HOMEBREW_NO_AUTO_UPDATE=1 brew install yq
exit
fi
- cd . && flutter build appbundle --release --build-name=$(yq e .version pubspec.yaml|awk -F+ '{print $1}') --build-number=$PROJECT_BUILD_NUMBER
- |
# generate signed universal apk with user specified keys
android-app-bundle build-universal-apk \
--bundle build/**/outputs/**/*.aab \
--ks $FCI_KEYSTORE_PATH \
--ks-pass $FCI_KEYSTORE_PASSWORD \
--ks-key-alias $FCI_KEY_ALIAS \
--key-pass $FCI_KEY_PASSWORD
artifacts:
- build/**/outputs/**/*.apk
- build/**/outputs/**/*.aab
- build/**/outputs/**/mapping.txt
- flutter_drive.log
publishing:
google_play:
credentials: Encrypted(xxx=)
track: internal
in_app_update_priority: 0