I have written a bash script it process some data and puts in one file. My intention is to give slack alert if there is content in that file if not it should not give the alert. Is there a way to do it? In Concourse
You should take advantage of the Concourse community's open source resource types. There's a list here. There is a slack resource listed on that page, but I use the one here (not included in the list above because it has not been added by the authors) https://github.com/cloudfoundry-community/slack-notification-resource.
That will give you the ability to add a put step in your job plan to send a slack resource. As for the logic of your original ask, you can use try and on_success. Your task might look something like this:
- try:
task: do-a-thing
config:
platform: linux
image_resource:
type: registry-image
source:
repository: YOUR_TASK_IMAGE
tag: latest
inputs:
- name: some-input
params:
FILE: some-file
run:
path: /bin/sh
args:
- -ec
- |
[ ! -z `cat some-input/${FILE}` ]
on_success:
put: slack
params:
<your slack resource put params go here>
The on_success part will run if the code defined in the task's run section returns 0. The script listed there just checks to see if there are more than zero bytes in the file. Because the task is wrapped in a try step, regardless of whether or not the task succeeds (and hence, sends you a message), the step will succeed and move to the next step in the plan.
Related
I have configured a manual workflow and it runs OK, but once I update/edit it and commit it to the same branch, the changes do not affect it. I mean the action still runs but uses the old version of the workflow file. is there any step I need to do?
Steps I followed for editing the workflow file:
https://docs.github.com/en/actions/learn-github-actions/finding-and-customizing-actions#browsing-marketplace-actions-in-the-workflow-editor
Here is workflow file details, just in case
The original:
# This is a basic workflow to help you get started with Actions
name: CI
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the "development" branch
release:
types: [created]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout#v3
# Runs a single command using the runners shell
- name: Run a one-line script
run: echo Hello, world Nikzad!, 3
Note: Let's say I just replace the last line with the bellow line, but my output still says Hello, world Nikzad!, 3 where it should say Hello, world Nikzad!, 4.
run: echo Hello, world Nikzad!, 4
I found my problem. actually, when we are creating a new release, we are providing a tag name for that release, and when we create that tag name and push it to the repo, the version of the workflow at that point is what matters, so when I want to edit or update my workflow, I do the followings:
Edit the workflow (Make any change you want, eg change the title or add echo)
Commit and push workflow changes
Create a new tag and push
From the GitHub panel, create a new release based on the new tag
Now I see the action is running based on the new workflow
Note: Maybe it is more about learning of the process, but I did not find it straight forward anywhere, I hope it helps someone.
TL; DR: Running jobs a,b in parallel. If a fails, stop the execution of b, while it's still running.
My company uses GitHub Actions to deploy our code.
The first step in our deployment is building dockers and pushing them to DockerHub.
We wrote a test for our code, which we want to run in parallel with building the dockers.
Both of these are separate jobs, and we have a few more jobs depending on the success of the first two.
Right now, if the test job fails, the other job continues to run, but obviously, the next one won't run, because the test job failed.
What I would like to do is cancel the docker building job while it's running, if the test failed.
Is that possible? After searching the web, StackOverflow and the GitHub Actions page, I haven't found a way to do that.
Thanks!
You can specify the needs option and refer to the job name. See: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idneeds
An example could be something like:
jobs:
build:
...
deploy:
needs: build
...
You can use the Cancel this build action.
The basic idea is to add it as a final step in each of your jobs that you want to cause a short-circuit in case of failure:
jobs
job_a:
steps:
- run: |
echo 'I am failing'
exit 1
- name: Cancelling parallel jobs
if: failure()
uses: andymckay/cancel-action#0.2
job_b:
steps:
- run: echo 'long task'
This will basically cancel job_b or any other in the same workflow whenever job_a fails.
Since you are working on an enterprise project, I would prefer to avoid using unverified actions from public repositories no matter how many stars they have. I think you can add a step to the end of each job a, b. This step will only run if previous steps failed. If it is failed then it will send a cancel-workflow api call.
- if: failure()
name: Check Job Status
uses: actions/github-script#v6
env:
RUN_ID: ${{ github.run_id }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
script: |
const runId = process.env.RUN_ID
const [owner, repo] = process.env.GITHUB_REPOSITORY.split("/");
const resp = await github.rest.actions.cancelWorkflowRun({
owner,
repo,
runId
})
Note: You may need to add another custom github_pat since this api-call may require higher permissions than default actions. I also suggest you to take a look at this post , I found it quite useful.
My use case is I want to have a unique version number for artifacts per each build/run. With current tools like CircleCI, Travis, etc. there is a build number available which is basically a counter that always goes up. So, I can create version strings like 0.1.0-27. This counter is increased each time even for the same commit.
How can I do something similar with GitHub Actions? Github actions only offer GITHUB_SHA and GITHUB_REF.
GitHub Actions now has a unique number and ID for a run/build in the github context.
github.run_id : A unique number for each workflow run within a repository. This number does not change if you re-run the workflow run.
github.run_number : A unique number for each run of a particular workflow in a repository. This number begins at 1 for the workflow's first run, and increments with each new run. This number does not change if you re-run the workflow run.
github.run_attempt : A unique number for each attempt of a particular workflow run in a repository. This number begins at 1 for the workflow run's first attempt, and increments with each re-run.
ref: https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#github-context
You can reference them in workflows like this:
- name: Output Run ID
run: echo ${{ github.run_id }}
- name: Output Run Number
run: echo ${{ github.run_number }}
- name: Output Run Attempt
run: echo ${{ github.run_attempt }}
I had the same problem and have just created an action to generate sequential build numbers. Use it like
- uses: einaregilsson/build-number#v1
with:
token: ${{secrets.github_token}}
In steps after that you'll have a BUILD_NUMBER environment variable. See more info about using the same build number for different jobs and more at https://github.com/einaregilsson/build-number/
UPDATE: There is now a $GITHUB_RUN_NUMBER variable built into GitHub Actions, so this approach is not needed anymore.
If you want a constant integer increment (1,2,3,4,5), I haven't found anything in the docs that you could use as such increment which is aware of how many times that particular action ran. There's two solutions I can think of:
Maintaining state on the repo: for example with a count.build file that uses the workflow ID and you increment it on build. This is my least favourite solution of the two because it adds other complexities, like it will itself trigger a push event. You could store this file somewhere else like S3 or in a Gist.
Using the Date: if you're not worried about sequence on the integer increment you could just use the current data and time, for example 0.1.0-201903031310 for Today at 13:10.
Regardless if you have Actions Beta Access, I would definitely feed this back to GitHub.
Hope it helps.
You can use GitVersion to generate incrementing versions from tags in Git. The PR at https://github.com/GitTools/GitVersion/pull/1787 has some details, but basically you can define this job:
- uses: actions/checkout#v1
- name: Get Git Version
uses: docker://gittools/gitversion:5.0.2-beta1-34-linux-debian-9-netcoreapp2.1
with:
args: /github/workspace /nofetch /exec /bin/sh /execargs "-c \"echo $GitVersion_MajorMinorPatch > /github/workspace/version.txt\""
I would like to tag my git commits as they are deployed to the various environments in my concourse pipeline with the name of the environment. For example, in my UAT deployment job, I would like to do something like:
- put: master-resource <-- a git resource
params:
repository: master <-- the resource local directory
tag: 'uat'
force: true <-- replace the tag, if it already exists
tag_only: true
This would seem like a common -or at least simple, thing to do however the value of the 'tag' parameter can only be the path to a file -there is no option to pass a constant/literal value.
I see two possible solutions but none of them seems 'simple' enough:
Create a file myself, but to do that (ideally?) I wish there were some kind of file resource that I could use to create the file.
The last alternative would be to create a custom task, and even there I was struggling to find a way to pass the name of the tag as a parameter.
Any suggestions on what would be the best way to accomplish my goal in the simplest way, or alternatively how to implement options 1 or 2?
Thanks!
The reason that tag takes in a file is so that you can dynamically set the tag of the commit based on information you imply during the course of the pipeline.
So, the best way I can see to do something like this would be workflow #2 that you described above.
So you would want something like this:
- task: generate-git-tag
params:
TAG: {{some-passed-in-tag}}
config:
platform: linux
image_resource:
type: docker-image
source:
repository: ruby
outputs:
- name: tag-file
params:
TAG:
run:
path: /bin/bash
args:
- -c
- |
echo "${TAG}" >> tag-file/tag.txt
- put: master-resource <-- a git resource
params:
repository: master <-- the resource local directory
tag: tag-file/tag.txt
force: true <-- replace the tag, if it already exists
tag_only: true
What's the best way to pass parameters between concourse tasks and jobs? For example; if my first task generates a unique ID, what would be the best way to pass that ID to the next job or task?
If you are just passing between tasks within the same job, you can use artifacts (https://concourse-ci.org/running-tasks.html#outputs) and if you are passing between jobs, you can use resources (like putting it in git or s3). For example, if you are passing between tasks, you can have a task file
---
platform: linux
image_resource: # ...
outputs:
- name: unique-id
run:
path: project-src/ci/fill-in-output.sh
And the script fill-in-output.sh will put the file that contains the unique ID into path unique-id/. With that, you can have another task that takes the unique-id output as an input (https://concourse-ci.org/running-tasks.html#inputs) and use that unique id file.
Additionally to tasks resources will place files automagically for you in their working directory.
For example I have a pipeline job as follows
jobs:
- name: build
plan:
- get: git-some-repo
- put: push-some-image
params:
build: git-some-repo/the-image
- task: Use-the-image-details
config:
platform: linux
image_resource:
type: docker-image
source:
repository: alpine
inputs:
- name: push-some-image
run:
path: sh
args:
- -exc
- |
ls -lrt push-some-image
cat push-some-image/repository
cat push-some-image/digest
Well see the details of the image push from push-some-image
+ cat push-some-image/repository
xxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/path/image
+ cat push-some-image/digest
sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Passing data within a job's tasks could easily be done with input/output artifacts (files), As Clara Fu noted.
For the case between jobs, when simple e.g. 'string' data has to be passed , and using a git is an overkill, the 'keyval' resource[1] seems to be a good solution.
The readme describes that the data is stored and managed as a standard properties file.
https://github.com/SWCE/keyval-resource