Jfrog pipelines embedded pipeline parent step is failing if one of the step in the embedded pipeline is skipped (without additional failures). Is there a way to make it green in case of skipped step?
Sample Pipeline Yaml
pipelines:
- name: top_pipeline
steps:
- name: scan_controller
type: TriggerPipeline
configuration:
pipelineName: scanner_pipeline
stepName: scan_it
integrations:
- name: myPlatformToken
environmentVariables:
scan_target:
default: "hello-world"
allowCustom: true
values:
- "vault"
- "redis"
- "postgresql"
- "hello-world"
execution:
onStart:
- set_trigger_payload pipelineVariables "scan_target=${scan_target}"
- set_trigger_payload stepVariables "notify=email" "uploadReport=true"
onComplete:
- echo "Final status is $nested_run_status"
- name: scanner_pipeline
steps:
- name: scan_it
type: Bash
execution:
onExecute:
- echo "Image to scan is $scan_target."
- echo "Triggered by parent step at $parent_step_url"
The status of the TriggerPipeline native step will only be "success" if the run that it triggers is also "success". Any other status in the child run will result in the TriggerPipeline step going to a "failure" status.
There are a few ways to better control the status of the child run:
allowFailure - This flag can be added to any step. If "true", then that step will not count towards calculating the final status of the run. This means the step could actually be in skipped, error, or failure status, and that status would not impact how the final run status is calculated.
newVersionOnly - If you configure an input resource with the newVersionOnly flag, it means that your step will skip if that resource was not updated during the run. If a step is skipped for this reason, it won't cause the run to be marked "skipped". That step will be ignored when calculating the final run status
If you have a scenario where it doesn't make sense to try and adjust the status of the child pipeline just to get the TriggerPipeline step to be successful, then you might want to consider setting the "allowFailure" flag on the TriggerPipeline step itself, so that even if it fails, the rest of your run can continue.
Related
We have a few status checks added but due to the amount of time it takes to run 1 job, i would like to only have it run when specific files are changed.
I can update the job via the paths: aka:
on:
pull_request:
paths:
- '**.tf'
but then we have to over right the merge request.
Is it possible to have a job that has to run for status checks but ONLY under a condition without having to do a manual intervention / override?
You could use the action dorny/paths-filter to create edited file filters and then run the intensive task as a conditional step in the job. This means the required status check will still run but just skip a step.
on:
pull_request:
jobs:
bigjob:
name: Some Big Job
runs-on: ubuntu-latest
steps:
- name: Check for Modified TF Files
uses: dorny/paths-filter#v2.2.1
id: filter
with:
filters: |
tf:
- '**/*.tf'
# Only run step if files found
- name: Run Task
if: steps.filter.outputs.tf== 'true'
shell: bash
run: echo "some task"
I have the following workflow outline
jobs:
job1:
steps:
- name: step 1 of job 1
run: exit 1
job2:
if: failure()
steps:
- name: step 1 of job 2
run: echo "Hello World"
I know the above runs. However it marks my corresponding status check as failed (red color). Is there a way to make the action execution successful?
You could use the continue-on-error step option:
jobs.<job_id>.steps[*].continue-on-error
Prevents a job from failing
when a step fails. Set to true to allow a job to pass when this step
fails
You. could check the outcome of the step in order to understand if was failed or not like:
steps.<id>.outcome != 'success'
See outcome doc here
TL; DR: Running jobs a,b in parallel. If a fails, stop the execution of b, while it's still running.
My company uses GitHub Actions to deploy our code.
The first step in our deployment is building dockers and pushing them to DockerHub.
We wrote a test for our code, which we want to run in parallel with building the dockers.
Both of these are separate jobs, and we have a few more jobs depending on the success of the first two.
Right now, if the test job fails, the other job continues to run, but obviously, the next one won't run, because the test job failed.
What I would like to do is cancel the docker building job while it's running, if the test failed.
Is that possible? After searching the web, StackOverflow and the GitHub Actions page, I haven't found a way to do that.
Thanks!
You can specify the needs option and refer to the job name. See: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idneeds
An example could be something like:
jobs:
build:
...
deploy:
needs: build
...
You can use the Cancel this build action.
The basic idea is to add it as a final step in each of your jobs that you want to cause a short-circuit in case of failure:
jobs
job_a:
steps:
- run: |
echo 'I am failing'
exit 1
- name: Cancelling parallel jobs
if: failure()
uses: andymckay/cancel-action#0.2
job_b:
steps:
- run: echo 'long task'
This will basically cancel job_b or any other in the same workflow whenever job_a fails.
Since you are working on an enterprise project, I would prefer to avoid using unverified actions from public repositories no matter how many stars they have. I think you can add a step to the end of each job a, b. This step will only run if previous steps failed. If it is failed then it will send a cancel-workflow api call.
- if: failure()
name: Check Job Status
uses: actions/github-script#v6
env:
RUN_ID: ${{ github.run_id }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
script: |
const runId = process.env.RUN_ID
const [owner, repo] = process.env.GITHUB_REPOSITORY.split("/");
const resp = await github.rest.actions.cancelWorkflowRun({
owner,
repo,
runId
})
Note: You may need to add another custom github_pat since this api-call may require higher permissions than default actions. I also suggest you to take a look at this post , I found it quite useful.
I've set up a workflow in GitHub actions to run my tests and create an artifact of the test coverage. The stripped down version of my YAML file looks like this:
name: Build
on: [pull_request]
jobs:
build:
runs-on: ubuntu-latest
steps:
# Other steps here
- name: Build app
- name: Run tests
- name: Create artifact of test coverage
# Other steps here
The problem is that the artifact does not get created when the tests fail.
I figured out about if: always() condition the from the docs, but this will also cause this step to run when my Build app step fails. I don't want that to happen because there is nothing to archive in that case.
How can I only run this step if the previous step has run (either succeeded or failed)?
Try checking success() OR failure().
name: Build
on: [pull_request]
jobs:
build:
runs-on: ubuntu-latest
steps:
# Other steps here
- name: Build app
- name: Run tests
- name: Create artifact of test coverage
if: success() || failure()
# Other steps here
Alternatively, create a step output of the exit code that you can check in later steps. For example:
- name: Build app
id: build
run: |
<build command>
echo ::set-output name=exit_code::$?
- name: Run tests
- name: Create artifact of test coverage
if: steps.build.outputs.exit_code == 0
A possible better alternative is <step>.outcome or <step>.conclusion
https://docs.github.com/en/actions/learn-github-actions/contexts#steps-context
steps.<step id>.conclusion. The result of a completed step after continue-on-error is applied. Possible values are success, failure, cancelled, or skipped. When a continue-on-error step fails, the outcome is failure, but the final conclusion is success.
steps.<step id>.outcome The result of a completed step before continue-on-error is applied. Possible values are success, failure, cancelled, or skipped. When a continue-on-error step fails, the outcome is failure, but the final conclusion is success.
- name: Build app
id: build
run: |
<build command>
- name: Run tests
- name: Create artifact of test coverage
if: steps.build.outcome == 'success'
I'm trying to follow an example Github has for testing my build with github actions, and then compressing the test results and uploading them as an artifact.
https://help.github.com/en/actions/automating-your-workflow-with-github-actions/persisting-workflow-data-using-artifacts#uploading-build-and-test-artifacts
I'm having trouble with what to do when my tests fail though. This is my action. When my tests pass everything works great, my results are zipped an exported as an artifact, but if my tests fail, it stops the rest of the steps in the job, so my results never get published.
I tried adding the continue-on-error: true https://help.github.com/en/actions/automating-your-workflow-with-github-actions/workflow-syntax-for-github-actions#jobsjob_idstepscontinue-on-error
This makes it continue after it fails and uploads my test results. but then the job is marked as passed, even though my test step failed. Is there some way to have it upload my artifact even if a step fails, while still marking the overall job as failed?
name: CI
on:
pull_request:
branches:
- master
push:
branches:
- master
jobs:
build-and-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: Test App
run: ./gradlew test
- name: Archive Rest Results
uses: actions/upload-artifact#v1
with:
name: test-results
path: app/build/reports/tests
You can add
if: always()
to your step to have it run even if a previous step fails
https://docs.github.com/en/actions/learn-github-actions/expressions#status-check-functions
so for a single step it would look like this:
steps:
- name: Build App
run: ./build.sh
- name: Archive Test Results
if: always()
uses: actions/upload-artifact#v1
with:
name: test-results
path: app/build
Or you can add it to a job:
jobs:
job1:
job2:
needs: job1
job3:
if: always()
needs: [job1, job2]
Additionally, as pointed out below, putting always() will cause the function to run even if the build is canceled.
If dont want the function to run when you manually cancel a job, you can instead put:
if: success() || failure()
Other way, you can add continue-on-error: true.
Look like
- name: Job fail
continue-on-error: true
run |
exit 1
- name: Next job
run |
echo Hello
Read more in here.
run a github-actions step, even if the previous step fails
If you only need to execute the step if it succeeds or fails, then:
steps:
- name: Build App
run: ./build.sh
- name: Archive Test Results
if: success() || failure()
uses: actions/upload-artifact#v1
with:
name: test-results
path: app/build
Why use success() || failure() instead of always()?
Reading the Status check functions documentation on Github:
always
Causes the step to always execute, and returns true, even when canceled. A job or step will not run when a critical failure prevents the task from running. For example, if getting sources failed.
Which means the job will run even when it gets cancelled, if that's what you want, then go ahead. Otherwise, success() || failure() would be more suitable.
Note -
The documentation made clear thanks to Vladimir Panteleev in which he submitted the following PR: Github Docs PR #8411
Addon: if you have following sitution. 2 steps i.e. build > deploy and in some cases i.e. workflow_dispatch with input parameters you might want to skip build and proceed with deploy. At the same time you might want deploy to be skipped, when build failed.
Logically that would be something like skipped or not failed as deploy conditional.
if: always() will not work, cause it will always trigger deploy, even if build failed.
Solution is pretty simple:
if: ${{ !failure() }}
Mind that you cannot skip brackets when negating in if:, cause it reports syntax error.
The other answers here are great and work, but you might want a little more granularity.
For instance, ./upload only if ./test ran, even if it failed.
However, if something else failed and prevented the tests from running, don't upload.
# ... Other steps
- run: ./test
id: test
- run: ./upload
if: success() || steps.test.conclusion == 'failure'
steps.*.conclusion will be success, failure, cancelled, or skipped.
success or failure indicate the step ran. cancelled or skipped means it didn't.
Note there is an important caveat that you must test at least one success() or failure() in if.
if: steps.test.conclusion == 'success' || steps.test.conclusion == 'failure' won't work as expected.
you can add || true to your command.
example:
jobs:
build-and-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: Test App
run: ./gradlew test || true