I have an Argo workflow YAML file responsible for running some templates on the cloud.
spec:
entrypoint: templateA
onExit: templateB
templates:
- name: templateA
<templateA YAML specs here>
- name: templateB
container:
image: <SOME IMAGE>
command: [sh, -c]
args: ["echo Hello World"]
outputs:
artifacts:
<CODE TO COPY FROM LOCAL TO S3>
I want to run a command when the transfer is completed or error has occurred. For eg: I want to record the timestamp and print it when the output artifacts transfer is completed.
What is the correct mechanism for me to achieve this?
Related
I'm currently doing a PoC to validate usage of Argo Workflow. I created a workflow spec with the following template (this is just a small portion of the workflow yaml):
templates:
- name: dummy-name
inputs:
parameters:
- name: params
container:
name: container-name
image: <image>
volumeMounts:
- name: vault-token
mountPath: "/etc/secrets"
readOnly: true
imagePullPolicy: IfNotPresent
command: ['workflow', 'f10', 'reports', 'expiry', '.', '--days-until-expiry', '30', '--vault-token-file-path', '/etc/secrets/token', '--environment', 'corporate', '--log-level', 'debug']
The above way of passing the commands works without any issues upon submitting the workflow. However, if I replace the command with {{inputs.parameters.params}} like this:
templates:
- name: dummy-name
inputs:
parameters:
- name: params
container:
name: container-name
image: <image>
volumeMounts:
- name: vault-token
mountPath: "/etc/secrets"
readOnly: true
imagePullPolicy: IfNotPresent
command: ['workflow', '{{inputs.parameters.params}}']
it fails with the following error:
DEBU[2023-01-20T18:11:07.220Z] Log line
content="Error: failed to find name in PATH: exec: \"workflow f10 reports expiry . --days-until-expiry 30 --vault-token-file-path /etc/secrets/token --environment corporate --log-level debug\":
stat workflow f10 reports expiry . --days-until-expiry 30 --vault-token-file-path /etc/secrets/token --environment corporate --log-level debug: no such file or directory"
Am I missing something here?
FYI: The Dockerfile that builds the container has the following ENTRYPOINT set:
ENTRYPOINT ["workflow"]
We have a Tekton pipeline and want to replace the image tags contents of our deployment.yml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: microservice-api-spring-boot
spec:
replicas: 3
revisionHistoryLimit: 3
selector:
matchLabels:
app: microservice-api-spring-boot
template:
metadata:
labels:
app: microservice-api-spring-boot
spec:
containers:
- image: registry.gitlab.com/jonashackt/microservice-api-spring-boot#sha256:5d8a03755d3c45a3d79d32ab22987ef571a65517d0edbcb8e828a4e6952f9bcd
name: microservice-api-spring-boot
ports:
- containerPort: 8098
imagePullSecrets:
- name: gitlab-container-registry
Our Tekton pipeline uses the yq Task from Tekton Hub to replace the .spec.template.spec.containers[0].image with the "$(params.IMAGE):$(params.SOURCE_REVISION)" name like this:
- name: substitute-config-image-name
taskRef:
name: yq
runAfter:
- fetch-config-repository
workspaces:
- name: source
workspace: config-workspace
params:
- name: files
value:
- "./deployment/deployment.yml"
- name: expression
value: .spec.template.spec.containers[0].image = \"$(params.IMAGE)\":\"$(params.SOURCE_REVISION)\"
Sadly the yq Task doesn't seem to work, it produces a green
Step completed successfully, but shows the following errors:
16:50:43 safelyRenameFile [ERRO] Failed copying from /tmp/temp3555913516 to /workspace/source/deployment/deployment.yml
16:50:43 safelyRenameFile [ERRO] open /workspace/source/deployment/deployment.yml: permission denied
Here's also a screenshot from our Tekton Dashboard:
Any idea on how to solve the error?
The problem seems to be related to the way how the Dockerfile of https://github.com/mikefarah/yq now handles file permissions (for example this fix among others). The 0.3 version of the Tekton yq Task uses the image https://hub.docker.com/layers/mikefarah/yq/4.16.2/images/sha256-c6ef1bc27dd9cee57fa635d9306ce43ca6805edcdab41b047905f7835c174005 which produces the error.
One work-around to the problem could be the usage of the yq Task version 0.2 which you can apply via:
kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/main/task/yq/0.2/yq.yaml
This one uses the older docker.io/mikefarah/yq:4#sha256:34f1d11ad51dc4639fc6d8dd5ade019fe57cf6084bb6a99a2f11ea522906033b and works without the error.
Alternatively you can simply create your own yq based Task that won't have the problem like this:
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: replace-image-name-with-yq
spec:
workspaces:
- name: source
description: A workspace that contains the file which need to be dumped.
params:
- name: IMAGE_NAME
description: The image name to substitute
- name: FILE_PATH
description: The file path relative to the workspace dir.
- name: YQ_VERSION
description: Version of https://github.com/mikefarah/yq
default: v4.2.0
steps:
- name: substitute-with-yq
image: alpine
workingDir: $(workspaces.source.path)
command:
- /bin/sh
args:
- '-c'
- |
set -ex
echo "--- Download yq & add to path"
wget https://github.com/mikefarah/yq/releases/download/$(params.YQ_VERSION)/yq_linux_amd64 -O /usr/bin/yq &&\
chmod +x /usr/bin/yq
echo "--- Run yq expression"
yq e ".spec.template.spec.containers[0].image = \"$(params.IMAGE_NAME)\"" -i $(params.FILE_PATH)
echo "--- Show file with replacement"
cat $(params.FILE_PATH)
resources: {}
This custom Task simple uses the alpine image as base and installs yq using the Plain binary wget download. Also it uses yq exactly as you would do on the command line locally, which makes development of your expression so much easier!
As a bonus it outputs the file contents so you can check the replacement results right in the Tekton pipeline!
You need to apply it with
kubectl apply -f tekton-ci-config/replace-image-name-with-yq.yml
And should now be able to use it like this:
- name: replace-config-image-name
taskRef:
name: replace-image-name-with-yq
runAfter:
- dump-contents
workspaces:
- name: source
workspace: config-workspace
params:
- name: IMAGE_NAME
value: "$(params.IMAGE):$(params.SOURCE_REVISION)"
- name: FILE_PATH
value: "./deployment/deployment.yml"
Inside the Tekton dashboard it will look somehow like this and output the processed file:
I'm exploring an easy way to read K8S resources in the Argo workflow. The current documentation is focusing mainly on create/patch with conditions (https://argoproj.github.io/argo/examples/#kubernetes-resources), while I'm curious if it's possible to perform "action: get", extra part of the resource state (or full resource) and pass it downstream as artifact or result output. Any ideas?
action: get is not a feature available from Argo.
However, it's easy to use kubectl from within a Pod and then send the JSON output to an output parameter. This uses a BASH script to send the JSON to the result output parameter, but an explicit output parameter or an output artifact are also viable options.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: kubectl-bash-
spec:
entrypoint: kubectl-example
templates:
- name: kubectl-example
steps:
- - name: generate
template: get-workflows
- - name: print
template: print-message
arguments:
parameters:
- name: message
value: "{{steps.generate.outputs.result}}"
- name: get-workflows
script:
image: bitnami/kubectl:latest
command: [bash]
source: |
some_workflow=$(kubectl get workflows -n argo | sed -n 2p | awk '{print $1;}')
kubectl get workflow "$some_workflow" -ojson
- name: print-message
inputs:
parameters:
- name: message
container:
image: alpine:latest
command: [sh, -c]
args: ["echo result was: '{{inputs.parameters.message}}'"]
Keep in mind that kubectl will run with the permissions of the Workflow's ServiceAccount. Be sure to submit the Workflow using a ServiceAccount which has access to the resource you want to get.
As per the Argo DAG template documentation.
tasks.<TASKNAME>.outputs.parameters: When the previous task uses
'withItems' or 'withParams', this contains a JSON array of the output
parameter maps of each invocation
When trying with the following simple workflow:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: test-workflow-
spec:
entrypoint: start
templates:
- name: start
dag:
tasks:
- name: with-items
template: hello-letter
arguments:
parameters:
- name: input-letter
value: "{{item}}"
withItems:
- A
- B
- C
- name: show-result
dependencies:
- with-items
template: echo-result
arguments:
parameters:
- name: input
value: "{{tasks.with-items.outputs.parameters}}"
- name: hello-letter
inputs:
parameters:
- name: input-letter
outputs:
parameters:
- name: output-letter
value: "{{inputs.parameters.input-letter}}"
script:
image: alpine
command: ["sh"]
source: |
echo "{{inputs.parameters.input-letter}}"
- name: echo-result
inputs:
parameters:
- name: input
outputs:
parameters:
- name: output
value: "{{inputs.parameters.input}}"
script:
image: alpine
command: ["sh"]
source: |
echo {{inputs.parameters.input}}
I get the following error :
Failed to submit workflow: templates.start.tasks.show-result failed to resolve {{tasks.with-items.outputs.parameters}}
Argo version (running in a minikube cluster)
argo: v2.10.0+195c6d8.dirty
BuildDate: 2020-08-18T23:06:32Z
GitCommit: 195c6d8310a70b07043b9df5c988d5a62dafe00d
GitTreeState: dirty
GitTag: v2.10.0
GoVersion: go1.13.4
Compiler: gc
Platform: darwin/amd64
Same error in Argo 2.8.1, although, using .result instead of .parameters in the show-result task worked fine (result was [A,B,C]), but doesn't work in 2.10 anymore
- name: show-result
dependencies:
- with-items
template: echo-result
arguments:
parameters:
- name: input
value: "{{tasks.with-items.outputs.result}}"
The result:
STEP TEMPLATE PODNAME DURATION MESSAGE
⚠ test-workflow-parallelism-xngg4 start
├-✔ with-items(0:A) hello-letter test-workflow-parallelism-xngg4-3307649634 6s
├-✔ with-items(1:B) hello-letter test-workflow-parallelism-xngg4-768315880 7s
├-✔ with-items(2:C) hello-letter test-workflow-parallelism-xngg4-2631126026 9s
└-⚠ show-result echo-result invalid character 'A' looking for beginning of value
I also tried to change the show-result task as:
- name: show-result
dependencies:
- with-items
template: echo-result
arguments:
parameters:
- name: input
value: "{{tasks.with-items.outputs.parameters.output-letter}}"
Executes without no errors:
STEP TEMPLATE PODNAME DURATION MESSAGE
✔ test-workflow-parallelism-qvp72 start
├-✔ with-items(0:A) hello-letter test-workflow-parallelism-qvp72-4221274474 8s
├-✔ with-items(1:B) hello-letter test-workflow-parallelism-qvp72-112866000 9s
├-✔ with-items(2:C) hello-letter test-workflow-parallelism-qvp72-1975676146 6s
└-✔ show-result echo-result test-workflow-parallelism-qvp72-3460867848 3s
But the parameter is not replaced by the value:
argo logs test-workflow-parallelism-qvp72
test-workflow-parallelism-qvp72-1975676146: 2020-08-25T14:52:50.622496755Z C
test-workflow-parallelism-qvp72-4221274474: 2020-08-25T14:52:52.228602517Z A
test-workflow-parallelism-qvp72-112866000: 2020-08-25T14:52:53.664320195Z B
test-workflow-parallelism-qvp72-3460867848: 2020-08-25T14:52:59.628892135Z {{tasks.with-items.outputs.parameters.output-letter}}
I don't understand what to expect as the output of a loop! What did I miss? Is there a way to find out what's happening?
There was a bug which caused this error before Argo version 3.2.5. Upgrade to latest and try again.
It looks like the problem was in the CLI only. I submitted the workflow with kubectl apply, and it ran fine. The error only appeared with argo submit.
The argo submit error was resolved when I upgraded to 3.2.6.
This is quite a common problem I've faced, I have not come across this in any bug report or feature documentation so far so it's yet to be determined if this is a feature or bug. However argo is clearly not capable in performing a "map-reduce" flow OOB.
The only "real" workaround I've found is to attach an artifact, write the with-items task output to it, and pass it along to your next step where you'll do the "reduce" yourself in code/script by reading values from the artifact.
---- edit -----
As mentioned by another answer this was indeed a bug which was resolved for the latest version, this resolves the usage of parameters as you mentioned as an option but outputs.result still causes an error post bugfix.
This issue is currently open on Argo Workflows Github: issue #6805
You could use a nested DAG to workaround this issue. This helps the parallel executions artifact resolution problem because each task output artifact is scoped to its inner nested DAG only, so there's only one upstream branch in the dependency tree. The error in issue #6805 happens when artifacts exist in the previous step and there's more than one upstream branch in the dependency tree.
I'm trying out Argo workflow and would like to understand how to freeze a step. Let's say that I have 3 step workflow and a workflow failed at step 2. So I'd like to resubmit the workflow from step 2 using successful step 1's artifact. How can I achieve this? I couldn't find the guidance anywhere on the document.
I think you should consider using Conditions and Artifact passing in your steps.
Conditionals provide a way to affect the control flow of a
workflow at runtime, depending on parameters. In this example
the 'print-hello' template may or may not be executed depending
on the input parameter, 'should-print'. When submitted with
$ argo submit examples/conditionals.yaml
the step will be skipped since 'should-print' will evaluate false.
When submitted with:
$ argo submit examples/conditionals.yaml -p should-print=true
the step will be executed since 'should-print' will evaluate true.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: conditional-
spec:
entrypoint: conditional-example
arguments:
parameters:
- name: should-print
value: "false"
templates:
- name: conditional-example
inputs:
parameters:
- name: should-print
steps:
- - name: print-hello
template: whalesay
when: "{{inputs.parameters.should-print}} == true"
- name: whalesay
container:
image: docker/whalesay:latest
command: [sh, -c]
args: ["cowsay hello"]
If you use conditions in each step you will be able to start from a step you like with appropriate condition.
Also have a loot at this article Argo: Workflow Engine for Kubernetes as author explains the use of conditions on coinflip example.
You can see many examples on their GitHub page.