Argo Workflow: "Bad Request: json: cannot unmarshal string into Go struct field" - unmarshalling

I have a Argo WorkflowTemplate that looks like this:
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: test-container-command
spec:
entrypoint: main
templates:
- name: main
inputs:
parameters:
- name: command
container:
image: alpine:latest
command: "{{ inputs.parameters.command }}"
env: # some predefined env
What I want to do is to create a WorkflowTemplate that can execute an arbitrary command specified by the input parameter command. That way, users of this WorkflowTemplate can supply the parameter command with an array of strings and then execute it like:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: test-run-
spec:
workflowTemplateRef:
name: test-container-command
entrypoint: main
arguments:
parameters:
- name: command
value:
- echo
- hello
However, when I try to save this WorkflowTemplate, the Argo server gave me this error message:
Bad Request: json: cannot unmarshal string into Go struct field Container.workflow.spec.templates.container.command of type []string
It seems that Argo expects the field .spec.templates.container.command to be an array of strings, but it treat "{{ inputs.parameters.command }}" as a string, even though I'm trying to supply the parameter command with an array of strings.
Is there any way to achieve what I was trying to do as the WorkflowTemplate test-container-command, i.e. provide a WorkflowTemplate for the user to execute arbitrary commands with a predefined container and env?

As it says in error command should be list of strings.
You need to reformat your template to:
container:
image: alpine:latest
command: ["{{ inputs.parameters.command }}"]
Now argo should create your workflow without any problems
You could also use
container:
image: alpine:latest
command:
- "{{ inputs.parameters.command }}"
Edit
As you want to run some commands within the image it would be much better instead of container use script template
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: test-container-command
spec:
entrypoint: main
templates:
- name: main
inputs:
parameters:
- name: command
- name: extraEnv
script:
image: alpine:latest
command: [ "sh" ]
env:
- { name: ENV1, value: "foo" }
- { name: ENV2, value: "{{ inputs.parameters.extraEnv }}" }
source: |
{{ inputs.parameters.command }}

Related

Combining ENV variables in helm chart

Based on this SO, this should work and I'm not sure what I'm missing.
I'm trying to combine env variables in a helm chart. TARGET and TARGET_KEY, but I'm getting:
- name: TARGET_KEY # combining keys together
value: Hello $(TARGET)
I'm expecting
- name: TARGET_KEY # combining keys together
value: Hello World
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: hello
namespace: myapp
spec:
template:
metadata:
annotations:
autoscaling.knative.dev/minScale: "1"
autoscaling.knative.dev/target: "10"
spec:
containers:
- image: gcr.io/knative-samples/helloworld-go
ports:
- containerPort: 8080
env:
- name: TARGET
value: "World"
- name: APIKEY
valueFrom: # get single key from secret at key
secretKeyRef:
name: {{ .Values.keys.name }}
key: apiKey
- name: TARGET_KEY # combining keys together
value: Hello $(TARGET)
envFrom: # set ENV variables from all the values in secret
- secretRef:
name: {{ .Values.keys.name }}
I am using ArgoCD to sync the helm charts. Checking the newly deployed pod's ENV vars.
#David is correct. The ENV variable shown in template and pod description keep the template name, but once I ssh'ed into the pod, doing printenv shows the env variable was properly filled in.
However, I did read there are issues with alphabetic sorting and ordering when trying to mix multiple ENV vars this way. That's a topic for another SO.

Configuring Argo output artifacts defined in a WorkflowTeamplate from Workflow

With the following WorkflowTemplate with an output artifact defined with the name messagejson. I am trying to configure it to use S3 in a Workflow:
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: file-output
spec:
entrypoint: writefile
templates:
- name: writefile
container:
image: alpine:latest
command: ["/bin/sh", "-c"]
args: ["echo hello | tee /tmp/message.json; ls -l /tmp; cat /tmp/message.json"]
outputs:
artifacts:
- name: messagejson
path: /tmp/message.json
---
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: read-file-
spec:
entrypoint: read-file
templates:
- name: read-file
steps:
- - name: print-file-content
templateRef:
name: file-output
template: writefile
arguments:
artifacts:
- name: messagejson
s3:
endpoint: 1.2.3.4
bucket: mybucket
key: "/rabbit/message.json"
insecure: true
accessKeySecret:
name: my-s3-credentials
key: accessKey
secretKeySecret:
name: my-s3-credentials
key: secretKey
However, I get Error (exit code 1): You need to configure artifact storage. More information on how to do this can be found in the docs: https://argoproj.github.io/argo-workflows/configure-artifact-repository/. The same works if I try to configure input artifacts from a Workflow but not output artifacts.
Any idea?

How can I get sub workflow steps/tasks output?

I created a cluster workflow template, which will do some tasks. And I will use last step output as current workflow parameters. When I ref this template, I don't know how can I get the output from cluster workflow task/step.
Cluster Workflow Template
apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
name: gen-params
spec:
templates:
- name: tasks
steps:
- - name: prepare
template: prepare
- - name: gen-params
template: gen-params
...
Workflow
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
name: demo
spec:
entrypoint: main
templates:
- name: main
steps:
- - name: gen-params
templateRef:
name: gen-params
template: tasks
clusterScope: true
- - name: calculate
template: calculate
arguments:
parameters:
- name: params
value: "{{steps.gen-params.steps.gen-params.outputs.result}}" # not work
...
Your issue is likely less about the usage of a WorkflowTemplate/ClusterWorkflowTemplate and more to do with the fact that you are attempting to access output from a "nested" workflow step.
You can achieve this by defining an output parameter of the top-level tasks template in your ClusterWorkflowTemplate which takes its value from the output result of the last step in that tasks template.
Your WorkflowTemplate would look like this:
apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
name: gen-params
spec:
templates:
- name: tasks
steps:
- - name: prepare
template: prepare
- - name: gen-params
template: gen-params
outputs:
parameters:
- name: "nested-gen-params-result"
valueFrom:
parameter: "{{steps.gen-params.outputs.result}}"
After making that change, you'll be able reference the output of the ClusterWorkflowTemplate-defined step of your top-level Workflow using {{steps.gen-params.outputs.parameters.nested-gen-params-result}}
Argo's nested-workflow example shows some other similar patterns.
templateRef is simply a link, used to populate the YAML of the Workflow step. You should interact with the gen-params step in the same way that you would if you'd just copy/pasted the YAML from the gen-params ClusterWorkflowTemplate directly into your new Workflow.
In this case, you should access the result of the gen-params step with this: {{steps.gen-params.outputs.result}}.

using generateName for yaml file to be installed by helm

I have upload.yaml file which is uploads a script to mongo, I package with helm.
apiVersion: batch/v1
kind: Job
metadata:
generateName: upload-strategy-to-mongo-v2
spec:
parallelism: 1
completions: 1
template:
metadata:
name: upload-strategy-to-mongo
spec:
volumes:
- name: upload-strategy-to-mongo-scripts-volume
configMap:
name: upload-strategy-to-mongo-scripts-v3
containers:
- name: upload-strategy-to-mongo
image: mongo
env:
- name: MONGODB_URI
value: ####
- name: MONGODB_USERNAME
valueFrom:
secretKeyRef:
name: mongodb-user
key: ####
- name: MONGODB_PASSWORD
valueFrom:
secretKeyRef:
name: mongodb-user
key: #####
volumeMounts:
- mountPath: /scripts
name: upload-strategy-to-mongo-scripts-volume
command: ["mongo"]
args:
- $(MONGODB_URI)/ravnml
- --username
- $(MONGODB_USERNAME)
- --password
- $(MONGODB_PASSWORD)
- --authenticationDatabase
- admin
- /scripts/upload.js
restartPolicy: Never
---
apiVersion: v1
kind: ConfigMap
metadata:
creationTimestamp: null
name: upload-strategy-to-mongo-scripts-v3
data:
upload.js: |
// Read the object from file and parse it
var data = cat('/scripts/strategy.json');
var obj = JSON.parse(data);
// Upsert strategy
print(db.strategy.find());
db.strategy.replaceOne(
{ name : obj.name },
obj,
{ upsert: true }
)
print(db.strategy.find());
strategy.json: {{ .Files.Get "strategy.json" | quote }}
now I am using generateName to generate a custom name every time I install it. I require to have multiple packages been installed and I require the name to be dynamic.
Error
When I install this script with helm install <name> <tar.gz file> -n <namespace> I get the following error
Error: rendered manifests contain a resource that already exists. Unable to continue with install: could not get information about the resource: resource name may not be empty
but I am able to install if I don't use generateName. Any ideas?
I looked at various resources but they don't seem to answer how to install via helm.
references looked:
Add random string on Kubernetes pod deployment name https://github.com/kubernetes/kubernetes/issues/44501 ;
https://zknill.io/posts/kubernetes-generated-names/
This seems to be a known issue. Helm doesn't work with generateName. For unique names, you can use the Helm's build in properties like Revision or Name. See the following link for reference:
https://github.com/helm/helm/issues/3348#issuecomment-482369133

Kubernetes Argo submit parameter into steps

I'm following the examples on the Argo GitHub but I am unable to change the parameter of message when I move the template into steps.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: hello-world-parameters-
spec:
# invoke the whalesay template with
# "hello world" as the argument
# to the message parameter
entrypoint: entry-point
templates:
- name: entry-point
steps:
- - name: print-message
template: whalesay
arguments:
parameters:
- name: message
value: hello world
- name: whalesay
inputs:
parameters:
- name: message # parameter declaration
container:
# run cowsay with that message input parameter as args
image: docker/whalesay
command: [cowsay]
args: ["{{inputs.parameters.message}}"]
If I submit the workflow using the following command:
argo submit .\workflow.yml -p message="goodbye world"
It still prints out hello world and not goodbye world. Not sure why
The -p argument sets the global workflow parameters defined in the arguments field of workflow spec. More information is available here . To use global parameters your workflow should be changed are the following:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: hello-world-parameters-
spec:
# invoke the whalesay template with
# "hello world" as the argument
# to the message parameter
entrypoint: entry-point
arguments:
parameters:
- name: message
value: hello world
templates:
- name: entry-point
steps:
- - name: print-message
template: whalesay
arguments:
parameters:
- name: message
value: "{{workflow.parameters.message}}"
- name: whalesay
inputs:
parameters:
- name: message # parameter declaration
container:
# run cowsay with that message input parameter as args
image: docker/whalesay
command: [cowsay]
args: ["{{inputs.parameters.message}}"]