Helm [stable/nginx-ingress] Getting issue while passing headers - kubernetes

Version of Helm and Kubernetes: Client: &version.Version{SemVer:"v2.14.1" and 1.13.7-gke.24
Which chart: stable/nginx-ingress [v0.24.1]
What happened: Trying to override headers using--set-string but it does not work as expected. It always gives issues with the parsing
/usr/sbin/helm install --name cx-nginx-1 --set controller.name=cx-nginx-1 --set controller.kind=Deployment --set controller.service.loadBalancerIP= --set controller.metrics.enabled=true --set-string 'controller.headers={"X-Different-Name":"true","X-Request-Start":"test-header","X-Using-Nginx-Controller":"true"}' . Error: release cx-nginx-1 failed: ConfigMap in version "v1" cannot be handled as a ConfigMap: v1.ConfigMap.Data: ReadMapCB: expect { or n, but found [, error found in #10 byte of ...|","data":["\"X-Diffe|..., bigger context ...|{"apiVersion":"v1","data":["\"X-Different-Name\":\"true\"","\"X-Request-Start|...
What you expected to happen: I want to override the header which the there by default in values.yam with custom headers
How to reproduce it (as minimally and precisely as possible):
I have provided the comment to reproduce,
helm install --name cx-nginx-1 --set controller.name=cx-nginx-1 --set controller.kind=Deployment --set controller.service.loadBalancerIP= --set controller.metrics.enabled=true --set-string 'controller.headers={"X-Different-Name":"true","X-Request-Start":"test-header","X-Using-Nginx-Controller":"true"}' .
I tried to run in debug mode (--dry-run --debug), It shows me configmap like below,
apiVersion: v1
kind: ConfigMap
metadata:
labels:
app: nginx-ingress
chart: nginx-ingress-1
component: "cx-nginx-1"
heritage: Tiller
release: foiled-coral
name: foiled-coral-nginx-ingress-custom-headers
namespace: cx-ingress
data:
- X-Different-Name:true
- X-Request-Start:test-header
- X-Using-Nginx-Controller:true
It seems like its adding intent 4 instead of intent 2. Below warning also i'm getting,
Warning: Merging destination map for chart 'nginx-ingress'. Cannot overwrite table item 'headers', with non table value: map[X-Different-Name:true X-Request-Start:test-header X-Using-Nginx-Controller:true]
Kindly help me to pass the headers in the right way.

Note: controller.headers is deprecated, make sure to use the controller.proxySetHeaders instead.
Helm --set has some limitations.
Your best option is to avoid using the --set, and use the --values instead.
You can declare all your custom values in a file like this:
# values.yaml
controller:
name: "cx-nginx-1"
kind: "Deployment"
service:
loadBalancerIP: ""
metrics:
enable: true
proxySetHeaders:
X-Different-Name: "true"
X-Request-Start: "true"
X-Using-Nginx-Controller: "true"
Then use it on install:
helm install --name cx-nginx-1 stable/nginx-ingress \
--values=values.yaml
If you want to use --set anyway, you should use this notation:
helm install --name cx-nginx-1 stable/nginx-ingress \
--set controller.name=cx-nginx-1 \
--set controller.kind=Deployment \
--set controller.service.loadBalancerIP= \
--set controller.metrics.enabled=true \
--set-string controller.proxySetHeaders.X-Different-Name="true" \
--set-string controller.proxySetHeaders.X-Request-Start="true" \
--set-string controller.proxySetHeaders.X-Using-Nginx-Controller="true"

Related

Helm cant pull registry image

After helm upgrade i got error:
Failed to pull image "myhostofgitlab.ru/common-core-executor:1bac97ef": rpc error: code = Unknown desc = Error response from daemon: Head https://myhostofgitlab.ruv2/common-core-executor/manifests/1bac97ef: denied: access forbidden
my run command:
k8s-deploy-Prod:
image: alpine/helm:latest
stage: deploy
script:
- helm upgrade ${PREFIX}-common-core-executor k8s/helm/common-core-executor --debug --atomic --install --wait --history-max 3
--set image.repository=${CI_REGISTRY_IMAGE}/common-core-executor
--set image.tag=${CI_COMMIT_SHORT_SHA}
--set name=${PREFIX}-common-core-executor
--set service.name=${PREFIX}-common-core-executor
--set branch=${PREFIX}
--set ingress.enabled=true
--set ingress.hosts[0].host=${PREFIX}.common-core-executor.k8s.test.zone
--set ingress.tls[0].hosts[0]=${PREFIX}.common-core-executor.k8s.test.zone
--set secret.name=${PREFIX}-${PROJECT_NAME}-secret
--timeout 2m0s
-f k8s/helm/common-core-executor/common-core-executor-values.yaml
-n ${NAMESPACE}
Where i wrong?
Before that error i make some steps from officially instruction. Firstable i create cred like this (its just sample data):
apiVersion: v1
kind: Secret
data:
.dockerconfigjson: eyJhdXRocyI6eyJodHRwczovL2hvc3QtZm9yLXN0YWNrLW92ZXJmbG93OnsidXNlcm5hbWUiOiJzdGFja292ZXJmbG93IiwicGFzc3dvcmQiOiJzdGFja292ZXJmbG93IiwiYXV0aCI6Inh4eCJ9fX0=
metadata:
name: regcred
namespace: prod-common-service
type: kubernetes.io/dockerconfigjson
And add in containers section of deployment.yaml
imagePullSecrets:
- name: regcred
Thanks!

Helm fails to pass double quotes into values.yaml from the command line?

I have a Helm chart with values.yaml containing:
# Elided
tolerations: []
I'm trying to pass the tolerations via the command line but it always removes the quotes (or adds double quotes inside single quotes) despite all the below attempts. As a result it fails on install saying it expected a string.
# Attempt 0
helm install traefik traefik/traefik --set tolerations[0].key=CriticalAddonsOnly --set tolerations[0].value="true" --set tolerations[0].operator=Equal --set tolerations[0].effect=NoExecute
# Attempt 1
helm install traefik traefik/traefik --set tolerations[0].key=CriticalAddonsOnly --set "tolerations[0].value="true"" --set tolerations[0].operator=Equal --set tolerations[0].effect=NoExecute
# Attempt 2
helm install traefik traefik/traefik --set tolerations[0].key=CriticalAddonsOnly --set "tolerations[0].value=\"true\"" --set tolerations[0].operator=Equal --set tolerations[0].effect=NoExecute
# Attempt 3
helm install traefik traefik/traefik --set tolerations[0].key=CriticalAddonsOnly --set tolerations[0].value="\"true\"" --set tolerations[0].operator=Equal --set tolerations[0].effect=NoExecute
# Attempt 4
helm install traefik traefik/traefik --set tolerations[0].key=CriticalAddonsOnly --set tolerations[0].value='"true"' --set tolerations[0].operator=Equal --set tolerations[0].effect=NoExecute
They all end up creating a yaml with value: true or value: '"true"', neither of which will install.
There appears to be two answers: the exceptionally verbose one that you're trying has a solution, or the more succinct one which doesn't prompt stack overflow questions for future readers to understand:
Helm offers --set-string which is the interpolation-free version of --set
helm install traefik traefik/traefik \
--set tolerations[0].key=CriticalAddonsOnly \
--set-string tolerations[0].value=true \
--set tolerations[0].operator=Equal \
--set tolerations[0].effect=NoExecute
However, as you experienced, that --set syntax is designed for the simplest cases only, for more complex cases --values is the correct mechanism. You can read them from stdin if created a temporary yaml file is too much work
printf 'tolerations: [{key: CriticalAddonsOnly, value: "true", operator: Equal, effect: NoExecute}]\n' | \
helm install traefik traefik/traefik --values /dev/stdin

how to convert nginx-ingress annotations to --set format to enable prometheus metrics

I want to set annotations on command line while installing nginx-ingress. my values.yaml file looks like below and i want to use command line argument instead of values.yaml file.
controller:
metrics:
port: 10254
enabled: true
service:
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "10254"
serviceMonitor:
enabled: true
namespace: monitoring
namespaceSelector:
any: true
I tried following arguments but its giving error
--set controller.metrics.service.annotations."prometheus\.io\/scrape"="true" --set controller.metrics.service.annotations."prometheus\.io\/port"="10254"
Error:
Error: release nginx-ingress failed: Service in version "v1" cannot be handled as a Service: v1.Service.ObjectMeta: v1.ObjectMeta.Annotations: ReadString: expects " or n, but found 1, error found in #10 byte of ...|io/port":10254,"prom|..., bigger context ...|,"metadata":{"annotations":{"prometheus.io/port":10254,"prometheus.io/scrape":true},"labels":{"app.k|...
Any suggestions how exactly these annotations should be passed ?
I just had the same issue! When you look at the chart, they define it as a string. So when I utilize the command below it successfully sets the values. The trick is to utilize --set-string rather than --set
helm upgrade ingress-controller ingress-nginx/ingress-nginx --namespace ingress-nginx --set controller.metrics.enabled=true --set-string controller.metrics.service.annotations."prometheus\.io/scrape"="true" --set-string controller.metrics.service.annotations."prometheus\.io/port"="10254"
Showing that the values are set when we validate this with helm get values ingress-controller --namespace ingress-nginx
controller:
metrics:
enabled: true
service:
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
Now if you would like to get the details into prometheus, then it appears that this did not work for me though. I had to utilize the controller.podAnnotations to get this working:
helm upgrade ingress-controller ingress-nginx/ingress-nginx \
--namespace ingress-nginx \
--set controller.metrics.enabled=true \
--set-string controller.podAnnotations."prometheus\.io/scrape"="true" \
--set-string controller.podAnnotations."prometheus\.io/port"="10254"
Let me know if this worked for you! :)

Provide nodeSelector to nginx ingress using helm

I spent some time looking into how to pass the parameters to helm in order to configure the nodeSelector properly.
Different tries led to different errors like:
Error: unable to build kubernetes objects from release manifest: error validating "": error validating data: ValidationError(Deployment.spec.template.spec.nodeSelector.kubernetes): invalid type for io.k8s.api.core.v1.PodSpec.nodeSelector: got "map", expected "string"
coalesce.go:196: warning: cannot overwrite table with non table for nodeSelector (map[])
Reference: https://learn.microsoft.com/en-us/azure/aks/ingress-static-ip
In the link above, we can see how it should be used:
helm install nginx-ingress stable/nginx-ingress \
--namespace $NAMESPACE \
--set controller.replicaCount=1 \
--set controller.nodeSelector."kubernetes\.io/hostname"=$LOADBALANCER_NODE \
--set controller.service.loadBalancerIP="$LOADBALANCER_IP" \
--set controller.extraArgs.default-ssl-certificate="$NAMESPACE/$LOADBALANCER_NODE-ssl"
In general it is a good source to look into helm help: https://helm.sh/docs/intro/using_helm/#the-format-and-limitations-of---set
Here you can find all the nginx parameters: https://github.com/helm/charts/tree/master/stable/nginx-ingress

Set extraEnvs ingress variable

I'm trying to use extraEnv property in order to add additional environment variables to set in the pod using helm charts.
I have a values.yaml file that includes:
controller:
service:
loadBalancerIP:
extraEnvs:
- name: ev1
value:
- name: ev2
value
first I've set the loadBalancerIP the following way:
helm template path/charts/ingress --set nginx-ingress.controller.service.loadBalancerIP=1.1.1.1 --output-dir .
In order to set extraEnvs values I've tried to use the same logic by doing:
helm template path/charts/ingress --set nginx-ingress.controller.service.loadBalancerIP=1.1.1.1 --set nginx-ingress.controller.extraEnvs[0].value=valueEnv1--set nginx.controller.extraEnvs[1].value=valueEnv2--output-dir .
But it doesn't work. I looked for the right way to set those variables but couldn't find anything.
Helm --set has some limitations.
Your best option is to avoid using the --set, and use the --values flag with your values.yaml file instead:
helm template path/charts/ingress \
--values=values.yaml
If you want to use --set anyway, the equivalent command should have this notation:
helm template path/charts/ingress \
--set=controller.service.loadBalancerIP=1.1.1.1 \
--set=controller.extraEnvs[0].name=ev1,controller.extraEnvs[0].value=valueEnv1 \
--set=controller.extraEnvs[1].name=ev2,controller.extraEnvs[1].value=valueEnv2 \
--output-dir .