We have some database migration jobs that we occationally want to run before deploy a new version of an app. The common approach for this in ArgoCD seems to be to use PreSync hooks, which I have tested and which seems to work, but I'm finding it a little bit limited in terms of functionality, and am unsure if I'm missing something or if that's just how it is.
How I would like it to work, is to only run the db migration jobs when they have changed in some way (most likely a new image), however the way presync jobs seem to be designed (and understandably so) is to always run the specified job on every sync. Functionally, this is fine, the migration job will take ~20 seconds to start and finish and end up doing nothing, however it's clearly not ideal to have this happen for every single unrelated change.
I'm hoping there is some way of accomplishing this "ArgoCD natively" that's I'm just missing.
The job template i'm using currently (and which runs each sync) is this:
{{- define "project.migration_job" -}}
{{- $appsettings := (get .Values.global.apps .name) }}
---
apiVersion: batch/v1
kind: Job
metadata:
generateName: {{ .name }}-
annotations:
argocd.argoproj.io/hook: PreSync
spec:
template:
spec:
automountServiceAccountToken: false
containers:
- name: {{ .name }}
image: "{{ .Values.global.repo }}/{{ .name }}:{{ $appsettings.image }}"
resources:
requests:
memory: {{ $appsettings.memory | default "256Mi" | quote }}
cpu: {{ $appsettings.cpu | default "75m" | quote }}
limits:
memory: {{ $appsettings.memory | default "256Mi" | quote }}
cpu: {{ $appsettings.cpu | default "75m" | quote }}
env:
{{- include "project.environment_variables" (dict "Values" .Values "env" .env) | trim | nindent 12 -}}
{{- include "project.secret_environment_variables" (dict "Values" .Values "secrets" .secrets) | trim | nindent 12 }}
restartPolicy: Never
backoffLimit: 2
{{ end -}}
Thanks for any help.
I don't know if there's a native solution, but this could help:
In your PreSync hook:
IMAGE_TAG_TO_DEPLOY: Get the image tag that will be deployed from the source repo (Ex.: curl -LSs https://x-access-token:"$GITHUB_TOKEN#raw.githubusercontent.com/company/project/master/path-to-image-tag.yaml" and parse the image tag value)
COMMIT_ID_TO_DEPLOY: Find the commit ID that corresponds to IMAGE_TAG_TO_DEPLOY
COMMIT_ID_DEPLOYED: Find the commit ID pointed to by a git tag named currently-deployed
If COMMIT_ID_TO_DEPLOY == COMMIT_ID_DEPLOYED, end the PreSync Hook
Perform the required actions for the PreSync hook.
Add the git tag currently-deployed to COMMIT_ID_TO_DEPLOY in your git repo
Related
I could not find anything by just googling, does Helm support private values?
So I have my chart and my values.yaml
privateProp: hello
publicProp: world
I have some values that I want to exposed to the end user of my chart and others that I do not want, however those "private" values are being used in many places.
For example: publicProps is overridable by the user of the chart, but I would like to block access to privateProp, however it is reused in many places:
containers:
name: {{.Values.privateProp}}
nodeSelector:
name: {{.Values.privateProp}}
I saw there is {{$privateProp := "hello"}}, but it is not clear how I can access it elsewhere in my files
How can I achieve this?
Ok, I have found a solution to my problem.
You can create a file called _variables.tpl, the name does not matter
and then declare a variable:
{{- define "privateProp" -}}
{{- print "hello" -}}
{{- end -}}
and then you can use it wherever you want in your chart by doing this:
spec:
containers:
- name: {{ .Values.dashboard.containers.name }}
image: {{ .Values.dashboard.containers.image.repository }}:{{ .Values.dashboard.containers.image.tag }}
imagePullPolicy: Always
ports:
- containerPort: {{ include "privateProp" . }} # <== This
I am trying to use if/else-if/else loop in helm chart. Basically, I want to add ENV configs in configfile based on the if/else condition. Below is the logic:
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Values.applicationName }}-configmap
labels:
projectName: {{ .Values.applicationName }}
environment: {{ .Values.environment }}
type: configmap
data:
{{- if eq .Values.environment "production" }}
{{ .Files.Get "config-prod.yaml" | nindent 2}}
{{- else if eq .Values.environment "development" }}
{{ .Files.Get "config-dev.yaml" | nindent 2}}
{{- else }}
{{ .Files.Get "config-stage.yaml" | nindent 2}}
{{- end }}
But I am not getting the desired output and facing some issue. Can anybody help me out with this?
Edit1: I have added my modified configmap.yaml as per the suggestions, helm install/template command gives Error: YAML parse error on demo2/templates/configmap.yaml: error converting YAML to JSON: yaml: line 14: did not find expected key error.
also my config-prod and config-stage is being rendered (as per the condition if I give environment: production then config-prod.yaml is being added and if I give environment: stage/null then config-stage.yaml is being added.
Your question would benefit from more specifics.
Please consider adding the following to your question:
How are you trying this? What commands exactly did you run?
How are you "not getting the desired output"? What output did you get?
Please also include:
the relevant entries from your values.yaml
the config-dev.yaml and config-stage.yaml files
Have you run helm template to generate the templates that Helm would apply to your cluster? This would be a good way to diagnose the issue.
I wonder whether you're chomping too much whitespace.
And you should just chomp left, i.e. {{- .... }} rather than left+right {{- ... -}}.
Sorry guys, it was my mistake, my dev-config.yaml has envs and it was defined like key=value, instead of key: value.
I'm using Helm for to deploy multiple K8s deployments. In some deployments I need to include extra environment variables, but for the majority of deployment the standard env. variables are enough. I would like to have named template for those deployments that must have extra env. variables.
Can I include a named template only if the named template exist?
Something like this:
{{ range $idx, $svc := .Values.services }}
kind: Deployment
metadata:
name: {{ $svc.name }}
spec:
containers:
- name: {{ $svc.name }}
env:
- name: JAVA_OPTS
- value: {{ $svc.javaOpts }}
# if template_exists (print $svc.name "-env")
{{ include (print $svc.name "-env") . | indent 12 }}
# end
{{- end -}}
It's in pseudo-code. How to do the # if part?
Thank you.
The easiest way would be to add additional key like templateExists: true to your services and check it with a simple if statement in your deployment:
{{- if $svc.templateExists }}
{{ include (print $svc.name "-env") . | indent 8 }}
{{- end -}}
services:
svc1:
templateExists: true
name: svc1
javaOpts: "-Xms128m -Xmx512m"
svc2:
name: svc2
javaOpts: "-Xms256m -Xmx512m"
Here is a solution that I came up now. I check if there are any files in the chart that matches the pattern [service]-env.yaml and if there are then I include the content of that file in the deployment.
{{- range $path, $_ := $f.Glob "**-env.yaml" }}
{{- if contains $svc.app.name $path }}
{{ $f.Get $path | indent 8 }}
{{- end }}
{{- end }}
That way, for services that require extra env. variables we can include a file in files/service-env.yaml and those variables will be added to the deployment. For services that do not require such variables, it's left empty.
I have 2 different values.yaml files per stage and production environment such as values.dev.yaml > values.prod.yaml and using with Helm 3. I would like to learn the best practices how to pass environment variables per environments.For instance we need to set different parameters to NODE_ENV variable.
-Should I specify the variable as hard coded as below and pass the environment variables when running helm upgrade/install command with --set flag?
-What is the correct way to use go template to do this. Can we specify something {{ .Values.node_env.value}} and then pass this env value in values yaml and use only -f values.yaml flag?
containers:
- name: {{ .Chart.Name }}
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- name: http
containerPort: 8080
protocol: TCP
resources:
{{- toYaml .Values.resources | nindent 12 }}
env:
- name: "NODE_ENV"
value: "stage"
- name: "NODE_ENV"
value: "production"
If you have one value file per environment (it is not clear to me this is your case.) like values.prod.yaml (for prod) and values.dev.yaml (for dev), then your templeate can look like this.
This will cause the template to look for extraEnv: in your values{dev/prod}.yaml and iterate over all key/values from that section.
env:
{{- range $key, $value := .Values.extraEnv }}
- name: {{ $key }}
value: {{ $value | quote }}
{{- end }}
In your values.dev.yaml files you add all your KEY: values that are specific for this environment. Note you can have multiple key values here, all of them will be loaded. In this case we have NODE_ENV, ANOTHER_KEY, YET_ANOTHER_KEY - all of them will be loaded.
extraEnv:
NODE_ENV: stage
ANOTHER_KEY: value
YET_ANOTHER_KEY: value
same in your values.prod.yaml multiple KEY: value pairs can be specified and all of them will be loaded.
extraEnv:
NODE_ENV: production
ANOTHER_KEY: value
currently we're adding features to 3rd party helm charts we're deploying (for example - in prometheus we're adding an authentication support as we use nginx ingress controller).
Obviously, this will cause us headaches when we want to upgrade those helm charts, we will need to perform "diffs" with our changes.
What's the recommended way to add functionality to existing 3rd party helm charts? Should i use umbrella charts and use prometheus as a dependency? then import value from the chart? (https://github.com/helm/helm/blob/master/docs/charts.md#importing-child-values-via-requirementsyaml)
Or any other recommended way?
-- EDIT --
Example - as you can see, i've added 3 nginx.ingress.* annotations to support basic auth on prometheus ingress resource - of course if i'll upgrade, i'll need to manually add them again, which will cause problems
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
{{- if .Values.prometheus.ingress.annotations }}
annotations:
{{ toYaml .Values.prometheus.ingress.annotations | indent 4 }}
{{- end }}
{{- if .Values.alertmanager.ingress.nginxBasicAuthEnabled }}
nginx.ingress.kubernetes.io/auth-realm: "Authentication Required - ok"
nginx.ingress.kubernetes.io/auth-secret: {{ template "prometheus-operator.fullname" . }}-prometheus-basicauth
nginx.ingress.kubernetes.io/auth-type: "basic"
{{- end }}
name: {{ $serviceName }}
labels:
app: {{ template "prometheus-operator.name" . }}-prometheus
{{ include "prometheus-operator.labels" . | indent 4 }}
{{- if .Values.prometheus.ingress.labels }}
{{ toYaml .Values.prometheus.ingress.labels | indent 4 }}
{{- end }}
spec:
rules:
{{- range $host := .Values.prometheus.ingress.hosts }}
- host: {{ . }}
http:
paths:
- path: "{{ $routePrefix }}"
backend:
serviceName: {{ $serviceName }}
servicePort: 9090
{{- end }}
{{- if .Values.prometheus.ingress.tls }}
tls:
{{ toYaml .Values.prometheus.ingress.tls | indent 4 }}
{{- end }}
{{- end }}
I think that might answer your question.
Subcharts and Globals
Requirements
Helm Dependencies
This led me to find the specific part I was looking for, where the parent chart can override sub-charts by specifying the chart name as a key in the parent values.yaml.
In the application chart's requirements.yaml:
dependencies:
- name: jenkins
# Can be found with "helm search jenkins"
version: '0.18.0'
# This is the binaries repository, as documented in the GitHub repo
repository: 'https://kubernetes-charts.storage.googleapis.com/'
Run:
helm dependency update
In the application chart's values.yaml:
# ...other normal config values
# Name matches the sub-chart
jenkins:
# This will be override "someJenkinsConfig" in the "jenkins" sub-chart
someJenkinsConfig: value
I would either fork and handle integrating the changes when you upgrade/rebase, or if possible disable the ingress elements for those you want to customise via the values.yaml file. Then create your own ingress instances manually with the customisations you need in another custom chart, and provide it the references it needs from the prometheus chart as normal values.yaml inputs.
Obviously this approach has it's limitations, if the customisations are too tightly coupled to the chart it might not be possible to split them out.
Hope this helps.