Currently, I am checking if lifecycle hooks are enabled, if yes add some extra delay:
{{- $delay := hasKey .Values "shutdownDelay" | ternary .Values.shutdownDelay 30 }}
{{- $graceperiod := hasKey .Values.service "terminationGracePeriodSeconds" | ternary .Values.service.terminationGracePeriodSeconds 120 }}
{{- $extraDelay := .Values.lifecycleHooks.enabled | ternary $delay 0 }}
terminationGracePeriodSeconds: {{ add $graceperiod $extraDelay }}
I want to cover a use case where the if .Values.lifecycleHooks.postStart and .Values.lifecycleHooks.prestart have some values then it should not add the extra delay in terminationGracePeriodSeconds
The values.yaml looks like
#shutdownDelay: 40
lifecycleHooks:
enabled: true
postStart:
exec:
command:
- echo
- "Run after starting container"
preStop:
exec:
command:
- echo
- "Run before stopping container"
service:
terminationGracePeriodSeconds: 120
So if the poststop hook value is defined like in values.yaml then it should not add any delay to terminationperiod.
The question is not very specific but if you are looking for "if"
condition with "and"/"or", below is an example might be helpfull.
As per your explanation assuming value for poststart/prestart, If the lifecycle.poststart is "false" and lifecycle.prestart is "true" then the terminationgraceperiodseconds wont have extradelay else condition will have an extradelay
{{- if and (eq .Values.lifecycleHooks.postStart "false") (eq .Values.lifecycleHooks.prestart "true")) }}
terminationGracePeriodSeconds: {{ $graceperiod }}
{{- else}}
terminationGracePeriodSeconds: {{ add $graceperiod $extraDelay }}
{{- end }}
conditional or
{{- if or (eq .Values.lifecycleHooks.postStart "false") (eq .Values.lifecycleHooks.prestart "true")) }}
terminationGracePeriodSeconds: {{ add $graceperiod $extraDelay }}
{{- end }}
Related
Given this deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment
spec:
revisionHistoryLimit: 5
template:
spec:
containers:
{{- include "app.container" (merge .Values.app $) | nindent 8 }}
{{- include "ports" (merge .Values.app ) | nindent 8 }}
{{- range $k, $v := .Values.extraContainers }}
{{- $nameDict := dict "name" $k -}}
{{- include "app.container" (mustMergeOverwrite $.Values.app $nameDict $v) | nindent 8 }}
{{- include "ports" (merge $nameDict $v ) | nindent 8 }}
{{- end }}
This helpers file...
{{/* vim: set filetype=helm: */}}
{{/*
app container base
*/}}
{{- define "app.containerBase" -}}
- name: {{ .name | default "app" }}
image: {{ .image.name }}:{{ .image.tag }}
{{- if .command }}
command:
{{- range .command }}
- {{ . | quote }}
{{- end }}
{{- end }}
{{- if .args }}
args:
{{- range .args }}
- {{ . | quote }}
{{- end }}
{{- end }}
{{- range .envVars }}
- name: {{ .name }}
{{- with .value }}
value: {{ . | quote }}
{{- end }}
{{- with .valueFrom }}
valueFrom:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
{{- if or .Values.configMaps .Values.secrets .Values.configMapRef }}
envFrom:
{{- if .Values.configMaps }}
- configMapRef:
name: "{{- include "app.fullname" $ }}"
{{- end }}
{{- range .Values.configMapRef }}
- configMapRef:
name: {{ . }}
{{- end }}
{{- range $name, $idk := $.Values.secrets }}
- secretRef:
name: "{{- include "app.fullname" $ }}-{{ $name }}"
{{- end }}
{{- end }}
{{- end -}}
{{/*
app container
*/}}
{{- define "app.container" -}}
{{- include "app.containerBase" . }}
resources:
limits:
cpu: {{ .resources.limits.cpu | default "100m" | quote }}
memory: {{ .resources.limits.memory | default "128Mi" | quote }}
{{- if .resources.limits.ephemeralStorage }}
ephemeral-storage: {{ .resources.limits.ephemeralStorage }}
{{- end }}
requests:
cpu: {{ .resources.requests.cpu | default "100m" | quote }}
memory: {{ .resources.requests.memory | default "128Mi" | quote }}
{{- if .resources.requests.ephemeralStorage }}
ephemeral-storage: {{ .resources.requests.ephemeralStorage }}
{{- end }}
securityContext:
runAsNonRoot: true
runAsUser: {{ .securityContext.runAsUser }}
runAsGroup: {{ .securityContext.runAsGroup }}
allowPrivilegeEscalation: false
{{- end -}}
{{/*
ports
*/}}
{{- define "ports" -}}
{{- if .port }}
ports:
- containerPort: {{ .port }}
protocol: {{ .protocol | default "tcp" | upper }}
{{- range .extraPorts}}
- containerPort: {{ required ".port is required." .port }}
{{- if .name }}
name: {{ .name }}
{{- end }}
{{- if .protocol }}
protocol: {{ .protocol | upper }}
{{- end }}
{{- end }}
{{- end }}
{{- end -}}
This values.yaml
extraContainers:
extra-container2:
image:
name: xxx.dkr.ecr.eu-west-1.amazonaws.com/foobar-two
tag: test
command:
- sleep
- 10
envVars:
- name: FOO
value: BAR
probes:
readinessProbe:
path: /ContainerTwoReadinessPath
livenessProbe:
path: /ContainerTwoLivenessPath
resources:
limits:
cpu: 200Mi
memory: 2Gi
requests:
cpu: 200Mi
memory: 2Gi
extra-container3:
image:
name: xx.dkr.ecr.eu-west-1.amazonaws.com/foobar-three
tag: latest
command:
- sleep
- 10
envVars:
- name: FOO
value: BAZ
probes:
readinessProbe:
enabled: false
livenessProbe:
enabled: false
app:
image:
name: xx.dkr.ecr.eu-west-1.amazonaws.com/foobar
tag: latest
probes:
readinessProbe:
path: /_readiness
enabled: true
livenessProbe:
path: /_liveness
enabled: true
port: 100
resources:
limits:
cpu: 100Mi
memory: 1Gi
requests:
cpu: 100Mi
memory: 1Gi
Why does helm template result in the below:
---
# Source: corp-service/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment
spec:
revisionHistoryLimit: 5
template:
spec:
containers:
- name: app
image: xx.dkr.ecr.eu-west-1.amazonaws.com/foobar:latest
resources:
limits:
cpu: "100Mi"
memory: "1Gi"
requests:
cpu: "100Mi"
memory: "1Gi"
securityContext:
runAsNonRoot: true
runAsUser: 2000
runAsGroup: 2000
allowPrivilegeEscalation: false
ports:
- containerPort: 100
protocol: TCP
- name: extra-container2
image: xx.dkr.ecr.eu-west-1.amazonaws.com/foobar-two:test
command:
- "sleep"
- "10"
- name: FOO
value: "BAR"
resources:
limits:
cpu: "200Mi"
memory: "2Gi"
requests:
cpu: "200Mi"
memory: "2Gi"
securityContext:
runAsNonRoot: true
runAsUser: 2000
runAsGroup: 2000
allowPrivilegeEscalation: false
- name: extra-container3
image: xx.dkr.ecr.eu-west-1.amazonaws.com/foobar-three:latest
command:
- "sleep"
- "10"
- name: FOO
value: "BAZ"
resources:
limits:
cpu: "200Mi"
memory: "2Gi"
requests:
cpu: "200Mi"
memory: "2Gi"
securityContext:
runAsNonRoot: true
runAsUser: 2000
runAsGroup: 2000
allowPrivilegeEscalation: false
i.e why does extra-container3 have resources: set with values from extra-container2 instead of taking them from the default set of resources found under .values.app.
essentially, i want extracontainers to use default values from .values.app unless i explicitly set them inside the extracontainers section. however it seems that if a previous iteration of the loop over extracontainers defines some values, they are used in the next iteration if they are not overriden.
the resources for extra-container3 i am expecting are:
resources:
limits:
cpu: 100Mi
memory: 1Gi
requests:
cpu: 100Mi
memory: 1Gi
what am i doing wrong?
So there are a couple of things going on here, but mostly the answer is that (mergeMustOverwrite) mutates the $dest map, which causes your range to "remember" the last value it saw, which according to your question isn't the behavior you want. The simplest answer is to use (deepCopy $.Values.app) as the $dest, but there's an asterisk to that due to another bug we'll cover in a second
{{- $nameDict := dict "name" $k -}}
- {{- include "app.container" (mustMergeOverwrite $.Values.app $nameDict $v) | nindent 8 }}
+ {{- include "app.container" (mustMergeOverwrite (deepCopy $.Values.app) $nameDict $v) | nindent 8 }}
{{- include "ports" (merge $nameDict $v ) | nindent 8 }}
{{- end }}
You see, (deepCopy $.Values.app) stack-overflows helm because of your inexplicable use of (merge .Values.app $) drags in Chart, Files, Capabilities and the whole world. I'm guessing your _helpers.tpl must have been copy-pasted from somewhere else, which explains the erroneous relative .Values reference inside that define. One way to fix that is to remove the .Values.configMaps, so it will track the .app context like I expect you meant, or you can change the first (merge) to artificially create a "Values": {} item just to keep the template from blowing up when in tries to reference .app.Values.configMaps. The correct one will depend on what you were intending, but (merge .Values.app $) is almost certainly not it
so, either:
--- a/templates/_helpers.tpl
+++ b/templates/_helpers.tpl
## -28,13 +28,13 ## app container base
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
- {{- if or .Values.configMaps .Values.secrets .Values.configMapRef }}
+ {{- if or .configMaps .secrets .configMapRef }}
envFrom:
- {{- if .Values.configMaps }}
+ {{- if .configMaps }}
- configMapRef:
name: "{{- include "app.fullname" $ }}"
{{- end }}
- {{- range .Values.configMapRef }}
+ {{- range .configMapRef }}
- configMapRef:
name: {{ . }}
{{- end }}
or
containers:
- {{- include "app.container" (merge .Values.app $) | nindent 8 }}
+ {{- include "app.container" (merge .Values.app (dict "Values" (dict))) | nindent 8 }}
{{- include "ports" (merge .Values.app ) | nindent 8 }}
I am trying to make a default value for CPU if its mentioned(if its not mentioned, i have handled that separately) value in values file is less than 5000m. I am trying this but I don't think I am doing it correctly.
Also, if I make the default value as 5000m and in my values file its mentioned just as 5. Would it be able to compare both?
resources:
requests:
{{- if .Values.resources.requests.cpu }}
cpu: {{ .Values.resources.requests.cpu }}
{{- end }}
{{- if .Values.resources.requests.memory }}
memory: {{ .Values.resources.requests.memory }}
{{- end }}
limits:
{{- if ((( .Values.resources).limits).cpu) }}
cpu: {{ .Values.resources.limits.cpu }}
{{- else }}
{{- $limit_value := .Values.resources.requests.cpu | toString | regexFind "[0-9.]+" }}
{{- $limit_suffix := .Values.resources.requests.cpu | toString | regexFind "[^0-9.]+" }}
cpu: {{ mulf $limit_value 3 }}{{ $limit_suffix }} }}
{{- end }}
{{- if (((.Values.resources).limits).memory) }}
memory: {{ .Values.resources.limits.memory }}
{{- else }}
{{- $limit_val := .Values.resources.requests.memory | toString | regexFind "[0-9.]+" }}
{{- $limit_suff := .Values.resources.requests.memory | toString | regexFind "[^0-9.]+" }}
memory: {{ mulf $limit_val 3 }}{{ $limit_suff }}
{{- end }}
{{- end }}
You have two separate issues here. There's not a built-in way to parse the Kubernetes resource values, so you'll have to do a lot of work to actually provide that default value.
If you just want to provide a default value and not try to check for a minimum, then you can just use the Helm (Sprig) default function:
resources:
requests:
cpu: {{ .Values.resources.requests.cpu | default "5000m" }}
The minimum bound is what leads to some trouble. I don't believe there's a function in Helm to parse 5000m, or to compare that to 5.0. You could try writing it in Helm template syntax, but it can become awkward.
{{/* Convert a resource quantity like "5000m" to a base number like "5".
Call with the quantity string as the parameter, returns the number
as a string. */}}
{{- define "resource-quantity" -}}
{{- if . | hasSuffix "m" -}}
{{- $quantity = . | trimSuffix "m" | float64 -}}
{{- divf $quantity 10000000 -}}
{{- else -}}
{{ . }}
{{- end -}}
{{- end -}}
Note that there are many suffixes besides m and you might want to handle those too, maybe using a dictionary structure. I'm using the Sprig floating-point math functions which should be included in Helm. This template is actual code and you also want to arrange things like tests for it, which Helm does not support well.
Once you have that, gt (greater-than) is a function that takes two parameters. You want to test both "is it present" and also "is it at least this minimum", so you'd have to repeat the value. For a long value like this one thing that can help is to use the standard template with operator, which both acts like an if instruction and also temporarily rebinds the . variable to the value you're testing.
So you could write something like
{{- with .Values.resources.requests.cpu -}}
{{- $quantity := include "resource-quantity" . | float64 }}
{{- if and . (gt $quantity 5.0) }}
cpu: {{ . }}
{{- else }}{{/* if */}}
cpu: 5000m
{{- end }}{{/* if */}}
{{- else }}{{/* with */}}
cpu: 5000m
{{- end }}{{/* with */}}
But with already tests if the value is non-empty, and you can use the maxf function to enforce a minimum value. So (given a complete working tested resource-quantity template function) you could write:
resources:
requests:
{{- with .Values.resources.requests.cpu }}
cpu: {{ include "resource-quantity" . | float64 | maxf 5.0 }}
{{- else }}
cpu: 5.0
{{- end }}
This is how I managed to compare. Below is the example of one of the unit m
{{- if eq $limit_suffix "m" }}
cpu: {{ mulf $limit_value 3 | max 5000 }}{{$limit_suffix}}
{{- else }}
cpu: {{ mulf $limit_value 3 | max 5 }}
{{- end }}
yaml file and in that below values are defined including one specific value called "environment"
image:
repository: my_repo_url
tag: my_tag
pullPolicy: IfNotPresent
releaseName: cron_script
schedule: "0 10 * * *"
namespace: deploy_cron
rav_admin_password: asdf
environment: testing
testing_forwarder_ip: 10.2.71.21
prod_us_forwarder_ip: 10.2.71.15
Now in my helm chart based on this environment value i need to assign a value to new variable and for that I have written code like below, but always it is not entering into the if else block itself
{{- $fwip := .Values.prod_us_forwarder_ip }}
{{- if contains .Values.environment "testing" }}
{{- $fwip := .Values.testing_forwarder_ip }}
{{- end }}
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: "{{ .Values.releaseName }}"
namespace: "{{ .Values.namespace }}"
labels:
....................................
....................................
....................................
spec:
restartPolicy: Never
containers:
- name: "{{ .Values.releaseName }}"
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: IfNotPresent
args:
- python3
- test.py
- --data
- 100
- {{ $fwip }}
In the above code always i get $fwip value as 10.2.71.21 what ever environment value is either testing or production for both i am getting same value
And if i don't declare the variable $fwip before the if else statement then it says $fwip variable is not defined error, So i am not sure why exactly if else statement is not getting used at all, How to debug further ?
This is a syntax problem of variables and local variables.
The fwip in if should use = instead of :=
{{- $fwip := .Values.prod_us_forwarder_ip }}
{{- if contains .Values.environment "testing" }}
{{- $fwip = .Values.testing_forwarder_ip }}
{{- end }}
I translated it into go code to make it easier for you to understand.
(In the go language, := means definition and assignment, = means assignment)
// :=
env := "testing"
test := "10.2.71.21"
prod := "10.2.71.15"
fwip := prod
if strings.Contains(env,"testing"){
fwip := test
fmt.Println(fwip) // 10.2.71.21
}
fmt.Println(fwip) // 10.2.71.15
// =
env := "testing"
test := "10.2.71.21"
prod := "10.2.71.15"
fwip := prod
if strings.Contains(env,"testing"){
fwip = test
fmt.Println(fwip) // 10.2.71.21
}
fmt.Println(fwip) // 10.2.71.21
I am newbie on spark and minikube. I faced the problem while running spark job in sparkapplication.yaml, spark driver and executor s' are created successfully but each of them does not mount hostPath. I referred Tom Louis's minikube-spark example. Everything runs fine if I put data into the sparkjob image file directly via Dockfile// COPY ~~//.
Currently, data(*.csv) are in localFolder - (mounted) - minikube - (not mounted) - spark driver Pod.
I don't know why hostPath does not mounted, there might be some error I did^^;
Anybody can take a look into my problem? Appreciated..!
template/sparkapplication.yaml
apiVersion: sparkoperator.k8s.io/v1beta2
kind: SparkApplication
metadata:
name: {{ .Release.Name | trunc 63 }}
labels:
chartname: {{ .Chart.Name | trunc 63 | quote }}
release: {{ .Release.Name | trunc 63 | quote }}
revision: {{ .Release.Revision | quote }}
sparkVersion: {{ .Values.sparkVersion | quote }}
version: {{ .Chart.Version | quote }}
spec:
type: Scala
mode: cluster
image: {{ list .Values.imageRegistry .Values.image | join "/" | quote }}
imagePullPolicy: {{ .Values.imagePullPolicy }}
{{- if .Values.imagePullSecrets }}
imagePullSecrets:
{{- range .Values.imagePullSecrets }}
- {{ . | quote }}
{{- end }}
{{- end }}
mainClass: {{ .Values.mainClass | quote }}
mainApplicationFile: {{ .Values.jar | quote }}
{{- if .Values.arguments }}
arguments:
{{- range .Values.arguments }}
- {{ . | quote }}
{{- end }}
{{- end }}
sparkVersion: {{ .Values.sparkVersion | quote }}
restartPolicy:
type: Never
{{- if or .Values.jarDependencies .Values.fileDependencies .Values.sparkConf .Values.hadoopConf }}
deps:
{{- if .Values.jarDependencies }}
jars:
{{- range .Values.jarDependencies }}
- {{ . | quote }}
{{- end }}
{{- end }}
{{- if .Values.fileDependencies }}
files:
{{- range .Values.fileDependencies }}
- {{ . | quote }}
{{- end }}
{{- end }}
{{- if .Values.sparkConf }}
sparkConf:
{{- range $conf, $value := .Values.sparkConf }}
{{ $conf | quote }}: {{ $value | quote }}
{{- end }}
{{- end }}
{{- if .Values.hadoopConf }}
hadoopConf:
{{- range $conf, $value := .Values.hadoopConf }}
{{ $conf | quote }}: {{ $value | quote }}
{{- end }}
{{- end }}
{{- end }}
driver:
{{- if .Values.envSecretKeyRefs }}
envSecretKeyRefs:
{{- range $name, $value := .Values.envSecretKeyRefs }}
{{ $name }}:
name: {{ $value.name}}
key: {{ $value.key}}
{{- end }}
{{- end }}
{{- if .Values.envVars }}
envVars:
{{- range $name, $value := .Values.envVars }}
{{ $name }}: {{ $value | quote }}
{{- end }}
{{- end }}
securityContext:
runAsUser: {{ .Values.userId }}
cores: {{ .Values.driver.cores }}
coreLimit: {{ .Values.driver.coreLimit | default .Values.driver.cores | quote }}
memory: {{ .Values.driver.memory }}
hostNetwork: {{ .Values.hostNetwork }}
labels:
release: {{ .Release.Name | trunc 63 | quote }}
revision: {{ .Release.Revision | quote }}
sparkVersion: {{ .Values.sparkVersion | quote }}
version: {{ .Chart.Version | quote }}
serviceAccount: {{ .Values.serviceAccount }}
{{- if .Values.javaOptions }}
javaOptions: {{ .Values.javaOptions | quote}}
{{- end }}
{{- if .Values.mounts }}
volumeMounts:
{{- range $name, $path := .Values.mounts }}
- name: {{ $name }}
mountPath: {{ $path }}
{{- end }}
{{- end }}
{{- if .Values.tolerations }}
tolerations:
{{ toYaml .Values.tolerations | indent 6 }}
{{- end }}
executor:
{{- if .Values.envVars }}
envVars:
{{- range $name, $value := .Values.envVars }}
{{ $name | quote }}: {{ $value | quote }}
{{- end }}
{{- end }}
securityContext:
runAsUser: {{ .Values.userId }}
cores: {{ .Values.executor.cores }}
coreLimit: {{ .Values.executor.coreLimit | default .Values.executor.cores | quote }}
instances: {{ .Values.executor.instances }}
memory: {{ .Values.executor.memory }}
labels:
release: {{ .Release.Name | trunc 63 | quote }}
revision: {{ .Release.Revision | quote }}
sparkVersion: {{ .Values.sparkVersion | quote }}
version: {{ .Chart.Version | quote }}
serviceAccount: {{ .Values.serviceAccount }}
{{- if .Values.javaOptions }}
javaOptions: {{ .Values.javaOptions }}
{{- end }}
{{- if .Values.mounts }}
volumeMounts:
{{- range $name, $path := .Values.mounts }}
- name: {{ $name }}
mountPath: {{ $path }}
{{- end }}
{{- end }}
{{- if .Values.tolerations }}
tolerations:
{{ toYaml .Values.tolerations | indent 6 }}
{{- end }}
{{- if .Values.jmxExporterJar }}
monitoring:
exposeDriverMetrics: true
exposeExecutorMetrics: true
prometheus:
port: {{ .Values.jmxPort | default 8090 }}
jmxExporterJar: {{ .Values.jmxExporterJar }}
{{- end }}
{{- if .Values.volumes }}
volumes:
- name: input-data
hostPath:
path: /input-data
- name: output-data
hostPath:
path: /output-data
{{- end }}
{{- if .Values.nodeSelector }}
nodeSelector:
{{ toYaml .Values.nodeSelector | indent 4 }}
{{- end }}
values.yaml
# Generated by build.sbt. Please don't manually update
version: 0.1
sparkVersion: 3.0.2
image: kaspi/kaspi-sparkjob:0.1
jar: local:///opt/spark/jars/kaspi-kaspi-sparkjob.jar
mainClass: kaspi.sparkjob
fileDependencies: []
environment: minikube
serviceAccount: spark-spark
imageRegistry: localhost:5000
arguments:
- "/mnt/data-in/"
- "/mnt/data-out/"
volumes:
- name: input-data
hostPath:
path: /input-data
- name: output-data
hostPath:
path: /output-data
mounts:
input-data: /mnt/data-in
output-data: /mnt/data-out
driver:
cores: 1
memory: "2g"
executor:
instances: 2
cores: 1
memory: "1g"
hadoopConf:
sparkConf:
hostNetwork: false
imagePullPolicy: Never
userId: 0
build.sbt
val sparkVersion = "3.0.2"
val sparkLibs = Seq(
"org.apache.spark" %% "spark-core" % sparkVersion,
"org.apache.spark" %% "spark-sql" % sparkVersion,
"org.apache.spark" %% "spark-streaming" % sparkVersion,
"org.apache.spark" %% "spark-mllib" % sparkVersion
)
lazy val commonSettings = Seq(
organization := "kaspi",
scalaVersion := "2.12.13",
version := "0.1",
libraryDependencies ++= sparkLibs
)
val domain = "kaspi"
// for building FAT jar
lazy val assemblySettings = Seq(
assembly / assemblyOption := (assemblyOption in assembly).value.copy(includeScala = false),
assembly / assemblyOutputPath := baseDirectory.value / "output" / s"${domain}-${name.value}.jar"
)
val targetDockerJarPath = "/opt/spark/jars"
val baseRegistry = sys.props.getOrElse("baseRegistry", default = "localhost:5000")
// for building docker image
lazy val dockerSettings = Seq(
imageNames in docker := Seq(
ImageName(s"$domain/${name.value}:latest"),
ImageName(s"$domain/${name.value}:${version.value}"),
),
buildOptions in docker := BuildOptions(
cache = false,
removeIntermediateContainers = BuildOptions.Remove.Always,
pullBaseImage = BuildOptions.Pull.Always
),
dockerfile in docker := {
// The assembly task generates a fat JAR file
val artifact: File = assembly.value
val artifactTargetPath = s"$targetDockerJarPath/$domain-${name.value}.jar"
new Dockerfile {
from(s"$baseRegistry/spark-runner:0.1")
}.add(artifact, artifactTargetPath)
}
)
// Include "provided" dependencies back to default run task
lazy val runLocalSettings = Seq(
// https://stackoverflow.com/questions/18838944/how-to-add-provided-dependencies-back-to-run-test-tasks-classpath/21803413#21803413
Compile / run := Defaults
.runTask(
fullClasspath in Compile,
mainClass in (Compile, run),
runner in (Compile, run)
)
.evaluated
)
lazy val root = (project in file("."))
.enablePlugins(sbtdocker.DockerPlugin)
.enablePlugins(AshScriptPlugin)
.settings(
commonSettings,
assemblySettings,
dockerSettings,
runLocalSettings,
name := "kaspi-sparkjob",
Compile / mainClass := Some("kaspi.sparkjob"),
Compile / resourceGenerators += createImporterHelmChart.taskValue
)
// Task to create helm chart
lazy val createImporterHelmChart: Def.Initialize[Task[Seq[File]]] = Def.task {
val chartFile = baseDirectory.value / "helm" / "Chart.yaml"
val valuesFile = baseDirectory.value / "helm" / "values.yaml"
val chartContents =
s"""# Generated by build.sbt. Please don't manually update
|apiVersion: v1
|name: $domain-${name.value}
|version: ${version.value}
|appVersion: ${version.value}
|description: ETL Job
|home: https://github.com/jyyoo0530/kaspi
|sources:
| - https://github.com/jyyoo0530/kaspi
|maintainers:
| - name: Jeremy Yoo
| email: jyyoo0530#gmail.com
| url: https://www.linkedin.com/in/jeeyoungyoo
|""".stripMargin
val valuesContents =
s"""# Generated by build.sbt. Please don't manually update
|version: ${version.value}
|sparkVersion: ${sparkVersion}
|image: $domain/${name.value}:${version.value}
|jar: local://$targetDockerJarPath/$domain-${name.value}.jar
|mainClass: ${(Compile / run / mainClass).value.getOrElse("__MAIN_CLASS__")}
|fileDependencies: []
|environment: minikube
|serviceAccount: spark-spark
|imageRegistry: localhost:5000
|arguments:
| - "/mnt/data-in/"
| - "/mnt/data-out/"
|volumes:
| - name: input-data
| hostPath:
| path: /input-data
| - name: output-data
| hostPath:
| path: /output-data
|mounts:
| input-data: /mnt/data-in
| output-data: /mnt/data-out
|driver:
| cores: 1
| memory: "2g"
|executor:
| instances: 2
| cores: 1
| memory: "1g"
|hadoopConf:
|sparkConf:
|hostNetwork: false
|imagePullPolicy: Never
|userId: 0
|""".stripMargin
IO.write(chartFile, chartContents)
IO.write(valuesFile, valuesContents)
Seq(chartFile, valuesFile)
}
lazy val showVersion = taskKey[Unit]("Show version")
showVersion := {
println((version).value)
}
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs # _*) => MergeStrategy.discard
case x => MergeStrategy.first
}
******2021/2/25 UPDATES ********
I tried below yaml for test purpose, then volume from hostpath mounted successfully in Pod. There are no differences, but the object characteristic is different, one is "container", one is "driver","executor"...etc.
(Same problem happened while using gaffer-hdfs which k8s object name is "namenode", "datanode"...etc).
Can it be a problem using custom kubernetes object name??
But if it is still inherited container properties,,, there is no reason to not to be mounted.
.... so.... still struggling..! :)
apiVersion: v1
kind: Pod
metadata:
name: hostpath
namespace: spark-apps
spec:
containers:
- name: nginx
image: nginx
volumeMounts:
- name: volumepath
mountPath: /mnt/data
volumes:
- name: volumepath
hostPath:
path: /input-data
type: Directory
I'm trying to create my own chart library from tutorial: https://helm.sh/docs/topics/library_charts/ . In the tutorial, there is really nice function, which merges 2 yamls together:
{{- define "libchart.util.merge" -}}
{{- $top := first . -}}
{{- $overrides := fromYaml (include (index . 1) $top) | default (dict ) -}}
{{- $tpl := fromYaml (include (index . 2) $top) | default (dict ) -}}
{{- toYaml (merge $overrides $tpl) -}}
{{- end -}}
I define web deployment template as:
{{- define "libchart.web.tpl" -}}
(...)
containers:
- env:
envFrom:
- configMapRef:
name: app-configmap
- secretRef:
name: app-secrets
image: {{ include "libchart.image" . }}
imagePullPolicy: Always
name: web
resources: {}
(...)
{{- end -}}
{{- define "libchart.web" -}}
{{- include "libchart.util.merge" (append . "libchart.web.tpl") -}}
{{- end -}}
When I want to override a dictionary it's ok:
{{- include "libchart.web" (list . "web") -}}
{{- define "web" -}}
metadata:
annotations:
test1: "test1"
{{- end -}}
And now I want to override resources:
{{- include "libchart.web" (list . "web") -}}
{{- define "web" -}}
spec:
template:
spec:
containers:
- resources:
requests:
cpu: '0.01'
memory: 109.0Mi
{{- end -}}
And the error which I get is:
Error: unable to build kubernetes objects from release manifest: error validating "": error validating data: ValidationError(Deployment.spec.template.spec.containers[0]): missing required field "name" in io.k8s.api.core.v1.Container
Which means that it tries to create 2 elements on the container list.
I found closed issue on githug: https://github.com/helm/charts/issues/19855.