I have a HELM Chart with a few requirements (i.e. subcharts).
When deploying that chart, I use a values.yaml containing all the values for both the main chart and the subcharts :
globalFoo: "bar"
subchart1:
foo: subchart1-{{ globalFoo }}
subchart2:
localFoo: "bar2"
foo: subchart2-{{ subchart2.localFoo }}
I'd like to achieve two things :
Reference a previously declared global variable (i.e. in the global chart scope) in a subchart value (subchart1.foo in my example)
Reference a previously declared local variable in the same subchart scope (subchart2.foo in my example)
The exemple above doesn't work. I tried several syntaxes and none of them worked. I didn't find anything like that in the HELM documentation.
Is it doable ?
Reference a previously declared global variable in a subchart value, Reference a previously declared local variable in the same subchart scope
This can be achieved to some extent using anchors and aliases.
global:
foo: &global-foo bar
subchart1:
# this verbatim copies the content of the anchor
foo: *global-foo
local: &subchart1-local bar
subchart2:
foo: *subchart1-local
Values can naturally be combined in a helm template:
kind: ConfigMap
...
data:
FOO: "subchart2-{{ .Values.subchart2.foo }}"
If find yourself needing "templated values" the tpl function may be helpful:
# values
global:
user: foo
pass: bar
dbhost: dbserver.com
mychart:
connection: "db://{{.d.user}}:{{d.pass}}/{{d.dbhost}}"
# template
kind: ConfigMap
...
data:
DBURL: "{{ tpl .Values.mychart.connection (dict "d" .Values.global "Template" $.Template }}"
Note the (dict ...) syntax is derived from a hint in this helm github comment. This helps to shorten the template string by giving the ".d" context instead ".Values", i.e. .d.user is short for .Values.global.user
Currently (as of Helm version 3) this is not supported.
A similar issue is discussed in Proposal: Allow templating in values.yaml
And is rejected for several reasons. One of them stated by the creator of Helm
The bigger constraint is that the values.yaml file MUST always be a valid YAML file.
And also in Support for using {{ values }} within values.yaml
the file that is passed into the template engine as a source of data for templates is itself not passed through the template engine. We almost definitely will not do that because it gets very confusing for users. It also breaks both standard YAML compatibility and backward compatibility with all existing Helm versions.
And last but not least here
The tl;dr
At its most simple, the reason behind not wanting to do this is you don’t template the file that provides the values that you template with. Furthermore, this feature will be made obsolete by Helm 3
...
Obsolete by Helm 3
In Helm 3 as part of the eventing/hook system, we will allow Lua-based modification of values in a much easier way than having to template them out. Seeing as we are well underway with development for Helm 3, adding a feature with all of the drawbacks mentioned above would cause more churn than the value added. So we aren’t rejecting this outright, but are implementing it (albeit in a different way) for Helm 3
But the mentioned support via Lua has been closed with this comment
We should close this as Lua support is not coming anytime soon and we have other (arguably better) ideas on handling this (like WASM). This was in the plan but isn't any longer.
Create yaml-bomb.yaml file:
apiVersion: v1
data:
a: &a ["web","web","web","web","web","web","web","web","web"]
b: &b [*a,*a,*a,*a,*a,*a,*a,*a,*a]
c: &c [*b,*b,*b,*b,*b,*b,*b,*b,*b]
d: &d [*c,*c,*c,*c,*c,*c,*c,*c,*c]
e: &e [*d,*d,*d,*d,*d,*d,*d,*d,*d]
f: &f [*e,*e,*e,*e,*e,*e,*e,*e,*e]
g: &g [*f,*f,*f,*f,*f,*f,*f,*f,*f]
h: &h [*g,*g,*g,*g,*g,*g,*g,*g,*g]
i: &i [*h,*h,*h,*h,*h,*h,*h,*h,*h]
kind: ConfigMap
metadata:
name: yaml-bomb
namespace: default
Send ConfigMap creation request to Kubernetes API by cmd kubectl apply -f yaml-bomb.yaml.
kube-api CPU/memory usage are very high, even later are getting restarted.
How do we prevent such yaml-bomb?
This is a billion laughts attack and can only be fixed in the YAML processor.
Note that the Wikipedia is wrong here when it says
A "Billion laughs" attack should exist for any file format that can contain references, for example this YAML bomb:
The problem is not that the file format contains references; it is the processor expanding them. This is against the spirit of the YAML spec which says that anchors are used for nodes that are actually referred to from multiple places. In the loaded data, anchors & aliases should become multiple references to the same object instead of the alias being expanded to a copy of the anchored node.
As an example, compare the behavior of the online PyYAML parser and the online NimYAML parser (full disclosure: my work) when you paste your code snippet. PyYAML won't respond because of the memory load from expanding aliases, while NimYAML doesn't expand the aliases and therefore responds quickly.
It's astonishing that Kubernetes suffers from this problem; I would have assumed since it's written in Go that they are able to properly handle references. You have to file a bug with them to get this fixed.
There's a couple of possible mitigations I could think of although as #flyx says the real fix here would be in the YAML parsing library used by Kubernetes.
Interestingly running this on a Kubernetes cluster on my local machine showed the CPU spike to be client-side (it's the kubectl process churning CPU) rather than server side.
If the issue was server side, then possible mitigations would be to use RBAC to minimize access to ConfigMap creation, and potentially to use an admission controller like OPA to review manifests before they are applied to the cluster.
This should probably be raised with the Kubernetes security vulnerability response team so that a proper fix can be implemented.
EDIT - I think where the problem manifests, might be down to the cluster version used. Server-side apply graduated to beta (should be enabled by default) in 1.16. So on a 1.16 cluster perhaps this would hit server side instead of client side.
EDIT - Just setup a 1.16 cluster, still showing the CPU usage as client-side in kubectl...
EDIT - I've filed an issue for this here also confirmed that the DoS can be achieved server-side by using curl instead of kubectl
Final EDIT - This got assigned a CVE (CVE-2019-11253) and is being fixed in Kubernetes 1.13+ . The fix has also been applied to the underlying YAML parsing lib here so any other Go programs should be ok as long as they're using an up to date version.
There was a TrustCom19 paper studying vulnerabilities in YAML parsers for different languages, it found that most parsers have some issues, so this is common and there are several recent CVEs in this space (details in paper: Laughter in the Wild: A Study into DoS Vulnerabilities in YAML Libraries, TrustCom19.
Preprint: https://www.researchgate.net/publication/333505459_Laughter_in_the_Wild_A_Study_into_DoS_Vulnerabilities_in_YAML_Libraries
I'm trying to deploy Node.js code to a Kubernetes cluster, and I'm seeing that in my reference (provided by the maintainer of the cluster) that the yaml files are all prefixed by numbers:
00-service.yaml
10-deployment.yaml
etc.
I don't think that this file format is specified by kubectl, but I found another example of it online: https://imti.co/kibana-kubernetes/ (but the numbering scheme isn't the same).
Is this a Kubernetes thing? A file naming convention? Is it to keep files ordered in a folder?
This is to handle the resource creation order. There's an opened issue in kubernetes:
https://github.com/kubernetes/kubernetes/issues/16448#issue-113878195
tl;dr kubectl apply -f k8s/* should handle the order but it does not.
However, except the namespace, I cannot imagine where the order will matter. Every relation except namespace is handled by label selectors, so it fixes itself once all resources are deployed. You can just do 00-namespace.yaml and everything else without prefixes. Or just skip prefixes at all unless you really hit the issue (I never faced it).
When you execute kubectl apply * the files are executed alphabetically. Prefixing files with a rising number allows you to control the order of the executed files. But in nearly all cases the order shouldn't matter.
Sequence helps in readability, user friendly and not the least maintainability. Looking at the resources one can conclude in which order the deployment needs to be performed. For example, deployment using configMap object would fail if the deployment is done before configMap is created.
I have a helm chart for an application that needs some kind of database.
Both mysql or postgresql would be fine.
I would like to give the chart user the option to install one of these as a dependency like this:
dependencies:
- name: mysql
version: 0.10.2
repository: https://kubernetes-charts.storage.googleapis.com/
condition: mysql.enabled
- name: postgresql
version: 3.11.5
repository: https://kubernetes-charts.storage.googleapis.com/
condition: postgresql.enabled
However this makes it possible to enable both of them.
Is there an easy way to make sure only one is selected?
I was thinking of a single variable selecting one of [mysql, postgres, manual] and depend on a specific database if it is selected. - Is there a way to do so?
I don't think there's a straightforward way to do this. In particular, it looks like the requirements.yaml condition: field only takes a boolean value (or a list of them) and not an arbitrary expression. From the Helm documentation:
The condition field holds one or more YAML paths (delimited by commas). If this path exists in the top parent’s values and resolves to a boolean value, the chart will be enabled or disabled based on that boolean value. Only the first valid path found in the list is evaluated and if no paths exist then the condition has no effect.
(The tags mechanism described below that is extremely similar and doesn't really help.)
When it comes down to actually writing your Deployment spec you have a more normal conditional system and can test that only one of the values is set; so I don't think you can prevent having redundant databases installed, but you'll at least only use one of them. You could also put an after-the-fact warning to this effect in your NOTES.txt file.
{{ if and .Values.mysql.enabled .Values.postgresql.enabled -}}
WARNING: you have multiple databases enabled in your Helm values file.
Both MySQL and PostgreSQL are installed as part of this chart, but only
PostgreSQL is being used. You can update this chart installation setting
`--set mysql.enabled=false` to remove the redundant database.
{{ end -}}
I have a working Kubernetes cluster that I want to monitor with Grafana.
I have been trying out many dashboards from https://grafana.com/dashboards but they all seem to have some problems: it looks like there's a mismatch between the Prometheus metric names and what the dashboard expects.
Eg if I look at this recently released, quite popular dashboard: https://grafana.com/dashboards/5309/revisions
I end up with many "holes" when running it:
Looking into the panel configuration, I see that the issues come from small key changes, eg node_memory_Buffers instead of node_memory_Buffers_bytes.
Similarly the dashboard expects node_disk_bytes_written when Prometheus provides node_disk_written_bytes_total.
I have tried out a lot of Kubernetes-specific dashboards and I have the same problem with almost all of them.
Am I doing something wrong?
The Prometheus node exporter changed a lot of the metric names in the 0.16.0 version to conform to new naming conventions.
From https://github.com/prometheus/node_exporter/releases/tag/v0.16.0:
Breaking changes
This release contains major breaking changes to metric names. Many
metrics have new names, labels, and label values in order to conform
to current naming conventions.
Linux node_cpu metrics now break out guest values into separate
metrics.
Many counter metrics have been renamed to include _total.
Many metrics have been renamed/modified to include
base units, for example node_cpu is now node_cpu_seconds_total.
See also the upgrade guide. One of its suggestion is to use compatibility rules that will create duplicate metrics with the old names.
Otherwise use version 0.15.x until the dashboards are updated, or fix them!