Unable to import grafana json file via the grafana helm chart - grafana

I am deploying the helm chart stable/grafana 4.3.0 onto a k8s cluster. I am using Helm 3. From a previous grafana installation, I have exported the json of a dashboard and saved it as my-dashboard.json. I want to have helm take care of uploading this file, so in my values.yaml I have
dashboardProviders:
dashboardproviders.yaml:
apiVersion: 1
providers:
- name: 'default'
orgId: 1
folder: ''
type: file
disableDeletion: false
editable: true
options:
path: /var/lib/grafana/dashboards
dashboards:
default:
my-dashboard:
file: my-dashboard.json
prometheus-stats:
gnetId: 2
revision: 2
datasource: Prometheus
I already have my Prometheus datasource (from the prometheus helm chart) defined as
datasources:
datasources.yaml:
apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
url: http://my-prometheus-release-server.default.svc.cluster.local
access: proxy
isDefault: true
And I've verified that the datasource works correctly.
If I run helm upgrade my-grafana-release stable/grafana --values values.yaml however, in the logs on the pod it repeats:
t=2020-01-17T21:33:35+0000 lvl=eror msg="failed to load dashboard from " logger=provisioning.dashboard type=file name=default file=/var/lib/grafana/dashboards/default/my-dashboard.json error=EOF
Seeing EOF makes me think the file isn't uploading. I have my-dashboard.json saved in the same folder as values.yaml, and I'm running the helm command from that folder. Do I need to store it somewhere else? I have searched all the documentation and it's not clear to me how it gets uploaded.

For anyone stumbling on this:
In case you installed grafana with the grafana helm chart or prometheus operator helm chart an easy way to add grafana dashboards is to set sidecar.dashboards.enabled: true in your values.yml (I recommend checking the docs for more info on this).
Then you can add dashboards with a simple configmap like this:
apiVersion: v1
kind: ConfigMap
metadata:
name: my-grafana-dashboard
labels:
grafana_dashboard: "1"
data:
my-dashboard.json: |
{
"annotations": {
"list": [
{
.....JSON.....
}

I found another way to handle this.
I am using Terraform to accomplish this, and I made values.yaml as a template file. This is the relevant section in values.yaml now:
dashboards:
default:
dashbaord1:
json: |
${my-dashboard-1}
dashboard2:
json: |
${my-dashboard-2}
And the templatefile block looks like this:
resource "helm_release" "grafana" {
name = "grafana-release"
repository = data.helm_repository.stable.metadata[0].name
chart = "grafana"
version = "4.3.0"
values = [
"${templatefile(
"${path.module}/values.yaml.tpl",
{
my-dashboard-1 = "${indent(8, data.template_file.my-dashboard-1.rendered)}}",
my-dashboard-2 = "${indent(8, data.template_file.my-dashboard-2.rendered)}}"
}
)}"
]
}
The indent is very important!

Where did you put the my-dashboard.jsonfile ?
It should be on the same level as values.yaml
Also check the grafana-dashboards-default configmap, it should contain the dashboard.

Related

ArgoCD helm chart how to override values yml in declarative helm chart deployment App/controller

I have an yaml which gets deployed by the ArgoCd controller, that deploys a helm chart from artifactory.
For my local development I use a sperate values.yml in the helm chart.
My controller looks like below refer git link
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: <name-to-the-app>
namespace: argocd
spec:
project: default
source:
repoURL: https://harbor.1000kit.org/chartrepo/1000kit/
targetRevision: <version-hardcode-in-repo>
chart: <chart-name-that-is-getting-deployed>
helm:
releaseName: <release-name-hardcoded>
# custom values to override the helm chart one
values: |
<pass-the-custom-values>>
destination:
server: https://kubernetes.default.svc
namespace: <namespace-where-to-be-deployed>
syncPolicy:
automated:
prune: true
selfHeal: true
The helm chart that is getting deployed is containing the values.yaml
I am trying to override the values.yml present in the helm chart in artifcatory, so passing all the values in part of the source -> helm -> values like above.
Question:
In the custom values, I skipped some value but the ArgoCd is fetching those values from the helm chart value.yml and using it. Is this the behavior?
Another observation is that, The helm chart repo values.yaml is being loaded as parmater in the ArgoCD, and the argocd.io application yaml the values are displayed in the UI.
From the documents i see there are parameters, which can be overridden but the values can't be overridden.
spec:
source:
helm:
parameters:
- name: app
value: $ARGOCD_APP_NAME
Is there any option to explicitly tell ArgoCD to ignore the values.yml from the helm chart in artifactory.
I am new to ArgoCd
Looks like,
The helm chart in the artifactory has a key : value combination say
# value present in the helm chart values.yaml
app-details:
name: "demoapp"
version: "1.0"
description: "simple demo app"
In argoCd application manifest
# .....
helm:
releaseName: <release-name-hardcoded>
# custom values to override the helm chart one
values: |
app-details :
name: "demoapp"
version: "1.0"
# if I didn't specify the description here <--------------- *
# ....
* - ArgoCd is setting the default value (from values.yaml) for description key in this case. This description value can be seen in the deployed manifest.
Also the values.yml data are displayed as parameter in the UI.
I had to use a different approach, where I need to make changes in the template charts that uses this description value.

How to add additional scrape config to Prometheus

I want to add an additional scrape config into Prometheus. I have followed the below method.
https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/additional-scrape-config.md
First, created a file prometheus-additional.yaml and added the new config
- job_name: "prometheus"
static_configs:
- targets: ["localhost:9090"]
Secondly, created a secret out of it.
kubectl create secret generic additional-scrape-configs --from-file=prometheus-additional.yaml --dry-run -oyaml > additional-scrape-configs.yaml
Then created the secret using the below command
kubectl apply -f additional-scrape-configs.yaml -n monitoring
Then in the above link it says
"Finally, reference this additional configuration in your prometheus.yaml CRD."
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
name: prometheus
labels:
prometheus: prometheus
spec:
replicas: 2
serviceAccountName: prometheus
serviceMonitorSelector:
matchLabels:
team: frontend
additionalScrapeConfigs:
name: additional-scrape-configs
key: prometheus-additional.yaml
Where I can find the above? Do I need to create a new CRD? Can't I update the existing running deployment?
This is somehow wrong in the Documentation, you have to use additionalScrapeConfigsSecret:
additionalScrapeConfigsSecret:
enabled: true
name: additional-scrape-configs
key: prometheus-additional.yaml
Else you get the error cannot unmarshal !!map into []yaml.MapSlice
Here is a better documentation:
https://github.com/prometheus-community/helm-charts/blob/8b45bdbdabd9b54766c4beb3c562b766b268a034/charts/kube-prometheus-stack/values.yaml#L2691
According to this, you could add scrape configs without packaging into a secret like this:
additionalScrapeConfigs: |
- job_name: "prometheus"
static_configs:
- targets: ["localhost:9090"]
That document applies to prometheus-operator. If you have deployed it, you should have your Prometheus CRD:
kubectl get prometheus -n monitoring
Then you can edit the Prometheus exactly as stated in above: adding additionalScrapeConfigs key in the spec. (after adding a secret)
After editing, new scrape configs will be reloaded and applied automatically (conf-reloaders).

Grafana dashboard not working with Ingress

I have installed below kube-prometheus-stack and getting an error when trying to access Grafana dashboard using it's own Ingress URL. I believe I am missing something silly here but unable to find any clues. I have looked at similar post here and others as well.
Chart: kube-prometheus-stack-9.4.5
App Version: 0.38.1
When I navigate to https://myorg.grafanatest.com URL, I get redirected to https://myorg.grafanatest.com/login with following message.
Changes made to grafana/values.yaml:
grafana.ini:
server:
# The full public facing url you use in browser, used for redirects and emails
root_url: https://myorg.grafanatest.com
Helm command used to install Prometheus-Grafana operator after making above changes.
helm install pg kube-prometheus-stack/ -n monitoring
I see below settings in grafana.ini file inside Grafana pod.
[analytics]
check_for_updates = true
[grafana_net]
url = https://grafana.net
[log]
mode = console
[paths]
data = /var/lib/grafana/data
logs = /var/log/grafana
plugins = /var/lib/grafana/plugins
provisioning = /etc/grafana/provisioning
[server]
root_url = https://myorg.grafanatest.com/
Posting a solution here as it's working now. Followed steps as gumelaragum mentioned above to create values.yaml and updated below values in that, and passed that values.yaml to helm install step. Not sure why it didn't work without enabling serve_from_sub_path, but it's ok as it's working now. Note that I didn't enable Ingress section since I have already created Ingress route outside the installation process.
helm show values prometheus-com/kube-prometheus-stack > custom-values.yaml
Then install by changing below values in custom-values.yaml. Change namespace as needed.
helm -n monitoring install -f ./custom-values.yaml pg prometheus-com/kube-prometheus-stack
grafana:
enabled: true
namespaceOverride: ""
# set pspUseAppArmor to false to fix Grafana pod Init errors
rbac:
pspUseAppArmor: false
grafana.ini:
server:
domain: mysb.grafanasite.com
#root_url: "%(protocol)s://%(domain)s/"
root_url: https://mysb.grafanasite.com/grafana/
serve_from_sub_path: true
## Deploy default dashboards.
##
defaultDashboardsEnabled: true
adminPassword: prom-operator
ingress:
## If true, Grafana Ingress will be created
##
enabled: false
## Annotations for Grafana Ingress
##
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
## Labels to be added to the Ingress
##
labels: {}
## Hostnames.
## Must be provided if Ingress is enable.
##
# hosts:
# - grafana.domain.com
hosts:
- mysb.grafanasite.com
## Path for grafana ingress
path: /grafana/
I see same values being reflected in grafana.ini file inside Grafana container mount path(/etc/grafana/grafana.ini).
[analytics]
check_for_updates = true
[grafana_net]
url = https://grafana.net
[log]
mode = console
[paths]
data = /var/lib/grafana/data
logs = /var/log/grafana
plugins = /var/lib/grafana/plugins
provisioning = /etc/grafana/provisioning
[server]
domain = mysb.grafanasite.com
root_url = https://mysb.grafanasite.com/grafana/
serve_from_sub_path = true
you need to edit from parent charts values.yaml
get default values.yaml from kube-prometheus-stack chart, save to file
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add stable https://kubernetes-charts.storage.googleapis.com/
helm repo update
helm show values prometheus-community/kube-prometheus-stack > values.yaml
in values.yaml file, edit like this :
## Using default values from https://github.com/grafana/helm-charts/blob/main/charts/grafana/values.yaml
##
#### This below line is in 509 line
grafana:
enabled: true
namespaceOverride: ""
## Deploy default dashboards.
##
defaultDashboardsEnabled: true
adminPassword: prom-operator
ingress:
## If true, Grafana Ingress will be created
##
enabled: true
## Annotations for Grafana Ingress
##
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
## Labels to be added to the Ingress
##
labels: {}
## Hostnames.
## Must be provided if Ingress is enable.
##
# hosts:
# - grafana.domain.com
hosts:
- myorg.grafanatest.com
## Path for grafana ingress
path: /
grafana.ingress.enabled to true
grafana.ingress.hosts add - myorg.grafanatest.com
Apply it with
helm -n monitoring install -f ./values.yaml kube-prometheus prometheus-community/kube-prometheus-stack
Hopefully help you
Update your grafana.ini config like this:
The grafana.ini can mostly be found under grafana config map
kubectl get cm
kubectl edit cm map_name
**data:
grafana.ini: |
[server]
serve_from_sub_path = true
domain = ingress-gateway.yourdomain.com
root_url = http://ingress-gateway.yourdomain.com/grafana/**
This grafana.ini is mostly saved under the config map or YAML files which can be edited.
reapply or edit the rules & create a mapping in the ingress, that should work.
Don't forget to restart your pod so that config map changes can be applied!

stable/prometheus-operator - adding persistent grafana dashboards

I am trying to add a new dashboard to the below helm chart
https://github.com/helm/charts/tree/master/stable/prometheus-operator
The documentation is not very clear.
I have added a config map to the name space like the below -
apiVersion: v1
kind: ConfigMap
metadata:
name: sample-grafana-dashboard
namespace: monitoring
labels:
grafana_dashboard: "1"
data:
etcd-dashboard.json: |-
{JSON}
According to the documentation, this should just be "picked" up and added, but its not.
https://github.com/helm/charts/tree/master/stable/grafana#configuration
The sidecar option in my values.yaml looks like -
grafana:
enabled: true
## Deploy default dashboards.
##
defaultDashboardsEnabled: true
adminPassword: password
ingress:
## If true, Grafana Ingress will be created
##
enabled: false
## Annotations for Grafana Ingress
##
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
## Labels to be added to the Ingress
##
labels: {}
## Hostnames.
## Must be provided if Ingress is enable.
##
# hosts:
# - grafana.domain.com
hosts: []
## Path for grafana ingress
path: /
## TLS configuration for grafana Ingress
## Secret must be manually created in the namespace
##
tls: []
# - secretName: grafana-general-tls
# hosts:
# - grafana.example.com
#dashboardsConfigMaps:
#sidecarProvider: sample-grafana-dashboard
sidecar:
dashboards:
enabled: true
label: grafana_dashboard
I have also tried adding this to the value.yml
dashboardsConfigMaps:
- sample-grafana-dashboard
Which, doesn't work.
Does anyone have any experience with adding your own dashboards to this helm chart as I really am at my wits end.
To sum up:
For sidecar you need only one option set to true - grafana.sidecar.dashboards.enabled
Install prometheus-operator witch sidecard enabled:
helm install stable/prometheus-operator --name prometheus-operator --set grafana.sidecar.dashboards.enabled=true --namespace monitoring
Add new dashboard, for example
MongoDB_Overview:
wget https://raw.githubusercontent.com/percona/grafana-dashboards/master/dashboards/MongoDB_Overview.json
kubectl -n monitoring create cm grafana-mongodb-overview --from-file=MongoDB_Overview.json
Now the tricky part, you have to set a correct label for your
configmap, by default grafana.sidecar.dashboards.label is set
tografana_dashboard, so:
kubectl -n monitoring label cm grafana-mongodb-overview grafana_dashboard=mongodb-overview
Now you should find your newly added dashboard in grafana, moreover every confimap with label grafana_dashboard will be processed as dashboard.
The dashboard is persisted and safe, stored in configmap.
UPDATE:
January 2021:
Prometheus operator chart was migrated from stable repo to Prometheus Community Kubernetes Helm Charts and helm v3 was released so:
Create namespace:
kubectl create namespace monitoring
Install prometheus-operator from helm chart:
helm install prometheus-operator prometheus-community/kube-prometheus-stack --namespace monitoring
Add Mongodb dashboard as an example
wget https://raw.githubusercontent.com/percona/grafana-dashboards/master/dashboards/MongoDB_Overview.json
kubectl -n monitoring create cm grafana-mongodb-overview --from-file=MongoDB_Overview.json
Lastly, label the dashboard:
kubectl -n monitoring label cm grafana-mongodb-overview grafana_dashboard=mongodb-overview
You have to:
define you dashboard json as a configmap (as you have done, but see below for an easier way)
define a provider: to tell where to load the dashboard
map the two together
from values.yml:
dashboardsConfigMaps:
application: application
dashboardProviders:
dashboardproviders.yaml:
apiVersion: 1
providers:
- name: application
orgId: 1
folder: "Application Metrics"
type: file
disableDeletion: true
editable: false
options:
path: /var/lib/grafana/dashboards/application
Now the application config map should create files in this directory in the pod, and as has been discussed the sidecar should load them into an Application Metrics folder, seen in the GUI.
That probably answers your issue as written, but as long as your dashboards aren't too big using kustonmise mean you can have the json on disk without needing to include the json in another file thus:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
# May choose to enable this if need to refer to configmaps outside of kustomize
generatorOptions:
disableNameSuffixHash: true
namespace: monitoring
configMapGenerator:
- name: application
files:
- grafana-dashboards/application/api01.json
- grafana-dashboards/application/api02.json
For completeness sake you can also load dashboards from url or from the Grafana site, although I don't believe mixing method in the same folder works.
So:
dashboards:
kafka:
kafka01:
url: https://raw.githubusercontent.com/kudobuilder/operators/master/repository/kafka/docs/latest/resources/grafana-dashboard.json
folder: "KUDO Kafka"
datasource: Prometheus
nginx:
nginx1:
gnetId: 9614
datasource: Prometheus
dashboardProviders:
dashboardproviders.yaml:
apiVersion: 1
providers:
- name: kafka
orgId: 1
folder: "KUDO Kafka"
type: file
disableDeletion: true
editable: false
options:
path: /var/lib/grafana/dashboards/kafka
- name: nginx
orgId: 1
folder: Nginx
type: file
disableDeletion: true
editable: false
options:
path: /var/lib/grafana/dashboards/nginx
Creates two new folders containing a dashboard each, from external sources, or maybe you point this at your git repo you de-couple your dashboard commits from your deployment.
If you do not change the settings in the helm chart. The default user/password for grafana is:
user: admin
password: prom-operator

How to use ConfigMap configuration with Helm NginX Ingress controller - Kubernetes

I've found a documentation about how to configure your NginX ingress controller using ConfigMap: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/
Unfortunately I've no idea and couldn't find it anywhere how to load that ConfigMap from my Ingress controller.
My ingress controller:
helm install --name ingress --namespace ingress-nginx --set rbac.create=true,controller.kind=DaemonSet,controller.service.type=ClusterIP,controller.hostNetwork=true stable/nginx-ingress
My config map:
kind: ConfigMap
apiVersion: v1
metadata:
name: ingress-configmap
data:
proxy-read-timeout: "86400s"
client-max-body-size: "2g"
use-http2: "false"
My ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
tls:
- hosts:
- my.endpoint.net
secretName: ingress-tls
rules:
- host: my.endpoint.net
http:
paths:
- path: /
backend:
serviceName: web
servicePort: 443
- path: /api
backend:
serviceName: api
servicePort: 443
How do I make my Ingress to load the configuration from the ConfigMap?
I've managed to display what YAML gets executed by Helm using the: --dry-run --debug options at the end of helm install command. Then I've noticed that there controller is executed with the: --configmap={namespace-where-the-nginx-ingress-is-deployed}/{name-of-the-helm-chart}-nginx-ingress-controller.
In order to load your ConfigMap you need to override it with your own (check out the namespace).
kind: ConfigMap
apiVersion: v1
metadata:
name: {name-of-the-helm-chart}-nginx-ingress-controller
namespace: {namespace-where-the-nginx-ingress-is-deployed}
data:
proxy-read-timeout: "86400"
proxy-body-size: "2g"
use-http2: "false"
The list of config properties can be found here.
One can pass config mag properties at the time of installation too:
helm install stable/nginx-ingress --name nginx-ingress --set controller.config.use-forwarded-headers='"true"'
NOTE: for non-string values had to use single quotes around double quotes to get it working.
If you used helm install to install the ingress-nginx, if no explicit value for which ConfigMap the nginx controller should look at was passed, the default value seems like it is {namespace}/{release-name}-nginx-ingress-controller. This is generated by https://github.com/helm/charts/blob/1e074fc79d0f2ee085ea75bf9bacca9115633fa9/stable/nginx-ingress/templates/controller-deployment.yaml#L67. (See similar if it's a dead link).
To verify for yourself, try to find your command that you installed the ingress-nginx chart with, and add --dry-run --debug to the command. This will show you the yaml files generated by Tiller to be applied to the cluster. The line # Source: nginx-ingress/templates/controller-deployment.yaml begins the controller deployment which has an arg of --configmap=. The value of this arg is what needs to be the name of the ConfigMap for the controller to sense, and use to update its own .conf file. This could be passed explicitly, but if it is not, it will have a default value.
If a ConfigMap is created with the RIGHT name, the controller's logs will show that it picked up the configuration change and reloaded itself.
This can be verified with kubectl logs <pod-name-of-controller> -n <namespace-arg-if-not-in-default-namespace>. My log messages contained the text Configuration changes detected, backend reload required. These log messages will not be present if the ConfigMap name was wrong.
I believe the official documentation for this is unnecessarily lacking, but maybe I'm incorrect? I will try to submit a PR with these details. Someone who knows more should help flesh them out so people don't need to stumble on this unnecessarily.
Cheers, thanks for your post.
If you want to give your own configuration while deploying nginx-ingress-controller, you can have a wrapper Helm chart over the original nginx-ingress Helm chart and provide your own values.yaml which can have custom configuration.
Using Helm 3 here.
Create a chart:
$ helm create custom-nginx
$ tree custom-nginx
So my chart structure looks like this:
custom-nginx/
├── Chart.yaml
├── charts
├── templates
│   ├── NOTES.txt
│   ├── _helpers.tpl
│   ├── deployment.yaml
│   ├── hpa.yaml
│   ├── ingress.yaml
│   ├── service.yaml
│   ├── serviceaccount.yaml
│   └── tests
│   └── test-connection.yaml
└── values.yaml
There are a few extra things here. Specifically, I don't need the complete templates/ directory and its contents, so I'll just remove those:
$ rm custom-nginx/templates/*
$ rmdir custom-nginx/templates
Now, the chart structure should look like this:
custom-nginx/
├── Chart.yaml
├── charts
└── values.yaml
Since, we've to include the original nginx-ingress chart as a dependency, my Chart.yaml looks like this:
$ cat custom-nginx/Chart.yaml
apiVersion: v2
name: custom-nginx
description: A Helm chart for Kubernetes
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 1.39.1
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
appVersion: 0.32.0
dependencies:
- name: nginx-ingress
version: 1.39.1
repository: https://kubernetes-charts.storage.googleapis.com/
Here, appVersion is the nginx-controller docker image version and version matches with the nginx-ingress chart version that I am using.
The only thing left is to provide your custom configuration. Here is an stripped down version of my custom configuration:
$ cat custom-nginx/values.yaml
# Default values for custom-nginx.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
nginx-ingress:
controller:
ingressClass: internal-nginx
replicaCount: 1
service:
externalTrafficPolicy: Local
publishService:
enabled: true
autoscaling:
enabled: true
minReplicas: 1
maxReplicas: 3
targetCPUUtilizationPercentage: "80"
targetMemoryUtilizationPercentage: "80"
resources:
requests:
cpu: 1
memory: 2Gi
limits:
cpu: 1
memory : 2Gi
metrics:
enabled: true
config:
compute-full-forwarded-for: "true"
We can check the keys that are available to use as configuration (config section in values.yaml) in https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/
And the rest of the configuration can be found here: https://github.com/helm/charts/tree/master/stable/nginx-ingress#configuration
Once configurations are set, just download the dependency of your chart:
$ helm dependency update <path/to/chart>
It's a good practice to do basic checks on your chart before deploying it:
$ helm lint <path/to/chart>
$ helm install --debug --dry-run --namespace <namespace> <release-name> <path/to/chart>
Then deploy your chart (which will deploy your nginx-ingress-controller with your own custom configurations).
Also, since you've a chart now, you can upgrade and rollback your chart.
When installing the chart through terraform, the configuration values can be set as shown below:
resource "helm_release" "ingress_nginx" {
name = "nginx"
repository = "https://kubernetes.github.io/ingress-nginx/"
chart = "ingress-nginx"
set {
name = "version"
value = "v4.0.2"
}
set {
name = "controller.config.proxy-read-timeout"
value = "86400s"
}
set {
name = "controller.config.client-max-body-size"
value = "2g"
}
set {
name = "controller.config.use-http2"
value = "false"
}
}
Just to confirm #NeverEndingQueue answer above, the name of the config map is present in the nginx-controller pod spec itself, so if you inspect the yaml of the nginx-controller pod: kubectl get po release-name-nginx-ingress-controller-random-sequence -o yaml, under spec.containers, you will find something like:
- args:
- /nginx-ingress-controller
- --default-backend-service=default/release-name-nginx-ingress-default-backend
- --election-id=ingress-controller-leader
- --ingress-class=nginx
- --configmap=default/release-name-nginx-ingress-controller
For example here, a config map named release-name-nginx-ingress-controller in the namespace default needs to be created.
Once done, you can verify if the changes have taken place by checking the logs. Normally, you will see something like:
I1116 10:35:45.174127 6 event.go:278] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"default", Name:"release-name-nginx-ingress-controller", UID:"76819abf-4df0-41e3-a3fe-25445e754f32", APIVersion:"v1", ResourceVersion:"62559702", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap default/release-name-nginx-ingress-controller
I1116 10:35:45.184627 6 controller.go:141] Configuration changes detected, backend reload required.
I1116 10:35:45.396920 6 controller.go:157] Backend successfully reloaded.
When you apply ConfigMap configuration with needful key-value data, Ingress controller picks up this information and insert it to the nested nginx-ingress-controller Pod's original configuration file /etc/nginx/nginx.conf, therefore it's easy afterwards to verify whether ConfigMap's values have been successfully reflected or not, by checking actual nginx.conf inside the corresponded Pod.
You can also check logs from the relevant nginx-ingress-controller Pod in order to check whether ConfigMap data already reloaded to the backend nginx.conf, or if not to investigate the reason.
Using enable-underscores-in-headers=true worked for me not enable-underscores-in-headers='"true"'
helm install nginx-ingress ingress-nginx/ingress-nginx
--namespace ingress-basic
--set controller.config.enable-underscores-in-headers=true
I managed to update the "large-client-header-buffers" in the nginx via configmap. Here are the steps I have followed..
Find the configmap name in the nginx ingress controller pod describition
kubectl -n utility describe pods/test-nginx-ingress-controller-584dd58494-d8fqr |grep configmap
--configmap=test-namespace/test-nginx-ingress-controller
Note: In my case, the namespace is "test-namespace" and the configmap name is "test-nginx-ingress-controller"
Create a configmap yaml
cat << EOF > test-nginx-ingress-controller-configmap.yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: test-nginx-ingress-controller
namespace: test-namespace
data:
large-client-header-buffers: "4 16k"
EOF
Note: Please replace the namespace and configmap name as per finding in the step 1
Deploy the configmap yaml
kubectl apply -f test-nginx-ingress-controller-configmap.yaml
Then you will see the change is updated to nginx controller pod after mins
i.g.
kubectl -n test-namespace exec -it test-nginx-ingress-controller-584dd58494-d8fqr -- cat /etc/nginx/nginx.conf|grep large
large_client_header_buffers 4 16k;
Based on the NeverEndingQueue's answer I want to provide an update for Kubernetes v1.23 / Helm 3
This is my installation command + --dry-run --debug part:
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx --create-namespace --dry-run --debug
This is the part we need from the generated output of the command above:
apiVersion: apps/v1
kind: DaemonSet
metadata:
...
spec:
...
template:
...
spec:
dnsPolicy: ClusterFirst
containers:
- name: controller
...
args:
- /nginx-ingress-controller
- --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
- --...
- --configmap=${POD_NAMESPACE}/ingress-nginx-controller
- --...
....
We need this part: --configmap=${POD_NAMESPACE}/ingress-nginx-controller.
As you can see, name of ConfigMap must be ingress-nginx-controller and namespace must be the one you use during chart installation (ie {POD_NAMESPACE}, in my example about this is --namespace ingress-nginx).
# nginx-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: ingress-nginx-controller
namespace: ingress-nginx
data:
map-hash-bucket-size: "128"
Then run kubectl apply -f nginx-config.yaml to apply ConfigMap and nginx's pod(s) will be auto-reloaded with updated config.
To check, that nginx config has been updated, find name of nginx's pod (you can use any one, if you have few nodes): kubectl get pods -n ingress-nginx (or kubectl get pods -A)
and then check config: kubectl exec -it ingress-nginx-controller-{generatedByKubernetesId} -n ingress-nginx cat /etc/nginx/nginx.conf
UPDATE:
The correct name (ie name: ingress-nginx-controller) is shown in the official docs. Conclusion: no need to reinvent the wheel.
What you have is an ingress yaml and not an Ingress controller deployment yaml , Ingress Controller is the Pod that actually does the work and usually is an nginx container itself. An example of such a configuration can be found here in the documentation you shared.
UPDATE
Using that example provided , you can also use following way to load config into nginx using config map
volumeMounts:
- name: nginx-config
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
volumes:
- name: nginx-config
configMap:
name: nginx-config
nginx-config contains your nginx configuration as part of config map
I read the above answers but could not make it work.
What worked for me was the following:
release_name=tcp-udp-ic
# add the helm repo from NginX and update the chart
helm repo add nginx-stable https://helm.nginx.com/stable
helm repo update
echo "- Installing -${release_name}- into cluster ..."
#delete the config map if already exists
kubectl delete cm tcp-udp-ic-cm
helm del --purge ${release_name}
helm upgrade --install ${release_name} \
--set controller.image.tag=1.6.0 \
--set controller.config.name=tcp-udp-ic-cm \
nginx-stable/nginx-ingress --version 0.4.0 #--dry-run --debug
# update the /etc/nginx/nginx.conf file with my attributes, via the config map
kubectl apply -f tcp-udp-ic-cm.yaml
and the tcp-udp-ic-cm.yaml is :
kind: ConfigMap
apiVersion: v1
metadata:
name: tcp-udp-ic-cm
namespace: default
data:
worker-connections : "10000"
Essentially I need to deploy with helm the release and set the name of the config-map that is going to use. Helm creates the config-map but empty. Then I apply the config-map file in order to update the config-map resource with my values. This sequence is the only one i could make work.
An easier way of doing this is just modifying the values that's deployed through helm. The values needed to be changed to enter to ConfigMap are now in controller.config.entries. Show latest values with: helm show values nginx-stable/nginx-ingress and look for the format on the version you are running.
I had tons of issues with this since all references online said controller.config, until I checked with the command above.
After you've entered the values upgrade with:
helm upgrade -f <PATH_TO_FILE>.yaml <NAME> nginx-stable/nginx-ingress
The nginx ingress controller may cause issues with forwarding. While we were able to get it working with nginx, via X-Forwarded-Proto etc., but it was a bit complicated and convoluted.
Moving to haproxy instead resolved this problem. As well, make sure you are interfacing with the ingress controller over https or that may cause issues with keycloak.
Keycloak v18 with --proxy edge
annotations:
kubernetes.io/ingress.class: haproxy
...