I am building a monitoring environment with data dock in GKE environment.
I have configured datadog helm-chart with logs set to enabled. At this time, doesn't the Datadog agent automatically get the logs of all pods on the Node?
As shown below, it seems that logs are being collected only when the annotation must be attached to the Pods you want to collect logs from
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "chart.fullname" . }}
labels:
{{- include "chart.labels" . | nindent 4 }}
tags.datadoghq.com/env: "test-env"
tags.datadoghq.com/service: "test-service"
tags.datadoghq.com/version: 0.0.1
spec:
....
In this case, what if you want to automatically get the logs of all Pods floating in the cluster?
I tried allowing datadog agent to work and show logs dashboard in datadog web
try to enable in datadog values file under logs part-
logs
Related
I have a cluster with many namespaces.
I'm trying to log data from a specific namespace in my Openshift cluster but it is logging the data from all the namespaces. I tried to follow the documentation of the Openshift regarding logging, but there is no mention of scoping the log data.
I followed this documentation:
https://docs.openshift.com/container-platform/4.7/logging/cluster-logging.html
I'm using fluentd as the log collector.
As Cluster Logging on OpenShift, you can transfer logs in namespaces or Pods matched label you select.
The sample CR like Forward logs in my-project namespace to Elasticserach which is deployed by Cluster Logging could be as follows:
apiVersion: "logging.openshift.io/v1"
kind: ClusterLogForwarder
metadata:
name: instance
namespace: openshift-logging
spec:
inputs:
- name: my-app-logs
application:
namespaces:
- my-project
pipelines:
- name: my-app
inputRefs:
- my-app-logs
outputRefs:
- default
You can customize inputs field as you want. It also could be specified Pods using matchLabels expression. *2
outputs default means send logs to default Elasticsearch on Cluster Logging.
*1: https://docs.openshift.com/container-platform/4.11/logging/cluster-logging-external.html
*2: https://docs.openshift.com/container-platform/4.7/logging/cluster-logging-external.html#cluster-logging-collector-log-forward-logs-from-application-pods_cluster-logging-external
i'm trying to assign pods to a specific node as part of helm command, so by the end the deployment yaml should look like this
spec:
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
nodeSelector:
node-name: dev-cpu-pool
i'm using this command as part of Jenkins file deployment
`sh "helm upgrade -f charts/${job_name}/default.yaml --set nodeSelector.name=${deployNamespace}-cpu-pool --install ${deployNamespace}-${name} helm/${name} --namespace=${deployNamespace} --recreate-pods --version=${version}`"
the deployment works good and the pod is up and running but from some reason i cannot see the nodeSelector key and value as part of the deployment yaml and as a results pods not assign to the specific node i want. any idea what is wrong ? should i put any place holder as part of my chart template or is not must ?
The artifacts that Helm submits to the Kubernetes API are exactly the result of rendering the chart templates; nothing more, nothing less. If your templates don't include a nodeSelector: block then the resulting Deployment never will either. Even if you helm install --set ... things that could match Kubernetes API fields, nothing will implicitly fill them in.
If you want an option to specify rarely-used fields like nodeSelector: then your chart code needs to include them. You can make the presence of the field conditional on the value being set, but you do need to explicitly list it out:
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
{{- if .Values.nodeSelector }}
nodeSelector: {{- .Values.nodeSelector | toYaml | nindent 8 }}
{{- end }}
What field should I add to the service / ingress yaml so that I can reach the service from another pod in the same cluster using its associated (external) hostname specified in ingress?
I'm using microk8s with the default ingress class (nginx), and I need a solution that works in any kubernetes platform (azure, gke, aks)
I need to reach my authentication server (keycloak) from my nodejs application, using ingress hostname. I can't use service name, because the token validation would fail (JWT ISS checking).
thanks!
Based on this SO post this can be done using Helm custom values and hostAliases.
A helm templated solution to the original question. I tested this with
helm 3.
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
{{- with .Values.hostAliases }}
hostAliases:
{{ toYaml . | indent 8 }}
{{- end }}
For values such as:
hostAliases:
- ip: "10.0.0.1"
hostnames:
- "host.domain.com"
If the hostAliases is omitted or commented out in the values, the
hostAliases section is omitted when the template is rendered.
I have a helm chart that deploys a number of Kubernetes resources. One of them is a resource that is of a Custom Resource Definition (CRD) type (ServiceMonitor used by prometheus-operator).
I am looking for a way, how to "tell" helm that I'd want to create this resource only if such a CRD is defined in the cluster OR to ignore errors only caused by the fact that such a CRD is missing.
Is that possible and how can I achieve that?
Helm's Capabilities object can tell you if an entire API class is installed in the cluster. I don't think it can test for a specific custom resource type.
In your .tpl files, you can wrap the entire file in a {{ if }}...{{ end }} block. Helm doesn't especially care if the rendered version of a file is empty.
That would lead you to a file like:
{{ if .Capabilities.APIVersions.Has "monitoring.coreos.com/v1" -}}
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
...
{{ end -}}
That would get installed if the operator is installed in the cluster, and skipped if not.
If you are on Helm 3 you can put your CRD in the crds/ directory. Helm will treat it differently, see the docs here.
In Helm 2 there is another mechanism using the crd-install hook. You can add the following to your CRD:
annotations:
"helm.sh/hook": crd-install
There are some limitations with this approach so if you are using Helm 3 that would be preferred.
In Helm v3, you can test for specific resources:
{{ if .Capabilities.APIVersions.Has "monitoring.coreos.com/v1/ServiceMonitor" -}}
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
...
spec:
...
{{- end }}
https://helm.sh/docs/chart_template_guide/builtin_objects/
We Could deploy applications using 'Helm Charts' with
helm install --name the-release helm/the-service-helm --namespace myns
And we cold 'Rolling Upgrade' the deployment using,
helm upgrade --recreate-pods the-release helm/the-service-helm --namespace myns
Is there a way to use 'Helm Charts' to achieve 'Blue/Green' Deployments?
Let's start from definitions
Since there are many deployment strategies, let's start from the definition.
As per Martin Flower:
The blue-green deployment approach does this by ensuring you have two production environments, as identical as possible. At any time one of them, let's say blue for the example, is live. As you prepare a new release of your software you do your final stage of testing in the green environment. Once the software is working in the green environment, you switch the router so that all incoming requests go to the green environment - the blue one is now idle.
Blue/Green is not recommended in Helm. But there are workaround solutions
As per to helm issue #3518, it's not recommended to use Helm for blue/green or canary deployment.
There are at least 3 solutions based on top of Helm, see below
However there is a Helm chart for that case.
Helm itself (TL;DR: not recommended)
Helm itself is not intended for the case. See their explanation:
direct support for blue / green deployment pattern in helm · Issue #3518 · helm/helm
Helm works more in the sense of a traditional package manager, upgrading charts from one version to the next in a graceful manner (thanks to pod liveness/readiness probes and deployment update strategies), much like how one expects something like apt upgrade to work. Blue/green deployments are a very different beast compared to the package manager style of upgrade workflows; blue/green sits at a level higher in the toolchain because the use cases around these deployments require step-in/step-out policies, gradual traffic migrations and rollbacks. Because of that, we decided that blue/green deployments are something out of scope for Helm, though a tool that utilizes Helm under the covers (or something parallel like istio) could more than likely be able to handle that use case.
Other solutions based on Helm
There are at least three solution based on top of Helm, described and compared here:
Shipper
Istio
Flagger.
Shipper by Booking.com - DEPRECATED
bookingcom/shipper: Kubernetes native multi-cluster canary or blue-green rollouts using Helm
It does this by relying on Helm, and using Helm Charts as the unit of configuration deployment. Shipper's Application object provides an interface for specifying values to a Chart just like the helm command line tool.
Shipper consumes Charts directly from a Chart repository like ChartMuseum, and installs objects into clusters itself. This has the nice property that regular Kubernetes authentication and RBAC controls can be used to manage access to Shipper APIs.
Kubernetes native multi-cluster canary or blue-green rollouts using Helm
Istio
You can try something like this:
kubectl create -f <(istioctl kube-inject -f cowsay-v1.yaml) # deploy v1
kubectl create -f <(istioctl kube-inject -f cowsay-v2.yaml) # deploy v1
Flagger.
There is guide written by Flagger team: Blue/Green Deployments - Flagger
This guide shows you how to automate Blue/Green deployments with Flagger and Kubernetes
You might try Helm itself
Also, as Kamol Hasan recommended, you can try that chart: puneetsaraswat/HelmCharts/blue-green.
blue.yml sample
{{ if .Values.blue.enabled }}
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: {{ template "blue-green.fullname" . }}-blue
labels:
release: {{ .Release.Name }}
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
app: {{ template "blue-green.name" . }}
spec:
replicas: {{ .Values.replicaCount }}
template:
metadata:
labels:
app: {{ template "blue-green.name" . }}
release: {{ .Release.Name }}
slot: blue
spec:
containers:
- name: {{ template "blue-green.name" . }}-blue
image: nginx:stable
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 80
protocol: TCP
# This (and the volumes section below) mount the config map as a volume.
volumeMounts:
- mountPath: /usr/share/nginx/html
name: wwwdata-volume
volumes:
- name: wwwdata-volume
configMap:
name: {{ template "blue-green.fullname" . }}
{{ end }}
Medium blog post: Blue/Green Deployments using Helm Charts