on kubernetes vm Im running for example : kubectl get endpoints
how can I get the same output inside the pod , what should I run within a pod?
I understood there is a kubeapi but Im new to kubernetes can someone explain how can I use it
this is my clusterrolebinding:
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: {{ template "elasticsearch.fullname" . }}
labels:
app: {{ template "elasticsearch.name" . }}
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
subjects:
- kind: ServiceAccount
name: {{ template "elasticsearch.serviceAccountName.client" . }}
namespace: {{ .Release.Namespace }}
roleRef:
kind: ClusterRole
name: {{ template "elasticsearch.fullname" . }}
apiGroup: rbac.authorization.k8s.io
clusterrole.yaml:
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: {{ template "elasticsearch.fullname" . }}
labels:
app: {{ template "elasticsearch.name" . }}
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
rules:
#
# Give here only the privileges you need
#
- apiGroups: [""]
resources:
- pods
- endpoints
verbs:
- get
- watch
- list
serviceaccount:
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app: {{ template "elasticsearch.name" . }}
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
component: "{{ .Values.client.name }}"
heritage: {{ .Release.Service }}
release: {{ .Release.Name }}
name: {{ template "elasticsearch.client.fullname" . }}
You don't have to have kubectl installed in pod to access the Kubernetes API. You will be ableto do it with any tool that can make HTTP requests.
The Kubernetes API is a simple HTTP REST API, and all the authentication information that you need is present in the container if it runs as a Pod in the cluster.
To get the Endpoints object named your-service from within a container in the cluster, you can do:
$ curl -k -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
https://kubernetes.default.svc:443/api/v1/namespaces/{namespace}/endpoints/your-service
Replace {namespace} with the namespace of the your-service Endpoints resource._
To extract the IP addresses of the returned JSON pipe the output to a tool like jq:
... | jq -r '.subsets[].addresses[].ip'
IMPORTANT:
The Pod from which you are executing this needs read permissions for the Endpoints resource, otherwise the API request will be denied.
You can do this by creating a ClusterRole, ClusterRoleBinding, and Service Account - set this up once:
$ kubectl create sa endpoint-reader-sa
$ kubectl create clusterrole endpoint-reader-cr --verb=get,list --resource=endpoints
$ kubectl create clusterrolebinding endpoint-reader-crb --serviceaccount=default:endpoint-reader-sa --clusterrole=endpoint-reader-cr
Next use created ServiceAccount - endpoint-reader-sa for the Pod from which you want to execute the above curl command by specifying it in the pod.spec.serviceAccountName field.
Granting permissions for any different API operations works in the same way.
Source: get-pod-ip.
And as also #ITChap mentioned similar answer: kubectl-from-inside-the-pod.
Related
I added this in my values.yaml expecting the serviceAccount to be created when I do the helm install but that did not work, am I missing something ?
helm version v3.9.0
kubernetes version v1.24.0
serviceAccount:
# Specifies whether a service account should be created
create: true
# Annotations to add to the service account
annotations: {}
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: my-service-account
I even tried adding the following (based on https://helm.sh/docs/chart_best_practices/rbac/#helm), with no luck:
rbac:
# Specifies whether RBAC resources should be created
create: true
Thanks
Thanks for the help, I ended up putting this file in the templates directory so it gets processed as you mentioned, I used helm lookup to check if the ServiceAccount is there or not so the first helm install does the installation (https://helm.sh/docs/chart_template_guide/functions_and_pipelines/#using-the-lookup-function)
{{- if not (lookup "v1" "ServiceAccount" "my-namespace" "my-service-account") }}
apiVersion: v1
kind: ServiceAccount
metadata:
name: my-service-account
namespace: {{ .Values.namespace }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: my-cluster-role
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: my-cluster-role-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: my-cluster-role
subjects:
- kind: ServiceAccount
name: my-service-account
namespace: {{ .Values.namespace }}
{{- end }}
You have to create the YAML or helm template into your template directory and helm will create/apply that config file to the K8s cluster.
service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app: {{ template "elasticsearch.name" . }}
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
heritage: {{ .Release.Service }}
release: {{ .Release.Name }}
name: {{ template "elasticsearch.fullname" . }}
Ref :https://github.com/CenterForOpenScience/helm-charts/blob/master/elasticsearch/templates/service-account.yaml
You can add your conditions accordingly to check if create is true or false etc.
Condition or flow control doc : https://helm.sh/docs/chart_template_guide/control_structures/
While deploying a Kubernetes application, I want to check if a resource is already present. If so it shall not be rendered. To archive this behaviour the lookup function of helm is used. As it seems is always empty while deploying (no dry-run). Any ideas what I am doing wrong?
---
{{- if not (lookup "v1" "ServiceAccount" "my-namespace" "my-sa") }}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ .Chart.Name }}-{{ .Values.environment }}
namespace: {{ .Values.namespace }}
labels:
app: {{ $.Chart.Name }}
environment: {{ .Values.environment }}
annotations:
"helm.sh/resource-policy": keep
iam.gke.io/gcp-service-account: "{{ .Chart.Name }}-{{ .Values.environment }}#{{ .Values.gcpProjectId }}.iam.gserviceaccount.com"
{{- end }}
running the corresponding kubectl command return the expected service account
kubectl get ServiceAccount my-sa -n my-namespace lists the expected service account
helm version: 3.5.4
i think you cannot use this if-statement to validate what you want.
the lookup function returns a list of objects that were found by your lookup. so, if you want to validate that there are no serviceaccounts with the properties you specified, you should check if the returned list is empty.
test something like
---
{{ if eq (len (lookup "v1" "ServiceAccount" "my-namespace" "my-sa")) 0 }}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ .Chart.Name }}-{{ .Values.environment }}
namespace: {{ .Values.namespace }}
labels:
app: {{ $.Chart.Name }}
environment: {{ .Values.environment }}
annotations:
"helm.sh/resource-policy": keep
iam.gke.io/gcp-service-account: "{{ .Chart.Name }}-{{ .Values.environment }}#{{ .Values.gcpProjectId }}.iam.gserviceaccount.com"
{{- end }}
see: https://helm.sh/docs/chart_template_guide/functions_and_pipelines/#using-the-lookup-function
I've created the following to objects:
apiVersion: v1
kind: Service
metadata:
name: {{ .Values.serviceName }}
namespace: {{ .Values.global.namespace }}
labels:
chart: {{ template "chartName" . }}
env: {{ .Values.global.env }}
annotations:
"helm.sh/hook": "pre-install"
"helm.sh/hook-weight": "10"
"helm.sh/hook-delete-policy": "before-hook-creation"
spec:
ports:
- port: {{ .Values.postgres.port}}
selector: {}
for a service and its endpoint:
kind: Endpoints
apiVersion: v1
metadata:
name: {{ .Values.serviceName }}
namespace: {{ .Values.global.namespace }}
labels:
chart: {{ template "chartName" . }}
env: {{ .Values.global.env }}
annotations:
"helm.sh/hook": "pre-install"
"helm.sh/hook-weight": "10"
"helm.sh/hook-delete-policy": "before-hook-creation"
subsets:
- addresses:
- ip: "{{ .Values.external.ip }}"
ports:
- name: "db"
port: {{ .Values.external.port }}
When I use helm even in a dry run mode I can see the service object and cant see the endpoint object.
Why? Doesn't helm support all k8s objects?
Helm is just a "templating" tool, so technically it supports everything that your underlying k8 supports.
In your case, please check that both files are in the templates directory
Actually it does work. The problem was that the service and the endpoint MUST have same names (which I new) and MUST have port names exactly the same
I am new to helm and I have tried to deploy a few tutorial charts. Had a couple of queries:
I have a Kubernetes job which I need to deploy. Is it possible to deploy a job via helm?
Also, currently my kubernetes job is deployed from my custom docker image and it runs a bash script to complete the job. I wanted to pass a few parameters to this chart/job so that the bash commands takes the input parameters. That's the reason I decided to move to helm because it provided a more flexibility. Is that possible?
You can use helm. Helm installs all the kubernetes resources like job,pods,configmaps,secrets inside the templates folder. You can control the order of installation by helm hooks. Helm offers hooks like pre-install, post-install, pre-delete with respect to deployment. if two or more jobs are pre-install then their weights will be compared for installing.
|-scripts/runjob.sh
|-templates/post-install.yaml
|-Chart.yaml
|-values.yaml
Many times you need to change the variables in the script as per the environment. so instead of hardcoding variable in script, you can also pass parameters to script by setting them as environment variables to your custom docker image. Change the values in values.yaml instead of changing in your script.
values.yaml
key1:
someKey1: value1
key2:
someKey2: value1
post-install.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: post-install-job
labels:
provider: stackoverflow
microservice: {{ template "name" . }}
release: "{{ .Release.Name }}"
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
annotations:
"helm.sh/hook": pre-install,pre-upgrade,pre-rollback
"helm.sh/hook-delete-policy": before-hook-creation
"helm.sh/hook-weight": "3"
spec:
template:
metadata:
name: "{{.Release.Name}}"
labels:
provider: stackoverflow
microservice: {{ template "name" . }}
release: "{{ .Release.Name }}"
app: {{ template "fullname" . }}
spec:
restartPolicy: Never
containers:
- name: post-install-job
image: "custom-docker-image:v1"
command: ["/bin/sh", "-c", {{ .Files.Get "scripts/runjob.sh" | quote }} ]
env:
#setting KEY1 as environment variable in the container,value of KEY1 in container is value1(read from values.yaml)
- name: KEY1
value: {{ .Values.key1.someKey1 }}
- name: KEY2
value: {{ .Values.key2.someKey2 }}
runjob.sh
# you can access the variable from env variable
echo $KEY1
echo $KEY2
# some stuff
You can use Helm Hooks to run jobs. Depending on how you set up your annotations you can run a different type of hook (pre-install, post-install, pre-delete, post-delete, pre-upgrade, post-upgrade, pre-rollback, post-rollback, crd-install). An example from the doc is as follows:
apiVersion: batch/v1
kind: Job
metadata:
name: "{{.Release.Name}}"
labels:
app.kubernetes.io/managed-by: {{.Release.Service | quote }}
app.kubernetes.io/instance: {{.Release.Name | quote }}
helm.sh/chart: "{{.Chart.Name}}-{{.Chart.Version}}"
annotations:
# This is what defines this resource as a hook. Without this line, the
# job is considered part of the release.
"helm.sh/hook": post-install
"helm.sh/hook-weight": "-5"
"helm.sh/hook-delete-policy": hook-succeeded
spec:
template:
metadata:
name: "{{.Release.Name}}"
labels:
app.kubernetes.io/managed-by: {{.Release.Service | quote }}
app.kubernetes.io/instance: {{.Release.Name | quote }}
helm.sh/chart: "{{.Chart.Name}}-{{.Chart.Version}}"
spec:
restartPolicy: Never
containers:
- name: post-install-job
image: "alpine:3.3"
command: ["/bin/sleep","{{default "10" .Values.sleepyTime}}"]
You can pass your parameters as secrets or configMaps to your job as you would to a pod.
I had a similar scenario where I had a job I wanted to pass a variety of arguments to. I ended up doing something like this:
Template:
apiVersion: batch/v1
kind: Job
metadata:
name: myJob
spec:
template:
spec:
containers:
- name: myJob
image: myImage
args: {{ .Values.args }}
Command (powershell):
helm template helm-chart --set "args={arg1\, arg2\, arg3}" | kubectl apply -f -
i.e. from name: {{ .Chart.Name }}-{{ .Values.module5678.name }}-pod below
# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: {{ template "project1234.name" . }}
chart: {{ template "project1234.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
name: {{ template "project1234.module5678.fullname" . }}
spec:
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
app: {{ template "project1234.name" . }}
template:
metadata:
labels:
app: {{ template "project1234.name" . }}
spec:
containers:
- image: "{{ .Values.image.name }}:{{ .Values.image.tag }}"
name: {{ .Chart.Name }}-{{ .Values.module5678.name }}-pod
ports:
- containerPort: 1234
imagePullSecrets:
- name: {{ .Values.image.pullSecret }}
I am expecting the pod name to be:
pod/project1234-module5678-pod
Instead, the resulting Pod name is:
pod/chartname-project1234-module5678-dc7db787-skqvv
...where (in my understanding):
chartname is from: helm install --name chartname -f values.yaml .
project1234 is from:
# Chart.yaml
apiVersion: v1
appVersion: "1.0"
description: project1234 Helm chart for Kubernetes
name: project1234
version: 0.1.0
module5678 is from:
# values.yaml
rbac:
create: true
serviceAccounts:
module5678:
create: true
name:
image:
name: <image location>
tag: 1.5
pullSecret: <pull secret>
gitlab:
secretName: <secret name>
username: foo
password: bar
module5678:
enabled: true
name: module5678
ingress:
enabled: true
replicaCount: 1
resources: {}
I've tried changing name: {{ .Chart.Name }}-{{ .Values.module5678.name }}-pod into a plain string value like "podname1234" and it isn't followed. I even tried removing the name setting entirely and the resulting pod name remains the same.
Pods created from a Deployment always have a generated name based on the Deployment's name (and also the name of the intermediate ReplicaSet, if you go off and look for it). You can't override it.
Given the YAML you've shown, I'd expect that this fragment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ template "project1234.module5678.fullname" . }}
expands out to a Deployment name of chartname-project1234-module5678; the remaining bits are added in by the ReplicaSet and then the Pod itself.
If you do look up the Pod and kubectl describe pod chartname-project1234-module5678-dc7db787-skqvv you will probably see that it has a single container that has the expected name project1234-module5678-pod. Pretty much the only time you need to use this is if you need to kubectl logs (or, more rarely, kubectl exec) in a multi-container pod; if you are in this case, you'll appreciate having a shorter name, and since the container names are always scoped to the specific pod in which they appear, there's nothing wrong with using a short fixed name here
spec:
containers:
- name: container