I want to know if helm has a option similar to kubectl config set-context --current --namespace test-namespace, So I wouldn't have to pass namespace every time.
The helm tool is used to deploy modeled kubernetes resource packages. So under the hood, helm uses go-template and kubectl and outside of the helm argument definition "-n" it necessarily refers to the current namespace.
By default (i.e. without using the "-n" option of the helm cli), the helm version will be deployed in the current namespace.
So you can ignore this option if you are in the right namespace with the commands :
kubens test-namespace # Install by krew plugin or binary release
OR
kubectl config set-context --current --namespace test-namespace # kubectl native subcommand
OR
kubie ns test-namespace # (recommended) Install by binary release or rust cargo
Related
In my CI I'm running a helm upgrade command to release an app.
But if it is a non existing app, I have to create the namespace, a secret and patch the serviceaccount. So I come up with this:
kubectl create namespace ${namespace} --dry-run=client -o yaml | kubectl apply -f -
kubectl create secret docker-registry gitlab-registry --namespace ${namespace} --docker-server="\${CI_REGISTRY}" --docker-username="\${CI_DEPLOY_USER}" --docker-password="\${CI_DEPLOY_PASSWORD}" --docker-email="\${GITLAB_USER_EMAIL}" -o yaml --dry-run=client | kubectl apply -f -
kubectl patch serviceaccount default -p '{"imagePullSecrets":[{"name":"gitlab-registry"}]}' --namespace ${namespace}
This is working, but I think it is not the perfect way as these three steps should only be done once.
: Only if app/namespace/secret is not existing
Helm provides the --create-namespace switch that will create the namespace of the release if it does not already exist.
The secret can be added in your helm chart and you can pass the variables (CI_REGISTRY, CI_DEPLOY_USER, etc.) in as helm chart values either as --set values or via the values.yaml file and using --values
The service account patching you can do as a post-install and/or a post-upgrade job (https://helm.sh/docs/topics/charts_hooks/)
I deployed a helm chart using helm install and after this I want to see if the pods/services/cms related to just this deployment have come up or failed. Is there a way to see this?
Using kubectl get pods and greping for the name works but it does not show the services and other resources that got deployed when this helm chart is deployed.
helm get manifest RELEASE_NAME
helm get all RELEASE_NAME
https://helm.sh/docs/helm/helm_get_manifest/
If you are using Helm3:
To list all resources managed by the helm, use label selector with label app.kubernetes.io/managed-by=Helm:
$ kubectl get all --all-namespaces -l='app.kubernetes.io/managed-by=Helm'
To list all resources managed by the helm and part of a specific release: (edit release-name)
kubectl get all --all-namespaces -l='app.kubernetes.io/managed-by=Helm,app.kubernetes.io/instance=release-name'
Update:
Labels key may vary over time, follow the official documentation for the latest labels.
I couldn't find anywhere that gave me what I wanted, so I wrote this one-liner using yq. It prints out all objects in Kind/name format. You might get some blank space if any manifests are nothing but comments.
helm get manifest $RELEASE_NAME | yq -N eval '[.kind, .metadata.name] | join("/")' - | sort
Published here: https://gist.github.com/bioshazard/e478d118fba9e26314bffebb88df1e33
By issuing:
kubectl get all -n <namespace> | grep ...
You will only query for the following resources:
pod
service
daemonset
deployment
replicaset
statefulset
job
cronjobs
I encourage you to follow this article for more explanation:
Studytonight.com: How to list all resources in a Kubernetes namespace
Using the example from the above link you can query the API for all resources by issuing:
kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1 kubectl get --show-kind -l LABEL=VALUE --ignore-not-found -o name
This command will query the API for all the resources types in the cluster and then query for each of the resources separately by label.
You can create resources in a Helm chart with labels and then query the API by specifying: -l LABEL=VALUE.
EXAMPLE
Assuming that you provisioned following Helm chart
$ helm install awesome-nginx stable/nginx-ingress
This Chart is deprecated but it's only for example purposes.
You can query the API for all resources with:
kubectl api-resources --verbs=list -o name | xargs -n 1 kubectl get --show-kind -l release=awesome-nginx --ignore-not-found -o name
where:
LABEL <- release
VALUE <- awesome-nginx (release name)
After that you should be able to see:
endpoints/awesome-nginx-nginx-ingress-controller
endpoints/awesome-nginx-nginx-ingress-default-backend
pod/awesome-nginx-nginx-ingress-controller-86b9c7d9c7-wwr8f
pod/awesome-nginx-nginx-ingress-default-backend-6979c95c78-xn9h2
serviceaccount/awesome-nginx-nginx-ingress
serviceaccount/awesome-nginx-nginx-ingress-backend
service/awesome-nginx-nginx-ingress-controller
service/awesome-nginx-nginx-ingress-default-backend
deployment.apps/awesome-nginx-nginx-ingress-controller
deployment.apps/awesome-nginx-nginx-ingress-default-backend
replicaset.apps/awesome-nginx-nginx-ingress-controller-86b9c7d9c7
replicaset.apps/awesome-nginx-nginx-ingress-default-backend-6979c95c78
podmetrics.metrics.k8s.io/awesome-nginx-nginx-ingress-controller-86b9c7d9c7-wwr8f
podmetrics.metrics.k8s.io/awesome-nginx-nginx-ingress-default-backend-6979c95c78-xn9h2
rolebinding.rbac.authorization.k8s.io/awesome-nginx-nginx-ingress
role.rbac.authorization.k8s.io/awesome-nginx-nginx-ingress
You can modify the output by changing the -o parameter.
Additional resources:
Github.com: Kubectl get all does not list all resources in a namespace #151
Stackoverflow.com: Questions: Listing all resources in a namespace
$ helm get manifest RELEASE-NAME
helm status RELEASE_NAME
This command shows the status of a named release. The status consists
of:
last deployment time
k8s namespace in which the release lives
state of the release (can be: unknown, deployed, uninstalled, superseded, failed, uninstalling, pending-install, pending-upgrade or
pending-rollback)
list of resources that this release consists of, sorted by kind
details on last test suite run, if applicable
additional notes provided by the chart
Usage: helm status RELEASE_NAME [flags]
Official docs
Also note that helm place some known labels/annotations on resource it manages, see here. You can use it with kubectl get ... -l ...
kubectl get all -n <namespace> | grep <helm chart keyword, ex: kibana, elasticsearch>
Should list all resources created by helm chart in a particular namespace
I am trying to deploy my microservice on a Kuberenetes cluster in 2 different environment dev and test. And I am using helm chart to deploy my Kubernetes service. I am using Jenkinsfile to deploy the chart. And inside Jenkinsfile I added helm command within the stage like the following ,
stage ('helmchartinstall')
{
steps
{
sh 'helm upgrade --install kubekubedeploy --namespace test pipeline/spacestudychart'
}
}
}
Here I am defining the --namespace test parameter. But when it deploying, it showing the console output with default namespace. I already created namespaces test and prod.
When I checked the Helm version, I got response like the following,
docker#mildevdcr01:~$ helm version
Client: &version.Version{SemVer:"v2.14.1",
GitCommit:"5270352a09c7e8b6e8c9593002a73535276507c0", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.14.0",
GitCommit:"05811b84a3f93603dd6c2fcfe57944dfa7ab7fd0", GitTreeState:"clean"}
Have I made any mistake here for defining the namespace?
The most likely issue here is that the Chart already specifies default as metadata.namespace which in Helm 2 is not overwritten by the --namespace parameter.
If this is the cause a solution would be to remove the namespace specified in the metadata.namespace or to make it a template parameter (aka release value).
Also see https://stackoverflow.com/a/51137448/1977182.
Approach 1:
export TILLER_NAMESPACE= your_namespace
helm upgrade -i -n release_name chart.tgz
Approach 2:
helm upgrade -i -n release_name --namespace your_namespace chart.tgz
I installed istio using these commands:
VERSION = 1.0.5
GCP = gcloud
K8S = kubectl
#$(K8S) apply -f istio-$(VERSION)/install/kubernetes/helm/istio/templates/crds.yaml
#$(K8S) apply -f istio-$(VERSION)/install/kubernetes/istio-demo-auth.yaml
#$(K8S) get pods -n istio-system
#$(K8S) label namespace default istio-injection=enabled
#$(K8S) get svc istio-ingressgateway -n istio-system
Now, how do I completely uninstall it including all containers/ingress/egress etc (everthing installed by istio-demo-auth.yaml?
Thanks.
If you used istioctl, it's pretty easy:
istioctl x uninstall --purge
Of course, it would be easier if that command were listed in istioctl --help...
Reference: https://istio.io/latest/docs/setup/install/istioctl/#uninstall-istio
Based on their documentation here, you can generate all specs as yml file then pipe it to simple kubectl's delete operation
istioctl manifest generate <your original installation options> | kubectl delete -f -
here's an example:
istioctl manifest generate --set profile=default | kubectl delete -f -
A drawback of this approach though is to remember all options you have used when you installed istio which might be quite hard to remember especially if you enabled specific components.
If you have installed istio using helm's chart, you can uninstall it easily
First, list all installed charts:
helm list -n istio-system
NAME NAMESPACE REVISION UPDATED STATUS
istiod istio-system 1 2020-03-07 15:01:56.141094 -0500 EST deployed
and then delete/uninstall the chart using the following syntax:
helm delete -n istio-system --purge istio-system
helm delete -n istio-system --purge istio-init
...
Check their website for more information on how to do this.
If you already installed istio using istioctl or helm in its own separate namespace, you can easily delete completely that namespace which will in turn delete all resources created inside it.
kubectl delete namespace istio-system
Just run kubectl delete for the files you applied.
kubectl delete -f istio-$(VERSION)/install/kubernetes/istio-demo-auth.yaml
You can find this in docs as well.
If you have installed it as described, then you will need to delete it in the same way.
kubectl delete -f istio-$(VERSION)/install/kubernetes/helm/istio/templates/crds.yaml
kubectl delete -f istio-$(VERSION)/install/kubernetes/istio-demo-auth.yaml
Then you would manually delete the folder, and istioctl, if you moved to anywhere.
IMPORTANT: Deleting a namespace is super comfortable to clean up, but you can't do it for all scenarios. In this situation, if you delete the namespace only, you are leaving all the permissions and credentials intact. Now, say you want to update Istio, and Istio team has made some security changes in their RBAC rules, but has not changed the name of the object. You would deploy the new yaml file, and it will throw an error saying the object (for example clusterrolebinding) already exists. If you don't pay attention to what that error was, you can end up with the worse type of errors (when there are no error, but something goes wrong).
Cleaning up Istio is a bit tricky, because of all the things it adds: CustomResourceDefinitions, ConfigMaps, MutatingWebhookConfigurations, etc. Just deleting the istio-system namespace is not sufficient. The safest bet is to use the uninstall instructions from istio.io for the method you used to install.
Kubectl: https://istio.io/docs/setup/kubernetes/install/kubernetes/#uninstall
Helm: https://istio.io/docs/setup/kubernetes/install/helm/#uninstall
When performing these steps, use the version of Istio you are attempting to remove. So if you are trying to remove Istio 1.0.2, grab that release from istio.io.
Don't forget to disable the injection:
kubectl delete -f istio-$(VERSION)/install/kubernetes/helm/istio/templates/crds.yaml
kubectl delete -f istio-$(VERSION)/install/kubernetes/istio-demo-auth.yaml
kubectl label default your-namespace istio-injection=disabled
Using the profile you used in installation, demo for example, run the following command
istioctl manifest generate --set profile=demo | kubectl delete -f -
After normal istio uninstall (depending on the way istio was installed by helm or istioctl) following steps can be performed
Check if anything still exists in the istio-system namespace, if exists then delete manually, also remove the istio-system namespace
Check if there is a sidecar associated with any pod (sometimes sidecars not get cleaned up in case of failed uninstallation)
kubectl get pods -o=jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.metadata.namespace}{"\t"}{..image}{"\n\n"}{end}' -A | grep 'istio/proxyv' | grep -v istio-system
Get the CRD that is still in use and remove associated resources
kubectl get crds | grep 'istio.io' | cut -f1-1 -d "." | xargs -n1 -I{} bash -c " echo {} && kubectl get --all-namespaces {} -o wide && echo -e '---'"
Delete all the CRD
kubectl get crds | grep 'istio.io' | xargs -n1 -I{} sh -c "kubectl delete crd {}"
Edit the labels back (optional)
kubectl label default <namespace name> istio-injection=disabled
Just delete the ns
k delete ns istio-system
Deleting CRDs without needing to find the helm charts:
kubectl delete crd -l chart=istio
Hi if you installated via helm-template you can use these commands :
For CRD's:
$ helm template ${ISTIO_BASE_DIR}/install/kubernetes/helm/istio-init --name istio-init --namespace istio-system | kubectl delete -f -
$ kubectl delete crd $(kubectl get crd |grep istio)
For Deployment/NS..etc other resources:
$ helm template install/kubernetes/helm/istio --name istio --namespace istio-system\
--values install/kubernetes/helm/istio/values-istio-demo.yaml \
--set global.controlPlaneSecurityEnabled=true \
--set global.mtls.enabled=true | kubectl delete -f -
I use the following helm (2.4.2) commands in my gitlab-ci.yml script:
- helm upgrade --install myapp-db --wait --set postgresUser=postgres,postgresPassword=postgres,postgresDatabase=myapp stable/postgresql
- helm upgrade --install myapp-web ./myapp-chart --wait --set env.DATABASE_URL="${DATABASE_URL}"
It's part of a deployment to my staging/review environment. After the above commands complete, I would like to execute commands against the my-app pod to create/migrate the database. At the moment this is achieved through the use of an initContainer (defined in the referenced yaml file). But I would prefer the logic to be part of the CI script - so I don't have to have a separate deployment file for production.
Is there a way to do this with helm? Or is my only option to use kubectl exec? If I use kubectl exec, is there an easy way to get the name of the pod using helm?
This GitHub issue addresses how you might use kubectl to find out the name of a pod based on a label:
https://github.com/kubernetes/kubernetes/issues/8876
I implemented the following:
- export POD_NAME=`kubectl get pod -l "app=myapp-web-chart" -o jsonpath='{.items[0].metadata.name}'`
- kubectl exec $POD_NAME -- mix ecto.migrate
Still, it would be much nicer if there was a way to do this with helm