How do I tell helm to create internal secrets in namespace - kubernetes-helm

When trying to run helm install to deploy an application to a private K8S cluster, I get the following error:
helm install myapp ./myapp
Error: create: failed to create: secrets is forbidden: User "u-user1"
cannot create resource "secrets" in API group "" in the namespace "default"
exit status 1
I know that this is happening because helm creates secrets behind the scene to hold information that it needs for managing the deployment. See Handling Secrets:
As of Helm v3, the release definition is stored as a Kubernetes Secret resource by default, as opposed to a ConfigMap.
The problem is that helm is trying to create the secrets in the default namespace, and I'm working in a private cloud and not allowed to create resources in the default namespace.
How can I tell helm to use a namespace when creating the internal secrets that it needs to use?
Searching for a solution
A search on the helm site found:
https://helm.sh/docs/faq/ - which says
In Helm 3, information about a particular release is now stored in the same namespace as the release itself
But I've set the deployment to be in the desired namespace. My myapp/templates/deployment.yaml file has:
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
namespace: myapp-namespace
So I'm not sure how to tell helm to create it's internal secrets in this myapp-namespace.
Other Searches
Helm Charts create secrets in different namespace - Is asking a different question about how to create user defined secrets in different namespaces.
Helm upgrade is creating multiple secrets - Different question, and no answer (yet).
Secret management in Helm Charts - is asking a different question.
Update 1)
When searching for a solution I tried adding the --namespace myapp-namespace argument to the helm install command (see below).
helm install --namespace myapp-namespace myapp ./myapp
Error: create: failed to create: secrets is forbidden: User "u-user1"
cannot create resource "secrets" in API group "" in the namespace "myapp-namespace"
exit status 1
Notice that the namespace is now myapp-namespace, so I believe that helm is now creating the internal secrets in my desired namespace, so I think this answers my original question.
I think I now have a permissions issue that I need to ask the K8S admins to address.

You must use the --namespace option in order to tell helm install what namespace you are using. The syntax you specified is correct.
helm install --namespace myapp-namespace myapp ./myapp
You could also put --namespace at the end of the command as below:
helm install myapp ./myapp --namespace myapp-namespace
With this syntax, helm will create the internal secrets in the namespace you've specified.
Doing this will prevent the default namespace from being polluted.
The following command is then needed to see the install.
helm list --namespace myapp-namespace
helm list --all-namespaces

Related

Failed to create nodeport error, after deploying ingress

Failed to create NodePort error, after deploying ingress
I have an ingress defined as in the screenshot:
Screenshot
The 2 replicas of an Ingress server are not spinning due to the Failed to create NodePort error. Please advice
Just like the error says. You are missing the NodePortPods CRD. It looks like that CRD existed at some point in time. But I don't see it anymore in the repo. You didn't specify how you deployed the ingress operator but you can make sure you install the latest.
helm repo add appscode https://charts.appscode.com/stable/
helm repo update
helm search repo appscode/voyager --version v13.0.0
# Generate the template to check or use helm install
helm template voyager-operator appscode/voyager --version v13.0.0 --namespace kube-system --no-hooks --set cloudProvider=baremetal 👈 Use the right cloud provider
✌️

cannot helm install rabbitmq servers (helm 2.16.9) : namespaces "rabbit" is forbidden

helm install --name my-rabbitserver stable/rabbitmq --namespace rabbit
Error: release my-rabbitserver failed: namespaces "rabbit" is forbidden: User "system:serviceaccount:kube-system:default" cannot get resource "namespaces" in API group "" in the namespace "rabbit"
I have tried with (and without a rabbit namespace created before the install attempt)
I am using helm 2.16.9, so I need to qualify the name of my installation with a --name
I am using this against a Google Cloud kubernetes cluster
It looks as though the Helm tiller pod did not have sufficient priveldeges.
I found this similar issue:
https://support.sumologic.com/hc/en-us/articles/360037704393-Kubernetes-Helm-install-fails-with-Error-namespaces-sumologic-is-forbidden-User-system-serviceaccount-kube-system-default-cannot-get-resource-namespaces-in-API-group-in-the-namespace-sumologic-
Basically I have to stop the tiller deployment, set up tiller ServiceAccount yaml and run it to give tiller access to the kube-system. And then execute helm init again with the new service account.
The helm rabbitmq installs then appear work as advertised
I thought helm was supposed to make life easier, but it still has its own limitations and additional yaml files to get it to work as advertised.

Helm RBAC rules to create namespaces and resources inside those created name

I found a lot of information on how to give helm permission to create resources in a particular namespace.
I am trying to see if I can create namespaces on the fly(with random names) and then use helm to install and delete resources inside the namespace.
My idea is to create a namespace with name such as Fixedsuffix-randomprefix and then allow helm to create all resources inside it. Is this possible ?
I can create a clusterrole and clusterrolebinding to allow tiller's serviceaccount to create namespaces, but I am not able to figure out how to have a serviceaccount that could create resources in the particular namespace( mainly because this serviceaccount to create resources cant would have to be created when the namespace is created and then assigned to tiller pod).
TIA
My question is why would you create sa, clusterrole and rolebinding to do that? Helm has it´s own resources which allow him to install and delete resources inside new namespace.
My idea is to create a namespace with name such as Fixedsuffix-randomprefix and then allow helm to create all resources inside it. Is this possible ?
Yes, you can create your new namespace and use helm to install everything in this namespace.Or even better you can just use helm install and it will create new namespace for you. For that purpose helm have helm install --namespace.
-n, --namespace string namespace scope for this request
For example you can install traefik chart in namespace tla.
helm install stable/traefik --namespace=tla
NAME: oily-beetle
LAST DEPLOYED: Tue Mar 24 07:33:03 2020
NAMESPACE: tla
STATUS: DEPLOYED
Another idea which came to my mind is you might want tiller not to use cluster-admin credentials, then this link could help.

Delete namespace and also delete Helm deployment

I deploy my Helm deploys to isolated namespaces.
Deleting a namespace deletes all the resources in it - except the Helm deployment.
Deleting a Helm deployment deletes all resource in it - except the namespace.
I have to do this which seems redundant:
helm del `helm ls NAMESPACE --short` --purge
kubectl delete namespace NAMESPACE
I'd rather just delete my namespace and have the Helm deploy also purged - is this possible?
Deleting a namespace deletes all the resources in it- except the helm deployment
This can't be (deleting a namespace implies deleting everything in it, there aren't any exceptions), and must means that the state representing Helm's concept of a deployment doesn't live in that namespace. Helm stores these as config maps in the TILLER_NAMESPACE. See here and here.
It's not surprising that if you create some resources with helm and then go "under the hood" and delete those resources directly via kubectl, Helm's state of the world won't result in that deployment disappearing.
Deleting a helm deployment deletes all resource in it- except the namespace
That sounds like expected behaviour. Presumably you created the namespace out of band with kubectl, it's not part of your Helm deployment. So deleting the Helm deployment wouldn't delete that namespace.
If you kubectl create namespace NS and helm install CHART --namespace NS then it's not surprising that to clean up, you need to helm delete the release and then kubectl delete the namespace.
The only way I could imagine to do that would be for the Helm chart itself to both create a namespace and create all subsequent namespace-scoped resources within that namespace. Here is an example that appears to do such a thing.
There is a PR created to cleanup all resources deployed from helm. follow the link --> https://github.com/helm/helm/issues/1464
hopefully in the future release it will be addressed

Deploying Images from gitlab in a new namespace in Kubernetes

I have integrated gitlab with Kubernetes cluster which is hosted on AWS. Currently it builds the code from gitlab to the default namespace. I have created two namespaces in kubernetes one for production and one for development. What are the steps if I want that to be deployed in a dev or a production namespace. Do I need to make changes at the gitlab level or on the kubernetes level.
This is done at the kubernetes level. Whether you're using helm or kubectl, you can specify the desired namespace in the command.
As in:
kubectl create -f deployment.yaml --namespace <desired-namespace>
helm install stable/gitlab-ce --namespace <desired-namespace>
Alternatively, you can just change your current namespace to the desired namespace and install as you did before. By default, helm charts or kuberenetes yaml files will install into your current namespace unless specified otherwise.