How to ssh to the node inside the cluster in local. I am using docker edge version which has kubernetes inbuilt. If i run
kubectl ssh node
I am getting
Error: unknown command "ssh" for "kubectl"
Did you mean this?
set
Run 'kubectl --help' for usage.
error: unknown command "ssh" for "kubectl"
Did you mean this?
set
There is no "ssh" command in kubectl yet, but there are plenty of options to access Kubernetes node shell.
In case you are using cloud provider, you are able to connect to nodes directly from instances management interface.
For example, in GCP: Select Menu -> Compute Engine -> VM instances, then press SSH button on the left side of the desired node instance.
In case of using local VM (VMWare, Virtualbox), you can configure sshd before rolling out Kubernetes cluster, or use VM console, which is available from management GUI.
Vagrant provides its own command to access VMs - vagrant ssh
In case of using minikube, there is minikube ssh command to connect to minikube VM. There are also other options.
I found no simple way to access docker-for-desktop VM, but you can easily switch to minikube for experimenting with node settings.
How to ssh to the node inside the cluster in local
Kubernetes is aware of nodes on level of secure communication with kubelets on nodes (geting hostname and ip from node), and as such, does not provide cluster-level ssh to nodes out of the box. Depending on your actual provide/setup there are different ways of connecting to nodes and they all boil down to locate your ssh key, open appropriate ports on firewall/security groups and issue ssh -i key user#node_instance_ip command to access node. If you are running locally with virtual machines you can setup your own ssh keypairs and do the trick..
You can effectively shell into a pod using exec(I know its not exactly what the question asks, but might be helpful).
An example usage would be kubectl exec -it name-of-your-pod -- /bin/bash. assuming you have bash installed.
Hope that helps.
You have to first Extend kubectl with plugins adding https://github.com/luksa/kubectl-plugins.
Basically, to "install" ssh, e.g.:
wget https://raw.githubusercontent.com/luksa/kubectl-plugins/master/kubectl-ssh
Then make sure the file is in kubectl-ssh your path.
Related
I know k3d can do this magically via k3d cluster create myname --token MYTOKEN --agents 1, but I am trying to figure out how to do the most simple version of that 'manually'. I want to create a server something like:
docker run -e K3S_TOKEN=MYTOKEN rancher/k3s:latest server
And connect an agent something like like:
docker run -e K3S_TOKEN=MYTOKEN -e K3S_URL=https://localhost:6443 rancher/k3s:latest agent
Does anyone know what ports need to be forwarded here? How can I set this up? Nearly everything I try, the agent complains about port 6444 already in use, even if I disable as much as possible about the server with any combination of --no-deploy servicelb --disable-agent --no-deploy traefik
Feel free to disable literally everything other than the server and the agent, I'm trying to make this ultra ultra simple, but just butting my head against a wall at the moment. Thanks!
The containers must "see" each other. Docker isolates the networks by default, so "localhost" in your agent container is the agent container itself.
Possible solutions:
Run both containers without network isolation using --net=host, map API port of the server to the host with --port and use the host IP in the agent container or use docker-compose.
A working example for docker-compose is described here: https://www.trion.de/news/2019/08/28/kubernetes-in-docker-mit-k3s.html
I am running a local Kubernetes cluster through Docker Desktop on Windows. I'm attempting to modify my kube-apiserver config, and all of the information I've found has said to modify /etc/kubernetes/manifests/kube-apiserver.yaml on the master. I haven't been able to find this file, and am not sure what the proper way is to do this. Is there a different process because the cluster is through Docker Desktop?
Is there a different process because the cluster is through Docker Desktop?
You can get access to the kubeapi-server.yaml with a Kubernetes that is running on Docker Desktop but in a "hacky" way. I've included the explanation below.
For setups that require such reconfigurations, I encourage you to use different solution like for example minikube.
Minikube has a feature that allows you to pass the additional options for the Kubernetes components. You can read more about --extra-config ExtraOption by following this documentation:
Minikube.sigs.k8s.io: Docs: Commands: Start
As for the reconfiguration of kube-apiserver.yaml with Docker Desktop
You need to run following command:
docker run -it --privileged --pid=host debian nsenter -t 1 -m -u -n -i sh
Above command will allow you to run:
vi /etc/kubernetes/manifests/kube-apiserver.yaml
This lets you edit the API server configuration. The Pod running kubeapi-server will be restarted with new parameters.
You can check below StackOverflow answers for more reference:
Stackoverflow.com: Answer: Where are the Docker Desktop for Windows kubelet logs located?
Stackoverflow.com: Answer: How to change the default nodeport range on Mac (docker-desktop)?
I've used this answer without $ screen command and I was able to reconfigure kubeapi-server on Docker Desktop in Windows
Currently...
I have a GKE/kubernetes/k8s cluster in GCP. I have a bastion host (Compute Engine VM Instance) in GCP. I have allowlisted my bastion host's IP in the GKE cluster's Master authorized networks section. Hence, in order to run kubectl commands to my GKE, I first need to SSH into my bastion host by running the gcloud beta compute ssh command; then I run the gcloud container clusters get-credentials command to authenticate with GKE, then from there I can run kubectl commands like usual.
Later...
I want to be able to run kubectl commands to my GKE cluster directly from my local development CLI. In order to do that, I can add my local development machine IP as an allowlisted entry into my GKE's Master authorized networks, and that should be it. Then i can run the gcloud container clusters get-credentials first and then run kubectl commands like usual.
However...
I am looking for a way to avoid having to allowlist my local development machine IP. Every time i take my laptop somewhere new, i have to update the allowlist my new IP from there before i can run the gcloud container clusters get-credentials command before running kubectl commands.
I wonder...
Is there a way to assign a port number in the bastion-host that can be used to invoke kubectl commands to the remote GKE cluster securely? And then, i can just use the gcloud compute start-iap-tunnel command (which BTW takes care of all permission issues using Cloud IAM) from my local dev CLI to establish a ssh-tunnel to that specific port number in the bastion host. That way, for the GKE cluster, it is receiving kubectl commands from the bastion host (which is already allowlisted in its Master authorized networks). But behind the scene, i am authenticating with the bastion host from my local dev CLI (using my glcoud auth credentails) and invoking kubectl commands from there securely.
Is this possible? Any ideas from anyone?
This would help accessing to your secured GKE cluster from localhost
https://github.com/GoogleCloudPlatform/gke-private-cluster-demo
Once the bastion host is setup with tinyproxy as in the above doc, we can use the below shell functions to quickly enable/disable the bastion host access
enable_secure_kubectl() {
# Aliasing kubectl/helm commands to use local proxy
alias kubectl="HTTPS_PROXY=localhost:8888 kubectl"
alias helm="HTTPS_PROXY=localhost:8888 helm"
# Open SSH tunnel for 1 hour
gcloud compute ssh my-bastion-host -- -o ExitOnForwardFailure=yes -M -S /tmp/sslsock -L8888:127.0.0.1:8888 -f sleep 3600
# Get kubernetes credentials with internal ip for kube-apiserver in kubeconfig
gcloud container clusters get-credentials my-gke-cluster --region us-east1 --project myproject --internal-ip
}
disable_secure_kubectl() {
unalias kubectl
unalias helm
ssh -S /tmp/sslsock -O exit my-bastion-host
}
I'm currently using minikube and I'm trying to access my application by utilizing the minikube tunnel since the service type is LoadBalancer.
I'm able to obtain an external IP when I execute the minikube tunnel, however, when I try to check it on the browser it doesn't work. I've also tried Postman and curl, they both don't work.
To add to this, if I shell into the pod I can use curl and it does work. Furthermore, I executed kubectl port-forward and I was able to access my application through localhost.
Does anyone have any idea as to why I'm not being able to access my application even though everything seems to be running correctly?
Your service is probably bound to localhost. Minikube starts the cluster in a VM or docker (depending on the driver you are using) that is bound to an external IP, $(minikube ip).
When you are running a minikube tunnel you're tunneling from minikube cluster external IP to the internal IP of the load balancer, the LB service in Kubernete the External IP goes from "Pending" to an actual internal IP and something like this should work:
curl -H 'Host: localhost' -v $(minikube ip)
However, it doesn't in the browser, since in the above command you are sending the request to the minikube's IP, not localhost. What I do for this to work is a ssh tunnel like this one:
ssh -i $(minikube ssh-key) docker#$(minikube ip) -L 8008:localhost:80
This maps the LB listener in port 80, in minikube's cluster, to 8008 in localhost. The external IP of the service remains pending but it works since the Kube controller can still find it. If you want to map port 80 then you will need to add sudo.
If the version of ssh on your system (the one in your path) is less than 8.0, 'minikube tunnel' will silently fail to instantiate the ssh tunnel for some port forwards. (e.g. privileged ports)
Open a command prompt as administrator, and type 'where.exe ssh'. Navigate to that location in windows explorer, and right-click on 'ssh.exe'. Choose Properties->Details to see the version.
If this is less than version 8.0 you must upgrade that to at least version 8.0 to prevent this silent failure of ssh by 'minikube tunnel'.
After upgrading, ssh, ensure that the newer version is the one that will be executed by using the 'where.exe' command again. If there are two on your system, then reorder the paths in your path environment variable. Restart your shell (or better) reboot the system so that all processes environments pick up the path changes.
Then try 'minikube tunnel' again. When it is working, you should see an ssh instance in the task manager for each tunnel that minikube creates.
In my case minikube service <serviceName> solved this issue.
For further details look here in minikube docs.
I am exploring and learning about containers and kubernetes using podman and minikube on a linux workstation. I use podman to build images on the workstation and would like to deploy these images in minikube also running on the workstation using the kvm2 virtual machine driver. I also start minikube using the CRI-O container runtime.
What are efficient workflows to deploy these images from the workstation to minikube in this scenario? Docker is not running on the minikube VM so the reusing the Docker daemon as described in the minikube documentation is not an option. Sharing the host file system with minikube also appears to not be viable at this time when using kvm2.
Is running a local registry that is visible to both the workstation and the minikube vm the best option? Answers to How to use local docker images with Minikube? and (Kubernetes + Minikube) can't get docker image from local registry appear to offer good solutions for configuring a local registry.
Would skopeo be a solution?
Edit: this is a nice post describing how to set up a registry using podman: https://computingforgeeks.com/create-docker-container-registry-with-podman-letsencrypt/
thank you
Brad
Minikube documentation provides the foundation for a potential workflow at https://minikube.sigs.k8s.io/docs/tasks/docker_registry/. In order to use podman in lieu of docker I did the following
Start minikube, as instructed, with the --insecure-registry flag. I specifically use
minikube start --network-plugin=cni --enable-default-cni --bootstrapper=kubeadm --container-runtime=cri-o --cpus 4 --memory 4g --insecure-registry "192.168.39.0/24"
Enable the minikube registry addon.
minikube addons enable registry
Configure podman to use the insecure minikube registry by adding the registry to the insecure registries section of /etc/containers/registries.conf. This section now looks like
[registries.insecure]
registries = ['192.168.39.175:5000']
where 192.168.39.175 is the minikube ip. This ip may change following minikube restarts.
Follow the build, push and run commands in https://minikube.sigs.k8s.io/docs/tasks/docker_registry/ substituting podman for docker. This assumes the test-img container file exists.
Build: podman build --tag $(minikube ip):5000/test-img .
Push: podman push $(minikube ip):5000/test-img
Run: kubectl run test-img --image=$(minikube ip):5000/test-img
This worked but suffers from a serious complication: there is no apparent way at this time to set the IP address for the minikube VM when using kvm2. The IP will always be in the 192.168.39.0/24 subnet but that is the only certainty. Each time minikube is started the IP address of the registry will change which has significant implications for podman and the workflow in general.
More to come an another solution.