I'm running a small openshift cluster and would like to provide our developers with an hosted instance of mongo on it, which they connect to externally.
Which is easy enough, I thought. Sadly it still looks like all traffic has to go over haproxy and is limited to http/https. But my developers need to transparently access the correct mongo port 27017.
is there some way to expose the internal pod port, to the outside world, without knowing which pod it run on.
right now our dirty workaround is
oc port-forward mongodb-1-2n1ov 27017:27017
and than the client does a ssh forwarding from there machine to this.
instead we would rather have an automated solution that allows tcp forwarding for virtual defined hostnames.
could anyone point me in the right direction please?
You are right. We too had similar issue and only other way we though was to update the serviceCIDR which was routable within our network. We did not go that route though. HAProxy is http/https..while the services do support tcp/udp and mongodb:27017 relies on UDP.
I too would like to know more about this if anyone else can share.
Related
I have a Kubernetes cluster running my production environments. I have bastion machine, and my own computer can connect to bastion & the bastion can access the cluster machines. I want to connect to some internal (i.e. not exposed to public network) services, such as MySQL, Redis, Kibana, etc, on my own computer. I need to have enough performance (e.g. the kubectl forward is toooo slow), and have enough security.
I have tried to use kubectl forward. But it is very slow, and after a search, they say it is just slow. So I guess I cannot make it faster.
I guess I can also expose every service as a NodePort. Then I can use things like ssh port forward. However, I am afraid whether the security is low? Because we have to create a NodePort, then if hacker can touch the cluster, he can use the nodeport to access my MySQL, Redis, Kafka, etc, which is terrible.
EDITED: In addition, I need not only my own computer, but my mobile phone to able to touch some services, such as my Spring Boot internal admin url. currently I do ssh port forward and bind to 0.0.0.0, so my mobile phone can connect to my_computer_ip:the_port to use it. But how can I do it without ssh port forward?
Thank you!
I have created a simple hello world service in my kubernetes cluster. I am not using any cloud provider and have created it in a simple Ubuntu 16.04 server from scratch.
I am able to access the service inside the cluster but now when I want to expose it to the internet, it does not work.
Here is the yml file - deployment.yml
And this is the result of the command - kubectl get all:
Now when I am trying to access the external IP with the port in my browser, i.e., 172.31.8.110:8080, it does not work.
NOTE: I also tried the NodePort Service Type, but then it does not provide any external IP to me. The state remains pending under the "External IP" tab when I do "kubectl get services".
How to resolve this??
I believe you might have a mix of networking problems tied together.
First of all, 172.31.8.110 belongs to a private network, and it is not routable via Internet. So make sure that the location you are trying to browse from can reach the destination (i.e. same private network).
As a quick test you can make an ssh connection to your master node and then check if you can open the page:
curl 172.31.8.110:8080
In order to expose it to Internet, you need a to use a public IP for your master node, not internal one. Then update your Service externalIPs accordingly.
Also make sure that your firewall allows network connections from public Internet to 8080 on master node.
In any case I suggest that you use this configuration for testing purposes only, as it is generally bad idea to use master node for service exposure, because this applies extra networking load on the master and widens security surface. Use something like an Ingress controller (like Nginx or other) + Ingress resource instead.
One option is also to do SSH local port forwarding.
ssh -L <local-port><private-ip-on-your-server><remote-port> <ip-of-your-server>
So in your case for example:
ssh -L 8888:172.31.8.110:8080 <ip-of-your-ubuntu-server>
Then you can simply go to your browser and configure a SOCKS Proxy for localhost:8888.
Then you can access the site on http://localhost:8888 .
I am trying to learn kubernetes and rancher. Here is what i want to accomplish :
I have few docker containers which i want to service only from my internal network using x.mydomain.com
I have same as above but those containers will be accessible from internet on x.mydomain.com
What i have at the moment is following :
Rancher server
RancherOS to be used for the cluster and as one node
I have made a cluster and added the node from 2. and disabled the nginx controller.
Install traefik app
I have forwarded port 80, 443 to my node.
Added few containers
Added ingress rules
So at the moments it works with the external network. I can write app1.mydomain.com from the internet and everything works as it should.
Now my problem is how can i add the internal network now ?
Do i create another cluster ? Another node on the same host ? Should i install two traefik and then use class in ingress for the internal stuff ?
My idea was to add another ip to the same interface on the rancheros then add another node on the same host but with the other ip but i can’t get it to work. Rancher sees both nodes with the same name and doesn’t use the information i give it i mean --address when creating the node. Of course even when i do this it would require that i setup a DNS server internally so it knows which domains are served internally but i haven’t done that yet since i can’t seem to figure out how to handle the two ip on the host and use them in two different nodes. I am unsure what is require, maybe it’s the wrong route i am going.
I would appreciate if somebody had some ideas.
Update :
I thought i had made it clear what i want from above. There is no YAML at the moment since i don't know how to do it. In my head it's simple what i want. Let me try to cook it down with an example :
I want 2 docker containers with web server to be able to be accessible from the internet on web1.mydomain.com and web2.mydomain.com and at the same time i want 2 docker containers with web server that i can access only from internal network on web3.mydomain.com and web4.mydomain.com.
Additional info :
- I only have one host that will be hosting the services.
- I only have one public IPv4 address.
- I can add additional ip alias to the one host i have.
- I can if needed configure an internal DNS server if required.
/donnib
I was using NodePort to host a webapp on Google Container Engine (GKE). It allows you to directly point your domains to the node IP address, instead of an expensive Google load balancer. Unfortunately, instances are created with HTTP ports blocked by default, and an update locked down manually changing the nodes, as they are now created using and Instance Group/and an Immutable Instance Template.
I need to open port 443 on my nodes, how do I do that with Kubernetes or GCE? Preferably in an update resistant way.
Related github question: https://github.com/nginxinc/kubernetes-ingress/issues/502
Using port 443 on your Kubernetes nodes is not a standard practice. If you look at the docs you and see the kubelet option --service-node-port-range which defaults to 30000-32767. You could change it to 443-32767 or something. Note that every port under 1024 is restricted to root.
In summary, it's not a good idea/practice to run your Kubernetes services on port 443. A more typical scenario would be an external nginx/haproxy proxy that sends traffic to the NodePorts of your service. The other option you mentioned is using a cloud load balancer but you'd like to avoid that due to costs.
Update: A deamonset with a nodeport can handle the port opening for you. nginx/k8s-ingress has a nodeport on 443 which gets exposed by a custom firewall rule. the GCE UI will not show「Allow HTTPS traffic」as checked, because its not using the default rule.
You can do everything you do on the GUI Google Cloud Console using the Cloud SDK, most easily through the Google Cloud Shell. Here is the command for adding a network tag to a running instance. This works, even though the GUI disabled the ability to do so
gcloud compute instances add-tags gke-clusty-pool-0-7696af58-52nf --zone=us-central1-b --tags https-server,http-server
This also works on the beta, meaning it should continue to work for a bit.
See https://cloud.google.com/sdk/docs/scripting-gcloud for examples on how to automate this. Perhaps consider running on a webhook when downtime is detected. Obviously none of this is ideal.
Alternatively, you can change the templates themselves. With this method you can also add a startup to new nodes, which allows you do do things like fire a webhook with the new IP Address for a round robin low downtime dynamic dns.
Source (he had the opposite problem, his problem is our solution): https://stackoverflow.com/a/51866195/370238
If I understand correctly, if nodes can be destroyed and recreated themselves , how are you going to rest assured that certain service behind port reliably available on production w/o any sort of load balancer which takes care of route orchestration diverting port traffic to new node(s)
I want to access k8s api resources. my cluster is 1node cluster. kube-api server is listening on 8080 and 6443 port. curl localhost:8080/api/v1 inside node is working. if i hit :8080, its not working because some other service (eureka) is running on this port. this leaves me option to access :6443 . in order to do make api accessible, there are 2 ways.
1- create service for kube-api with some specific port which will target 6443. For that ca.crt , key , token etc are required. How to create and configure such things so that i will be able to access api.
2- make change in waeve (weave is available as service in k8s setup) so that my server can access k8s apis.
anyone of option is fine with me. any help will be appreciated .
my cluster is 1node cluster
One of those words does not mean what you think it does. If you haven't already encountered it, you will eventually discover that the memory and CPU pressure of attempting to run all the components of a kubernetes cluster on a single Node will cause memory exhaustion, and then lots of things won't work right with some pretty horrible error messages.
I can deeply appreciate wanting to start simple, but you will be much happier with a 3 machine cluster than trying to squeeze everything into a single machine. Not to mention the fact that only having a single machine won't surface any networking misconfigurations, which can be a separate frustration when you think everything is working correctly and only then go to scale your cluster up to more Nodes.
some other service (eureka) is running on this port.
Well, at the very real risk of stating the obvious: why not move one of those two services to listen on a separate port from one another? Many cluster provisioning tools (I love kubespray) have a configuration option that allows one to very easily adjust the insecure port used by the apiserver to be a port of your choosing. It can even be a privileged port (that is: less than 1024) because docker runs as root and thus can --publish a port using any number it likes.
If having the :8080 is so important to both pieces of software that it would be prohibitively costly to relocate the port, then consider binding the "eureka" software to the machine's IP and bind the kubernetes apiserver's insecure port to 127.0.0.1 (which is certainly the intent, anyway). If "eureka" is also running in docker, you can change its --publish to include an IP address on the "left hand side" to very cheaply do what I said: --publish ${the_ip}:8080:8080 (or whatever). If it is not using docker, there is still a pretty good chance that the software will accept a "bind address" or "bind host" through which you can enter the ip address, versus "0.0.0.0".
1- create service for kube-api with some specific port which will target 6443. For that ca.crt , key , token etc are required. How to create and configure such things so that i will be able to access api.
Every Pod running in your cluster has the option of declaring a serviceAccountName, which by default is default, and the effect of having a serviceAccountName is that every container in the Pod has access to those components you mentioned: the CA certificate and a JWT credential that enables the Pod to invoke the kubernetes API (which from within the cluster one can always access via: the kubernetes Service IP, the environment variable $KUBERNETES_SERVICE_HOST, or the hostname https://kubernetes -- assuming you are using kube-dns). Those serviceAccount credentials are automatically projected into the container at /var/run/secret/kubernetes.io without requiring that your Pod declare those volumeMounts explicitly.
So, if your concern is that one must have credentials from within the cluster, that concern can go away pretty quickly. If your concern is access from outside the cluster, there are a lot of ways to address that concern which don't directly involve creating all 3 parts of that equation.