I have been playing with Digital Ocean's new managed Kubernetes service. I have created a new cluster using Digital Ocean's dashboard and, seemingly, successfully deployed my yaml file (attached).
running in context kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
api-svc NodePort XX.XXX.XXX.XXX <none> 8080:30000/TCP 2h
kubernetes ClusterIP XX.XXX.X.X <none> 443/TCP 2h
My question is, how do I go exposing my service without a load balancer?
I have been able to do this locally using minikube. To get the cluster IP I run minikube ip and use port number 30000, as specified in my nodePort config, to reach the api-svc service.
From what I understand, Digital Ocean's managed service abstracts the master node away. So where would I find the public IP address to access my cluster?
Thank you in advance!
my yaml file for reference
apiVersion: v1
kind: Secret
metadata:
name: regcred
data:
.dockerconfigjson: <my base 64 key>
type: kubernetes.io/dockerconfigjson
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: api-deployment
labels:
app: api-deployment
spec:
replicas: 1
strategy: {}
template:
metadata:
labels:
app: api
spec:
containers:
- name: api
image: <my-dockerhub-user>/api:latest
ports:
- containerPort: 8080
imagePullSecrets:
- name: regcred
---
apiVersion: v1
kind: Service
metadata:
name: api-svc
spec:
type: NodePort
ports:
- port: 8080
targetPort: 8080
nodePort: 30000
protocol: TCP
selector:
app: api
type: NodePort
You can hit any of your worker nodes' ip. Example http://worker-node-ip:30000/. You can get the worker nodes ip from the digitalocean dashboard or use doctl cli.
Slightly more detailed answer: DigitalOcean manages firewall rules for your NodePort services automatically, so once you expose the service, the NodePort is automatically open to public traffic from all worker nodes in your cluster. See docs
To find the public IP of any of your worker nodes, execute the following doctl commands:
# Get the first worker node from the first node-pool of your cluster
NODE_NAME=$(doctl kubernetes cluster node-pool get <cluster-name> <pool-name> -o json | jq -r '.[0].nodes[0].name')
WORKER_NODE_IP=$(doctl compute droplet get $NODE_NAME --template '{{.PublicIPv4}}')
Using "type: NodePort" presume use of node external address (any node) and may be unsustainable because nodes might be changed/upgraded.
Related
I was trying to apply service externalIPs feature on EKS cluster.
What I do
I've created EKS cluster with eksctl:
eksctl create cluster --name=test --region=eu-north-1 --nodes=1
I've opened all security groups to make sure I don't have issue with firewall. ACL also allow all traffic.
I took public IP for the only available worker node and try to use it with simple service + deployment.
This should be only 1 deployment with 1 replicaset and 1 pod with nginx. This should be attached to a service with external/public IP everyone can reach.
apiVersion: apps/v1
kind: Deployment
metadata:
name: app
spec:
replicas: 1
selector:
matchLabels:
app: app
template:
metadata:
labels:
app: app
spec:
containers:
- name: nginx
image: nginx
---
apiVersion: v1
kind: Service
metadata:
name: app
labels:
app: app
spec:
ports:
- port: 80
targetPort: 80
selector:
app: app
externalIPs:
- 13.51.55.82
When I apply it then everything seems to work just fine. I can port-forward my app service to localhost and I can see the output (kubectl port-forward svc/app 9999:80 -> curl localhost:9999).
But the problem is I cannot reach this service via public IP.
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
app ClusterIP 10.100.140.38 13.51.55.82 80/TCP 49m
kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 62m
$ curl 13.51.55.82:80
curl: (7) Failed to connect to 13.51.55.82 port 80: Connection refused
Thoughts
For me it looks like the service is not connected to node itself. When I ssh to the node and setup simple web server on port 80 it respond immediately.
I know I can use NodePort but in my case I want finally use fixed port 4000 and NodePort allow me only to use ports in range 30000-32768.
Question
I want to be able to curl my service via public IP on certain port below 30000 (NodePort doesn't apply).
How can I make it work with Kubernetes Service externalIPs on EKS cluster?
Edit I:
FYI: I do not want to use LoadBalancer.
I running a kubernetes LB but the external ip says "pending", looks like it is trying to get a IP but, I need it as "localhost" to access it in my browser:
What do I miss?
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/dev-pypi LoadBalancer 10.106.128.15 <pending> 80:30914/TCP 2m7s
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 47h
service/qa-pypi LoadBalancer 10.97.62.94 <pending> 8200:30114/TCP 94m
Thanks in advance.
This is my yaml file:
kind: Service
apiVersion: v1
metadata:
name: qa-pypi
spec:
type: LoadBalancer
selector:
app: pypi-qa
ports:
- protocol: TCP
port: 8200
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: qa-pypi
labels:
app: pypi-qa
spec:
replicas: 2
selector:
matchLabels:
app: pypi-qa
template:
metadata:
labels:
app: pypi-qa
spec:
containers:
- name: pypi-qa
imagePullPolicy: IfNotPresent
image: myimg2
ports:
- containerPort: 8080
volumeMounts:
- name: storageqa
mountPath: /app/local
volumes:
- name: storageqa
persistentVolumeClaim:
claimName: persistvolumeqa
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: persistvolumeqa
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
Thank you !!!!!!!!!!!!!!!!!!!!
You already have here a lot of advises how to use localhost, what you can do with NodePort. Let me fill the gap and explain why you see Pending state.
Look, minikube itself, doesnt allocate and provide you LoadBalancer ip-address. LoadBalancer service type is widely used in cloud implementations, like EKS, AKS, GKE and others. Because cloud service providers create you loadbalancer in the background as soon as you chose this type.
If you want to use LoadBalancer with minikube you should configure minikube first. What you can do is use built-in metallb minikube addon.
MetalLB had addressed the gap and provides the network LoadBalancer
implementation as an addon.
During metallb installation and configuration you are able to set the range og local ip-adressed that would be assigned to LoadBalancer service instead of Pending state
If you want to know more about this and check real example of configuration - check for example MetalLB Configuration in Minikube — To enable Kubernetes service of type “LoadBalancer” article
you can use ingress to access your application from the browser
ingress
settingup_ingress
tutorial
if you doesn't find your host after setup it may need to register at root ./etc/hosts
127.0.0.1 localhost
127.0.0.1 https-my-nginx.com
To get expose your app in localhost you can try the NodePort type. If you are trying with kubeadm you can use NodePort type or ClusterIp type, so that you can able to expose it locally. Usually, Load-balancer is widely used in deployment on cloud machines.
If you are trying with minikube, then run minikube tunnel so that external-IP will be added for Loadbalancer.
you have to configure external IP on your machine(platform) first then only external ip will be allocated to your service. It can be VIP which you can advertise or address pool needs to make, other wise setting up ingress will also not going to serve the purpose as ingress itself needs external IP to be allocated for communication.
it's a networking thing you need to setup on your private machine.
if you don't want to use this option then can go for node port with that you can access externally with nodeip:nodeport.
Because docker supports out of the box kubernetes (on my Mac) I thought I try it out and see if I can load balance a simple webservice. For that, I created a simple image, which exposes port 3000 and only returns Hello World. And I created a k8s config yaml
apiVersion: v1
kind: Service
metadata:
name: hello-kubernetes
spec:
type: NodePort
externalIPs:
- 192.168.2.85
ports:
- port: 8080
targetPort: 3000
selector:
app: hello-kubernetes
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-kubernetes
spec:
replicas: 3
selector:
matchLabels:
app: hello-kubernetes
template:
metadata:
labels:
app: hello-kubernetes
spec:
containers:
- name: hello-kubernetes
image: hello/world:latest
ports:
- containerPort: 3000
Apply it
$> kubectl apply -f ./example.yaml
I see 3 pods running, and a service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-kubernetes NodePort 10.99.38.46 192.168.2.85 8080:30244/TCP 42m
I've used NodePort above, but I'm not sure if I can use Loadbalancer here as well.
Anyway, in the browser I get the message This site can’t be reached when I goto http://192.168.2.85:8080 or `http://192.168.2.85:30244 (I never know which port to use)
So, I think I'm close, but I still missed something :( Any help would be appreciated!
the port number is wrong.
use http://NODEIP:NODEPORT
in your case, try
http://NODEIP:30244
k explain service.spec.externalIPs
KIND: Service VERSION: v1
FIELD: externalIPs <[]string>
DESCRIPTION:
externalIPs is a list of IP addresses for which nodes in the cluster will
also accept traffic for this service. These IPs are not managed by
Kubernetes. The user is responsible for ensuring that traffic arrives at a
node with this IP. A common example is external load-balancers that are not
part of the Kubernetes system.
Problem here is we don't know your network settings. IS this minikube for mac? Is the 192.168.2.x network reachable for you? In my case using minikube all I had to do was to edit the externalIP to be reachable from my network. So what I did to get this working was:
minikube IP in my case 192.168.99.100 (IP address of minikubeVM)
changed externalIP to 192.168.99.100
k get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-kubernetes NodePort 10.105.212.118 192.168.99.100 8080:32298/TCP 46m
And I was able to reach the application using 192.168.99.100:8080.
Also note that in your case you have 8081 port (But I guess P Ekambaram already mentioned this).
I have the following pods hello-abc and hello-def.
And I want to send data from hello-abc to hello-def.
How would pod hello-abc know the IP address of hello-def?
And I want to do this programmatically.
What's the easiest way for hello-abc to find where hello-def?
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: hello-abc-deployment
spec:
replicas: 1
template:
metadata:
labels:
app: hello-abc
spec:
containers:
- name: hello-abc
image: hello-abc:v0.0.1
imagePullPolicy: Always
args: ["/hello-abc"]
ports:
- containerPort: 5000
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: hello-def-deployment
spec:
replicas: 1
template:
metadata:
labels:
app: hello-def
spec:
containers:
- name: hello-def
image: hello-def:v0.0.1
imagePullPolicy: Always
args: ["/hello-def"]
ports:
- containerPort: 5001
---
apiVersion: v1
kind: Service
metadata:
name: hello-abc-service
spec:
ports:
- port: 80
targetPort: 5000
protocol: TCP
selector:
app: hello-abc
type: NodePort
---
apiVersion: v1
kind: Service
metadata:
name: hello-def-service
spec:
ports:
- port: 80
targetPort: 5001
protocol: TCP
selector:
app: hello-def
type: NodePort
Preface
Since you have defined a service that routes to each deployment, if you have deployed both services and deployments into the same namespace, you can in many modern kubernetes clusters take advantage of kube-dns and simply refer to the service by name.
Unfortunately if kube-dns is not configured in your cluster (although it is unlikely) you cannot refer to it by name.
You can read more about DNS records for services here
In addition Kubernetes features "Service Discovery" Which exposes the ports and ips of your services into any container which is deployed into the same namespace.
Solution
This means, to reach hello-def you can do so like this
curl http://hello-def-service:${HELLO_DEF_SERVICE_PORT}
based on Service Discovery https://kubernetes.io/docs/concepts/services-networking/service/#environment-variables
Caveat: Its very possible that if the Service port changes, only pods that are created after the change in the same namespace will receive the new environment variables.
External Access
In addition, you can also reach this your service externally since you are using the NodePort feature, as long as your NodePort range is accessible from outside.
This would require you to access your service by node-ip:nodePort
You can find out the NodePort which was randomly assigned to your service with kubectl describe svc/hello-def-service
Ingress
To reach your service from outside you should implement an ingress service such as nginx-ingress
https://github.com/helm/charts/tree/master/stable/nginx-ingress
https://github.com/kubernetes/ingress-nginx
Sidecar
If your 2 services are tightly coupled, you can include both in the same pod using the Kubernetes Sidecar feature. In this case, both containers in the pod would share the same virtual network adapter and accessible via localhost:$port
https://kubernetes.io/docs/concepts/workloads/pods/pod/#uses-of-pods
Service Discovery
When a Pod is run on a Node, the kubelet adds a set of environment
variables for each active Service. It supports both Docker links
compatible variables (see makeLinkVariables) and simpler
{SVCNAME}_SERVICE_HOST and {SVCNAME}_SERVICE_PORT variables, where the
Service name is upper-cased and dashes are converted to underscores.
Read more about service discovery here:
https://kubernetes.io/docs/concepts/services-networking/service/#environment-variables
You should be able to reach hello-def-service from pods in hello-abc via DNS as specified here: https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#services
However, kube-dns or CoreDNS has to be configured/installed in your k8s cluster before DNS records can be utilized in your cluster.
Specifically, you should be reach hello-def-service via the DNS record http://hello-def-service for the service running in the same namespace as hello-abc-service
And you should be able to reach hello-def-service running in another namespace ohter_namespace via the DNS record hello-def-service.other_namespace.svc.cluster.local.
If, for some reason, you do not have DNS add-ons installed in your cluster, you still can find the virtual IP of the hello-def-service via environment variables in hello-abc pods. As is documented here.
I created a pod with an api and web docker container in kuberneters using a yml file (see below).
apiVersion: v1
kind: Pod
metadata:
name: test
labels:
purpose: test
spec:
containers:
- name: api
image: gcr.io/test-1/api:latest
ports:
- containerPort: 8085
name: http
protocol: TCP
- name: web
image: gcr.io/test-1/web:latest
ports:
- containerPort: 5000
name: http
protocol: TCP
It show my pod is up and running
NAME READY STATUS RESTARTS AGE
test 2/2 Running 0 5m
but I don't know how to expose it from here.
it seems odd I would have to run kubectl run .... again as the pod is already running. It does not show a deployment though.
if I try something like
kubectl expose deployment test --type="NodePort"--port 80 --target-port 5000
it complains about deployments.extensions "test' not found. What is the cleanest way to deploy from here?
To expose a deployment to the public internet, you will want to use a Service. The service type LoadBalancer handles this nicely, as you can just use pod selectors in the yaml file.
So if my deployment.yaml looks like this:
kind: Deployment
apiVersion: apps/v1beta2
metadata:
name: test-dply
spec:
selector:
# Defines the selector that can be matched by a service for this
deployment
matchLabels:
app: test_pod
template:
metadata:
labels:
# Puts the label on the pod, this must match the matchLabels
selector
app: test_pod
spec:
# Our containers for training each model
containers:
- name: mycontainer
image: myimage
imagePullPolicy: Always
command: ["/bin/bash"]
ports:
- name: containerport
containerPort: 8085
Then the service that would link to it is:
kind: Service
apiVersion: v1
metadata:
# Name of our service
name: prodigy-service
spec:
# LoadBalancer type to allow external access to multiple ports
type: LoadBalancer
selector:
# Will deliver external traffic to the pod holding each of our containers
app: test_pod
ports:
- name: sentiment
protocol: TCP
port: 80
targetPort: containerport
You can deploy these two items by using kubectl create -f /path/to/dply.yaml and kubectl create -f /path/to/svc.yaml. Quick note: The service will allocate a public IP address, which you can find using kubectl get services with the following output:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
carbon-relay ClusterIP *.*.*.* <none> 2003/TCP 78d
comparison-api LoadBalancer *.*.*.* *.*.*.* 80:30920/TCP 15d
It can take several minutes to allocate the ip, just a forewarning. But the LoadBalancer's ip is fixed, and you can delete the pod that it points to and re-spin it without consequence. So if I want to edit my test.dply, I can without worrying about my service being impacted. You should rarely have to spin down services
You have created a pod, not a deployment.
Then you have exposed a deployment (and not your pod).
Try:
kubectl expose pod test --type=NodePort --port=80 --target-port=5000
kubectl expose pod test --type=LoadBalancer --port=XX --target-port=XXXX
If you already have pod and service running, you can create an ingress for the service you want to expose to the internet.
If you want to create it through console, Google Cloud provides really easy way to create an ingress from an existing service. Go to Services and Ingress tab, select the service, click on create ingress, fill the name and other mandatory fields.
or you can create using yaml file
apiVersion: "networking.k8s.io/v1"
kind: "Ingress"
metadata:
name: "example-ingress"
namespace: "default"
spec:
defaultBackend:
service:
name: "example-service"
port:
number: 8123
status:
loadBalancer: {}