I've provisioned a kubernetes cluster on my own couple of virtual machines via kubespray. Kubespray uses project-calico as default network-plugin which fits my requirements of proxying services in the cluster network to the outer world pretty well.
Kubespray deploys the apiserver itself as a ClusterIP Service. To make it reachable from outside it defines an Endpoint of this service with the master nodes Host IP Adress, which is routed to the internal ClusterIP by Calico as far as I could figure it out by myself.
My Question is: How is it possible to define my own endpoint (for another service), as these get implicietly defined already by provisioning the service.yaml and cannot be overwritten. I would like to follow a similar approach to get my Rook/Ceph Dashboard visible from outside the cluster.
EDIT: Note that kubectl get ingresses.networking.k8s.io --all-namespaces returns No resources found. and kubectl describe service kubernete returns
Name: kubernetes
Namespace: default
Labels: component=apiserver
provider=kubernetes
Annotations: <none>
Selector: <none>
Type: ClusterIP
IP: 10.233.0.1
Port: https 443/TCP
TargetPort: 6443/TCP
Endpoints: 192.168.103.254:6443
Session Affinity: None
Events: <none>
I'll refer to your question:
How is it possible to define my own endpoint?
You'll have to:
1 ) Create a Service without a Pod selector:
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 9376
(At this point, no auto-generated Endpoints will be created by K8S because it can't decide to which pods those Endpoints should be referring).
2 ) Crate an Endpoints object and map it to the desired network address and port where the external resource is running:
apiVersion: v1
kind: Endpoints
metadata:
name: my-service
subsets:
- addresses:
- ip: 192.0.2.45
ports:
- port: 9376
(*) Notice that there should be a match between the service name and the name of the Endpoints object.
I am not exactly sure if what You mean but i think what You are looking for is ability to expose services externally.
You can expose Your services like Rook/Ceph Dashboard with "Publishing Services" (service types that expose internal services externally).
As quoted from kubernetes documentation:
For some parts of your application (for example, frontends) you may
want to expose a Service onto an external IP address, that’s outside
of your cluster.
Kubernetes ServiceTypes allow you to specify what kind of Service
you want. The default is ClusterIP.
Type values and their behaviors are:
ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the
cluster. This is the default ServiceType.
NodePort:
Exposes the Service on each Node’s IP at a static port (the
NodePort). A ClusterIP Service, to which the NodePort
Service routes, is automatically created. You’ll be able to contact
the NodePort Service, from outside the cluster, by requesting
<NodeIP>:<NodePort>.
LoadBalancer:
Exposes the Service externally using a cloud provider’s load balancer.
NodePort and ClusterIP Services, to which the external load
balancer routes, are automatically created.
ExternalName:
Maps the Service to the contents of the externalName field (e.g.
foo.bar.example.com), by returning a CNAME record with its value. No proxying of any kind is set up.
Here is an example from documentation.
You can also define the Services with yaml manifests like this:
apiVersion: v1
kind: Service
metadata:
name: examplelb
spec:
type: LoadBalancer
selector:
app: asd
ports:
-
name: koala
port: 22223
targetPort: 22225
nodePort: 31913
-
name: grisly
port: 22224
targetPort: 22226
nodePort: 31914
-
name: polar
port: 22225
targetPort: 22227
nodePort: 31915
This makes pods with label: app: asd have following ports exposed with pattern
internal port 22223 exposed on 31913.
$ kubectl get svc examplelb
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
examplelb LoadBalancer 10.111.8.204 <pending> 22223:31913/TCP,22224:31914/TCP,22225:31915/TCP 7d2h
If service with type LoadBalancer has External-IP pending you can still access all those ports on each node as NodePort.
Hope this helps.
Related
I have following configuration of a service:
apiVersion: v1
kind: Service
metadata:
name: academy-backend-service
spec:
selector:
app: academy-backend-app
type: NodePort
ports:
- port: 8081
targetPort: 8081
nodePort: 30081
Behind this service there is a deployment that runs a docker image of a spring boot application that expose port 8081.
When I try to reach the application from browser on http://localhost:30081 I don't get anything (not reachable). However if I connect inside minikube cluster, the application is available on http:{servcieip}:8081.
Any clues what is not configured properly? I thought that nodePort is enough.
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
academy-backend-service NodePort 10.97.44.87 <none> 8081:30081/TCP 34m
Use NodePorts to expose the service nodePort on all nodes in the cluster.
From https://kubernetes.io/docs/concepts/services-networking/service/
NodePort: Exposes the Service on each Node's IP at a static port (the NodePort). To make the node port available, Kubernetes sets up a cluster IP address, the same as if you had requested a Service of type: ClusterIP.
If you want to expose your service outside the cluster, use LoadBalancer type:
LoadBalancer: Exposes the Service externally using a cloud provider's load balancer.
or use ingress-controller which is a reverse-proxy that routs traffics from outside to your cluster:
https://github.com/kubernetes/ingress-nginx
I have a set of pods running based on the following fleet:
apiVersion: "agones.dev/v1"
kind: Fleet
metadata:
name: bungee
spec:
replicas: 2
template:
metadata:
labels:
run: bungee
spec:
ports:
- name: default
containerPort: 25565
protocol: TCP
template:
spec:
containers:
- name: bungee
image: a/b:test
I can access these pods outside the cluster with <node-IP>:<port> where the port is random per pod given by Agones.
My goal is to be able to connect to these pods through a single IP, meaning I have to add some sort of load balancer. I tried using this service of type LoadBalancer, but I can't connect to any of the pods with it.
apiVersion: v1
kind: Service
metadata:
name: bungee-svc
spec:
type: LoadBalancer
loadBalancerIP: XXX.XX.XX.XXX
ports:
- port: 25565
protocol: TCP
selector:
run: bungee
externalTrafficPolicy: Local
Is a service like this the wrong approach here, and if so what should I use instead? If it is correct, why is it not working?
Edit: External IP field says pending while checking the service status. I am running Kubernetes on bare-metal.
Edit 2: Attempting to use NodePort as suggested, I see the service has not been given an external IP address. Trying to connect to <node-IP>:<nodePort> does not work. Could it be a problem related to the selector?
LoadBalancer Services could have worked, in clusters that are integrating with the API of the cloud provider hosting your Kubernetes nodes (cloud-controller-manager component). Since this is not your case, you're looking for a NodePort Service.
Something like:
apiVersion: v1
kind: Service
metadata:
name: bungee-svc
spec:
type: NodePort
ports:
- port: 25565
protocol: TCP
selector:
run: bungee
Having created that service, you can check its description - or yaml/json representation:
# kubectl describe svc xxx
Type: NodePort
IP: 10.233.24.89 <- ip within SDN
Port: tcp-8080 8080/TCP <- ports within SDN
TargetPort: 8080/TCP <- port on your container
NodePort: tcp-8080 31655/TCP <- port exposed on your nodes
Endpoints: 10.233.108.232:8080 <- pod:port ...
Session Affinity: None
Now, I know the port 31655 was allocated to my NodePort Service -- ports are unique on your cluster, they are picked within a range, depends on your cluster configuration.
I can connect to my service, accessing any Kubernetes node IP, on the port that was allocated to my NodePort service.
curl http://k8s-worker1.example.com:31655/
As a sidenote: a LoadBalancer Service extends a NodePort Service. While the externalIP won't ever show up, note that your Service was already allocated with its own port, as any NodePort Service - which is meant to receive traffic from whichever LoadBalancer would have been configured on behalf of your cluster, onto the cloud infrastructure it is integrated with.
And ... I have to say I'm not familiar with Agones. When you say "I can access these pods outside the cluster with <node-IP>:<port> where the port is random per pod given by Agones". Are you sure ports are allocated on a per-pod basis, and bound to a given node? Or could it be they're already using a NodePort Service. Give it another look: have you tried connecting that port on other nodes of your cluster?
How I understand that I could be able to talk with other pods from a specific pod by sending from within the pod an HTTP request with the fully qualified domain name of the service (FQDN).
The system runs locally with minikube.
The service's YML -
apiVersion: v1
kind: Service
metadata:
name: kubia
spec:
sessionAffinity: ClientIP
ports:
- port: 80
targetPort: 8080
selector:
app: kubia
The describe of the service -
Name: kubia
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=kubia
Type: ClusterIP
IP: 10.111.178.111
Port: <unset> 80/TCP
TargetPort: 8080/TCP
Endpoints: 172.17.0.7:8080,172.17.0.8:8080,172.17.0.9:8080
Session Affinity: ClientIP
Events: <none>
I'm trying to do that with -
kubectl exec -it kubia-gqd5l bash
where kubia-gqd5l is the pod.
In the bash I tried to sent a request by -
curl http://kubia
Where kubia is the name of the service.
and I got error -
curl: (6) Could not resolve host: kubia.
It is important to note that I manage to communicate with the service by -
kubectl exec kubia-gqd5l -- curl -s http://10.111.178.111
any idea?
Kubernetes clusters usually have DNS deployed. That allows pod to pod communications within the cluster (among other things) by using the name of the corresponding Kubernetes services. See https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
Does your Kubernetes cluster/minikube have DNS running?
Something else to check is the selector in the Service definition - make sure the pod/deployment has the app: kubia label as specified in the selector.
Otherwise, and per the doc at the link above, because the lookup of the service is from a pod in the same namespace, it shouldn't be needed to use the namespace along with the service name: (quote) "...Assume a Service named foo in the Kubernetes namespace bar. A Pod running in namespace bar can look up this service by simply doing a DNS query for foo. A Pod running in namespace quux can look up this service by doing a DNS query for foo.bar".
Have a look at this answer 2 Kubernetes pod communicating without knowing the exposed address, to target a service it's better to add the namespace with the service.
I have create a pod with the below yaml definition.
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
spec:
containers:
- name: myapp-container
image: praveensripati/docker-demo:1.2
ports:
- containerPort: 3000
And now I expose the pod, which creates a service.
kubectl expose pod myapp-pod --type=NodePort
The port 3000 on the container is exposed to port 31728 on the nodes. And I am able to do access the page using curl on port 31728.
kubectl get service myapp-pod
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
myapp-pod NodePort 10.107.254.254 <none> 3000:31728/TCP 5s
This time I wanted to expose the service not a random port, but on port 80. And so I specify the port number as 80, by using --port. The service details are a bit odd. It says that port 80 on the container is exposed to port 31316 on the nodes. Also, I am able to access the page using curl on the random port (31316 in this case) and not port 80.
kubectl expose pod myapp-pod --type=NodePort --target-port=3000 --port=80
kubectl get service myapp-pod
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
myapp-pod NodePort 10.105.123.73 <none> 80:31316/TCP 12s
I am not able to expose a service on a specific port and not on a random port. I tried a few combinations and read the k8s documentation, but no success.
How do I expose a service on a specific port instead of a random port?
Your question is about exposing the NodePort type of service on a specific port. For that you need to specify the nodePort field under ports in your service definition.
kind: Service
apiVersion: v1
metadata:
name: my-service
spec:
selector:
app: myapp
ports:
- protocol: TCP
port: 3000
nodePort: 32321
type: NodePort
Note that it has to be within a given range provided in the configs. Which defaults to 30000-32767. This range can be specified in the kube-apiserver configs using the --service-node-port-range option.
When an existing Dashboard service already exists, remove it.
kubectl delete service kubernetes-dashboard -n kube-system
Expose the Dashboard deployment as a NodePort.
kubectl expose deployment kubernetes-dashboard -n kube-system --type=NodePort
The above will assign a random port >= 30000. So use the Patch command to assign the port to a known, unused and desired port >= 30000.
kubectl patch service kubernetes-dashboard --namespace=kube-system --type='json' --patch='[{"op": "replace", "path": "/spec/ports/0/nodePort", "value":30000}]'
Caution: Never expose your dashboard publicly without authentication.
If your cluster does not have a LoadBalancer Provider, you can specify externalIPs in IP of nodes' network interface.
For example:
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
type: ClusterIP
externalIPs:
- 125.100.99.101 # Node1-IP
- 125.100.99.102 # Node2-IP
- 192.168.55.112 # Node2-IP2
selector:
pod: nginx
ports:
- name: http
port: 80
- name: https
port: 443
This will listen 80 and 443 on the specified node, and forward to the nginx service.
I had the same problem and the only way I found to do it without modifying the files was:
k expose --type=NodePort deployment nginx --port 80 --name nginx-ep-patch --overrides '{ "apiVersion": "v1","spec":{"ports": [{"port":80,"protocol":"TCP","targetPort":80,"nodePort":30080}]}}'
service/nginx-ep-patch exposed
In this way we path online the configuration and the port 30080 has been exposed:
$ k describe svc nginx-ep-patch
Name: nginx-ep-patch
Namespace: default
Labels: app=nginx
Annotations: <none>
Selector: app=nginx
Type: NodePort
IP: 10.96.51.148
Port: <unset> 80/TCP
TargetPort: 80/TCP
NodePort: <unset> 30080/TCP
Endpoints: 10.244.0.6:80
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
I will try to answer your query here.
Also, I am able to access the page using curl on the random port
(31316 in this case) and not port 80.
-- Because, kubernetes exposed the port 31316 on the host (maps to the service) and hence it can be accessed on host:31316.
-- Service port is visible only within the kubernetes cluster. You can exec into a pod container and do a curl on servicename:service port instead of the NodePort.
Note the terms - container port: the port container listens on. Service port: the port where kubernetes service is exposed on cluster internal ip and mapped to the container port. Nodeport: the port exposed on the host and mapped to kubernetes service.
we can expose Kubernetes service on specific node port.
Port value must be between 30000-32767.
We can expose service to specific port of below service types:
NodePort
LoadBalancer
Find the sample myservice.yaml file below:
apiVersion: v1
kind: Service
metadata:
name: app1
spec:
type: NodePort/LoadBalancer
ports:
- name: "80"
port: 80
nodePort: 32062
targetPort: 80
selector:
appone: app1
app: test
Note: In above service yaml file we can specify service type either NodePort or Loadbalancer.
For those who need to use kubectl commands, you can create a NodePort service with a specified port using create nodeport command:
kubectl create nodeport NAME [--tcp=port:targetPort] [--dry-run=server|client|none]
For example:
kubectl create service nodeport myservice --node-port=31000 --tcp=3000:80
You can check Kubectl reference for more:
https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#-em-service-nodeport-em-
I'm running a local kubernetes bundled with docker on Mac OS.
How can I expose a service, so that I can access the service via a browser on my Mac?
I've created:
a) deployment including apache httpd.
b) service via yaml:
apiVersion: v1
kind: Service
metadata:
name: apaches
spec:
selector:
app: web
type: NodePort
ports:
- protocol: TCP
port: 80
externalIPs:
- 192.168.1.10 # Network IP of my Mac
My service looks like:
$ kubectl get service apaches
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
apaches NodePort 10.102.106.158 192.168.1.10 80:31137/TCP 14m
I can locally access the service in my kubernetes cluster by wget $CLUSTER-IP
I tried to call http://192.168.1.10/ on my Mac, but it doesn't work.
This question deals to a similar issue. But the solution does not help, because I do not know which IP I can use.
Update
Thanks to Michael Hausenblas I worked out a solution using Ingress.
Nevertheless there are still some open questions:
What is the meaning of a service's externalIP? Why do I need an externalIP when I do not directly access a service from external?
What is the meaning of the service port 31137?
The kubernetes docs describe a method to [publish a service in minikube via NodePort][4]. Is this also possible with kubernetes bundled on docker?
There are several solutions to expose services in kubernetes:
http://alesnosek.com/blog/2017/02/14/accessing-kubernetes-pods-from-outside-of-the-cluster/
Here are my solutions according to alesnosek for a local kubernetes bundled with docker:
1. hostNetwork
hostNetwork: true
Dirty (the host network should not be shared for security reasons) => I did not check this solution.
2. hostPort
hostPort: 8086
Does not apply to services => I did not check this solution.
3. NodePort
Expose the service by defining a nodePort:
apiVersion: v1
kind: Service
metadata:
name: apaches
spec:
type: NodePort
ports:
- port: 80
nodePort: 30000
selector:
app: apache
4. LoadBalancer
EDIT
#MathObsessed posted the solution in his anwer.
5. Ingress
a. Install Ingress Controller
git clone https://github.com/jnewland/local-dev-with-docker-for-mac-kubernetes.git
kubectl apply -f nginx-ingress/namespaces/nginx-ingress.yaml -Rf nginx-ingress
b. Configure Ingress
kubectl apply -f apache-ing.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: apache-ingress
spec:
rules:
- host: localhost
http:
paths:
- path: /
backend:
serviceName: apaches
servicePort: 80
Now I can access my apache deployed with kubernetes by calling http://localhost/
Remarks for using local-dev-with-docker-for-mac-kubernetes
The repo simplifies the deployment of the offical ingress-nginx controller
For production use I would follow the official guide.
The repos ships with a tiny full featured ingress example. Very useful for getting quickly a working example application.
Further documentation
https://kubernetes.io/docs/concepts/services-networking/ingress
For those still looking for an answer. I've managed to achieve this by adding another Kube service just to expose my app to localhost calls (via browser or Postman):
kind: Service
apiVersion: v1
metadata:
name: apaches-published
spec:
ports:
- name: http
port: 8080
targetPort: 80
protocol: TCP
selector:
app: web
type: LoadBalancer
Try it now on: http://localhost:8080
Really simple example
METHOD1
$ kubectl create deployment nginx-dep --image=nginx --replicas=2
Get the pods
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-dep-5c5477cb4-76t9q 1/1 Running 0 7h5m
nginx-dep-5c5477cb4-9g84j 1/1 Running 0 7h5m
Access the pod using kubectl port
$ kubectl port-forward nginx-dep-5c5477cb4-9g84j 8888:80
Forwarding from 127.0.0.1:8888 -> 80
Forwarding from [::1]:8888 -> 80
Now do a curl to the localhost:8888
$ curl -v http://localhost:8888
METHOD2
You can expose port 80 of the deployment (where the application is runnin i.e. nginx port)
via a NodePort
$ kubectl expose deployment nginx-dep --name=nginx-dep-svc --type=NodePort --port=80
Get the service
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 31d
nginx-dep-svc NodePort 10.110.80.21 <none> 80:31239/TCP 21m
Access the deployment using hte NodePort
$ curl http://localhost:31239
As already mentioned in Matthias Ms answer there are several ways.
As the offical Kubernetes documentation specifically describes using a Service with a type NodePort I wanted to describe the workflow.
NodePort: Exposes the Service on each Node’s IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service routes, is automatically created. You’ll be able to contact the NodePort Service, from outside the cluster, by requesting <NodeIP>:<NodePort>.
If you set the type field to NodePort, the Kubernetes control plane allocates a port from a range specified by --service-node-port-range flag (default: 30000-32767). Each node proxies that port (the same port number on every Node) into your Service. Your Service reports the allocated port in its .spec.ports[*].nodePort field.
Setup a Service with a type of NodePort
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
selector:
app: MyApp
ports:
- protocol: TCP
port: 80
targetPort: 9376
clusterIP: 10.0.171.239
type: NodePort
Then you can check on which port the Service is exposed to via
kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
my-service NodePort 10.103.218.215 <none> 9376:31040/TCP 52s
and access it via localhost using the exposed port. E.g.
curl http://localhost:31040