I am using a baremetal cluster of 1 master and 2 nodes on premise in my home lab with istio, metallb and calico.
I want to create a DNS server in kubernetes that translates IPs for the hosts on the LAN.
Is it possible to use the coreDNS already installed in k8s?
Yes, it's possible but there are some points to consider when doing that. Most of them are described in the Stackoverflow answer below:
Stackoverflow.com: Questions: How to expose Kubernetes DNS externally
For example: The DNS server would be resolving the queries that are internal to the Kubernetes cluster (like nslookup kubernetes.default.svc.cluster.local).
I've included the example on how you can expose your CoreDNS to external sources and add a Service that would be pointing to some IP address
Steps:
Modify the CoreDNS Service to be available outside.
Modify the configMap of your CoreDNS accordingly to:
CoreDNS.io: Plugins: K8s_external
Create a Service that is pointing to external device.
Test
Modify the CoreDNS Service to be available outside.
As you are new to Kubernetes you are probably aware on how Services work and which can be made available outside. You will need to change your CoreDNS Service from ClusterIP to either NodePort or LoadBalancer (I'd reckon LoadBalancer would be a better idea considering the metallb is used and you will access the DNS server on a port: 53)
$ kubectl edit --namespace=kube-system service/coredns (or kube-dns)
A side note!
CoreDNS is using TCP and UDP simultaneously, it could be an issue when creating a LoadBalancer. Here you can find more information on it:
Metallb.universe.tf: Usage (at the bottom)
Modify the configMap of your CoreDNS
If you would like to resolve domain like for example: example.org you will need to edit the configMap of CoreDNS in a following way:
$ kubectl edit configmap --namespace=kube-system coredns
Add the line to the Corefile:
k8s_external example.org
This plugin allows an additional zone to resolve the external IP address(es) of a Kubernetes service. This plugin is only useful if the kubernetes plugin is also loaded.
The plugin uses an external zone to resolve in-cluster IP addresses. It only handles queries for A, AAAA and SRV records; all others result in NODATA responses. To make it a proper DNS zone, it handles SOA and NS queries for the apex of the zone.
-- CoreDNS.io: Plugins: K8s_external
Create a Service that is pointing to external device.
Following on the link that I've included, you can now create a Service that will point to an IP address:
apiVersion: v1
kind: Service
metadata:
name: test
namespace: default
spec:
clusterIP: None
externalIPs:
- 192.168.200.123
type: ClusterIP
Test
I've used minikube with --driver=docker (with NodePort) but I'd reckon your can use the ExternalIP of your LoadBalancer to check it:
dig #192.168.49.2 test.default.example.org -p 32261 +short
192.168.200.123
where:
#192.168.49.2 - IP address of minikube
test.default.example.org - service-name.namespace.k8s_external_domain
-p 32261 - NodePort port
+short - to limit the output
Additional resources:
Linux.die.net: Man: Dig
Related
I am following this guide to expose a service running on my bare metal k8s cluster to the world.
The guide suggests using metallb for giving external access. The problem is, during the setup process of metallb, I am asked to give a range of available IP addresses.
The hosting provider I am using is very basic, and all I have is the IP address of the Linux instance that is running my K8s node. So my question is, how can I provision an IP address for assigning to my application? Is this possible with a single IP?
Alternatively I'd love get this done with a NodePort, but I need to support HTTPS traffic and I am not sure its possible if I go that way.
Specify a single IP using CIDR notation. your-ip/32 (192.168.10.0/32 for example)
Single IP Address Load Balancer
Using a single IP address is possible. In this case you don't need the speaker pods that announce the external IP addresses and thus no pod security labels. So, if you install using helm, prepare a metallb-values-helm.yaml file:
speaker:
enabled: false
Then install metallb:
kubectl create namespace metallb-system
helm repo add metallb https://metallb.github.io/metallb
helm install metallb metallb/metallb --namespace metallb-system -f metallb-values-helm.yaml
Now prepare a configuration of the public IP address in a metallb-config-ipaddress.yaml file:
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: metallb-ip
namespace: metallb-system
spec:
addresses:
# The IP address of the kubernetes master node
- 192.168.178.10/32
And apply:
kubectl apply -f metallb-config-ipaddress.yaml
Multiple Services Sharing the Same IP Address
This should already work for a single service. However, if you want to apply multiple services on different ports of the same IP address, you need to provide an annotation in every service manifest, as described here. A service manifest will look like:
apiVersion: v1
kind: Service
metadata:
name: cool-service
namespace: default
annotations:
metallb.universe.tf/allow-shared-ip: "key-to-share-192.168.178.10"
...
The string "key-to-share-192.168.178.10" is arbitrary, but must be equal for all services. If there is really just a single IP address in your pool (as specified above), you don't have to specify it as loadBalancerIP: 192.168.178.10. This would be only required if you had multiple IP addresses and wanted to select one. So that's all.
What's next?
You can also use nginx-ingress as ingress controller behind your metallb load balancer, which is still required to expose the nginx-ingress service. Then you can e. g. separate your services via subdomains (pointing to the same IP address) like
service1.domain.com
service2.domain.com
I have a single node Kubernetes cluster, installed using k3s on bare metal. I also run some services on the host itself, outside the Kubernetes cluster. Currently I use the external IP address of the machine (192.168.200.4) to connect to these services from inside the Kubernetes network.
Is there a cleaner way of doing this? What I want to avoid is having to reconfigure my Kubernetes pods if I decide to change the IP address of my host.
Possible magic I which existed: a Kubernetes service or IP that automagically points to my external IP (192.168.200.4) or a DNS name that points the node's external IP address.
That's what ExternalName services are for (https://kubernetes.io/docs/concepts/services-networking/service/#externalname):
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
type: ExternalName
externalName: ${my-hostname}
ports:
- port: 80
Then you can access the service from withing kubernetes as my-service.${namespace}.svc.cluster.local.
See: https://livebook.manning.com/concept/kubernetes/external-service
After the service is created, pods can connect to the external service
through the external-service.default.svc.cluster.local domain name (or
even external-service) instead of using the service’s actual FQDN.
This hides the actual service name and its location from pods
consuming the service, allowing you to modify the service definition
and point it to a different service any time later, by only changing
the externalName attribute or by changing the type back to ClusterIP
and creating an Endpoints object for the service—either manually or by
specifying a label selector on the service and having it created
automatically.
ExternalName services are implemented solely at the DNS level—a simple
CNAME DNS record is created for the service. Therefore, clients
connecting to the service will connect to the external service
directly, bypassing the service proxy completely. For this reason,
these types of services don’t even get a cluster IP.
This relies on using a resolvable hostname of your machine. On minikube there's a DNS alias host.minikube.internal that is setup to resolve to an IP address that routes to your host machine, I don't know if k3s supports something similar.
Thanks #GeertPt,
With minikube's host.minikube.internal in mind I search around and found that CoreDNS has a DNS entry for each host it's running on. This only seems the case for K3S.
Checking
kubectl -n kube-system get configmap coredns -o yaml
reveals there is the following entry:
NodeHosts: |
192.168.200.4 my-hostname
So if the hostname doesn't change, I can use this instead of the IP.
Also, if you're running plain docker you can use host.docker.internal to access the host.
So to sum up:
from minikube: host.minikube.internal
from docker: host.docker.internal
from k3s: <hostname>
I have two-zone, each has to master node. Today I created a simple ingress-nginx controller and successfully pointed a DNS test.example.com to one of the public IP in zone-1.
But now I want to create another nginx-controller in zone-2 and point test.example.com to the public IP address of that zone with cloud DNS.
What approach should I take? Is there any best practice?
Your question is unclear and needs to be improved with any minimal reproducible example. You can find the manual here.
According to your subject, you're using Kubernetes cluster in bare-metal, however you mentioned using cloud DNS. Where did you get cloud DNS?
If you're using pure bare metal, then consider using MetalLB.
MetalLB provides a network load-balancer implementation for Kubernetes clusters that do not run on a supported cloud provider, effectively allowing the usage of LoadBalancer Services within any cluster.
But this approach has a few other limitations one ought to be aware of, one of them is about Ingress status:
Because NodePort Services do not get a LoadBalancerIP assigned by definition, the NGINX Ingress controller does not update the status of Ingress objects it manages
$ kubectl get ingress
NAME HOSTS ADDRESS PORTS
test-ingress myapp.example.com 80
Despite the fact there is no load balancer providing a public IP address to the NGINX Ingress controller, it is possible to force the status update of all managed Ingress objects by setting the externalIPs field of the ingress-nginx Service.
Please see the example below:
Given the following 3-node Kubernetes cluster (the external IP is added as an example, in most bare-metal environments this value is )
$ kubectl get node
NAME STATUS ROLES EXTERNAL-IP
host-1 Ready master 203.0.113.1
host-2 Ready node 203.0.113.2
host-3 Ready node 203.0.113.3
one could edit the ingress-nginx Service and add the following field to the object spec
spec:
externalIPs:
- 203.0.113.1
- 203.0.113.2
- 203.0.113.3
which would in turn be reflected on Ingress objects as follows:
$ kubectl get ingress -o wide
NAME HOSTS ADDRESS PORTS
test-ingress myapp.example.com 203.0.113.1,203.0.113.2,203.0.113.3 80
See more detailed info here
I got the following service defined:
apiVersion: v1
kind: Service
metadata:
name: customerservice
spec:
type: NodePort
selector:
app: customerapp
ports:
- protocol: TCP
port: 31004
nodePort: 31004
targetPort: 8080
Current situation: I am able to hit the pod via the service IP.
Now my goal is to reach the customerservice via the name of the service, which does not work right now. So I would simply type http://customerservice:31004 instead of http://<IP>:31004.
DNS resolution of services is ONLY available within the cluster, provided by CoreDNS/KubeDNS.
Should you wish to have access to this locally on your machine, you'd need to use another tool. One such tool is kubefwd:
https://github.com/txn2/kubefwd
A slightly simpler solution, is to use port-forward; which is a very simple way to access a single service locally.
kubectl port-forward --namespace=whatever svs/service-name port
EDIT:// I've made the assumption that you want to use the service DNS locally, as I'm assuming by saying:
I would simply type http://customerservice:31004
is in the context of your web browser.
Normal Services are assigned a DNS A record for a name of the form my-svc.my-namespace.svc.cluster.local. This resolves to the cluster IP of the Service. This DNS entry is present inside the kubernetes cluster only and hence you're able to access the service by name from inside the kubernetes pod.
Now, if you want to access your kubernetes service by name from one of the node you need to modify the /etc/resolve.conf of your node with <svc_name>.<namespace>.svc.cluster.local, please have a look at following /etc/resolve.conf
search ec2.internal default.svc.cluster.local svc.cluster.local cluster.local
nameserver 10.96.0.10
nameserver is clusterIP of kube-dns service, you can find it using kubectl get svc kube-dns -n kube-system
Now you will be able to curl your service as curl ui.default.svc.cluster.local:80
I've setup a NodePort service using the following config:
wordpress-service.yaml
apiVersion: v1
kind: Service
metadata:
labels:
app: wordpress
name: wordpress
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: wordpress
Is this sufficient to access the service externally, if so how can I now access the service? What details do I need - and how do I determine them - for example node IP.
For Kubernetes on GCE:
We had the same question regarding services of type NodePort: How do we access node port services from our own host?
#ivan.sim 's answer (nodeIp:nodePort) is on mark however, you still wouldn't be able to access your service unless you add a firewall ingress (inbound to google cloud) traffic rule on the VPC network console to allow your host to be able to access your compute node
the above rule is dangerous and should be used only during development
You can find the node port using either the Google Cloud console or by running subsequent kubectl commands to find out the node running your pod which has your container. i.e kubectl get pods , kubectl describe pod your-pod-name, kubectl describe node node-that-runs-you-pod .status.addresses has your ExternalIP
It would be great if we could extract the node ip running our container in the pod using only a label/selector and a few line of commands, so here is what we did, in this case our selector is app: your-label:
$ nodename=$(kubectl get pods -o jsonpath='{.items[?(#.metadata.labels.app=="your-label")].spec.nodeName}')
$ nodeIp=$(kubectl get nodes -o jsonpath='{.items[?(#.metadata.name=="'$(echo $nodename)'")].status.addresses[?(#.type=="ExternalIP")].address}')
$ echo nodeIp
notice: we used json path to extract the information we desired, for more on json path see: json path
You could certainly turn this into a script that takes a label/selector as input and outputs an external ip of the node running your container !!!
To get the nodeport just type:
$ kubectl get services
under the PORT(S) columns you will see something like tagetPort:nodePort. this nodeport is what you want .
nodeIp:nodePort
When you define a service as type NodePort, every node in your cluster will proxy that port to your service. If you nodes are reachable from outside the Kubernetes cluster, you should be able to access the service at nodeIP:nodePort.
To determine nodeIP of a particular node, you can use either kubectl get no <node> -o yaml or kubectl describe no <node>. The status.Addresses field will be of interest. Generally, you will see fields like HostName, ExternalIP and InternalIP there.
To determine nodePort of your service, you can use either kubectl get svc wordpress -o yaml or kubectl describe svc wordpress. The spec.ports.nodePort is the port you need.
Service defined like this got assgned a high port number and is exposed on all your cluster nodes on that port (probably something like 3xxxx). Hard to tell the rest without proper knowledge of how your cluster is provisioned. kubectl get nodes should give you some knowledge about your nodes.
Although I assume you want to expose the service to the outside world. In the long run I suggest getting familiar with LoadBalancer type services and Ingress / IngressController