I have set up a baremetal k8 cluster ( 1 master node - intel NUC & 2 worker nodes on Raspberry pi). I managed to set up a metal-lb load balance and nginx ingress controller. I have launched two applications, ghost (listens on default port 2368) and nextcloud ( listens on default port 80) . I'm trying to access the applications from public ip myhomeserver.io ( to access the ghost application) and nextcloud.myhomeserver.io ( to access the next cloud application). I can access the ghost application but I can't seem to access nextcloud.Given below are the yaml files for ingress and services. Not sure where am I going wrong.
kubectl get services --all-namespaces
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 98d
ghost ghost-service ClusterIP 10.107.116.108 <none> 2368/TCP 7h37m
ingress-nginx ingress-nginx LoadBalancer 10.109.177.223 192.168.178.200 80:31619/TCP,443:30365/TCP 7d23h
kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 98d
nextcloud nextcloud-service ClusterIP 10.105.24.162 <none> 8080/TCP 137m
=============================================================================================================================
NAMESPACE NAME HOSTS ADDRESS PORTS AGE
ghost ingress-ghost myhomeserver.io 192.168.178.200 80 7d22h
nextcloud ingress-nextcloud nextcloud.myhomeserver.io 192.168.178.200 80 140m
=============================================================================================================================
cat ingress-object-ghost.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-ghost
namespace: ghost
spec:
rules:
- host: myhomeserver.io
http:
paths:
- backend:
serviceName: ghost-service
servicePort: 2368
=============================================================================================================================
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-nextcloud
namespace: nextcloud
spec:
rules:
- host: nextcloud.myhomeserver.io
http:
paths:
- backend:
serviceName: nextcloud-service
servicePort: 8080
================================================================================================================================
cat ingress-object-nextcloud.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-nextcloud
namespace: nextcloud
spec:
rules:
- host: nextcloud.myhomeserver.io
http:
paths:
- backend:
serviceName: nextcloud-service
servicePort: 8080
===================================================================================
apiVersion: apps/v1
kind: Deployment
metadata:
name:
deployment-nextcloud
namespace: nextcloud
labels:
env: prod
app: nextcloud-app
spec:
template:
metadata:
name: nextcloud-app-pod
labels:
app: nextcloud-app
env: production
spec:
containers:
- name: nextcloud
image: arm32v7/nextcloud
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
volumeMounts:
- mountPath: /var/www/html
name: nextcloud-data
securityContext:
privileged: True
volumes:
- name: nextcloud-data
persistentVolumeClaim:
claimName: pvc-nextcloud
nodeSelector:
kubernetes.io/arch: arm
replicas: 2
selector:
matchLabels:
app: nextcloud-app
================================================================================================================
apiVersion: v1
kind: Service
metadata:
name: nextcloud-service
namespace: nextcloud
labels:
app: nextcloud-app
spec:
type: ClusterIP
selector:
app: nextcloud-app
ports:
- port: 8080
targetPort: 8080
protocol: TCP
Note your nginx ingress controller is running in the ghost namespace so it only knows about the ghost service. You need to have another ingress controller for your nextcloud namespace if you want to have an ingress there. If you don't want another ingress controller then you can resolve the nextcloud service by targeting its dns in the following way servicename.namespacename.svc.cluster.local
On a side, there is not really a point in dividing your applications that much. Kubernetes already gives you enough privacy among applications in the same namespace.
UPDATE
Ingress that works for you given you have only 1 INGRESS CONTROLLER. Since there are two services I have added a path rule which will be rewritten to / so each service will receive a clean URI. Use myhomeserver.io/ghost to reach ghost and myhomeserver.io/nextcloud to reach nextcloud.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-ghost
namespace: ghost
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: myhomeserver.io
http:
paths:
- path: /ghost
backend:
serviceName: ghost-service
servicePort: 2368
- path: /nextcloud
backend:
serviceName: nextcloud-service.nextcloud.svc.cluster.local
servicePort: 8080
UPDATE 2
So your ingress controller is running in the ghost namespace. Thus, your ingress has to be deployed in the ghost namespace. Note the http rules for each host.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-ghost
namespace: ghost
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: myhomeserver.io
http:
paths:
- path: /
backend:
serviceName: ghost-service
servicePort: 2368
- host: nextcloud.myhomeserver.io
http:
- path: /
backend:
serviceName: nextcloud-service.nextcloud.svc.cluster.local
servicePort: 8080
Related
I have a cluster IP service and a Ingress. What should my custom domain name point to if I need to route traffic using Ingress? Backend is plain http.
Do I have to create a AWS Loadbalancer with target groups pointing to k8s nodes? And use domain alias pointing to aws loadbalancer? I was reading this K8s article and they're pointing to a subdomain.
Ingress.yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
creationTimestamp: "2022-08-05T00:50:41Z"
generation: 1
labels:
app: testing
name: httpd
namespace: default
spec:
rules:
- host: www.example.com
http:
paths:
- backend:
service:
name: httpd
port:
number: 8080
path: /
pathType: ImplementationSpecific
tls:
- hosts:
- www.example.com
secretName: tls-secret
status:
loadBalancer: {}
service.yaml:
kind: Service
metadata:
creationTimestamp: "2022-08-05T00:50:41Z"
labels:
app: testing
name: httpd
namespace: default
spec:
clusterIP: 100.65.xxx.xx
clusterIPs:
- 100.65.xxx.xx
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
selector:
name: httpd
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
Yes you have to create the Load Balancer however that will auto managed by the K8s service.
You can use the Nginx or other ingress controller as per requirement.
You can checkout this Nice official doc from AWS : https://aws.amazon.com/blogs/opensource/network-load-balancer-nginx-ingress-controller-eks/
Once you deploy the Nginx ingress controller it will manage the ingress resource and the Nginx controller will get the public LB.
Example :
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- anthonycornell.com
secretName: tls-secret
rules:
- host: anthonycornell.com
http:
paths:
- path: /apple
backend:
serviceName: apple-service
servicePort: 5678
- path: /banana
backend:
serviceName: banana-service
servicePort: 5678
I have deployed the kubernetes cluster on vagrant machine with config as:
one master and two worker nodes.
Two services are deployed with named as nodeport-svc-rc and nodeport-svc-rs
Services config:
# nodeport-svc-rc
apiVersion: v1
kind: Service
metadata:
name: nodeport-svc-rc
spec:
type: NodePort
ports:
- port: 5001
targetPort: 5001
nodePort: 30001
selector:
app: controller
# nodeport-svc-rs
apiVersion: v1
kind: Service
metadata:
name: nodeport-svc-rs
spec:
type: NodePort
ports:
- port: 5002
targetPort: 5002
nodePort: 30002
selector:
app: controller-rs
Ingress Config:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: traefik
ingress.kubernetes.io/auth-type: "basic"
ingress.kubernetes.io/auth-secret: "mysecret"
spec:
rules:
- host: example.com
http:
paths:
- path: /demo
backend:
serviceName: nodeport-svc-rc
servicePort: 5001
- path: /demof
backend:
serviceName: nodeport-svc-rs
servicePort: 5002
Traefik is able to detect the ingress resource on its dashboard as backends services:
But no Frontends have been detected on dashboard and no IP address are detected on Backends.
Entry in /etc/hosts file: XXX.XXX.X.X example.com
I'm unable to route traffic using ingress. If i hit from browser example.com/demo, error shows Site can't be reached where i'm wrong? can someone help me.
# sudo kubectl describe ing
Name: ingress
Namespace: default
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
example.com
/demo nodeport-svc-rc:5001 (10.244.171.95:5001,10.244.171.96:5001,10.244.235.150:5001)
/demof nodeport-svc-rs:5002 (10.244.171.98:5002,10.244.235.157:5002,10.244.235.158:5002)
Annotations: ingress.kubernetes.io/auth-secret: mysecret
ingress.kubernetes.io/auth-type: basic
kubernetes.io/ingress.class: traefik
Events: <none>
And when i hit directly on nodePort service example.com:30001 or example.com:30002 it successfully give response.
Edited: Below Traefik controller config:
kind: Deployment
apiVersion: apps/v1
metadata:
name: traefik-ingress-controller
namespace: kube-system
labels:
k8s-app: traefik-ingress-lb
spec:
replicas: 1
selector:
matchLabels:
k8s-app: traefik-ingress-lb
template:
metadata:
labels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-lb
spec:
serviceAccountName: traefik-ingress-controller
terminationGracePeriodSeconds: 60
containers:
- image: traefik:v1.7.26-alpine
name: traefik-ingress-lb
args:
- --web
- --kubernetes
I am setting up kubernetes on a AWS environment using kubeadm. I have setup ingress-nginx to access the service on port 443. I have checked the service configurations which look good. I am receiving 502 bad gateway and also the Address field in ingress is empty.
Front end service
apiVersion: v1
kind: Service
metadata:
labels:
name: voyager-configurator-webapp
name: voyager-configurator-webapp
spec:
ports:
-
port: 443
targetPort: 443
selector:
component: app
name: voyager-configurator-webapp
type: ClusterIP
Ingress yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress-resource
spec:
tls:
- hosts:
- kubernetes-test.xyz.com
secretName: default-server-secret
rules:
- host: kubernetes-test.xyz.com
http:
paths:
- backend:
serviceName: voyager-configurator-webapp
servicePort: 443
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx-ingress-resource <none> kubernetes-test.xyz.com 80, 443 45m
What could be the issue here ? Any help will be appreciated.
Make sure that your service is created in proper namespace - if not add namespace field in service definition. It is not good approach to add label called name with the same name as your service, instead you can use different one to avoid mistake and configurations problem.
Read more about selectors and labels: labels-selectors.
Your frontend service should look like that:
piVersion: v1
kind: Service
name: voyager-configurator-webapp
metadata:
labels:
component: app
appservice: your-example-app
spec:
ports:
- protocol: TCP
port: 443
targetPort: 443
selector:
component: app
app: your-example-app
type: ClusterIP
Your ingress should look like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress-resource
annotations:
kubernetes.io/ingress.class: nginx
spec:
tls:
- hosts:
- kubernetes-test.xyz.com
secretName: default-server-secret
rules:
- host: kubernetes-test.xyz.com
http:
paths:
- path: /
backend:
serviceName: voyager-configurator-webapp
servicePort: 443
You have to define path to backend to with Ingress should send traffic.
Remember that is good to follow some examples and instructions during setup to avoid problems and waste of time during debugging.
Take a look: nginx-ingress-502-bad-gateway, aws-kubernetes-ingress-nginx.
I have a cluster on Digital Ocean. 1 master with 2 nodes.
I'm using the Nginx Controller with the Digital Ocean Load Balancer.
Three Items in my Ingress Service Work fine. The fourth where I use Nginx Doesn't.
Does anyone know why?
Here are the configs. I left out the Ingress Services for the 1st through 3rd Deployments that are working. I can add them if needed.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-kubernetes-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: hw1.example.com
http:
paths:
- backend:
serviceName: hello-kubernetes-first
servicePort: 80
- host: hw2.example.com
http:
paths:
- backend:
serviceName: hello-kubernetes-second
servicePort: 80
- host: hw3.example.com
http:
paths:
- backend:
serviceName: hello-kubernetes-third
servicePort: 80
- host: hw4.example.com
http:
paths:
- backend:
serviceName: hello-kubernetes-fourth
servicePort: 80
apiVersion: v1
kind: Service
metadata:
name: hello-kubernetes-fourth
spec:
type: ClusterIP
ports:
- port: 80
targetPort: 8080
selector:
app: hello-kubernetes-fourth
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-kubernetes-fourth
spec:
replicas: 3
selector:
matchLabels:
app: hello-kubernetes-fourth
template:
metadata:
labels:
app: hello-kubernetes-fourth
spec:
containers:
- name: nginx
image: nginx:1.8
ports:
- containerPort: 8080
First of all ,if you use the nginx image, the container Port should be the one exposed by the Dockerfile of the image.
For the nginx image the ContainerPort should be port 80
https://hub.docker.com/layers/nginx/library/nginx/stable/images/sha256-cab8e4374e1e32bac58a8c7ae96c252cadcb1049545ed4bb3e3bfd5d087983b9
Now you should test if the nginx is available by accessing the podip:containerPort from the minikube node:
kubectl get po -o wide
hello-kubernetes-fourth-cb4fb668c-7tkd4 1/1 Running 0 25m 172.17.0.12
curl 172.17.0.12
After this, you should modify the ports of the service : targetPort should match the containerPort (80) and port 8080
Now access the Nginx by service URL:
kubectl describe svc hello-kubernetes-fourth
curl ClusterIP:8080
If OK, modify also the servicePort of the ingress to match the Service Port. Don't forget to enable the ingress, as it is disabled by default on minikube:
minikube addons enable ingress
* ingress was successfully enabled
after ingress pod is up, and adding in your host machine the MINIKUBEIP hw4.example.com in your /etc/hosts, you should be able to curl from the host machine:
curl hw4.example.com
The deployment configuration is incorrect. update the YAMLs as shown below
apiVersion: v1
kind: Service
metadata:
name: hello-kubernetes-fourth
spec:
type: ClusterIP
ports:
- port: 80
targetPort: 80
selector:
app: hello-kubernetes-fourth
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-kubernetes-fourth
spec:
replicas: 3
selector:
matchLabels:
app: hello-kubernetes-fourth
template:
metadata:
labels:
app: hello-kubernetes-fourth
spec:
containers:
- name: nginx
image: nginx:1.8
ports:
- containerPort: 80
get inside nginx pod and verify that you are able to reach the service and getting the response
curl http://hello-kubernetes-fourth
Then you should be able to reach the service from ingress.
Say I have a service that isn't hosted on Kubernetes. I also have an ingress controller and cert-manager set up on my kubernetes cluster.
Because it's so much simpler and easy to use kubernetes ingress to control access to services, I wanted to have a kubernetes ingress that points to a non-kubernetes service.
For example, I have a service that's hosted at https://10.0.40.1:5678 (ssl required, but self signed certificate) and want to access at service.example.com.
You can do it by manual creation of Service and Endpoint objects for your external server.
Objects will looks like that:
apiVersion: v1
kind: Service
metadata:
name: external-ip
spec:
ports:
- name: app
port: 80
protocol: TCP
targetPort: 5678
clusterIP: None
type: ClusterIP
---
apiVersion: v1
kind: Endpoints
metadata:
name: external-ip
subsets:
- addresses:
- ip: 10.0.40.1
ports:
- name: app
port: 5678
protocol: TCP
Then, you can create an Ingress object which will point to Service external-ip with port 80:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: external-service
spec:
rules:
- host: service.example.com
http:
paths:
- backend:
serviceName: external-ip
servicePort: 80
path: /
So I got this working using ingress-nginx to proxy an managed external service over a non-standard port
apiVersion: v1
kind: Service
metadata:
name: external-service-expose
namespace: default
spec:
type: ExternalName
externalName: <external-service> # eg example.example.com
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: external-service-expose
namespace: default
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" #important
spec:
rules:
- host: <some-host-on-your-side> # eg external-service.yourdomain.com
http:
- path: /
pathType: Prefix
backend:
service:
name: external-service
port:
number: <port of external service> # eg 4589
tls:
- hosts:
- external-service.yourdomain.com
secretName: <tls secret for your domain>
of-course you need to make sure that the managed url is reachable from inside the cluster, a simple check can be done by launching a debug pod and doing
curl -v https://example.example.com:4589
If your external service has a dns entry configured on it, you can use kubernetes externalName service.
---
apiVersion: v1
kind: Service
metadata:
name: my-service
namespace: prod
spec:
type: ExternalName
externalName: myexternal.http.service.com
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: externalNameservice
namespace: prod
spec:
rules:
- host: service.example.com
http:
paths:
- backend:
serviceName: my-service
servicePort: 80
path: /
In this way, kubernetes create cname record my-service pointing to myexternal.http.service.com
I just want to update #Moulick answer here according to Kubernetes version v1.21.1, as for ingress the configuration has changed a little bit.
In my example I am using Let's Encrypt for my nginx controller:
apiVersion: v1
kind: Service
metadata:
name: external-service
namespace: default
spec:
type: ExternalName
externalName: <some-host-on-your-side> eg managed.yourdomain.com
ports:
- port: <port of external service> eg 4589
---
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
name: external-service
namespace: default
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
nginx.ingress.kubernetes.io/proxy-body-size: 100m
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" #important
spec:
tls:
- hosts:
- <some-host-on-your-side> eg managed.yourdomain.com
secretName: tls-external-service
rules:
- host: <some-host-on-your-side> eg managed.yourdomain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: external-service
port:
number: <port of external service> eg 4589