I have a service object of type LoadBalancer:
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default user-api LoadBalancer 10.106.120.69 <pending> 9411:31622/TCP,3000:31878/TCP,4000:30202/TCP,3001:31656/TCP,4001:31455/TCP 96m
I can reach to the service internally with 3000 port via internal_ip:31878.
Supposing I have a public_ip and ingress, how should I correctly configure it to let the internal service to reach to the internet?
NAME HOSTS ADDRESS PORTS AGE
app-ingress user-api.my-domain.com 80 95m
again, I'm guessing that this was a cluster setup by kubeadm because of your other questions and again the solution is to install an external load-balancer plugin such as metalLB. After that your loadbalancers will get IPs assigned automatically and all ports of your services/ingress will be open on this IP and will lead to your pods.
Installing metalLB is very easy:
kubectl apply -f https://raw.githubusercontent.com/google/metallb/v0.8.3/manifests/metallb.yaml
create and apply config of IP pools to be used for your load-balancers similarly to this one:
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 92.18.1.200-92.18.1.250
If you have a service object of type LoadBalancer, you should already be able to reach to the service from the Internet. If you have a Load Balancer that is attached to an Ingress Controller, you should change the type of your service object (to something like ClusterIP) and register the service in the Ingress Controller manifest. Example is:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /<context-path>
backend:
serviceName: <service-name>
servicePort: <service-port>
Related
I have the following topology in my kubernetes cluster:
So, I have 2 Nodes: 1 Master and 1 Worker Node.
Now I created an application with my deployment.yml and my service.yml, using nodePort configuration, see:
apiVersion: v1
kind: Service
metadata:
name: administrativo-service
spec:
type: NodePort
selector:
app: administrativo
ports:
- protocol: TCP
port: 80
targetPort: 8080
And this is my service:
Now I need to access this API using my DNS, something like: myapi.localdns, so I followed this steps to install Ingress Controller based on nginx:
https://kubernetes.github.io/ingress-nginx/deploy/#quick-start
https://kubernetes.github.io/ingress-nginx/deploy/#bare-metal-clusters
After 1 hour this is POD status in ingress-nginx namespace:
And finally, this is my Ingress yml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: administrativo-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: myapi.localdns
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: administrativo-service
port:
number: 80
Well, my idea is to create an entry in my company to DNS to point to this DNS myapi.localdns:
but to do it I need the Ingress Address, that don't show in my ingress resource, see:
I solved the problem, using this steps:
First create in my company DNS the CNAMEs pointing to my kubernetes workernode IP.
Reinstall ingress-nginx controller using bare-metal configuration: kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.5/deploy/static/provider/baremetal/deploy.yaml.
Change the deploy.yaml to use NodePort before use kubectl apply
Use externalIPS to expose my service in port 80.
I am trying to expose kubernetes-dashboard app externally using Ingress resource. I have installed Nginx Controller and a service called Kubernetes-dashboard is clusterIP type service with port 443.
I have created Ingress resource with YAML file and pointing to backend service which is kubernetes-dashboard but somehow I am not getting the IP address of my host (dashboard.com) so that I can add this entry in /etc/hosts file. what is the resolution here. I am not able to paste the yaml file here as this website complain about code formatting.
I tried to put YAML file here in various ways but it does not work.
yaml file of kubernetes-dashboard as below:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: dashboard-ingress namespace: kubernetes-dashboard spec: tls: - hosts: - dashboard.com secretName: kubernetes-dashboard-certs rules: - host: dashboard.com http: paths: - pathType: ImplementationSpecific path: / backend: service: name: kubernetes-dashboard port: number: 443
Kubernetes-dashboard service config as below: Name: kubernetes-dashboard Namespace: kubernetes-dashboard Labels: k8s-app=kubernetes-dashboard Annotations: Selector: k8s-app=kubernetes-dashboard Type: ClusterIP IP Family Policy: SingleStack IP Families: IPv4 IP: 10.106.1.186 IPs: 10.106.1.186 Port: 443/TCP TargetPort: 8443/TCP Endpoints: 10.44.0.3:8443 Session Affinity: None Events:
I am not getting the IP address of my host
You have to use the Nginx ingress controller service IP everywhere so traffic gets forwarded and managed via Nginx ingress.
you can check the IP of Nginx controller using the
Kubectl get svc -n ingress-nginx
Nginx controller service will be exposed as the type LoadBalancer you can use this IP into the DNS route as A or CNAME record.
Any request coming to your domain will get forwarded to
ingress > Nginx ingress controller > K8s service > K8s PODs
I have created a Nginx Ingress and Service with the following code:
apiVersion: v1
kind: Service
metadata:
name: myservice
spec:
type: ClusterIP
selector:
name: my-app
ports:
- port: 8000
targetPort: 8000
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myingress
annotations:
kubernetes.io/ingress.class: nginx
labels:
name: myingress
spec:
rules:
- host: mydomain.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: my-service
port:
number: 8000
Nginx ingress installed with:
helm install ingress-nginx ingress-nginx/ingress-nginx.
I have also enabled proxy protocols for ELB. But in nginx logs I don't see the real client ip for X-Forwarded-For and X-Real-IP headers. This is the final headers I see in my app logs:
X-Forwarded-For:[192.168.21.145] X-Forwarded-Port:[80] X-Forwarded-Proto:[http] X-Forwarded-Scheme:[http] X-Real-Ip:[192.168.21.145] X-Request-Id:[1bc14871ebc2bfbd9b2b6f31] X-Scheme:[http]
How do I get the real client ip instead of the ingress pod IP? Also is there a way to know which headers the ELB is sending to the ingress?
One solution is to use externalTrafficPolicy: Local (see documentation).
In fact, according to the kubernetes documentation:
Due to the implementation of this feature, the source IP seen in the target container is not the original source IP of the client.
...
service.spec.externalTrafficPolicy - denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. There are two available options: Cluster (default) and Local. Cluster obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading. Local preserves the client source IP and avoids a second hop for LoadBalancer and NodePort type services, but risks potentially imbalanced traffic spreading.
If you want to follow this route, update your nginx ingress controller Service and add the externalTrafficPolicy field:
apiVersion: v1
kind: Service
metadata:
name: nginx-ingress-controller
spec:
...
externalTrafficPolicy: Local
A possible alternative could be to use Proxy protocol (see documentation)
The proxy protocol should be enabled in the ConfigMap for the ingress-controller as well as the ELB.
L4 uses proxy-protocol
For L7 use use-forwarded-headers
# configmap.yaml
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
data:
use-forwarded-headers: "true"
use-proxy-protocol: "true"
https://kubernetes.github.io/ingress-nginx/user-guide/miscellaneous/#proxy-protocol
Just expanding on #strongjz answer.
By default, the Load balancer that will be created in AWS for a Service of type LoadBalancer will be a Classic Load Balancer, operating on Layer 4, i.e., proxying in the TCP protocol level.
For this scenario, the best way to preserve the real ip is to use the Proxy Protocol, because it is capable of doing this at the TCP level.
To do this, you should enable the Proxy Protocol both on the Load Balancer and on Nginx-ingress.
Those values should do it for a Helm installation of nginx-ingress:
controller:
service:
annotations:
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
config:
use-proxy-protocol: "true"
real-ip-header: "proxy_protocol"
The service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*" annotation will tell the aws-load-balancer-controller to create your LoadBalancer with Proxy Protocol enabled. I'm not sure what happens if you add it to a pre-existing Ingress-nginx, but it should work too.
The use-proxy-protocol and real-ip-header are options passed to Nginx, to also enable Proxy Protocol there.
Reference:
https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/service/annotations/#proxy-protocol-v2
https://kubernetes.github.io/ingress-nginx/user-guide/miscellaneous/#proxy-protocol
https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#use-proxy-protocol
Cluster information:
Installation Method: kubeadm
Kubernetes: 1.19.2
Master & Nodes: Ubuntu 20.04.1 (Oracle Virutalbox)
Docker: 19.03.12
Calico: 3.16.1
Ingress : Bare-metal - 0.40.1
I want to access the Kubernetes dashboard from my laptop using ingress without proxy?
Can anyone help me with the steps... ( I tried multiple ways with the help of the internet... not sure where I am missing?)
Note: As per discussion forums I have added "hostNetwork: true" under the deployment section in ingress YAML to resolve "not working without host parameter" and commented "type: NodePort".
Updated info:
I have created ingress-controller as daemon instead of deployments/pod - this helps in accessing directly with worker IPs. (this is what I am expecting - but unable to access kubernetes dashboard as it is in different namespace)
Ingress yaml: this is running in default namespace
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: kdash-in-ns
port:
number: 443
kdash-in-ns yaml - svc with External Name
kind: Service
apiVersion: v1
metadata:
name: kdash-in-ns
namespace: default
spec:
type: ExternalName
externalName: kubernetes-dashboard.kubernetes-dashboard.svc.cluster.local
ports:
- name: https
port: 443
Below details about kdash-in-ns svc with ExternalName
dockeras#ubuntu3:~/simplek8s/kubernetes/yamls/ingress-demo$ kubectl describe svc kdash-in-ns
Name: kdash-in-ns
Namespace: default
Labels: <none>
Annotations: <none>
Selector: <none>
Type: ExternalName
IP:
External Name: kubernetes-dashboard.kubernetes-dashboard.svc.cluster.local
Port: https 443/TCP
TargetPort: 443/TCP
Endpoints: <none>
Session Affinity: None
Events: <none>
kubectl describe for the updated ingress route: in this i have ngnix - which is working fine (i guess both ingress and nginx are in same namespace.. getting error for dashboard - as it in different namespace (kubernetes-dasbhoard))
dockeras#ubuntu3:~$ kubectl describe ing nginx-ingress
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
Name: nginx-ingress
Namespace: default
Address: 192.168.1.31,192.168.1.32
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
*
/nginx nginx-deploy-main:80 )
/foo kubernetes-dashboard:443 (<error: endpoints "kubernetes-dashboard" not found>)
/dashboard kdash-in-ns:443 (<error: endpoints "kdash-in-ns" not found>)
Annotations: kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 4m40s nginx-ingress-controller Ingress default/nginx-ingress
When I tried the same URLs in browser below are the responses (One of my worker iP - 192.168.1.31)
192.168.1.31/nginx - responds with nginx default page (pod - nginx-deploy-main)
192.168.1.31/foo - error page - 503 service temporarily Unavailable (default nginx)
192.168.1.31/dashboard - 504 Gateway Time-out (default nginx)
running svc, pods:
All Pods and svcs
If I understand correctly, you want to access kubernetes service (dashboard) from outside cluster. You may deploy metallb LoadBalancer and manage a pull of IPs from external cluster network, assigned to your cluster.
So, you can assign an IP and a LoadBalancer through which you will access your service. Below is an example for mssql server, but you can easily adapt it to your needs with dashboard:
apiVersion: v1
kind: Service
metadata:
name: sql-server-lb
namespace: database-server
annotations:
metallb.universe.tf/address-pool: default
spec:
selector:
app: sql-server
ports:
- port: 1433
targetPort: 1433
type: LoadBalancer
https://metallb.universe.tf/
FYI:
I run Kubernetes on docker desktop for mac
The website based on Nginx image
I run 2 simple website deployments on Kubetesetes and use the NodePort service. Then I want to make routing to the website using ingress. When I open the browser and access the website, I get an error 503 like images below. So, how do I fix this error?
### Service
apiVersion: v1
kind: Service
metadata:
name: app-svc
labels:
app: app1
spec:
type: NodePort
ports:
- port: 80
selector:
app: app1
---
apiVersion: v1
kind: Service
metadata:
name: app2-svc
labels:
app: app2
spec:
type: NodePort
ports:
- port: 80
selector:
app: app2
### Ingress-Rules
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: app-ingress
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /app1
backend:
serviceName: app-svc
servicePort: 30092
- path: /app2
backend:
serviceName: app2-svc
servicePort: 30936
Yes, i end up with same error. once i changed the service type to "ClusterIP", it worked fine for me.
Found this page after searching for a solution to nginx continually returning 503 responses despite the service names all being configured correctly. The issue for me was that I had configured the kubernetes service in a specific namespace, but did not update the ingress component to be in the same namespace. Despite being such a simple solution it was not at all obvious!
I advise you to use service type ClusterIP
Take look on this useful article: services-kubernetes.
If you use Ingress you have to know that Ingress isn’t a type of Service, but rather an object that acts as a reverse proxy and single entry-point to your cluster that routes the request to different services. The most basic Ingress is the NGINX Ingress Controller, where the NGINX takes on the role of reverse proxy, while also functioning as SSL. On below drawing you can see workflow between specific components of environment objects.
Ingress is exposed to the outside of the cluster via ClusterIP and Kubernetes proxy, NodePort, or LoadBalancer, and routes incoming traffic according to the configured rules.
Example of service definition:
---
apiVersion: v1
kind: Service
metadata:
name: app-svc
labels:
app: app1
spec:
type: ClusterIP
ports:
- port: 80
selector:
app: app1
---
apiVersion: v1
kind: Service
metadata:
name: app2-svc
labels:
app: app2
spec:
type: ClusterIP
ports:
- port: 80
selector:
app: app2
Let me know if it helps.
First, You need to change the service type of your app-service to ClusterIP, because the Ingress object is going to access these Pods(services) from inside the cluster. (ClusterIP service is used when you want to allow accessing a pod inside a cluster).
Second, Make sure the services are running by running kubectl get services and check the running services names against the names in backend section in Ingress routing rules
Little late to this journey but here is my comment on the issue.
I was having the same issue and having the same environment. (Docker Desktop-based Kubernetes with WSL2)
a couple of items probably can help.
add the host entry in the rules section. and the value will be kubernetes.docker.internal like below
rules:
- host: kubernetes.docker.internal
http:
paths:
- path....
check the endpoints using kubectl get services to confirm that the same port is in your ingress rule definition for each of those backend services.
backend:
service:
name: my-apple-service
port:
number: 30101
kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
my-apple-service ClusterIP 10.106.121.95 <none> 30101/TCP 9h
my-banada-service ClusterIP 10.99.192.112 <none> 30101/TCP 9h