Use LoadBalancer for bare metal single node kubernetes clusters - kubernetes

Environment: Ubuntu 18.06 bare metal, set up the cluster with kubeadm (single node)
I want to access the cluster via port 80. Currently I am able to access it via the nodePort: domain.com:31668/ but not via port 80. I am using metallb Do I need something else to handle incoming traffic?
So the current topology would be:
LoadBalancer > Ingress Controller > Ingress > Service
kubectl -n ingress-nginx describe service/ingress-nginx:
Name: ingress-nginx
Namespace: ingress-nginx
Labels: app.kubernetes.io/name=ingress-nginx
app.kubernetes.io/part-of=ingress-nginx
Annotations: <none>
Selector: app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx
Type: LoadBalancer
IP: 10.99.6.137
LoadBalancer Ingress: 192.168.1.240
Port: http 80/TCP
TargetPort: 80/TCP
NodePort: http 31668/TCP
Endpoints: 192.168.0.8:80
Port: https 443/TCP
TargetPort: 443/TCP
NodePort: https 30632/TCP
Endpoints: 192.168.0.8:443
Session Affinity: None
External Traffic Policy: Cluster
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal IPAllocated 35m metallb-controller Assigned IP "192.168.1.240"
As I am using a bare metal environment I am using metallb.
metallb config:
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 192.168.1.240-192.168.1.250
Ingress controller yml's:
apiVersion: v1 kind: Namespace metadata: name: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap apiVersion: v1 metadata: name: nginx-configuration namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
--- kind: ConfigMap apiVersion: v1 metadata: name: tcp-services namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
--- kind: ConfigMap apiVersion: v1 metadata: name: udp-services namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
--- apiVersion: v1 kind: ServiceAccount metadata: name: nginx-ingress-serviceaccount namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
--- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: nginx-ingress-clusterrole labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
resources:
- ingresses/status
verbs:
- update
--- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: Role metadata: name: nginx-ingress-role namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
# Defaults to "<election-id>-<ingress-class>"
# Here: "<ingress-controller-leader>-<nginx>"
# This has to be adapted if you change either parameter
# when launching the nginx-ingress-controller.
- "ingress-controller-leader-nginx"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
--- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: nginx-ingress-role-nisa-binding namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: nginx-ingress-role subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
--- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: nginx-ingress-clusterrole-nisa-binding labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: nginx-ingress-clusterrole subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: extensions/v1beta1 kind: Deployment metadata: name: nginx-ingress-controller namespace: ingress-nginx labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx spec: replicas: 1 selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
spec:
serviceAccountName: nginx-ingress-serviceaccount
containers:
- name: nginx-ingress-controller
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.21.0
args:
- /nginx-ingress-controller
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
securityContext:
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
# www-data -> 33
runAsUser: 33
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: http
containerPort: 80
- name: https
containerPort: 443
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
---
output of curl -v http://192.168.1.240 (executing inside the server)
* Rebuilt URL to: http://192.168.1.240/
* Trying 192.168.1.240...
* TCP_NODELAY set
* Connected to 192.168.1.240 (192.168.1.240) port 80 (#0)
> GET / HTTP/1.1
> Host: 192.168.1.240
> User-Agent: curl/7.61.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Server: nginx/1.15.6
< Date: Thu, 27 Dec 2018 19:03:28 GMT
< Content-Type: text/html
< Content-Length: 153
< Connection: keep-alive
<
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.15.6</center>
</body>
</html>
* Connection #0 to host 192.168.1.240 left intact
kubectl describe ingress articleservice-ingress
Name: articleservice-ingress
Namespace: default
Address: 192.168.1.240
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
host.com
/articleservice articleservice:31001 (<none>)
Annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
Events: <none>
curl -vH 'host: elpsit.com' http://192.168.1.240/articleservice/system/ipaddr
I can reach the ingress as expected from inside the server.

Related

Kubernetes Ingress failing to get address

I am trying to set up an ingress for services using nginx ingress on AWS EKS.
I have installed nginx ingress with the code provided on Kubernetes' github page.
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- endpoints
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resourceNames:
- ingress-controller-leader
resources:
- configmaps
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
- namespaces
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission
rules:
- apiGroups:
- admissionregistration.k8s.io
resources:
- validatingwebhookconfigurations
verbs:
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: v1
data:
allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-controller
namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
service.beta.kubernetes.io/aws-load-balancer-type: nlb
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
externalTrafficPolicy: Local
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: http
- appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
ports:
- appProtocol: https
name: https-webhook
port: 443
targetPort: webhook
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
spec:
containers:
- args:
- /nginx-ingress-controller
- --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
- --election-id=ingress-controller-leader
- --controller-class=k8s.io/ingress-nginx
- --ingress-class=nginx
- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: LD_PRELOAD
value: /usr/local/lib/libmimalloc.so
image: k8s.gcr.io/ingress-nginx/controller:v1.2.0#sha256:d8196e3bc1e72547c5dec66d6556c0ff92a23f6d0919b206be170bc90d5f9185
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
livenessProbe:
failureThreshold: 5
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: controller
ports:
- containerPort: 80
name: http
protocol: TCP
- containerPort: 443
name: https
protocol: TCP
- containerPort: 8443
name: webhook
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
requests:
cpu: 100m
memory: 90Mi
securityContext:
allowPrivilegeEscalation: true
capabilities:
add:
- NET_BIND_SERVICE
drop:
- ALL
runAsUser: 101
volumeMounts:
- mountPath: /usr/local/certificates/
name: webhook-cert
readOnly: true
dnsPolicy: ClusterFirst
nodeSelector:
kubernetes.io/os: linux
serviceAccountName: ingress-nginx
terminationGracePeriodSeconds: 300
volumes:
- name: webhook-cert
secret:
secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission-create
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission-create
spec:
containers:
- args:
- create
- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
- --namespace=$(POD_NAMESPACE)
- --secret-name=ingress-nginx-admission
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1#sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
imagePullPolicy: IfNotPresent
name: create
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission-patch
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission-patch
spec:
containers:
- args:
- patch
- --webhook-name=ingress-nginx-admission
- --namespace=$(POD_NAMESPACE)
- --patch-mutating=false
- --secret-name=ingress-nginx-admission
- --patch-failure-policy=Fail
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1#sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
imagePullPolicy: IfNotPresent
name: patch
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: nginx
spec:
controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.2.0
name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
- v1
clientConfig:
service:
name: ingress-nginx-controller-admission
namespace: ingress-nginx
path: /networking/v1/ingresses
failurePolicy: Fail
matchPolicy: Equivalent
name: validate.nginx.ingress.kubernetes.io
rules:
- apiGroups:
- networking.k8s.io
apiVersions:
- v1
operations:
- CREATE
- UPDATE
resources:
- ingresses
sideEffects: None
It seems that I was able to create the ingress controller successfully.
Image of ingress controller success.
I have created the deployment and service for the application that I want to deploy on the cluster via yaml like this
apiVersion: v1
kind: Namespace
metadata:
name: evcloud
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: evcloud-vertx
namespace: evcloud
spec:
replicas: 3
selector:
matchLabels:
app: evcloud-tcp
template:
metadata:
labels:
app: evcloud-tcp
spec:
containers:
- name: evcloud-tcp
image: location of ecr container
ports:
- containerPort: 2238
- containerPort: 2237
---
apiVersion: v1
kind: Service
metadata:
name: evcloud-tcp
namespace: evcloud
spec:
selector:
app: evcloud-tcp
ports:
- port: 2238
name: remote
protocol: TCP
targetPort: 2238
- port: 2237
name: central
protocol: TCP
targetPort: 2237
It seems that the pods and services have been deployed successfully.
Image of pod running successfully
I have tried to connect the service of these pods using an ingress via the yaml file below.
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: evcloud
namespace: evcloud
annotations:
kubernetes.io/ingress.class: ingress-nginx
spec:
rules:
- host:
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: evcloud-tcp
port:
number: 2237
- host:
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: evcloud-tcp
port:
number: 2238
I was able to successfully make the ingress but the address seems to be empty.
error log
I am really new to Kubernetes, so any sort of feedback would be deeply appreciated!
Thank you in advance!!
The problem was cause because they were in different namespaces.
Solved it by using a headless service with externalName.

Ingress: Connection refused, however, it works from cluster

I have setup Ingress controller:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: {{ template "mychart.fullname" . }}-app
annotations:
# type of authentication [basic|digest]
nginx.ingress.kubernetes.io/auth-type: basic
# name of the secret that contains the user/password definitions
nginx.ingress.kubernetes.io/auth-secret: {{ template "mychart.fullname" . }}-myauthsecret
# message to display with an appropriate context why the authentication is required
nginx.ingress.kubernetes.io/auth-realm: "Authentication Required - foo"
spec:
rules:
- host: "test.example.com"
http:
paths:
- path: /
backend:
serviceName: {{ template "mychart.fullname" . }}-app
servicePort: 80
But, when I test it, I get connection refused:
curl -H 'Host: test.example.com' http://{public ip}/
When I test it on machine, where cluster run, it works properly:
curl -H 'Host: test.example.com' https://10.96.183.247/
10.96.183.247 is local cluster IP
Thank you for comments, I havent noticed, I had no Nginx ingress controller installed on new baremetal.
Here is missing part, ingress with hostport:
apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
---
# tcp-services-configmap
kind: ConfigMap
apiVersion: v1
metadata:
name: tcp-services
namespace: ingress-nginx
---
# udp-services-configmap
kind: ConfigMap
apiVersion: v1
metadata:
name: udp-services
namespace: ingress-nginx
# rbac start
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: nginx-ingress-clusterrole
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
resources:
- ingresses/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: nginx-ingress-role
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
# Defaults to "<election-id>-<ingress-class>"
# Here: "<ingress-controller-leader>-<nginx>"
# This has to be adapted if you change either parameter
# when launching the nginx-ingress-controller.
- "ingress-controller-leader-nginx"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: nginx-ingress-role-nisa-binding
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress-role
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nginx-ingress-clusterrole-nisa-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress-clusterrole
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
# rbac end
# with-rbac start
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
spec:
replicas: 1
selector:
matchLabels:
app: ingress-nginx
template:
metadata:
labels:
app: ingress-nginx
annotations:
prometheus.io/port: '10254'
prometheus.io/scrape: 'true'
spec:
serviceAccountName: nginx-ingress-serviceaccount
containers:
- name: nginx-ingress-controller
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.14.0
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --annotations-prefix=nginx.ingress.kubernetes.io
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: http
containerPort: 80
hostPort: 80 # !!!!!!
- name: https
containerPort: 443
hostPort: 443 # !!!!!!
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
securityContext:
runAsNonRoot: false
---
# with-rbac end
# default-backend start
apiVersion: apps/v1
kind: Deployment
metadata:
name: default-http-backend
namespace: ingress-nginx
spec:
selector:
matchLabels:
app: default-http-backend
template:
metadata:
labels:
app: default-http-backend
spec:
terminationGracePeriodSeconds: 60
containers:
- name: default-http-backend
# Any image is permissible as long as:
# 1. It serves a 404 page at /
# 2. It serves 200 on a /healthz endpoint
image: gcr.io/google_containers/defaultbackend:1.4
livenessProbe:
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 30
timeoutSeconds: 5
ports:
- containerPort: 8080
resources:
limits:
cpu: 10m
memory: 20Mi
requests:
cpu: 10m
memory: 20Mi
---
apiVersion: v1
kind: Service
metadata:
name: default-http-backend
namespace: ingress-nginx
spec:
selector:
app: default-http-backend
ports:
- port: 80
targetPort: 8080
---
# default-backend end

I am trying to use externalname together with nodeport service for Nginx-Controller but I am getting 502 Bad Gateway

Environment:
I have:
1- NGINX Ingress controller version: 1.15.9, image: 0.23.0
2- Kubernetes version:
Client Version: version.Info{Major:"1", Minor:"13",
GitVersion:"v1.13.4",
GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1",
GitTreeState:"clean", BuildDate:"2019-02-28T13:37:52Z",
GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13",
GitVersion:"v1.13.4",
GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1",
GitTreeState:"clean", BuildDate:"2019-02-28T13:30:26Z",
GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Cloud provider or hardware configuration: Virtual Machines on KVM
OS (e.g. from /etc/os-release):
NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel
fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7"
Kernel (e.g. uname -a):
Linux node01 3.10.0-957.5.1.el7.x86_64 #1 SMP Fri Feb 1 14:54:57 UTC
2019 x86_64 x86_64 x86_64 GNU/Linux
Install tools: kubeadm
More details:
CNI : WEAVE
Setup:
2 Resilient HA Proxy, 3 Masters, 2 infra, and worker nodes.
I am exposing all the services as node ports, where the HA-Proxy re-assign them to a public virtual IP.
Dedicated project hosted on the infra node carrying the monitoring and logging tools (Grafana, Prometheus, EFK, etc)
Backend NFS storage as persistent storage
What happened:
I want to be able to use external Name rather than node ports, so instead of accessing grafana for instance via vip + 3000 I want to access it via http://grafana.wild-card-dns-zone
Deployment
I have created a new namespace called ingress
I deployed it as follow:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-ingress-controller
namespace: ingress
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
replicas: **2**
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
name: nginx-ingress
spec:
serviceAccountName: nginx-ingress-serviceaccount
nodeSelector:
node-role.kubernetes.io/infra: infra
terminationGracePeriodSeconds: 60
containers:
- name: nginx-ingress-controller
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.23.0
readinessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
livenessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
args:
- /nginx-ingress-controller
- --default-backend-service=ingress/ingress-controller-nginx-ingress-default-backend
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
- --v3
securityContext:
allowPrivilegeEscalation: true
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
# www-data -> 33
runAsUser: 33
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: http
containerPort: 80
- name: https
containerPort: 443
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
generation: 1
labels:
app: nginx-ingress
chart: nginx-ingress-1.3.1
component: default-backend
name: ingress-controller-nginx-ingress-default-backend
namespace: ingress
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: nginx-ingress
component: default-backend
release: ingress-controller
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: nginx-ingress
component: default-backend
release: ingress-controller
spec:
nodeSelector:
node-role.kubernetes.io/infra: infra
containers:
- image: k8s.gcr.io/defaultbackend:1.4
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
name: nginx-ingress-default-backend
ports:
- containerPort: 8080
name: http
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 60
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-configuration
namespace: ingress
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: tcp-services
namespace: ingress
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: udp-services
namespace: ingress
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx
namespace: ingress
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
- port: 443
targetPort: 443
protocol: TCP
name: https
selector:
name: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
labels:
app: nginx-ingress
chart: nginx-ingress-1.3.1
component: default-backend
name: ingress-controller-nginx-ingress-default-backend
namespace: ingress
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
selector:
app: nginx-ingress
component: default-backend
release: ingress-controller
sessionAffinity: None
type: ClusterIP
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nginx-ingress-serviceaccount
namespace: ingress
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: nginx-ingress-clusterrole
namespace: ingress
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
resources:
- ingresses/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: nginx-ingress-role
namespace: ingress
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
# Defaults to "<election-id>-<ingress-class>"
# Here: "<ingress-controller-leader>-<nginx>"
# This has to be adapted if you change either parameter
# when launching the nginx-ingress-controller.
- "ingress-controller-leader-nginx"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nginx-ingress-clusterrolebinding
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress-clusterrole
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: nginx-ingress-rolebinding
namespace: ingress
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress-role
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress
INGRESS SETUP:
services
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2019-03-25T16:03:01Z"
labels:
app: jaeger
app.kubernetes.io/component: query
app.kubernetes.io/instance: jeager
app.kubernetes.io/managed-by: jaeger-operator
app.kubernetes.io/name: jeager-query
app.kubernetes.io/part-of: jaeger
name: jeager-query
namespace: monitoring-logging
resourceVersion: "3055947"
selfLink: /api/v1/namespaces/monitoring-logging/services/jeager-query
uid: 778550f0-4f17-11e9-9078-001a4a16021e
spec:
externalName: jaeger.example.com
ports:
- port: 16686
protocol: TCP
targetPort: 16686
selector:
app: jaeger
app.kubernetes.io/component: query
app.kubernetes.io/instance: jeager
app.kubernetes.io/managed-by: jaeger-operator
app.kubernetes.io/name: jeager-query
app.kubernetes.io/part-of: jaeger
sessionAffinity: None
type: ExternalName
status:
loadBalancer: {}
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2019-03-25T15:40:30Z"
labels:
app: grafana
chart: grafana-2.2.4
heritage: Tiller
release: grafana
name: grafana
namespace: monitoring-logging
resourceVersion: "3053698"
selfLink: /api/v1/namespaces/monitoring-logging/services/grafana
uid: 51b9d878-4f14-11e9-9078-001a4a16021e
spec:
externalName: grafana.example.com
ports:
- name: http
port: 3000
protocol: TCP
targetPort: 3000
selector:
app: grafana
release: grafana
sessionAffinity: None
type: ExternalName
status:
loadBalancer: {}
INGRESS
Ingress 1
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
ingress.kubernetes.io/service-upstream: "true"
creationTimestamp: "2019-03-25T21:13:56Z"
generation: 1
labels:
app: jaeger
app.kubernetes.io/component: query-ingress
app.kubernetes.io/instance: jeager
app.kubernetes.io/managed-by: jaeger-operator
app.kubernetes.io/name: jeager-query
app.kubernetes.io/part-of: jaeger
name: jaeger-query
namespace: monitoring-logging
resourceVersion: "3111683"
selfLink: /apis/extensions/v1beta1/namespaces/monitoring-logging/ingresses/jaeger-query
uid: e6347f6b-4f42-11e9-9e8e-001a4a16021c
spec:
rules:
- host: jaeger.example.com
http:
paths:
- backend:
serviceName: jeager-query
servicePort: 16686
status:
loadBalancer: {}
Ingress 2
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{},"labels":{"app":"grafana"},"name":"grafana","namespace":"monitoring-logging"},"spec":{"rules":[{"host":"grafana.example.com","http":{"paths":[{"backend":{"serviceName":"grafana","servicePort":3000}}]}}]}}
creationTimestamp: "2019-03-25T17:52:40Z"
generation: 1
labels:
app: grafana
name: grafana
namespace: monitoring-logging
resourceVersion: "3071719"
selfLink: /apis/extensions/v1beta1/namespaces/monitoring-logging/ingresses/grafana
uid: c89d7f34-4f26-11e9-8c10-001a4a16021d
spec:
rules:
- host: grafana.example.com
http:
paths:
- backend:
serviceName: grafana
servicePort: 3000
status:
loadBalancer: {}
EndPoints
Endpoint 1
apiVersion: v1
kind: Endpoints
metadata:
creationTimestamp: "2019-03-25T15:40:30Z"
labels:
app: grafana
chart: grafana-2.2.4
heritage: Tiller
release: grafana
name: grafana
namespace: monitoring-logging
resourceVersion: "3050562"
selfLink: /api/v1/namespaces/monitoring-logging/endpoints/grafana
uid: 51bb1f9c-4f14-11e9-9e8e-001a4a16021c
subsets:
- addresses:
- ip: 10.42.0.15
nodeName: kuinfra01.example.com
targetRef:
kind: Pod
name: grafana-b44b4f867-bcq2x
namespace: monitoring-logging
resourceVersion: "1386975"
uid: 433e3d21-4827-11e9-9e8e-001a4a16021c
ports:
- name: http
port: 3000
protocol: TCP
Endpoint 2
apiVersion: v1
kind: Endpoints
metadata:
creationTimestamp: "2019-03-25T16:03:01Z"
labels:
app: jaeger
app.kubernetes.io/component: service-query
app.kubernetes.io/instance: jeager
app.kubernetes.io/managed-by: jaeger-operator
app.kubernetes.io/name: jeager-query
app.kubernetes.io/part-of: jaeger
name: jeager-query
namespace: monitoring-logging
resourceVersion: "3114702"
selfLink: /api/v1/namespaces/monitoring-logging/endpoints/jeager-query
uid: 7786d833-4f17-11e9-9e8e-001a4a16021c
subsets:
- addresses:
- ip: 10.35.0.3
nodeName: kunode02.example.com
targetRef:
kind: Pod
name: jeager-query-7d9775d8f7-2hwdn
namespace: monitoring-logging
resourceVersion: "3114693"
uid: fdac9771-4f49-11e9-9e8e-001a4a16021c
ports:
- name: query
port: 16686
protocol: TCP
I am able to curl the endpoints from inside the ingress-controller pod:
# kubectl exec -it nginx-ingress-controller-5dd67f88cc-z2g8s -n ingress -- /bin/bash
www-data#nginx-ingress-controller-5dd67f88cc-z2g8s:/etc/nginx$ curl -k https://localhost
Found.
www-data#nginx-ingress-controller-5dd67f88cc-z2g8s:/etc/nginx$ curl http://localhost
<html>
<head><title>308 Permanent Redirect</title></head>
<body>
<center><h1>308 Permanent Redirect</h1></center>
<hr><center>nginx/1.15.9</center>
</body>
</html>
www-data#nginx-ingress-controller-5dd67f88cc-z2g8s:/etc/nginx$ exit
But from out side when I am trying to reach jaeger.example.com or grafana.example.com I am getting 502 bad gatway and the following error log:
10.39.0.0 - [10.39.0.0] - - [25/Mar/2019:16:40:32 +0000] "GET /search HTTP/1.1" 502 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36" 514 0.001 [monitoring-logging-jeager-query-16686] vip:16686, vip:16686, vip:16686 0, 0, 0 0.001, 0.000, 0.000 502, 502, 502 b7c813286fccf27fffa03eb6564edfd1
2019/03/25 16:40:32 [error] 2816#2816: *4617326 connect() failed (111: Connection refused) while connecting to upstream, client: 10.39.0.0, server: _, request: "GET /favicon.ico HTTP/1.1", upstream: "http://vip:16686/favicon.ico", host: "jeager.example.com", referrer: "http://jeager.example.com/search"
2019/03/25 16:40:32 [error] 2816#2816: *4617326 connect() failed (111: Connection refused) while connecting to upstream, client: 10.39.0.0, server: _, request: "GET /favicon.ico HTTP/1.1", upstream: "http://vip:16686/favicon.ico", host: "jeager.example.com", referrer: "http://jeager.example.com/search"
2019/03/25 16:40:32 [error] 2816#2816: *4617326 connect() failed (111: Connection refused) while connecting to upstream, client: 10.39.0.0, server: _, request: "GET /favicon.ico HTTP/1.1", upstream: "http://vip:16686/favicon.ico", host: "jeager.example.com", referrer: "http://jeager.example.com/search"
10.39.0.0 - [10.39.0.0] - - [25/Mar/2019:16:40:32 +0000] "GET /favicon.ico HTTP/1.1" 502 559 "http://jeager.example.com/search" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36" 494 0.001 [monitoring-logging-jeager-query-16686] vip:16686, vip:16686, vip:16686 0, 0, 0 0.000, 0.001, 0.000 502, 502, 502 9e582912614e67dfee6be1f679de5933
I0325 16:40:32.497868 8 socket.go:225] skiping metric for host jeager.example.com that is not being served
I0325 16:40:32.497886 8 socket.go:225] skiping metric for host jeager.example.com that is not being served
First thanks for cookiedough for the clue to help regarding the service issue, but later I faced an issue to create service using external name but I found my mistake thanks for "Long" user in the slack, the mistake is that I was using service of type ExternalName and it should be type cluster IP here are the steps to solve the problems (Remark https issue is a separate problem):
1- Create wild character DNS zone pointing the public IP
1- For new service just create it of type ClusterIP
2- In the namespace for the service create an ingress using the following example (yaml):
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
labels:
app: grafana
name: grafana
namespace: grafana-namespace
spec:
rules:
host: grafana.example.com
http:
paths:
backend:
serviceName: grafana
servicePort: 3000
3- kubectl -f apply -f grafana-ingress.yaml
Now you can reach your grafana on http://grafana.example,com

Traefik-ingress dashboard return 404

I deploy traefik ingress controller pod and then two services, one of them a LoadBalancer type for reverse-proxy and the other a ClusterIP for dashboard.
Also I create ingress for redirect all <elb-address>/dashboard to my traefik dashboard.
but for some reason I get 404 error code when I trying to request my dashboard at aws-ip/dashboard
That is the manifest yamls that I use to set up traefik
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: traefik-ingress-controller
namespace: kube-system
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: traefik-ingress-controller
namespace: kube-system
labels:
k8s-app: traefik-ingress-lb
spec:
replicas: 1
selector:
matchLabels:
k8s-app: traefik-ingress-lb
template:
metadata:
labels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-lb
spec:
serviceAccountName: traefik-ingress-controller
terminationGracePeriodSeconds: 60
containers:
- image: traefik
name: traefik-ingress-lb
ports:
- name: http
containerPort: 80
- name: admin
containerPort: 8080
args:
- --api
- --kubernetes
- --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
name: traefik-ingress-service
namespace: kube-system
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- protocol: TCP
targetPort: 80
port: 80
type: LoadBalancer
---
kind: Service
apiVersion: v1
metadata:
name: traefik-web-ui
namespace: kube-system
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- name: web
port: 80
targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: kube-system
name: traefik-ingress
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- http:
paths:
- path: /dashboard
backend:
serviceName: traefik-web-ui
servicePort: web
Update
I am watching the log and get a the follow errors with rbac activated and the ClusterRole, ServiceRole and ServiceAccount created:
E1124 18:56:23.267560 1 reflector.go:205] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Endpoints: endpoints is forbidden: User "system:serviceaccount:kube-system:traefik-ingress" cannot list endpoints in the namespace "default"
E1124 18:56:23.648207 1 reflector.go:205] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:kube-system:traefik-ingress" cannot list services in the namespace "default"
E1124 18:56:23.267560 1 reflector.go:205] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Endpoints: endpoints is forbidden: User "system:serviceaccount:kube-system:traefik-ingress" cannot list endpoints in the namespace "default"
This are my serviceAccount, clusterRole and RoleBingind
kind: ServiceAccount
apiVersion: v1
metadata:
name: traefik-ingress
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: traefik-ingress
rules:
- apiGroups:
- ""
resources:
- pods
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses/status
verbs:
- update
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: traefik-ingress
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress
subjects:
- kind: ServiceAccount
name: traefik-ingress
namespace: default
Solution
I apply this
kubectl create serviceaccount --namespace kube-system tiller
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'
and then installed the stable/traefik template with helm
helm install stable/traefik --name=traefik-ingress-controller --values values.yaml
values.yaml file is:
dashboard:
enabled: true
domain: traefik-ui.k8s.io
rbac:
enabled: true
kubernetes:
namespaces:
- default
- kube-system
Thanks for help
I tried this myself. So basically when you create your Ingress it gets created with a host of traefik-ui.minikube (default), so you won't be able to access the dashboard with <elb-address>/dashboard/.
You will have to access it with traefik-ui.minikube/dashboard/. As an example:
$ kubectl -n kube-system get ingress
NAME HOSTS ADDRESS PORTS AGE
traefik-ingress * 80 8m13s
traefik-web-ui traefik-ui.minikube xxxx.elb.amazonaws.com 80 71d
$ curl -H 'Host: traefik-ui.minikube' xxxx.elb.amazonaws.com/dashboard/
<!doctype html><html class="has-navbar-fixed-top">
...
</html>
You can also add an entry to your /etc/hosts file if you'd like to see it on your browser.
<one-of-the-ips-of-your-elb> traefik-ui.minikube
And you can also use the host to the rules in your Ingress definition:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: kube-system
name: traefik-ingress
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: yourown.hostname.com
http:
paths:
- path: /dashboard
backend:
serviceName: traefik-web-ui
servicePort: web
Just because I ran into this, the docs say:
The trailing slash / in /dashboard/ is mandatory

K8s Ingress, initiate ingress controller nginx error?

I have two spring boot container, I want to setup ingress service. As document here says, ingress has two parts, one is controller, the other is resources.
My two resources are two containers: gearbox-rack-eureka-server and gearbox-rack-config-server. The difference is port so that ingress could route traffic by different ports. My yaml files are listed below:
eureka_pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: gearbox-rack-eureka-server
labels:
app: gearbox-rack-eureka-server
purpose: platform_eureka_demo
spec:
containers:
- name: gearbox-rack-eureka-server
image: 192.168.1.229:5000/gearboxrack/gearbox-rack-eureka-server
ports:
- containerPort: 8761
eureka_svc.yaml
apiVersion: v1
kind: Service
metadata:
name: gearbox-rack-eureka-server
labels:
name: gearbox_rack_eureka_server
spec:
selector:
app: gearbox-rack-eureka-server
type: NodePort
ports:
- port: 8761
nodePort: 31501
name: tcp
config_pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: gearbox-rack-config-server
labels:
app: gearbox-rack-config-server
purpose: platform-demo
spec:
containers:
- name: gearbox-rack-config-server
image: 192.168.1.229:5000/gearboxrack/gearbox-rack-config-server
ports:
- containerPort: 8888
env:
- name: EUREKA_SERVER
value: http://172.16.100.83:8761
config_svc.yaml
apiVersion: v1
kind: Service
metadata:
name: gearbox-rack-config-server
labels:
name: gearbox-rack-config-server
spec:
selector:
app: gearbox-rack-config-server
type: NodePort
ports:
- port: 8888
nodePort: 31502
name: tcp
My ingress-nginx controller is mostly copied from the link above,
ingress_nginx_ctl.yaml:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
spec:
type: LoadBalancer
selector:
app: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: ingress-nginx
spec:
replicas: 1
template:
metadata:
labels:
app: ingress-nginx
spec:
terminationGracePeriodSeconds: 60
containers:
- image: nginx:1.13.12
name: ingress-nginx
imagePullPolicy: Always
ports:
- name: http
containerPort: 80
protocol: TCP
- name: https
containerPort: 443
protocol: TCP
livenessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 30
timeoutSeconds: 5
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/nginx-default-backend
I did following commands, they are successful.
kubectl apply -f eureka_pod.yaml
kubectl apply -f eureka_svc.yaml
kubectl apply -f config_pod.yaml
kubectl apply -f config_svc.yaml
Then I got error from execute kubectl apply -f ingress_nginx_ctl.yaml, the pod does not start, logs are listed below:
[root#master3 nginx-ingress-controller]# kubectl get pods
NAME READY STATUS RESTARTS AGE
gearbox-rack-config-server 1/1 Running 0 39m
gearbox-rack-eureka-server 1/1 Running 0 40m
ingress-nginx-686c9975d5-7d464 0/1 CrashLoopBackOff 6 7m
[root#master3 nginx-ingress-controller]# kubectl logs -f ingress-nginx-686c9975d5-7d464
container_linux.go:247: starting container process caused "exec: \"/nginx-ingress-controller\": stat /nginx-ingress-controller: no such file or directory"
I created a directory /nginx-ingress-controller under root, and repeat the steps again, it still said same error. Does someone could point me the problem?
I put my ingress_nginx_res.yaml as follows for reference, it may have errors also.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-ingress
spec:
rules:
- host: 172.16.100.83
http:
paths:
- backend:
serviceName: gearbox-rack-eureka-server
servicePort: 8761
- host: 172.16.100.83
http:
paths:
- path:
backend:
serviceName: gearbox-rack-config-server
servicePort: 8888
==========================================
second edition
After change image link, The previous errors disappear, but still it has following permission problem:
[root#master3 ingress]# kubectl get pods
NAME READY STATUS RESTARTS AGE
gearbox-rack-config-server 1/1 Running 0 15m
gearbox-rack-eureka-server 1/1 Running 0 15m
ingress-nginx-8679f9c8ff-5sxw7 0/1 CrashLoopBackOff 5 12m
The log message is as follows:
[root#master3 kube]# kubectl logs ingress-nginx-8679f9c8ff-5sxw7
W0530 07:54:22.290114 5 client_config.go:533] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0530 07:54:22.290374 5 main.go:158] Creating API client for https://10.96.0.1:443
-------------------------------------------------------------------------------
NGINX Ingress controller
Release: 0.15.0
Build: git-df61bd7
Repository: https://github.com/kubernetes/ingress-nginx
-------------------------------------------------------------------------------
I0530 07:54:22.298248 5 main.go:202] Running in Kubernetes Cluster version v1.9 (v1.9.2) - git (clean) commit 5fa2db2bd46ac79e5e00a4e6ed24191080aa463b - platform linux/amd64
F0530 07:54:22.298610 5 main.go:80] ✖ It seems the cluster it is running with Authorization enabled (like RBAC) and there is no permissions for the ingress controller. Please check the configuration
It is RBAC problem. I check the install script which is downloaded from forum:
heapster-rbac.yaml:
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: heapster
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:heapster
subjects:
- kind: ServiceAccount
name: heapster
namespace: kube-system
One of related kubelet start argument is as follows: (I do not know whether it is relevant).
Environment="KUBELET_AUTHZ_ARGS=--authorization-mode=Webhook --client-ca-file=/etc/kubernetes/pki/ca.crt"
By which way, I could grant permission to ingress controller? Just put namespace kube-system to ingress_nginx_ctl.yaml?
================================================================
Third edition
I put Kun Li's codes into ingress_nginx_role_rb.yaml, and run the following commands:
kubectl apply -f eureka_pod.yaml
kubectl apply -f eureka_svc.yaml
kubectl apply -f config_pod.yaml
kubectl apply -f config_svc.yaml
kubectl apply -f ingress_nginx_role_rb.yaml (just copy paste from Kun Li's answer)
kubectl apply -f nginx_default_backend.yaml
kubectl apply -f ingress_nginx_ctl.yaml
nginx_default_backend.yaml file is listed below:
kind: Service
apiVersion: v1
metadata:
name: nginx-default-backend
namespace: kube-system
spec:
ports:
- port: 80
targetPort: http
selector:
app: nginx-default-backend
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: nginx-default-backend
namespace: kube-system
spec:
replicas: 1
template:
metadata:
labels:
app: nginx-default-backend
spec:
terminationGracePeriodSeconds: 60
containers:
- name: default-http-backend
image: chenliujin/defaultbackend
livenessProbe:
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 30
timeoutSeconds: 5
resources:
limits:
cpu: 10m
memory: 20Mi
requests:
cpu: 10m
memory: 20Mi
ports:
- name: http
containerPort: 8080
protocol: TCP
ingress_nginx_ctl.yaml is listed below:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
spec:
type: LoadBalancer
selector:
app: ingress-nginx
ports:
- name: http
port: 80
targetPort: http
- name: https
port: 443
targetPort: https
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: ingress-nginx
namespace: kube-system
spec:
replicas: 1
template:
metadata:
labels:
app: ingress-nginx
spec:
terminationGracePeriodSeconds: 60
serviceAccount: lb
containers:
- image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0
name: ingress-nginx
imagePullPolicy: Always
ports:
- name: http
containerPort: 80
protocol: TCP
- name: https
containerPort: 443
protocol: TCP
livenessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 30
timeoutSeconds: 5
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/nginx-default-backend
From here, we could see service ingress-nginx namespace is default, not kube-system. But anyway, controller is up.
[root#master3 ingress]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
calico-etcd-cdn8z 1/1 Running 0 11m
calico-kube-controllers-d554689d5-tzdq5 1/1 Running 0 11m
calico-node-dz4d6 2/2 Running 1 11m
coredns-65dcdb4cf-h62bh 1/1 Running 0 11m
etcd-master3 1/1 Running 0 10m
heapster-5c448886d-swp58 1/1 Running 0 11m
ingress-nginx-6ccc799fbc-hq2rm 1/1 Running 0 9m
kube-apiserver-master3 1/1 Running 0 10m
ingress-nginx pod's namespace is kube-system (shown above), but its service's namespace is default.(shown below).
[root#master3 ingress]# kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
gearbox-rack-config-server NodePort 10.97.211.136 <none> 8888:31502/TCP 43m
gearbox-rack-eureka-server NodePort 10.106.69.13 <none> 8761:31501/TCP 43m
ingress-nginx LoadBalancer 10.105.114.64 <pending> 80:30646/TCP,443:31332/TCP 42m
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 44m
as mentioned in the comments, expert's response help me to move forward.
For ingress-controller, image quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0 should be used. And you need setup nginx-default-backend pod and service.
About RBAC, I think you need a seviceaccount to deploy your nginx-ingress-controller, with the following roles and bindings:
apiVersion: v1
kind: ServiceAccount
metadata:
name: lb
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: nginx-ingress-normal
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
resources:
- ingresses/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: nginx-ingress-minimal
namespace: kube-system
rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
- "ingress-controller-leader-dev"
- "ingress-controller-leader-prod"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: nginx-ingress-minimal
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress-minimal
subjects:
- kind: ServiceAccount
name: lb
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nginx-ingress-normal
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress-normal
subjects:
- kind: ServiceAccount
name: lb
namespace: kube-system