ingress points to wrong port on a service - kubernetes

Question

I have a Kubernetes service that exposes two ports as follows
Name: m-svc
Namespace: m-ns
Labels:
Annotations: <none>
Selector: app=my-application
Type: ClusterIP
IP: 10.233.43.40
Port: first 8080/TCP
TargetPort: 8080/TCP
Endpoints: 10.233.115.178:8080,10.233.122.166:8080
Port: second 8888/TCP
TargetPort: 8888/TCP
Endpoints: 10.233.115.178:8888,10.233.122.166:8888
Session Affinity: None
Events: <none>
And here is the ingress definition:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: f5
virtual-server.f5.com/http-port: "80"
virtual-server.f5.com/ip: controller-default
virtual-server.f5.com/round-robin: round-robin
creationTimestamp: 2018-10-05T18:54:45Z
generation: 2
name: m-ingress
namespace: m-ns
resourceVersion: "39557812"
selfLink: /apis/extensions/v1beta1/namespaces/m-ns
uid: 20241db9-c8d0-11e8-9fac-0050568d4d4a
spec:
rules:
- host: www.myhost.com
http:
paths:
- backend:
serviceName: m-svc
servicePort: 8080
path: /first/path
- backend:
serviceName: m-svc
servicePort: 8080
path: /second/path
status:
loadBalancer:
ingress:
- ip: 172.31.74.89
But when I go to www.myhost.com/first/path I end up at the service that is listening on port 8888 of m-svc. What might be going on?
Another piece of information is that I am sharing a service between two ingresses that point to different ports on the same service, is this a problem? There is a different ingress port the port 8888 on this service which works fine
Also I am using an F5 controller
After a lot of time investigating this, it looks like the root cause is in the F5s, it looks like because the name of the backend (Kubernetes service) is the same, it only creates one entry in the pool and routes the requests to this backend and the one port that gets registered in the F5 policy. Is there a fix for this? A workaround is to create a unique service for each port but I dont want to make this change , is this possible at the F5 level?

Answer 1

From what I see you don't have a Selector field in your service. Without it, it will not forward to any backend or pod. What makes you think that it's going to port 8888? What's strange is that you have Endpoints in your service. Did you manually create them?
The service would have to be something like this:
Name: m-svc
Namespace: m-ns
Labels:
Annotations: <none>
Selector: app=my-application
Type: ClusterIP
IP: 10.233.43.40
Port: first 8080/TCP
TargetPort: 8080/TCP
Endpoints: 10.233.115.178:8080,10.233.122.166:8080
Port: second 8888/TCP
TargetPort: 8888/TCP
Endpoints: 10.233.115.178:8888,10.233.122.166:8888
Session Affinity: None
Events: <none>
Then in your deployment definition:
selector:
matchLabels:
app: my-application
Or in a pod:
apiVersion: v1
kind: Pod
metadata:
annotations: { ... }
labels:
app: my-application
You should also be able to describe your Endpoints:
$ kubectl describe endpoints m-svc
Name: m-svc
Namespace: default
Labels: app=my-application
Annotations: <none>
Subsets:
Addresses: x.x.x.x
NotReadyAddresses: <none>
Ports:
Name Port Protocol
---- ---- --------
first 8080 TCP
second 8081 TCP
Events: <none>

Answer 2

Your Service appears to be what is called a headless service: https://kubernetes.io/docs/concepts/services-networking/service/#headless-services. This would explain why the Endpoints was created automatically.
Something is amiss because it should be impossible for your HTTP request to arrive at you pods without the .spec.selector populated.
I suggest deleting the Service you created and delete the Endpoints with the same name and then recreate the Service with type=ClusterIP and the spec.selector properly populated.