I am a newbie to Kubernetes and trying to learn calico networking.
I am following this documentation (https://docs.aws.amazon.com/eks/latest/userguide/calico.html)
and I tried to create a networkpolicy for the traffic to flow between backend to client :
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
namespace: stars
name: backend-client
spec:
podSelector:
matchLabels:
role : client
ingress:
- from:
- namespaceSelector:
matchLabels:
role: backend
ports:
- protocol: TCP
port: 9000
I finished all the 10 steps in the documentation, and i tried to test by creating a policy that would send traffic from the backend to the client with the above policy.
When i applied the policy there was no error , but i don't see the traffic/connection between the two.
Please let me know what is wrong.
Creating NetworkPolicy alone will not help in ensuring that the NetworkPolicy is enforced. We should configure the network plugin like Calico which is integrated with Kubernetes and executes the necessary operations to achieve the intent of the given Network Policy
https://kubernetes.io/docs/concepts/services-networking/network-policies/ says
"Network policies are implemented by the network plugin, so you must be using a networking solution which supports NetworkPolicy - simply creating the resource without a controller to implement it will have no effect."
I believe you need to put your policy in the client namespace instead of the stars namespace. I don't believe there are any pods with role: client in the stars namespace. A pod selector like you've specified only applies to pods in the namespace the policy is in.
While I don't think it is as direct as it could be the Kubernetes Network Policy docs do mention that a NetworkPolicy applies in the given namespace. I suggest you check them out if you haven't already.
I hope that helps.
Related
I have a mosquitto broker running on a pod, this server is exposed as a service as both DNS and IP address.
But this service is accessible by any pod in the cluster.
I want to restrict access to this service such that pods trying to connect to this DNS or IP address should only be able to if the pods have certain name/metadata.
One solution I guess will be to use namespaces? What other solution is there?
The UseCase you are describing is exactly what NetworkPolicies are here for.
Basically you define selector for pods which the network traffic should be restricted (i.e. your mosquito broker) and what specifica pods need to have in order to be allowed to reach it. For example a label "broker-access: true" or whatever seems to be suitable for you.
an example network policy could look like this:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: broker-policy
namespace: default
spec:
podSelector:
matchLabels:
role: message-broker
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
broker-access: true
ports:
- protocol: TCP
port: 6379
this network policy would be applied to every pod with label role=message-broker.
and it would restrict all incoming traffic except for traffic from pods with label broker-acces=true on port 6379.
Hope this helps and gives you a bit of a skaffold for your NetworkPolicy
I have an EKS cluster with an nginx deployment on namespace gitlab-managed-apps. Exposing the application to the public from ALB ingress. I'm trying to block a specific Public IP (ex: x.x.x.x/32) from accessing the webpage. I tried Calico and K8s network policies. Nothing worked for me. I created this Calico policy with my limited knowledge of Network policies, but it blocks everything from accessing the nginx app, not just x.x.x.x/32 external IP. Showing everyone 504 Gateway timeout from ALB
apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
name: ingress-external
namespace: gitlab-managed-apps
spec:
selector:
app == 'nginx'
types:
- Ingress
ingress:
- action: Deny
source:
nets:
- x.x.x.x/32
Try this:
apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
name: ingress-external
namespace: gitlab-managed-apps
spec:
selector:
app == 'nginx'
types:
- Ingress
ingress:
- action: Deny
source:
nets:
- x.x.x.x/32
- action: Allow
calico docs suggests:
If one or more network policies apply to a pod containing ingress rules, then only the ingress traffic specifically allowed by those policies is allowed.
So this means that any traffic is denied by default and only allowed if you explicitly allow it. This is why adding additional rule action: Allow should allow all other traffic that was not matched by the previous rule.
Also remember what docs mention about rules:
A single rule matches a set of packets and applies some action to them. When multiple rules are specified, they are executed in order.
So default Allow rule has to follow the Deny rule for the specific IP, not the other way around.
I have deployed a statefulset in AKS - My goal is to load balance traffic to my statefulset.
From my understanding I can define a LoadBalancer Service that can route traffic based on Selectors, something like this.
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
type: LoadBalancer
ports:
- port: 80
name: web
selector:
app: nginx
However I don't want to necessarily go down the LoadBalance route and I would prefer Ingress doing this work for me, My question is can any of the ingress controller support routing rules which can do Path based routing to endpoints based on selectors? Instead of routing to another service.
Update
To elaborate more on the scenario - Each pod in my statefulset is a stateless node doing data processing of a HTTP feed. I want my ingress service to be able to load balance traffic across these statefulset pods ( honoring keep-alives etc), however given the nature of statefulsets in k8s they are currently exposed through a headless service. I am not sure if a headless service can load balance traffic to my statefulsets?
Update 2
Quick search reveals headless service does not loadbalance
Sometimes you don't need load-balancing and a single Service IP. In this case, you can create what are termed "headless" Services, by explicitly specifying "None" for the cluster IP (.spec.clusterIP).
As much i know it's not possible to do the selector-based routing with ingress.
selector based routing is mostly used during a Blue-green deployment or canary deployment you can only achieve this by using the service mesh. You can use any of the service mesh like istio or APP mesh and you can do the selector base routing.
I have deployed a statefulset in AKS - My goal is to load balance
traffic to my statefulset.
if your goal is to just load balance traffic you can use the ingress controller maybe still not sure about scenrio you are trying to explain.
By default kubernetes service also Load balance the traffic across the PODs.
Flow will be something like DNS > ingress > ingress controller > Kubernetes service (Load balancing here) > any of statefulset
+1 to Harsh Manvar's answer but let me add also my 3 cents.
My question is can any of the ingress controller support routing rules
which can do Path based routing to endpoints based on selectors?
Instead of routing to another service.
To the best of my knowledge, the answer to your question is no, it can't as it doesn't even depend on a particular ingress controller implementation. Note that various ingress controllers, no matter how different they may be when it comes to implementation, must conform to the general specification of the ingress resource, described in the official kubernetes documentation. You don't have different kinds of ingresses, depending on what controller is used.
Ingress and Service work on a different layer of abstraction. While Service exposes a set of pods using a selector e.g.:
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
selector:
app: MyApp 👈
path-based routing performed by Ingress is always done between Services:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: minimal-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /testpath
pathType: Prefix
backend:
service:
name: test 👈
port:
number: 80
I am not sure if a headless service can load balance traffic to my statefulsets?
The first answer is "no". Why?
k8s Service is implemented by the kube-proxy. Kube-proxy itself can work in two modes:
iptables (also known as netfilter)
ipvs (also known as LVS/Linux Virtual Server)
load balancing in case of iptables mode is a NAT iptables rule: from ClusterIP address to the list of Endpoints
load balancing in case of ipvs mode is a VIP (LVS Virtual IP) with the Endpoints as upstreams
So, when you create k8s Service with clusterIP set to None you are exactly saying:
"I need this service WITHOUT load balancing"
Setting up the clusterIP to None causes kube-proxy NOT TO CREATE NAT rule in iptables mode, VIP in ipvs mode. There will be nothing for traffic load balancing across the pods selected by this particular Service selector
The second answer is "it could be". Why?
You are free to create headless Service with desired pods selector. DNS query to this Service will return the list of DNS A records for selected pods. Then you can use this data to implement load balancing YOUR way
The IP whitelisting/blacklisting example explained here https://kubernetes.io/docs/tutorials/services/source-ip/ uses source.ip attribute. However, in kubernetes (kubernetes cluster running on docker-for-desktop) source.ip returns the IP of kube-proxy. A suggested workaround is to use request.headers["X-Real-IP"], however it doesn't seem to work and returns kube-proxy IP in docker-for-desktop in mac.
https://github.com/istio/istio/issues/7328 mentions this issue and states:
With a proxy that terminates the client connection and opens a new connection to your nodes/endpoints. In such cases the source IP will always be that of the cloud LB, not that of the client.
With a packet forwarder, such that requests from the client sent to the loadbalancer VIP end up at the node with the source IP of the client, not an intermediate proxy.
Loadbalancers in the first category must use an agreed upon protocol between the loadbalancer and backend to communicate the true client IP such as the HTTP X-FORWARDED-FOR header, or the proxy protocol.
Can someone please help how can we define a protocol to get the client IP from the loadbalancer?
Maybe your are confusing with kube-proxy and istio, by default Kubernetes uses kube-proxy but you can install istio that injects a new proxy per pod to control the traffic in both directions to the services inside the pod.
With that said you can install istio on your cluster and enable it for only the services you need and apply a blacklisting using the istio mechanisms
https://istio.io/docs/tasks/policy-enforcement/denial-and-list/
To make a blacklist using the source IP we have to leave istio manage how to fetch the source IP address and use som configuration like this taken from the docs:
apiVersion: config.istio.io/v1alpha2
kind: handler
metadata:
name: whitelistip
spec:
compiledAdapter: listchecker
params:
# providerUrl: ordinarily black and white lists are maintained
# externally and fetched asynchronously using the providerUrl.
overrides: ["10.57.0.0/16"] # overrides provide a static list
blacklist: false
entryType: IP_ADDRESSES
---
apiVersion: config.istio.io/v1alpha2
kind: instance
metadata:
name: sourceip
spec:
compiledTemplate: listentry
params:
value: source.ip | ip("0.0.0.0")
---
apiVersion: config.istio.io/v1alpha2
kind: rule
metadata:
name: checkip
spec:
match: source.labels["istio"] == "ingressgateway"
actions:
- handler: whitelistip
instances: [ sourceip ]
---
You can use the param providerURL to maintain an external list.
Also check to use externalTrafficPolicy: Local on the ingress-gateway servce of istio.
As per comments my last advice is to use a different ingress-controller to avoid the use of kube-proxy, my recomendation is to use the nginx-controller
https://github.com/kubernetes/ingress-nginx
You can configure this ingress as a regular nginx acting as a proxy
In a cluster with 2 namespaces (ns1 and ns2), I deploy the same app (deployment) and expose it with a service.
I thought separate namespaces would prevent from executing curl http://deployment.ns1 from a pod in ns2, but apparently, it's possible.
So my question is, how to allow/deny such cross namespaces operations? For example:
pods in ns1 should accept requests from any namespace
pods (or service?) in ns2 should deny all requests from other namespaces
Good that you are working with namespace isolation.
Deploy a new kind Network Policy in your ns1 with ingress all. You can lookup the documentation to define network ingress policy to allow all inbound traffic
Likewise for ns2, you can create a new kind Network Policy and deploy the config in ns2 to deny all ingress. Again the docs will come to rescue to help with you the yaml construct.
It may look something like this:
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
namespace: ns1
name: web-allow-all-namespaces
spec:
podSelector:
matchLabels:
app: app_name_ns1
ingress:
- from:
- namespaceSelector: {}
It would not be answer you want, but I can provide the helpful feature information to implement your requirements.
AFAIK Kubernetes can define network policy to limit the network access.
Refer Declare Network Policy for more details of Network Policy.
Default policies
Setting a Default NetworkPolicy for New Projects in case OpenShift.