I had started with a Service Fabric Cluster (SFC) that was using the Basic SKU Load Balancer (LB). The SFC has Primary, FrontEnd, and BackEnd node types. Then I needed to change the FrontEnd LB to the Standard SKU in order to use the additional capability provided. The only thing modified in the LB ARM template was to specify the different SKU. Now the VMSS using the Standard LB cannot connect to the SFC and all the nodes appear in the 'Down' state. In order to deploy the different LB SKU, I had to delete the affected VMSS and LB. The LB is internal, connecting only to subnets with the VNet.
"sku": {
"name": "Standard"
}
Other than changing the SKU property, are there any other settings required to allow the VMSS nodes to notify and connect to the SFC?
When using an internal Standard Load Balancer, outbound NAT is not available until outbound connectivity has been explicitly declared. You can define outbound connectivity using an outbound rule to create outbound connectivity for VMs behind an internal Standard Load Balancer with these steps:
Create a public Standard Load Balancer.
Create a backend pool and place the VMs into a backend pool of the public Load Balancer in addition to the internal Load Balancer.
Configure an outbound rule on the public Load Balancer to program outbound NAT for these VMs.
https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-outbound-rules-overview#outbound-nat-for-internal-standard-load-balancer-scenarios
https://learn.microsoft.com/en-us/azure/load-balancer/configure-load-balancer-outbound-cli
Related
I have set up my application to be served by a Kubernetes NGINX ingress in AKS. Today while experimenting with the Azure API management, I tried to set it up so that all the traffic to the ingress controller would go through the API management. I pointed its backend service to the current public address of the ingress controller but I was wondering when I make the ingress controller private or remove it altogether to rely on the Kubernetes services instead, how API management could access it and how I would define the backend service in API management. By the way, while provisioning the API management instance, I added a new subnet to the existing virtual network of the AKS instance so they are in the same network.
There are two modes of deploying API Management into a VNet – External and Internal.
If API consumers do not reside in the cluster VNet, the External mode (Fig below) should be used. In this mode, the API Management gateway is injected into the cluster VNet but accessible from public internet via an external load balancer. It helps to hide the cluster completely while still allowing external clients to consume the microservices. Additionally, you can use Azure networking capabilities such as Network Security Groups (NSG) to restrict network traffic.
If all API consumers reside within the cluster VNet, then the Internal mode (Figure below) could be used. In this mode, the API Management gateway is injected into the cluster VNET and accessible only from within this VNet via an internal load balancer. There is no way to reach the API Management gateway or the AKS cluster from public internet.
In both cases, the AKS cluster is not publicly visible. The Ingress Controller may not be necessary. Depending on your scenario and configuration, authentication might still be required between API Management and your microservices. For instance, if a Service Mesh is adopted, it always requires mutual TLS authentication.
Pros:
The most secure option because the AKS cluster has no public endpoint
Simplifies cluster configuration since it has no public endpoint
Ability to hide both API Management and AKS inside the VNet using the Internal mode
Ability to control network traffic using Azure networking capabilities such as Network Security Groups (NSG)
Cons:
Increases complexity of deploying and configuring API Management to work inside the VNet
Reference
To restrict access to your applications in Azure Kubernetes Service (AKS), you can create and use an internal load balancer. An internal load balancer makes a Kubernetes service accessible only to applications running in the same virtual network as the Kubernetes cluster.
You can either expose your the backends on the AKS cluster through internal Ingress or simply using Services of type internal load balancer.
You can then point the API Gateway's backend to the internal Ingress' Private IP address or the internal load balancers Service's EXTERNAL IP (which would also be a private IP address). These private IP addresses are accessible within the Virtual Network and any connected network (i.e. Azure virtual networks connected through peering or Vnet-to-Vnet Gateway, or on-premises networks connected to the AKS Vnet). In your case, if the API Gateway is deployed in the same Virtual Network then, it should be able to access these private IP addresses. If the API Gateway is deployed in a different Virtual Network, please connect it to the AKS virtual network using VNET Peering or Vnet-to-Vnet Gateway, depending on your use-case.
Is it working now. If not, please try to add that vnet and subnet in apim. Mostly it won't required, because both of them are in same vnet,we can access directly via privateip. Please check the routing is properly configured in the ingress controller. Another option is, just for testing, you can directly call the service from api by avoiding ingress controller. So that we can make sure that, there is no request is getting blocked by nsg or others
.
According to AWS documentation:
You can configure Service Discovery for an ECS Service that is behind
a Load Balancer, but Service Discovery traffic is always routed to the
Task and not the Load Balancer.
If this the case, how does the load balancing happens here?
Also, without the Load Balancer, how does the service discovery works, will the traffic routed to a random Container Instances?
TL;DR Yes, the traffic will be sent to random instances.
When you use ECS Service Discovery, you have two options for discovering your services. One is via Route 53 DNS, which in case of ECS Service Discovery leverages Multivalue Routing Policy, so that your client application receives up to eight healthy endpoints, selected at random.
The other option is to use Cloud Map DiscoverInstances API, which returns up to 100 endpoints for a given service name, selected at random.
Can someone explain the load balancing mechanism in AWS ECS for me? I clearly understand how inter service communication is handled within a kubernetes cluster, there is an automatic load balancer applied when accessing a defined internal service. This means Container/Pod scalability is simply predefined:
when a Pod-1A from within the service-A is accessing another Pod-1B
from within a different Service-B (Service to Service communication)
this call is automatically load balanced to this Pod-1B from
service-B.
So with service Registry in kubernetes we simply need to define Services and communication is automatically load balanced to the available Pods within the services.
So assuming that Pods are equal to Tasks and Services are equal to Services in AWS ECS, how is this load balancing mechanism handled wihtin ECS? Do we really need to apply an Elastic Load balancer at the task/pod level manually compared to kubernetes? (So that we need to define manually a load balancer for every service, to make this service and its tasks with its container scalable?)
Edit:
What is the reason in AWS ECS, to define a service which instantiates
multiple replicas of a Task, when no load balancer has been defined?
Will the traffic be routed only to the same Task replica (Container)
all the time? (No scaling at all?)
Please note, this is not about access from external ip addresses, where an ingress controller is needed. I am talking about microservices where each service exposes its own http api to communicate with other services within the cluster (internal microservice Application), typically there is an API Gateway handling external traffic (ingress controller).
I am running some internal services and also some customer facing services in one K8s cluster. The internal ones should only be accessible from some specific ips and the customer facing services should be accessible worldwide.
So I created my Ingresses and an nginx Ingress Controller and some K8s LoadBalancer Services with the proper ip filters.
Now I see those Firewall rules in GCP are created behind the scenes. But they are conflicting and the "customer facing" firewall rules overrule the "internal" ones. And so everything of my K8s Cluster is visible worldwide.
The usecase sounds not that exotic to me - do you have an idea how to get some parts of a K8s cluster protected by firewall rules and some accessible everywhere?
As surprising as it is, the L7 (http/https) load balancer in GCP created by a Kubernetes Ingress object has no IP whitelisting capabilities by default, so what you described is working as intended. You can filter on your end using the X-Forwarded-For header (see Target Proxies under Setting Up HTTP(S) Load Balancing).
Whitelisting will be available trough Cloud Armour, which is in private beta at the moment.
To make this situation slightly more complicated: the L4 (tcp/ssl) load balancer in GCP created by a Kubernetes LoadBalancer object (so, not an Ingress) does have IP filtering capability. You simply set .spec.loadBalancerSourceRanges on the Service for that. Of course, a Service will not give you url/host based routing, but you can achieve that by deploying an ingress controller like nginx-ingress. If you go this route you can still create Ingresses for your internal services you just need to annotate them so the new ingress controller picks them up. This is a fairly standard solution, and is actually cheaper than creating L7s for each of your internal services (you will only have to pay for 1 forwarding rule for all of your internal services).
(By "internal services" above I meant services you need to be able to access from outside of the itself cluster but only from specific IPs, say a VPN, office, etc. For services you only need to access from inside the cluster you should use type: ClusterIP)
I have a service fabric cluster with two node types, Frontend and Backend. Each node type has a single application in there that listen on a REST interface. The front end app should be accessible from the outside world, but the backend node type should only be accessible from the front end app.
Each node type has an associated Load Balancer and I have setup rules to allow access to each of the apps and this all works fine. However I would like to make sure that the load balancer only allows comms to backend node type if the comms originates from the front end app. I cannot see a way to configure this in the load balancer rules.
Can someone tell me how to prevent public access to my backend application?
I believe you can solve this problem by using Network Security Groups.
A network security group (NSG) contains a list of security rules that
allow or deny network traffic to resources connected to Azure Virtual
Networks (VNet).
Here's an example on how to deploy this.
Use this template as a sample for setting up a three nodetype secure
cluster and to control the inbound and outbound network traffic using
Network Security Groups. The template has a Network Security Group for
each of the VMSS to control the traffic in and out of the VMSS.