How do I connect to an ECS service running on Fargate? - aws-api-gateway

I'm building a slack app, and my application is expected to receive webhook calls from slack. SO I need to give Slack the "endpoint URL for my service"
I'm not sure I understand what AWS bricks I need to put together to make this work.
So far, I have configured my service using Fargate and it is running on ECS with 1 task.
I'm not really sure how I can successfully connect the internet (slack) to my container instance. How do I make an "endpoint" that slack can send requests to ? I believe I need to use API Gateway for this, but I'm not sure how I'm supposed to configure API Gateway so it redirect my URLs to my ECS service...?
Notes :
I am not planning to have more than one task right now, so it would be more convenient if I did not have to setup a load balancer
However I'm planning to make some regular updates to the service itself, and the tasks will be restarted often

I'm not sure if there's an alternative or not, but when setting up a service on fargate, in order to receive traffic you will need
To have a load balancer (either an ALB or NLB should work)
In case of an ALB, a target group targeting "IP" must be created
It's only during the creation of the service that you can select to put your containers behind a target group or ALB
For instance, to distribute traffic from a target group, you can use this to configure the service
"loadBalancers": [
{
"containerName": "your-container-name-app",
"containerPort": 80,
"targetGroupArn": "arn:aws:elasticloadbalancing:eu-central-1:account-id:targetgroup/targetgroup-name/RANDOM-ID"
}
],
Create the target group like this
aws elbv2 create-target-group \
--name targetgroup-name \
--protocol HTTP \
--port 80 \
--vpc-id vpc-YOUR_VPC_ID \
--health-check-protocol HTTP \
--health-check-path /healthcheck \
--target-type ip
So you'll basically use the load balancer DNS to route traffic to your instances. Using an ALB, you can easily intercept a certain path (eg /slack/*) to route to your specific target group, so the same ALB can be used for multiple different services.
But you need a load balancer, and cannot target Fargate containers directly from what I understand.

Related

Using HTTP API to call multiple services running on AWS ECS

My goal here is to deploy two spring boot services using AWS ECS Fargate in a private subnet and access them via AWS API Gateway. Basically, I want to use a single HTTP API and then based on the path it should call the appropriate service. I am using VPC Links, and Cloud Map for linking services running in a private subnet, for service discovery. First of all - Is this assumption even correct, i.e. can we use a single HTTP API to call two different services based on a path?
Some considerations of how I created the ECS services.
ECS Service A is deployed in a private subnet, it has no public IP enabled and the service discovery has been enabled. While enabling service discovery I choose the DNS record type to be SRV, giving a port number and TTL as 60 secs.
ECS Service B is also deployed similarly.
Both ECS Service A and B have a separate Service discovery endpoint.
Now in the API Gateway, the steps I followed were
Created a new HTTP API using the defaults, this means the default stage and no routes and integrations configured yet.
Then I created a VPC Link for HTTP API by assigning it a name (service-a-vpclink), assigning a VPC, subnet and appropriate security group (security that was assigned to the ECS service for service A).
Now I created a route where the method is "ANY" and the path is "$default" and assigned an integration to it, I am able to reach all my endpoints of service A running in the private subnet. (So all good here, as this shows that I am able to reach the service running in a private subnet using API Gateway.)
For the integration that I mentioned in point 3, this was of type "Private Resource", target service as "Cloud Map" and then selecting the namespace and appropriate service (serviceA) along with the VPC link that was created in step 2.
But this is what I don't want to do. I want something like the below:
Hitting any endpoint like "https://uzhgtf6t8u.execute-api.eu-west-2.amazonaws.com/serviceA/any-serviceA-endpoints" where /serviceA is a path that is configured in API Gateway and then any-serviceA-endpoints are the actual endpoints configured in the backend service running, navigates to service A endpoints.
Hitting any endpoint like "https://uzhgtf6t8u.execute-api.eu-west-2.amazonaws.com/serviceB/any-serviceB-endpoints" where /serviceB is a path that is configured in API Gateway and then any-serviceB-endpoints are the actual endpoints configured in the backend service running, navigates to service B endpoints.
Here I attach separate integrations to path /serviceA and to path /serviceB, but this does not work. Rather this way the response is 404, not found.
What exactly am I not following?
Many thanks..
Screenshot of route

Within a Kubernetes cluster catch outgoing requests from a Pod and redirect to a different target

I have a cluster with 3 nodes. In each node i have a frontend application running in a Pod and backend application running in a separate Pod.
I send data from the frontend application to the backend application, to do this i utilise the Cluster IP Service and k8 dns resource.
I also have a function in my frontend where i send data to a separate service unrelated to my k8s cluster. I send this data using a standard AJAX request to a url with a payload i.e http://my-seperate-service-unrelated-tok8.com.
All of this works correctly and the cluster operates as i want. - i have this cluster deployed to GKE. 

I now want to run this cluster local using minikube, which i have been able to do, however, when i am running locally i do not want to send data to my external service - instead i want to forward it to either a new Pod i will create or just not send it.


The problem here is i need a proxy to intercept outgoing network traffic, check if the outgoing request is the request i am looking for and if it is then redirect it.
I understand each node running in a cluster has a kube-proxy service running within the node - which is used to forward traffic to the relevant services in the cluster. 

I would like to either extend this service, or create a new proxy service where i can listen for outgoing traffic to a specific url and redirect it. 

Is this possible to do in a k8 cluster? I assume there is a Service i can create to listen for all outgoing requests and redirect specific requests based on rules i set. 

I wasn’t sure if k8 clusters have a Service already configured i can simply add to - that’s why i thought of the kube-proxy, would anyone be able to advice on this?

I wanted to add this proxy so i don’t have to change my code when its ran locally in minikube or deployed to GKE.


Any help is greatly appreciated. Thanks!
I did a tool that help you to forward a service to another service,local port, service from other cluster, etc...
This way you can have exactly your same urls, ports and code... but the underlying services gets "replaced", if I understand correctly this is what you are looking for.
Here is a quick example of an stage service being replaced with my local 3000 port
This is the repository with more info and examples: linker-tool
If you are interested let me know if you need help or have any question.

Securing an exposed load balancer service in kubernetes

I have a workload deployed in kubernetes. I have exposed it using a load balancer service because I need an external IP to communicate with the workload.
The external IP is now publicly accessible. How do I secure it so that only I will be able to access it from an external application?
Kubernetes doesn't come with out-of-the-box authentication for external services. If you have more services and security is important for you I would take a look into istio project. You can configure authentication for your services in decalarative way using authentication policy:
https://istio.io/docs/tasks/security/authn-policy/#end-user-authentication
Using istio you can secure not only incoming connections, but also outgoing and internal traffic.
If you are new to service mesh concept and you don't know how to start, you can check kyma-project where istio is already configured and you can apply token validation with one click in UI or single kubectl command. Check the example:
https://github.com/kyma-project/examples/tree/master/gateway

How do I find out the external IP of a Load Balancer service?

I am using Kubernetes Engine on the Google Cloud Platform. I have a pod running a process in a Docker scratch container. I also have a load balancer service that gives me access to the pod from the outside world.
The process running in the pod needs to know what its external IP address is. How can I get this?
Prior to using Kubernetes Engine I was using Compute Engine and could find the external IP address by the following:
curl -H "Metadata-Flavor: Google" http://metadata/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip
Are there any internal tools I can use that would be available to my process? Or would I need the process to call an external site that can mirror back the IP address?
Every Pod (unless configured not to do so) has valid kubernetes credentials in /var/run/secrets/kubernetes.io/serviceaccount/token as described here so the answer is to use the kubernetes API to ask the Service in front of the Pod(s) for its status:loadBalancer:ingress:ip: as described here which I have every reason to believe GKE will keep up-to-date with any changes to the load balancer. The kubernetes API is always(?) located at https://kubernetes (that's normally enough, or https://kubernetes.default.svc.cluster.local is its full name), so there should be very little configuration the Pod would need in order to carry out the lookup.
The asterisk to that response is that one must provide the name of the Service to the Pod(s) of the Service sitting in front of it, because (for the most part) there is no way for the Pod to know how many Services point to it.

Azure Container Service with Kubernetes - Containers not able to reach Internet

I created an ACS (Azure Container Service) using Kubernetes by following this link : https://learn.microsoft.com/en-us/azure/container-service/container-service-kubernetes-windows-walkthrough & I deployed my .net 4.5 app by following this link : https://learn.microsoft.com/en-us/azure/container-service/container-service-kubernetes-ui . My app needs to access Azure SQL and other resources that are part of some other resource groups in my account, but my container is not able to make any outbound calls to network - both inside azure and to internet. I opened some ports to allow outbound connections, that is not helping either.
When I create an ACS does it come with a gateway or should I create one ? How can I configure ACS so that it allows outbound network calls ?
Thanks,
Ashok.
Outbound internet access works from an Azure Container Service (ACS) Kubernetes Windows cluster if you are connecting to IP Addresses other than the range 10.0.0.0/16 (that is you are not connecting to another service on your VNET).
Before Feb 22,2017 there was a bug where Internet access was not available.
Please try the latest deployment from ACS-Engine: https://github.com/Azure/acs-engine/blob/master/docs/kubernetes.windows.md., and open an issue there if you still see this, and we (Azure Container Service) can help you debug.
For the communication with service running inside the cluster, you can use the Kube-dns which allows you to access service by its name. You can find more details at https://kubernetes.io/docs/admin/dns/
For the external communication (internet), there is no need to create any gateway etc. By default your containers inside a pod can make outbound connections. To verify this, you can run powershell in one of your containers and try to run
wget http://www.google.com -OutFile testping.txt
Get-Contents testping.txt
and see if it works.
To run powershell, ssh to your master node - instructions here
kubectl exec -it <pod_name> -- powershell