Can we have ECS task metrics over the availability zone,
say I have 2 tasks in 1a, 2 in 1b can i have seperate metrics for each of the Availabilty Zone.
I went through these docs:-
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cloudwatch-metrics.html
but did not find a way.
Related
I have an ECS Fargate service which uses CloudWatch alarms to scale-in/scale-out using service auto-scaling. The task containers have long processing times (upto 40 minutes) and I don't want a running container to get killed when a scale-in happens. Is there way to do that for an ECS task/service?
PS: I have looked at the stopTimeout property in a task-definition but its max value is only 120 seconds. I have also looked at scale-in protection for EC2 instances but haven't found any such solution for an ECS Fargate task.
Support for ECS task scale-in protection was released on 2022-11-10: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-scale-in-protection.html
In summary, you can use the new ECS container agent endpoint from inside a task to mark it as protected:
PUT $ECS_AGENT_URI/task-protection/v1/state -d
'{"ProtectionEnabled":true}'
Alternatively, you can use the UpdateTaskProtection API to achieve the same result from outside the task: https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_UpdateTaskProtection.html
I have a ECS fargate cluster up and running and it has 1 service and 1 task definition attached to it.
The task definition already has 2 container images described.This cluster is up and running.
Can I create a new service and for another application and configure it with this Existing ECS cluster.
If yes, will both the service run simultaneously.
From the AWS Documentation in regards Amazon ECS Clusters
An Amazon ECS cluster is a logical grouping of tasks or services. Your
tasks and services are run on infrastructure that is registered to a
cluster.
So I believe, you should be able to run multiple services in a cluster that is attached to its related task definition in the ECS.
Source Documentation - https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html
We have an application to create/start/stop containers inside AWS ECS. we are not making use of ecs services because we don't want container to be started if it is stopped by an application.
So how to automate scale-in/scale-out of the cluster instances in ecs without using ecs services?
Below is the documentation which will tell you step by step how to scale your container instances.
Scaling Container Instances
So how this works is :
Say you have one Container Instance and 2 services running on it.
You are required to increase the ECS Service but it will not scale as it doesn't have resources available on one Container Instance.
Following up the documentation, you can set up CloudWatch Alarms on let's say MemoryReservation metric for your cluster.
When the memory reservation of your cluster rises above 75% (meaning that only 25% of the memory in your cluster is available to for new tasks to reserve), the alarm triggers the Auto Scaling group to add another instance and provide more resources for your tasks and services.
Depending on the Amazon EC2 instance types that you use in your
clusters, and quantity of container instances that you have in a
cluster, your tasks have a limited amount of resources that they can
use while running. Amazon ECS monitors the resources available in the
cluster to work with the schedulers to place tasks. If your cluster
runs low on any of these resources, such as memory, you are eventually
unable to launch more tasks until you add more container instances,
reduce the number of desired tasks in a service, or stop some of the
running tasks in your cluster to free up the constrained resource.
we have a scenario where we want to stop the cluster (worker nodes) in Night when it is not being used and start it again in the morning when people start using the application running on AWS EKS. Any suggestions will be helpful.
I think you can achieve it by changing the desired capacity of auto scaling group using aws cli. You can run it as a CRON job:
aws autoscaling update-auto-scaling-group --auto-scaling-group-name <my-auto-scaling-group> --desired-capacity 0 --min-size 0
I am running a job with a kubernetes POD and I need to measure the time between the creation the job by the user and the time of starting running this job on the node .
I want to get it through some api.
Does anyone know how can I get it ?
Monitoring Kubernetes ( number of pending pods/jobs)
Use the kube-state-metrics package for monitoring and a small Go program called veneur-prometheus to scrape the Prometheus metrics kube-state-metrics emits and publish them as statsd metrics to monitoring system.
For example, here’s a chart of the number of pending pods in the cluster over the last hour. Pending means that they’re waiting to be assigned a worker node to run on. You can see that the number spikes at 11am, because a lot of cron jobs run at the 0th minute of the hour in this case.
An example chart showing pending pods in a cluster over the last hour