How to not rebuild a DockerImageAsset at every deploy using aws-cdk in TypeScript? - amazon-ecs

My app is a Python API that I package as a Docker image and use with ECS Fargate (Spot Instances). The code below works.
My issue is that it rebuilds the entire image every time I deploy this – which is very time-consuming (downloads all dependencies, makes the image, uploads, etc). I want it to reuse the exact same image uploaded to ECR by aws-cdk itself.
Is there a way (env variable or else) for me to skip this when I don't touch the app's code and just make changes to the stack?
#!/usr/bin/env node
import * as cdk from "#aws-cdk/core"
import * as ecs from "#aws-cdk/aws-ecs"
import * as ec2 from "#aws-cdk/aws-ec2"
import * as ecrassets from "#aws-cdk/aws-ecr-assets"
// See https://docs.aws.amazon.com/cdk/api/latest/docs/aws-ecs-readme.html
export class Stack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props)
/**
* Repository & Image
*/
const apiDockerImage = new ecrassets.DockerImageAsset(
this,
`my-api-image`,
{
directory: `.`,
exclude: [`cdk.out`, `cdk`, `.git`]
}
)
/**
* Cluster
*/
const myCluster = new ecs.Cluster(this, "Cluster", {})
// Add Spot Capacity to the Cluster
myCluster.addCapacity(`spot-auto-scaling-group-capacity`, {
maxCapacity: 2,
minCapacity: 1,
instanceType: new ec2.InstanceType(`r5a.large`),
spotPrice: `0.0400`,
spotInstanceDraining: true
})
// A task Definition describes what a single copy of a task should look like
const myApiFargateTaskDefinition = new ecs.FargateTaskDefinition(
this,
`api-fargate-task-definition`,
{
cpu: 2048,
memoryLimitMiB: 8192,
}
)
// Add image to task def
myApiFargateTaskDefinition.addContainer(`api-container`, {
image: ecs.ContainerImage.fromEcrRepository(
apiDockerImage.repository,
`latest`
),
})
// And the service attaching the task def to the cluster
const myApiService = new ecs.FargateService(
this,
`my-api-fargate-service`,
{
cluster: myCluster,
taskDefinition: myApiFargateTaskDefinition,
desiredCount: 1,
assignPublicIp: true,
}
)
}
}

The proper solution is to build your image outside of this deployment process and just get a reference to that image in ECR.

Related

How to copy file from k8 pod to GCP storage programmatically?

I am deploying my springboot application image to GCP.My springboot application processes data and creates one file.I want to copy this file to GCP Storage.Can we achieve this from Kubernetes pod to GCP Storage?
You can use GCSFUSE to mount GCS bucket to Kubernetes Pod as a volume.
Here is an example of Container Lifecycle Hooks to mount and unmount GCS bucket.
#...
securityContext:
privileged: true
capabilities:
add:
- SYS_ADMIN
lifecycle:
postStart:
exec:
command: ["gcsfuse", "-o", "nonempty,allow_other", "bucket-name", "/usr/share/nginx/bucket-data"]
preStop:
exec:
command: ["fusermount", "-u", "/usr/share/nginx/bucket-data"]
#...
Reference: MOUNTING GOOGLE CLOUD STORAGE BUCKET TO KUBERNETES POD
And to copy the file from your Pod to GCS bucket, use Fabric8 Kubernetes Client, which is described in detail here.
Example code:
public class DownloadFileFromPod {
public static void main(String[] args) {
try (KubernetesClient client = new KubernetesClientBuilder().build()) {
// Path Where to copy file to cloud storage
Path downloadToPath = new File("path_of_gcs").toPath();
// Using Kubernetes Client to copy file from pod
client.pods()
.inNamespace("default") // <- Namespace of pod
.withName("quarkus-84dc4885b-tsck6") // <- Name of pod
.file("/deployments/quarkus-1.0.0-runner.jar") // <- Path of file inside pod
.copy(downloadToPath); // <- path where to copy downloaded file
}
}
}
You can also use the Cloud Storage client libraries to programmatically achieve this.
Here is an example for uploading files using java
import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.BlobInfo;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
public class UploadObject {
public static void uploadObject(
String projectId, String bucketName, String objectName, String filePath) throws IOException {
// The ID of your GCP project
// String projectId = "your-project-id";
// The ID of your GCS bucket
// String bucketName = "your-unique-bucket-name";
// The ID of your GCS object
// String objectName = "your-object-name";
// The path to your file to upload
// String filePath = "path/to/your/file"
Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();
BlobId blobId = BlobId.of(bucketName, objectName);
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).build();
// Optional: set a generation-match precondition to avoid potential race
// conditions and data corruptions. The request returns a 412 error if the
// preconditions are not met.
Storage.BlobTargetOption precondition;
if (storage.get(bucketName, objectName) == null) {
// For a target object that does not yet exist, set the DoesNotExist precondition.
// This will cause the request to fail if the object is created before the request runs.
precondition = Storage.BlobTargetOption.doesNotExist();
} else {
// If the destination already exists in your bucket, instead set a generation-match
// precondition. This will cause the request to fail if the existing object's generation
// changes before the request runs.
precondition =
Storage.BlobTargetOption.generationMatch(
storage.get(bucketName, objectName).getGeneration());
}
storage.create(blobInfo, Files.readAllBytes(Paths.get(filePath)), precondition);
System.out.println(
"File " + filePath + " uploaded to bucket " + bucketName + " as " + objectName);
}
}
Also check this thread with python examples for more details.
Yes you can, you need to install gsutil in your springboot application image then mount your storage token in that container as configmap or secret.
In the end you can run gsutil like this:
gsutil cp source_file gs://my-bucket

How to deploy the kinesis-video-producer Docker image from AWS's own ECR to Fargate using CDK in TypeScript?

I'm trying to stand up a proof of concept that ingests an RTSP video stream into Kinesis Video. The provided documentation has a docker image all set up that seems to have everything I need to do this, hosted by AWS on 546150905175.dkr.ecr.us-west-2.amazonaws.com. What I am having trouble with, though, is getting that deployment (via an Amplify Custom category, in TypeScript CDK) to work.
I've tried different variations on
import * as iam from "#aws-cdk/aws-iam";
import * as ecs from "#aws-cdk/aws-ecs";
import * as ec2 from "#aws-cdk/aws-ec2";
const kinesisUserAccessKey = new iam.AccessKey(this, 'KinesisStreamUserAccessKey', {
user: kinesisStreamUser,
})
const servicePrincipal = new iam.ServicePrincipal('ecs-tasks.amazonaws.com');
const executionRole = new iam.Role(this, 'IngestVideoTaskDefExecutionRole', {
assumedBy: servicePrincipal,
managedPolicies: [
iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AmazonECSTaskExecutionRolePolicy'),
]
});
const taskDefinition = new ecs.FargateTaskDefinition(this, 'IngestVideoTaskDef', {
cpu: 512,
memoryLimitMiB: 1024,
executionRole,
})
const image = ecs.ContainerImage.fromRegistry('546150905175.dkr.ecr.us-west-2.amazonaws.com/kinesis-video-producer-sdk-cpp-amazon-linux:latest');
taskDefinition.addContainer('IngestVideoContainer', {
command: [
'gst-launch-1.0',
'rtspsrc',
`location="${locationParam.secretValue.toString()}"`,
'short-header=TRUE',
'!',
'rtph264depay',
'!',
'video/x-h264,',
'format=avc,alignment=au',
'!',
'kvssink',
`stream-name="${cfnStream.name}"`,
'storage-size=512',
`access-key="${kinesisUserAccessKey.accessKeyId}"`,
`secret-key="${kinesisUserAccessKey.secretAccessKey.toString()}"`,
`aws-region="${REGION}"`,
// `aws-region="${cdk.Aws.REGION}"`,
],
image,
logging: new ecs.AwsLogDriver({
streamPrefix: 'IngestVideoContainer',
}),
})
const service = new ecs.FargateService(this, 'IngestVideoService', {
cluster,
taskDefinition,
desiredCount: 1,
securityGroups: [
ec2.SecurityGroup.fromSecurityGroupId(this, 'DefaultSecurityGroup', SECURITY_GROUP_ID)
],
vpcSubnets: {
subnets: SUBNET_IDS.map(subnetId => ec2.Subnet.fromSubnetId(this, subnetId, subnetId)),
}
})
But it seems like regardless of what I do, an amplify push just stays in 'in progress' for like an hour until I go into the CloudFormation console and cancel the stack update, but deep in the my way to the ECS Console I managed to find an actual error message:
Resourceinitializationerror: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve ecr registry auth: service call has been retried 3 time(s): RequestError: send request failed caused by: Post "https://api.ecr.us-west-2.amazonaws.com/": dial tcp 52.94.177.118:443: i/o timeout
It seems to be some kind of networking issue, but I'm not sure how to proceed. Any assistance you can provide would be wonderful. Cheers!
Figured it out. For those stuck with similar issues, you have to give it an execution role with AmazonECSTaskExecutionRolePolicy, which I already edited above, and set assignPublicIp: true in the service.

Referencing a loop object

i am currently checking out tanka + jsonnet. But evertime i think i understand it... sth. new irritates me. Can somebody help me understand how to do a loop-reference? (Or general better solution?)
Trying to create multiple deployments with a corresponding configmapVolumeMount and i am not sure how to reference to the according configmap object here?
(using a configVolumeMount it works since it refers to the name, not the object).
deployment: [
deploy.new(
name='demo-' + instance.name,
],
)
+ deploy.configMapVolumeMount('config-' + instance.name, '/config.yml', k.core.v1.volumeMount.withSubPath('config.yml'))
for instance in $._config.demo.instances
],
configMap: [
configMap.new('config-' + instance.name, {
'config.yml': (importstr 'files/config.yml') % {
name: instance.name,
....
},
}),
for instance in $._config.demo.instances
]
regards
Great to read that you're making progress with tanka, it's an awesome tool (once you learned how to ride it heh).
Find below a possible answer, see inline comments in the code, in particular how we ab-use tanka layout flexibility, to "populate" deploys: [...] array with jsonnet objects containing each paired deploy+configMap.
config.jsonnet
{
demo: {
instances: ['foo', 'bar'],
image: 'nginx', // just as example
},
}
main.jsonnet
local config = import 'config.jsonnet';
local k = import 'github.com/grafana/jsonnet-libs/ksonnet-util/kausal.libsonnet';
{
local deployment = k.apps.v1.deployment,
local configMap = k.core.v1.configMap,
_config:: import 'config.jsonnet',
// my_deploy(name) will return name-d deploy+configMap object
my_deploy(name):: {
local this = self,
deployment:
deployment.new(
name='deploy-%s' % name,
replicas=1,
containers=[
k.core.v1.container.new('demo-%s' % name, $._config.demo.image),
],
)
+ deployment.configMapVolumeMount(
this.configMap,
'/config.yml',
k.core.v1.volumeMount.withSubPath('config.yml')
),
configMap:
configMap.new('config-%s' % name)
+ configMap.withData({
// NB: replacing `importstr 'files/config.yml';` by
// a simple YAML multi-line string, just for the sake of having
// a simple yet complete/usable example.
'config.yml': |||
name: %(name)s
other: value
||| % { name: name }, //
}),
},
// Tanka is pretty flexible with the "layout" of the Kubernetes objects
// in the Environment (can be arrays, objects, etc), below using an array
// for simplicity (built via a loop/comprehension)
deploys: [$.my_deploy(name) for name in $._config.demo.instances],
}
output
$ tk init
[...]
## NOTE: using https://kind.sigs.k8s.io/ local Kubernetes cluster
$ tk env set --server-from-context kind-kind environments/default
[... save main.jsonnet, config.jsonnet to ./environments/default/]
$ tk apply --dry-run=server environments/default
[...]
configmap/config-bar created (server dry run)
configmap/config-foo created (server dry run)
deployment.apps/deploy-bar created (server dry run)
deployment.apps/deploy-foo created (server dry run)

CannotPullContainerError: failed to extract layer

I'm trying to run a task on a windows container in fargate mode on aws
The container is a .net console application (Fullframework 4.5)
This is the task definition generated programmatically by SDK
var taskResponse = await ecsClient.RegisterTaskDefinitionAsync(new Amazon.ECS.Model.RegisterTaskDefinitionRequest()
{
RequiresCompatibilities = new List<string>() { "FARGATE" },
TaskRoleArn = TASK_ROLE_ARN,
ExecutionRoleArn = EXECUTION_ROLE_ARN,
Cpu = CONTAINER_CPU.ToString(),
Memory = CONTAINER_MEMORY.ToString(),
NetworkMode = NetworkMode.Awsvpc,
Family = "netfullframework45consoleapp-task-definition",
EphemeralStorage = new EphemeralStorage() { SizeInGiB = EPHEMERAL_STORAGE_SIZE_GIB },
ContainerDefinitions = new List<Amazon.ECS.Model.ContainerDefinition>()
{
new Amazon.ECS.Model.ContainerDefinition()
{
Name = "netfullframework45consoleapp-task-definition",
Image = "XXXXXXXXXX.dkr.ecr.eu-west-1.amazonaws.com/netfullframework45consoleapp:latest",
Cpu = CONTAINER_CPU,
Memory = CONTAINER_MEMORY,
Essential = true
//I REMOVED THE LOG DEFINITION TO SIMPLIFY THE PROBLEM
//,
//LogConfiguration = new Amazon.ECS.Model.LogConfiguration()
//{
// LogDriver = LogDriver.Awslogs,
// Options = new Dictionary<string, string>()
// {
// { "awslogs-create-group", "true"},
// { "awslogs-group", $"/ecs/{TASK_DEFINITION_NAME}" },
// { "awslogs-region", AWS_REGION },
// { "awslogs-stream-prefix", $"{TASK_DEFINITION_NAME}" }
// }
//}
}
}
});
these are the role policies contained used by the task AmazonECSTaskExecutionRolePolicy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "*"
}
]
}
i got this error when lunch the task
CannotPullContainerError: ref pull has been retried 1 time(s): failed to extract layer sha256:fe48cee89971abac42eedb9110b61867659df00fc5b0b90dd91d6e19f704d935: link /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/212/fs/Files/ProgramData/Microsoft/Event Viewer/Views/ServerRoles/RemoteDesktop.Events.xml /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/212/fs/Files/Windows/Microsoft.NET/assembly/GAC_64/Microsoft.Windows.ServerManager.RDSPlugin/v4.0_10.0.0.0__31bf3856ad364e35/RemoteDesktop.Events.xml: no such file or directory: unknown
some search drived me here:
https://aws.amazon.com/it/premiumsupport/knowledge-center/ecs-pull-container-api-error-ecr/
the point 1 says that if i run the task on the private subnet (like i'm doing) i need a NAT with related route to garantee the communication towards the ECR, but
note that in my infrastructure i've a VPC Endpoint to the ECR....
so the first question is: is a VPC Endpoint sufficent to garantee the comunication from the container to the container images registry(ECR)? or i need necessarily to implement what the point 1 say (NAT and route on the route table) or eventually run the task on a public subnet?
Can be the error related to the missing communication towards the ECR, or could be a missing policy problem?
Make sure your VPC endpoint is configured correctly. Note that
"Amazon ECS tasks hosted on Fargate using platform version 1.4.0 or later require both the com.amazonaws.region.ecr.dkr and com.amazonaws.region.ecr.api Amazon ECR VPC endpoints as well as the Amazon S3 gateway endpoint to take advantage of this feature."
See https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-endpoints.html for more information
In the first paragraph of the page I linked: "You don't need an internet gateway, a NAT device, or a virtual private gateway."

aws-ecs-patterns error: Cluster for this service needs Ec2 capacity. Call addXxxCapacity() on the cluster

Hoping someone can help me here, according to AWS CDK documentation if I declare my VPC then I shouldn't declare 'capacity', but when I run cdk synth I get the following error...
throw new Error(Validation failed with the following errors:\n ${errorList});
Error: Validation failed with the following errors:
[PrerenderInfrasctutureStack/preRenderApp/Service] Cluster for this service needs Ec2 capacity. Call addXxxCapacity() on the cluster.
here is my code...
(i hope Nathan Peck sees this)
const ec2 = require('#aws-cdk/aws-ec2');
const ecsPattern = require('#aws-cdk/aws-ecs-patterns');
const ecs = require('#aws-cdk/aws-ecs');
class PrerenderInfrasctutureStack extends cdk.Stack {
/**
*
* #param {cdk.Construct} scope
* #param {string} id
* #param {cdk.StackProps=} props
*/
constructor(scope, id, props) {
super(scope, id, props);
const myVPC = ec2.Vpc.fromLookup(this, 'publicVpc', {
vpcId:'vpc-xxx'
});
const preRenderApp = new ecsPattern.ApplicationLoadBalancedEc2Service(this, 'preRenderApp', {
vpcId: myVPC,
certificate: 'arn:aws:acm:ap-southeast-2:xxx:certificate/xxx', //becuase this is spcified, then the LB will automatically use HTTPS
domainName: 'my-dev.com.au.',
domainZone:'my-dev.com.au',
listenerPort: 443,
publicLoadBalancer: true,
memoryReservationMiB: 8,
cpu: 4096,
desiredCount: 1,
taskImageOptions:{
image: ecs.ContainerImage.fromRegistry('xxx.dkr.ecr.region.amazonaws.com/express-prerender-server'),
containerPort: 3000
},
});
}
}
module.exports = { PrerenderInfrasctutureStack }
This is because if you don't explicitly pass a cluster then it uses the default cluster that exists on your account. However the default cluster starts out with no EC2 capacity, since EC2 instances cost money when they run. You can use the empty default cluster with Fargate mode since Fargate does not require EC2 capacity, it just runs your container inside Fargate, but the default cluster won't work with EC2 mode until you add EC2 instances to the cluster.
The easy solution here is to switch to ApplicationLoadBalancedFargateService instead, because Fargate services run using Fargate capacity, so they don't require EC2 instances in the cluster. Alternatively you should define your own cluster using something like:
// Create an ECS cluster
const cluster = new ecs.Cluster(this, 'Cluster', {
vpc,
});
// Add capacity to it
cluster.addCapacity('DefaultAutoScalingGroupCapacity', {
instanceType: new ec2.InstanceType("t2.xlarge"),
desiredCapacity: 3,
});
Then pass that cluster as a property when creating the ApplicationLoadBalancedEc2Service
Hope this helps!