I am setting up a pipeline with an Amazon ECR source to ECS deploy. Have been following the steps in the tutorial here.
My issue is when my private ECR is updated with a docker image the pipeline is not triggered. I am not applying the latest tag on the image, just using a semantic versioning tag which includes a build number and a short Git commit hash, for eg:
myserver:b21-6d22b379a
myserver:b20-c90b134a
etc..
In the Image Tag option in the ECR source action it says: Choose the image tag that triggers your pipeline when a change occurs in the image repository.
If I leave it blank and just specify the ECR repository name such as myserver, will it look for a new image only if the latest tag is moved to another image with a different SHS digest in ECR?
Or is it smart enough to detect the change in ECR based on the timestamp + SHA digest of a new image even if the image did not have the latest tag applied?
I want to avoid using the latest image tag, as with an ECS Fargate cluster my understanding is a new container will simply pull the latest tag irrespective of if CodeDeploy has published a new task def with a new image tag.
So how does one specify the Image & Tag in the ECR source action if not using the latest tag on the docker image in ECR? Does it require a fixed tag to be used for the auto deploy from ECR to ECS to work?
As per https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-ECR.html#action-reference-ECR-config :
If a value for ImageTag is not specified, the value defaults to latest
So if you leave "ImageTag" empty, only images with the "latest" tag will get deployed.
And yes it requires a fixed tag for deployment to work. As such the use of tags is limited - you can, for example, use tags to ensure a specific image only gets deployed in a specific environment (e.g. images tagged "staging" get deployed in staging environment, and images tagged "production" get deployed in production environment) - but you can't use tags dynamically.
Related
Backstory:
We have a web app that creates batch jobs in Azure using docker images. In the application configuration there is a parameter to defines which version of the docker image the batch job should use. In our current setup we need to manually change the parameter if we deploy a new version of the docker image.
What I want to do is choose which docker image to use when I create a release for the web app. I already have a working release pipeline where I manually type in which version of the docker image I want to use, but I would like to be able to choose from the available docker images in the repository. The docker images are built in Azure devops and we have a tag on each build with the version number.
Is it possible to achieve this?
Synopsis
My overarching objective is to automate the build and publish of a container image to the GitHub Packages registry (and/or DockerHub). I have already set up a project that accomplishes this, however there's a "gotcha."
I want to use output from the container image build process to update the tag that's assigned to the container image.
For example:
ghcr.io/pcgeek86/aws-powershell:<versionNumber>
The Caveat
Unfortunately, the GitHub Action for Docker does a build and a push simultaneously. Hence, I am unable to capture the output from the container image build, parse the version number, and then update the environment variable containing the new tag value.
Question
Is there a way to separate the container 1) build and 2) push into separate steps, so that I can capture the build output and use that to modify the container image tag, before it's pushed up to the GitHub Packages registry?
I have a Kubernetes cron job that runs off the latest image of my repo. In the yaml definition of the cron job the image definition is like this:
image: 123456789.abc.ecr.us-west-1.amazonaws.com/company/myservice:latest
The issue is our latest tag gets updated anytime there is a merge to master in GitHub. However, the code is not deployed to prod at this point. There is no continuous deployment though so our code is not deployed in prod, however the latest code from master will be used by the cron job.
Does it work as a solution to tag a 'prod' image that the cron job will run from? In this case when we deploy, the prod image would get updated to that image. Also, if there is a rollback we would update the prod image to be the image that prod is now reverting to.
Or is there a better solution/pattern to this issue?
You can create git release tags or use commit ids. Put this value in config map, and the cron job container should pull code by using this commit-id or release tag. So staging and production configs can have a different tags.
I would not recommend using latest tag for deployments on kubernetes. It works well with docker but not with kubernetes + docker. Create immutable image tags.
Kuberentes compares only tag, not tag + SHA.
https://discuss.kubernetes.io/t/use-latest-image-tag-to-update-a-deployment/2929
Or you can use imagePullPolicy: always with latest tag.
Generally you should set up two Git repos of your application, for example master and prod. You may then use Jenkins to build and deploy those repose to Kubernetes on commit (prod branch can be set to build on demand). So master branch will push to latest tag, and prod branch will push to prod branch. Then you specify tag latest in your job manifest. That's how it is configured in most cases.
Any questions?
I cannot for the life of me figure out how to associate an image from the elastic container registry with an elastic beanstalk environment. I'm modifying a project that used to upload the image to S3, and I'm changing it so that it stores the image in ECR instead.
The registry for the image and the EB environment are both currently being set up with terraform. Previously the S3 bucket for the image was set in the docker settings of my build.sbt file, but I cannot find an equivalent for stating the name of the ECR repository. I am able to successfully deploy my docker image to ECR using the sbt-ecr plugin, but I cannot find anything that would then let elastic beanstalk know where this image is.
This is my first time using EB so I might just be misunderstanding how it works, but I honestly can't see any link between the code and EB. The deployment pipeline previously just ran
sbt realease with-defaults
and then
aws elasticbeanstalk update-environment --environment-name <env-name> --version-label <version>
How am I supposed to tell it to look for the image in the ECR repository? I don't even understand how it was working before, let alone how to get it to work now. All of the EB plugins I've found have just deployed the local image to EB rather than using something deployed somewhere. When I google the general issue I keep seeing things telling me to add stuff to my Dockkerrun.aws.json file, but I guess that's something that sbt has previously been building for me as I've never seen it before.
Any help would be greatly appreciated.
Trying to familiarize myself with ECR - i understand that you should be able to push as many repositories as you like to Docker registry.
However, ECR has a concept of a 'repository'.
So if i have 10 different containers comprising my app, does it mean i need 10 repositories in a registry?
Can i have one repository and push 10 different containers with their 'latest' tags?
Currently if i tag another image with the same {registry}/{repository_name} pattern, it replaces latest tag on my other image:
If you want to get the detailed descriptions, I would check out the What Is Amazon EC2 Container Registry? page, which describes the components in detail, but the high level difference between the two is this: Each account has a Registry, each Registry can contain several repositories. Each Repository can contain several Images. An image can have Several Tags, a Tag can only exist once per Repository.
If you look at the reference to a repository:
[account].dkr.ecr.[region].amazonaws.com/[repository_name]
The part in front of the first / is your registry, the part after the first / is your repository.
So what you're experiencing here is that by pushing a second image to the same repository, you're changing the reference that the latest tag is pointing to.
If you want to have multiple distinct images, each with their own latest tag, each one should have it's own Repository. Based on the Pricing for ECR, you only pay based on the storage size and transfer from an ECR Repo, so there's little benefit to not creating additional repos.