dockerhub provides automated build option on every new git push.
With automated build option,
we know, which image corresponds to which docker filethis is a important.
On git push of Dockerfile in GitHub repo or Bitbucket, build starts immediately.
1)
Does AWS ECR provide such automated build option? because I don't prefer the approach of running docker push through some automation
2)
if yes, can the automated build support docker-compose build?
ECR does not contain a build tool. But you can use AWS Codebuild AWS's continuous integration tool. I am pretty sure it supports both of your requirements.
Related
I want to create a cicd pipeline from github to aws eks.
Is there possible to create pipeline from GitHub to AWS EKS deployments on Git actions ?
Yes its possible you need to use some kind of CI/CD tool (Jenkins/Gitlab/AWS Native services) in between to automate this whole process.
Flow would be something like
Developer commit changes --> Trigger CI/CD pipeline --> Build Docker image --> Push it to ECR -- Deploy latest image to EKS using (Kubectl or Helm charts)
Please refer :
https://www.eksworkshop.com/intermediate/260_weave_flux/ this has example for end to end implementation.
https://www.weave.works/blog/gitops-with-github-actions-eks
https://aws.amazon.com/blogs/opensource/git-push-deploy-app-eks-gitkube/
I am trying to use AWS CodeBuild for building my code from github. These are the steps I followed so far,
1) Created a windows docker image with all the pre-req software
needed (git, npm, node.js etc) and pushed to Amazon ECS.
2)Created a project in AWS CodeBuild using
a) github as the source (What to build)
b) docker image created in Step 1 (How to build)
I setup buildspec.yml as below:
env:
#variables:
#parameter-store:
phases:
#install:
#pre_build:
build:
commands:
- git clone https://github.com/OrgName/RepName.git "c:\www\localfolder"
#post_build:
#artifacts:
#files:
But this is always failing during DOWNLOAD_SOURCE STEP saying "CodeBuild is experiencing Issues"
Please suggest how to setup buildspec.yml for github clone\fetch\checkout purpose.
Thanks.
The issue you encountered may not be related to git clone\fetch\checkout failure. The build could also fail at "DOWNLOAD_SOURCE" step if CodeBuild failed/timed out when pulling the Windows Docker image; especially when the image is large.
Workarounds you can try:
1) use the windows image provided by CodeBuild and install the pre-req software during the install phase. (you will need to update your buildspec.yml)
OR
2) use a BUILD_GENERAL1_LARGE instance. maybe you will need to increase the timeout too.
We are using GitHub as our source repository, AWS CodeBuild to compile the code from GitHub, Elastic Beanstalk to host environments and CodePipeline to trigger a build on commit and to deploy the code to different environments, with production being the final environment.
What I would like to add as a final step to CodePipeline is a merge back to master after a build has been deployed to production. I did a brief search on google but could not find any good references for how to initiate a git merge.
Does anybody have any experience with triggering a merge from CodePipeline?
Currently there isn't built-in support for merging.
Today most users run their pipeline on master, and merge into that before the code enters their pipeline. One advantage of this approach is that it ensures your pipeline is run on the exact merged version on mainline, rather than a pre-merge version.
However, we're aware that some workflows like a pull-request based workflow would benefit from being able to merge at the end of a pipeline.
The best workaround today is to use a Lambda function, custom action, or CodeBuild step to perform the merge.
In Gitlab it is now possible to automatically mirror remote GIT repo:
http://docs.gitlab.com/ee/workflow/repository_mirroring.html
Synchronization is either done manually or via gitlab cron script (running every hour).
I would like to sync in this way my github repo and run Gitlab CI jobs using my own runners.
Is is possible to automatize sync task, i.e. via Github webhooks ? Do you know if there is any other way to do it with Gitlab infrastructure ?
I would like to avoid hacking like:
- cloning github repo in gitlab runner
- running my own cron jobs which do sync more often
Mirror does work, but it's slow. If your goal is to run gitlab-ci for a github repository, good news, gitlab has released a new version which lets you use github.com repository with gitlab-ci:
https://about.gitlab.com/2018/03/22/gitlab-10-6-released/
GitLab CI/CD for GitHub feature a part of our GitLab.com Free tier
Instructions:
https://about.gitlab.com/2018/03/22/gitlab-10-6-released/#gitlab-cicd-for-external-repos
https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/
I'm wondering if there are any convenient ways to automate deployment of code to a live server in GO, either standard built-in methods, or otherwise.
I want something google app engine like, I just run the command and it uploads to the server and triggers a restart.
(Ultimately I want a git commit to trigger a rebuild and redeploy, but thats for down the track in the future)
I recommend Travis CI + Heroku.
You can deploy to heroku directly with just a git push, but I like to use Travis to build and run the tests before that.
There are some guides online but I'll try to go directly to the point:
What you will need?
Github account
Travis account (linked with github, free if open source)
Empty Heroku app (Free dyno works great)
Setup
In your github repo, create the following files:
.travis.yml (more info on the Travis CI documentation)
Procfile
.go-dir
After that go to your Travis account, add your repository and enabled the build for it.
Here is a sample minimal config file content (based on my app that I deploy to heroku):
.travis.yml
language: go
go:
- tip
deploy:
provider: heroku
buildpack: https://github.com/kr/heroku-buildpack-go.git
api_key:
secure: <your heroku api key encripted with travis encrypt>
on: master
Procfile
worker: your-app-binary
.go-dir
your-app-binary
Procfile and .go-dir are heroku configs so it can vary if you are deploying a web app, you can read more at the heroku documentation
One important and easily missed point is the build pack, without it the deploy will not work.
Read the Travis docs to see how to encrypt the heroku key
How it works?
Basically, every push to your repository will trigger the Travis CI build, if it passes it will deploy the app to heroku, so you set this up once and build + deploy is just a push away ;)
Also Travis will build and updated the status of all Pull Requests to your repository automagically.
To see my config and build, please take a look at my Travis build and my repository with my working configs