I am using gcp and kubernetes.
I have gcp repository and container registry.
I have a trigger for build container after pushing into the master branch.
I don't know how to set some auto-trigger to deploy new version of the container (docker file).
How can I automate the build process?
You need some extra pieces to do it, for example if you use Helm to package your deployment you can use Flux to trigger the automated deployment.
https://helm.sh/
https://fluxcd.github.io/flux/
There are two solutions here.
You can expand the build step. Cloud Build can also push changes to your GKE cluster. You can read more about this here
What you currently have is a solid CI pipeline, for the CD, you can use Spinnaker for GCP, which was released recently. This integrates well with GCE, GKE and GAE and allows you to automate the CD portion.
Related
I am trying to install kubeflow pipelines (KFP) for kubeflow on AWS, as shown here. I am using an overlay for some simple labeling and other cosmetic changes. Installing KFP in the way shown in the documentation will also deploy instances of argo and other necessary services. I already have an instance of argo running on the cluster, so how can I point KFP at that installation of argo instead of deploying a duplicate instance?
I am trying to understand the Gitlab K8s agent. But it looks like it requires the developer to commit changes to a manifest file before it can deploy them to K8s. This is a challenge when trying to do auto deploys using Gitlab pipelines because those pipelines run after the commit. So how is the user supposed to create a new commit in an automated way that the Gitlab K8s agent can pick up?
I am wondering if anyone is using Gitlab and their K8s agent for auto deploying to K8s. Would really appreciate if you could throw some light on this.
My situation is as follows:
have a kubernetes cluster with a couple of nodes
have argocd installed on the cluster and working great
using gitlab for my repo and build pipelines
have another repo for storing my helm charts
have docker images being built in gitlab and pushed to my gitlab registry
have argocd able to point to my helm chart repo and sync the helm chart with my k8s cluster
have helm chart archive files pushed to my gitlab repo
While this is a decent setup, it's not ideal.
The first problem i faced with using a helm chart git repo is that I can't (or don't know) how to differentiate my staging environment with my production environment. Since I have a dev environment and prod environment in my cluster, argocd syncs both environments with the helm chart repo. I could get around this with separate charts for each environment but that isn't a valid solution.
The second problem i faced, while trying to get around the above problem, is that I can't get argocd to pull helm charts from a gitlab oci registry. I made it so that my build pipeline pushed the helm chart archive file to my gitlab container registry with the tag dev-latest or prod-latest, which is great, just what I want. The problem is that argocd, as far as I can tell, can't pull from gitlab's container registry.
How do I go about getting my pipeline automated with gitlab as my repo and build pipeline, helm for packaging my application, and argocd for syncing my helm application with my k8s cluster?
is that I can't get argocd to pull helm charts from a gitlab oci registry.
You might be interested by the latest Jul. 2021 GitLab 14.1:
Build, publish, and share Helm charts
Helm defines a chart as a Helm package that contains all of the resource definitions necessary to run an application, tool, or service inside of a Kubernetes cluster.
For organizations that create and manage their own Helm charts, it’s important to have a central repository to collect and share them.
GitLab already supports a variety of other package manager formats.
Why not also support Helm? That’s what community member and MVP from the 14.0 milestone Mathieu Parent asked several months ago before breaking ground on the new GitLab Helm chart registry. The collaboration between the community and GitLab is part of our dual flywheel strategy and one of the reasons I love working at GitLab. Chapeau Mathieu!
Now you can use your GitLab project to publish and share packaged Helm charts.
Simply add your project as a remote, authenticating with a personal access, deploy, or CI/CD job token.
Once that’s done you can use the Helm client or GitLab CI/CD to manage your Helm charts.
You can also download the charts using the API or the user interface.
What’s next? First, we’d like to present additional metadata for charts.
Then we’ll start dogfooding the feature by using it as a replacement for https://charts.gitlab.io/.
So, try out the feature and let us know how it goes by commenting in the epic GitLab-#6366.
See Documentation and issue.
I want to create a cicd pipeline from github to aws eks.
Is there possible to create pipeline from GitHub to AWS EKS deployments on Git actions ?
Yes its possible you need to use some kind of CI/CD tool (Jenkins/Gitlab/AWS Native services) in between to automate this whole process.
Flow would be something like
Developer commit changes --> Trigger CI/CD pipeline --> Build Docker image --> Push it to ECR -- Deploy latest image to EKS using (Kubectl or Helm charts)
Please refer :
https://www.eksworkshop.com/intermediate/260_weave_flux/ this has example for end to end implementation.
https://www.weave.works/blog/gitops-with-github-actions-eks
https://aws.amazon.com/blogs/opensource/git-push-deploy-app-eks-gitkube/
In my organization we use IBM Urban code to deploy docker images to Kubernetes. Deploying using Urban code is not easy and process is not transparent. Sometimes output of UC is confusing to release management. Are there any better tools used by the industry to deploy docker application in kubernetes, docker EE platform?
I can share how we are doing it in our start-up.
We've built our own pipeline around Jenkins and Google Kubernetes Engine. There are not that many steps involved:
Create a tag of your built image(s): docker tag <source_image> <target_image>
Push image(s) to the Google Container Registry: gcloud docker -- push <target_image>
Change yaml file definitions to select new <target_image>
Update K8s configuration: kubectl apply -f <yaml_file>
Of course in real life this is a little more complex and automatically updates tons of microservices but you get the gist.
Because you asked for tools, there are lots of solutions out there to help you, please have a look at this list to get an overview. It all pretty much depends what kind of environment you want to use it in. Some prominent examples are:
Werker
Codefresh
Spinnaker
KubeCI
You can use below tools for deploying docker apps to kubernetes
Jenkins with kubernetes CD plugin
https://github.com/jenkinsci/kubernetes-cd-plugin
Spinnaker