Currently, I have two separate Bitbucket repositories for Frontend / backend respectively. Each repo has its own Bitbucket Pipelines configuration which works independently from each other. Sometimes, we run into a situation when a frontend is broken because there are no corresponding backend changes deployed or vice versa - backend deployed something not yet supported by frontend.
I have an idea on how to solve it - triggering backend deployment from frontend merging related backed pull requests with the help of merge checks to prevent merges if backend is not ready yet.
Probably, there are other strategies exist or I might try some tool out to achieve my goal.
Please advise on this topic
Related
I have a database-driven application with a single code base configured for multiple clients using the database setting and config files.
The main code base consists of common/core code/files that are being used by all the clients and some client-specific code/files. Both types of files are in different folders of the same repository.
We have been planning to integrate CI/CD using GitHub and Jenkins. I am new to Jenkins.
In GitHub, we have a single repository that contains all the code/files. I want to use Jenkins to deploy to different client environments but make sure that only files related to a specific client should be deployed to that client environment.
What could be the best way or possible solutions for this?
Edit: Basically I want to deploy specific files that are client related to specific client environments.
Any and all suggestions would be highly appreciated.
During development a monolithic component (one project in one repo) is actually quite useful:
Especially in the early stages, renaming classes and methods is easy.
Also integration tests can be written, since all services are available inside the same monolith.
But also during development there is no need to start other services. Just start the monolith and everything works end2end.
Dependency management can be done in one place.
However, when deploying, micro-services are preferable. That's why my question is, whether there are any examples or best practices for automatically splitting a monolith into micro-services, but only for deployment.
A few ideas that come to mind:
Deploying the same monolith image for several micro-services. So basically pod1, pod2 and pod3 have the same image, but each pod has its own service1, service2 and service3 (kubernetes resource "Service"). Via a configMap the monolith image for pod1 for example is configured to only serve requests for service1. Same respectively for pod2 and pod3
For REST services and Ingress configuration could allow only service1 requests on pod1.
Using the CI/CD pipeline to build several different docker images from the same monolith repo. Effectively removing those parts that are not necessary. E.g. image1 only contains service1-implementation, but service2-implementation and service3-implementation have been removed. Already during build process several different jars (for a Java application) could be build.
I'd like to know, if anyone has experience with such an approach or sees clear counter-arguments to work like this.
Thanks.
For the MicroService Architecture based application, I'm trying to understand a standard process about how to logically group and manage correct version compatibility among independently deployable microservices. Let me elaborate with practical scenario :
Say, I am building a software application which is composed of 10 microservices. All the microservices have their independent repositories(branching workflow etc.) and their separate CI/CD Pipeline.
The CI/CD Pipeline gets triggered whenever any change pushed to 'master' branch for respective microservice.
Considering Helm chart and Kubernetes based deployment, all the microservices will get deployed with version 1.0 for the very first deployment and our system would work. For subsequent releases, we might have only couple of services that will get deploy. So after couple of production releases, each microservice will be at different version to constituent an application at that point of time.
My question is :
How to logically group independently deployable microservices in order to deploy or rollback to earlier release i.e. how to determine what was the version of different microservices for earlier releases?
Is there any existing tool or standard practice to track versions of each microservice for given release to seamlessly rollback to expected release?
If not automated solution, what would be the right approach to address such requirement?
Appreciate your thoughts and suggestion on this.
With consideration kuberenets:
1. Helm is nice tool to deploy and track.
2. Native k8s deployment works nice, you need to use deployment properly especially look --record flag in k8s commands eg check this link
With AWS ECS clusters:
1. they have task definations and tasks. I think that works for you.
Not have pointers for docker-compose, swarm, and other tools. But you can always use the power of git and some scripting.
the idea is make a file that lists all versions of services/containers/code . and commit that file in git with code. Make tag out of it for simplicity. your script should compare this state file and current state and apply specific changes only. Look at git submodules also. it is nothing but a group of many git projects and it tracks status of each project with help of commit id of each project. This helped us in the situation you mention.
This is a fairly new problem, we just launched a new tool Reliza Hub to solve that. Also here is my post on the subject: Microservices – Combinatorial Explosion of Versions. Currently, we are at the MVP stage and a lot of work is going on - see this video tutorial if our direction makes sense for you https://www.youtube.com/watch?v=yDlf5fMBGuI
If you decide to implement and have any questions or need help with integration, just tag me on SO and I'd be very much willing to make it work for you.
To sum up few things that we are doing - we denote developer facing projects (those that map to source code) as Projects and customer facing projects (bundles that customer sees) as Products.
And we say that Products are essentially composition of Projects and provide tooling how you can compile different versions of Projects into what's called a Product bundle. You can then integrate this into any CI or CD tool out there or start manually if you haven't configured CICD yet.
Other than that, yes - I highly recommend helm and kubernetes - this is what we use on newer projects. (And I can also add ArgoCD and Spinnaker to the existing tooling). But it is not enough to track permutations of different versions of microservices and establishing which configurations are good and which are not between different environments.
I have been playing with Spring Cloud Configuration. I like the simplicity of the solution and the fact that it uses git as it's default configuration store.
There are two aspects I need to figure out before pushing it as a solution for centralized configuration management.
The aspects are:
High availability
How to gradually roll out configuration changes (to support canary releases)
If you already implemented this in your data center or just playing with that please share your ideas!
Also I would like to hear from the creators, how they see the recommended deployment in single/cross data-center environments.
The Config Server itself is stateless, so you can spin up as many as these as you need and find them via eureka. Underneath the server itself, the git implementation you point to needs to be highly available as well. So if you point to github (private or public), then git is as available as github is. If the config server can't reach git it will continue to serve what it has checked out even if it is stale.
As far as gradual config changes, you could use a different branch and configure the canary to use that branch via spring.cloud.config.label and them merge the branch. You could also use profiles (eg application-<profilename>.properties) and configure the canary to use the specified profile.
I think the branch makes a little more sense, because you wouldn't have to reconfigure the non-canary nodes to use the new profile each time, just configure canary to use the branch.
Either way, the only time apps see config chages (when using spring cloud config client) is on startup or when you POST to /refresh on each node. You can also POST to /bus/refresh?destination=<servicename> if you use the Spring Cloud Bus to refresh all instances of a service at once.
We're currently doing continuous deployment to our dev/qa servers, and manually triggered automated deployment to our production boxes. Currently we're using TeamCity/PowerShell/MsDeploy. We now have a requirement to deploy to a server that sits on an external network, on which the target server cannot be accessed externally. Instead, it will have to "call home" for updates - and presumably then push the results back if it succeeds or not.
I'm thinking we could write a service that requests a particular URL on our build server with delivers the artifacts that would have been used for deployment, pull that down - and then fire off the build script.
However, I'm not entirely sure how we'd deal with updating the updater, and failures when they occur. Does anyone have any recommendations on how to approach this?
Sounds like you need a release repository. The build server pushes files into it and each deploy job pulls from it. This would neatly decouple the two processes.
A release repository could be as simple as a shared NAS, or something more sophisticated such as the Nexus repository manager.