I currently have a VS2015 solution that comprises of 6 applications, Web site, DB, Web Service, Console Apps.
This solution is deployed multiple times to different servers internally. At the moment i have only 5 working instances. Potentially i could end up with 10 - 20 50 etc.
I'm struggling to understand how Release Management could help me deploy this solution n number of times. Is there something else that i could perhaps use to orchestrate the deployment?
A release definition is a collection of environments. An environment is a logical entity that represents where you want to deploy a release. You can add environment in build definition to achieve multiple deployments.
More information, check https://msdn.microsoft.com/en-us/library/vs/alm/release/overview
Related
I work with teams members to develop a microservices architecture but I have a problem with the way to work. Indeed, I have too many microservices and when I run them during my development, it consumes too memory even with a good workstation. So I use docker compose to build and execute my MSA but it takes a long time. One often hears about how technically build an MSA but never about the way to work efficiently to build it. How do you do in this case ? How do you work ? Do you use tools or any others to improve and facilitate your developments. I've heard about skaffold but I don't see what the difference is with docker compose or with a simple ci/cd in a cluster env for example. Feel free to give tips and your opinion. Thanks
I've had a fair amount of experience with microservices and local development and here's been some approaches I've seen:
Run all the things locally on docker or k8. If using k8, then a tool like skaffolding can make it easier to run and debug a service locally in the IDE but put it into your local k8 so that it can communicate with other k8 services. It works OK but running more than 4 or 5 full services locally in k8 or docker requires dedicating a substantial amount of CPU and memory.
Build mock versions of all your services. Use those locally and for integration tests. The mock services are intentionally much simpler and therefore easier to run lots of them locally. Obvious downside is that you have to build mock version of every service, and you can easily miss bugs that are caused by mock services not behaving like the real service. Record/replay tools like Hoveryfly can help in building mock services.
Give every developer their own Cloud environment. Run most services in the cloud but use a tool like Telepresence to swap locally running services in and out of the cloud cluster. This eliminates the problem of running too many services on a single machine but can be spendy to maintain separate cloud sandboxes for each developer. You also need a DevOps resource to help developers when their cloud sandbox gets out of whack.
Eliminate unnecessary microservice complexity and consolidate all your services into 1 or 2 monoliths. Enjoy being able to run everything locally as a single service. Accept the fact that a microservice architecture is overkill for most companies. Too many people choose a microservice architecture upfront before their needs demand it. Or they do it out of fear that they will need it in the future. Inevitably this leads to guessing how they should decompose the system into many microservices, and getting the boundaries and contracts wrong, which makes it just as hard or harder to fix in the future compared to a monolith. And they incur the costs of microservices years before they need to. Microservices make everything more costly and painful, from local development to deployment. For companies like Netflix and Amazon, it's necessary. For most of us, it's not.
I prefer option 4 if at all possible. Otherwise option 2 or 3 in that order. Option 1 should be avoided in my opinion but it is probably the option everyone tries first.
In GKE and assuming you have a private cluster. You can utilize port forwarding while hooked up to the GKE environment through the CLI. Create a script that forwards your local ports to the GKE environment. I believe on the services tab in your cluster is where you will find the "port-forwarding" button that will give you the CMD command. This way you can work on one microservice with all of its traffic being routed to the actual DEV cluster. This prevents you from having to run multiple projects at the same time.
I would say create a staging environment which will have all services running. This staging environment will specifically be curated for development. E.g. if it's deployed using k8s then you expose some ports using nodeport service if you need them for your specific microservice. And have a DevOps pipeline to always keep this environment up to date with the code.
This environment should always be built from master branch. If you have single repo for app or repo per service, it's fair assumption that the will always have most recent code when you create your dev/feature branch.
Then when you want to develop a feature or fix a bug you checkout your microservice. And if you are following the microservice pattern appropriately, that single microservice should be an executable and have it's own docker file and should be debuggable from your local IDE. Many enterprises follow this pattern, and enforce at the organization level that the master branch is always production ready and high quality.
Let's say, you discover a bug in some other microservice running in k8s cluster. You will very likely get tempted to find a way to debug that remote microservice. However, that should be written as a bug for the team that owns the microservice. If your team owns it then you fix it and then start working on your feature. If you really think you need to debug multiple microservices, then I think you have real tight coupling between the services or you don't really need the microservice architecture.
We are using mta to structure our application consisting of multiple micro-services.
The mtar file is deployed to SAP Cloud Foundry in blue-green mode using the SAP Cloud SDK pipeline (Piper Project -> cloudFoundryDeploy step).
We are running into the below issue while trying partial deployment with the blue-green strategy.
Issue: If MTA with same ID is redeployed, pipeline creates new MTA color (Blue to Green and vice-versa) and all applications with the old color are deleted. This approach creates issue during partial deployments.
For instance - using blue-green deployment approach, assume that we have deployed 10 micro-services to SAP CF space.
After bug fixes, we want to do partial redeployment for only a couple of micro-services.
In this case, if we use the same MTA ID and include only the two micro-services in mta.yaml, pipeline deletes the other 8 micro-services which are already deployed to SAP CF.
We tried updating the MTA version but it doesn't make any difference.
As a result of this issue, we unnecessarily need to redeploy all the 10 micro-services again although fix was done for only a couple of micro-services.
On the flip side, if we use a different MTA ID for redeploying the two micro-services, pipeline treats those as initial deployment and triggers an action which is same as the standard deployment type. This results in downtime and defies the purpose of blue-green deployment approach.
Appreciate if someone can help us here to resolve this issue of partial deployment using blue-green strategy and guide us on the best practices to follow.
I don't think this is currently possible, and the right place to build this would probably be multiapps-cli-plugin. I think it would be best if you opened an issue in that repo.
On another note: With that requirement of "partial deployments" I'm not sure if MTA is the best choice. Is there a reason why you need MTA? If you had one maven project per microservice, with one pipeline each this would be closer to the general concept of microservices where each unit can be deployed independently.
Hope this helps
Florian
This post is not for a specific coding issue. Rather, what would be the overall, best practice file structure in a Visual Studio solution of microservices in Azure Service Fabric as well as test and debug?
This may be an obvious question with simple answer. However, I couldn't find an explicit answer in MSDNs or other articles.
Two questions which are also tied together:
I currently have a couple of Visual Studio solutions with each solution containing multiple projects (services). My concern is how decoupled these services are in terms of publish, deployment and versioning in the Service Fabric environment. I hope I am wrong in thinking that these services are now tied together in terms of deployments and versioning. Hopefully each individual service can be deployed and published independently onto Service Fabric.
A lot of times these multiple services in multiple Visual Studio solutions work in chain in achieving end to end business flow. I literally have to open multiple Visual Studio instances to debug and test. I almost thought of merging them into a single solution to make debugging more convenient. I wish there was a more effective and convenient way to test and debug without having to combine them all into single solution or running 10 Visual Studio solutions.
Yes, services are individually versioned. Even the elements that make up a service are versioned (code, data, config). Everything can be updated independently.
When you deploy an application to a (dev) cluster, you don't have to delete it after debugging. You can just leave it running. This way, you can debug a single service, while the others are running without a debugger.
I have introduced branching/merging to my team and have talked before about how it would be great to automatically build and deploy code checked into the staging/master branches, but I'm a junior dev, not very ops-y.
The trouble I'm having, is that we create intranet applications and store them on our own VM's which we have access to, but we also have load balancing which is causing me grief!
I can get a build to automate (well, I haven't got all the bugs figured out but I'm working my way through them) - and I can even get the build to automatically create a zip file ready for deployment.
Is it possible to configure several servers for deployment?
I.E
1) I check in some code to stage
***Automatically***
2) Code builds
3) Build completes, Unit tests run and they complete
4) Code is packaged into a .zip
5) .Zip is deployed across the three load balancing servers (all with the same file path).
***
Maybe worth noting we currently have our TFS server running Visual Studio so the code is built on the same server it is all stored, but this is not the server we run live code from.
Any help or tutorials specific to my setup would be GREATLY appreciated, I really want to turn this departments releasing strategies around!
I am going to address only the deployment aspect. There are a lot of different ways that this can be handled, such as:
Customizing the build template
Writing custom .Net code and inserting it into the build template (which would also involve customizing the template)
Creating a Batch or Powershell script set to run after the build completes
Using a separate tool such as OctoDeploy or Release Manager to handle the deployments
The first thing you need to do is separate the build and deployment steps in your head. While they are tightly coupled in your model, they are two totally different tasks that need to be handled different ways.
The second thing is to stop thinking like a developer when it comes to the deployment portion. While there will likely be a programmatic solution, you'll need to identify the manual steps first.
You stated that you're not very ops-y, by which I assume you mean you're more Developer and not Systems Analyst. If that is the case, then the third thing you'll need to do is get someone who is involved, such as your current release team.
There are 3 major things that need to be done then:
EVERYTHING needs to be standardized. If you can't standardize something, then standardize the way that it's non-standard (example: You have a bulk list of servers you need to deploy to, and you need to figure out which ones to deploy to based on their name, which can be anything. In that case, a rule needs to be put in place that all QA servers need to have QA in their name, User Acceptance servers need UAT, Production need PROD, etc.).
Figure out how you're going to communicate from the build to the deployment, which builds are going to deployed, to which servers, and where the code is going to be picked up from
You need to document every manual step, and every exception to those steps, and every exception to those exceptions.
Once you have all those pieces in place, you need to then go through each manual step and automate it, whether that's through Batch, Powershell, or a custom-built application. Once you have all the steps automated, you'll have both the build and deploy pieces complete.
After you're able to execute a single "manual" automatic deployment to a single environment, you're then ready to figure out how you want to run it for multiple environments. This can be as complex as an XML file that is iterated through, to simply calling the same command multiple times with different parameters.
A quick summary of how I've done this at my current job (where using a third-party deployment tool was not an option):
Created a tool using .Net WinForms to allow us to "manually" run automated builds (We use the interface to determine the input parameters, and the custom classes under the hood do all the heavy lifting. These custom classes are in a separate project that builds to their own dll. This also allows us to test tweaks and changes to the process in a testing environment before we roll it out to our production build server)
Set up an XML file for each set of environment (QA, UAT, Prod, etc.) that contains all of the servers that need to be deployed to in that environment, including destination paths, scheduled tasks, and Windows Services
Customize the TFS build template and include the custom classes created for the custom tool, which will read the XML file and iterate through each server entry to perform the deployments
I'm more than happy to help with more specific examples and assistance, I look at things a bit different than most people and it helps when it comes to release management.
I have a question regarding azure deploys. My specific scenario is that I would like to deploy N projects on the same WebRole / WebSite.
The solution (don't think only as a visual studio way of organizing projects) will involve 1 web service (that could be scaled-out later) and N web sites (that also could be scaled-out later)
Because right now my traffic is very low I was planning to deploy the web app on port 80 (of course) and the web services on say port 5000 of the same instance to save some money.
What i can't figure out right now is how to do this... in the end i have 2 questions:
-1: Can I deploy multiple projects in a single instance? (this question is similar but is not straighfoward... I was hoping for a more direct answer...)
-2: How?
EDIT: Don't get this question wrong! I do not want to create a azure web role project in VS 2012 and "Add" the 2 minor projects... i want to create 2 projects completaly independent... 2 separated solutions... separated deploys (ex: I want to correct a bug in the service without updating the web-site!)
Am I being clear? if not just say and I will re-explain...
Based on the edits to your question, the short answer is no, you can't have two completely different solutions that deploy to the same instance without overwriting the other. For Azure one role is the smallest unit of deployment.
You might be able to write your own auto updating code, but you'd be on your own there, there's nothing baked into the service to help you with this.