We are running out of space on our D:\Temporary Storage drive on our Service Fabric Cluster VM's (5 nodes). I have conversed back and forth with MS support about what is safe to delete from this drive and the answers I'm getting are ambiguous at best.
I've noted that we have many older versions of our applications and services on the VM's that we don't need anymore. Getting rid of these will definitely help free up space. I've asked MS support if it's safe to delete the old versions of the applications and they said yes, but then directed me to these links:
https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-deploy-remove-applications#remove-an-application-package-from-the-image-store
https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-deploy-remove-applications#remove-an-application
https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-deploy-remove-applications#unregister-an-application-type
So the three sections we have are:
Remove an application package from the image store
Remove an application
Unregister an application type
These all deal with PowerShell scripts that need to be run, which I am very novice with. I have direct RDP access to the VM's and have the ability to simply delete the files via Windows File Explorer. Is it ok to do it this way, or do I need to go the Powershell route for deletion and Unregistering the application? At least for #1, removing the application package from the image store, there shouldn't be any issue with me just deleting that from Windows File Explorer in the VM's, correct?
EDIT: this is not a duplicate of Run out of storage on Service Fabric scale set :
I am asking about manually clearing space on the SFC VM's - the above thread is talking about setting up your application deployment to auto-delete old versions of applications. These are not duplicates.
You shouldn't delete manually from within the VM, SF should handle it and you may cause issues.
The right way to remove them is doing like the documentation says, using powershell like:.
Remove-ServiceFabricApplicationPackage -ApplicationPackagePathInImageStore MyApplicationV1
You can also remove it manually via Service Fabric Explorer:
The option in the left will try to delete all the Application Package Versions registered in the cluster(if none in use)
The one in the right will delete a specific version (if not in use)
Keep in mind that to remove a package you should remove any running application that is using the same package version.
The other option is deleting the old version when you deploy a new one. I will link you to this other SO question: Run out of storage on Service Fabric scale set
Related
The concrete question
For those who just want the direct questions:
Is there a way to temporarily disable default services on a ServiceFabric application type so that a new application can be installed (using Powershell) without automatically installing any default services?
A proposed solution here is to remove the default services from the manifest and later restoring them. I am able to write a PowerShell script to adjust the application manifest accordingly, but how do I update the application type using Powershell - assuming I already have altered the manifest?
Any solution that solves the contextual problem without requiring manual config meddling is acceptable - my proposed solution is probably not the only possible solution. We do explicitly want to avoid manual meddling.
When allowing meddling, we are already able to just comment out the default services when we need to. We're specifically looking for a solution that requires no meddling as this reduces bugs and debugging issues.
The context
I'm running into an issue with using the application manifest's default services during local development.
I am aware of the general "don't use default services" advice, and it is being followed. During CI build, the default services are removed and will not be relied upon for any of our clusters in Azure. The only exception here is local developer machines, which use default services to keep the developer F5 experience nicer by enabling all services when starting a debug session.
We have written specialized scripts that provision a new tenant (SF application) with their own set of services (SF service). Not every tenant should get every service, we want to opt-in to the services, which is what the script already does (based on a mapping that we manage elsewhere, which is not part of the current question as the provisioning script exists and works).
However, when default services are enabled, every tenant already gets every service and the actual opt-in provisioning is useless. This is the issue we're trying to fix.
This same script works in our production cluster since there are no default services configured there. The question is solely focus on the local development environment.
Essentially, we're dealing with two scenarios during local development:
When debugging, we want the default services to be on because it allows us to run all of our services by pressing F5 (and not requiring any further action)
When testing our provisioning script, we don't want default services because it gets in the way of our selective provisioning behavior
I'm aware that commenting the default services out of the manifest solves the issue, but this requires developers constantly toggling the content of the manifest and reinstalling the application type, which we'd like to avoid.
Ideally, we want to have the default services in the manifest (as is currently the case) but then have the provisioning script "disable" the default services for its own runtime (and restore the default services before exiting), as this gets us the desired behavior in both cases.
What is the solution that requires the least manual developer meddling to get the desired behavior in both scenarios?
I'm currently trying to implement it so that the provisioning script:
Copies the application manifest to a backup location
Removes the default services from the real manifest
Updates the application type using the new manifest (i.e. without default services)
Runs the provisioning logic
Restores the real manifest using the backup manifest from step 1
Updates the application type using the restored manifest (i.e. with default services)
It is specifically steps 3 and 6 that I do not know how to implement.
Consider having two sfproj projects in the solution. One with default services, one without.
Also look into using a start-service.ps1 script instead of default services. This way the two projects can use the same application manifest.
https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-debugging-your-application#running-a-script-as-part-of-debugging
I am using a library which searches in registry for a dll. That dll can be installed by running MSI in the Service Fabric cluster and this path will be set.
But I wanted to avoid the installation of MSI in the cluster, and provided the required dlls in the package itself. During start up of the service, I am creating the registry entry and giving the location of the dll in my package. Everything is working as expected.
Is this approach ideal? Are we allowed to make changes to registry? If not, how do we solve this problem? Any pointers are appreciated.
If the library has to use the registry, there is nothing you can do about it other than register the values. If you could change the DLL to retrieve this information from the configuration file would be the ideal solution.
You can do it in SF, the right way to do it is using the SetupEntryPoint option of the ServiceManifest to do these management tasks, and from the Application manifest you can set the policies to specify which user you should run these policies. it is described here with more details
The main issue you have on SF with this approach is that you application might move around the cluster and you have to register it on every node, and maybe also remove it when the application is not running there anymore to avoid garbage in the registry.
We have a c#, .NET 4.0, windows application which we deploy to a terminal server. (Developed using VS 2010). This application makes use of several WCF services sitting on another server.
Our users access the front-end via remote desktop session. (They all have a .RDP file on their desktops.)
My question is regarding the deployment of this front-end. Currently, if we need to do an emergency deployment during business hours, we need to kick all the users off that are hooked into the app (as they are using the dll's that we need to replace). This is not ideal, obviously. We work in quite a business-critical environment, so these deployments are unavoidable. I've investigated ClickOnce, but have read that you cannot use this with terminal services application here. (Which kind of makes sense since it's essentially one app being "accessed" by several clients...)
I would like to be able to do a "silent" deployment whereby the user knows nothing about the fix until they restart their instance of the application. I'm not sure this is even possible?
I would appreciate any guidance or suggestions on this!
Yep, I do this all the time with a RD app -- you just need to move or rename the DLLs instead of deleting them. Windows allows moves and renames when DLLs are in use, but prevents you from deleting them. If you use Windows Installer to deploy your app, it will do the moves automatically (and delete the old versions when the system is next rebooted).
Once you replace the DLLs this way, existing sessions will continue to use the old, renamed versions, and new sessions will use the new versions. Of course, depending on how many DLLs you have, how long it takes your app to load them into memory, and how much activity you have on your server, you could run into a scenario where the app loads some of the old DLLs and some of the new ones when you're in the middle of updating them, but that would likely be rare.
We have WCF services deployed in azure cloud and runnig. We have some changes in some dlls and want to update in VM but dont want to go through regular deployment/redeployment process.
We are thinking of manually coping dlls to approot and siteroot folders. Will it work?
Will it pick up new dlls when VM restart anytime in future?
To answer your questions
Will manually copying dlls to approot and sitesroot folders work: Yes (make sure you do this on each instance if you have multiple instances running)
Will these dlls survive a reboot: Yes (see Reboot Role Instance: ... Any data that is written to the local disk is persisted across reboots. ...)
But I would suggest to only do this if you're planning to test some things while developing your service.
Do NOT plan to use this for production deployments, because if something goes wrong with your instance, the Fabric Controller might decide to destroy that instance and deploy a new one (same could apply for Windows Updates). This new instance would go back to the initial state of your deployment (the content of the cspkg you deployed).
To make your development deployments even easier you could also activate WebDeploy on your Web Role to deploy from Visual Studio: Enabling Web Deploy for Windows Azure Web Roles with Visual Studio (again, do not use this for real deployments, this is only for when you're testing out some things).
Note: Web Deploy will not work with multiple instances.
No,
And this is not the way to go. If you want to be more dynamic, you have to take the approach of Windows Azure Accelerator for WebRoles. Although not anymore supported and developed project, it will give you a good foundation of dynamically loading assemblies (in this case entire sites) from Blob storage.
I'm looking for best practice in continuous delivery of windows services.
Currently we hava a set of powershell scripts that unintall, reboot, install updates but error handling is tricky. We are reviewing System center but are there any other options available for deploying a windows service?
We've been using Presto since Dec 2011, and have done over 1,000 deployments. Most of what we deploy are Windows services.
What's nice is that we set up our apps and servers in Presto, then we can repeatedly deploy, to any server (or multiple servers at once), by just hitting a button. Presto will copy our official release binaries, update all of the items in our app config files, create and start the service, etc...
So, if you have an application that has 30 manual steps to deploying it, you can enter these steps in Presto, then it's done automatically for you after that.
It's worth a look: http://presto.codeplex.com/
Your most basic and generally accepted best option comes from this thread, which basically links to a Microsoft support article on creating an installer for the windows service.