Slow upgrade deployment of guest executable in Service Fabric - azure-service-fabric

I have guest executable (real .net core 1.1 console app) and first deployment take few minutes only. Upgrade deployments take 12+ minutes. There are 3 service instances on 5 node cluster. SF is for developers and there is no workload.
Guest executable is start.cmd
<EntryPoint>
<ExeHost>
<Program>start.cmd</Program>
<Arguments />
<WorkingFolder>CodePackage</WorkingFolder>
</ExeHost>
</EntryPoint>
Start.cmd is really simple
start /b /wait "SomeWebAPI" "C:\Program Files\dotnet\dotnet.exe" SomeWebAPI.dll
Appreciate any recommendation to fix slow upgrades.

That's true that there are cases when upgrade could take longer than the initial deployment. It's happening because SF applies a very robust workflow to ensure that everything works after making an update on a particular upgrade domain. It conducts a number of health checks depending on the health policies, and those checks happen within particular time intervals that were defined for your cluster. Here are some parameters that define the upgrade behavior:
HealthCheckWaitDurationSec
The time to wait (in seconds) after the upgrade has finished on the upgrade domain before Service Fabric evaluates the health of the application
HealthCheckStableDurationSec
The duration (in seconds) to verify that the application is stable before moving to the next upgrade domain or completing the upgrade.
UpgradeHealthCheckInterval
The frequency that the health status is checked.
...and so on. There is also a difference in how upgrade goes for Stateless and Statefull services, which is determined by UpgradeReplicaSetCheckTimeout parameter. Check out Application upgrade parameters and Service Fabric application upgrade for more details.

Looks like SF does not like applications launched from cmd as another process.
I have found solution for my problem. Upgrade deployment time dropped from 12 minutes to 4 minutes. Mainly you need to run executable directly from you SF. Here are steps to achieve it with .Net Core 1.1.
Compile it into EXE by adding following section into project file (Build .NET Core console application to output an EXE?)
<PropertyGroup>
<RuntimeIdentifiers>win10-x64</RuntimeIdentifiers>
</PropertyGroup>
Create self-contained application
dotnet publish -c Release -r win10-x64
Set Entry Point to your executable
<EntryPoint>
<ExeHost>
<Program>SomeWebAPI.exe</Program>
<Arguments />
<WorkingFolder>CodePackage</WorkingFolder>
</ExeHost>
</EntryPoint>

Related

Application and service(s) deployment in Azure Service Fabric

I am not clear enough yet how Service Fabric allows deployment.
From the applications being created in a single VS solution, let me try to ask with file formats for better understanding.
In a single Visual Studio solution, there are
a single .sln
a single .sfproj
multiple .csproj(s)
As I see these files, multiple services (.csproj files) are bound to a single Service Fabric application (.sfproj file), which is under single solution file (.sln file).
Can I individually deploy a .csproj project to the Service Fabric cluster, or are these now bound to a .sfproj so that I have to deploy multiple services (each created with .csproj and bound to .sfproj) together?
The answer to your question is yes and no at the same time. Let me explain it in detail.
Can I individually deploy .csproj project to the Service Fabric cluster
The answer is no you can't deploy a service - in term of Service Fabric the minimal unit of deployment is the application (the .sfproj one). So no matter what changes you have you still need to deploy the application.
But as we all understand performing a full deployment of all application services is very hard, consumes lots of time and causes lots of disturbance to the cluster. To avoid this massive update, all Service Fabric components have their own versions (you can take a closer look at ServiceManifest.xml and ApplicationManifest.xml). So each time application is deployed to the cluster, Service Fabric goes through all services included in the application and updates only components that have been changed (i.e. have different version).
This approach allows you to perform updates of very high granularity i.e. you can update only <Config /> package of the single service.

Is it possible to deploy only Application Parameters (cloud.xml) in Service Fabric without having to change the Application version?

Is there a possibility to deploy only Application Parameters (cloud.xml) somehow without having to redeploy the code or upgrade the application version? For example after deploying the Application on Service fabric cluster, there could be a minor thing I might need to change in one of the parameters in cloud.xml. And it won't make sense to upgrade the version of the Application for this minor change.
I have combed through various sites but did not find anything suitable.
I'd say - no, you can't do this. Here is what Service Fabric application upgrade says on this -
During an upgrade, Service Fabric compares the new application manifest with the previous version and determines which services in the application require updates. Service Fabric compares the version numbers in the service manifests with the version numbers in the previous version. If a service has not changed, that service is not upgraded.

Service Fabric - How to deploy multiple branches of code on one cluster?

Due to the hardware requirements that each cluster must have a minimum of 3 servers I'd like to use that hardware to support multiple branches/environments. Specifically we generally have 3 Dev and 3 Test branches running simultaneously for an application to support multiple parallel development project. After we release to production the code gets merged back into the other braches.
I understand I can create multiple instances of an Application Type, but what I think I really need is to have multiple versions of an Application Type on the same cluster. Its very possible that development could be happening in the A and B branches at the same time. We would want to test and deploy both branches to the Dev Cluster.
Similarly I would like to use the same cluster to expose a test environment endpoint. So as the code gets promoted I could deploy a TestB version of the application, if bug fixes happen those would be fixed and deployed in the DevB Version of that Application Type.
To handle the WebAPI endpoints Port issues we are planning on having the build script chose the environment specific WEBAPI Service manifest because it contains the port number that exposes the Service Fabric application to calling applications. So i'll have a ServiceManifest-DevB.xml file that gets renamed as plain old ServiceManifest.xml and packaged up with the DevB build when it goes out. Then ServiceManifest-TestB.xml will do the same but have a different port. Another option here is Tokenizer.
But I'm struggling on how can I have different versions of the same Application Type running on the cluster? Can I override Application Type in the parameter files or something along those lines? I am really hoping I don't need to build 6 clusters for this? That's a ton of hardware which won't fly.
Please help and thanks in advance,
Greg
I had this question a year ago and had put this down. Now its back so this time I will document it!!!
I am using one cluster for both my Dev and Test environments and we use two branches for these. I needed to be able to deploy the application for these two branches under different application names.
To figure this out I followed the ps1 trail. First you look at Deploy-FabricApplication.ps1 which just passed the PublishProfile to Publish-NewServiceFabricApplication.ps1. This guy uses a method in the Utilities.ps1 called Get-ApplicationNameFromApplicationParameterFile. All this does is open the environment specific application parameter file and read it off the top:
Use this is my Dev Application Paramter file:
<Application xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="fabric:/MyAppDEVA" xmlns="http://schemas.microsoft.com/2011/01/fabric">
Use this in my Test Application Paramter file"
<Application xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="fabric:/MyAppTEST" xmlns="http://schemas.microsoft.com/2011/01/fabric">
Easy breezy when you know, and knowing is half the battle.
You can pass the desired version of an application when you create the application instance via New-ServiceFabricApplication. Just copy, register, and then new-up the application types and versions you need.

Azure PaaS Malwre protection using Symantec

I want to enable Malware protection/Virus scanning in my PaaS.
For example i have a Web Api for file upload and store the files to Blob staorage. I want to scan the files before store it using symantec. I dint find any article or code for the same.
Thanks in advance,
Subbiah K
Firstable you need to resolve the licensing with Symantec. Once that is resolved you must configure some startup Tasks for your compute instance.
This Tasks are intended to run every time the application is started including when the package is deployed and when the instance is destroyed-recreated by azure maintenance or error conditions.
The startup Task purpose should be to copy / install symantec products to be accessible by the cloud service. As I previously say this Task run everytime the package is deployed including instance reconstruction by azure.
Once the symantec product is installed you only need to run the scan each X time, probably this process should be called from a Worker Role.
Create the Task is quite easy , you need to include something like this in the service definition file
<Startup>
<Task commandLine="Startup.cmd" executionContext="limited" taskType="simple" >
<Environment>
<Variable name="MyVersionNumber" value="1.0.0.0" />
</Environment>
</Task>
</Startup>
the cmd file is a windows batch file created to call the Symantec product installer. This Will ensure that the product is installed on each instance even after instance recreation or package deployment.
This link goes to Azure documentation with more detailed information
How to configure and run startup tasks for a cloud service

Wix: The service cannot be started during installation

I'm having a problem with Wix Service as the service cannot be started during install progress. It throws the error:
Error 1053: The service did not respond to the start or control request in a timely fashion
I've tried with both [WIX_ACCOUNT_LOCALSYSTEM] and [WIX_ACCOUNT_LOCALSERVICE] but no one of them work.
But there is weird here as I have an installer which using ClickOne, it includes the same service component as the one I have been using in Wix. The ClickOne installs service just fine (using InstallUtil.exe), so it proves the account has right to start a service.
Then, I uninstall the software (installed by ClickOne), and running the Wix installer again, the service starts well now. I don't know the reason why?
I'd like to put some flows for more clearly:
1- On a fresh machine
2- Running Wix software installer --> the service cannot be started and throwing error message --> Cancel install
3- Running ClickOne software installer --> service starts well --> Uninstalling software
4- Running Wix software installer --> service starts well
Also note that, I've tried 2 times on 2 fresh machines but it's the same. Anyone can shed some light on this weird behavior? Or anything I should verify against?
Thanks in advance,
Thank you #Stephen Connolly, #Alexey Ivanov, #Cosmin Pirvu for your comments.
I'd like to add your comments above as the answer.
Using CheckAsm, a great tool to verify the assembly dependencies
Looking at the log information in Event Viewer for anything could stop the service starting (i.e. timeout, services dependency ...)
Verifying all stuffs would be needed for service operations. They should be available once installation completed (i.e. configuration, registry, working folder ...)
If the installer is installing files to the GAC using the Windows Installer tables, the dependencies won't be available when the installer runs the StartServices action