Removing application from service fabric cluster - azure-service-fabric

I tried removing application from service fabric using service fabric explorer.
I deleted my application using Delete Application action.
Then When I tried Unprovision application type I got error saying,
Error: Application Type of version 1.0.0 could not be unprovisioned as
it still contains active applications.
I could see that even after deleting the application , the actor service inside the application is still active in some of the nodes. Am attaching a screenshot of my service fabric explorer.
Any help regarding completely removing the applications?

This can happen if services in your application don't play nice by not shutting down when requested by the platform, for example by ignoring the cancellation token in RunAsync.
Here's a quick PowerShell script that will go through an application and force remove all replicas of all stateful services:
Connect-ServiceFabricCluster -ConnectionEndpoint localhost:19000
$nodes = Get-ServiceFabricNode
foreach($node in $nodes)
{
$replicas = Get-ServiceFabricDeployedReplica -NodeName $node.NodeName -ApplicationName "fabric:/MyApp"
foreach ($replica in $replicas)
{
Remove-ServiceFabricReplica -ForceRemove -NodeName $node.NodeName -PartitionId $replica.Partitionid -ReplicaOrInstanceId $replica.ReplicaOrInstanceId
}
}

Go to http://localhost:19080/-->Expand Cluster-->Applications-->Service
Click on the 3 dots beside service and select option "Delete service"
Similarly delete the application and then unprovision the type

Related

Release Agent Configuration necessary for Web App Add task

To create a directory under and add a web app to IIS, I put this script in Azure DevOps Server Deployment Group job Powershell task:
New-Item -Path "C:\inetpub\wwwroot" -Name "MyNewApp" -ItemType "directory" `
C:\Windows\system32\inetsrv\appcmd.exe add app `
/site.name:"Default Web Site" `
/path:"/MyNewApp" `
/physicalPath:"C:\inetpub\wwwroot\MyNewApp"`
When this task runs it throws an error: (timestamps removed for clarity)
APP object "Default Web Site/MyNewApp" changed
ERROR ( hresult:80090016, message:Failed to commit configuration changes.
Keyset does not exist )
After executing the above code from a Powershell console while logged in as either myself or the service account, the following messages are returned:
APP object "Default Web Site/MyNewApp" added
VDIR object "Default Web Site/MyNewApp" added
I can confirm in IIS Manager that the directory and application was created.
I cannot reproduce the error in a console. It is only when it is running as the Release Agent and initiated through the DevOps Server web interface that this error occurs.
I have tried the following:
Keyset Issue - https://support.microsoft.com/en-us/help/977754/keyset-does-not-exist-error-message-when-you-try-to-change-the-identit for both the Local Service and service accounts.
Up-to Modify permissions on C:\inetpub\wwwroot for the service account
Adding the service account to IIS Manager Permissions
Update 1:
I determined the "Keyset does not exist" error was coming from trying to use the DefaultAppPool application pool as part of the setup. This application pool was not setup with the service account but was still set with default Application Pool Identity.
However, after making this change I have a new error:
APP object "Default Web Site/MyNewApp" changed
ERROR ( hresult:80070057, message:Failed to commit configuration changes.
The parameter is incorrect. )
...
Process 'appcmd.exe' exited with code '87'.
Any guidance is greatly appreciated.
Update 2:
I logged an issue with Microsoft community that is currently being looked at. You can see it here.
You could also look at the .net implementation with the servermanager:
https://learn.microsoft.com/en-us/iis/manage/scripting/how-to-use-microsoftwebadministration
https://learn.microsoft.com/en-us/dotnet/api/microsoft.web.administration.servermanager?view=iis-dotnet
Should not be very hard to use this in powershell.

Issue with service fabric resource manager file deployment (apim.json and apim.parameters)

My requirement is as follows:
I have an web API whose port I have removed from the ServiceManifest.xml file. This is done so that I can implement multiple node multiple calls feature from API management. (i.e. I want to remove dependency on port number)
While deploying API management resource files, I am facing issues while deploying apim.json and apim.parameters.json file.
Following is the exception I am getting always.
"Service activation failed. Please look at the details in Activity Log on the left side. In case you are deploying into VNET please make sure prerequisites are followed as described on https://aka.ms/apiminvnet"
There is nothing in the log files when I am uploading.
I am using following link for the deployment and testing. https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-tutorial-deploy-api-management
I am getting exception in running powershell script
New-AzureRmResourceGroupDeployment -ResourceGroupName $groupname -TemplateFile "$templatepath\apim.json" -TemplateParameterFile "$templatepath\apim.parameters.json" -Verbose

Run Service Fabric App under Group Managed Service Account (gMSA)

I'm testing using a gMSA account to run an SF app, instead of NETWORKSERVICE.
Following the instructions from here:
https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-application-runas-security
Created the gMSA on the domain controller using the powershell cmdlet:
New-ADServiceAccount -name MySA$ -DnsHostName MySA.contoso -ServicePrincipalNames http/MySA.contoso -PrincipalsAllowedToRetrieveManagedPassword Node0Machine$, Node1Machine$, Node2Machine$
Install-AdServiceAccount returned an "unspecified error" on each of the nodes, however Test-AdServiceAccount returns true for MySA$ (when running powershell as a domain user)
ApplicationManifest.xml has the following changes:
<Principals>
<Users>
<User Name="MySA" AccountType="ManagedServiceAccount" AccountName="Contoso\MySA$"/>
</Users>
</Principals>
<Policies>
<SecurityAccessPolicies>
<SecurityAccessPolicy ResourceRef="ConfigurationEncipherment" PrincipalRef="MySa" ResourceType="Certificate" />
</SecurityAccessPolicies>
<DefaultRunAsPolicy UserRef="MySA"/>
</Policies>
The Service Fabric explorer shows the following error for each service:
Error event: SourceId='System.Hosting', Property='CodePackageActivation:Code:SetupEntryPoint'.
There was an error during CodePackage activation.Service host failed to activate. Error:0x8007052e
I have also tried creating the cluster using the gMSA (we are using X509 successfully at the moment). Using the gMSA cluster config as a template, it fails with a timeout (presumably the "WindowsIdentities section is incorrect - there seems to be little documentation on this)
"security": {
"WindowsIdentities": {
"ClustergMSAIdentity": "MySA$#contoso",
"ClusterSPN": "http/MySa.contoso",
"ClientIdentities": [
{
"Identity": "contoso\\MySA$",
"IsAdmin": true
}
]
},
The Error:0x8007052e may be linked to a logon failure.
According to Secure a standalone cluster on Windows by using Windows security and Connect to a secure cluster
If you have more than 10 nodes or for clusters that are likely to grow or shrink. Microsoft strongly recommend using the Group Managed Service Account (gMSA) approach.
You will see also:
You can establish trust in two different ways:
Specify the domain group users that can connect.
Specify the domain node users that can connect.
[...]
Administrators have full access to management capabilities (including read/write capabilities). Users, by default, have only read access to management capabilities (for example, query capabilities), and the ability to resolve applications and services.
You may also find help on Getting Started with Group Managed Service Accounts
According to your comment, as soon as you add the gMSA to the ServiceFabricAdministrators group everything will work and it is probably due to the fact that "administrators have full access to management capabilities"

Hosting a Console application in Service Fabric

I'm starting with Service Fabric. I have created a very simple console application that runs the following code:
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Hello world!");
File.AppendAllText("c:\\temp\\hello.txt", "Hello world!" + DateTime.Now.ToString() + "\r\n");
Console.ReadLine();
}
}
Then I create a guest executable project with Visual Studio and point it to the exe application. It gets installed in Service Fabric, I can see that the file is created, but then service fabric throws an error:
Error event: SourceId='System.FM', Property='State'.
Partition is below target replica or instance count.
fabric:/Test3/Test3Service -1 1 5ef5a0eb-5621-4821-95cb-4c1920ab7f0c
(Showing 0 out of 0 replicas. Total available replicas: 0.)
Is this approach correct? Can I have exe applications hosted in Service Fabric or do I need to implement/inherit from something?
EDIT
When the application is deployed it enters in a Warning state, showing the following messages:
Soon afterwards it transitions to an error state:
Yes you can host a simple Console Application in Service Fabric as a Guest Executable, that should not be a problem.
The issue you are seeing is likely because the application is trying to write to a file in c:/temp where your Guest Exe by default doesnt have permissions. Try removing that part of your sample code, or change it to write to hello.txt and it will end up in the same folder your Guest Exe is running in.
You should consider file storage on a Service Fabric node as temporary however and not rely on storing data there as your service could be moved between nodes by Service Fabric as part of it's cluster maintenance.
See this answer for some more details on file system access in SF https://stackoverflow.com/a/37966158/1062217

Azure Service Fabric - change config settings for a deployed Application

How do I change settings for a deployed application in Service Fabric?
I have a provisioned cluster and an application deployed to the cluster with two applications. I would like to be able to change my services' settings and have them pick up those changes, but I don't see how I can do that.
Previously, we've done all of our services with worker roles in Cloud Services, and the portal allows for changing configurations, but it does not appear to do so for Service Fabric. From the Service Fabric Explorer I can drill down to the service, go to MANIFEST and view the XML with the settings. I just don't see a way to edit or change it. I've struggled finding anything in the SF documentation addressing this.
The portal doesn't expose a way to do this. It needs to be done via an upgrade of the application. Just change the settings in your settings XML file and perform an upgrade. In the VS publish dialog for your application project, you can update your version numbers appropriately by changing the config package version which will automatically bubble up to update the containing service and application versions.
Building on Matt Thalman's answer, here's documentation on modifying the settings in the application or service manifest XML files, updating the version numbers, and performing an application upgrade: Service Fabric application upgrade tutorial using Visual Studio. You can also perform the app upgrade using PowerShell.
Additional to above answers, adding some powershell code..
we may use below powershell code to connect to Service Fabric from powershell and get the application parameters and then update specific parameter and re deploy..
### Change the connection here (from Profile-Cloud.xml
$ConnectArgs = #{
ConnectionEndpoint="devxxxxxx.westus.cloudapp.azure.com:19000"
X509Credential="true"
ServerCertThumbprint="52BFxxxxxxxxxx"
FindType="FindByThumbprint"
FindValue="EF3A2xxxxxxxxxxxxxx"
StoreLocation="CurrentUser"
StoreName="My"
}
Connect-ServiceFabricCluster #ConnectArgs
$myApplication = Get-ServiceFabricApplication -ApplicationName fabric:/ABC.MyService
$appParamCollection = $myApplication.ApplicationParameters
### Update your parameter here..
$applicationParameterMap.ElasticSearch_Username="sachin2"
$applicationParameterMap = #{}
foreach ($pair in $appParamCollection)
{
$applicationParameterMap.Add($pair.Name, $pair.Value);
}
### Start Udpating
Start-ServiceFabricApplicationUpgrade -ApplicationName $myApplication.ApplicationName.OriginalString -ApplicationTypeVersion $myApplication.ApplicationTypeVersion -ApplicationParameter $applicationParameterMap -Monitored -FailureAction Rollback -ForceRestart $true
### Check the status until it is Ready
(Get-ServiceFabricApplication -ApplicationName fabric:/ABC.MyService).ApplicationStatus
### Check the parameters to confirm those're updated
Get-ServiceFabricApplication -ApplicationName fabric:/ABC.MyService
You may change or remove the -ForceRestart as per your requriements