Azure Service Fabric - Configure service instance parameters using Powershell - powershell

Is it possible to inject parameters at run time into a Guest Executable via the Settings.xml file or another means? I have a GuestExecutable which I need to pass some configuration to it - a URL at service creation time.
I need two instances of the service running with different parameters, the service instance information has to differ in terms of a custom parameter I need to pass through . Is this possible using Powershell or, do I need to version the config and create a new version?
Thanks in advance

Have you tried the following command: New-ServiceFabricApplication ?
When you create your application manifest file, the parameters will contain replaceable parameters that you set when you register a new application.
An ApplicationManifest.xml example like this:
<?xml version="1.0" encoding="utf-8"?>
<ApplicationManifest ApplicationTypeName="MyAppTypeName" ApplicationTypeVersion="1.0.0" xmlns=...>
<Parameters>
<Parameter Name="Web1_InstanceCount" Value="-1" />
<Parameter Name="ENVIRONMENT_NAME" Value="DEV" />
<Parameter Name="FEPlacementConstraints" Value="NodeTypeName==FrontEnd" />
</Parameters>
<ServiceManifestImport>
<ServiceManifestRef ServiceManifestName="MyServicePkg" ServiceManifestVersion="1.0.0" />
<ConfigOverrides />
<EnvironmentOverrides />
</ServiceManifestImport>
<DefaultServices>
<Service Name="Web1">
<StatelessService ServiceTypeName="MyServiceType" InstanceCount="[Web1_InstanceCount]">
<SingletonPartition />
<PlacementConstraints>[FEPlacementConstraints]</PlacementConstraints>
</StatelessService>
</Service>
</DefaultServices>
</ApplicationManifest>
You would change the settings file using config overrides like this:
<ConfigOverrides>
<ConfigOverride Name="Config">
<Settings>
<Section Name="MyConfigSection">
<Parameter Name="MySetting" Value="[ENVIRONMENT_NAME]"/>
</Section>
</Settings>
</ConfigOverride>
</ConfigOverrides>
Or you could set environment variables on your service like this:
<EnvironmentOverrides CodePackageRef="Code">
<EnvironmentVariable Name="ASPNETCORE_ENVIRONMENT" Value="[ENVIRONMENT_NAME]" />
</EnvironmentOverrides>
In my opinion the environment overrides would work better on your case,
because most guest executable are not flexible on how you can configure them but generally they accept environment variables.
Then you:
Upload your application with: Copy-ServiceFabricApplicationPackage
Register your application with Register-ServiceFabricApplicationType
And then you register each new app with the settings you need:
.
New-ServiceFabricApplication -ApplicationName fabric:/myapp/todolist-dev -ApplicationTypeName "MyAppTypeName" -ApplicationTypeVersion "1.0.0" -ApplicationParameter #{Web1_InstanceCount='-1'; ENVIRONMENT_NAME='DEV'}
New-ServiceFabricApplication -ApplicationName fabric:/myapp/todolist-uat -ApplicationTypeName "MyAppTypeName" -ApplicationTypeVersion "1.0.0" -ApplicationParameter #{Web1_InstanceCount='-1'; ENVIRONMENT_NAME='UAT'}
The only down side of this approach is that you would end up with two applications, but don't think would be a problem for you, they will be managed mostly the same way as you would do with a single application.
if you strictly need to run both together on same application, you could do some workarounds:
Using multiple Config packages(don't think would work well with guest executables, but works well with reliable services
Using the startup script, where you would add the logic to pass the parameter to your startup exe, something like the following:
In your ServiceManifest.xml:
<CodePackage Name="Code" Version="1.0.0">
<EntryPoint>
<ExeHost>
<Program>start.bat</Program>
<WorkingFolder>CodePackage</WorkingFolder>
</ExeHost>
</EntryPoint>
</CodePackage>
On the same folder (code) of your .exe, you create the start.bat file with following contents:
myApp.exe %EnvironmentVariableNameSetBySF%

Related

WIX: Install Services using runner and multible dlls

I'm working on a Wix installer which should install multiple Services based on the same runner. This a some dlls which will be loaded by the runner. With sc.exe this works fine on my test system. Now my question, can I use standard Wix ServiceInstall for this? I have only one runner.exe and I'm not sure how to write this in the XML. Or would a Custom Action be the right way?
Thanks!!
The ServiceInstall and ServiceControl elements don't come after the file element per say but they are children on the Component element. They target the keypath of the component which happens to be the File element. You can easily have multiple services defined in a single component pointing to the same executable.
<Component Id="c1" Guid="dbc1b8dd-14e1-380f-5793-4a746fa0c5c5">
<File Id="f1" Source="$(var.SourceDir)\TestService.exe" KeyPath="yes" />
<ServiceInstall Id="si1" Name="TestService1" DisplayName="TestService1 Service" Description="TestService1 Service" ErrorControl="normal" Start="auto" Type="ownProcess" />
<ServiceControl Id="sc1" Name="TestService1" Start="install" Stop="both" Remove="both" Wait="yes" />
<ServiceInstall Id="si2" Name="TestService2" DisplayName="TestService2 Service" Description="TestService Service" ErrorControl="normal" Start="auto" Type="ownProcess" />
<ServiceControl Id="sc2" Name="TestService2" Start="install" Stop="both" Remove="both" Wait="yes" />
</Component>
To have each service behave differently you'll have to write code in your service to access ServiceBase.ServiceName (this.ServiceName in the OnStart method likely). From here you can dynamically load different classes from different assemblies.

"Failed to create endpoint [XXX] on network because of a duplicate name" on local 5-node Service Fabric cluster

I have a local Service Fabric cluster of 5 nodes and I have a problem deploying my application on all nodes. When set to "1 Node", the cluster works fine. When set to "5 Nodes" it gives an error on all nodes but one. This is the error/warning message:
Error event: SourceId='System.Hosting', Property='CodePackageActivation:Code:EntryPoint:131919316927686034'.
There was an error during CodePackage activation.System.Fabric.FabricException (-2147017731)
Failed to start Container. ContainerName=sf-0-28e0002f-fd7d-412c-81b8-b78ca5339ce4_865991cc-9c36-493f-9b3d-95f6eba43851, ApplicationId=Proton.SFType_App0, ApplicationName=fabric:/Proton.SF.
DockerRequest returned StatusCode=InternalServerError with ResponseBody={"message":"failed to create endpoint sf-0-28e0002f-fd7d-412c-81b8-b78ca5339ce4_865991cc-9c36-493f-9b3d-95f6eba43851 on network nat: HNS failed with error : You were not connected because a duplicate name
The error message looks truncated. The application loads up fine, but on one node only. Am I missing something in 5-node configuration? The application we are deploying is a container which runs a .NET Core app. I've attached a screenshot of the error in Service Fabric Explorer.
Error Screenshot
ServiceManifest.xml
<?xml version="1.0" encoding="utf-8"?>
<ServiceManifest Name="Proton.TestingPkg"
Version="1.0.0"
xmlns="http://schemas.microsoft.com/2011/01/fabric"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ServiceTypes>
<!-- This is the name of your ServiceType.
The UseImplicitHost attribute indicates this is a guest service. -->
<StatelessServiceType ServiceTypeName="Proton.TestingType" UseImplicitHost="true" />
</ServiceTypes>
<!-- Code package is your service executable. -->
<CodePackage Name="Code" Version="1.0.0">
<EntryPoint>
<!-- Follow this link for more information about deploying Windows containers to Service Fabric: https://aka.ms/sfguestcontainers -->
<ContainerHost>
<ImageName>proton.azurecr.io/protontesting:latest</ImageName>
</ContainerHost>
</EntryPoint>
<!-- Pass environment variables to your container: -->
<!--
<EnvironmentVariables>
<EnvironmentVariable Name="VariableName" Value="VariableValue"/>
</EnvironmentVariables>
-->
</CodePackage>
<!-- Config package is the contents of the Config directoy under PackageRoot that contains an
independently-updateable and versioned set of custom configuration settings for your service. -->
<ConfigPackage Name="Config" Version="1.0.0" />
<Resources>
<Endpoints>
<!-- This endpoint is used by the communication listener to obtain the port on which to
listen. Please note that if your service is partitioned, this port is shared with
replicas of different partitions that are placed in your code. -->
<Endpoint Name="Proton.TestingTypeEndpoint" Port="8001" />
</Endpoints>
</Resources>
</ServiceManifest>
ApplicationManifest.xml
<?xml version="1.0" encoding="utf-8"?>
<ApplicationManifest ApplicationTypeName="Proton.SFType"
ApplicationTypeVersion="1.0.0"
xmlns="http://schemas.microsoft.com/2011/01/fabric"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Parameters>
<Parameter Name="Proton.Testing_InstanceCount" DefaultValue="-1" />
</Parameters>
<!-- Import the ServiceManifest from the ServicePackage. The ServiceManifestName and ServiceManifestVersion
should match the Name and Version attributes of the ServiceManifest element defined in the
ServiceManifest.xml file. -->
<ServiceManifestImport>
<ServiceManifestRef ServiceManifestName="Proton.TestingPkg" ServiceManifestVersion="1.0.0" />
<ConfigOverrides />
<Policies>
<ContainerHostPolicies CodePackageRef="Code">
<!-- See https://aka.ms/I7z0p9 for how to encrypt your repository password -->
<RepositoryCredentials AccountName="ProtonCluster" Password="XXX" PasswordEncrypted="false" />
<PortBinding ContainerPort="80" EndpointRef="Proton.TestingTypeEndpoint" />
</ContainerHostPolicies>
</Policies>
</ServiceManifestImport>
<DefaultServices>
<!-- The section below creates instances of service types, when an instance of this
application type is created. You can also create one or more instances of service type using the
ServiceFabric PowerShell module.
The attribute ServiceTypeName below must match the name defined in the imported ServiceManifest.xml file. -->
<Service Name="Proton.Testing" ServicePackageActivationMode="ExclusiveProcess">
<StatelessService ServiceTypeName="Proton.TestingType" InstanceCount="[Proton.Testing_InstanceCount]">
<SingletonPartition />
</StatelessService>
</Service>
</DefaultServices>
</ApplicationManifest>

Service Fabric: are multiple service types allowed in ServiceManifest.xml

I'm trying to register multiple service-fabric services within a single exe (ServiceRuntime.RegisterServiceAsync). Is this supported? If so, how would I configure them?
Eg: ServiceManifest.xml supports multiple StatelessServiceType elements within ServiceTypes:
<?xml version="1.0" encoding="utf-8"?>
<ServiceManifest Name="EchoGatewayPkg"
Version="1.0.0"
xmlns="http://schemas.microsoft.com/2011/01/fabric"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ServiceTypes>
<StatelessServiceType ServiceTypeName="Service1Type" >
</StatelessServiceType>
<StatelessServiceType ServiceTypeName="Service2Type" >
</StatelessServiceType>
</ServiceTypes>
...
and ApplicationManifest.xml does not support multiple StatelessService elements within DefaultServices/Service:
<?xml version="1.0" encoding="utf-8"?>
<ApplicationManifest xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ApplicationTypeName="EchoServiceType" ApplicationTypeVersion="1.0.0" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Parameters>
<Parameter Name="Service1_InstanceCount" DefaultValue="1" />
<Parameter Name="Service2_InstanceCount" DefaultValue="1" />
</Parameters>
<ServiceManifestImport>
<ServiceManifestRef ServiceManifestName="MyServicePkg" ServiceManifestVersion="1.0.0" />
<ConfigOverrides />
</ServiceManifestImport>
<DefaultServices>
<Service Name="MyService1">
<StatelessService ServiceTypeName="Service1Type" InstanceCount="[Service1_InstanceCount]">
<SingletonPartition />
</StatelessService>
</Service>
<Service Name="MyService2">
<StatelessService ServiceTypeName="Service2Type" InstanceCount="[Service2_InstanceCount]">
<SingletonPartition />
</StatelessService>
</Service>
</DefaultServices>
</ApplicationManifest>
Hence, this effectively spawns 2 processes, and each process' activation-context has both default service types listed (I would have expected only one with this configuration).
Any suggestions (on how to configure multiple service types within a single exe) or clarifications are welcome.
Disregarding default services for a second..
Yes, this is how you get multiple service types to share a host process. You're seeing 2 processes because Service Fabric is placing those two service instances on different nodes and each one needs a host process. Try creating each service with InstanceCount="-1" to see the two types share host processes.
OK, back to default services. This is inherently confusing because application manifest and service manifest are only meant to describe type information, not instance information. But in order to create service instances by default when an application instance is created, you have to specify, declaratively, the instances and their parameters. So what you see under default services there in your example is the XML equivalent of:
PS > New-ServiceFabricService -Stateless -PartitionSchemeSingleton -InstanceCount 1 -ApplicationName fabric:/EchoService -ServiceName fabric:/EchoService/MyService1 -ServiceTypeName Service1Type
PS > New-ServiceFabricService -Stateless -PartitionSchemeSingleton -InstanceCount 1 -ApplicationName fabric:/EchoService -ServiceName fabric:/EchoService/MyService2 -ServiceTypeName Service2Type
So it doesn't make sense to have more than one StatelessService element in the Service element, because you're defining the service instances you want to create, along with the type and version of the service instance.
And just for posterity, this is the registration code that goes with the service manifest you posted:
internal static class Program
{
private static void Main()
{
try
{
ServiceRuntime.RegisterServiceAsync("Service1Type",
context => new Stateless1(context)).GetAwaiter().GetResult();
ServiceRuntime.RegisterServiceAsync("Service2Type",
context => new Stateless2(context)).GetAwaiter().GetResult();
Thread.Sleep(Timeout.Infinite);
}
catch (Exception e)
{
ServiceEventSource.Current.ServiceHostInitializationFailed(e.ToString());
throw;
}
}
}
}

How do I configure local cluster for addtional node types

I have a cluster configuration with two Node Types specified in the ServiceManifest.xml
<?xml version="1.0" encoding="utf-8"?>
<ServiceManifest Name="MKopa.M2M.ConfigurationPkg"
Version="1.0.0"
xmlns="http://schemas.microsoft.com/2011/01/fabric"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ServiceTypes>
<!-- This is the name of your ServiceType.
This name must match the string used in RegisterServiceType call in Program.cs. -->
<StatelessServiceType ServiceTypeName="ConfigurationType">
<PlacementConstraints>(NodeType == Internal)</PlacementConstraints>
</StatelessServiceType>
</ServiceTypes>
<!-- Code package is your service executable. -->
<CodePackage Name="Code" Version="1.0.0">
<EntryPoint>
<ExeHost>
<Program>MKopa.M2M.Configuration.Service.exe</Program>
</ExeHost>
</EntryPoint>
</CodePackage>
<!-- Config package is the contents of the Config directoy under PackageRoot that contains an
independently-updateable and versioned set of custom configuration settings for your service. -->
<ConfigPackage Name="Config" Version="1.0.0" />
<Resources>
<Endpoints>
<!-- This endpoint is used by the communication listener to obtain the port on which to
listen. Please note that if your service is partitioned, this port is shared with
replicas of different partitions that are placed in your code. -->
<Endpoint Name="ServiceEndpoint" />
<Endpoint Name="HttpEndpoint" Protocol="http" Port="8081"/>
</Endpoints>
</Resources>
</ServiceManifest>
My issue that this causes the deployment to the local cluster to fail as this NodeType doesn't exist in the Local Cluster.
I've seen mention of the cluster.xml file and I've found it but making changes to it doesn't seem to have any effect. I've tried a reset, start and stop but the reset overrides the changes.
Here's is hoping that the answer is not starting the services dynamically :-)
I don't know how it works while the cluster is running, but I was able to do it by re-installing the local cluster. These were my steps:
Go to C:\Program Files\Microsoft SDKs\Service Fabric\ClusterSetup\
Uninstall the existing cluster by calling .\CleanCluster.ps1
Create a backup of the file C:\Program Files\Microsoft SDKs\Service Fabric\ClusterSetup\NonSecure\ClusterManifestTemplate.xml
Now you can adjust this file and add placement properties to every node:
<NodeType ...>
<Endpoints>...</Endpoints>
<PlacementProperties>
<Property Name="NodeType" Value="Internal" />
</PlacementProperties>
</NodeType>
Re-create the cluster by calling .\DevClusterSetup.ps1

Service Fabric Multiple service instances with config override

Our service fabric application includes a stateless service that exposes an HTTP endpoint through OwinCommunicationListener.
The ServiceManifest.Xml for this service specifies the service endpoint <Endpoint Name="ServiceEndpoint" Type="Input" Protocol="http" Port="8090" />
The stateless service can then be accessed via a browser on http://localhost:8090/
What we are trying to do is instantiate multiple instances of this service on different endpoints in the same Service Fabric application through the ApplicationManifest.
The ServiceManifestImport imports our service package and allows configuration overrides at the application level. We're not able to override the ServiceEndpoint this way, only the values in Settings.xml
<ServiceManifestImport>
<ServiceManifestRef ServiceManifestName="FooServicePkg" ServiceManifestVersion="1.0.0" >
<ConfigOverrides Name="Config">
<Settings>
<SectionName Name="MySettings">
<Parameter Name="MySetting" Value="SomeValue">
</Settings>
</ConfigOverrides>
</ServiceManifestImport>
We can create named instances of the service by specifying multiple Service nodes under DefaultServices
<DefaultServices>
<Service Name="FooInstanceA">
<StatelessService ServiceTypeName="FooType" InstanceCount="1" />
<SingletonPartition />
</StatelessService>
</Service>
<Service Name="FooInstanceB">
<StatelessService ServiceTypeName="FooType" InstanceCount="1" />
<SingletonPartition />
</StatelessService>
</Service>
</DefaultServices>
Is it possible to specify configuration overrides per instance of a service through configuration?
I tried to make the service instances listen on a specific port by using their service name to work out which port so FooInstanceA listens on port 8090 and FooInstanceB listens on 8091.
Clearly Service Fabric is doing some magic during deployment because when FooInstanceB listens on a port other than the one specified on the ServiceEndpoint configuration the service is not accessible.
The first reason is the DACL is not set on the endpoint, this is resolved by running;
netsh http add urlacl http://+:8091/ user=everyone listen=yes
This allows the services to come up and show healthy in the Service Fabric Explorer, however the FooInstanceB is responding with an HTTP 503 error when we access with http://localhost:8091/
How can we get the service instances listening on different ports?
I hope that's clear, thank you.
Not a lot of great options to accomplish this. Here are some ideas:
Create multiple application instances instead of multiple services of the same type within an app. That would allow you to use the app parameters to configure the behavior of the particular service.
Create multiple config packages within your service type. Each config package would be intended for one of the service instances. Determining which config package a service instance is assigned to would need to be dynamic, maybe based on the service instance's name? Not a great option, of course, since it couples the service definition with the number of instances that will be created.
A custom configuration implementation. Maybe have your service expose an endpoint that allows you to configure it post-deployment. Or have the service call some other management service that provides its configuration at activation time.
You can also let Service Fabric assign the ports automatically and then use the reverse proxy that comes with Service Fabric.
I think its a very good idea to have a url rewriting service in front of a SF cluster .
This can give you multiple end points and do url rewriting , firewalls/ black listing , https etc .
Other way is to package the service as a lib or in Nuget and create the N services you need with different service manifests.
I've managed to do this using ApplicationParameter xml files and choose the right one during deployment in VSTS.
This, for example, is my Cloud.xml which I use for deployment to my test environment. The trick is the Name parameter on the second line. I've specified the port using this documentation: https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-how-to-specify-port-number-using-parameters
<?xml version="1.0" encoding="utf-8"?>
<Application xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="fabric:/DotNetCoreTest" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Parameters>
<Parameter Name="Web1_InstanceCount" Value="2" />
<Parameter Name="ServiceEndpoint_PortNumber" Value="8909" />
</Parameters>
</Application>
This is the xml file for Staging: Staging.xml
<?xml version="1.0" encoding="utf-8"?>
<Application xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="fabric:/DotNetCoreStaging" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Parameters>
<Parameter Name="Web1_InstanceCount" Value="2" />
<Parameter Name="ServiceEndpoint_PortNumber" Value="8910" />
</Parameters>
</Application>
Using the Traefik Reverse Proxy as described here I can use a hostname to reach my service.
In VSTS I'm using a tokenizer to replace both the host name in the ServiceManifest and replacing the ApplicationTypeName in the ApplicationManifest during deployment.
This is my ServiceManifest:
<ServiceTypes>
<!-- This is the name of your ServiceType.
This name must match the string used in RegisterServiceType call in Program.cs. -->
<StatelessServiceType ServiceTypeName="Web1Type">
<Extensions>
<Extension Name="Traefik">
<Labels xmlns="http://schemas.microsoft.com/2015/03/fabact-no-schema">
<Label Key="traefik.frontend.rule.hostname">Host: #{HostName}#</Label>
<Label Key="traefik.expose">true</Label>
<Label Key="traefik.frontend.passHostHeader">true</Label>
</Labels>
</Extension>
</Extensions>
</StatelessServiceType>
</ServiceTypes>