What is the best way to invoke the same method on all active instances of a stateless service in SFC?
I mean doing something similar to below, just on ALL active instances:
var service = ServiceProxy.Create<IMyService>("<my-service-uri>");
await service.MyServiceMethod();
Unfortunately looks like there is no way to send remoting request to the particular service instance / replica.
According to the documentation of ServiceProxy.Create method there is a very limited control over target instance / replica selection (it is strict to the TargetReplicaSelector enumeration).
Related
Hystrix is predominantly meant for applications built using spring cloud.
Having said that there could be multiple services layer for an application.
e.g. Amazon (Amazon site must be having multiple services like login, products, carts, orders, payment and so on)
Client (say web user) -> web application X -> Service A (it uses Data Source A ) -> Service B (Data Source B) -> Service C (Data Source C) -> Service D (Data Source D) -> Service E (Data Source E)
with this kind of scenario when something breaks in Service E, How that gets navigated back to client?
How Hystrix can be useful here to know unavailability of one specific functionality in Service E?
If that example is wrong, then is Hystrix scope limited to multiple processes inside one service and not multiple services used in one application?
Having said that above example can be tweaked something like below
Client (say web user) -> web application X -> Service A -> inside Service A Lets say there are processes like process 1 ->process 2 ->process 3 ->process 4->process 5
and anything fails in process 5 gets navigated back to process 1 and then back to client.
My question is more about maintaining thread state here.
With try-catch thread scope is limited per service (please correct me if wrong).
How Hystrix maintains state of thread during this whole transaction?
Hystrix is predominantly meant for applications built using spring
cloud
Not exactly. Hystrix is generally used to enable Circuit Breaker functionality. It could be used everywhere. Even for a simple method call
For example
class A {
B b;
public void methodA() {
b.methodB();
}
}
class A {
DatabaseConnectionPool object;
#HystrixCommand(fallbackMethod = "abcd")
public void methodB() {
// Get db object from pool.
// call db and fetch something.
}
}
Even for a simple method call, it can be used. Doesn't matter what is being done inside the code wrapped around Hystrix.
But generally we use Hystrix around pieces of code which would throw exceptions for unknown reasons (especially when calling differnt applications)
with this kind of scenario when something breaks in Service E, How
that gets navigated back to client? How Hystrix can be useful here to
know unavailability of one specific functionality in Service E?
If you wrap each method call i.e from Service A --> Service B and Service B --> Service C and further with Hystrix, then each call is treated as a circuit and you can visualize using Hystrix-dashboard, what is the state(closed, open, half-open) of each circuit.
Let's assume the call from Service B --> Service C fails(throws exception) then Hystrix would wrap the original exception in Hystrix Exception and throws it back. If you have a fallback method, then it goes to the fallback method in Service B and returns the value specified in fallback. If you don't have fallback then it throws the exception higher up the chain. And same thing repeats higher up the chain.
How Hystrix maintains state of thread during this whole transaction ?
For each call wrapped with Hystrix, Hystrix maintains a Thread Pool and you can configure this completely.
If I already have an existing Java feature of using try-catch why will
someone go for Hystrix explicitly?
Hystrix provides much more functionality. You cannot even compare that with try catch. I suggest you read Circuit breaker pattern.
When you create a service fabric application project using visual studio, you get an implementation of EventSource (called ServiceEventSource). For example, here is one of the method implementation:
private const int ServiceRequestStopEventId = 6;
[Event(ServiceRequestStopEventId, Level = EventLevel.Informational, Message = "Service request '{0}' finished", Keywords = Keywords.Requests)]
public void ServiceRequestStop(string requestTypeName)
{
WriteEvent(ServiceRequestStopEventId, requestTypeName);
}
As you can see, this method has Event attribute which has Level argument set.
Where would I set that Level argument value?
I am thinking that setting this Level's argument value will show how much output gets generated. Am I correct?
Can I modify this Level argument value dynamically at run time and at will?
You can set Level only in the Event attribute.
The amount of output getting generated depends on consumers of logs. If there are no consumers or listeners, event will not be generated at any level. We can say that level depends on amount of output, but only if there are consumers of such event.
No, you can't modify the level dynamically. To do this, you could have two methods with the same signature and different levels.
You can find all the interesting information about ETW and its configuration here.
The code just indicates information about the ETW events it generates. Setting the level indicates in which category the event will be put. It doesn't configure whether the event is output. The logging tool determines whether it's logged or not. And you can usually change that level in the logging tool at run time.
Some useful links:
Configure WAD:
https://azure.microsoft.com/en-us/documentation/articles/service-fabric-diagnostics-how-to-setup-wad/
Use Elastic Search
https://azure.microsoft.com/en-us/documentation/articles/service-fabric-diagnostic-how-to-use-elasticsearch/
Use OMS to analyse the events.
https://azure.microsoft.com/en-us/documentation/articles/log-analytics-service-fabric/
Use Service Profiler (Actors)
https://www.azureserviceprofiler.com/
When I create a proxy to my service like this:
ServiceProxy.Create<IMyService>(new Uri("fabric:/MyApplication/MyService"));
What values will OperationRetrySettings instance has?
Will there be any logic in default TryHandleException implementation?
Will any logic related to retrying in case of exceptions even be involved in case of mentioned code?
Is there a way to adjust retrying in case of exception logic, change values of default OperationRetrySettings?
I use FabricTransportServiceRemotingListener.
Based on my observations, which, definitely, do not cover all the questions completely because there is not so much information:
Some defaults. At least DefaultMaxRetryCount is 10 and 2 seconds on each backoff interval. I found these values by instantiating FabricTransportServiceRemotingClientFactory and passing custom IExceptionHandler. Probably, this property even has no meaning if you are using default ServiceProxyFactory or ServiceProxy.
Looks like yes. I didn't find the exact default IExceptionHandler which is used in proxy and factory, but noticed numerous retries in case if I throw an TimeoutException exception in a service. Probably, logic of ActorRemotingExceptionHandler and ServiceRemotingExceptionHandler is used by default.
Yes.
Yes. You need to instantiate ServiceProxyFactory or ActorProxyFactory, passing there IServiceRemotingClientFactory implementation (for example, FabricTransportServiceRemotingClientFactory), specifying OperationRetrySettings and IExceptionHandler, passing as an exception handler your own implementation.
The ServiceProxy.Create uses a default ServiceProxyFactory.
It is created with the default OperationRetrySettings which are documented here:
https://msdn.microsoft.com/en-us/library/mt711955.aspx
The default ServiceProxyFactory also uses the default service removing client factory; FabricTransportServiceRemotingClientFactory.
https://msdn.microsoft.com/en-us/library/microsoft.servicefabric.services.remoting.fabrictransport.client.fabrictransportserviceremotingclientfactory.fabrictransportserviceremotingclientfactory.aspx
Following the remarks on this documentation, you can see that it uses ServiceRemotingExeptionHandler which is described here:
https://msdn.microsoft.com/en-us/library/microsoft.servicefabric.services.remoting.client.serviceremotingexceptionhandler.aspx
I am writing a reliable actor in service fabric who's job it will be to listen to changes in a Firebase DB and run logic based on those changes. I have it functioning, but not correctly. What I've done so far is write the actor code with a method called MonitorRules() which is what is listening to Firebase using a C# Firebase client wrapper called FireSharp. MonitorRules() looks like this:
public async Task MonitorRules()
{
FireSharp.FirebaseClient client = new FireSharp.FirebaseClient(new FireSharp.Config.FirebaseConfig
{
AuthSecret = "My5up3rS3cr3tAu7h53cr37",
BasePath = "https://myapp.firebaseio.com/"
});
await client.OnAsync("businessRules",
added: (sender, args) =>
{
ActorEventSource.Current.ActorMessage(this, $"{args.Data} added at {args.Path}");
},
changed: (sender, args) =>
{
ActorEventSource.Current.ActorMessage(this, $"{args.OldData} changed to {args.Data} at {args.Path}");
}
);
}
I then call MonitorRules() after the service is registered like so in the service's Main() method:
fabricRuntime.RegisterActor<RuleMonitor>();
var serviceUri = new Uri("fabric:/MyApp.RuleEngine/RuleMonitorActorService");
var actorId = ActorId.NewId();
var ruleMonitor = ActorProxy.Create<IRuleMonitor>(actorId, serviceUri);
ruleMonitor.MonitorRules();
This "works" in that the service opens a connection to Firebase and responds to data changes. The problem is that since the service is run on three nodes of a five node cluster, it's actually listening three times and processes each message three times. Also, if there is no activity for a while, the service is deactivated and no longer responds to changes in Firebase. All in all, not the right way to set something like this up I'm sure, but I can not find any documentation on how to set up a polling client like this in service fabric. Is there a way to set this up that will adhere to the spirit of azure service fabric?
Yeah, there are a few things to familiarize yourself with here. The first is the Actor lifecycle and garbage collection. Tl;dr: Actors are deactivated if they do not receive a client request (via ActorProxy) or a reminder for some period of time, which is configurable.
Second, Actors have Timers and Reminders that you can use to do periodic work, like polling a database for changes. The difference between a timer and reminder is that a timer doesn't count as "being used" meaning that the actor can still be deactivated which shuts down the timer, but a reminder counts as "being used" and can also re-activate a deactivated actor. The way to think about timers and reminders is that you're doing the polling, rather than waiting for a callback from something else like you have here with FireSharp.
Finally, calling MonitorRules from Main() is not the best idea. The reason is that Main() is actually the entry point for your actor service host process, which is just an EXE that is used to host instances of your actors. The only thing that should happen in Main() is registering your actor type and nothing else. Let's look at what's happening here in more detail:
So you deploy your actor service to a cluster. The first thing that happens is we start the host process on as many nodes as necessary to run the actor service (in your case that's 3). We enter Main() where the actor service type gets registered and at this point, that's all we should do, because once the actor service is registered with the host process, we'll then create an instance (or multiple instances or replicas if it's stateful) of the service, and then the service can start doing its work. For actors, that means the actor service is ready to start activating actors when a client application makes a call using ActorProxy. But with the ActorProxy call you have in Main(), you're basically saying "activate an actor on every node where this host is when the host starts" which is why you're listening three times.
With all that in mind, the first question to ask yourself is whether actors are the right model for you. If you just want a simple place to monitor Firebase with a FireSharp client, it might be easier just use a reliable service instead because you can put your monitoring in RunAsync, which is started automatically when the service starts, unlike actors which need a client to activate them.
In CF10 the new operator calls init implicitly and we need not call it explicitly while creating an object.
But what happens if I have a REST service with an init method; do the properties defined in init become available inside any REST resource?
My tests shows they do not. Ultimately the REST call is invoking a function and I believe that each implicit function invocation results in object creation behind the scenes.
Is REST different than normal object creation/instantiation? Can an init even be created (it can be, but is it a workable/good practice)?
I'm just beginning to look more closely at REST. One thing that I will point out is that REST is stateless. Take a look at the wikipedia description. Each request stands alone, the client should maintain any state that is necessary, and pass that in with the request. The service just responds to requests. I'd say that having an init() in your RESTful service is neither a good nor a bad practice. It might be a handy place to collect function calls and variable settings that you would commonly use during requests. Then when you pass your state in with a request, that method could then call the init(), then go on about its business.