About the Microsoft.ServiceFabric.Actors.Runtime.Actor, I know it single-threaded execution from the document "An actor is an isolated, independent unit of compute and state with single-threaded execution. ".
Is Microsoft.ServiceFabric.Actors.Runtime.ActorService thread-safe? There are few document about it. We used it in our application, It seems not thread-safe in multi-nodes and multi-instance. Does anyone know about this?
Reliable Actors are packaged in Reliable Services (ActorService) that can be deployed in the Service Fabric infrastructure. Actor instances are activated in a named service instance.
ActorService is just a Stateful Service. Services allow concurrent access.
edit:removed wrong remark about StateManager
What issues are you seeing?
Related
I am using vert.x as an api gateway to route calls to downstream services.
As of now, I am using single web client instance which is shared across multiple verticles (injected through guice)
Does it make sense for each verticle to have it's own webclient? Will it help in boosting performance? (My each gateway instance runs 64 vericles and handles approximately 1000 requests per second)
What are the pros and cons of each approach?
Can someone help to figure out what's the ideal strategy for the same?
Thanks
Vert.x is optimized for using a single WebClient per-Verticle. Sharing a single WebClient instance between threads might work, but it can negatively affect performance, and could lead to some code running on the "wrong" event-loop thread, as described by Julien Viet, the lead developer of Vert.x:
So if you share a web client between verticles, then your verticle
might reuse a connection previously open (because of pooling) and you
will get callbacks on the event loop you won't expect. In addition
there is synchronization in the web client that might become contented
when used intensively from different threads.
Additionally, the Vert.x documentation for HttpClient, which is the underlying object used by WebClient, explicitly states not to share it between Vert.x Contexts (each Verticle gets its own Context):
The HttpClient can be used in a Verticle or embedded.
When used in a Verticle, the Verticle should use its own client
instance.
More generally a client should not be shared between different Vert.x
contexts as it can lead to unexpected behavior.
For example a keep-alive connection will call the client handlers on
the context of the request that opened the connection, subsequent
requests will use the same context.
I'm new to Service Fabric Reliable Actors technology and trying to figure out best practices for this specific scenario:
Let's say we have some legacy code that we want to run new code built on SF Reliable Actors. Actors of certain type "ActorExecutor" are going to asynchronously call some third-party service that sometimes could stuck for pretty long time, longer than actor's calling client is ready to wait, or even experience some prolonged underling communication issues. We do not want client (legacy code) to get blocked by any sort of issues in ActorExecutor, it does not expect to receive any value or status back from actor. Should we use SF ReliableQueue for that? Should we use some sort of actor-broker to receive requests from client and storing them to queue: Client->ActorBroker->ActorExecutor? Are reminders could be helpful here?
One more question in this regard: Giving the situation is possible when many thousands of actors might stuck in 'third-party incomplete call' in the same time, and we want to reactivate and repeat the very last call for them, should we write a new tool for that? In NServiceBus you can create an error queue in MSMQ where all failed like 'unable to process' messages to be landed, and then we were able to simply re-process them anytime in the future. From my understanding, there is no such thing in Service Fabric and it's something we need to built on our own.
An event driven approach can help you here. Instead of waiting for the Actor to return from the call to a service, you can enqueue some task on it, to request it to perform some action. The service calling Actor would function autonomously, processing items from it's task queue. This will allow it to perform retries and error handling. After a successful call, a new event can notify the rest of the system.
Maybe this project can help you to get started.
edits:
At this time, I don't believe you can use reliable collections in Actors. So a queue inside the state of an Actor, is a regular (read-only) collection.
Process the queue using an Actor Timer. Don't use the threadpool, as it's not persistent and won't survive crashes and Actor garbage collections.
Hi I have some reliable actors which I want to backup.
I'm thinking of full backup every day and incremental backup once the state changes and maybe before each deployment. Reading the documentation and example didn't help me because there are using services.
I found an example on github: https://github.com/Microsoft/azure-docs/blob/master/articles/service-fabric/service-fabric-reliable-actors-platform.md
But I have some doubts using the remoting contract and call all actors. This will block the actor since its kind of single threaded. Is this really the best practice?
Maybe it will be beter to forward all the changes to an event hub and the store it in a real database. Or should I use a reminder which will trigger the backup task.
Reliable Actors abstraction is built on ActorService which. is an implementation of StatefulServiceBase (Reliable Service). As explained in the Backup and restore Reliable Actors, you can implement a custom ActorService to get access to the same BackupAsync and RestoreAsync APIs as if you were building a Reliable Service. So you can program your ActorService to backup periodically as you were thinking. Note that when an ActorService backups up, it backs up all the Actors that reside on that replica. Hence, backing up every time an Actor's state changes might be expensive. I would recommend deciding on the acceptable Recovery Point Objective for your application and incrementally backing up at the relevant periodicity.
More information on how to build a custom ActorService can be found in Custom Actor Service section.
Please note that in the current release, ActorService does not support incremental backups: "The KvsActorStateProvider currently only supports full backup" Backup and Restore Reliable Actors.
Say you have 30+ console applications running on the Windows machine which can invoked manually or through Windows Scheduled Tasks, what would be recommended way to move to/implement them in Service Fabric?
One way of implementing this would be as one Service Fabric application with many stateless services (Reliable Actor using Timers/Reminders) each listening to the Service Bus queue/topic, and then use Azure Scheduler to send messages to the queue/topic.
What would be the pros/cons of such implementation? This article seems to list few of them.
What would be other ways to implement this?
Seems like some people are trying to advocate for including pub/sub framework into Service Fabric, if that becomes part of Service Fabric would that a valid option?
I would look at using Azure Functions, this would be great for simplicity and trendy being Serverless compute, meaning no need to spin up and configure a bus or queue, then use Stateless reliable API services and have the Azure timed Function call the stateless service directly.
See here for a start:
https://azure.microsoft.com/en-us/services/functions/
This Video is doing a timer with db clean up by no reason why this couldn't be a HTTP call.
Video
I like your idea of converting the console applications to actors and using reminders. However, I don't see the need for Service Bus or the Azure Scheduler.
It seems to me that you only need to expose a few API methods on the actors. One to create/modify the run schedule, and a second that would allow the actor to be manually/immediately invoked (while still maintaining turn-based concurrency). The actor could store its full schedule internally, but it only only ever needs to calculate the next time to execute - and set the reminder accordingly.
Also, keep in mind that actor reminders are triggered under all circumstances, whereas a timer stopped if Service Fabric deactivates the actor.
I am new to Azure Service Fabric and the biggest questions I have are
When should I use reliable actors? Give me practical examples please.
When should I use reliable services? Give me practical examples please.
Taken a look at the differences:
State analogy : Actors work on a single instance of an object graph.
Services usually have state for multiple callers.
Scope : Actors can’t work alone, because of their size (more like objects).
Life-cycle : Actors are only active when used, so
more will fit on your available server resources
Concurrency : Actors
enforce single threaded access
State : Actors just modify the
aggregate, services work on sets so often use transactions on sets
for ACID behavior.
Communication : Actors communicate through
channels provided by the platform. Services may choose otherwise.
Access : Actors in the cluster can’t be reached from the outside by
default. You’ll probably need a Service that provides access.
Samples when to use an actor:
For every user of your mobile app you could have one actor.
For every thermostat that sends information to your application you could have one actor.
For every customer of your e-commerce site, you could have one shopping-basket actor.
Create a service in the cases that you are probably used to. Create a reliable service that provides a service for multiple users at once. For example a weather service.
I don't mean to use a word to define itself, but use Reliable Actors only if you've determined your problem fits the actor design pattern. Actors are a design pattern much like many of the Gang of Four design patterns. If your problem fits one of the patterns, use it. If it doesn't, it's best not to try to shoehorn your problem into the wrong pattern.
In Service Fabric, Reliable Actors are an implementation of the Virtual Actor pattern. It has certain rules of operation and caveats that go with them. This is a good doc to read to get an idea of how the Reliable Actor framework works and whether or not it meets your requirements: https://azure.microsoft.com/en-us/documentation/articles/service-fabric-reliable-actors-platform/
Reliable Actors are in fact just a framework built on top of Reliable Services, so all the same scaling, partitioning, and distribution rule apply.