I'm investigating converting my service fabric application to kubernetes. One area I'm struggling with, is what the equivalent of service fabric actors is in kubernetes? I mainly use reminder actors in the application, is there an equivalent in kubernetes?
To use actor programming model on k8s you should use the combination of actor framework (orleans or akka.net) and k8s stateful sets.
Talk from Oslo NDC 2019 Real-Time, Distributed Applications with Akka.NET, Kubernetes and .NET Core
Have a look at dapr runtime. It is heavily influenced by Microsoft's learning of Service Fabric Reliable Services/Actors and runs on top of Kubernetes (and others).
Take into account that it is still in development and not yet released for production scenarios.
Related
I am looking at some strategies how to make bidirectional communication of applications hosted on seperate clusters. Some of them are hosted in Service Fabric and the others are in Kubernetes. One of the options is to use a DNS service on the Service Fabric and the counterpart on Kubernetes. On the other hand the Reverse Proxy seems to be a way to go. After going through the options I was thinking...what is actually the best way to create microservices that can be deployed either in SF or in K8s without worrying about the communication model which requires least changes if we wish suddenly to migrate one app from SF to K8s but still making it avaiable to the SF apps and vice versa?
We are in the process of migrating our Service Fabric services to Kubernetes. Most of them were "stateless" services and were easy to migrate. However, we have one "stateful" service that uses SF's Reliable Collections pretty heavily.
K8s has Statefulsets, but that's not really comparable to SF's reliable collections.
Is there a .NET library or other solution to implement something similar to SF's Reliable Collections in K8s?
AFAIK this cannot be done by using a .Net library.
K8 is all about orchestration. SF on the other hand is both an orchestrator + rich programming /application model + state management.
If you want to do something like reliable collection in K8 then you have to either
A) built your own replication solution with leader election and all.
B) use private etcd/cockroachdb etc. store
This article is pretty good in terms of differences.
https://learn.microsoft.com/en-us/archive/blogs/azuredev/service-fabric-and-kubernetes-comparison-part-1-distributed-systems-architecture#split-brain-and-other-stateful-disasters
"Existing systems provide varying levels of support for microservices, the most prominent being Nirmata, Akka, Bluemix, Kubernetes, Mesos, and AWS Lambda [there’s a mixed bag!!]. SF is more powerful: it is the only data-ware orchestration system today for stateful microservices"
However, they don't solve the coordination problem (saving data on the primary instance will automatically replicate to the others for recovery when the primary instance dies). That's what SF reliable collections does out of the box.
StatefulSets are valuable for applications that require one or more of the following.
Stable (persistent), unique network identifiers.
Stable, persistent storage.
Ordered, graceful deployment and scaling.
Ordered, automated rolling updates.
If your application doesn't require any of these, you should deploy your application using a Deployment.
There is a good Kubernetes guide on how to Run a Replicated Stateful Application here.
This page shows how to run a replicated stateful application using a StatefulSet controller. This application is a replicated MySQL database. The example topology has a single primary server and multiple replicas, using asynchronous row-based replication.
The StatefulSet controller starts Pods one at a time, in order by their ordinal index. It waits until each Pod reports being Ready before starting the next one. And it is used to perform orderly startup of MySQL replication by running Init Containers.
Operators can help
While operators are not necessary, they can help run stateful apps on Kubernetes with features like application-level HA management, backups, and restore.
You can use existing Operators or develop your own. The operator package includes all the configurations needed to deploy and manage the application from a Kubernetes point of view – from a StatefulSet to be used to any required storage, rollout strategies, persistence and affinity configuration, and more. Kubernetes will then rely on the operator to validate instances of the application against the specification to ensure it runs in the same way across instances in all clusters it is deployed in.
Some DB operators:
You can deploy MySQL database using Kubernetes operator developed by Oracle (currently is in a preview state):
https://github.com/mysql/mysql-operator
There’s also a PostgreSQL operator by Crunchydata to deploy PostgreSQL to Kubernetes:
https://github.com/CrunchyData/postgres-operator
MongoDB owns an operator to deploy MongoDB Enterprise to a Kubernetes cluster:
https://github.com/mongodb/mongodb-enterprise-kubernetes
You can find ready-made operators on OperatorHub.io to suit your use case.
I don't believe that it is a good idea to host a cluster of message queues that persist messages inside Kubernetes.
I don't think hosting any cluster inside kubernetes nor any stateful system inside Kubernetes is a good idea.
Kubernetes is designed for stateless restful services, to scale in and out, etc.
First of all, am I right?
If I am right, then
I just need some good valid points to convince some of my colleagues.
Thanks
Kubernetes was primarily designed for stateless applications, and managing state in Kubernetes introduces additional complexities that you have to manage (persistence, backups, recovery, replication, high availability, etc.).
Let's cite some sources (emphases added):
While it’s perfectly possible to run stateful workloads like databases in Kubernetes with enterprise-grade reliability, it requires a large investment of time and engineering that it may not make sense for your company to make [...] It’s usually more cost-effective to use managed services instead.
Cloud Native DevOps with Kubernetes, p. 13, O'Reilly, 2019.
The decision to use Statefulsets should be taken judiciously because usually stateful applications require much deeper management that the orchestrator cannot really manage well yet.
Kubernetes Best Practices, Chapter 16, O'Reilly, 2019.
But...
Support for running stateful applications in Kubernetes is steadily increasing, the main tools being StatefulSets and Operators:
Stateful applications require much more due diligence, but the reality of running them in clusters has been accelerated by the introduction of StatefulSets and Operators.
Kubernetes Best Practices, Chapter 16, O'Reilly, 2019.
Managing stateful applications such as database systems in Kubernetes is still a complex distributed system and needs to be carefully orchestrated using the native Kubernetes primitives of pods, ReplicaSets, Deployments, and StatefulSets, but using Operators that have specific application knowledge built into them as Kubernetes-native APIs may help to elevate these systems into production-based clusters.
Kubernetes Best Practices, Chapter 16, O'Reilly, 2019.
Conclusion
As a best practice at the time of this writing, I would say:
Avoid managing state inside Kubernetes if you can. Use external services (e.g. cloud services, like DynamoDB or CloudAMQP) for managing state.
If you have to manage state inside the Kubernetes cluster, check if an Operator exists exists for the type of application that you want to run, and if yes, use it.
I have worked on Kubernetes and currently reading about Service Fabric, I know Service Fabric provides microservices framework models like stateful, stateless and actor but other than that it also provides GuestExecutables or Containers as well which is what Kubernetes also does manage/orchestrate containers. Can anyone explain a detailed difference between the two?
You can see in this project paolosalvatori/service-fabric-acs-kubernetes-multi-container-appthe same containers implemented both in Service Fabric, and in Kubernetes.
Their "service" (for external ingress access) is different, with Kubernetes being a bit more complete and diverse: see Services.
The reality is: there are "two slightly different offering" because of market pressure.
The Microsoft Azure platform, initially released in 2010, has implemented its own Microsoft Azure Fabric Controller, in order to ensure the services and environment do not fail if one or more of the servers fails within the Microsoft data center, and which also provides the management of the user's Web application such as memory allocation and load balancing.
But in order to attract other clients on their own Microsoft Data Center, they had to adapt to Kubernetes, released initially in 2014, which is now (2018) either adopted or closely considered by... pretty much everybody (as reported in late December)
(That does not mean one is "better" than the other,
only that the "other" is more "visible" than the first ;) )
So it is less about "a detailed difference between the two", and more about the ability to integrate Kubernetes-based system on Microsoft Data Centers.
This is in line (source: detailed here) with Microsoft continued its unprecedented shift toward an open (read: non-proprietary) staging platform for Azure (with Deis).
And Kubernetes orchestrator is available on Microsoft's Azure Container Service since February 2017.
You can see other differences in their architecture of a deployed application:
Service Fabric:
Vs. Kubernetes:
thieme mentions in the comments the article "Service Fabric and Kubernetes comparison, part 1 – Distributed Systems Architecture", from Marcin Kosieradzki.
Both are different. Kubernetes manages rkt or other containers.
Service Fabric is not for managing containers. In case it manages some, that does not make it its purpose. That does not enable it for a comparison with Kubernetes.
eg: When a pod dies Kubernetes puts it to other nodes immediately. The part of SF that manages containers does not do this, it is done by some other area of Service Fabric. And outside containers. And was not designed with containers in mind.
I have a bunch of services deploying as guest executables to service fabric and all seems fine. I was wondering if there was any point in porting the services to be native Fabric Service services.
Looking at the documentation I can't seem to find any benefits of having implementing them as such, am I missing something obvious?
If your services are stateless there is probably no compelling reason to migrate them into native stateless services. It could be different if your services were stateful; in this context I mean that they store some state inside the process.
The state in native stateful services is stored redundantly, so your services can cope with node failures. This could increase the resilience of your service. In general, you usually create native services in green field situations and rely on guest executables and containers in migration/hybrid situations.
Guest Executable is missing out on some advanced features but it is up to you to decide if you need them.
Benefits of running a guest executable in Service Fabric
There are several advantages to running a guest executable in a Service Fabric:
High availability. Applications that run in Service Fabric are made highly available. Service Fabric ensures that instances of an application are running.
Health monitoring. Service Fabric health monitoring detects if an application is running, and provides diagnostic information if there is a failure.
Application lifecycle management. Besides providing upgrades with no downtime, Service Fabric provides automatic rollback to the previous version if there is a bad health event reported during an upgrade.
Density. You can run multiple applications in a cluster, which eliminates the need for each application to run on its own hardware.
Discoverability: Using REST you can call the Service Fabric Naming service to find other services in the cluster.
I.e. There is something called Stateless Reliable Services - http://www.jamessturtevant.com/posts/Service-Fabric-Service-Types/
The above link will explain it more.