I need to deploy Grafana in a Kubernetes cluster in a way so that I can have multiple persistent volumes stay in sync - similar to what they did here.
Does anybody know how I can use the master/slave architecture so that only 1 pod writes while the others read? How would I keep them in sync? Do I need additional scripts to do that? Can I use Grafana's built-in sqlite3 database or do I have to set up a different one (Mysql, Postgres)?
There's really not a ton of documentation out there about how to deploy statefulset applications other than Mysql or MongoDB.
Any guidance, experience, or even so much as a simple suggestion would be a huge help. Thanks!
StatefulSets are not what you think and have nothing to do with replication. They just handle the very basics of provisioning storage for each replica.
The way you do this is as you said by pointing Grafana at a "real" database rather than local Sqlite.
Once you do that, you use a Deployment because Grafana itself is then stateless like any other webapp.
Related
I deployed this week a Redis instance using Bitnami's Helm Chart into a GKE (Google Kubernetes Engine) cluster. Although I've been successful on this part, the challenge now is to make a failover disaster recovery strategy that replicates the data to another Redis instance in another GKE cluster (but same GCP project). How can I do this? I tried Persistence Volume Claims but they are only visible inside the cluster.
Redis Enterprise does have a WAN (multi-geo) replication mode, however I've never used it and it looks like it dramatically limits which features of Redis you can use to things that are compatible with a CRDT model. Basically first you need to answer how you would do this by hand, and then investigate automating that. WAN failover is a very complex thing, and generally you wouldn't even want to do it since you wouldn't fail over at the Redis level. Instead you would fail over your entire DC (or whatever you want to call your failure zones). Distributed database modelling and management is very tricky, here be dragons.
I have a MySQL database pod with 3 replicas.Now I'm making some changes in one pod(pod data,not pod configuration), say I'm adding a table.How will the change reflect on the other replicas of the pod?
I'm using kubernetes v1.13 with 3 worker nodes.
PODs do not sync. Think of them as independend processes.
If you want a clustered MySQL installation, the Kubernetes docs describe how to do this by using a StatefulSet: https://kubernetes.io/docs/tasks/run-application/run-replicated-stateful-application/#deploy-mysql
In essence you have to configure master/slave instances of MySQL yourself.
Pods are independent from each other, if you modify one pod the others will not be affected
As per your configuration - changes applied in one pod wont be reflected on all others. These are isolated resources.
There is a good practice to deploy such things using PersistentVolumeClaims and StatefulSets.
You can always find explanation with examples and best practices in Run a Replicated Stateful Application documentation.
If you have three mysql server pods, then you have 3 independent databases. Even though you created them from the same Deployment. So, depending on what you do, you might end up with bunch of databases in the cluster.
I would create 1 mysql pod, with persistence, so if one pod dies, the next one would take if from where the other one left. Would not lose data.
If what you want is high availability, or failover replica, you would need to manage it on your own.
Generally speaking, K8s should not be used for storage purposes.
You are good to have common storage among those 3 pods (PVC) and also consider STS when running databases on k8s.
I would like to know if it is possible for multiple pods in the same Kubernetes cluster to access a database which is configured using persistent volumes on a Google cloud persistent disk.
Currently I am building a microservices achitecture web app which has 3 node apis in different pods all accessing the same database. So how do I achieve this with kubernetes.
Kindly let me know if my architecture is right as well
You can certainly connect multiple node-based app pods to the same database. It is sometimes said that microservices shouldn't share a database but this depends on what your apps are doing, the project history and the extent to which you want the parts to be worked on separately.
There are questions you have to answer about running databases at scale, such as your future load and whether you want to use relational databases if you're going to try to span availability zones. And there are
some specific to kubernetes, especially around how you associate DB Pods to data. See https://stackoverflow.com/a/53980021/9705485. Another popular option is to use a managed DB service from a cloud provider. If you do run the DB in k8s then I'd suggest looking for a helm chart or looking at an operator, such as the kubeDB operator, to avoid crafting the kubernetes descriptors yourself and to get more guidance on running the DB and setting it up.
If it's a new project and you've not used k8s before then you'll also have to decide where to host your code, your docker images and your deployment descriptors and how to setup your CI pipelines. If you've not got answers to these questions already then I'd suggest looking at Jenkins-X as it will provide you with out of the box defaults for a whole cluster and CI setup and a template ('build pack') for building node apps and deploying them to staging and prod environments through a pipeline.
I am trying to deploy Kong in GKE as per the documentation https://github.com/Kong/kong-dist-kubernetes
I noticed cassandra is available as StatefulSet but Postgres as ReplicationController. Can I understand the difference? Also can anyone suggest how to choose between these 2?
ReplicationControllers predates StatefulSets. It was a way to manage your pod replicas. The 'newer' approach to manage your replicas is ReplicaSets which is used by Deployments.
StatefulSets is meant for applications that require your pods to start in an ordered way together with some sort of data stored on disk. So it's very suitable for master/slave datastore or ring topology datastores like Cassandra. I would strongly recommend using StatefulSets for these types of workloads.
StatefulSet is better for managing stateful applications (which postgres and cassandra definitely are) as it provides possibility to create PersistentVolumeClaim to use GKE PD in your case so your state will be stored on separate partition on dedicated PD. In comparison Postgres deployment using ReplicationController which you provided use emptyDir so it means when you delete by accident/failure POD with Postgres all data will be lost so in that case you will need probably to re-initialize your Kong deployment (run Kong migrations, configure routers, etc.)
When using Rubbernecks do I need to still use Containers for Oracle?
Here is my case, I am designing microservice architecture for my application using kubernetes.
I have deployed all my front-end, middleware and back-end services in different containers.
But for the data layer, I am not sure is it the right way to deploy Oracle in a container, as the database cannot be scaled horizontally. And if I want to scale the data container vertically in Kubernetes, How can I do that?
Generally how this is handled in the Kubernetes world?
You really need to come to terms with what you want/expect.
You can run a database on kube, specialy since StatefulSets. You might want to look into PersistentVolumes and PersistentVolumeClaims as well.
You can connect from kube services to an off cluster database managed in traditional way.
Which way you choose depends largely on what are you projects constraints and team experience or expected solution portability between envs and level of automation you want/have around it.