Is it best practice to place monitoring tools like Prometheus and Grafana inside a Kubernetes cluster or outside a Kubernetes cluster?
I can see the case for either. It seems very easy to place it inside the cluster. But that seems less robust.
It seems people do this typically, likely they are running everything in their environment or app under K8S. In a bigger picture view if you have use cases outside of one specific app it likely makes sense to run this on another architecture. The reason why is that Prometheus doesn't support clustering. You can write to two instances, but there is not really an HA plan for this product. To me, that's a major problem for a monitoring technology.
Most organizations who use this tool heavily end up not meeting use cases which APM (transaction tracing and diagnostics) can do. Additionally, you'll need to deploy an ELK/Splunk type stack, so it gets very complex. They also find it difficult to manage and often will look towards a Datadog, SingalFx, Sysdig, or another system which can do more and is fully managed. Naturally, most of what I have said here has cost, so if you do not have a budget then you'll have to spend your time (time=money) doing all the work.
Related
Sorry if this question might sound "convoluted" but here it goes...
I'm currently designing a k8s solution based on Firecracker and Kata-containers. I'd like the environment to be as isolated/secure as possible. My thoughts around this are:
deploy k8s masters as Firecracker nodes having API-server,
Controller, Scheduler and etcd
deploy k8s workers as Firecracker nodes having Kubelet, Kube-proxy and using Kata-containers + Firecracker for
deployed workload. The workload will be a combination of MQTT cluster components and in-house developed FaaS components (probably using OpenFaaS)
It's point 2 above which makes me feel a little awkward/convoluted. Am I over complicating things, introducing complexity which will cause problems related to (CNI) networking among worker nodes etc? Isolation and minimizing attack vectors are all important, but maybe I'm trying "to be too much of a s.m.a.r.t.a.s.s" here :)
I really like the concept with Firecrackers microVM architecture with reduced security risks and reduced footprint and it would make for a wonderful solution to tenant isolation. However, am I better of to use another CRI-conforming runtime together with Kata for the actual workload being deployed on the workers?
Many thanks in advance for your thoughts/comments on this!
You might want to take a look at https://github.com/weaveworks-liquidmetal and consider whether contributing to that would get you further towards your goal? alternative runtimes (like kata) for different workloads are welcomed in PR’s. There is a liquid-metal slack channel in the Weaveworks user group of you have any queries. Disclosure I currently work at Weaveworks :)
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I am totally new to this two technologies (I know docker and kubernetes btw).
Haven't find much an the web about this comparison topic.
I have read that Openshift is used by more companies,but a nightmare to install,pricier and on upgrade data loss can occur.
But nothing else.
What should be the deciding factor for which one to use for kubernete cluster orchestration?
I currently work for Rancher. I've also been building Internet infrastructure since 1996 and owned an MSP for 14 years that built and managed Internet datacenters for large US media companies. I've been working with containers since 2014, and since then I've tried pretty much everything that exists for managing containers and Kubernetes.
"The deciding factor" varies by individual and organization. Many companies use OpenShift. Many companies use Rancher. Many companies use something else, and everyone will defend their solution because it fits their needs, or because of the psychological principle of consistency, which states that because we chose to walk a certain path, that path must be correct. More specifically, the parameters around the solution we chose must be what we need because that was the choice we made.
Red Hat's approach to Kubernetes management comes from OpenShift being a PaaS before it was ever a Kubernetes solution. By virtue of being a PaaS, it is opinionated, which means it's going to be prescriptive about what you can do and how you can do it. For many people, this is a great solution -- they avoid the "analysis paralysis" that comes from having too many choices available to them.
Rancher's approach to Kubernetes management comes from a desire to integrate cloud native tooling into a modular platform that still lets you choose what to do. Much like Kubernetes itself, it doesn't tell you how to do it, but rather gives fast access to the tooling to do whatever you want to do.
Red Hat's approach is to create large K8s clusters and manage them independently.
Rancher's approach is to unify thousands of clusters into a single management control plane.
Because Rancher is designed for multi-cluster management, it applies global configuration where it benefits the operator (such as authentication and identity management) but keeps tight controls on individual clusters and namespaces within them.
Within the security boundaries Rancher gives developers access to clusters and namespaces, easy app deployment, monitoring and metrics, service mesh, and access to Kubernetes features without having to go and learn all about Kubernetes first.
But wait! Doesn't OpenShift give developers those things too?
Yes, but often with Red Hat-branded solutions that are modified versions of open source software. Rancher always deploys unadulterated versions of upstream software and adds management value to it from the outside.
The skills you learn using software with Rancher will transfer to using that same software anywhere else. That's not always the case with skills you learn while using OpenShift.
There are a lot of things in Kubernetes that are onerous to configure, independent of the value of using the thing itself. It's easy to spend more time fussing around with Kubernetes than you do using it, and Rancher wants to narrow that gap without compromising your freedom of choice.
What is it that you want to do, not only now, but in the future? You say that you already know Kubernetes, but something has you seeking a management solution for your K8s clusters. What are your criteria for success?
No one can tell you what you need to be successful. Not me, not Red Hat, not Rancher.
I chose to use Rancher and to work there because I believe that they are empowering developers and operators to hit the ground running with Kubernetes. Everything that Rancher produces is free and open source, and although they're a business, the vast majority of Rancher deployments make no money for Rancher.
This forces Rancher to create a product that has true value, not a product that they can convince other people to buy.
The proof is in the deployments - Red Hat has roughly 1,000 OpenShift customers, which means roughly 1,000 OpenShift deployments. Rancher has fewer paying customers than Red Hat, but Rancher has over 30,000 deployments that we know about.
You can be up and running with Rancher in under ten minutes, and you can import the clusters you already have and start working with them a few minutes later. Why not just take it for a spin and see if you like it?
I also invite you to join the Rancher Users slack. There you will not only find a community of Rancher users, but you will be able to find other people who compared Rancher and OpenShift and chose Rancher. They will be happy to help you with information that will lead you to feel confident about whatever choice you make.
I have some apps in production working in Azure. All these applications belong to the same company and communicate with each other. I want to migrate them to Kubernetes.
My question is: What are the best practices in this case and why ?
Some peoples recommend one cluster and multiples namespaces and I don't know why.
For example: https://www.youtube.com/watch?v=xygE8DbwJ7c recommends apps within a cluster doing intra-cluster multi-tenancy but the arguments of this choice are not enough for me.
My question is: What are the best practices in this case? and why ?
Answer is: it depends...
To try to summarize it from our experience:
Cluster for each app is usually quite a bit waste of resources, especially giving HA clusters requirements, and it can mainly be justified in case when single app is comprised of larger number of microservices that are naturally clustered together or when some special security considerations has to be taken into account. That is, however, in our experience, rare the case (but it depends)...
Namespaces for apps in a cluster are more in line with our experience and needs, but again, this should not be overdone either (so, again it depends) since, for example your CNI can be bottleneck leading to one rogue app (or setup) degrading performance for other apps in seemingly unrelated case. Loadbanalcing and rollout downtimes, clashes for resources and other things can happen if all is crammed into one cluster at all cost. So this has it's limits as well.
Best of both worlds - we started with single cluster, and when we reached naturally separate (and separately performant) use cases (say, qa, dev, stage environments, different client with special security considerations etc) we migrated to more clusters, keeping in each cluster reasonably namespaced apps.
So all in all: depending on available machine pool (number of nodes), size of the cluster, size of apps themselves (microservice/service complexity), HA requirements, redundance, security considerations etc.... you might want to fit all into one cluster with namespaced apps, then again maybe separate in several clusters (again with namespaced apps within each cluster) or keep everything totally separate with one app per cluster. So - it depends.
It really depends on the scenario. I can think of one scenario where some of the apps need dedicated higher configuration nodes (Say GPU).
In such scenarios having a dedicated cluster with GPU nodes can be beneficial for such apps. And having a normal CPU nodes for other normal apps.
I have been reading about Docker for a long time and tried few examples. While going through blogs, I didn't find any unanimous answer as whether a product having multiple components like JEE deployables, database, webserver etc should be deployed, in one single container or on different containers. Can someone please provide detailed answer in terms of
Manageability
Complexity
Risk (like data loss, security etc.)
Any other point, welcome
Also will it be worth going Kubernetes route or Docker is still sufficient?
There are alternative views (that may work well depending on your use case) but the official docker stance is one process per container. From my experience, you'll be able to fit into the docker ecosystem and re-use things more effectively if you go with the grain on that one. However, there are alternative solutions (again, that may work well depending on your use case) such as passenger-docker.
I'm developing a web app on the MEAN stack (and probably other stuff, like some way of storing images).
In the startup accelerator I'm working in I've been suggested to let go the IAAS approach for a PAAS one (namely AWS). I must admit being used to working on small scale apps on single virtual machines I'm very confused about how to approach the task and the whole PAAS thing.
Reading through AWS documentation looks like Elastic Beanstalk is the way to go for building a scalable web app. From my understanding it abstracts away the infrastructure management workload taking care of load balancing and resource scaling.
I decided to give it a try.
Now I'm a bit confused on how to setup the infrastructure. Particularly I'm wondering how to fit MongoDB it the puzzle.
I guess I shouldn't install it on the node.js machine but on a different one, so that the two can scale out depending on needs.
My questions are:
where and how should I install Mongo?
should I let go MongoDB in favour of something like DynamoDB? in this case how can I set up a local development environment?
Any suggestions would be appreciated.