I have a Kubernetes cluster running in the Google Cloud consisting of 4 servers. What is the easiest way to monitor networking latency/bandwidth between the servers ?
Is it possible to have a "scriptable" solution so I can repeat the deployment on different clusters in the future with minimal need for overhead ?
Thanks
PS - Kind of new at this so apologies if I didn't get the terms exact
Have a look at that post: http://paulbakker.io/docker/docker-cloud-network-performance/
They use a Docker Image on each host and test network performance between two machines. Maybe that's something you could do and even script ;)
Related
I am trying to set up a ZFS cluster on two nodes running Enterprise Storage OS (ESOS). This is based on Redhat, and running the newest ESOS production release (4.0.12).
I have been reading up on this for a bit, and think I finally understand that I have to use Corosync, DRBD and Pacemaker for this to be done correctly.
Though, I haven't done anything like this before, and still have some questions about the different modules.
The complete setup is like the following:
2 ESOS nodes running a ZFS active/passive cluster.
3 ESXi hosts connecting to this cluster using iSCSI. These are connected using fiber.
The 2 ESOS nodes got a dedicated 10G fiber link for synchronization.
First of, I am not able to find any answers to whether or not this configuration would ever be possible to archive, considering I am using ZFS.
If I understand what I have read correctly, you configure a shared iSCSI initiator address when this is set up. Then you use that on ESXi, where Corosync, DRBD & Pacemaker does the rest on the SAN side of things. Have I understood this correctly?
Corosync uses rings to communicate date between the two hosts (not so sure about this one, nor what it exactly means).
Do I need to use all three modules (Corosync, DRBD & Pacemaker), and in essence, what do they actually do.
In the different guides I have been reading, I have seen Asymmetric Logical Unit Access (ALUA) been mentioned a couple times. Is this possible to use to instruct iSCSI initiators which SAN node to use, and thereby not have to use a shared initiator?
Does anyone by any chance know of a website where someone has done something like this?
I will try this one tomorrow, and see if it helps me in the right direction: https://marcitland.blogspot.com/2013/04/building-using-highly-available-esos.html
Thanks.
Say you have 3 or more services that communicate with each other constantly, if they are deployed remotely to the same cluster all is good cause they can see each other.
However, I was wondering how could I deploy one of those locally, using minikube for instance, in a way that they are still able to talk to each other.
I am aware that I can port-forward the other two so that the one I have locally deployed can send calls to the others but I am not sure how I could make it work for the other two also be able to send calls to the local one.
TL;DR Yes, it is possible but not recommended, it is difficult and comes with a security risk.
Charlie wrote very well in the comment and is absolutely right:
Your local service will not be discoverable by a remote service unless you have a direct IP. One other way is to establish RTC or Web socket connection between your local and remote services using an external server.
As you can see, it is possible, but also not recommended. Generally, both containerization and the use of kubernetes tend to isolate environments. If you want your services to communicate with each other anyway being in completely different clusters on different machines, you need to configure the appropriate network connections over the public internet. It also may come with a security risk.
If you want to set up the environment locally, it will be a much better idea to run these 3 services as an independent whole. Also take into account that the Minikube is mainly designed for learning and testing certain solutions and is not entirely suitable for production solutions.
I deploying a python + tensorflow + flask application using a fully managed Google Cloud Run Service (1 vCPUs and 4 GB Ram).
System works fine but it is really slow, so I am evaluating ways of making it fast (it needs to run 20-30 times faster than what is doing now)
What would be the best approach?
To use a Kubernetes Cluster with one or two powerful machines
To use a Kubernetes Cluster with 3-5 weaker machines
To forget about Kubernets/Docker and run everything on single powerfull VM
Something else maybe?
For now I don't expect to have more than 10 users at a time but I want to be able to scale it up eventually.
You might want to evaluate according to your use case
Per this article, Fully managed Cloud Run is an ideal serverless platform for stateless containerized microservices that don’t require Kubernetes features like namespaces, co-location of containers in pods (sidecars) or node allocation and management.
GKE is a great choice if you are looking for a container orchestration platform that offers advanced scalability and configuration flexibility.
You mentioned you are looking the cheaper/easier method to develop, but this will probably not be as scalable, efficient or manageable, you might want to take a closer look at all cloud compute options in GCP to see what could benefit your use case the most.
You mentioned your use case is CPU intensive, so you might want to leverage the high CPU machine types, these might be used directly by creating a VM, creating an instance group or using them in other services like GKE or App Engine
I'm new to Akka Clusters, however as I am understanding its documentation, I need to know at least one "seed node" to join an existing cluster.
So when using clusters with OpenShift I would need to know if the current gear is the first node - then I would create a new cluster - or if there are already some other gears around - I would need to know at least one of their IPs to join them.
Is this possible with OpenShift cloud? (I'm using the DIY catridge, so customizing the start up script wouldn't be a problem. However I can't find any environment variable which provides me relevant data.)
DIY gears on OpenShift Online do not scale. And if you are spinning up separate applications for each of the nodes in your cluster, you are going to (probably) run into inter-gear communication issues. You might need to create your own akka cartridge (http://docs.openshift.org/origin-m4/oo_cartridge_developers_guide.html), then you can set your own scaling options. You might check out this cartridge (https://github.com/smarterclayton/openshift-redis-cart) which supports scaling and might give you some ideas about how to implement yours.
I'm trying to learn a deployment process that can guarantee a headackeless scaling of a meanjs application (not in the level that people do it in big companies, but also not at a hobby level).
So as long as I understood, this could be a solution to work on:
Having mongodb on digitalocean on Ubuntu
Having the meanjs application (all other than mongodb) in a docker
Then one can scale! Because mongodb could be clustered separately and docker keeps the scaling of the application easy.
Well, I know it sounds trivial and that's why I'm asking here: I just want to go and learn docker and want to be sure before investing time on the above assumed solution!
Do you think this guarantee an easy scaling, say, for a simple online multiplayer game on meanjs please? Thank you.
UPDATE 31/07/2018
Digital Ocean introducing Kubernates which does all the orchestration they have also released load balancer which I think will work well with kubernates
==============
There is no off the shelf solution.
You can use docker with swamp but for small deployment it brings additional issues of monitoring and networking.
So here is what I did:
Create a script to generate HAProxy config when you start/stop instance
Have mongo in a cluster or replica or whatever. Database usually does not need to be scaled dynamically. You just have single mongo server then you scale it up and when you can't scale it vertically anymore you scale it horizontally by creating replica set and then scale it up until you can't then you do sharding.
So have HAProxy as load balancer that accepts connections on port 80 and forwards to your droples oven private network.
You can also write scripts to use DO API to create an image with your deployment and fire it up once you have more traffic either dynamically by detecting response time or cpu load or whatever other metric you have or statically.
I hope this helps.