Scenario:
I have two groups of backend servers. And then I set weight 1 to group A, and set weight 2 to group B. The backends in group A are located at 172.16.11.XXX, the backends in group B are located at 172.16.12.XXX.
Question:
I don't find a wildcard for specifying the backends in haproxy documentation.
And I don't want add another haproxy layer in the middle of frontend and backends. Any suggestion ids appreciated!
Thanks
you can create happroxy backend and putt all A group server in 1 backend and group B will be backend 2 after that create master upstream and put both backend and give priority weight 1 , weight 2
Related
I want to build a system with the following data replication requirements.
In the image attached:
Node 1 has 2 entities Entity 1 and Entity 2.
Each entity has multiple rows of data say (Row1, Row2, Row3)
Node 2 and Node 3 are a full replica of Node1 and possibly in the same data center.
Node 4 sits in a different place altogether and has only Row 1 from Entity1 and Entity2.
Node 5 sits in another place and has only Row2 from Entity 1 and Entity2.
The idea is Node4 and Node5 will be in the geographic vicinity of the consumer system and the consumer can communicate with local copies in Node 4 and Node5 if the network is down.
On a normal business day - Its acceptable to limit all writes to Node1 and allow Node 4 or Node 5 to do the write only when Node 1 is down.
I am not sure which Database can support this without extensive management through code.
Data Model Replication
So far I have found this:
Cassandra can do keyspace based replication but it might be tricky as I have 2000+ remote locations for partial data. I can think of having to say 200 keyspaces with 10 locations sharing same keyspace, thus creating less overhead, even though data copied to local nodes will not always be useful to them.
Mongodb has an open request for this feature (https://jira.mongodb.org/browse/SERVER-1559)
Couchbase has XDCR based filtering, which looks like a potential solution.
Can you please suggest if my understanding is correct?
Yes, Couchbase XDCR is a viable solution. You could
1. set up Node 1, Node 4, and Node 5 as three separate data clusters
2. set up a uni-directional XDCR from Node 1 to Node 4 with a filtering expression that matches only Row 1
3. set up a uni-directional XDCR from Node 1 to Node 5 with a filtering expression that matches only Row 2.
For more information, please refer to https://docs.couchbase.com/server/6.0/learn/clusters-and-availability/xdcr-overview.html.
XDCR filtering is at: https://docs.couchbase.com/server/6.0/learn/clusters-and-availability/xdcr-filtering.html
I would like to be able to use leastconn in HAProxy while still having different weights on backend hosts.
Context: I have ~100 backend hosts that are being accessed by ~2000 front end hosts. All backend hosts process requests the same way (no faster hosts), however some backend hosts can process more requests (they have more cores). The problem is that I cannot use round robin as it is, because sometimes some backend host get stuck with long connections, and with round robin it will keep receiving more and more front end connections, which it never recovers from. In the current situation, I use leastconn, so all backend hosts process ~ the same number of requests, but I don't optimize their CPU usage.
What I would like to achieve is to be able to still use leastconn, but allowing more connections to certain hosts. For example if we have only 2 hosts: host A with 1 core and host B with 2 cores. At any moment, I would like HAProxy to decide which host to pick based on:
x= num_current_connections_A, y = 0.5*num_current_connections_B. If x<=y go to A, otherwise go to B.
I read this post which states the same issue, but no answer really solved my problem: http://haproxy.formilux.narkive.com/I6hSmq8H/balance-leastconn-does-not-honor-weight
Thank you
I have Azure Service Fabric stateless service which doesn't use any endpoints. It takes message from queue, processing it and save results to db.
I want to deploy 10 instances on my 2 nodes. By default I have -1 instancecount - it means that there will be 2 instances for 2 nodes. I can specify instancecount as 1 or 2 and it will be OK, but I cannot set 10 instances, it gives me error.
So I decide to create another instance of my application type. Is it right solution? Is there more elegant way to do this?
There are a few ways you can currently choose from:
Multiple Application instances (as you're doing). Multiple Applications hosting a service. Increases complexity, because instances must be managed.
Multiple Services. One application, hosting multiple services. Same downside as #1.
Multiple Partitions within a service (instead of one SingletonPartition). Downside of this, is that this number is 'fixed'. Changes require redeployment of the type with some downtime.
Use multiple receivers inside one service. Probably a good option, because it gives the least overhead, as creating multiple processes (#1, 2 & 3) creates some overhead.
(an important question is: do you really require multiple instances?)
More info here.
Can I have nodes with different machine specifications (e.g, low
performer, medium performer, high performer) in my cluster?
Can I target a partition to run on a specific node?
Can I specify I may use up to 100 nodes, initially specify 10
partitions each running on its own node of the 100 (so 10 partitions
10 nodes), but over time drop and add partitions such that hours
later I'm using 5 partitions on 5 nodes and later 96 partitions on
96 nodes (all this ignores replicas)?
Yes, you can use Node Types. NodeType is the node definition used to create the cluster virtual machines. It is based on Virtual Machines Scale Sets, this scale set has the definition of OS, Memory, Disk, and so on. In you case you would create the nodeTypes low performer, medium performer, high performer, and can define how many instances(VMs) each nodeType will have. For more information, check here
On Service Fabric you have placement constraints where you define conditions for your services to be validated before deploying the service to a particular node, for example, one of the constraints you can create is (NodeType==MediumPerformer). This will make SF place your service on any node of Type MediumPerformer. The only caveat is that it use the same rule for all replicas and partitions, if you want different behavior, you would have to create new service named instances with different rules. For more information, check here
Service Partitions are immutable, so you won't be able to change the number of partitions after the service is deployed. You can bypass this limitation by creating multiple named services instead. For more information, check here
I have 3 x mongodb nodes configured as one shard (1 x primary and 2 x secondary) but I only have 2 data centers. If I host 1 x node in DataCenterA and 2 x nodes in DataCenterB and DataCenterB is down. Can I have anyway to get the node in DataCenterA to perform both read write instead of in read-only mode and once other nodes are back online the clusters can be retained?
Understand that the best practice is to have a third location to host each of node in one location but if I only have 2 x locations available do I have any way to make this option work?
Thanks a lot.
Yes you can by removing the node in data center A from the replica set and starting it back as a standalone node. You can add the node back once the other servers in the replica set are up.