HAProxy 1.8 adds an experimental nbthread option which has some benefits over nbproc. Is it possible to bind frontends to certain threads in the same way that bind-process binds them to processes?
Related
I have been testing with haproxy which does cookie based load balancing to our streaming servers, but lets say for example haproxy falls over (I know unlikely)
the streamer gets disconnected, is there a way of passing on the connection without it relying on haproxy, basically laving the streamer connected to the destination and cutting all ties with haproxy.
That is not possible by design.
HAProxy is a proxy (as the name suggests). As such, for each communication, you have two independent TCP-connections, one between the client and HAProxy and another between HAProxy and your backend server.
If HAProxy fails or you need to failover, the standing connections will have to be re-created. You can't pass over existing connections to another server since there is a lot of state attached to each connection that can't be transferred.
If you want to remove the loadbalancer from the equation after the initial connection initialization, you should look at Layer-3 loadbalancing solutions like LVS on Linux with Direct Routing. Note that these solutions are much less flexible than HAProxy. There is no such thing as a free lunch after all :)
I use postgresql as database. I have a master/slave with streaming replication. I want to use HAProxy for load balancing. I want to send the writes to the master, and the reads to the slave. Can I do this with haproxy?
No, you can't. HAProxy doesn't understand the PostgreSQL protocol so it has no idea what "reads" or "writes" are.
Take a look at PgPool-II, which can do this to a limited extent. In practice it's usually better to configure the application so it knows to route its read-only queries to a different server if possible.
We are doing it by defining a frontend for reading and other for writing, each one listening on different ports, and routing them to backends where you have your db cluster organized.
Example of HAProxy config:
frontend writes
bind *:5439
default_backend writes_db
frontend reads
bind *:5438
default_backend reads_db
backend writes_db
option pgsql-check user haproxy
server master_db ip.for.my.master:5432 check
backend reads_db
balance roundrobin
option pgsql-check user haproxy
server replica_db ip.for.my.replica:5432 check
In our case, we use Django so we need to define the routers and settings.databases so all write operations are done on one port of the HAProxy server (5438) and all read operations are done on the other one (5439).
I have an HAproxy instance used as load balancer of BOSH (http-bind, http://xmpp.org/extensions/xep-0206.html) servers. It was running with "roundrobin" load balancing method, but I experimented some issues, when some instances go down, all the connections are redistributed to the active instances. When the death nodes come up again, they don't have the same amount of connections that the other instances, and they aren't using the same resources. If other instances go down, the sessions will be redistributed again and some servers will be overloaded and some other that are running in their limits go down, so all the service is interrupted, and I need to restart all instances at the same time in order try that the the sessions could be evenly redistributed.
I was reading about how can I configure a BOSH load balancing using HAproxy and I found this book: "Professional XMPP Programming with JavaScript and jQuery". In this book the author recommends that we can use "leastconn" as balance method for Haproxy.
The HAproxy documentation says that we shouldn't use "leastconn" with HTTP connections, but it says that we should use it where very long sessions are expected.
I think that this balancing method can help with the issue when the servers go down, because it will redistribute the sessions equally in the active nodes, and when the instance is up again, all the new sessions will go to this instance until it has the same amount of sessions that the other servers.
Has anyone some experience in this kind of configuration? What HApoxy settings or tuning do you recommend me in order to balance BOSH connections?
If your sessions are long, and they may be when I read SMPP, then leastconn will provide a better load-balancing than roundrobin.
Roundrobin works well for very short connections.
cheers
I have posted this to ServerFault, but the Node.js community seems tiny there, so I'm hoping this bring more exposure.
I have a Node.js (0.4.9) application and am researching how to best deploy and maintain it. I want to run it in the cloud (EC2 or RackSpace) with high availability. The app should run on HTTPS. I'll worry about East/West/EU full-failover later.
I have done a lot of reading about keep-alive (Upstart, Forever), multi-core utilities (Fugue, multi-node, Cluster), and proxy/load balancers (node-http-proxy, nginx, Varnish, and Pound). However, I am unsure how to combine the various utilities available to me.
I have this setup in mind and need to iron out some questions and get feedback.
Cluster is the most actively developed and seemingly popular multi-core utility for Node.js, so use that to run 1 node "cluster" per app server on non-privileged port (say 3000). Q1: Should Forever be used to keep the cluster alive or is that just redundant?
Use 1 nginx per app server running on port 80, simply reverse proxying to node on port 3000. Q2: Would node-http-proxy be more suitable for this task even though it doesn't gzip or server static files quickly?
Have minimum 2x servers as described above, with an independent server acting as a load balancer across these boxes. Use Pound listening 443 to terminate HTTPS and pass HTTP to Varnish which would round robin load balance across the IPs of servers above. Q3: Should nginx be used to do both instead? Q4: Should AWS or RackSpace load balancer be considered instead (the latter doesn't terminate HTTPS)
General Questions:
Do you see a need for (2) above at all?
Where is the best place to terminate HTTPS?
If WebSockets are needed in the future, what nginx substitutions would you make?
I'd really like to hear how people are setting up current production environments and which combination of tools they prefer. Much appreciated.
It's been several months since I asked this question and not a lot of answer flow. Both Samyak Bhuta and nponeccop had good suggestions, but I wanted to discuss the answers I've found to my questions.
Here is what I've settled on at this point for a production system, but further improvements are always being made. I hope it helps anyone in a similar scenario.
Use Cluster to spawn as many child processes as you desire to handle incoming requests on multi-core virtual or physical machines. This binds to a single port and makes maintenance easier. My rule of thumb is n - 1 Cluster workers. You don't need Forever on this, as Cluster respawns worker processes that die. To have resiliency even at the Cluster parent level, ensure that you use an Upstart script (or equivalent) to daemonize the Node.js application, and use Monit (or equivalent) to watch the PID of the Cluster parent and respawn it if it dies. You can try using the respawn feature of Upstart, but I prefer having Monit watching things, so rather than split responsibilities, I find it's best to let Monit handle the respawn as well.
Use 1 nginx per app server running on port 80, simply reverse proxying to your Cluster on whatever port you bound to in (1). node-http-proxy can be used, but nginx is more mature, more featureful, and faster at serving static files. Run nginx lean (don't log, don't gzip tiny files) to minimize it's overhead.
Have minimum 2x servers as described above in a minimum of 2 availability zones, and if in AWS, use an ELB that terminates HTTPS/SSL on port 443 and communicates on HTTP port 80 to the node.js app servers. ELBs are simple and, if you desire, make it somewhat easier to auto-scale. You could run multiple nginx either sharing an IP or round-robin balanced themselves by your DNS provider, but I found this overkill for now. At that point, you'd remove the nginx instance on each app server.
I have not needed WebSockets so nginx continues to be suitable and I'll revisit this issue when WebSockets come into the picture.
Feedback is welcome.
You should not bother serving static files quickly. If your load is small - node static file servers will do. If your load is big - it's better to use a CDN (Akamai, Limelight, CoralCDN).
Instead of forever you can use monit.
Instead of nginx you can use HAProxy. It is known to work well with websockets. Consider also proxying flash sockets as they are a good workaround until websocket support is ubiquitous (see socket.io).
HAProxy has some support for HTTPS load balancing, but not termination. You can try to use stunnel for HTTPS termination, but I think it's too slow.
Round-robin load (or other statistical) balancing works pretty well in practice, so there's no need to know about other servers' load in most cases.
Consider also using ZeroMQ or RabbitMQ for communications between nodes.
This is an excellent thread! Thanks to everyone that contributed useful information.
I've been dealing with the same issues the past few months setting up the infrastructure for our startup.
As people mentioned previously, we wanted a Node environment with multi-core support + web sockets + vhosts
We ended up creating a hybrid between the native cluster module and http-proxy and called it Drone - of course it's open sourced:
https://github.com/makesites/drone
We also released it as an AMI with Monit and Nginx
https://aws.amazon.com/amis/drone-server
I found this thread researching how to add SSL support to Drone - tnx for recommending ELB but I wouldn't rely on a proprietary solution for something so crucial.
Instead I extended the default proxy to handle all the SSL requests. The configuration is minimal while the SSL requests are converted to plain http - but I guess that's preferable when you're passing traffic between ports...
Feel free to look into it and let me know if it fits your needs. All feedback welcomed.
I have seen AWS load balancer to load balance and termination + http-node-proxy for reverse proxy, if you want to run multiple service per box + cluster.js for mulicore support and process level failover doing extremely well.
forever.js on cluster.js could be good option for extreme care you want to take in terms of failover but that's hardly needed.
Running a web server on node.js is a simple thing to do (as seen by its excellent examples and documentation) but I wonder how you can fully use the CPU resources of a dedicated server?
Since node.js is single-threaded the only way to take advantage of multiple processors is via multiple processes. Of course, only one process can bind to a port so it seems there would have to be a master/worker pattern wherein the master forks children, binds to the incoming port, and delegates incoming connections (and the actual processing work) to the children. (Perhaps via a hungry-consumer pattern?)
Is this the best way to scale a web server running node.js? If so, are there libraries to simplify the master/worker pattern? If not, what patterns or deployment setups are recommended to best use the entire resources of a dedicated machine?
(Is this a better question for ServerFault?)
Multi-node is a library that provides the master/worker pattern.
If the server processes don't need to be able to talk to each other, and you aren't using Socket.IO, a simple option would be to just start one process/core, bind to local ports, and use something like nginx or HAProxy to load balance between them.
If you're using express, I'd use tj's Cluster: http://learnboost.github.com/cluster/
It provides 'transparent' cpu based load balancing, which is nice because you can use your existing express app, and it scales it across cores relatively painlessly.