I have a web application spread over multiple servers and the incoming traffic is handled by HAProxy in order to balance the load. When we do the distribution, we do it at night because the users are much less and therefore we are less in service. To make the distribution we use the following strategy:
we shut down half of the servers
we deploy on servers that are turned off
we reactivate the servers that are turned off
we perform the same procedure on the other servers
The problem is that in any case I turn off the servers we close connections to users. Is there a better strategy for doing this? How could I improve this and avoid disservices and maybe be able to make distributions even during the day?
I hope I was clear. Thanks
I strongly suggest to use health checks for the servers.
Using HAProxy as an API Gateway, Part 3 [Health Checks]
You should have a URL ("/health") which you can use for health check of the backend server and add option redispatch to the config.
Now when you want to maintain the backend server just "remove" the "/health" URL and haproxy automagically routes the user to the other available servers.
Related
I would like to create a web service with GoLang that runs either on IIS (7, 8 or 10) or under Tomcat 7.0. We have multiple environments, each with multiple servers, all being Windows 2008 R2, 2012 or 2016. All servers are private (10.x). My goal is to add some REST services to a COTS product that uses both IIS and Tomcat. I'd prefer to avoid gluing nginx or something else onto either server at the risk of impairing the COTS product or having their tech support people not answer the phone. Again .. the servers are only accessible via corporate VPN and are not public internet-facing.
Which server would offer the easiest path to get something working -- Tomcat or IIS?
That's not really about Go, but still there exist at least two solutions I can think of:
Reverse proxying of HTTP requests.
Write a plain Go server serving requests via HTTP.
Maybe turn it into a proper Windows service using golang.org/x/sys/windows/svc.
Deploy it.
If it's to be hosted on the same machine which runs IIS, then it's fine to make it listen on localhost only. Note that it will need a dedicated TCP port to listen on, and you'll need to make it possible for your server to be somehow configurable in this regard.
Set up reverse proxying in your IIS so that it forwards requests coming to whatever (part of an) URL you want to the Go server.
Use FastCGI.
Go supports serving requests over the FastCGI
protocol by means of its standard library,
and IIS suports FastCGI workers.
So it's possible to (re-)write your Go server to use FastCGI
instead of HTTP and then hook it into IIS via this protocol.
The pros and cons of these solutions—as I view them—are:
Serving over plain HTTP is more familiar to a developer and
makes the server more "portable"—in the sense it will be easier to change its deployment scheme if/when you'll need it.
Right to making it available to the Internet directly.
Conversely, with FastCGI, you'll always need a FastCGI host as a "middleware".
There were rumors that HTTP code is more fine-tuned in terms
of performance than that of FastCGI.
Still, this only will be of concern for reasonably hard-core loads.
One possible upside of FastCGI over HTTP is that it can
be served over pipes rather than TCP. For instance, you might
get it served over named pipes as it's supported by IIS's FastCGI module and there exists 3rd-party packages for Go implementing support for them
(even including one from Microsoft®).
The upside of this is that pipes are beleived to incur lesser overhead for data transfer (basically it's just shoveling bytes between in-kernel buffers belonging to two processes instead of pushing them through the whole TCP/IP stack), and using pipes frees you from the need of dedicating a TCP port to the Go server.
Still, I have no personal experience with this kind of setup and its performance trade-offs.
I'm building a software agent that run on a server, this software agent act as a server manager i.e. starting/stoping Docker container, monitoring etc.
This server will host/serve many services, these services are programs running in Docker container, 1 program/service per container.
There may be so many servers and these servers aren't necessary be a high performance server, they ranges from a small VM to high performance computer. Right now, I assume that every service uses HTTP to serve request.
The function that I want to implement in this software agent is tracking the number of clients that are currently connecting (requesting) to server for every service (e.x. server A is processing 500 requests) or specific program is ok (e.x. program A is processing 100 requests, program B is processing 200 request).
I want to know this number because I want to do workload balancing across servers that host the same service.
The following is ideas that I have.
Implementing load balancer/reverse proxy inside this agent (I would use this load balancer https://github.com/nwoodthorpe/Load-Balancer-Golang). This may be the last choice because I think it will use pretty much resources for load balancing.
Letting service programs that are running on server tell agent whenever they start and finish processing request. I simply implement UDP socket server in agent to listen for a datagram that tell unique ID of request (actually, can be anything that help me distinguish specific request that being processed) and status whether is being processed or finish processing.
So, I would like to ask for a suggestion for above approaches, which one is better or how should I implement it? Is there any better approach to do this?
I'm trying to think through the advantages and disadvantages of haproxy health checks happening on a different port from regular traffic.
If a server becomes overloaded having health checks on a different port may mark the server as being up even when overloaded. I think this is a good thing because taking servers offline may make an overloading problem worse, but want to confirm that that makes sense. I can't seem to find any good docs on the tradeoffs though and was wondering if someone has a good analysis on the tradeoffs.
The port keyword is often used with address to send health checks somewhere else than directly to the service you are checking. One example might be enabling option httpchk to monitor a non-HTTP service. What you then do is have a HTTP-compatible service that when queried can execute complex health checks against the service you are actually testing.
The above is often done with agent-check nowdays, but some people prefer to use an HTTP interface.
This also has nothing to do with server load, the only idea is to send health checks to some other service, not the one directly monitored, which is more capable of testing the actual service (possibly by using a more-complex logic) and returning a result. As an example, one could have a MySQL backend which instead of being tested just for authentication by option mysql-check, could be tested by a PHP script that, for example, checks if backup is running and if it is returns a 5xx HTTP error. The configuration could be something like:
backend mysql
mode tcp
option httpchk GET /mysql-status.php
server mysqlserver 10.0.0.1:3306 check port 80
we’ve got a strange little problem we’re experiencing for months now:
The load on our cluster (http, long lasting keepalive connections with a lot of very short (<100ms) requests) is distributed very uneven.
All servers are configured the same way but some connections that push through thousands of requests per second just end up being sent to only one server.
We tried both load balancing strategies but that does not help.
It seems to be strictly keepalive related.
The misbehaving backend has the following settings:
option tcpka
option http-pretend-keepalive
Is the option http-server-close made to cover that issue?
If I get it right it will close and re-open a lot of connections which means load to the systems? Isn't there a way to keep the connections open but evenly balance the traffic anyway?
I tried to enable that option but it kills all of our backends when under load.
HAProxy currently only support keep-alive HTTP-connections toward the client, not the server. If you want to be able to inspect (and balance) each HTTP request, you currently have to use one of the following options
# enable keepalive to the client
option http-server-close
# or
# disable keepalive completely
option httpclose
The option http-pretend-keepalive doesn't change the actual behavior of HAProxy in regards of connection handling. Instead, it is intended as a workaround for servers which don't work well when they see a non-keepalive connection (as is generated by HAProxy to the backend server).
Support for keep-alive towards the backend server is scheduled to be in the final HAProxy 1.5 release. But the actual scope of that might still vary and the final release date is sometime in the future...
Just FYI, it's present in the latest release 1.5-dev20 (but take the fixes with it, as it shipped with a few regressions).
Does anyone have experience deploying GWT apps to EC2?
If I were to install tomcat or apache on a ec2 instance, could I have users connect directly to a url pointing there?
Would that be cost effective, or would java hosting services be best?
Is there any downside to hosting the edge HTTP server on a regular hosting service and have that direct requests to EC2? Performance ever an issue here?
Other answers are correct but I just wanted to share the fact that we are are developing a product that is 100% EC2/S3 based and also have a pure GWT front end.
We use maven2 for builds and the excellent gwt-maven plugin. This makes it easy to produce a WAR package of our web application as output. We use Jetty but Tomcat would work just as well.
We have pound (a http accelerator/load balancer) running on the VM listening for http & https, which then forwards to requests to lighttpd (static) or jetty (app). This also simplifies SSL certificates because pound handles SSL. I've found Java servers have always been a pain to configure with SSL certs.
Yes, you can host pretty much whatever you want, as you effectively have a dedicated Linux machine at your command.
As I last recall, the basic rate for an EC2 instance, on their "low end box" worked out to around $75/month, so you can use that as a benchmark against other vendors. That also assumed that the machine is up 24x7 (since you pay for it by the hour).
The major downside of an EC2 instance is simply that it can "go away" at any time, and when it does, any data written to your instance will "go away" as well.
That means you need to set it up so that you can readily restart the server, but also you need to offline any data that you generate and wish to keep (either to one of Amazons other services, like S3, or to some other external service). That will incur some extra costs depending on volume.
Finally, you will also be billed for any traffic to the service.
The thing to compare it against is another "Virtual Server" from some other vendor. There is a lot of interesting things that can be done with EC2, but it may well be easier to go with a dedicated Virtual hosting service if you're just using a single machine.
Others have given good answers. I would have to add that you need to spend programmer time getting to know EC2's quirks and addressing them (e.g. with EBS). It's not completely trivial, and though it is useful knowledge to have and may be worth it for that reason alone, if you want to get up and running quickly with just a few servers, you should probably look at other hosted options.
On the other hand, if you plan to scale up massively enough (eventually hosting many servers on EC2) then I would highly recommend it. You have to architect a few things, but you need to do that anyways. The flexibility of on-demand computing, and the generally low price, makes this a killer platform once you reach a certain scale of operation.
You definitely can host an http server in EC2, but you need to take into consideration the following:
As mentioned before the cost can be much higher than alternative hosting solutions
Your instance (the machine you've started in EC2) can go off unexpectedly. There is no guarantee from Amazon for 24x7 availability. This mean that the data you've stored in local storage will be lost and when you've start a new instance, it will get a new IP.
To successfully host a server in EC2, you therefore need to employ some other services from Amazon. You need Elastic IP, so that you can circumvent the new IP address problem. You can also use Elastic Block Storage. This is a service that will allow you to mount in your machine a disk, that will not go away when your instance is lost.