Background
We are writing a Messenger-like app. We have setup Websockets to Inbox and Chat.
Question
My question is simple. What are the advantages and disadvantages when sending data from Client to Server using REST instead of Websockets? (I am not interested in updates now.)
We know that REST has higher overhead in terms of message sizes and that WS is duplex (thus open all time). What about the other things we didn't keep in mind?
Here's a summary of the tradeoffs I'm aware of.
Reasons to use webSocket:
You need/want server-push of data.
You are sending lots of small pieces of data from client to server and doing it very regularly. Using webSocket has significantly less overhead per transmission.
Reasons to use REST:
You want to use server-side frameworks or modules that are built for REST, not for webSocket (such as auth, rate limiting, security, streaming, etc...).
You aren't sending data very often from client to server and thus the server-side burden of keeping a webSocket connection open all the time may lessen your server scalability.
You want your client to run in places where a long-connected webSocket during inactive periods of time may not be practical (perhaps mobile).
You want your client to run in old browsers that don't support webSocket.
You want the browser to enforce same-origin restrictions (those are enforced for REST Ajax calls, but not for webSocket connections).
You don't want to have to write code that detects when the webSocket connection has died and then auto-reconnects and handles back-offs and handles mobile issues with battery usage issues, etc...
You need to run in situations where there are proxies or other network infrastructure that may not support long running webSocket connections.
If you want request/response built in. REST is request/response. WebSocket is not - it's message based. Responses from a webSocket are done by sending a messge back. That message back is not, by itself, a response to any specific request, it's just data being sent back. If you want request/response with webSocket, then you have to build some infrastructure yourself where you tag an id into a request and the response for that particular request contains that specific id. Otherwise, if there are every multiple requests in flight at the same time, then you don't know which response belongs with which request because all the data is being sent over the same connection and you would have no way of matching response with request.
If you want other clients to be able to carry out this operation via an Ajax call.
So, if you already have a webSocket implementation, don't have any problem with it that are lessened with REST and aren't interested in any of the reasons that REST might be better, then stick with your webSocket implementation.
Related references:
websocket vs rest API for real time data?
Ajax vs Socket.io
Adding comments per your request:
It sounds like you're expecting someone to tell you the "right" way to do it. There are reasons to pick one way over the other. If none of those reason compel you one way vs. the other, then it's just an architectural choice and you must take in the whole context of what you are doing and decide which architectural choice makes more sense to you. If you already have the reliably established webSocket connection and none of the advantages of REST apply to your situation then you can optimize for "efficiency" and send your data to the server over the webSocket connection.
On the other hand, if you wanted there to be a simple API on your server that could be reached with an Ajax call from other clients, then you'd want your server to support this operation via REST so it would simplest for these other clients to carry out this one operation. So, it all depends upon which direction your requirements drive you and, if there is no particular driving reason to go one way or the other, you just make an architectural choice yourself.
I'm a newbie with Rabbitmq(and programming) so sorry in advance if this is obvious. I am creating a pool to share between threads that are working on a queue but I'm not sure if I should use connections or channels in the pool.
I know I need channels to do the actual work but is there a performance benefit of having one channel per connection(in terms of more throughput from the queue)? or am I better off just using a single connection per application and pool many channels?
note: because I'm pooling the resources the initial cost is not a factor, as I know connections are more expensive than channels. I'm more interested in throughput.
I have found this on the rabbitmq website it is near the bottom so I have quoted the relevant part below.
The tl;dr version is that you should have 1 connection per application and 1 channel per thread.
Connections
AMQP connections are typically long-lived. AMQP is an application
level protocol that uses TCP for reliable delivery. AMQP connections
use authentication and can be protected using TLS (SSL). When an
application no longer needs to be connected to an AMQP broker, it
should gracefully close the AMQP connection instead of abruptly
closing the underlying TCP connection.
Channels
Some applications need multiple connections to an AMQP broker.
However, it is undesirable to keep many TCP connections open at the
same time because doing so consumes system resources and makes it more
difficult to configure firewalls. AMQP 0-9-1 connections are
multiplexed with channels that can be thought of as "lightweight
connections that share a single TCP connection".
For applications that use multiple threads/processes for processing,
it is very common to open a new channel per thread/process and not
share channels between them.
Communication on a particular channel is completely separate from
communication on another channel, therefore every AMQP method also
carries a channel number that clients use to figure out which channel
the method is for (and thus, which event handler needs to be invoked,
for example).
It is advised that there is 1 channel per thread, even though they are thread safe, so you could have multiple threads sending through one channel. In terms of your application I would suggest that you stick with 1 channel per thread though.
Additionally it is advised to only have 1 consumer per channel.
These are only guidelines so you will have to do some testing to see what works best for you.
This thread has some insights here and here.
Despite all these guidelines this post suggests that it will most likely not affect performance by having multiple connections. Though it is not specific whether it is talking about client side or server(rabbitmq) side. With the one point that it will of course use more systems resources with more connections. If this is not a problem and you wish to have more throughput it may indeed be better to have multiple connections as this post suggests multiple connections will allow you more throughput. The reason seems to be that even if there are multiple channels only one message goes through the connection at one time. Therefore a large message will block the whole connection or many unimportant messages on one channel may block an important message on the same connection but a different channel. Again resources are an issue. If you are using up all the bandwidth with one connection then adding an additional connection will have no increase performance over having two channels on the one connection. Also each connection will use more memory, cpu and filehandles, but that may well not be a concern though might be an issue when scaling.
In addition to the accepted answer:
If you have a cluster of RabbitMQ nodes with either a load-balancer in front, or a short-lived DNS (making it possible to connect to a different rabbit node each time), then a single, long-lived connection would mean that one application node works exclusively with a single RabbitMQ node. This may lead to one RabbitMQ node being more heavily utilized than the others.
The other concern mentioned above is that the publishing and consuming are blocking operations, which leads to queueing messages. Having more connections will ensure that 1. processing time for each messages doesn't block other messages 2. big messages aren't blocking other messages.
That's why it's worth considering having a small connection pool (having in mind the resource concerns raised above)
The "one channel per thread" might be a safe assumption (I say might as I have not made any research by myself and I have no reason to doubt the documentation :) ) but beware that there is a case where this breaks:
If you you use RPC with RabbitMQ Direct reply-to then you cannot reuse the same channel to consume for another RPC request. I asked for details about that in the google user group and the answer I got from Michael Klishin (who seems to be actively involved in RabbitMQ development) was that
Direct Reply to is not meant to be used with channel sharing either way.
I've email Pivotal to update their documentation to explain how amq.rabbitmq.reply-to is working under the hood and I'm still waiting for an answer (or an update).
So if you want to stick to "one channel per thread" beware as this will not work good with Direct reply-to.
I am building a distributed system that consists of potentially millions of clients which all need to keep an open (preferrably HTTP) connection to wait for a command from the server (which is running somewhere else). The load of messages / commmands will not be very high, maybe one message / sec / 1000 clients which means it would be 1000 msg/sec # 1 million clients. => it's basically about the concurrent connections.
The requirements are simple too. One way messaging (server->client), only 1 client per "channel".
I am pretty open in terms of technology (xmpp / websockets / comet / ...). I am using Google App Engine as server, but their "channels" won't work for me unfortunately (too low quotas and no Java client). XMPP was an option but is quite expensive. So far I was using URL Fetch & pubnub, but they just started charging for connections (big time).
So:
Does anyone know of a service out there that can do that for me in an affordable way? Most I have found restrict or heavily charge for connections.
Any experience with implementing such a server yourself? I have actually done that already and it works pretty well (based on Tomcat & NIO) but I haven't had the time yet to actually set up a large load test environment (partially because this is still a fallback solution, I'd prefer a battle hardened msg server). Any experience to how many users you get per GB? Any hard limits?
My architecture also allows to fragment the msg servers, but I'd like to maximize the concurrent connections because the msg processing CPU overhead is minimal.
I have meanwhile implemented my own message server using netty.io. Netty makes use of Java NIO and scales extremely well. For idle connections I get a memory footprint of 500 bytes per connection. I am doing only very simple message forwarding (no caching, storage or other fancy stuff) but with that am easily getting 1000 - 1500 msg / sec (each half a KB) on the small amazon instance (1ECU / 1.6GB).
Otherwise if you are looking for a (paid) service then I can recommend spire.io (they do not charge for connections but have a higher price per message) or pubnub (they do charge for connections but are cheaper per message).
You have to look more in architecture of making such environment.
First of all, if you will write sockets management by yourself, then don't use Thread per Client Socket. Use Asynchronous methods for receiving and sending data.
WebSockets might be too heavy if your messages are small. Because it implements framing, which has to be applied to each message for each socket individually (caching can be used for different versions of WebSockets protocols), that makes them slower to process both directions: for receive and for send, especially because of data masking.
It is possible to create millions of sockets, but only most advanced technologies are capable to do so. Erlang is able to handle millions connections, and is pretty scalable.
If you would like to have millions of connections using other higher level technologies, then you need to think about clustering of what you are trying to accomplish.
For example using gateway server that will keep track of all processing servers. And have data of them (IP, ports, load (if it will be one internal network, firewalling and port forwarding might be handy here).
Client software connects to that gateway server, gateway server checks the least loaded server and sends ip and port to client. Client creates connection directly to working server using provided address.
That way you will have gateway which as well can handle authorization, and wont hold connections for long, so one of them might be enough. And many workers that are doing publishing of data and keeping connections.
This is very related to your needs, and might not be suitable for your solutions.
I've developed some web-based applications till now using PHP, Python and Java. But some fundamental but very important questions are still beyond my knowledge, so I made this post to get help and clarification from you guys.
Say I use some programming language as my backend language(PHP/Python/.Net/Java, etc), and I deploy my application with a web server(apache/lighttpd/nginx/IIS, etc). And suppose at time T, one of my page got 100 simultaneous requests from different users. So my questions are:
How does my web server handle such 100 simultaneous requests? Will web server generate one process/thread for each request? (if yes, process or thread?)
How does the interpreter of the backend language do? How will it handle the request and generate the proper html? Will the interpreter generate a process/thread for each request?(if yes, process or thread?)
If the interpreter will generate a process/thread for each request, how about these processes(threads)? Will they share some code space? Will they communicate with each other? How to handle the global variables in the backend codes? Or they are independent processes(threads)? How long is the duration of the process/thread? Will they be destroyed when the request is handled and the response is returned?
Suppose the web server can only support 100 simultaneous requests, but now it got 1000 simultaneous requests. How does it handle such situation? Will it handle them like a queue and handle the request when the server is available? Or other approaches?
I read some articles about Comet these days. And I found long connection may be a good way to handle the real-time multi-users usecase. So how about long connection? Is it a feature of some specific web servers or it is available for every web server? Long connection will require a long-existing interpreter process?
EDIT:
Recently I read some articles about CGI and fastcgi, which makes me know the approach of fastcgi should be a typical approach to hanlde request.
the protocol multiplexes a single transport connection between several independent FastCGI requests. This supports applications that are able to process concurrent requests using event-driven or multi-threaded programming techniques.
Quoted from fastcgi spec, which mentioned connection which can handle several requests, and can be implemented in mutli-threaded tech. I'm wondering this connection can be treated as process and it can generate several threads for each request. If this is true, I become more confused about how to handle the shared resource in each thread?
P.S thank Thomas for the advice of splitting the post to several posts, but I think the questions are related and it's better to group them together.
Thank S.Lott for your great answer, but some answers to each question are too brief or not covered at all.
Thank everyone's answer, which makes me closer to the truth.
Update, Spring 2018:
I wrote this response in 2010 and since then, a whole lot of things have changed in the world of a web backend developer. Namely, the advent of the "cloud" turning services such as one-click load balancers and autoscaling into commodities have made the actual mechanics of scaling your application much easier to get started.
That said, what I wrote in this article in 2010 still mostly holds true today, and understanding the mechanics behind how your web server and language hosting environment actually works and how to tune it can save you considerable amounts of money in hosting costs. For that reason, I have left the article as originally written below for anyone who is starting to get elbows deep in tuning their stack.
1. Depends on the webserver (and sometimes configuration of such). A description of various models:
Apache with mpm_prefork (default on unix): Process per request. To minimize startup time, Apache keeps a pool of idle processes waiting to handle new requests (which you configure the size of). When a new request comes in, the master process delegates it to an available worker, otherwise spawns up a new one. If 100 requests came in, unless you had 100 idle workers, some forking would need to be done to handle the load. If the number of idle processes exceeds the MaxSpare value, some will be reaped after finishing requests until there are only so many idle processes.
Apache with mpm_event, mpm_worker, mpm_winnt: Thread per request. Similarly, apache keeps a pool of idle threads in most situations, also configurable. (A small detail, but functionally the same: mpm_worker runs several processes, each of which is multi-threaded).
Nginx/Lighttpd: These are lightweight event-based servers which use select()/epoll()/poll() to multiplex a number of sockets without needing multiple threads or processes. Through very careful coding and use of non-blocking APIs, they can scale to thousands of simultaneous requests on commodity hardware, provided available bandwidth and correctly configured file-descriptor limits. The caveat is that implementing traditional embedded scripting languages is almost impossible within the server context, this would negate most of the benefits. Both support FastCGI however for external scripting languages.
2. Depends on the language, or in some languages, on which deployment model you use. Some server configurations only allow certain deployment models.
Apache mod_php, mod_perl, mod_python: These modules run a separate interpreter for each apache worker. Most of these cannot work with mpm_worker very well (due to various issues with threadsafety in client code), thus they are mostly limited to forking models. That means that for each apache process, you have a php/perl/python interpreter running inside. This severely increases memory footprint: if a given apache worker would normally take about 4MB of memory on your system, one with PHP may take 15mb and one with Python may take 20-40MB for an average application. Some of this will be shared memory between processes, but in general, these models are very difficult to scale very large.
Apache (supported configurations), Lighttpd, CGI: This is mostly a dying-off method of hosting. The issue with CGI is that not only do you fork a new process for handling requests, you do so for -every- request, not just when you need to increase load. With the dynamic languages of today having a rather large startup time, this creates not only a lot of work for your webserver, but significantly increases page load time. A small perl script might be fine to run as CGI, but a large python, ruby, or java application is rather unwieldy. In the case of Java, you might be waiting a second or more just for app startup, only to have to do it all again on the next request.
All web servers, FastCGI/SCGI/AJP: This is the 'external' hosting model of running dynamic languages. There are a whole list of interesting variations, but the gist is that your application listens on some sort of socket, and the web server handles an HTTP request, then sends it via another protocol to the socket, only for dynamic pages (static pages are usually handled directly by the webserver).
This confers many advantages, because you will need less dynamic workers than you need the ability to handle connections. If for every 100 requests, half are for static files such as images, CSS, etc, and furthermore if most dynamic requests are short, you might get by with 20 dynamic workers handling 100 simultaneous clients. That is, since the normal use of a given webserver keep-alive connection is 80% idle, your dynamic interpreters can be handling requests from other clients. This is much better than the mod_php/python/perl approach, where when your user is loading a CSS file or not loading anything at all, your interpreter sits there using memory and not doing any work.
Apache mod_wsgi: This specifically applies to hosting python, but it takes some of the advantages of webserver-hosted apps (easy configuration) and external hosting (process multiplexing). When you run it in daemon mode, mod_wsgi only delegates requests to your daemon workers when needed, and thus 4 daemons might be able to handle 100 simultaneous users (depends on your site and its workload)
Phusion Passenger: Passenger is an apache hosting system that is mostly for hosting ruby apps, and like mod_wsgi provides advantages of both external and webserver-managed hosting.
3. Again, I will split the question based on hosting models for where this is applicable.
mod_php, mod_python, mod_perl: Only the C libraries of your application will generally be shared at all between apache workers. This is because apache forks first, then loads up your dynamic code (which due to subtleties, is mostly not able to use shared pages). Interpreters do not communicate with each other within this model. No global variables are generally shared. In the case of mod_python, you can have globals stay between requests within a process, but not across processes. This can lead to some very weird behaviours (browsers rarely keep the same connection forever, and most open several to a given website) so be very careful with how you use globals. Use something like memcached or a database or files for things like session storage and other cache bits that need to be shared.
FastCGI/SCGI/AJP/Proxied HTTP: Because your application is essentially a server in and of itself, this depends on the language the server is written in (usually the same language as your code, but not always) and various factors. For example, most Java deployment use a thread-per-request. Python and its "flup" FastCGI library can run in either prefork or threaded mode, but since Python and its GIL are limiting, you will likely get the best performance from prefork.
mod_wsgi/passenger: mod_wsgi in server mode can be configured how it handles things, but I would recommend you give it a fixed number of processes. You want to keep your python code in memory, spun up and ready to go. This is the best approach to keeping latency predictable and low.
In almost all models mentioned above, the lifetime of a process/thread is longer than a single request. Most setups follow some variation on the apache model: Keep some spare workers around, spawn up more when needed, reap when there are too many, based on a few configurable limits. Most of these setups -do not- destroy a process after a request, though some may clear out the application code (such as in the case of PHP fastcgi).
4. If you say "the web server can only handle 100 requests" it depends on whether you mean the actual webserver itself or the dynamic portion of the webserver. There is also a difference between actual and functional limits.
In the case of Apache for example, you will configure a maximum number of workers (connections). If this number of connections was 100 and was reached, no more connections will be accepted by apache until someone disconnects. With keep-alive enabled, those 100 connections may stay open for a long time, much longer than a single request, and those other 900 people waiting on requests will probably time out.
If you do have limits high enough, you can accept all those users. Even with the most lightweight apache however, the cost is about 2-3mb per worker, so with apache alone you might be talking 3gb+ of memory just to handle the connections, not to mention other possibly limited OS resources like process ids, file descriptors, and buffers, and this is before considering your application code.
For lighttpd/Nginx, they can handle a large number of connections (thousands) in a tiny memory footprint, often just a few megs per thousand connections (depends on factors like buffers and how async IO apis are set up). If we go on the assumption most your connections are keep-alive and 80% (or more) idle, this is very good, as you are not wasting dynamic process time or a whole lot of memory.
In any external hosted model (mod_wsgi/fastcgi/ajp/proxied http), say you only have 10 workers and 1000 users make a request, your webserver will queue up the requests to your dynamic workers. This is ideal: if your requests return quickly you can keep handling a much larger user load without needing more workers. Usually the premium is memory or DB connections, and by queueing you can serve a lot more users with the same resources, rather than denying some users.
Be careful: say you have one page which builds a report or does a search and takes several seconds, and a whole lot of users tie up workers with this: someone wanting to load your front page may be queued for a few seconds while all those long-running requests complete. Alternatives are using a separate pool of workers to handle URLs to your reporting app section, or doing reporting separately (like in a background job) and then polling its completion later. Lots of options there, but require you to put some thought into your application.
5. Most people using apache who need to handle a lot of simultaneous users, for reasons of high memory footprint, turn keep-alive off. Or Apache with keep-alive turned on, with a short keep-alive time limit, say 10 seconds (so you can get your front page and images/CSS in a single page load). If you truly need to scale to 1000 connections or more and want keep-alive, you will want to look at Nginx/lighttpd and other lightweight event-based servers.
It might be noted that if you do want apache (for configuration ease of use, or need to host certain setups) you can put Nginx in front of apache, using HTTP proxying. This will allow Nginx to handle keep-alive connections (and, preferably, static files) and apache to handle only the grunt work. Nginx also happens to be better than apache at writing logfiles, interestingly. For a production deployment, we have been very happy with nginx in front of apache(with mod_wsgi in this instance). The apache does not do any access logging, nor does it handle static files, allowing us to disable a large number of the modules inside apache to keep it small footprint.
I've mostly answered this already, but no, if you have a long connection it doesn't have to have any bearing on how long the interpreter runs (as long as you are using external hosted application, which by now should be clear is vastly superior). So if you want to use comet, and a long keep-alive (which is usually a good thing, if you can handle it) consider the nginx.
Bonus FastCGI Question You mention that fastcgi can multiplex within a single connection. This is supported by the protocol indeed (I believe the concept is known as "channels"), so that in theory a single socket can handle lots of connections. However, it is not a required feature of fastcgi implementors, and in actuality I do not believe there is a single server which uses this. Most fastcgi responders don't use this feature either, because implementing this is very difficult. Most webservers will make only one request across a given fastcgi socket at a time, then make the next across that socket. So you often just have one fastcgi socket per process/thread.
Whether your fastcgi application uses processing or threading (and whether you implement it via a "master" process accepting connections and delegating or just lots of processes each doing their own thing) is up to you; and varies based on capabilities of your programming language and OS too. In most cases, whatever is the default the library uses should be fine, but be prepared to do some benchmarking and tuning of parameters.
As to shared state, I recommend you pretend that any traditional uses of in-process shared state do not exist: even if they may work now, you may have to split your dynamic workers across multiple machines later. For state like shopping carts, etc; the db may be the best option, session-login info can be kept in securecookies, and for temporary state something akin to memcached is pretty neat. The less you have reliant on features that share data (the "shared-nothing" approach) the bigger you can scale in the future.
Postscript: I have written and deployed a whole lot of dynamic applications in the whole scope of setups above: all of the webservers listed above, and everything in the range of PHP/Python/Ruby/Java. I have extensively tested (using both benchmarking and real-world observation) the methods, and the results are sometimes surprising: less is often more. Once you've moved away from hosting your code in the webserver process, You often can get away with a very small number of FastCGI/Mongrel/mod_wsgi/etc workers. It depends on how much time your application stays in the DB, but it's very often the case that more processes than 2*number of CPU's will not actually gain you anything.
How does my web server handle such 100 simultaneous requests? Does web server generate one process/thread for each request? (if yes, process or thread?)
It varies. Apache has both threads and processes for handling requests. Apache starts several concurrent processes, each one of which can run any number of concurrent threads. You must configure Apache to control how this actually plays out for each request.
How does the interpreter of the backend language do? How will it handle the request and generate the proper html? Will the interpreter generate a process/thread for each request?(if yes, process or thread?)
This varies with your Apache configuration and your language. For Python one typical approach is to have daemon processes running in the background. Each Apache process owns a daemon process. This is done with the mod_wsgi module. It can be configured to work several different ways.
If the interpreter will generate a process/thread for each request, how about these processes(threads)? Will they share some code space? Will they communicate with each other? How to handle the global variables in the backend codes? Or they are independent processes(threads)? How long is the duration of the process/thread? Will they be destroyed when the request is handled and the response is returned?
Threads share the same code. By definition.
Processes will share the same code because that's the way Apache works.
They do not -- intentionally -- communicate with each other. Your code doesn't have a way to easily determine what else is going on. This is by design. You can't tell which process you're running in, and can't tell what other threads are running in this process space.
The processes are long-running. They do not (and should not) be created dynamically. You configure Apache to fork several concurrent copies of itself when it starts to avoid the overhead of process creation.
Thread creation has much less overhead. How Apaches handles threads internally doesn't much matter. You can, however, think of Apache as starting a thread per request.
Suppose the web server can only support 100 simultaneous requests, but now it got 1000 simultaneous requests. How does it handle such situation? Will it handle them like a queue and handle the request when the server is available? Or other approaches?
This is the "scalability" question. In short -- how will performance degrade as the load increases. The general answer is that the server gets slower. For some load level (let's say 100 concurrent requests) there are enough processes available that they all run respectably fast. At some load level (say 101 concurrent requests) it starts to get slower. At some other load level (who knows how many requests) it gets so slow you're unhappy with the speed.
There is an internal queue (as part of the way TCP/IP works, generally) but there's no governor that limits the workload to 100 concurrent requests. If you get more requests, more threads are created (not more processes) and things run more slowly.
To begin with, requiring detailed answers to all your points is a bit much, IMHO.
Anyway, a few short answers about your questions:
#1
It depends on the architecture of the server. Apache is a multi-process, and optionally also, multi-threaded server. There is a master process which listens on the network port, and manages a pool of worker processes (where in the case of the "worker" mpm each worker process has multiple threads). When a request comes in, it is forwarded to one of the idle workers. The master manages the size of the worker pool by launching and terminating workers depending on the load and the configuration settings.
Now, lighthttpd and nginx are different; they are so-called event-based architectures, where multiple network connections are multiplexed onto one or more worker processes/threads by using the OS support for event multiplexing such as the classic select()/poll() in POSIX, or more scalable but unfortunately OS-specific mechanisms such as epoll in Linux. The advantage of this is that each additional network connection needs only maybe a few hundred bytes of memory, allowing these servers to keep open tens of thousands of connections, which would generally be prohibitive for a request-per-process/thread architecture such as apache. However, these event-based servers can still use multiple processes or threads in order to utilize multiple CPU cores, and also in order to execute blocking system calls in parallel such as normal POSIX file I/O.
For more info, see the somewhat dated C10k page by Dan Kegel.
#2
Again, it depends. For classic CGI, a new process is launched for every request. For mod_php or mod_python with apache, the interpreter is embedded into the apache processes, and hence there is no need to launch a new process or thread. However, this also means that each apache process requires quite a lot of memory, and in combination with the issues I explained above for #1, limits scalability.
In order to avoid this, it's possible to have a separate pool of heavyweight processes running the interpreters, and the frontend web servers proxy to the backends when dynamic content needs to be generated. This is essentially the approach taken by FastCGI and mod_wsgi (although they use custom protocols and not HTTP so perhaps technically it's not proxying). This is also typically the approach chosen when using the event-based servers, as the code for generating the dynamic content seldom is re-entrant which it would need to be in order to work properly in an event-based environment. Same goes for multi-threaded approaches as well if the dynamic content code is not thread-safe; one can have, say, frontend apache server with the threaded worker mpm proxying to backend apache servers running PHP code with the single-threaded prefork mpm.
#3
Depending on at which level you're asking, they will share some memory via the OS caching mechanism, yes. But generally, from a programmer perspective, they are independent. Note that this independence is not per se a bad thing, as it enables straightforward horizontal scaling to multiple machines. But alas, some amount of communication is often necessary. One simple approach is to communicate via the database, assuming that one is needed for other reasons, as it usually is. Another approach is to use some dedicated distributed memory caching system such as memcached.
#4
Depends. They might be queued, or the server might reply with some suitable error code, such as HTTP 503, or the server might just refuse the connection in the first place. Typically, all of the above can occur depending on how loaded the server is.
#5
The viability of this approach depends on the server architecture (see my answer to #1). For an event-based server, keeping connections open is no big issue, but for apache it certainly is due to the large amount of memory required for every connection. And yes, this certainly requires a long-running interpreter process, but as described above, except for classic CGI this is pretty much granted.
Web servers are multi-threaded environment; besides using application scoped variables, a user request doesn't interact with other threads.
So:
Yes, a new thread will be created for every user
Yes, HTML will be processed for every request
You'll need to use application scoped variables
If you get more requests than you can deal, they will be put on queue. If they got served before configured timeout period, user will get his response, or a "server busy" like error.
Comet isn't specific for any server/language. You can achieve same result by quering your server every n seconds, without dealing with other nasty threads issues.