I have used Sails framework for my web application. Now we are calling services from mobile app.
I found that the Sails executes only 4 request at a time. Is there any way I can increase that?
Are you sure about that?
I've tested my sails app multiple times with: https://github.com/alexfernandez/loadtest
For example: loadtest http://localhost:1337 --rps 150 -c 20 -k
You can set it to any route of your app, and send GET/POST/PUT/DEL requests. The -c option means concurrent, so you can play with that number and check your app limits, but 5 concurrent is too low.
Most likely your browser or http client is limiting the number of requests per server. Refer to https://stackoverflow.com/a/985704/401025 or lookup the maximum number of requests from the manual of your http client.
There are no tcp connection limits imposed by sails or by nodejs. In matter of fact nodejs is known to be able to handle thousands of simultaneous connections.
Without knowing your server/nodejs setup it is hard to say why you are seeing a limit of 4 connections.
Depending on which nodejs version you use, you might check the http module of nodejs and following attributes:
https://nodejs.org/api/net.html#net_server_maxconnections
Set this property to reject connections when the server's connection
count gets high.
https://nodejs.org/api/http.html#http_agent_maxsockets
By default set to Infinity. Determines how many concurrent sockets the
agent can have open per origin. Origin is either a 'host:port' or
'host:port:localAddress' combination.
Related
I need to issue a long sequence of REST calls to the same server (let's call it myapi.com). At the moment, I am using the Rust library reqwest as follows:
I create a reqwest::Client with all default settings.
For each REST call:
I use client.post("https://myapi.com/path/to/the/api") to create a reqwest::RequestBuilder.
I configure the RequestBuilder to obtain a reqwest::Request.
I send() the Request, read the reqwest::Response.
I drop everything except the Client, start again.
I read in the docs that reqwest is supposed to pool connections within the same Client. Given that I always reuse the same Client, I would expect the first API call to take some more (owing to the initial TCP and HTTPS handshakes). However, I observe always a consistent, quite high latency across all requests. So, I am wondering if connections are reused at all, or re-established every time. If they are not, how do I get to recycle the same connection? I feel that latency would be drastically reduced if I could save myself some roundtrips.
I am building a distributed system that consists of potentially millions of clients which all need to keep an open (preferrably HTTP) connection to wait for a command from the server (which is running somewhere else). The load of messages / commmands will not be very high, maybe one message / sec / 1000 clients which means it would be 1000 msg/sec # 1 million clients. => it's basically about the concurrent connections.
The requirements are simple too. One way messaging (server->client), only 1 client per "channel".
I am pretty open in terms of technology (xmpp / websockets / comet / ...). I am using Google App Engine as server, but their "channels" won't work for me unfortunately (too low quotas and no Java client). XMPP was an option but is quite expensive. So far I was using URL Fetch & pubnub, but they just started charging for connections (big time).
So:
Does anyone know of a service out there that can do that for me in an affordable way? Most I have found restrict or heavily charge for connections.
Any experience with implementing such a server yourself? I have actually done that already and it works pretty well (based on Tomcat & NIO) but I haven't had the time yet to actually set up a large load test environment (partially because this is still a fallback solution, I'd prefer a battle hardened msg server). Any experience to how many users you get per GB? Any hard limits?
My architecture also allows to fragment the msg servers, but I'd like to maximize the concurrent connections because the msg processing CPU overhead is minimal.
I have meanwhile implemented my own message server using netty.io. Netty makes use of Java NIO and scales extremely well. For idle connections I get a memory footprint of 500 bytes per connection. I am doing only very simple message forwarding (no caching, storage or other fancy stuff) but with that am easily getting 1000 - 1500 msg / sec (each half a KB) on the small amazon instance (1ECU / 1.6GB).
Otherwise if you are looking for a (paid) service then I can recommend spire.io (they do not charge for connections but have a higher price per message) or pubnub (they do charge for connections but are cheaper per message).
You have to look more in architecture of making such environment.
First of all, if you will write sockets management by yourself, then don't use Thread per Client Socket. Use Asynchronous methods for receiving and sending data.
WebSockets might be too heavy if your messages are small. Because it implements framing, which has to be applied to each message for each socket individually (caching can be used for different versions of WebSockets protocols), that makes them slower to process both directions: for receive and for send, especially because of data masking.
It is possible to create millions of sockets, but only most advanced technologies are capable to do so. Erlang is able to handle millions connections, and is pretty scalable.
If you would like to have millions of connections using other higher level technologies, then you need to think about clustering of what you are trying to accomplish.
For example using gateway server that will keep track of all processing servers. And have data of them (IP, ports, load (if it will be one internal network, firewalling and port forwarding might be handy here).
Client software connects to that gateway server, gateway server checks the least loaded server and sends ip and port to client. Client creates connection directly to working server using provided address.
That way you will have gateway which as well can handle authorization, and wont hold connections for long, so one of them might be enough. And many workers that are doing publishing of data and keeping connections.
This is very related to your needs, and might not be suitable for your solutions.
I am consuming a rate-limited web service, which only allows me to make 5 calls per second. I am using my server to proxy these calls to a web client:
Mashery > My Web Server > Client's Browser
I have optimized the usage of this web service, but there are still occasional times when I go over the rate limit. What I would like to do instead is hold the client's request for one second (or longer if warranted) before making the web service call to Mashery.
There are some ways I can solve this by building a simple queue system with a database back-end, but I'd rather avoid that if something already exists. Does something already exist to rate-limit the consuming side of this?
To reliably throttle requests to only 5 per second, I would employ a queue/worker system. But before that I would just request a higher rate limit by contacting API support for that platform. Of course, I would also consider using caching if it's a read-only method and you're pulling down a lot of the same data and it's compliant with the API TOS.
This is specifically about GWT's RequestBuilder, but should apply to general XHR as well. My company is having me build a near realtime chat application over HTTP. Yes, I do realize there are better ways to do chat aplications, but this is what they want. Eventually we want it working on the iPad/iPhone as well so flash is out, which rules out websockets and comet as well, I think?
Anyway, I'm running into issues were I've set GWT's RequestBuilder timeout to 10 seconds and we get very random and sporadic timeouts. We've got error handling and emailing on the server side and never get any errors, which suggests the underlying XHR request that RequestBuilder is built on, never gets to the server and times out after 10 seconds.
We're using these request to poll the server for new messages rather often and also for sending new messages to the server and also polling (less frequently) for other parts of application. What I'm afraid of is that we're running into the browsers limit on concurrent connections to the same domain (2 for IE by default?).
Now my question is - If I construct a RequestBuilder and call it's send() method and the browser blocks it from sending until one of the 2 connections per domain is free, does the timeout still start while the request is being blocked or will it not start until the browser actually releases the underlying XHR?
I hope that's clear, if not please let me know and I'll try to explain more.
On the GWT Incubator doc page is an article explaining server push.
With said technique you only hold one connection open all the time.
Browsers allowed only 2 connections per hostname; that has now changed. 'Modern' browsers allow upto 6 simultaneous connections - it varies between browsers. See http://www.browserscope.org/ - network tab.
As regards the timer, it starts before GWT invokes xhr.send(), so your suspicion is right. See Request.java and RequestBuilder.java if you want to trace it out.
Seems like half the time, you answer your own question as soon as you post it.
Via: http://google-web-toolkit.googlecode.com/svn/javadoc/1.6/com/google/gwt/http/client/package-summary.html
Pending Request Limit
- Modern web browsers are limited to having only two HTTP requests outstanding at any one time. If your server experiences an error that prevents it from sending a response, it can tie up your outstanding requests. If you are concerned about this, you can always set timeouts for the request via RequestBuilder.setTimeoutMillis(int).
I have a chat application (xmpp / muc) that is going to be served by apache (we might change to nginx later but right now it's not easily done). If a user is in 2 rooms, he'll have between 2 and 4 active connections to the server (long-polling connections), so if we have 200 users per room and we have 5 rooms, what should the ServerLimit, MaxClients be set to? For example, these are the default values:
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000
Thanks,
Each user should only have a single connection to the XMPP server if your using something like BOSH to make the Web->XMPP connection. Each MUC room activity will be sent to the XMPP server with the user's JID as the target and that will cause the server to send that to the appropriate BOSH connection. Same for presence.
The information you need to load balance the Apache setup is the same as load balancing any long-polling connection - XMPP doesn't add anything exotic to the mix. An XMPP BOSH connection is a long-polling connection to your web server which is then sent to a persistent connection to the XMPP server - causing 2 sockets to be opened per user connect.
I'm guessing this a pre-fork Apache....
If you've got 1000 concurrent chat connections, then you need at least 1000 webservers (presumably you'll be serving up static content too, and you say that each presence results in 2 connections - so you can double the 1000) - a ServerLimit/MaxClients of 256 is not going to cut it, you probably need around 2200 to support this (but without hard metrics its hard to give an exact figure).
This is a ridiculously large amount. To support this I'd be looking for 3 boxes, each with about 2Gb free memory before the webserver is started.
MaxRequestsPerChild is not really relevant other than the fact you want some turnover of processes, particularly if you are using long polling.
This is one of the reasons why COMET is not a good idea. Using AJAX polling would be much more efficient. Assuming this is not something you can change, you might want to have a look at using a threaded Apache webserver which is marginally more memory efficient,
C.