I am using Dropbox Business API to extract folders/files. However, it is taking some time (can get 10 folders deep).
So I am thinking to run the process with multiple computers, say four computers per 1/4 members' records each.
Will it be OK to run all the four processes over one Dropbox Business API connection?
Should create four different Dropbox Business API connections and run those processes per different connections?
It doesn't matter either way?
Let me know if you have any idea.
Thanks!
I created multiple Dropbox API connections and run processes per connections instead of running processes on one API connection.
Related
I am trying to build a messaging app such that each message will be inserted to the database. Also, my backend will hit the database every second to retrieve the latest messages. I am worried that if I have many users who are using the messages feature, I will hit the database maximum number of connection quickly and the app will not work for other users. So, how can I make sure this problem will not happen?
I am just wondering if I created two (db-g1-small) instances, will I have twice the number of connection (1000 per instance)?. Google documentation says (db-g1-small) 1,000 Maximum Connections.
How can I keep track of the number of connections?. What will happen if the number of connection to the database reaches the maximum?
https://cloud.google.com/sql/pricing#2nd-gen-instance-pricing
You shouldn't have a unique connection per user to your database. Instead, your backend should use connection pooling to maintain a consistent number of connections to your instance. You can view some example of how best to do this on the Managing Database Connections page.
It's incredibly unlikely that you'll need 1000 open connections. Most applications use far, far less for optimal performance. You can check out this article about benchmarking different connection pool sizes.
Assume I'm working on a multiplayer online game. Each group of players may start an instance of the game to play. Take League Of Legends as an example.
At any moment of time, there are many game matches being served at the same time. My question is about the architecture of this case. Here are my suggestions:
Assume we have a cloud with a gateway. Any game instance requires a game server behind this gateway to serve the game. For different clients outside the cloud to access different game servers in the cloud, the gateway may differentiate between connections according to ports. It is like we have one machine with many processes each of them listening on a different port.
Is this the best we can get?
Is there another way for the gateway to differentiate connections and forward them to different game instances?
Notice that these are socket connections NOT HTTP requests to an API gateway.
EDIT 1: This question is not about Load Balancing
The keyword is ports. Will each match be served on a different port? or is there another way to serve multiple services on the same host (host = IP)?
Elaboration: I'm using client-server model for each match instance. So multiple clients may connect to the same match server to participate in the same match. Each match need to be server by a match server.
The limitation in mind is: For one host (=IP) to serve multiple services it need to provide them on different ports. Match 1 on port 1234. So clients participating in match 1 will connect to and communicate with the match server on port 1234.
EDIT 2: Scalability is the target
My match server does not calculate and maintain the world of many matches. It maintains the world of one match. This is why each match need another instance of the match server. It is not scalable to have all clients communicating about different matches to connect to one process and to be processed by one process.
My idea is to serve the world of each match by different process. This will require each process to be listening on a different port.
Example: Any client will start a TCP connection with a server listening on port A. Is there is a way to serve multiple MatchServers on the same port A (so that more simultaneous MatchServers won't result in more ports)?
Is there a better scalable way to serve the different worlds of multiple matches?
Short answer: you probably shouldn't use proxy-gateway to handle user connections unless you are absolutely sure there's no other way - you are severely limiting your scaling ability.
Long answer:
What you've described is just a load balancing problem. You can find plenty of solutions based on given restrictions via google.
For League Of Legends it can be quite simple: using some health-check find server with lowest amount of load and stick (kinda like sticky sessions) current game to this server - until the game is finished any computations for particular game are made there. You could use any kind of caching mechanism to store game - server relation for subsequent requests on gateway side.
Another, a bit more complicated example could be data storage for statistics for particular game - it's usually solved via sharding which is a usual consequence of distributed computing. It could be solved this way: use some kind of hashing function (for example, modulo) with game ID as parameter to calculate server number. For example 18283 mod 15 = 13 for game ID = 18283 and 15 available shards - so 13th server should store/serve this data.
Main problem here would be "rebalancing" - adding/remove a shard from cluster, for example.
Those are just two examples, you can google more of them using appropriate keywords. Just keep in mind that all of this is just a subset of problems of distributed computing.
I intend to use sails.io to build a chat website. There will be some 1000 user online at the same time. is sails.io is enough to do that? And is there a way to test performance of chat website? with some normal website i known jmeter but for chat website i know nothing at all
That will depend mostly of the server you will be using for your service.
Sockets are simply an array of connections. You can have as many as you want (within normal memory usage limits of your server machine).
You can checkout this answer for more information on socket costs What's the maximum number of rooms socket.io can handle?
I am currently sails.io for a chat product with 2000+ simultaneous users using it during business hours. Sails socket.io have been holding it pretty well. Nevertheless I got it prepared for horizontal scaling when I maximum capacity starts to show symptoms.You should too.
I want to figure out how many users can ADFS 2.0 stand-alone server support. I mean load of the server. My customers said that it supports just 100 users (seems strange for a server and so simple operations) and they have 700 users at the same time.
So he recomended to have a federation farm instead of stand-alone server. But I prefere to check first.
So, can you share info about load limitations of stand-alon SSO server VS server farms?
Any docs, articles with numbers, experts ideas or so on...
We have standalone servers that support WAY more than 100 users - easily over 1000.
Not sure what the upper limit is?
A farm is only going to help if you have a load balancer in front of them
I also found an article about this problem.
The auther writes that they use 2 ADFS servers for 10 000 users.
And there is a calculator to get number of servers depending on the load and users
There is no such restriction and it solely depends on how often users login to your system.
We have few deployments of the adfs, one of them supports like 50000 users and only TWO servers are enough. I even suspect one would do however this is, as always, not a good idea to have just one server (at least two servers = failover, you wouldn't want the whole environment to be inaccessible just because your login server just died).
The idea would be then to start with two servers and monitor the infrastructure. Add other instances only when necessary.
I have to perform load test on REST web services of 1000 concurrent users activities. I have selected NeoLoad tool for it. Is it required to have license for 1000 virtual users to simulate 1000 users or form one virtual user I can simulate 1000 users activities?
Load testing tools (like NeoLoad) price their license based on concurrent virtual users. So, if you have a license for a single user, you can create many requests, but only one at a time. To general load of 1000 users concurrently, you'll need a 1000 users license.
To simulate 1000 concurrent users you really need 1000 VUs as if you use only one you would be:
having wrongly good responsectimes due to caching effect
having wrongly negative response times due to contention on writing
you would probably not be reproducing the production behaviour of server regarding memory usage, caching....
But note that for http restful load testing you could go for a lot of free production ready open sources like Apache JMeter, locust...
https://www.blazemeter.com/blog/rest-api-testing-how-to-do-it-right
https://www.redline13.com/blog/2018/05/test-rest-apis-authentication-using-jmeter/