Anybody who can advise or have experience on the possibility to have an SqlBase database in a cloud environment and run a Gupta application which is stored on local PCs?
Thanks.
We have some experience running a SQL-Database (Oracle, SqlServer, SqlBase) on a remote Server connected over WAN. Most often data access is very slow and you have to write your application carefully.
The reason for the slowness is usually not the bandwidth but the number of hops an IP-packet takes. Each hop adds some milliseconds of delay which oftens sums up to a painful experience. So it's ok to get one big blob from a database. It's also ok to fetch large resultsets. But when there are a lot of smaller queries it will get very slow.
There are two solutions to this problem:
1) Use a dedicated line from client to server if possible.
2) Write your application in a way that minimizes the number of queries.
Related
Did anyone observed slowness when querying data from snowflake(select statement) with OS Windows 2016 + pycharm environment.
Getting result quickly with window 10 OS with same environment.
When checked network performance using wireshark, round trip time is more with windows 10 when compared with window 2016.
Other difference is window 10 is on physical on-prem system and windows 2016 is workspace in AWS.
Any one experienced slowness with similar setup as above.
Any suggestion to troubleshoot further.
Do you think the query execution at Snowflake is slow?
Have you compared the query execution time by keeping other conditions (like warehouse size, load on the warehouse) the same?
If the slowness is in data transfer over network, it could be because of AWS region etc, not to mention several other factors that could impact network data transfer speeds.
Snowflake's query execution performance should not be impacted based on your location or OS etc.
Snowflake is running as a SaaS on the cloud. In Ragesh's case I think it is running on AWS. So, from a query performance perspective on snowflake, it ideally should not matter whether you are initiating the connection from a Windows 2010 or 2016. But the network bandwidth may be a factor to consider. Your bandwidth on premise may not be as good as when you are on AWS. To be sure about that. Can you please do this test.
Run your query with Windows 10 - Before running the query, please set a query tag like
ALTER SESSION SET QUERY_TAG ='ONPREMISE';
Run your query with Windows 16 - Before running the query, please set a query tag like
ALTER SESSION SET QUERY_TAG ='ONAWS';
After this, go to Snwoflake console history view and filter based on query_tag and share the query profile details for both the scenarions
I have a fairly busy DB2 on Windows server - 9.7, fix pack 11.
About 60% of the CPU time used by all queries in the package cache is being used by the following two statements:
CALL SYSIBM.SQLSTATISTICS(?,?,?,?,?,?)
CALL SYSIBM.SQLPRIMARYKEYS(?,?,?,?)
I'm fairly decent with physical tuning and have spent a lot of time on SQL tuning on this system as well. The applications are all custom, and educating developers is something I also spend time on.
I get the impression that these two stored procedures are something that perhaps ODBC calls? Reading their descriptions, they also seem like things that are completely unnecessary to do the work being done. The application doesn't need to know the primary key of a table to be able to query it!
Is there anything I can tell my developers to do that will either eliminate/reduce the execution of these or cache the information so that they're not executing against the database millions of times and eating up so much CPU? Or alternately anything I can do at the database level to reduce their impact?
6.5 years later, and I have the answer to my own question. This is a side effect of using an ORM. Part of what it does is to discover the database schema. Rails also has a similar workload. In Rails, you can avoid this by using the schema cache. This becomes particularly important at scale. Not sure if there are equivalencies for other ORMs, but I hope so!
For our architecture we are contemplating something like multi-tenancy. In our approach each tenant would get their own database. When I say database, I don't mean server. I mean a database within an OrientDB server.
The question is... Is there a best practice way to do this. The three options we see are:
Stand up an entire OrientDB server to host a single database.
This seems inefficient. Especially since we are going to look towards a clustered / replicated architecture.
Put multiple databases into a single OrientDB Server
Here I am curious as to scalability. Is there a practical limit to how many databases a single OrientDB cluster can hold? Each tenant may make many connections to the database. If say each tenant makes 20 or so database connections and we have 1,000 tenants, I now have 20,000 connections going to the database. Obviously we would have many servers supporting this load so that would be distributed.
Some middle ground where we have a certain number of tenants hosted in each clustered instance of OrientDB
Not sure how to draw the line here.
Just wondering if there are best practices around this? Thanks and keep up the good work.
The physical limits are given by memory size, number of transactions managed per second and number of open files on the OS.
Every db in OrientDB is just a folder on the filesystem, if you never access the db it does not use system resources, but as soon as you access and query it, OrientDB has to keep files open, establish connections to the clients, allocate disk cache and so on.
My suggestion is to have at most a few tens of small databases on the same OrientDB instance.
How scalable is Odoo with regard to the number of users it can support and still run smoothly?
Also, how many records can it entertain without facing server load issues when many users are using it simultaneously?
It depends on your server configuration, disk performance, processor, memory, etc. There are many performance factors not unique to Odoo.
For performance information you must read this: Odoo Performance
Odoo has a PostgreSQL database, for better performance you must read about PostgreSQL as well.
Also, see the Performance Comparisons like speed test, load test, scalability etc.
What's the best solution for using Node.js and Redis to create an uptime monitoring system? Can I use Redis as a queue but is not the best way to save information, maybe MongoDB is?
It seems pretty simple but needing to have more than 1 server to guarantee the server is down and make everything work together is not so easy.
To monitor uptime, you would use a Cron job on the system. With each call, you would check to see if the host is up, and how long it would take. And in that script, you would save your data in Redis.
To do this in Node.JS, you would create a script that checks the status of the server. Just making a HTTP request to the server (Or Ping, w.e.) and recording if it fails or not. Then I would just record it to Redis. How you do it does not matter, because the script (if you run the cron every 30 seconds) has [30] seconds before the next run, so you dont have to worry about getting your query to the server. How you save your data is up to you, but in this case even MySQL would work (if you are only doing a small number of sites)
More on Cron # Wikipedia
Can I use Redis as a queue but is not
the best way to save information,
maybe MongoDB is?
You can(should) use Redis as your queue. It is going to be extremely fast.
I also think it is going to be very good option to save the information inside Redis. Unfortunately Redis does not do any timing(yet). I think you could/should use Beanstalkd to put messages on the queue that get delivered when needed(every x seconds). I also think cron is not that a very good idea because you would be needing a lot of them and when using a queue you could do your work faster(share load among multiple processes) also.
Also I don't think you need that much memory to save everything in memory(makes site fast) because dataset is going to be relative simple. Even if you aren't able(smart to get more memory if you ask me) to fit entire dataset in memory you can rely on Redis's virtual memory.
It seems pretty simple but needing to
have more than 1 server to guarantee
the server is down and make everything
work together is not so easy.
Sharding/replication is what I think you should read into to solve this problem(hard). Luckily Redis supports replication(sharding can also be achieved). MongoDB supports sharding/replication out of the box. To be honest I don't think you need sharding yet and your dataset is rather simple so Redis is going to be faster:
http://redis.io/topics/replication
http://www.mongodb.org/display/DOCS/Sharding+Introduction
http://www.mongodb.org/display/DOCS/Replication
http://ngchi.wordpress.com/2010/08/23/towards-auto-sharding-in-your-node-js-app/