Yeah, I'm just wondering whether is it a good idea to use a MySQL or MongoDB driver written in pure JS.. I mean, it shouldn't be problem when running a small app with small db for 100 users / month, but what about heavy load / really huge DBs?
Aren't there any professional MySQL and Mongo drivers for NodeJS that I can compile? The performance of these should be way much better.
Or am I wrong about this? For example, Mongoose uses a driver written in pure JS. Is that good enough to efficiently query 500 million documents?
Any suggestion / advice would be appreciated!
Thanks
EDIT:
So thanks for the response guys. Well I'm still a but unsure about this :).
I mean, writing drivers in Python or Java or even C# surely makes sense, but those languages are much more powerful and faster than JS.
Here is what makes me worried:
My MySQL driver (written in pure JS) executes the query SHOW COLUMNS FROM Table in 300-400ms. If I execute the exact same query from MySQL shell, it takes 20ms.
I use an ORM (JugglingDB) which makes use of https://github.com/felixge/node-mysql module. The 300ms is the raw query query execution time, as printed in debug mode.
Why do we see such a big difference? Is it the ORM, or Node/JS or the driver is too slow?
Most MongoDB drivers are written in the language that they are used with. The Python driver is written in Python, the Perl driver in Perl. There are a few exceptions, as the PHP driver is written in C and the Python driver as an optional C extension to speed things up.
The node-mongodb-native driver is written all in JavaScript: https://github.com/mongodb/node-mongodb-native. It make sense as the NodeJS platform is optimised for this and there should be no adverse effects.
Related
I have a fairly busy DB2 on Windows server - 9.7, fix pack 11.
About 60% of the CPU time used by all queries in the package cache is being used by the following two statements:
CALL SYSIBM.SQLSTATISTICS(?,?,?,?,?,?)
CALL SYSIBM.SQLPRIMARYKEYS(?,?,?,?)
I'm fairly decent with physical tuning and have spent a lot of time on SQL tuning on this system as well. The applications are all custom, and educating developers is something I also spend time on.
I get the impression that these two stored procedures are something that perhaps ODBC calls? Reading their descriptions, they also seem like things that are completely unnecessary to do the work being done. The application doesn't need to know the primary key of a table to be able to query it!
Is there anything I can tell my developers to do that will either eliminate/reduce the execution of these or cache the information so that they're not executing against the database millions of times and eating up so much CPU? Or alternately anything I can do at the database level to reduce their impact?
6.5 years later, and I have the answer to my own question. This is a side effect of using an ORM. Part of what it does is to discover the database schema. Rails also has a similar workload. In Rails, you can avoid this by using the schema cache. This becomes particularly important at scale. Not sure if there are equivalencies for other ORMs, but I hope so!
When querying Sphinx through SphinxQL would you gain the standard benefits of using mysqli/pdo in PHP?
In additions is there any benefit to using prepared statements with SphinxQL? Are they even supported?
I don't think proper binary (ie in the protocol - server-side) prepared statements are supported. It would have to be software emulated (client-side), which wouldn't bring much benefit.
In general one of the main reasons (other than sql injection protection) for prepared statements, is to avoid the overhead of full SQL parsing on every command. the sql dialect understood by sphinx is much simpler than a full blown database server, so it should in general be much quicker that parsing the incoming statements.
You may as well use mysqli I would think, but PDO wouldnt bring much benefit.
But at the end of the day, use which is most familiar to you, rather than worrying about the tiny benefits each might bring :)
Casbah is another layer on top of Java driver for MongoDb. Would this reduce performance of queries and updates in MongoDb database in scala projects when using Casbah as opposed to directly using the Java driver?
(I'm the author and maintainer of Casbah)
It should be negligible - I have worked hard over the last 2 years to keep the thickness of any wrapping to the bare minimum.
I'd venture to guess that most users will never begin to experience ANY kind of noticable pain or slowdown as a result of Casbah's wrapper code over using the straight Java driver.
In many cases because of optimisations Scala is able to make at compile time, I have also observed places where Casbah performs flat out better (especially for iteration type operations) than the Java driver alone.
I am writing a custom server, which should be very performant.
It has 100.000-600.000 clients connected, and like 10 million records stored.
Database will run on a single server.
The server code is realized via twisted framework (in python).
Now I had it use MySQL, but I think a NoSQL database would be much more efficient (no complex queries, many simple writes / timestamp changes and many simple reads).
Which NoSQL database should I go for? Easy indexing would be a plus, I want the option to search the database from an administration system, create groups from logs containing a specific keyword and stuff like that.
I had a look at Cassandra and MongoDB, MongoDB seemed easier to get in / use for me.
Thanks for the help!
As far as pure learning curve goes, MongoDB has positioned itself to be a very friendly alternative to MySQL. Cassandra is a very different beast and will have a higher learning curve. That said, both have the potential to solve your problem based upon what you describe.
You have pretty simple requirements: easy indexing, arbitrary searches, grouping on keyword, etc -- pretty much every NoSQL system would work. It really comes down to the technologies with which you're comfortable. Like C#? Then go with RavenDB -- it can even automatically add indices as you execute queries. Like Erlang? Then you're a freak, but you should go with CouchDB. Like Javascript and JSON? Go with MongoDB.
Personally I really like Mongo, as it feels like a lovely hybrid of SQL and NoSQL databases. You can index the hell out of it (and get amazing performance!), which makes it almost like a RDBMS. You can also use it like a key/value store, and use it like a "giant hashtable in the sky". Still, YMMV. Play with them and see what works for you.
Cassandra is really designed for multiple server nodes, providing transparent replication. So you won't get the best value out of it with a single server host. Cassandra is also designed primarily for large-scale (and sacrifices indexing and flexible queries as a result). 10 million records isn't really very big, so you can afford to try something more flexible but less scalable.
I'm wondering about which driver is the best between the following :
mongodb-csharp driver
simple-mongodb driver
NoRM
which one consider the best !>
I think there are even more flavours: the one you call mongodb-csharp is actually two:
https://github.com/samus/mongodb-csharp
https://github.com/mongodb/mongo-csharp-driver
The first is a bit more mature and is used widely in the field. The second is a recent development, but is coming from 10gen, the creators of mongodb. Implementation is looking good and is rather like the samus driver. If you need in production something right now, I'm not sure what to advise, but in the long run, I'd go for the 10gen driver.
The one thing that it currently doesn't offer is Linq integration. This could be important to you.
I have no experience with the NORM and simple-mongdb drivers.
I would use the official c# driver released by mongoDB.
http://www.mongodb.org/display/DOCS/CSharp+Language+Center
I've been using it and I like it so far.