i've been developing a asp mvc website for almost a year now exclusively on mongodb. i've loved it for the most part. development productivity has been great using a C# mongodb driver and tools like mongovue.
however, i've started to reach a point where there are things i really wish i had a SQL server database for. simple tasks like updating a record in the DB and only mildly complex queries to generate some type of report are becoming a pain.
i read an article somewhere that in order for NOSSQL to succeed there needs to be a standard query language for it, and tools developed around it. i'm guessing this is far far away, so right now i'm stuck trying to deal with these things.
i think eventually i will have to have a dual solution with monogDB and sql server. i don't think i will ever get to the point where i am as productive updating and writing queries for mongoDB as i was with sql server.
how are you guys dealing with this when using NOSQL like mongodb? are you facing the same issues as me?
One solution you may consider is LINQPad. You can set up a template with a reference to 10Gen's drivers and write ad-hoc, C# MongoDB queries like you would in your code. My team and I use this method to address the very problem you mention.
Try it out (it's free) and see if it can help with the simple, day-to-day queries you come up with.
Edit I also support Chris's suggestion of familiarizing yourself with the native JSON query language. Nothing beats a quick console window for speed, if you know the syntax.
The official C# driver will probably get a LINQ provider some time in the future, so that'd give .NET devs a familiar syntax for querying and maybe help with initial productivity. There're also some nice docs that help relate MongoDB queries back to SQL:
SQL to Mongo Mapping Chart
SQL to MongoDB (PDF)
These are great for learning, but to get the most out of Mongo it's well worth investing time getting used to the native JSON query syntax and Mongo-specific concepts like map-reduce.
Since your questions asks,
how are you guys dealing with this when using NOSQL like mongodb?
I thought I'd chime in. I felt your pain when working with another NOSQL database, RavenDB.
I wrote a Linqpad driver specifically for ad hoc interactions with RavenDB.
https://github.com/ronnieoverby/RavenDB-Linqpad-Driver
Related
I have been previously learning, how to make applications with ionic2, Angular2 --> For Frontend.
And backend--> Firebase.
Now I have to change my backend to MongoDB.So, What all important things I should learn in order to use MongoDB?Please suggest.
Thank you.
Note: Please let me know if any more inputs are needed relating to this.
You can learn following to accomplish a decent backend with mongodb :
Learn mongodb with some latest aggregation features to make sure the your queries are optimized. mongodb has great documentation which can help you. Install mongo shell and play around to get hands on it.
Make sure your have good cluster-replica management to make sure your data is backed up properly.
Extremely important, make sure you add indexes on the fields which you find are reference points for majority of your queries. Trust me, adding a simple index to a commonly referenced field makes a huge difference. Also, just as an addon, check what type of index you would need based on your requirement. There are various available in mongodb documentation.
Make sure you structure your collections in such a way that there are not many $looups(inner joins) however still making sure how you can keep it scalable. It totally depends on your application schema though.
Try hands on Mongodb Map-Reduce as its very handy.
Install RoboMongo, a GUI client for mongodb if you are used to stuff like phpmyadmin.
Node js(optional if required) :
1. Learn express which is really easy. It will help you to get started with basic routing, cors handling , middlewares and such http-web related stuff.
2. Mongoose, which is a mongodb package for node. There are many others, however if you are familiar with ORM then mongoose is really handy to get you going with simple steps.
3. Authentication for RESTful APIs : There are many options available to be honest. However, based on my experience, you can use passport-jwt, which is really simple and effective.
I'm currently planning a social-media application - especially the backend.
Basically I have all the social aspects for which I want to use SQL (PostgreSQL I guess) but I also have geolocations organized in lists (so many-to-one) which will propably make out the biggest ammount of data. I know that PostgreSQL has modules for GIS capabilities and my initial thought was to just use PostgreSQL for everything, just for the sake of simplicity and because performance of Geolocation searches should be around the same for both systems, if not even in favor of PostgreSQL. I can also use JSON Type in PostgreSQL so it basically has the most obvious advantages of MongoDB covered.
On the other hand I'm affraid of scalability as the geolocations are going to be the biggest chunk of data and the tables are propably going have heaps of rows.
So my thought now is to implement geolocations in MongoDB with its easy scalability, easy to use geolocation search and embedd e.g Comments/Likes for a geolocation directly into the document, which would make the geolocation reads/searches way easier but then again I had to combine this data with social data from SQL, e.g fetch all users that commented a geolocation and get their profile info from PostgreSQL and 'manually' combine it. Even though parts of this could be done on frontend saving me a lot of resources.
I'm not sure how good this idea performs and if I'm really doing myself a favor there.
tldr: Use PostgreSQL.
Long answer:
You are trying to pre-optimize for a problem you don't even know if you will have. You don't know how many geolocations you will have, what the usage behaviors will be of your users and you probably don't even have any users yet.
I've used MongoDB before and migrated to PostgreSQL. There are many, many features and benefits to using a 'real' database for storing highly structured data. I suggest googling around for 'PostgreSQL vs X' articles, but the overall consensus that I've found is that PGSQL is extremely mature, reliable, performant and supported.
From my personal experience using Mongo then switching to PGSQL, I will never use Mongo again unless PGSQL (or another full-fledged SQL database) is completely falling over and I've spent months fixing it. Even then I'd take a hard look at other NoSQL databases too. PGSQL has so many amazing features and powerful tools that make it a joy to use.
For the seemingly few things you think you need Mongo for, PGSQL can do, and do just as well or better. It has native JSON types with indexes, geo support, full text indexing, etc. PGSQL has been around longer and has more support (useful for debugging, performance tuning, etc).
Regardless of which technologies you are thinking of using, you can't make any sort of informed decision if you don't:
Test with large data sets
and
Know your usage patterns and data volumes
So at this point I'd pick the more matured and powerful tool and setup monitoring for it. Watch the usage and performance of PGSQL, see how it holds up. Research best practices for PGSQL. Get to know it, learn it, dive in deep. When it comes to scaling individual services, each one is somewhat unique and will not fit a simple "Should I use X or Y?" question.
Good luck!
I'm developing a brand new project in Scala. It's just an application for a bunch of CRUD operations, however, because of some eccentric requirements, Play2 or Lift does not fit the bill, so I'm going to develop the application from the ground up. This means that Anorm or ScalaQuery becomes less obvious choices for database integration, and leaves me with the question: is it time to try something new?
My past technology stacks mostly included Java and PostgreSQL and I have experience with both ORM and plain SQL. Are NoSQL database management systems like MongoDB a good replacement for a typical RDBMS or are they special case application data stores? Also, how does the choice of database effect the greater Scala system design (if at all)? For example, the fact that you are using a JSON-like interface to talk to the database, and JSON between the web and a REST service, does not mean that much if everything in the middle becomes Scala objects, or does it?
I'm basically asking for someone's experience on moving from relational to object/document type databases, using Scala in particular. I know that good RDBMS integration is promised in the upcoming release of SLICK. So, if a company like TypeSafe decides to make a RDBMS integration part of the TypeSafe stack, then will I be swimming upstream by integrating to MongoDB using Casbah for example?
Apologies if this question appears a bit vague. I do hope that someone with the right insights or experience will be able to help though.
Update:
Apologies for not adding links to SLICK (it being fairly new). Here goes:
Quick overview
Project home
Update 2:
My personal first win for a technology is usually developer productivity - this translates to lightweight and simple: quick to learn, easy to maintain, no magic
I am currently in a similar situation, and since I have some experience with web development and SQL databases, I took it as an opportunity to work with MongoDB, Cashbah (and Scalatra). My experience is still very limited and the project and the amount of data I am working with is pretty small, but here are a few observations I've made.
For the few sets of data I have, performance does not seem to motivate either SQL or NoSQL. However, performance in the presence of huge amounts of data is often listed as a reason for using NoSQL, e.g., by Wikipedia
My documents (entries in the database) arise from benchmarking test suits, and mainly have a static structure, and I am optimistic that I could store them in a fixed-schema SQL database. However, a few substructures are not static, e.g., new test cases are added, new statistics are tracked, others are removed. This was my main motivation for trying a schema-free NoSQL database. Also, because I had the feeling that the document approach of MongoDB makes it much more obvious which data belongs together (i.e., to a document), in contrast to entries in a relational database, where the data would be distributed over various tables and rows, and where a full "document" would need to be reconstructed by joins.
Tools such as Lift-Json or Rogue allow you to work with regular Scala objects in a type-safe, although the data is regularly (de-)serialised as (from) JSON. However, this naturally works best if the structure of your data is mainly static, otherwise, you you are left with using strings to access your data (e.g., for expanding the results of a query using Cashbah).
If you are mainly concerned about a coherent representation of data on server and client side, languages such as Opa or Haxe might be of interest, since they compile to code that can executed on both sides. See this page for "multitarget" or "tierless" languages.
Got too long for a comment. Was just trying to relate my short experience with Scala (about 6 months now, since about when Play2 came out--it's quickly become my go to language).
I've enjoyed using Salat/Casbah with MongoDB in my last few projects; most have been in Play2, but the latest was without a webapp framework. It definitely hasn't felt like swimming upstream.
I would say that there are particular use cases for which I wouldn't use mongo, but it works nicely as a general purpose object data store, especially if you expect to query by id or index and don't need transactions (and will need minimal ad-hoc aggregation type stuff).
Expect to require a separate set of servers dedicated to mongodb (or to use a service dedicated to mongodb), but I guess that's normal for most serious database apps.
I've also used Play2/Anorm, which was surprisingly enjoyable to use for some ad-hoc query dashboard-style report pages. I started trying to go the Squeryl route, but Anorm seemed easier to use for one-off aggregation queries. Haven't looked at SLICK, but it sounds interesting.
It's really hard to say without knowing what problems you would like the app to solve.
I've personally found my productivity increased using NoSQL DBs via REST/JSON. Though bear in mind most NoSQL DBs offer REST interfaces which preclude the need for much middleware, Scala or otherwise, unless you intend to write a webapp with a UI.
If this is a learning exercise, I recommend you try multiple things out, as each NoSQL DB has something different to offer to your toolkit, and have personally found CouchDB, Riak, Neo4j, and MongoDb all with various pluses and drawbacks and good for different purposes.
Hope this helps, good luck.
I'm a computer engineer student and i've got a basically simple assignment but i'd like to make it a little more interesting :). Basically I have to create a simple multiuser, online document versioning system using Java or .NET. Because I'm more a .NET guy I'm gonna use ASP.NET or Silverlight (it's not yet decided).
Anyway the intersting part: I want to use MongoDB to store the documents (they can be virtually anything, video, audio, simple MSOffice files, plain text), each version of them and the related metadata (which library, folder they're in etc). I'd like to put anything else, like users, permissions etc.(the specs are not yet completed) in to an ordinary relational database. The question is what do you think about this? Do you think this makes any sense or I'm just trying to over complicate the whole thing? Would it be simpler to use only mongodb for everything or leave the nosql thingy entirely out of it? Is there any conventions for this kind of stuff? :)
BTW. under any other circumstences I'd use some free, battle tested solution for doc versioning, but I have to design and implement this myself, and I'm trying to do it at least a little unconventionally :).
Thanks for every comment:), any help appreciated:)
Greets
If you were using something like Mongoid on Rails, this would be trivial. http://mongoid.org/docs/extras.html Look for Versioning. The ODM has it built in for you. I've yet to use Mongo in the .NET world, it's always been something like Raven, so I'm not sure if the libraries for it easily work like Mongoid does.
Mongo handles relational data as well. Things you want normalized (e.q. user records) can have references to other documents in the DB. And of course what ever magic document you want to store works as well.
Doing things unconventionnaly is usually a good way to discover new ways (better ways ?) so it's a good idea.
Using mongodb is a good idea too because of the schemaless property.
USing a relationa database for storing extra information is quite weird and will add more complexity without any advantage (IMO)
I'm new to MongoDB, can anyone explain how it could be used in efficiently in enterprise applications, so as to give good performance (using joins, indexing etc.)
And perhaps also point me to any MongoDB production applications on the web.
For a good introduction to MongoDB, check out The Little MongoDB Book. Here's a list of sites currently using MongoDB in production.
You talk about Joins and Indexes. It seems your head is still in the RDBMS world. NoSQL and Mongo are not just different Relational Databases there a completely different way of managing and thinking of Data. You need to think of your data schema in terms of Structured Objects rather than rows.
Sounds like you need to start from the beginning. MongoDB.org has a lot of the info you're asking for already available. Specifically, read their page on use cases, and the page on production deployments.
A more specific question would receive more comprehensive answers and fewer downvotes.