I have a MongoDB database with about 100 collections. The collections are very similar to each in structure, but the data is different enough that I need to keep the entries in the collections separate. So I am trying to figure out how I can aggregate all the individual collections into a single collection under different keys.
For example, my database currently has the collections:
collection_set_A
collection_set_B
collection_set_C
collection_set_D
...
and I would like to have a single collection that looks like this:
collection_set
|
+-collection_set_A
+-collection_set_B
+-collection_set_C
+-collection_set_D
+-...
So that collection_set_A can now be accessed as collection_set['collection_set_A'].
Is this possible? I have seen a lot of references to map/reduce, but those seem to be more for joining data, and not full collections like this. I'm basically wanting to move data, not join it.
Does anyone know if this is possible?
This is not possible, but a work-around would be to use the copyto method db.collection.copyTo() to copy the contents of one collection to another collection
You can find more information here http://docs.mongodb.org/manual/reference/method/db.collection.copyTo/#db.collection.copyTo
A somewhat similar question is answered here
Can a MongoDB collection have inside it another collection?
Related
I want to use the collections as DBs!.. make a connection from nodejs where you can use a collection as a normal database.??
The purpose of this is that I'm going to create two totally different apps, with different collections and documents, but there are data that I need to relate between the two apps. And if the two apps are using the same database, it would be much easier for me.
Well.. MongoDB doesnt do that, what can you do is something like give a prefix name to your collections, "projectA_Users , projectB_Users" and to relate data use the $lookup Aggregate operator to get your relationship. I see something like that on FireBase that you can create a collection inside a document and use find, update, remove on it (here)
I want to manage multiple projects data in mongoDB. Each project contains multiple users from multiple departments with multiple role assigned to them. plus certain task is assigned to each user. Now I am confused about schema, not able to decide which entity should be kept as collection & which one as document ? What is the best efficient way to store ?
should I keep all under single collection as embedded documents or in separate collection ?
Thanks
First of all if you are using mongodb you should know why are you using it. MongoDB is not about normalize stuff. If you are able to create data structure is de-normalize way then and only then go for MongoDB.
I think you should maintain one single document containing all the mentioned things above. But the scenario which you have mentioned above is good for relational database. you need only 3 entities in relational database and your problem is solved.
Still if you want to go for mongodb you can go with one collection only. which contains project details number of users working there and their roles and department.
Do many mongodb collections have a big impact on mongodb performance, memory and capacity? I am designing an api with mvc pattern, and a collection is being created for each model. I question the way I am doing now.
MongoDB with the WirdeTiger engine supports an unlimited number of collections. So you are not going to run into any hard technical limitations.
When you wonder if something should be in one collection or in multiple collections, these are some of the considerations you need to keep in mind:
More collections = more maintenance work. Sharding is configured on the collection level. So having a large number of collections will make shard configuration a lot more work. You also need to set up indexes for each collection separately, but this is quite easy to automatize, because createIndex on an index which already exists does nothing.
The MongoDB API is designed in a way that every database query operates on one collection at a time. That means when you need to search for a document in n different collections, you need to perform n queries. When you need to aggregate data stored in multiple collections, you run into even more problems. So any data which is queried together should be stored together in the same collection.
Creating one collection for each class in your model is usually a god rule of thumb, but it is not a golden hammer solution. There are situations where you want to embed object in their parent-object documents instead of putting them into a separate collection. There are also cases where you want to put all objects with the same base-class in the same collection to benefit from MongoDB's ability to handle heterogeneous collections. But that goes beyond the scope of this question.
Why don't you use this and test your application ?
https://docs.mongodb.com/manual/tutorial/evaluate-operation-performance/
By the way your question is not completely clear... is more like a "discussion" rather than question. And you're asking others to evaluate your work instead of searching the web the rigth approach.
I am currently evaluating mongodb for a project I have started but I can't find any information on what the equivalent of an SQL view in mongodb would be. What I need, that an SQL view provides, is to lump together data from different tables (collections) into a single collection.
I want nothing more than to clump some documents together and label them as a single document. Here's an example:
I have the following documents:
cc_address
us_address
billing_address
shipping_address
But in my application, I'd like to see all of my addresses and be able to manage them in a single document.
In other cases, I may just want a couple of fields from collections:
I have the following documents:
fb_contact
twitter_contact
google_contact
reddit_contact
each of these documents have fields that align, like firstname lastname and email, but they also have fields that don't align. I'd like to be able to compile them into a single document that only contains the fields that align.
This can be accomplished by Views in SQL correct? Can I accomplish this kind of functionality in MongoDb?
The question is quite old already. However, since mongodb v3.2 you can use $lookup in order to join data of different collections together as long as the collections are unsharded.
Since mongodb v3.4 you can also create read-only views.
There are no "joins" in MongoDB. As said by JonnyHK, you can either enormalize your data or you use embedded documents or you perform multiple queries
However, you could also use Map-Reduce.
or if you're prepared to use the development branch, you could test the new aggregation framework though maybe it's too much? This new framework will be in the soon-to-be-released 2.2, which is production-ready unlike 2.1.x.
Here's the SQL-Mongo chart also, which may be of some help in your learning.
Update: Based on your re-edit, you don't need Map-Reduce or the Aggregation Framework because you're just querying.
You're essentially doing joins, querying multiple documents and merging the results. The place to do this is within your application on the client-side.
MongoDB queries never span more than a single collection as there is no support for joins. So if you have related data you need available in the results of a query you must either add that related data to the collection you're querying (i.e. denormalize your data), or make a separate query for it from another collection.
I am currently evaluating mongodb for a project I have started but I
can't find any information on what the equivalent of an SQL view in
mongodb would be
In addition to this answer, mongodb now has on-demand materialized views. In a nutshell, this feature allows you to use aggregate and $merge (in 4.2) to create/update a quick view collection that you can query from faster. The strategy is used to update the quick view collection whenever the main collection has a record change. This has the side effect unlike SQL of increasing your data storage size. But the benefits can be huge depending on your querying needs.
In the project I am currently working on, it seems to make more sense efficiency wise if I create a nested document that contains a list of "pointers" to information stored in other collections. That way this nested document can be easily used to retrieve a list of relevant information. The question is, how to do this? Is there a way to store locations of other information in a field in MongoDB? If not, could anyone suggest a scheme that is equally or more efficient? Thanks very much!
There is no GOOD way to do this. If this is what you're looking for, you should be using a relational database.
But if you HAVE to go by this route then, why not store ID's in a document, and then link those ID's to documents in the other collection.
Unfortunately, this would require you to do 2 separate queries, as Mongo does not support compound queries that span documents.