I have two collections which have a similar document structure besides one has some additional fields. I have written an aggregation for data I need for one of these collections but I would rather prefer merging these collections into one before aggregating so I dont have to run that aggregation twice. Is there any way to do so?
Related
Can anyone tell me if it is possible to search multiple collections at once in Firebase?
What I need to do is search the data in both the 2009 and 2010 collections in bio_data (more collections will need to be added in the future) and extract the document from one of them.
Or it doesn't matter if there is a way to get a name list of the collections inside a document
Firestore does not have the concept of server-side joins or projections across collections. Each query or document read can only take data from a single collection, or from all collections that have the same name with collection group queries.
If you need to load data from two collections, you'll need at least two reads.
For a better understanding I highly recommend watching the Get to know Cloud Firestore playlist.
Michel's answer is correct.
In addition, I would suggest a change to your data model: Instead of having one sub-collection per year, you could have one unique sub-collection and add a year field to the docs in this unique collection.
This way it would be easy to query by year: For one year, for several years with the in operator (up to 10 equality (==) clauses) or for all the years.
And with a Collection Group query you could even query all the data for all the students.
I have a DB containing thousands of collections, each collection containing documents.
How would I query all these documents from all the collections at once?
I have considered using the $lookup aggregation method but from my knowledge, it only combines an additional collection at once, whereas I have thousands of collections that I want to combine and query altogether.
Is there a way to achieve this?
MongoDB operations work on a single collection at a time. As you point out, $lookup can perform a cross-collection lookup, but it won't help you with a large number collections.
Re-design your data model.
I have a mongodb collection that stores raw information coming from an app. I wrote a multi-pipeline aggregation method to generate more meaningful data from the raw documents.
Using the $out operator in my aggregation function I store the aggregation results in another collection.
I would like to be able to either delete raw documents that were already aggregated, or somehow mark those documents so I know not to aggregate again.
I am worried that I cannot guaranty I won't miss out some documents that are created in between or create duplicate aggregated documents.
Is there a way to achieve this?
I have a MongoDB database with about 100 collections. The collections are very similar to each in structure, but the data is different enough that I need to keep the entries in the collections separate. So I am trying to figure out how I can aggregate all the individual collections into a single collection under different keys.
For example, my database currently has the collections:
collection_set_A
collection_set_B
collection_set_C
collection_set_D
...
and I would like to have a single collection that looks like this:
collection_set
|
+-collection_set_A
+-collection_set_B
+-collection_set_C
+-collection_set_D
+-...
So that collection_set_A can now be accessed as collection_set['collection_set_A'].
Is this possible? I have seen a lot of references to map/reduce, but those seem to be more for joining data, and not full collections like this. I'm basically wanting to move data, not join it.
Does anyone know if this is possible?
This is not possible, but a work-around would be to use the copyto method db.collection.copyTo() to copy the contents of one collection to another collection
You can find more information here http://docs.mongodb.org/manual/reference/method/db.collection.copyTo/#db.collection.copyTo
A somewhat similar question is answered here
Can a MongoDB collection have inside it another collection?
I am currently evaluating mongodb for a project I have started but I can't find any information on what the equivalent of an SQL view in mongodb would be. What I need, that an SQL view provides, is to lump together data from different tables (collections) into a single collection.
I want nothing more than to clump some documents together and label them as a single document. Here's an example:
I have the following documents:
cc_address
us_address
billing_address
shipping_address
But in my application, I'd like to see all of my addresses and be able to manage them in a single document.
In other cases, I may just want a couple of fields from collections:
I have the following documents:
fb_contact
twitter_contact
google_contact
reddit_contact
each of these documents have fields that align, like firstname lastname and email, but they also have fields that don't align. I'd like to be able to compile them into a single document that only contains the fields that align.
This can be accomplished by Views in SQL correct? Can I accomplish this kind of functionality in MongoDb?
The question is quite old already. However, since mongodb v3.2 you can use $lookup in order to join data of different collections together as long as the collections are unsharded.
Since mongodb v3.4 you can also create read-only views.
There are no "joins" in MongoDB. As said by JonnyHK, you can either enormalize your data or you use embedded documents or you perform multiple queries
However, you could also use Map-Reduce.
or if you're prepared to use the development branch, you could test the new aggregation framework though maybe it's too much? This new framework will be in the soon-to-be-released 2.2, which is production-ready unlike 2.1.x.
Here's the SQL-Mongo chart also, which may be of some help in your learning.
Update: Based on your re-edit, you don't need Map-Reduce or the Aggregation Framework because you're just querying.
You're essentially doing joins, querying multiple documents and merging the results. The place to do this is within your application on the client-side.
MongoDB queries never span more than a single collection as there is no support for joins. So if you have related data you need available in the results of a query you must either add that related data to the collection you're querying (i.e. denormalize your data), or make a separate query for it from another collection.
I am currently evaluating mongodb for a project I have started but I
can't find any information on what the equivalent of an SQL view in
mongodb would be
In addition to this answer, mongodb now has on-demand materialized views. In a nutshell, this feature allows you to use aggregate and $merge (in 4.2) to create/update a quick view collection that you can query from faster. The strategy is used to update the quick view collection whenever the main collection has a record change. This has the side effect unlike SQL of increasing your data storage size. But the benefits can be huge depending on your querying needs.