Mongo Query: how to $lookup with DBRef - mongodb

I have a trouble with $lookup with DBRef. I couldn't find the solution for below scenario anywhere. Someone please help me here?
Suppose the Collection A is
{
"_id" : ObjectId("582abcd85d2dfa67f44127e0"),
"status" : NumberInt(1),
"seq" : NumberInt(0) }
and Collection B:
{
"_id" : ObjectId("582abcd85d2dfa67f44127e1"),
"Name" : "from B Collection"
"bid" : DBRef("B", ObjectId("582abcd85d2dfa67f44127e0")) }
I have spent lot of time in aggregating above two collections. I am looking for the output as below.
{
"_id" : ObjectId("582abcd85d2dfa67f44127e0"),
"status" : NumberInt(1),
"seq" : NumberInt(0),
B: [
{
"_id" : ObjectId("582abcd85d2dfa67f44127e1"),
"Name" : "from B Collection"
}
]}
Please help me with the Mongo query to retrieve the result in the above format. Thanks in advance

Ideally you would be able to change the DBRef to a plain objectId or just string type. As noted in this post, it can be convoluted to use a DBRef in a lookup. The key is an $addFields stage with {$objectToArray: "$$ROOT.bid"} to get the DBRef value into a usable format.
You'll need to start the aggregation from collection B since that is where the reference is -- and that DBRef needs massaging before doing the lookup. Knowing that is the case, maybe the goal output shape might change; however, here is an aggregation that works to get you what you need:
db.getCollection('B').aggregate([
{$addFields: {fk: {$objectToArray: "$$ROOT.bid"}}},
{$lookup: {
from: 'A',
localField: 'fk.1.v',
foreignField: '_id',
as: 'A'
}},
// the below is transforming data into the format in the example
{$addFields: {'A.B': {_id: '$_id', Name: '$Name'}}},
{$unwind: '$A'},
{$replaceRoot: {newRoot: '$A'}}
])
You might need to do a groupBy if there are multiple B matches you need to group into an array.

Related

MongoDB 3.6 - How to modify this aggregate query with $lookup so it can be run on multiple documents?

I have 2 collections, I want to do a join on them using the $lookup operator. The fields I want to join have different types. I am not allowed to use the $toObjectId Operator due to challenges of Upgrading the DB, nor I am allowed to modify the existing data.
Prescription collection
{"_id" : ObjectId("p1"), "currentOrderId" : "o1"}
{"_id" : ObjectId("p2"), "currentOrderId" : "o2"}
{"_id" : ObjectId("p3"), "currentOrderId" : "o3"}
{"_id" : ObjectId("p4"), "currentOrderId" : "o4"} ...
Order Collection
{"_id" : ObjectId("o1"), "orderNumber": "123"}
{"_id" : ObjectId("o2"), "orderNumber": "123"}
{"_id" : ObjectId("o3"), "orderNumber": "123"}
{"_id" : ObjectId("o4"), "orderNumber": "123"} ...
I have to join the order documents to the prescription documents so my result collection looks something like this :
{"_id" : ObjectId("p1"),
"currentOrderId" : "o1",
"currentOrder": {
"_id" : ObjectId("o1"),
"orderNumber": "123"
}
...... ( many more documents) .......
I was able to join only one pair of prescription and order document using the following query :
db.prescription.aggregate([
{ $match: {"_id":ObjectId("p1")}},
{ $addFields: { "coid": ObjectId("o1")}},
{ $lookup: {
from: "order",
localField: "coid",
foreignField: "_id",
as: "currentOrder"
}},
{
$unwind: "$currentOrder",
},
])
Note I hardcoded the $match and $addfields to test my query. Now I want to apply this query to be able to join all orders to the prescription collection in batch.
Here is what I think:
We have no knowledge of how many prescriptions are there, nor their _id's, therefore we need to run one query to get their _ids. And store it in a list.
I need to apply this query in batch, so I might remove the match operator.
I also need to get a list of order _id's, I can run another query on the prescription collection in order to get a string list of order ids.
I need to somehow convert the string list from step 3 into objectID, in batch.
Sorry, that is all I have right now. I will consider answers involving more queries, as long as the number of queries is constant. Note that I cannot use any operators introduced after version 3.6.
Any bits of help or tips are greatly appreciated.

$lookup on Embedded Documents in MongoDB: Does order of values matter?

I have two similar collections within the same database which I am trying to merge using $lookup and the aggregate pipeline. Their _ids, which I'm using as the matching field, contain the same values, but in a different order:
Collection1:
{ "_id" : { "State" : "Vermont", "Race" : "Black American or African American" }, "Population" : 6456 }
Collection2:
{ "_id" : { "Race" : "Multiracial", "State" : "Arkansas" }, "Population" : 48996 }
I tried running the aggregate pipeline as follows:
db.Collection1.aggregate([{$lookup: {from: "Collection2", localField: "_id", foreignField: "_id", as: "Population"}}])
However, when I do that, I get:
{ "_id" : { "Race" : "Multiracial", "State" : "Arkansas" }, "Population" : [ ] }
I'd like to get the values for population within the array. I'm fairly new to MongoDB. Is there something wrong with my syntax for the aggregate command, or is it failing because 'Race' and 'State' are listed in a different order within the embedded document _id? Does the order of the values matter for matching on embedded documents?
Thank you so much for your time, and I appreciate any suggestions.

Mongo Db : $lookup (aggregation) / join with two different data types

I am trying to use mongo 3.4 $lookup:function
db.orders.aggregate([
{
$lookup:
{
from: "inventory",
localField: "item",
foreignField: "sku",
as: "inventory_docs"
}
}
])
orders:
{ "_id" : 1, "itemid" : "1234", "price" : 12, "quantity" : 2 }
invdentory :
{ "_id" : 1, "skuid" : 123, description: "product 1", "instock" : 120 }
The problem here is the fields to be joined are of string and integer. How can i make this lookup possible in mongo
It's not possible to change the datatype inside the $lookup step of the aggregation pipeline. The topic has already been discussed here:
Change type of field inside mongoDB aggregation and does $lookup utilises index on fields or not?
how to convert string to numerical values in an aggregate query in MongoDB.
In both threads the final solution was: you must previously convert the datatype programmatically

Query MongoDB subfield query select/find? not sure what to do

I have a dataset that looks like this:
{ "_id" : ObjectId( "----number-----" ),
"interaction" : { "author" : { "link" : "------",
"avatar" : "----link---",
"name" : "-----name----",
"id" : "12345678" },
How do I query mongodb to give me a list of id's from this sort of field? If you know, I also need to group them descending by count. Is there an equivalent of group by as in sql?
Thanks!
Look into the MongoDB Aggregation Framework.
The query you're looking for should be like this.
db.collection.aggregate([{$group: {_id: '$iteraction.id', hits: {$sum: 1}}}, {$sort: {hits: -1}}])

Save Subset of MongoDB Collection to Another Collection

I have a set like so
{date: 20120101}
{date: 20120103}
{date: 20120104}
{date: 20120005}
{date: 20120105}
How do I save a subset of those documents with the date '20120105' to another collection?
i.e db.subset.save(db.full_set.find({date: "20120105"}));
I would advise using the aggregation framework:
db.full_set.aggregate([ { $match: { date: "20120105" } }, { $out: "subset" } ])
It works about 100 times faster than forEach at least in my case. This is because the entire aggregation pipeline runs in the mongod process, whereas a solution based on find() and insert() has to send all of the documents from the server to the client and then back. This has a performance penalty, even if the server and client are on the same machine.
Here's the shell version:
db.full_set.find({date:"20120105"}).forEach(function(doc){
db.subset.insert(doc);
});
Note: As of MongoDB 2.6, the aggregation framework makes it possible to do this faster; see melan's answer for details.
Actually, there is an equivalent of SQL's insert into ... select from in MongoDB. First, you convert multiple documents into an array of documents; then you insert the array into the target collection
db.subset.insert(db.full_set.find({date:"20120105"}).toArray())
The most general solution is this:
Make use of the aggregation (answer given by #melan):
db.full_set.aggregate({$match:{your query here...}},{$out:"sample"})
db.sample.copyTo("subset")
This works even when there are documents in "subset" before the operation and you want to preserve those "old" documents and just insert a new subset into it.
Care must be taken, because the copyTo() command replaces the documents with the same _id.
There's no direct equivalent of SQL's insert into ... select from ....
You have to take care of it yourself. Fetch documents of interest and save them to another collection.
You can do it in the shell, but I'd use a small external script in Ruby. Something like this:
require 'mongo'
db = Mongo::Connection.new.db('mydb')
source = db.collection('source_collection')
target = db.collection('target_collection')
source.find(date: "20120105").each do |doc|
target.insert doc
end
Mongodb has aggregate along with $out operator which allow to save subset into new collection. Following are the details :
$out Takes the documents returned by the aggregation pipeline and writes them to a specified collection.
The $out operation creates a new collection in the current database if one does not already exist.
The collection is not visible until the aggregation completes.
If the aggregation fails, MongoDB does not create the collection.
Syntax :
{ $out: "<output-collection>" }
Example
A collection books contains the following documents:
{ "_id" : 8751, "title" : "The Banquet", "author" : "Dante", "copies" : 2 }
{ "_id" : 8752, "title" : "Divine Comedy", "author" : "Dante", "copies" : 1 }
{ "_id" : 8645, "title" : "Eclogues", "author" : "Dante", "copies" : 2 }
{ "_id" : 7000, "title" : "The Odyssey", "author" : "Homer", "copies" : 10 }
{ "_id" : 7020, "title" : "Iliad", "author" : "Homer", "copies" : 10 }
The following aggregation operation pivots the data in the books collection to have titles grouped by authors and then writes the results to the authors collection.
db.books.aggregate( [
{ $group : { _id : "$author", books: { $push: "$title" } } },
{ $out : "authors" }
] )
After the operation, the authors collection contains the following documents:
{ "_id" : "Homer", "books" : [ "The Odyssey", "Iliad" ] }
{ "_id" : "Dante", "books" : [ "The Banquet", "Divine Comedy", "Eclogues" ] }
In the asked question, use following query and you will get new collection named 'col_20120105' in your database
db.products.aggregate([
{ $match : { date : "20120105" } },
{ $out : "col_20120105" }
]);
You can also use $merge aggregation pipeline stage.
db.full_set.aggregate([
{$match: {...}},
{ $merge: {
into: { db: 'your_db', coll: 'your_another_collection' },
on: '_id',
whenMatched: 'keepExisting',
whenNotMatched: 'insert'
}}
])