I have a mongo database that I didn't create, but need to extract some additional data from. I have access to the database, and can look at and query the tables, but I have been unable to join two tables to provide all the necessary information in my query output.
I have 2 tables that I am interested in. First Setting, which has documents that looks like this (truncated):
{
"_id" : ObjectId("5aff382669153dc20edf945a"),
"key" : "mgmt",
"site_id" : "5aff382669153dc20edf9456",
"advanced_feature_enabled" : false,
"auto_upgrade" : true
}
and site, that looks like this (truncated):
{
"_id" : ObjectId("5aff382669153dc20edf9456"),
"desc" : "Joe's Crab Shack",
}
site_id in the setting table is a foreign key that refers to the hex component _id in the site table.
I would like the output to resemble this, but I have been unsuccessful:
{
"_id" : ObjectId("5aff382669153dc20edf9456"),
"desc" : "Joe's Crab Shack",
"setting" : [
"_id" : ObjectId("5aff382669153dc20edf945a"),
"key" : "mgmt",
"site_id" : "5aff382669153dc20edf9456",
"advanced_feature_enabled" : false,
"auto_upgrade" : true
]
}
I would like to be able to do only a database query, currently using Robo 3T, and not have to resort to scripting or programming. The closest that I have been able to get to the desired outcome is below. This query returns the site documents, with all the setting documents that don't have a site_id foreign key.
db.site.aggregate([
{
$lookup:
{
from: "setting",
localField:"_id.str",
foreignField:"site_id",
as: "setting"
}
}
])
I'm sure that I am missing something simple, but I am very new to MongoDB, and am a little lost due to the terminology differences between SQL and Mongo. For any that are wondering, the database is actually the back end for a large, multisite Ubiquiti controller, and I am looking to create "reports" that provide more insight to devices that require upgrades, what settings don't meet our "default" configuration, etc.
If site_id in setting collection is of type ObjectId, then you can try this
db.site.aggregate([
{
$lookup: {
from: "setting",
localField: "_id",
foreignField: "site_id",
as: "setting"
}
}
])
and you can test it here Mongo Playground
If site_id is of type string, then you cannot do that lookup, as _id and site_id should be matched in type too (ObjectIds), so you can change the setting schema to make the site_id of type ObjectId and refers to site schema
instead of this
site_id: String
use this
site_id: { type: ObjectId, ref: 'site' }
Update
If site_id is a string and you are not able to change the setting schema, then we can do some work around
we can add a new property to the site document in the aggregate pipeline, which will store the _id of the document but as a string not ObjectId
something like this
db.site.aggregate([
{
$addFields: {
siteIdString: { // this is the stringified version of the _id
$toString: "$_id"
}
}
},
{
$lookup: {
from: "setting",
localField: "siteIdString", // then use this string id in lookup stage
foreignField: "site_id",
as: "setting"
}
}
])
you can test it here Mongo Playground2
hope it helps
Related
Here is have two collections
Product
individual Product stock (collectionname is like mongoID)
1.Product collection
db.getCollection("products").insert({ _id:"60dcc2f4c8b03e500a019fda",
"name":"mobile"
})
2 dynamicID collection
db.getCollection("60dcc2f4c8b03e500a019fda").insert(
{ quantity:5,product:"60dcc2f4c8b03e500a019fda" }
)
My Query is like
db.getCollection('products').aggregate([{
$lookup: {
from: '_id',
localField: '_id',
foreignField: 'product',
as: 'stocks'
}
},
])
Getting blank Array
As of now, you can't dynamically lookup collections.
The from field cannot be an expression and must be a string literal.
There is an open feature request for this however it seems it is very unlikely to happen:
Thanks to everyone for voting and giving your input on this request. Currently, there are no plans for supporting variable `from' collection per document, that would require significant changes in the security and query optimization architectures.
Your second insert statement inserted into a collection called "60dcc2f4c8b03e500a019fda". Your $lookup refers to a non-existent collection called "_id". To make this work try this...
Insert into 2 collections...
db.getCollection("products").insert({
_id:"60dcc2f4c8b03e500a019fda",
"name":"mobile"
})
db.getCollection("inventoryStock").insert({
quantity:5,
product:"60dcc2f4c8b03e500a019fda"
})
Run Aggregation...
db.getCollection('products').aggregate([
{
$lookup:
{
from: 'inventoryStock',
localField: '_id',
foreignField: 'product',
as: 'stocks'
}
}
])
Output:
{ "_id" : "60dcc2f4c8b03e500a019fda", "name" : "mobile", "stocks" : [ { "_id" : ObjectId("60e0ffff1e165946802c244a"), "quantity" : 5, "product" : "60dcc2f4c8b03e500a019fda" } ] }
a simple question (hopefully) to mongoDB users
i have 2 collection
collection A have a data with ID which the name (describing the ID) located in collection B.
i would like to query mongo to get all the Items from Collection A sorted ASC by the mapped name from Collection B.
TIA
well,
i found a simple solution for it:
db.createCollection("ColA")
db.createCollection("ColB")
db.ColA.insert({name:"Item1",priorityid:1})
db.ColA.insert({name:"Item2",priorityid:2})
db.ColA.insert({name:"Item3",priorityid:3})
db.ColA.insert({name:"Item4",priorityid:1})
db.ColA.insert({name:"Item5",priorityid:2})
ColA:
{"_id":"5fd7c79f774870001fa36872","name":"Item1","priorityid":1}
{"_id":"5fd7c7a9774870001fa36873","name":"Item2","priorityid":2}
{"_id":"5fd7c7bb774870001fa36875","name":"Item3","priorityid":3}
{"_id":"5fd7c7c8774870001fa36876","name":"Item4","priorityid":1}
{"_id":"5fd7c7d5774870001fa36877","name":"Item5","priorityid":2}
db.ColB.insert({name:"b_high",priorityid:1})
db.ColB.insert({name:"a_medium",priorityid:2})
db.ColB.insert({name:"c_low",priorityid:3})
ColB:
{"_id":"5fd7c810774870001fa3687b","name":"b_high","priorityid":1}
{"_id":"5fd7c842774870001fa3687c","name":"a_medium","priorityid":2}
{"_id":"5fd7c851774870001fa3687d","name":"c_low","priorityid":3}
Query:
db.ColA.aggregate([
{
$lookup:
{
from: "ColB",
localField: "priorityid",
foreignField: "priorityid",
as: "priorityname"
}
}, { $unwind:"$priorityname" }, {
$project:{
_id : 1,
name : 1,
priorityname : "$priorityname.name"
}
},{ $sort : { priorityname : 1 } }
])
result:
{"_id":"5fd7c7a9774870001fa36873","name":"Item2","priorityname":"a_medium"}
{"_id":"5fd7c7d5774870001fa36877","name":"Item5","priorityname":"a_medium"}
{"_id":"5fd7c79f774870001fa36872","name":"Item1","priorityname":"b_high"}
{"_id":"5fd7c7c8774870001fa36876","name":"Item4","priorityname":"b_high"}
{"_id":"5fd7c7bb774870001fa36875","name":"Item3","priorityname":"c_low"}
I am wondering how exactly I would build query to filter based on the attributes of a stored document within a document using MongoDB.
Example:
My first collection is called Movies, it has a document as so:
_id: ObjectID("5d872b7f927f2538e4eefbf5")
mvNumb : "1"
Director : ObjectID("5d8abd243372eb2850ad71e7")
The second collection is called Director, and the director has the following fields:
_id: ObjectID("5d8abd243372eb2850ad71e7")
Name : "Sam"
What I am trying to do, is filter al my movies based on their director name. An ideal query would look like :
db.Movies.Find({"Director.Name : "Sam"})
Try;
db.directors.aggregate([
{
$match: {
Name: "Sam"
}
},
{
$lookup: {
from: "movies",
localField: "_id",
foreignField: "Director",
as: "movies"
}
},
{
$project: {
_id: 0,
movies: 1
}
}
])
You can first get the target director with Name using $match. Then find the movies that director is affiliated with using $lookup, which is essentially a join-like operation, that fetches all such movies, and puts them under a movies field. Then just project those fields only.
Check on mongoplayground
For an existing mongo database, the link between 2 collections is done by :
collA : field collB_id
collB : field _id = ObjectId("a string value")
where collB_id is _id.valueOf()
i.e. : the value of collB_id of collA is "a string value"
but in a $lookup :
localField: "collB_id",
foreignField: _id.valueOf(),
don't work, so what can I do ?
Mongodb v3.6
If i understood you correctly, you have two collections where documents from first collection (collA) reference documents from second collection (collB). And the problem is that you store reference as a string value of that objectId, so you cant use $lookup to join those docs.
collA:
{
"_id" : ObjectId(...),
"collB_id" : "123456...",
...
}
collB:
{
"_id" : ObjectId("123456..."),
...
}
If you are using mongo 4.0+ you can do it with following aggregation:
db.getCollection('collA').aggregate([
{
$addFields: {
converted_collB_id: { $toObjectId: "$collB_id" }
}
},
{
$lookup: {
from: 'collB',
localField: 'converted_collB_id',
foreignField: '_id',
as: 'joined'
}
}
]);
Mongo 4.0 introduces new aggregation pipeline operators $toObjectId and $toString.
That allows you to add new field which will be an ObjectId created from the string value stored in collB_id, and use that new field as localField in lookup.
I would strongly advise you not to store ObjectIds as strings.
You already experienced the problem with $lookup.
Also, size of ObjectId is 12 bytes while its hex representation is 24 bytes (that is twice the size). You will probably want to index that field as well, so you want it to be as small as possible.
ObjectId also contains timestamp which you can get by calling getTimestamp()
Make your life easier by using native types when possible!
Hope this helps!
I'm developing an api in which I will return user A blog post's detail.
So my response from api will be as below:
{
BlogDetail : {
id : 1,
title : 'My blog',
....
totalComments : 123, //Always present
//Add comments based on user parameter getComment in api -- If true append else dont
comments : [{
user : 'Steve Smith',
comment ; 'Virat learn from me how to bat in test cricket'
},
....]
},
..... //Other Fields
}
Blog Schema: It wont have totalComments field in schema.
{
_id : 1,
title: 'xyz',
....
}
Comment schema :
{
_id : 1,
blog_id: 1,
user : 'abc',
comment: 'xyz'
....
}
Here Blog Detail & Comments are stored in Blogs and Comments collections respectively.
Based on user parameter I will append Comments in response. But will always return total number of comments on that post.
Now my question is :
1 How to join two collection in mongoose?
2 As per MongoDB db.collection.count() and db.collection.find().count() are equal. So it will scan through all the documents? So its better to use db.collection.find() and store it some where so I can letter use them?
How to improve performance?
check lookup in aggregate
```
BlogDetail.aggregate([{
$lookup: {
from: "Comments",
localField: "_id",
foreignField: "Comments",
as: "Comments"
}
}]).exec(function(err, students) {
// do whatever you want
});
```