I'm developing an api in which I will return user A blog post's detail.
So my response from api will be as below:
{
BlogDetail : {
id : 1,
title : 'My blog',
....
totalComments : 123, //Always present
//Add comments based on user parameter getComment in api -- If true append else dont
comments : [{
user : 'Steve Smith',
comment ; 'Virat learn from me how to bat in test cricket'
},
....]
},
..... //Other Fields
}
Blog Schema: It wont have totalComments field in schema.
{
_id : 1,
title: 'xyz',
....
}
Comment schema :
{
_id : 1,
blog_id: 1,
user : 'abc',
comment: 'xyz'
....
}
Here Blog Detail & Comments are stored in Blogs and Comments collections respectively.
Based on user parameter I will append Comments in response. But will always return total number of comments on that post.
Now my question is :
1 How to join two collection in mongoose?
2 As per MongoDB db.collection.count() and db.collection.find().count() are equal. So it will scan through all the documents? So its better to use db.collection.find() and store it some where so I can letter use them?
How to improve performance?
check lookup in aggregate
```
BlogDetail.aggregate([{
$lookup: {
from: "Comments",
localField: "_id",
foreignField: "Comments",
as: "Comments"
}
}]).exec(function(err, students) {
// do whatever you want
});
```
Related
I have a mongo database that I didn't create, but need to extract some additional data from. I have access to the database, and can look at and query the tables, but I have been unable to join two tables to provide all the necessary information in my query output.
I have 2 tables that I am interested in. First Setting, which has documents that looks like this (truncated):
{
"_id" : ObjectId("5aff382669153dc20edf945a"),
"key" : "mgmt",
"site_id" : "5aff382669153dc20edf9456",
"advanced_feature_enabled" : false,
"auto_upgrade" : true
}
and site, that looks like this (truncated):
{
"_id" : ObjectId("5aff382669153dc20edf9456"),
"desc" : "Joe's Crab Shack",
}
site_id in the setting table is a foreign key that refers to the hex component _id in the site table.
I would like the output to resemble this, but I have been unsuccessful:
{
"_id" : ObjectId("5aff382669153dc20edf9456"),
"desc" : "Joe's Crab Shack",
"setting" : [
"_id" : ObjectId("5aff382669153dc20edf945a"),
"key" : "mgmt",
"site_id" : "5aff382669153dc20edf9456",
"advanced_feature_enabled" : false,
"auto_upgrade" : true
]
}
I would like to be able to do only a database query, currently using Robo 3T, and not have to resort to scripting or programming. The closest that I have been able to get to the desired outcome is below. This query returns the site documents, with all the setting documents that don't have a site_id foreign key.
db.site.aggregate([
{
$lookup:
{
from: "setting",
localField:"_id.str",
foreignField:"site_id",
as: "setting"
}
}
])
I'm sure that I am missing something simple, but I am very new to MongoDB, and am a little lost due to the terminology differences between SQL and Mongo. For any that are wondering, the database is actually the back end for a large, multisite Ubiquiti controller, and I am looking to create "reports" that provide more insight to devices that require upgrades, what settings don't meet our "default" configuration, etc.
If site_id in setting collection is of type ObjectId, then you can try this
db.site.aggregate([
{
$lookup: {
from: "setting",
localField: "_id",
foreignField: "site_id",
as: "setting"
}
}
])
and you can test it here Mongo Playground
If site_id is of type string, then you cannot do that lookup, as _id and site_id should be matched in type too (ObjectIds), so you can change the setting schema to make the site_id of type ObjectId and refers to site schema
instead of this
site_id: String
use this
site_id: { type: ObjectId, ref: 'site' }
Update
If site_id is a string and you are not able to change the setting schema, then we can do some work around
we can add a new property to the site document in the aggregate pipeline, which will store the _id of the document but as a string not ObjectId
something like this
db.site.aggregate([
{
$addFields: {
siteIdString: { // this is the stringified version of the _id
$toString: "$_id"
}
}
},
{
$lookup: {
from: "setting",
localField: "siteIdString", // then use this string id in lookup stage
foreignField: "site_id",
as: "setting"
}
}
])
you can test it here Mongo Playground2
hope it helps
I have post schema like this
const postSchema = new mongoose.Schema({
title: { type: String, minlength: 2 },
description: { type: String },
categoryId: [{ type: mongoose.Schema.ObjectId, ref: "category" }],
});
I have to fetch random documents from the posts and populate all categories in it. What I have achieved so far is this
await Post.aggregate([
{
$lookup: {
from: "categories",
localField: "categoryId",
foreignField: "_id",
as: "categoryId",
},
},
]).sample(5)
Need to do
Suppose i have an id of a category in a variable catId, what I need to do is to filter out those posts which contains this id in their array of categoryId before getting those random posts. By filtering out I mean i need random posts which has catId in their array.
You can do it like below :
Post.aggregate([
/** Use match to Get docs where 'categoryId' has passed in value */
{
$match: {
categoryId: catId /** Input 'catId', make sure you convert input request of string to 'ObjectId()' to match with type in DB*/
}
},
{
$sample: { /** Picks some random 3 docs (Sometimes a single doc can be repeated) */
size: 3
}
},
{
$lookup: {
from: "categories",
localField: "categoryId",
foreignField: "_id",
as: "categoryId"
}
}
])
Test : MongoDB-Playground
Ref : $sample
Note : So $sample can get the job done, but it has few limitations read through the doc for more info. Use $explain to check if your query is performing well or not, In any case if $sample is not working for you, then you can actually use $skip & $limit to get this job done.
Let's say if you can auto generate some number at least 3 less than collections size for skip & skip those docs to eventually do limit of 3 on remaining docs. You can have a sort in between skip & limit - such way do test this for better uniqueness. Check this : skip-and-limit-in-aggregation-framework - Usually this used for pagination but might help you in least case.
I am wondering how exactly I would build query to filter based on the attributes of a stored document within a document using MongoDB.
Example:
My first collection is called Movies, it has a document as so:
_id: ObjectID("5d872b7f927f2538e4eefbf5")
mvNumb : "1"
Director : ObjectID("5d8abd243372eb2850ad71e7")
The second collection is called Director, and the director has the following fields:
_id: ObjectID("5d8abd243372eb2850ad71e7")
Name : "Sam"
What I am trying to do, is filter al my movies based on their director name. An ideal query would look like :
db.Movies.Find({"Director.Name : "Sam"})
Try;
db.directors.aggregate([
{
$match: {
Name: "Sam"
}
},
{
$lookup: {
from: "movies",
localField: "_id",
foreignField: "Director",
as: "movies"
}
},
{
$project: {
_id: 0,
movies: 1
}
}
])
You can first get the target director with Name using $match. Then find the movies that director is affiliated with using $lookup, which is essentially a join-like operation, that fetches all such movies, and puts them under a movies field. Then just project those fields only.
Check on mongoplayground
This question already has an answer here:
Matching ObjectId to String for $graphLookup
(1 answer)
Closed 4 years ago.
I am trying to use the Mongoose aggregate pipeline to query my User collection and Company collection simultaneously.
Specifically, I am trying to return all users and return the user's associated companyName. Each user has a companyId attribute
that maps to a company _id field. Here are my sample user and company documents:
User:
{
_id: ObjectId("5b12ef5ba07ce1f8b212f07b"),
companyId: "12345"
first: "John",
last: "Doe",
}
Company:
{
_id: ObjectId("12345"),
companyName: "UPS"
}
Here is my query code:
User.aggregate([{
$match: {}
},
{
$lookup: {
localField: "companyId",
from: "company",
foreignField: "_id",
as: "companyInfo"
}
},
{
$unwind: "$companyInfo"
},
{
$project: {
first: 1,
last: 1,
"companyInfo.companyName": 1,
}
}
], (err, users) => {
console.log(users) // returns []
});
I THINK the reason is the companyId is stored as a string in my User collection, but it is an objectId in the company collection.
Can anyone confirm? And if it is the issue, I'm not sure how to resolve it... Can someone help? Thanks in advance!
Your assumption is correct.
When doing $lookup, local field and remote field must be of the same type.
There are no way to do a converter when lookup, afaik.
This is the related issue:
https://jira.mongodb.org/browse/SERVER-22781
The version 3.7.3 has it implemented
For now you might want to change your data type
How to achieve below SQL in MongoShell?
Select TableA.* from TableA where TableA.FieldB in (select TableB.FieldValue from TableB)
Mongo doc gives some example of
db.inventory.find( { qty: { $in: [ 5, 15 ] } } )
I want that array be dynamically from another query. Is it possible?
Extending my question
I have a collection of bot names
bots collection
{
"_id" : ObjectId("53266697c294991f57c36e42"),
"name" : "teoma"
}
I have a collection of user traffic, in that traffic collection, I have a field useragent
userTraffic Collection
{
"_id" : ObjectId("5325ee6efb91c0161cbe7b2c"),
"hosttype" : "http",
"useragent" : "Mediapartners-Google",
"is_crawler" : false,
"City" : "Mountain View",
"State" : "CA",
"Country" : "United States"
}
I want to select all user traffic documents where its useragent contains any name of bot collection
This is what I have come up with
var botArray = db.bots.find({},{name:1, _id:0}).toArray()
db.Sessions.find({
useragent: {$in: botArray}
},{
ipaddress:1
})
Here i believe it is doing equals to comparison, but I want it to do like %% comparison
Once I get the result, I want to do an update to that result set as is_crawler= true
Tried something like this, isn't helpful
db.bots.find().forEach( function(myBot) {
db.Sessions.find({
useragent: /myBot.name/
},{
ipaddress:1
})
});
Another way of looping through the records, but no match found.
var bots = db.bots.find( {
$query: {},
$orderby:{
name:1}
});
while( bots.hasNext()) {
var bot = bots.next();
//print(bot.name);
var botName = bot.name.toLowerCase();
print(botName);
db.Sessions.find({
useragent: /botName/,
is_crawler:false
},{
start_date:1,
ipaddress:1,
useragent:1,
City:1,
State:1,
Country:1,
is_crawler:1,
_id:0
})
}
Not in a single query it isn't.
There is nothing wrong with getting the results from a query and feeding that in as your in condition.
var list = db.collectionA.find({},{ "_id": 0, "field": 1 }).toArray();
results = db.collectionB.find({ "newfield": { "$in": list } });
But your actual purpose is not clear, as using SQL queries alone as the only example of what you want to achieve are generally not a good guide to answer the question. The main cause of this is that you probably should be modelling differently than as you do in relational. Otherwise, why use MongoDB at all?
I would suggest reading the documentation section on Data Modelling which shows several examples of how to approach common modelling cases.
Considering that information, then perhaps you can reconsider what you are modelling, and if you then have specific questions to other problems there, then feel free to ask your questions here.
Finally this is how I could accomplish it.
// Get a array with values for name field
var botArray = db.bots.find({},{name:1}).toArray();
// loop through another collection
db.Sessions.find().forEach(function(sess){
if(sess.is_crawler == false){ // check a condition
// loop in the above array
botArray.forEach(function(b){
//check if exists in the array
if(String(sess.useragent).toUpperCase().indexOf(b.name.toUpperCase()) > -1){
db.Sessions.update({ _id : sess._id} // find by _id
,{
is_crawler : true // set a update value
},
{
upsert:false // do update only
})
}
});
}
});