I have two table
master_document_table document_table
id | title id | master_document_id | userId
1 | Profile
select * from master_document_table as md
left join document_table as d on md.id = d.master_document_id and d.userId = 2
Result:
id | title | master_document_id | userId
1 | Profile null null
how can this be achieved using mongodb i have tried & also did some research from stack overflow and did not got the expected result.
From mongo v3.2, it is possible to aggregate two or more than two collections.
But here one thing, you need to know is, you won't get the same structure as sql. You will get your data on nested document.
Your tables will look something like this in mongodb:
master_document_table (collection / table)
{
_id: ObjectId("5f28d53c00613e6ada45f702"),
title: "Profile",
}
document_table (collection / table)
{
_id: ObjectId("5f28d53c00613e6ada45e790"),
master_document_id: ObjectId("5f28d53c00613e6ada45f702"), // master_doc_table's ref id.
userId: ObjectId("5f28d53c00613e6ada327f231"), // user's ref id.
}
Your query will be:
db.document_table.aggregate([
{$match: {title: "Profile"}}, // you can give any condition here.
{
$lookup:
{
from: "master_document_table",
localField: "master_document_id",
foreignField: "_id",
as: "master_document_id"
}
}
])
Your query result will look something like:
[{
_id: ObjectId("5f28d53c00613e6ada45e790"),
master_document_id: {
_id: ObjectId("5f28d53c00613e6ada45f702"),
title: "Profile"
},
userId: ObjectId("5f28d53c00613e6ada327f231"),
}]
Now you can restructure it based on your requirement.
I would like to recommend you
https://docs.mongodb.com/manual/reference/operator/aggregation/match/
&
https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/
article from mongodb docs.
Hope this will help to resolve your problem.
Inorder to join two tables you can use the $lookup aggregation pipeline operator in version 3.6 and newer. Since you need the results like LEFT OUTER JOIN, you have to $unwind the array to objects and filter out the desired results using $match conditions (ie document_table.userId = 2). Dont forget to use preserveNullAndEmptyArrays to true when unwinding.
Sample code for reference:
db.master_document_table.aggregate([
{ "$lookup": {
"from": "document_table",
"localField": "id",
"foreignField": "master_document_id",
"as": "document_table"
}},
{ $unwind: { path: "$document_table", preserveNullAndEmptyArrays: true } },
{
$match:{
$or:[
{ "document_table": { $exists: false } },
{"document_table.userId" : 2 }
]
}
}
])
I some how managed to get desire result thanks for the help
db.master_document_table.aggregate([
{
$lookup: {
from: 'document_table',
localField: 'id',
foreignField: 'master_document_id',
as: 'users'
}
},
{
$project: {
title: 1,
users: {
$filter: {
input: '$users',
as: 'user',
cond: {
$eq: ['$$user.userId', 2]
}
}
}
}
}
]);
Related
I have a collection called Post, which has an ID field called makeupProduct. This ID field is a foreign key to my MakeupProduct collection.
In MongoDB Compass, I'm trying to find all Posts
where productUrl === null and
populate the Makeup record along with it
Is this possible?
I have the 1st filter down, but don't know how to write the rest.
db.Post.aggregate([
{
$match: {
productUrl: null
}
},
{
$lookup:
{
from: "MakeupProduct",
localField: "makeupProduct",
foreignField: "_id",
as: "makeupProduct"
}
},
{
$set: {
makeupProduct: {
$arrayElemAt: ["$makeupProduct", 0]
}
}
}
])
I have an aggregate query where I join 3 collections. I'd like to filter the search based on fields from two of those collections. The problem is, I'm only able to use $match on the initial collection that mongoose initialized with.
Here's the query:
var pipeline = [
{
$lookup: {
from: 'blurts',
localField: 'followee',
foreignField: 'author.id',
as: 'followerBlurts'
}
},
{
$unwind: '$followerBlurts'
},
{
$lookup: {
from: 'users',
localField: 'followee',
foreignField: '_id',
as: 'usertbl'
}
},
{
$unwind: '$usertbl'
},
{
$match: {
'follower': { $eq: req.user._id },
//'blurtDate': { $gte: qryDateFrom, $lte: qryDateTo }
}
},
{
$sample: { 'size': 42 }
},
{
$project: {
_id: '$followerBlurts._id',
name: '$usertbl.name',
smImg: '$usertbl.smImg',
text: '$followerBlurts.text',
vote: '$followerBlurts.vote',
blurtDate: '$followerBlurts.blurtDate',
blurtImg: '$followerBlurts.blurtImg'
}
}
];
keystone.list('Follow').model.aggregate(pipeline)
.sort({blurtDate: -1})
.cursor().exec()
.toArray(function(err, data) {
if (!err) {
res.json(data);
} else {
console.log('Error getting following blurts --> ' + err);
}
});
Within the pipeline, I can only use $match on the 'Follow' model. When I use $match on the 'Blurt' model, it simply ignores the condition (you can see where I tried to include it in the commented line under $match).
What's perplexing is that I can utilize this field in the .sort method, but not in the $match conditions.
Any help much appreciated.
You can use the mongo dot notation to access elements of the collection that is being looked up via $lookup.
https://docs.mongodb.com/manual/core/document/#dot-notation
So, in this case followerBlurts.blurtDate should give you the value you are looking for.
I would like unmatched data from the USERS collection. Here I have two collections 1) USERS, 2) COMPANY.I am able to get the matched data from both USERS using aggregate function. but in this case I want data from USERS table which are not assigned to a company.
USERS table
{
_id: "AAA",
fullName:"John Papa"
},
{
_id: "BBB",
fullName:"Robin Son"
}
COMPANY table
{
_id: "1sd1s",
Name:"Lumbar Company"
User:"AAA"
},
{
_id: "23s1dfs3",
Name:"Patricia"
User:"AAA"
}
$lookup works like LEFT OUTER JOIN so it will remove empty array when there's no match. Then you can use $size to get only empty arrays:
db.users.aggregate([
{
$lookup: {
from: "company",
localField: "_id",
foreignField: "User",
as: "companies"
}
},
{
$match: {
$expr: {
$eq: [ { "$size": "$companies" }, 0 ]
}
}
}
])
Mongo Playground
I'm fairly new to MongoDB and need help doing a select, or perhaps some sort of left join, on one collection based on another collection's data.
I have two collections, animals and meals, and I want to get the animal(s) that has had it's last registered meal after a certain date (let's say 20171001) to determine if the animal is still active.
collection animals:
{
name: mr floof,
id: 12345,
lastMeal: abcdef
},
{
name: pluto,
id: 6789,
lastMeal: ghijkl
}
collection meals:
{
id: abcdef,
created: 20171008,
name: carrots
},
{
id: ghijkl,
created: 20170918,
name: lettuce
}
So the expected output of the query in this case would be:
{
name: mr floof,
id: 12345,
lastMeal: abcdef
}
As Mr Floof has had his last meal 20171008, i.e. after 20171001.
Hope I was clear enough, but if not, don't hesitate to ask.
You can try below aggregation query.
db.animals.aggregate([ [
{
"$lookup": {
"from": "meals",
"localField": "lastMeal",
"foreignField": "id",
"as": "last_meal"
}
},
{
"$unwind": "$last_meal"
},
{
"$match": {
"last_meal.created": {
"$gt": 20171001
}
}
}
])
More info here.
You can use $project with exclusion after $match stage to format the response to exclude joined fields. Something like { $project: {"last_meal":0} }
MongoDB supports joins with $lookup , In your case you can use query like:-
db.animals.aggregate([
{
$lookup:
{
from: "meals",
localField: "lastMeal",
foreignField: "id",
as: "last_meal"
}
},
{
$match: {
"created" : {
$gt: "date" //your date format
}
}
}
])
thanks !
Is it possible to do SQL inner joins kind of stuff in MongoDB?
I know there is the $lookup attribute in an aggregation pipeline and it is equivalent to outer joins in SQL, but I want to do something similar to inner joins.
I have three collections which need to merge together:
// User Collection
db.User.find({});
// Output:
{
ID : 1,
USER_NAME : "John",
password : "pass"
}
{
ID : 2,
USER_NAME : "Andrew",
PASSWORD : "andrew"
}
// Role Collection
db.ROLE.find({});
// Output:
{
ID : 1,
ROLE_NAME : "admin"
},
{
ID : 2,
ROLE_NAME : "staff"
}
// USER_ROLE Collection
db.USER_ROLE.find({});
// Output:
{
ID : 1,
USER_ID : 1,
ROLE_ID : 1
}
I have the above collections and I want to extract only the documents matched with users and their respective roles, not all the documents. How can I manage it in MongoDB?
I found answer my self it was
$unwind
done the trick to me following query worked for me
db.USER.aggregate([{
$lookup: {
from: "USER_ROLE",
localField: "ID",
foreignField: "USER_ID",
as: "userRole"
}
}, {
$unwind: {
path: "$userRole",
preserveNullAndEmptyArrays: false
}
}, {
$lookup: {
from: "ROLE",
localField: "userRole.ROLE_ID",
foreignField: "ID",
as: "role"
}
}, {
$unwind: {
path: "$role",
preserveNullAndEmptyArrays: false
}
}, {
$match: {
"role.ROLE_NAME": "staff"
}, {
$project: {
USER_NAME: 1,
_id: 0
}
}
]).pretty()
Anyway thanks for the answers
As Tiramisu wrote this looks like schema issue.
You can make a manual inner join, by removing documents where $lookup returned empty array.
....
{$lookup... as myArray},
{$match: {"myArray":{$ne:[]}}},
{$lookup... as myArray2},
{$match: {"myArray2":{$ne:[]}}},
schema change
I personally will go for schema update, like this:
db.User.find({})
{
ID : 1,
USER_NAME : "John",
password : "pass"
roles:[{ID : 1, ROLE_NAME : "admin"}]
}
db.ROLE.find({})
{
ID : 1,
ROLE_NAME : "admin"
},
Will this help
const RolesSchema = new Schema({
....
});
const Roles = mongoose.model('Roles', RolesSchema);
const UserSchema = new Schema({
...
roles: [{ type: mongoose.Schema.Types.ObjectId, ref: "Roles" }]
});
using the populate on userschema you can also reduce the redundancy
Well you are correct, $lookup attribute is equivalent to outer joins in SQL, but in mongoDB, you need additional aggregation stages so as to perform a similar INNER JOIN in mongo. Here's an example for joining User and ROLE collections based on the ID and displaying the results based on USER_NAME and ROLE_NAME
db.User.aggregate([{
$lookup: {
from: "ROLE",
localField: "ID",
foreignField: "ID",
as: "result"
}
},{
$match: {
result: {
$ne: []
}
}
},{
$addFields: {
result: {
$arrayElemAt: ["$result",0]
}
}
},{
$project: {
USER_NAME: "$USER_NAME",
ROLE_NAME: "$result.ROLE_NAME"
}
}])
Hope it helps!!
MongoDB $lookup aggregation is the most formal and the best-optimized method for this question. However, If you are using Node.js as the server-side, Then you can use a little hack as follows.
CollectionOne.find().then((data0) => {
if (data0.length > 0) {
let array = [];
for (let i = 0; i < data0.length; i++) {
let x = data0[i]
let y = x.yourForeignKey;
array.push({_id: y});
}
CollectionTwo.find(
{$or: array}
).then((data1) => {
res.status(200).json(data1);
}).catch((error1) => {
return error1;
})
}
}).catch((error0) => {
return error0;
});
I used the Array Push() method and the $or operator of MongoDB. You can use the $nor operator instead of the $or to find outer join documents. And also you can change the finding algorithm by using $ne, $nor, $or, $and and etc.
$lookup aggregation
Performs a left outer join to a collection in the same database to filter in documents from the "joined" collection for processing. The
$lookup
stage adds a new array field to each input document. The new array field contains the matching documents from the "joined" collection. The
$lookup
stage passes these reshaped documents to the next stage.
Starting in MongoDB 5.1,
$lookup
works across sharded collections.
To combine elements from two different collections, use the
$unionWith
pipeline stage.
Syntax
The $lookup stage has the following syntaxes:
Equality Match with a Single Join Condition
To perform an equality match between a field from the input documents with a field from the documents of the "joined" collection, the
$lookup
stage has this syntax:
{
$lookup:
{
from: <collection to join>,
localField: <field from the input documents>,
foreignField: <field from the documents of the "from" collection>,
as: <output array field>
}
}
More details: https://www.mongodb.com/docs/manual/reference/operator/aggregation/lookup/
> show dbs
admin 0.000GB
config 0.000GB
local 0.002GB
> use local
switched to db local
> show collections
startup_log
test1
test2
> db.test2.aggregate([{
... $lookup: {
... from: "test1",
... localField: "id",
... foreignField: "id",
... as: "aggTest"
... }
... }])