I have a collection (collectionA) that stores an event ID in an event array. The event array information comes from (collectionB).
Lately when an event is deleted from CollectionB via the web app, it sometimes does not get removed from Collection A as it should.
Is there a query i can do in mongo 3.0 to check to see what event_id's exist in CollectionA that are not in collectionB. Those will be the ones that need to be removed while the development team resolves the issue?
Here is a sample query that will give you list of such objects, assuming, collectionA has events array with IDs from collectionB
db.collectionA.aggregate([
{$unwind: '$events'},
{$lookup: {
from: 'collectionB',
localField: 'events',
foreignField: '_id',
as: 'event'
}},
{$unwind: {path: '$event', preserveNullAndEmptyArrays:true}},
{$match:{ 'event': {$exists:false}}},
])
Related
I am having 2 collections -
Collection name is content
Collection name is surveys
I actually want to add the "type" information from the content collection to every content in every survey in the "surveys" collection.
I know in SQL we have to join these 2 tables first on _id and content_id commun columns and then add type column to suryeys table but not sure how should i do it here. I want the type value from content collection to add in every content field inside surveys collection. Please help me.
One option is:
Using $lookup to get the data from content collection
Merge the arrays using $mergeObjects and $indexOfArray. After the lookup, the content data is on someField array. This step finds the releant content from someField and put it inside the matching survey's content item, under the key type. This step also removes someField.
Format the result using $map. This step uses $map to iterate over the content array and format the data under type to contain only the wanted part.
Save it back using $merge. This step saves the result back into the collection.
db.surey.aggregate([
{$lookup: {
from: "content",
localField: "content.content_id",
foreignField: "_id",
as: "someField"
}},
{$set: {
someField: "$$REMOVE",
content: {$map: {
input: "$content",
in: {$mergeObjects: [
"$$this",
{type: {
$arrayElemAt: [
"$someField",
{$indexOfArray: ["$someField._id", "$$this.content_id"]}
]
}}
]}
}}
}},
{$set: {content: {$map: {
input: "$content",
in: {$mergeObjects: ["$$this", {type: "$$this.type.type"}]}
}}}},
{$merge: {into: "surey"}}
])
See how it works on the playground example
In mongoose (I assume you could be using mongoose, sind you added the tag) you can specify the relation in Schema definition and just just populate to add these fields to the output.
If you are not using mongoose you can use $lookup (https://www.mongodb.com/docs/manual/reference/operator/aggregation/lookup/) in your aggregation pipeline.
{
from: "content",
localField: "content._id",
foreignField: "_id",
as: "someField"
}
How I can make a lookup without to overwrite the existing data.
I recreated the situation in mopngodb playground: mongo db playground example
The problem is that I need to lookup the events and all the object ids (subevents, tags).
But the problem starts on the first lookup to the subevents. I need to have all the lookup data on the same place like the id for the lookup. But the rest from the data from the event is gone only the subevents are there.
Any ideas?
as you are giving lookup output name same as existing field it overwrites existing data, if you give new name it persists all data.
for example below aggregation gives the main events and sub events details, if required we can use project stages to put subevents under main event:
[{$match: {
type: "EVENT"
}}, {$lookup: {
from: "events",
localField: "markedItemID",
foreignField: "_id",
as: "marked_event"
}}, {$unwind: {
path: "$marked_event"
}}, {$lookup: {
from: "events",
localField: "marked_event.baseData.subEvents",
foreignField: "_id",
as: "marked_subEvents"
}}]
https://mongoplayground.net/p/dlxqiK1PKdH
Iam working on the Go project and I am using mongodb to store my data. But suddenly the mongodb query execution took too much time to get data.
I have a collection named "cars" with around 25000 documents and each document containing around 200 fields (4.385KB). I have an aggregate query like this:
db.cars.aggregate([
{
$lookup:
{
from: "users",
localField: "uid",
foreignField: "_id",
as: "customer_info"
}
},{
$unwind: "$customer_info"
},{
$lookup:
{
from: "user_addresses",
localField: "uid",
foreignField: "_id",
as: "address"
}
},{
$unwind: "$address"
},{
$lookup:
{
from: "models",
localField: "_id",
foreignField: "car_id",
as: "model_info"
}
},{
$match:{
purchased_on:{$gt:1538392491},
status:{$in:[1,2,3,4]},
"customer_info.status":{$ne:9},
"model_info.status":{$ne:9},
}
},{
$sort:{
arrival_time:1
}
},{
$skip:0
},{
$limit:5
}
])
My document structure is like: https://drive.google.com/file/d/1hM-lPwvE45_213rQDYaYuYYbt3LRTgF0/view.
Now, If run this query with out indexing then it take around 10 mins to load the data. Can anyone suggest me how can I reduce its execution time ?
There are many things to do to optimize your query. What I would try :
As Anthony Winzlet said in comments, use as possible $match stage as first stage. This way, you can reduce number of documents passed to the following stages, and use indexes.
Assuming you use at least 3.6 mongo version, change your lookup stages using the 'let/pipeline' syntax (see here). This way, you can integrate your 'external filters' ( "customer_info.status":{$ne:9}, "model_info.status":{$ne:9} ) in a $match stage in your lookups pipeline. With indexes on right fields / collections, you will gain some time / memory in your $lookup stages.
Do your unwind stages as late as possible, to restrict number of documents passed to the following stages.
It's important to understand how works aggregation pipeline : each stage receive data, do its stuff, and pass data to next stage. So the less data is passed to the pipeline, the faster will be your query.
I have two collection conversation and usercollection
MongoDB format of
conversation looks
[
{participants:["ram","shyam"], recentchat:"hello shyam"},
{participants:["ram","hari"], recentchat:"namaste"},
{participants:["jhon","raju"], recentchat:"what's up"}
]
Likewise usercollection looks
[
{"name" : "shyam", "address":"Kathmandu"},
{"name" : "hari", "address":"Dolakha"},
{"name:"jhon", "address":"Pokhara"},
]
The assumption is:
Ram is the user and it needs to access all the conversation list
I want to use aggregate method of mongodb for joining two collection and get the result in following format.
[
{friend:{name:"shyam",address:"Kathmandu"}, recentchat:"hello shyam"},
{friend:{name:"hari", address:"Dolakha"},recentchat:"namaste"}
]
If you are using MongoDB 3.2+ then you can use $lookup aggregator for that.
Example:
db.getCollection('conversation').aggregate([
{$match: {"participants":"ram"}},
{$lookup: {
from: "usercollection",
localField: "participants",
foreignField: "name",
as: "friend"
}},
{$project: {"friend":1, "recentchat": 1, "_id":0}}
]);
I use:
$match - for select the parent document.
$lookup - for data collecting from usercollection collection
$project - for definig which fields must be included into document
Get the recent chat for particular user from conversation collection
var recentChat = db.conversation.find({participants:"shyam"}).sort({$natural:-1}).recentchat
Update recent chat for that user in usercollection
db.usercollection.update({name:"shyam"}, {$set:{ recentchat: recentChat }})
I have two collections, one that has _id and UserId, and another that has UserId (same unique identifier) and "other data".
I want to filter the latter collection based on a list of _ids from the former collection.
Can someone provide an example query for this scenario?
The only way to 'join' collections in MongoDB is a $lookup aggregation stage (available in version 3.2).
firstCollection.aggregate([
{ $match: { _id: {$in: [1,2,3] }}}, // filter by _ids
{
$lookup:
{
from: "secondCollection",
localField: "UserId",
foreignField: "UserId",
as: "data"
}
}
])
That will add 'data' field to the documents from the first collection which will contain all related documents from second collection. If relation is not 1:1, you can add $unwind stage to flatten results:
{$unwind: "$data"}