I'm trying to build an aggregate query from a users collection. Each user has an id property that is associated in a clients collection array:
// users...
[
{
id: '12345',
name: 'John'
}
// ...
}
// clients...
[
{
name: 'Foo',
members: [ { id: '1234', role: 'Admin' } ]
},
// ....
]
So what I'm trying to do is aggregate the users collection and do a $lookup to "join" the clients with which a user is a member (by the id)
db.users.aggregate([
$lookup: {
from: 'clients',
as: 'clients',
let: { user_id: '$id' },
pipeline: [
{
$match: {
members: {
$elemMatch: { id: '$user_id' },
},
},
},
],
};
}])
If I hard-code any user's id into the $elemMatch (replacing $user_id) it works, but I can't seem to get it to work as a variable from the user records.
From $lookup let,
A $match stage requires the use of an $expr operator to access the variables. The $expr operator allows the use of aggregation expressions inside of the $match syntax.
From your scenario,
Need $expr to access the variable.
Apply the $in operator instead of $elemMatch.
To reference the variable in the pipeline, use $$<variable> but not $<variable>.
{
$match: {
$expr: {
$in: [
"$$user_id",
"$members.id"
]
}
}
}
Sample Mongo Playground
Related
I have two collections energyOffers and energyOfferLogs. When a user deactivated their account I'm looking for all the remaining active energyOffers where the entity of the user is in the assignees array, not in the declinedEntities array and the offerValidTill date is less than the current timestamp.
const [energyOffers] = await EnergyOffer.find([{
'assignees.id': entityID,
declinedEntities: {
$ne: leadID
},
offerValidTill: { $gt: Date.now() }
}], { session });
Based on these energyOffers I need to update the corresponding energyOfferLogs. I can find these with { entityID: entityID, 'offer.offerID': offer._id } but how can I look for all these offers in the same query?
If I loop through the energyOffers I will have to perform multiple updates while my guess is that this can be done in one updateMany. I was looking into the $lookup aggregate operator (https://www.mongodb.com/docs/v6.0/reference/operator/aggregation/lookup/) but it seems that the EnergyOffer find query is too complex to perform in this.
await EnergyOfferLog.updateMany({ ??? }, {
$set: {
'offer.action': 'declined',
'offer.action_date': Math.floor(Date.now()),
'offer.action_user': user.first_name,
'offer.action_user_id': userID
}
});
Get all offer ids from the first query, e.g.
let ids = energyOffers.map(o => o._id)
Use $in to match logs for all matching offers:
await EnergyOfferLog.updateMany({ entityID: entityID, 'offer.offerID': {$in: ids} }, {
$set: {
'offer.action': 'declined',
'offer.action_date': Math.floor(Date.now()),
'offer.action_user': user.first_name,
'offer.action_user_id': userID
}
});
If you want to do it with one query only, it is not complex. You can use $lookup with a pipeline for this:
Start with your $match query on the energyOffers collection
Use '$lookupto get the matchingenergyOfferLogs`
Clean the pipeline to contain only the energyOfferLogs docs
Perform the $set
Use $merge to save it back to energyOfferLogs collection
db.energyOffers.aggregate([
{$match: {
"assignees.id": entityID,
declinedEntities: {$ne: leadID},
offerValidTill: {$gt: Date.now()}
}
},
{$lookup: {
from: "energyOfferLogs",
let: {offerId: "$_id"},
pipeline: [
{$match: {
$and: [
{entityID: entityID},
{$expr: {$eq: ["$offer.offerID", "$$offerId"]}}
]
}
}
],
as: "energyOfferLogs"
}
},
{$unwind: "$energyOfferLogs"},
{$replaceRoot: {newRoot: "$energyOfferLogs"}},
{$set: {
"offer.action": "declined",
"offer.action_date": Math.floor(Date.now()),
"offer.action_user": user.first_name,
"offer.action_user_id": userID
}
},
{$merge: {into: "$energyOfferLogs"}}
])
See how it works on the playground example
Answer was updated according to a remark by #Alex_Blex
I have two collections that I want to join with $lookup based on two id fields. Both fields are from type guid and looke like this in mongodb compass: 'Binary('cavTZa/U2kqfHtf08sI+Fg==', 3)'
This syntax in the compass aggregation pipeline builder gives the expected result:
{
from: 'clients',
localField: 'ClientId',
foreignField: '_id',
as: 'ClientData'
}
But i want to add some projection and tried to change it like this:
{
from: 'clients',
'let': {
id: '$_id.clients'
},
pipeline: [
{
$match: {
$expr: {
$eq: [
'$ClientId',
'$$id'
]
}
}
},
{
$project: {
Name: 1,
_id: 0
}
}
],
as: 'ClientData'
}
But the result here is that every client from collection 'clients' is added to every document in the starting table. I have to use MongoDB 3.6 so the new lookup syntax from >=5.0 is not available.
Any ideas for me? Does $eq work for binary stored guid data?
In the first example, you say that the local field is ClientId and the foreign field is _id. But that's not what you used in your second example.
This should work better:
{
from: 'clients',
'let': {
ClientId: '$ClientId'
},
pipeline: [
{
$match: {
$expr: {
$eq: [
'$$ClientId',
'$_id'
]
}
}
},
{
$project: {
Name: 1,
_id: 0
}
}
],
as: 'ClientData'
}
I have no idea about how to build a query which does this:
I have a collection of users, each user has a field userdata which contains an array of String.
Each string is the string of the ObjectID of other documents (news already seen) in another collection.
I need, knowing the username of this user, to perform a query which gets all the news but not those which have been already seen.
I think the $nin operator does what I need but I don't know how to mix it with data from another collection.
Users
user
username: String
userdata: Object
news: Array of String
News
news1
_id: ObjectID
news2
_id: ObjectID
EXAMPLE:
Users: [{
username: 'mario',
userdata: {
news: ['10', '11']
}
}]
News: [{
_id: '10',
content: 'hello world10'
},{
_id: '11',
content: 'hello world11'
},{
_id: '12',
content: 'hello world12'
}]
Passing to the query the username (as a String) 'mario', I need to query the collection News and get back only the one with _id '12'.
Thanks
You need to run $lookup with custom pipeline. There's no $nin for aggregations but you can use $not along with $in. Then you can also try $unwind with $replaceRoot to promote filtered News to the root level:
db.Users.aggregate([
{ $match: { username: "mario" } },
{
$lookup: {
from: "News",
let: { user_news: "$userdata.news" },
pipeline: [{ $match: { $expr: { $not: { $in: [ "$_id", "$$user_news" ] } } } }],
as: "filteredNews"
}
},
{ $unwind: "$filteredNews" },
{ $replaceRoot: { newRoot: "$filteredNews" }}
])
Right now, it's not possible to use the $in operator in the $filter array aggregation operator.
Let's say this is the document schema:
{
_id: 1,
users: [
{
_id: 'a',
accounts: ['x', 'y', 'z']
},
{
_id: 'b',
accounts: ['j','k','l']
}
]
}
I want, using aggregate, to get the documents with filtered array of users based on the contents of the accounts array.
IF the $in would work with the $filter operator, I would expect it to look like this:
db.test.aggregate([
{
$project: {
'filtered_users': {
$filter: {
input: '$users',
as: 'user',
cond: {
$in: ['$$user.accounts', ['x']]
}
}
}
}
}
])
and return in the filtered_users only the first user since x is in his account.
But, as I said, this doesn't work and I get the error:
"invalid operator '$in'"
because it isn't supported in the $filter operator.
Now I know I can do it with $unwind and using regular $match operators, but then it will be much longer (and uglier) aggregation with the need of using $group to set the results back as an array - I don't want this
My question is, if there is some other way to manipulate the $filter operator to get my desired results.
Since $in is not supported in aggregate operation for array, the alternative would be for you to use $setIsSubset. For more information on this you can refer this link. The aggregate query would now look like
db.test.aggregate([
{
$project: {
'filtered_users': {
$filter: {
input: '$users',
as: 'user',
cond: {
$setIsSubset: [['x'], '$$user.accounts']
}
}
}
}
}])
This query will return only elements which have [x] as a subset of the array in user.accounts.
Starting From MongoDB 3.4, you can use the $in aggregation operator in the $project stage
db.collection.aggregate([
{
"$project": {
"filtered_users": {
"$filter": {
"input": "$users",
"as": "user",
"cond": { "$in": [ "x", "$$user.accounts" ] }
}
}
}
}
])
I am trying new pipeline query of MongoDB so i try to execute below query.
{
aggregate: 'Posts',
pipeline: [
{ $unwind: '$Comments'},
{ $match: {'$Comments.Owner': 'Harry' }},
{$group: {
'_id': '$Comments._id'
}
}
]
}
And nothing match to query so empty result returns. I guess problem can be on $match command . I am using dotted notation match comment Owner but not sure it is exactly true or not. Why this query does not return Ownders who is 'Harry' . I am sure it is exist in db.
You don't use the $ prefix for the $match field names.
Try this:
{
aggregate: 'Posts',
pipeline: [
{ $unwind: '$Comments'},
{ $match: {'Comments.Owner': 'Harry' }},
{ $group: {
'_id': '$Comments._id'
}}
]
}
I encounter the same problem with aggregation framework with MongoDB 2.2.
$match didn't work for me for subdocument (but I am just learning MongoDB, so I could do something wrong).
I added extra projection to remove subdocument (Comments in this case):
{
aggregate: 'Posts',
pipeline: [
{ $unwind: '$Comments'},
{ $project: {
comment_id: "$Comments._id",
comment_owner: "$Comments.Owner"
}},
{ $match: {'$comment_Owner': 'Harry' }},
{$group: {
'_id': '$comment_id'
}
}
]
}