I have a situation were i got the following database-structure
Collection "User":
[
{ _id: ObjectId("507f1f77bcf86c0000000001"), name: "Mike", status: "ACTIVE", verified: true },
{ _id: ObjectId("507f1f77bcf86c0000000002"), name: "Ben", status: "INACTIVE", verified: true },
{ _id: ObjectId("507f1f77bcf86c0000000003"), name: "Anastasia", status: "ACTIVE", verified: true }
]
Collection "Reports"
[
{ userRef: ObjectId("507f1f77bcf86c0000000001"), reportVerified: true },
{ userRef: ObjectId("507f1f77bcf86c0000000003"), reportVerified: false },
]
As you can see I have a collection with all of my users and a different collection called "Report" were entries references to a user and have a separated flag-field called "reportVerified". Now I want to find all entries from the "User"-collection which have specific properties in the "User"-collection but are also references with a specific property in the "Report"-collection.
Example: I want to find all users which have User-Collection.status "ACTIVE" and have a reference in the "Report"-Table with "reportVerified" set true. This should match only "Mike" in my case.
Having the properties of the "Report"-collection in the "User"-collection directly is not an option for me.
The situation would be quite easy if i only got find-criterias either in the "User"-collection (simple find) or in the "Report"-collection (using populate) but I need a combination of both.
The best way would be using aggregate. First you need to use lookup for adding user object to the report object.
for example
mongoose.db(dbName).collection(cName).aggregate([
{
$match :{} // your match condition for report
},
{
$lookup:
{
from: "user-collection-name",
let: { user_id: "$_id", user_conditon: "$status" },
pipeline: [
{ $match:
{ $expr:
{ $and:
[
{ $eq: [ "$userRef", "$$user_id" ] }, // for joining collections
{ $eq: [ conditionInput, "$$status" ] }, // for querying on user collection
]
}
}
}
],
as: "user"
}
}
])
Related
I have a collection in the following format:
[
{
"postId": ObjectId("62dffd0acb17483cf015375f"),
"userId": ObjectId("62dff9584f5b702d61c81c3c"),
"state": [
{
"id": ObjectId("62dffc49cb17483cf0153220"),
"notes": "these are my custom notes!",
"lvl": 3,
},
{
"id": ObjectId("62dffc49cb17483cf0153221"),
"notes": "hello again",
"lvl": 0,
},
]
},
]
My goal is to be able to update and add an element in this array in the following situation:
If the ID of the new element is not in the state array, push the new element in the array
If the ID of the new element is in the state array and its lvl field is 0, update that element with the new information
If the ID of the new element exists in the array, and its lvl field is not 0, then nothing should happen. I will throw an error by seeing that no documents were matched.
Basically, to accomplish this I was thinking about using findOneAndUpdate with upsert, but I am not sure how to tell the query to update the state if lvl is 0 or don't do anything if it is bigger than 0 when the match is found.
For solving (1) this is what I was able to come up with:
db.collection.findOneAndUpdate(
{
"postId": ObjectId("62dffd0acb17483cf015375f"),
"userId": ObjectId("62dff9584f5b702d61c81c3c"),
"state.id": {
"$ne": ObjectId("62dffc49cb17483cf0153222"),
},
},
{
"$push": {"state": {"id": ObjectId("62dffc49cb17483cf0153222"), "lvl": 1}}
},
{
"new": true,
"upsert": true,
}
)
What is the correct way to approach this issue? Should I just split the query into multiple ones?
Edit: as of now I have done this in more than one query (one to fetch the document, then I iterate over its state array to check if the ID exists in it, and then I perform (1), (2) and (3) in a normal if-else clause)
If the ID of the new element exists in the array, and its lvl field is not 0, then nothing should happen. I will throw an error by seeing that no documents where matched.
First thing FYI,
upsert is not possible in the nested array
upsert will not add new elements to the array
upsert can add a new document with the new element
if you want to throw an error if the record does not present then you don't need upsert
Second thing, you can achieve this in one query by using an update with aggregation pipeline in MongoDB 4.2,
Note: Here i must inform you, this query will respond updated document but there will be no flag or any clue if this query fulfilled your first situation or second situation, or the third situation out of 3, you have to check in your client-side code through query response.
check conditions for postId and userId fields only
we are going to update state field under $set stage
check the condition if the provided id is present in state's id?
true, $map to iterate loop of state array
check conditions for id and lvl: 0?
true, $mergeObjects to merge current object with the new information
false, it will not do anything
false, then add that new element in state array, by $concatArrays operator
db.collection.findOneAndUpdate(
{
postId: ObjectId("62dffd0acb17483cf015375f"),
userId: ObjectId("62dff9584f5b702d61c81c3c")
},
[{
$set: {
state: {
$cond: [
{ $in: [ObjectId("62dffc49cb17483cf0153221"), "$state.id"] },
{
$map: {
input: "$state",
in: {
$cond: [
{
$and: [
{ $eq: ["$$this.id", ObjectId("62dffc49cb17483cf0153221")] },
{ $eq: ["$$this.lvl", 0] }
]
},
{
$mergeObjects: [
"$$this",
{
// update your new fields here
"notes": "new note"
}
]
},
"$$this"
]
}
}
},
{
$concatArrays: [
"$state",
[
// add new element
{
"id": ObjectId("62dffc49cb17483cf0153221"),
"lvl": 1
}
]
]
}
]
}
}
}],
{ returnNewDocument: true }
)
Playrgound
Third thing, you can execute 2 update queries,
The first query, for the case: element does not present and it will push a new element in state
let response = db.collection.findOneAndUpdate({
postId: ObjectId("62dffd0acb17483cf015375f"),
userId: ObjectId("62dff9584f5b702d61c81c3c"),
"state.id": { $ne: ObjectId("62dffc49cb17483cf0153221") }
},
{
$push: {
state: {
id: ObjectId("62dffc49cb17483cf0153221"),
lvl: 1
}
}
},
{
returnNewDocument: true
})
The second query on the base of if the response of the above query is null then this query will execute,
This will check state id and lvl: 0 conditions if conditions are fulfilled then execute the update fields operation, it will return null if the document is not found
You can throw if this will return null otherwise do stuff with response data and response success
if (response == null) {
response = db.collection.findOneAndUpdate({
postId: ObjectId("62dffd0acb17483cf015375f"),
userId: ObjectId("62dff9584f5b702d61c81c3c"),
state: {
$elemMatch: {
id: ObjectId("62dffc49cb17483cf0153221"),
lvl: 0
}
}
},
{
$set: {
// add your update fields
"state.$.notes": "new note"
}
},
{
returnNewDocument: true
});
// not found and throw an error
if (response == null) {
return {
// throw error;
};
}
}
// do stuff with "response" data and return result
return {
// success;
};
Note: As per the above options, I would recommend you that I explained in the Third thing that you can execute 2 update queries.
What you're trying became possible with the introduction pipelined updates, here is how I would do it by using $concatArrays to concat the exists state array with the new input and $ifNull in case of an upsert to init the empty value, like so:
const inputObj = {
"id": ObjectId("62dffc49cb17483cf0153222"),
"lvl": 1
};
db.collection.findOneAndUpdate({
"postId": ObjectId("62dffd0acb17483cf015375f"),
"userId": ObjectId("62dff9584f5b702d61c81c3c")
},
[
{
$set: {
state: {
$ifNull: [
"$state",
[]
]
},
}
},
{
$set: {
state: {
$concatArrays: [
{
$map: {
input: "$state",
in: {
$mergeObjects: [
{
$cond: [
{
$and: [
{
$in: [
inputObj.id,
"$state.id"
]
},
{
$eq: [
inputObj.lvl,
0
]
}
]
},
inputObj,
{},
]
},
"$$this"
]
}
}
},
{
$cond: [
{
$not: {
$in: [
inputObj.id,
"$state.id"
]
}
},
[
],
[]
]
}
]
}
}
}
],
{
"new": true,
"upsert": true
})
Mongo Playground
Prior to version 4.2 and the introduction of this feature what you're trying to do was not possible using the naive update syntax, If you are using an older version then you'd have to split this into 2 separate calls, first a findOne to see if the document exists, and only then an update based on that. obviously this can cause stability issue's if you have high update volume.
I recently updated my subschemas (called Courses) to have timestamps and am trying to backfill existing documents to include createdAt/updatedAt fields.
Courses are stored in an array called courses in the user document.
// User document example
{
name: "Joe John",
age: 20,
courses: [
{
_id: <id here>,
name: "Intro to Geography",
units: 4
} // Trying to add timestamps to each course
]
}
I would also like to derive the createdAt field from the Course's Mongo ID.
This is the code I'm using to attempt adding the timestamps to the subdocuments:
db.collection('user').updateMany(
{
'courses.0': { $exists: true },
},
{
$set: {
'courses.$[elem].createdAt': { $toDate: 'courses.$[elem]._id' },
},
},
{ arrayFilters: [{ 'elem.createdAt': { $exists: false } }] }
);
However, after running the code, no fields are added to the Course subdocuments.
I'm using mongo ^4.1.1 and mongoose ^6.0.6.
Any help would be appreciated!
Using aggregation operators and referencing the value of another field in an update statement requires using the pipeline form of update, which is not available until MongoDB 4.2.
Once you upgrade, you could use an update like this:
db.collection.updateMany({
"courses": {$elemMatch: {
_id:{$exists:true},
createdAt: {$exists: false}
}}
},
[{$set: {
"courses": {
$map: {
input: "$courses",
in: {
$mergeObjects: [
{createdAt: {
$convert: {
input: "$$this._id",
to: "date",
onError: {"error": "$$this._id"}
}
}},
"$$this"
]
}
}
}
}
}
])
this is my schema:
new Schema({
code: { type: String },
toy_array: [
{
date:{
type:Date(),
default: new Date()
}
toy:{ type:String }
]
}
this is my db:
{
"code": "Toystore A",
"toy_array": [
{
_id:"xxxxx", // automatic
"toy": "buzz"
},
{
_id:"xxxxx", // automatic
"toy": "pope"
}
]
},
{
"code": "Toystore B",
"toy_array": [
{
_id:"xxxxx", // automatic
"toy": "jessie"
}
]
}
I am trying to update an object. In this case I want to update the document with code: 'ToystoreA' and add an array of subdocuments to the array named toy_array if the toys does not exists in the array.
for example if I try to do this:
db.mydb.findOneAndUpdate({
code: 'ToystoreA,
/*toy_array: {
$not: {
$elemMatch: {
toy: [{"toy":'woddy'},{"toy":"buzz"}],
},
},
},*/
},
{
$addToSet: {
toy_array: {
$each: [{"toy":'woddy'},{"toy":"buzz"}],
},
},
},
{
new: false,
}
})
they are added and is what I want to avoid.
how can I do it?
[
{
"code": "Toystore A",
"toy_array": [
{
"toy": "buzz"
},
{
"toy": "pope"
}
]
},
{
"code": "Toystore B",
"toy_array": [
{
"toy": "jessie"
}
]
}
]
In this example [{"toy":'woddy'},{"toy":"buzz"}] it should only be added 'woddy' because 'buzz' is already in the array.
Note:when I insert a new toy an insertion date is also inserted, in addition to an _id (it is normal for me).
As you're using $addToSet on an object it's failing for your use case for a reason :
Let's say if your document look like this :
{
_id: 123, // automatically generated
"toy": "buzz"
},
{
_id: 456, // automatically generated
"toy": "pope"
}
and input is :
[{_id: 789, "toy":'woddy'},{_id: 098, "toy":"buzz"}]
Here while comparing two objects {_id: 098, "toy":"buzz"} & {_id: 123, "toy":"buzz"} - $addToSet consider these are different and you can't use $addToSet on a field (toy) in an object. So try below query on MongoDB version >= 4.2.
Query :
db.collection.updateOne({"_id" : "Toystore A"},[{
$addFields: {
toy_array: {
$reduce: {
input: inputArrayOfObjects,
initialValue: "$toy_array", // taking existing `toy_array` as initial value
in: {
$cond: [
{ $in: [ "$$this.toy", "$toy_array.toy" ] }, // check if each new toy exists in existing arrays of toys
"$$value", // If yes, just return accumulator array
{ $concatArrays: [ [ "$$this" ], "$$value" ] } // If No, push new toy object into accumulator
]
}
}
}
}
}])
Test : aggregation pipeline test url : mongoplayground
Ref : $reduce
Note :
You don't need to mention { new: false } as .findOneAndUpdate() return old doc by default, if you need new one then you've to do { new: true }. Also if anyone can get rid of _id's from schema of array objects then you can just use $addToSet as OP was doing earlier (Assume if _id is only unique field), check this stop-mongoose-from-creating-id-property-for-sub-document-array-items.
Having issues with aggregate and lookup in multiple stages. The issue is that I cannot match by userId In the last lookup. If I omit the { $eq: ['$userId', '$$userId'] } it works and match by the other criteria. But not by the userid.
I've tried added pools as a let and use it as { $eq: ['$userId', '$$pools.userId'] } in the last stage but that doesn't work either. I get an empty coupon array.
I get this with the below query. I think I need to use $unwind in some way? But haven't got that to work yet. Any pointers?
There is three collections total to be joined. First the userModel, it should contain pools and then the pools should contain a users coupons.
{
"userId": "5df344a1372f345308dac12a", // Match this usedId with below userId coming from the coupon
"pools": [
{
"_id": "5e1ebbc6cffd4b042fc081ab",
"eventId": "id999",
"eventStartTime": "some date",
"trackName": "tracky",
"type": "foo bar",
"coupon": []
}
]
},
I need the coupon array to be filled with the correct data (below) which has a matching userId in it.
"coupon": [
{
"eventId": "id999",
"userId": "5df344a1372f345308dac12a", // This userId need to match the above one
"checked": true,
"pool": "a pool",
}
poolProject:
const poolProject = {
eventId: 1,
eventStartTime: 1,
trackName: 1,
type: 1,
};
Userproject:
const userProjection = {
_id: {
$toString: '$_id',
},
paper: 1,
correctBetsLastWeek: 1,
correctBetsTotal: 1,
totalScore: 1,
role: 1,
};
The aggregate query
const result = await userModel.aggregate([
{ $project: userProjection },
{
$match: {
$or: [{ role: 'User' },
{ role: 'SuperUser' }],
},
},
{ $addFields: { userId: { $toString: '$_id' } } },
{
$lookup: {
from: 'pools',
as: 'pools',
let: { eventId: '$eventId' },
pipeline: [
{ $project: poolProject },
{
$match: {
$expr: {
$in: ['$eventId', eventIds],
},
},
},
{
$lookup: {
from: 'coupons',
as: 'coupon',
let: { innerUserId: '$$userId' },
pipeline: [
{
$match: {
$expr: {
$eq: ['$userId', '$$innerUserId'],
},
},
},
],
},
},
],
},
},
]);
Thanks for any input!
Edit:
If i move the second lookup (coupon) so they are in the same "level" it works but i would like to have it inside of the pool. If I add as: 'pools.coupon', in the last lookup it overwrites the lookedup pool data.
When you access fields with the $$ prefix it means they are defined as "special" system variables by Mongo.
We don't know exactly how Mongo the magic happens but you're naming two variables with the same name, which causes a conflict as it seems.
So either remove userId: '$userId' from the first lookup as you're not even using it.
Or rename or second userId: '$userId' a different name like innerUserId: '$userId' to avoid conflicts when you access it.
Just dont forget to change { $eq: ['$userId', '$$userId'] } to { $eq: ['$userId', '$$innerUserId'] } after.
EDIT:
Now that its clear theres no field userId in pools collection just change the variable in the second lookup collection from:
let: { innerUserId: '$userId' } //userId does not exist in pools.
To:
let: { innerUserId: '$$userId' }
I have a mongoose Schema that looks likes this :
var AnswerSchema = new Schema({
author: {type: Schema.Types.ObjectId, ref: 'User'},
likes: [{type: Schema.Types.ObjectId, ref: 'User'}]
text: String,
....
});
and I have an API endpoint that allow to get answers posted by a specific user (which exclude the likes array). What I want to do is add a field (with "true/false" value for example) to the answer(s) returned by the mongoose query, when a specific user_id is (or is not) in the likes array of an answer. This way, I can display to the user requesting the answers if he already liked an answer or not.
How could I achieve this in an optimised way ? I would like to avoid fetching the likes array, then look into it myself in my Javascript code to check if specified userId is present in it, then remove it before sending it back to the client... because it sounds wrong to fetch all this data from mongoDB to my node app for nothing. I'm sure there is a better way by using aggregation but I never used it and am a bit confused on how to do it right.
The database might grow very large so it must be quick and optimised.
One approach you could take is via the aggregation framework which allows you to add/modify fields via the $project pipeline, applying a host of logical operators that work in cohort to achieve the desired end result. For instance, in your above case this would translate to:
Answer.aggregate()
.project({
"author": 1,
"matched": {
"$eq": [
{
"$size": {
"$ifNull": [
{ "$setIntersection": [ "$likes", [userId] ] },
[]
]
}
},
1
]
}
})
.exec(function (err, docs){
console.log(docs);
})
As an example to test in mongo shell, let's insert some few test documents to the test collection:
db.test.insert([
{
"likes": [1, 2, 3]
},
{
"likes": [3, 2]
},
{
"likes": null
},
{
"another": "foo"
}
])
Running the above aggregation pipeline on the test collection to get the boolean field for userId = 2:
var userId = 2;
db.test.aggregate([
{
"$project": {
"matched": {
"$eq": [
{
"$size": {
"$ifNull": [
{ "$setIntersection": [ "$likes", [userId] ] },
[]
]
}
},
1
]
}
}
}
])
gives the following output:
{
"result" : [
{
"_id" : ObjectId("564f487c7d3c273d063cd21e"),
"matched" : true
},
{
"_id" : ObjectId("564f487c7d3c273d063cd21f"),
"matched" : true
},
{
"_id" : ObjectId("564f487c7d3c273d063cd220"),
"matched" : false
},
{
"_id" : ObjectId("564f487c7d3c273d063cd221"),
"matched" : false
}
],
"ok" : 1
}