I'm looking for the solution since yesterday, but can't find it. I tried different approaches like $elemMatch, aggregation etc. but nothing works. Either I get all objects or just the first one, but never only that ones with the matching IDs.
What I try to achieve: Get all objects of an array with one of multiple IDs.
First parameter is the user ID like 60a3f7d0f988f1e57212a81e. Every Document in this collection is unique by the user ID.
Second parameter is an array like ['609d1cfe0806daba1b0502bf', '609d1bba887035b9cd4292aa'] that consists of different IDs to be matched with the objects of the array words.
My approach gets only the first object and doesn't even work with an array of IDs (wordIDs) as parameter.
Words.prototype.getSpecificPersonalWords = async function(userID, wordIDs) {
const words = await personalWords.aggregate([
{$match:
{"user_id": userID}
},
{$unwind:
{"path": "$words"}
},
{$match:
{"words.word_id": { $in: ['609d1cfe0806daba1b0502bf', '609d1bba887035b9cd4292aa'] } }
}
])
return words
}
Document, that I want to query:
{
"_id": {
"$oid": "60a3f7d0f988f1e57212a81e"
},
"user_id": "609d22188ea8aebac56f9dc3",
"__v": {
"$numberInt": "0"
},
"words": [{
"word_id": "609d1bba887035b9cd4292aa",
"last_practiced": "2021-05-15 16:00:00",
"strength": {
"$numberInt": "1"
}
}, {
"word_id": "609d1c35861d99b9effc0027",
"last_practiced": "2021-05-15 16:00:00",
"strength": {
"$numberInt": "1"
}
}, {
"word_id": "609d1dcbc99e8dba538208d7",
"last_practiced": "2021-05-15 16:00:00",
"strength": {
"$numberInt": "1"
}
}]
}
Schema of that document
{
user_id: {
type: String,
unique: true
},
words: Array
});
I hope someone can help me to figure this out. Thanks in advance.
$match both user_id and word_id conditions
$filter to iterate loop of words and match condition id word_id in array of ids
const words = await personalWords.aggregate([
{
$match: {
"user_id": "609d22188ea8aebac56f9dc3",
"words.word_id": {
$in: ["609d1cfe0806daba1b0502bf", "609d1bba887035b9cd4292aa"]
}
}
},
{
$addFields: {
words: {
$filter: {
input: "$words",
cond: {
$in: [
"$$this.word_id",
["609d1cfe0806daba1b0502bf", "609d1bba887035b9cd4292aa"]
]
}
}
}
}
}
])
Playground
In this code i am trying to match the document based on product size from mongoose query.
I tried this query but it doesn't works. Can anyone tell me what's wrong in this code?
Query i have passed:
{ $match: { "product_items.product_size": { value: 22, unit: "ml" } } }
** Structure:**
[
{
"product_name": "sample product",
"product_items": [
{
"product_item_number": "796363011028",
"product_size": {
"value": 22,
"unit": "ml"
}
}
]
}
]
It doesn't work this way because product_items.product_size evaluates to an array of objects and you are trying to compare a single object with such array. It is more reliable to use $elemMatch when working with arrays of objects:
db.collection.aggregate([
{
$match: {
"product_items": {
$elemMatch: {
"product_size.value": 22,
"product_size.unit": "ml"
}
}
}
}
])
Mongo Playground
How can I change & display particular fields in an array in a mongoDB document? I have a field like this:
"mainField_e": {
"routesArr_e": [
{
"eField_a": {
"field_a1": 8197,
"field_a2": "a string"
},
"field_b": "b string"
}, ..
]
}
..and I want this:
"routesArr_e": [
{
"field_a1": 8197,
"field_a2": "a string",
"field_b": "b string"
}, ..
]
I tried the following command:
db.coll.aggregate([
{
$addFields: {
"routesArr_e.field_a1": "$mainField_e.routesArr_e.eField_a.field_a1",
"routesArr_e.field_a2": "$mainField_e.routesArr_e.eField_a.field_a2",
"routesArr_e.field_b": "$mainField_e.routesArr_e.field_b"
}
}, {
$project: {
"routesArr_e": 1
}
}
]).pretty();
However this puts individual fields into outputs:
"routesArr_e": {
"field_a1": [8197, 6873, ..],
"field_a2": ["a string", "different string", ..],
"field_b": ["b string", "another string", ..]
}
..rather than keeping routesArr_e as an array with fields
I really need some guidance because I've tried a number of combinations of $addFields and $project but cannot get this right. Many thanks
You can use $map to get to your desired output.
db.coll.aggregate([{
$project: {
_id: false, // if you want to remove _id
routesArr_e: {
$map: {
input: "$mainField_e.routesArr_e",
in: {
field_a1: "$$this.eField_a.field_a1",
field_a2: "$$this.eField_a.field_a2",
field_b: "$$this.field_b"
}
}
}
}
}])
Mongo Playground
So I have a questionnaire model:
const schema = new mongoose.Schema({
title: String,
category: String,
description: String,
requirementOption: String,
creationDate: String,
questions: [],
answers: []
})
As you can see the answers is an array. This array contains object that have this structure
{
"participantEmail": "someEmail#email.email"
"currentIndex": 14,
...
}
Now I want to get a specific questionnaire by id, but in answers array I only want specific participant email. So the answers array should have either one element or no element. But I don't want to get null result if there is no such email in the answers array.
I figure it out how to get that specific element from array with this query:
dbModel.findOne({_id: id, 'answers': {$elemMatch: {participantEmail: "someEmail#email.com"}}}, {'answers.$': 1}).exec();
And if that email exists in the answer array I will get this:
"data": {
"questionnaireForParticipant": {
"id": "5d9ca298cba039001b916c55",
"title": null,
"category": null,
"creationDate": null,
"description": null,
"questions": null,
"answers": [
{
"participantEmail": "someEmail#email.com",
....
}
}
}
But if that email is not in the answers array I will get only null. Also I would like to get the title and category and all of the other fields. But I can't seem to find a way to do this.
Since you've this condition 'answers': {$elemMatch: {participantEmail: "someEmail#email.com"}} in filter part of .findOne() - If for given _id document there are no elements in answers. participantEmail array match with input value "someEmail#email.com" then .findOne() will return null as output. So if you wanted to return document irrespective of a matching element exists in answers array or not then try below query :
db.collection.aggregate([
{
$match: { "_id": ObjectId("5a934e000102030405000000") }
},
/** addFields will re-create an existing field or will create new field if there is no field with same name */
{
$addFields: {
answers: {
$filter: { // filter will result in either [] or array with matching elements
input: "$answers",
cond: { $eq: [ "$$this.participantEmail", "someEmail#email.com" ] }
}
}
}
}
])
Test : mongoplayground
Ref : aggregation-pipeline
Note : We've used aggregation as you wanted to return either answers array with matched element or an empty array. Also you can use $project instead of $addFields to transform the output as you wanted to.
The accepted answer is correct, but if you are using mongoose like I do this is how you have to write the accepted answer query:
dbModel.aggregate([
{
$match: { "_id": mongoose.Types.ObjectId("5a934e000102030405000000") }
}]).addFields({
answers: {
$filter: {
input: "$answers",
cond: { $eq: [ "$$this.participantEmail", "someEmail#email.com" ] }
}
}
}).exec();
With this sample input document:
{
_id: 1,
title: "t-1",
category: "cat-abc",
creationDate: ISODate("2020-05-05T07:01:09.853Z"),
questions: [ ],
answers: [
{ participantEmail: "someEmail#email.email", currentIndex: 14 }
]
}
And, with this query:
EMAIL_TO_MATCH = "someEmail#email.email"
db.questionnaire.findOne(
{ _id: 1 },
{ title: 1, category: 1, answers: { $elemMatch: { participantEmail: EMAIL_TO_MATCH } } }
)
The query returns (when the answers.participantEmail matches):
{
"_id" : 1,
"title" : "t-1",
"category" : "cat-abc",
"answers" : [
{
"participantEmail" : "someEmail#email.email",
"currentIndex" : 12
}
]
}
And, when the answers.participantEmail doesn't match or if the amswers array is empty, the result is:
{ "_id" : 1, "title" : "t-1", "category" : "cat-abc" }
NOTE: The $elemMatch used in the above query is a projection operator.
Is it possible to reference the root document during an update operation such that a document like this:
{"name":"foo","value":1}
can be updated with new values and have the full (previous) document pushed into a new field (creating an update history):
{"name":"bar","value":2,"previous":[{"name:"foo","value":1}]}
And so on..
{"name":"baz","value":3,"previous":[{"name:"foo","value":1},{"name:"bar","value":2}]}
I figure I'll have to use the new aggregate set operator in Mongo 4.2, but how can I achieve this?
Ideally I don't want to have to reference each field explicitly. I'd prefer to push the root document (minus the _id and previous fields) without knowing what the other fields are.
In addition to the new $set operator, what makes your use case really easier with Mongo 4.2 is the fact that db.collection.update() now accepts an aggregation pipeline, finally allowing the update of a field based on its current value:
// { name: "foo", value: 1 }
db.collection.update(
{},
[{ $set: {
previous: {
$ifNull: [
{ $concatArrays: [ "$previous", [{ name: "$name", value: "$value" }] ] },
[ { name: "$name", value: "$value" } ]
]
},
name: "bar",
value: 2
}}],
{ multi: true }
)
// { name: "bar", value: 2, previous: [{ name: "foo", value: 1 }] }
// and if applied again:
// { name: "baz", value: 3, previous: [{ name: "foo", value: 1 }, { name: "bar", value: 2 } ] }
The first part {} is the match query, filtering which documents to update (in our case probably all documents).
The second part [{ $set: { previous: { $ifNull: [ ... } ] is the update aggregation pipeline (note the squared brackets signifying the use of an aggregation pipeline):
$set is a new aggregation operator and an alias of $addFields. It's used to add/replace a new field (in our case "previous") with values from the current document.
Using an $ifNull check, we can determine whether "previous" already exists in the document or not (this is not the case for the first update).
If "previous" doesn't exist (is null), then we have to create it and set it with an array of one element: the current document: [ { name: "$name", value: "$value" } ].
If "previous" already exist, then we concatenate ($concatArrays) the existing array with the current document.
Don't forget { multi: true }, otherwise only the first matching document will be updated.
As you mentioned "root" in your question and if your schema is not the same for all documents (if you can't tell which fields should be used and pushed in the "previous" array), then you can use the $$ROOT variable which represents the current document and filter out the "previous" array. In this case, replace both { name: "$name", value: "$value" } from the previous query with:
{ $arrayToObject: { $filter: {
input: { $objectToArray: "$$ROOT" },
as: "root",
cond: { $ne: [ "$$root.k", "previous" ] }
}}}
Imho, you are making your life indefinitely more complex for no reason with such complicated data models.
Think of what you really want to achieve. You want to correlate different values in one or more interconnected series which are written to the collection consecutively.
Storing this in one document comes with some strings attached. While it seems to be reasonable in the beginning, let me name a few:
How do you get the most current document if you do not know it's value for name?
How do you deal with very large series, which make the document hit the 16MB limit?
What is the benefit of the added complexity?
Simplify first
So, let's assume you have only one series for a moment. It gets as simple as
[{
"_id":"foo",
"ts": ISODate("2019-07-03T17:40:00.000Z"),
"value":1
},{
"_id":"bar",
"ts": ISODate("2019-07-03T17:45:00.000"),
"value":2
},{
"_id":"baz",
"ts": ISODate("2019-07-03T17:50:00.000"),
"value":3
}]
Assuming the name is unique, we can use it as _id, potentially saving an index.
You can actually get the semantic equivalent by simply doing a
> db.seriesa.find().sort({ts:-1})
{ "_id" : "baz", "ts" : ISODate("2019-07-03T17:50:00Z"), "value" : 3 }
{ "_id" : "bar", "ts" : ISODate("2019-07-03T17:45:00Z"), "value" : 2 }
{ "_id" : "foo", "ts" : ISODate("2019-07-03T17:40:00Z"), "value" : 1 }
Say you only want to have the two latest values, you can use limit():
> db.seriesa.find().sort({ts:-1}).limit(2)
{ "_id" : "baz", "ts" : ISODate("2019-07-03T17:50:00Z"), "value" : 3 }
{ "_id" : "bar", "ts" : ISODate("2019-07-03T17:45:00Z"), "value" : 2 }
Should you really need to have the older values in a queue-ish array
db.seriesa.aggregate([{
$group: {
_id: "queue",
name: {
$last: "$_id"
},
value: {
$last: "$value"
},
previous: {
$push: {
name: "$_id",
value: "$value"
}
}
}
}, {
$project: {
name: 1,
value: 1,
previous: {
$slice: ["$previous", {
$subtract: [{
$size: "$previous"
}, 1]
}]
}
}
}])
Nail it
Now, let us say you have more than one series of data. Basically, there are two ways of dealing with it: put different series in different collections or put all the series in one collection and make a distinction by a field, which for obvious reasons should be indexed.
So, when to use what? It boils down wether you want to do aggregations over all series (maybe later down the road) or not. If you do, you should put all series into one collection. Of course, we have to slightly modify our data model:
[{
"name":"foo",
"series": "a"
"ts": ISODate("2019-07-03T17:40:00.000Z"),
"value":1
},{
"name":"bar",
"series": "a"
"ts": ISODate("2019-07-03T17:45:00.000"),
"value":2
},{
"name":"baz",
"series": "a"
"ts": ISODate("2019-07-03T17:50:00.000"),
"value":3
},{
"name":"foo",
"series": "b"
"ts": ISODate("2019-07-03T17:40:00.000Z"),
"value":1
},{
"name":"bar",
"series": "b"
"ts": ISODate("2019-07-03T17:45:00.000"),
"value":2
},{
"name":"baz",
"series": "b"
"ts": ISODate("2019-07-03T17:50:00.000"),
"value":3
}]
Note that for demonstration purposes, I fell back for the default ObjectId value for _id.
Next, we create an index over series and ts, as we are going to need it for our query:
> db.series.ensureIndex({series:1,ts:-1})
And now our simple query looks like this
> db.series.find({"series":"b"},{_id:0}).sort({ts:-1})
{ "name" : "baz", "series" : "b", "ts" : ISODate("2019-07-03T17:50:00Z"), "value" : 3 }
{ "name" : "bar", "series" : "b", "ts" : ISODate("2019-07-03T17:45:00Z"), "value" : 2 }
{ "name" : "foo", "series" : "b", "ts" : ISODate("2019-07-03T17:40:00Z"), "value" : 1 }
In order to generate the queue-ish like document, we need to add a match state
> db.series.aggregate([{
$match: {
"series": "b"
}
},
// other stages omitted for brevity
])
Note that the index we created earlier will be utilized here.
Or, we can generate a document like this for every series by simply using series as the _id in the $group stage and replace _id with name where appropriate
db.series.aggregate([{
$group: {
_id: "$series",
name: {
$last: "$name"
},
value: {
$last: "$value"
},
previous: {
$push: {
name: "$name",
value: "$value"
}
}
}
}, {
$project: {
name: 1,
value: 1,
previous: {
$slice: ["$previous", {
$subtract: [{
$size: "$previous"
}, 1]
}]
}
}
}])
Conclusion
Stop Being Clever when it comes to data models in MongoDB. Most of the problems with data models I saw in the wild and the vast majority I see on SO come from the fact that someone tried to be Smart (by premature optimization) ™.
Unless we are talking of ginormous series (which can not be, since you settled for a 16MB limit in your approach), the data model and queries above are highly efficient without adding unneeded complexity.
addMultipleData: (req, res, next) => {
let name = req.body.name ? req.body.name : res.json({ message: "Please enter Name" });
let value = req.body.value ? req.body.value : res.json({ message: "Please Enter Value" });
if (!req.body.name || !req.body.value) { return; }
//Step 1
models.dynamic.findOne({}, function (findError, findResponse) {
if (findResponse == null) {
let insertedValue = {
name: name,
value: value
}
//Step 2
models.dynamic.create(insertedValue, function (error, response) {
res.json({
message: "succesfully inserted"
})
})
}
else {
let pushedValue = {
name: findResponse.name,
value: findResponse.value
}
let updateWith = {
$set: { name: name, value: value },
$push: { previous: pushedValue }
}
let options = { upsert: true }
//Step 3
models.dynamic.updateOne({}, updateWith, options, function (error, updatedResponse) {
if (updatedResponse.nModified == 1) {
res.json({
message: "succesfully inserted"
})
}
})
}
})
}
//This is the schema
var multipleAddSchema = mongoose.Schema({
"name":String,
"value":Number,
"previous":[]
})