mongodb query map by key - mongodb

I have the following structure in mongodb:
person:
{
"1" [personId] : ["some_text", "some_text"],
"2" [personId] : ["some_text", "some_text"],
"3" [personId] : ["some_text", "some_text"]
}
I would like to query a person map structure and get only values when personid (key) is between 1 to 2
I try to use $elemMatch but its not a good idea because I would like query by a dynamic range.
I use map strucute becuase I have another process that insert person data dynamically by person id.
Is there any way to filter map structure data by key?
Thanks


The code is not actually tested. But this might work:
const searchParams = [
{ $elemMatch: { fieldA: "val1", fieldB: "val2" } },
{ $elemMatch: { fieldC: "val3", fieldD: "val4" } },
]
const results = await yourDB.find({
person: {
$in: searchParam
}
})
The elements of searchParams can be dynamically generated based on your need.
searchParams.psuh({ $elemMatch: { fieldX: "val_x", fieldZ: "val_z })


You need an to run an aggregate operation where you can convert the hash map into an array of key value documents, filter the array based on the above condition and convert the filtered array back to the hash map.
Take for instance the following operation, it uses the $addFields pipeline step to do the above operations with the aid of the $objectToArray, $filter and $arrayToObject operators:
db.collection.aggregate([
{ '$addFields': {
'person': {
'$arrayToObject': {
'$filter': {
'input': { '$objectToArray': '$person' },
'cond': {
'$in': ['$$this.k', ['1', '2']]
}
}
}
}
} }
])

Related

MongoDb 4.x Query with projection without empty entries using find

We are now using mongo-db to store data from tests. I am using Mongo-Shell
The document is structured like this:
{
static1:"abc",
static2:"xyz",
static3:"asd", [...],
nested:[
{
data1: "d1",
data2: "d2",
},
{
data1: "dx",
data4: "d4",
data5: "d5",
data6: "d6",
},
{
data1: "ds",
data8:"data8"
}, [...]
]
}
So the static-data is always in the same structure, but for every measurement the object can look different. It could be voltage with upper, lower and actual value. Or just comparison of a target value and an actual value. Highly dynamic.
Same data1-name represent same attributes.
Now e.g. I want to display only some static data and one (or several) attributes of a nested document.
I am using this query:
find({}, {_id:0, data1:1, "nested.data8":1}).pretty()
As expected, only the static-data1 is displayed, but the dynamic measurements has lots of empty objects in the shell-output
Example-Output:
{
"static1" : "123",
"nested" : [
{ },
{ },
{ }, [...] ,
{ "data8" : "OK" }
] }
The desired output would be:
{
"static1" : "123",
"nested" : [
{ "data8" : "d8" }
]
}
I also tried this query on the mongo shell:
aggregate( { $addFields: {"static":"$static1", "data8":"$nested.data8"} }, { $project:{"static1":1, "nested.data8":1} } ).pretty()
But the result is the same.
I hope there is a ways to get rid of the empty documents in the output.
Thanks

You can try $filter operator to filter result of nested array by checking condition not equal to empty object {}
db.collection.aggregate([
{
$project: {
_id: 0,
data1: 1,
"nested.data8": 1
}
},
{
$set: {
nested: {
$filter: {
input: "$nested",
cond: { $ne: ["$$this", {}] }
}
}
}
}
])
Playground

Conditionally set element of array or push new element in mongo update [duplicate]

I have the following collection
{
"_id" : ObjectId("57315ba4846dd82425ca2408"),
"myarray" : [
{
userId : ObjectId("570ca5e48dbe673802c2d035"),
point : 5
},
{
userId : ObjectId("613ca5e48dbe673802c2d521"),
point : 2
},
]
}
These are my questions
I want to push into myarray if userId doesn't exist, it should be appended to myarray. If userId exists, it should be updated to point.
I found this
db.collection.update({
_id : ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId" : ObjectId("570ca5e48dbe673802c2d035")
}, {
$set: { "myarray.$.point": 10 }
})
But if userId doesn't exist, nothing happens.
and
db.collection.update({
_id : ObjectId("57315ba4846dd82425ca2408")
}, {
$push: {
"myarray": {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
})
But if userId object already exists, it will push again.
What is the best way to do this in MongoDB?

Try this
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{ $pull: {"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")}}
)
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{ $push: {"myarray": {
userId:ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}}
)
Explination:
in the first statment $pull removes the element with userId= ObjectId("570ca5e48dbe673802c2d035") from the array on the document where _id = ObjectId("57315ba4846dd82425ca2408")
In the second one $push inserts
this object { userId:ObjectId("570ca5e48dbe673802c2d035"), point: 10 } in the same array.

The accepted answer by Flying Fisher is that the existing record will first be deleted, and then it will be pushed again.
A safer approach (common sense) would be to try to update the record first, and if that did not find a match, insert it, like so:
// first try to overwrite existing value
var result = db.collection.update(
{
_id : ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")
},
{
$set: {"myarray.$.point": {point: 10}}
}
);
// you probably need to modify the following if-statement to some async callback
// checking depending on your server-side code and mongodb-driver
if(!result.nMatched)
{
// record not found, so create a new entry
// this can be done using $addToSet:
db.collection.update(
{
_id: ObjectId("57315ba4846dd82425ca2408")
},
{
$addToSet: {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
}
);
// OR (the equivalent) using $push:
db.collection.update(
{
_id: ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId": {$ne: ObjectId("570ca5e48dbe673802c2d035"}}
},
{
$push: {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
}
);
}
This should also give (common sense, untested) an increase in performance, if in most cases the record already exists, only the first query will be executed.

There is a option called update documents with aggregation pipeline starting from MongoDB v4.2,
check condition $cond if userId in myarray.userId or not
if yes then $map to iterate loop of myarray array and check condition if userId match then merge with new document using $mergeObjects
if no then $concatArrays to concat new object and myarray
let _id = ObjectId("57315ba4846dd82425ca2408");
let updateDoc = {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
};
db.collection.update(
{ _id: _id },
[{
$set: {
myarray: {
$cond: [
{ $in: [updateDoc.userId, "$myarray.userId"] },
{
$map: {
input: "$myarray",
in: {
$mergeObjects: [
"$$this",
{
$cond: [
{ $eq: ["$$this.userId", updateDoc.userId] },
updateDoc,
{}
]
}
]
}
}
},
{ $concatArrays: ["$myarray", [updateDoc]] }
]
}
}
}]
)
Playground

Unfortunately "upsert" operation is not possible on embedded array. Operators simply do not exist so that this is not possible in a single statement.Hence you must perform two update operations in order to do what you want. Also the order of application for these two updates is important to get desired result.

I haven't found any solutions based on a one atomic query. Instead there are 3 ways based on a sequence of two queries:
always $pull (to remove the item from array), then $push (to add the updated item to array)
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{ $pull: {"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")}}
)
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{
$push: {
"myarray": {
userId:ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
}
)
try to $set (to update the item in array if exists), then get the result and check if the updating operation successed or if a $push needs (to insert the item)
var result = db.collection.update(
{
_id : ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")
},
{
$set: {"myarray.$.point": {point: 10}}
}
);
if(!result.nMatched){
db.collection.update({_id: ObjectId("57315ba4846dd82425ca2408")},
{
$addToSet: {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
);
always $addToSet (to add the item if not exists), then always $set to update the item in array
db.collection.update({_id: ObjectId("57315ba4846dd82425ca2408")},
myarray: { $not: { $elemMatch: {userId: ObjectId("570ca5e48dbe673802c2d035")} } } },
{
$addToSet : {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
},
{ multi: false, upsert: false});
db.collection.update({
_id: ObjectId("57315ba4846dd82425ca2408"),
"myArray.userId": ObjectId("570ca5e48dbe673802c2d035")
},
{ $set : { myArray.$.point: 10 } },
{ multi: false, upsert: false});
1st and 2nd way are unsafe, so transaction must be established to avoid two concurrent requests could push the same item generating a duplicate.
3rd way is safer. the $addToSet adds only if the item doesn't exist, otherwise nothing happens. In case of two concurrent requests, only one of them adds the missing item to the array.

Possible solution with aggregation pipeline:
db.collection.update(
{ _id },
[
{
$set: {
myarray: { $filter: {
input: '$myarray',
as: 'myarray',
cond: { $ne: ['$$myarray.userId', ObjectId('570ca5e48dbe673802c2d035')] },
} },
},
},
{
$set: {
myarray: {
$concatArrays: [
'$myarray',
[{ userId: ObjectId('570ca5e48dbe673802c2d035'), point: 10 },
],
],
},
},
},
],
);
We use 2 stages:
filter myarray (= remove element if userId exist)
concat filtered myarray with new element;

When you want update or insert value in array try it
Object in db
key:name,
key1:name1,
arr:[
{
val:1,
val2:1
}
]
Query
var query = {
$inc:{
"arr.0.val": 2,
"arr.0.val2": 2
}
}
.updateOne( { "key": name }, query, { upsert: true }
key:name,
key1:name1,
arr:[
{
val:3,
val2:3
}
]

In MongoDB 3.6 it is now possible to upsert elements in an array.

array update and create don't mix in under one query, if you care much about atomicity then there's this solution:
normalise your schema to,
{
"_id" : ObjectId("57315ba4846dd82425ca2408"),
userId : ObjectId("570ca5e48dbe673802c2d035"),
point : 5
}

You could use a variation of the .forEach/.updateOne method I currently use in mongosh CLI to do things like that. In the .forEach, you might be able to set all of your if/then conditions that you mentioned.
Example of .forEach/.updateOne:
let medications = db.medications.aggregate([
{$match: {patient_id: {$exists: true}}}
]).toArray();
medications.forEach(med => {
try {
db.patients.updateOne({patient_id: med.patient_id},
{$push: {medications: med}}
)
} catch {
console.log("Didn't find match for patient_id. Could not add this med to a patient.")
}
})
This may not be the most "MongoDB way" to do it, but it definitely works and gives you the freedom of javascript to do things within the .forEach.

Sorting query results by the order of items in provided conditions array in Mongoose

I am looking for a way to query a database with Mongoose based on an array of document IDs and return the results in the order in which they were presented to Mongoose within that array.
For example if I provided Mongoose with this array ["112", "111", "113"]
and assuming that all the documents with these IDs exist the result will be an array of documents ordered accordingly as: [{//112},{//111},{//113}]
Just to be crystal clear, I don't need to sort the array by any specific document field - I am looking for a way to tell Mongoose to sort by the order of the IDs as presented in the conditions array itself. Is this possible?

Considering following data:
db.col.save({ a: "111"})
db.col.save({ a: "112"})
db.col.save({ a: "113"})
db.col.save({ a: "114"})
you can use Aggregation Framework's $match to filter out all the items that are not present in specified array and the $addFields with $indexOfArray to get the index property. Then you can $sort by that property and use $project to remove temporary field. Try:
db.col.aggregate([
{
$match: { a: { $in: ["112", "111", "113"] } }
},
{
$addFields: {
index: { $indexOfArray: [ ["112", "111", "113"], "$a" ] }
}
},
{
$sort: { index: 1 }
},
{
$project: { index: 0, _id: 0 }
}
])
Outputs:
{ "a" : "112" }
{ "a" : "111" }
{ "a" : "113" }

Mongo Query to return common values in array

I need a Mongo Query to return me common values present in an array.
So if there are 4 documents in match, then the values are returned if those are present in in all the 4 documents
Suppose I have the below documents in my db
Mongo Documents
{
"id":"0",
"merchants":["1","2"]
}
{
"id":"1",
"merchants":["1","2","4"]
}
{
"id":"2",
"merchants":["4","5"]
}
Input : List of id
(i) Input with id "0" and "1"
Then it should return me merchants:["1","2"] as both are present in documents with id "0" & id "1"
(ii) Input with id "1" and "2"
Then it should return me merchants:["4"] as it is common and present in both documents with id "1" & id "2"
(iii) Input with id "0" and "2"
Should return empty merchants:[] as no common merchants between these 2 documents

You can try below aggregation.
db.collection.aggregate(
{$match:{id: {$in: ["1", "2"]}}},
{$group:{_id:null, first:{$first:"$merchants"}, second:{$last:"$merchants"}}},
{$project: {commonToBoth: {$setIntersection: ["$first", "$second"]}, _id: 0 } }
)

Say you have a function query that does the required DB query for you, and you'll call that function with idsToMatch which is an array containing all the elements you want to match. I have used JS here as the driver language, replace it with whatever you are using.
The following code is dynamic, will work for any number of ids you give as input:
const query = (idsToMatch) => {
db.collectionName.aggregate([
{ $match: { id: {$in: idsToMatch} } },
{ $unwind: "$merchants" },
{ $group: { _id: { id: "$id", data: "$merchants" } } },
{ $group: { _id: "$_id.data", count: {$sum: 1} } },
{ $match: { count: { $gte: idsToMatch.length } } },
{ $group: { _id: 0, result: {$push: "$_id" } } },
{ $project: { _id: 0, result: "$result" } }
])
The first $group statement is to make sure you don't have any
repetitions in any of your merchants attribute in a document. If
you are certain that in your individual documents you won't have any
repeated value for merchants, you need not include it.
The real work happens only upto the 2nd $match phase. The last two
phases ($group and $project) are only to prettify the result,
you may choose not use them, and instead use the language of your
choice to transform it in the form you want
Assuming you want to reduce the phases as per the points given above, the actual code will reduce to:
aggregate([
{ $match: { id: {$in: idsToMatch} } },
{ $unwind: "$merchants" },
{ $group: { _id: "merchants", count: {$sum: 1} } },
{ $match: { count: { $gte: idsToMatch.length } } }
])
Your required values will be at the _id attribute of each element of the result array.

The answer provided by #jgr0 is correct to some extent. The only mistake is the intermediate match operation
(i) So if input ids are "1" & "0" then the query becomes
aggregate([
{"$match":{"id":{"$in":["1","0"]}}},
{"$unwind":"$merchants"},
{"$group":{"_id":"$merchants","count":{"$sum":1}}},
{"$match":{"count":{"$eq":2}}},
{"$group":{"_id":null,"merchants":{"$push":"$_id"}}},
{"$project":{"_id":0,"merchants":1}}
])
(ii) So if input ids are "1", "0" & "2" then the query becomes
aggregate([
{"$match":{"id":{"$in":["1","0", "2"]}}},
{"$unwind":"$merchants"},
{"$group":{"_id":"$merchants","count":{"$sum":1}}},
{"$match":{"count":{"$eq":3}}},
{"$group":{"_id":null,"merchants":{"$push":"$_id"}}},
{"$project":{"_id":0,"merchants":1}}
])
The intermediate match operation should be the count of ids in input. So in case (i) it is 2 and in case (2) it is 3.

MongoDB - Update or Insert object in array

I have the following collection
{
"_id" : ObjectId("57315ba4846dd82425ca2408"),
"myarray" : [
{
userId : ObjectId("570ca5e48dbe673802c2d035"),
point : 5
},
{
userId : ObjectId("613ca5e48dbe673802c2d521"),
point : 2
},
]
}
These are my questions
I want to push into myarray if userId doesn't exist, it should be appended to myarray. If userId exists, it should be updated to point.
I found this
db.collection.update({
_id : ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId" : ObjectId("570ca5e48dbe673802c2d035")
}, {
$set: { "myarray.$.point": 10 }
})
But if userId doesn't exist, nothing happens.
and
db.collection.update({
_id : ObjectId("57315ba4846dd82425ca2408")
}, {
$push: {
"myarray": {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
})
But if userId object already exists, it will push again.
What is the best way to do this in MongoDB?

Try this
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{ $pull: {"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")}}
)
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{ $push: {"myarray": {
userId:ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}}
)
Explination:
in the first statment $pull removes the element with userId= ObjectId("570ca5e48dbe673802c2d035") from the array on the document where _id = ObjectId("57315ba4846dd82425ca2408")
In the second one $push inserts
this object { userId:ObjectId("570ca5e48dbe673802c2d035"), point: 10 } in the same array.

The accepted answer by Flying Fisher is that the existing record will first be deleted, and then it will be pushed again.
A safer approach (common sense) would be to try to update the record first, and if that did not find a match, insert it, like so:
// first try to overwrite existing value
var result = db.collection.update(
{
_id : ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")
},
{
$set: {"myarray.$.point": {point: 10}}
}
);
// you probably need to modify the following if-statement to some async callback
// checking depending on your server-side code and mongodb-driver
if(!result.nMatched)
{
// record not found, so create a new entry
// this can be done using $addToSet:
db.collection.update(
{
_id: ObjectId("57315ba4846dd82425ca2408")
},
{
$addToSet: {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
}
);
// OR (the equivalent) using $push:
db.collection.update(
{
_id: ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId": {$ne: ObjectId("570ca5e48dbe673802c2d035"}}
},
{
$push: {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
}
);
}
This should also give (common sense, untested) an increase in performance, if in most cases the record already exists, only the first query will be executed.

There is a option called update documents with aggregation pipeline starting from MongoDB v4.2,
check condition $cond if userId in myarray.userId or not
if yes then $map to iterate loop of myarray array and check condition if userId match then merge with new document using $mergeObjects
if no then $concatArrays to concat new object and myarray
let _id = ObjectId("57315ba4846dd82425ca2408");
let updateDoc = {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
};
db.collection.update(
{ _id: _id },
[{
$set: {
myarray: {
$cond: [
{ $in: [updateDoc.userId, "$myarray.userId"] },
{
$map: {
input: "$myarray",
in: {
$mergeObjects: [
"$$this",
{
$cond: [
{ $eq: ["$$this.userId", updateDoc.userId] },
updateDoc,
{}
]
}
]
}
}
},
{ $concatArrays: ["$myarray", [updateDoc]] }
]
}
}
}]
)
Playground

Unfortunately "upsert" operation is not possible on embedded array. Operators simply do not exist so that this is not possible in a single statement.Hence you must perform two update operations in order to do what you want. Also the order of application for these two updates is important to get desired result.

I haven't found any solutions based on a one atomic query. Instead there are 3 ways based on a sequence of two queries:
always $pull (to remove the item from array), then $push (to add the updated item to array)
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{ $pull: {"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")}}
)
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{
$push: {
"myarray": {
userId:ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
}
)
try to $set (to update the item in array if exists), then get the result and check if the updating operation successed or if a $push needs (to insert the item)
var result = db.collection.update(
{
_id : ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")
},
{
$set: {"myarray.$.point": {point: 10}}
}
);
if(!result.nMatched){
db.collection.update({_id: ObjectId("57315ba4846dd82425ca2408")},
{
$addToSet: {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
);
always $addToSet (to add the item if not exists), then always $set to update the item in array
db.collection.update({_id: ObjectId("57315ba4846dd82425ca2408")},
myarray: { $not: { $elemMatch: {userId: ObjectId("570ca5e48dbe673802c2d035")} } } },
{
$addToSet : {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
},
{ multi: false, upsert: false});
db.collection.update({
_id: ObjectId("57315ba4846dd82425ca2408"),
"myArray.userId": ObjectId("570ca5e48dbe673802c2d035")
},
{ $set : { myArray.$.point: 10 } },
{ multi: false, upsert: false});
1st and 2nd way are unsafe, so transaction must be established to avoid two concurrent requests could push the same item generating a duplicate.
3rd way is safer. the $addToSet adds only if the item doesn't exist, otherwise nothing happens. In case of two concurrent requests, only one of them adds the missing item to the array.

Possible solution with aggregation pipeline:
db.collection.update(
{ _id },
[
{
$set: {
myarray: { $filter: {
input: '$myarray',
as: 'myarray',
cond: { $ne: ['$$myarray.userId', ObjectId('570ca5e48dbe673802c2d035')] },
} },
},
},
{
$set: {
myarray: {
$concatArrays: [
'$myarray',
[{ userId: ObjectId('570ca5e48dbe673802c2d035'), point: 10 },
],
],
},
},
},
],
);
We use 2 stages:
filter myarray (= remove element if userId exist)
concat filtered myarray with new element;

When you want update or insert value in array try it
Object in db
key:name,
key1:name1,
arr:[
{
val:1,
val2:1
}
]
Query
var query = {
$inc:{
"arr.0.val": 2,
"arr.0.val2": 2
}
}
.updateOne( { "key": name }, query, { upsert: true }
key:name,
key1:name1,
arr:[
{
val:3,
val2:3
}
]

In MongoDB 3.6 it is now possible to upsert elements in an array.

array update and create don't mix in under one query, if you care much about atomicity then there's this solution:
normalise your schema to,
{
"_id" : ObjectId("57315ba4846dd82425ca2408"),
userId : ObjectId("570ca5e48dbe673802c2d035"),
point : 5
}

You could use a variation of the .forEach/.updateOne method I currently use in mongosh CLI to do things like that. In the .forEach, you might be able to set all of your if/then conditions that you mentioned.
Example of .forEach/.updateOne:
let medications = db.medications.aggregate([
{$match: {patient_id: {$exists: true}}}
]).toArray();
medications.forEach(med => {
try {
db.patients.updateOne({patient_id: med.patient_id},
{$push: {medications: med}}
)
} catch {
console.log("Didn't find match for patient_id. Could not add this med to a patient.")
}
})
This may not be the most "MongoDB way" to do it, but it definitely works and gives you the freedom of javascript to do things within the .forEach.