how to match the last value of array in mongo db? [duplicate] - mongodb

I have a sample document like shown below
{
"_id" : "docID",
"ARRAY" : [
{
"k" : "value",
"T" : "20:15:35",
"I" : "Hai"
},
{
"K" : "some value",
"T" : "20:16:35",
"I" : "Hello"
},
{
"K" : "some other value",
"T" : "20:15:35",
"I" : "Update"
}
]
}
I am trying to update the last element in the "ARRAY" based on field "ARRAY.T"(which is only field i know at the point of update), but what my problem is first element in the array matches the query and its ARRAY.I field is updated.
Query used to update:
db.collection.update( { _id: "docID","ARRAY.T" : "20:15:35"},
{ $set: { "ARRAY.$.I": "Updated value" }
})
Actually i don't know index of the array where to update so i have to use ARRAY.I in the query, is there any way to to tell Mongodb to update the first element matched the query from last of the array.

I understand what you are saying in that you want to match the last element in this case or in fact process the match in reverse order. There is no way to modify this and the index stored in the positional $ operator will always be the "first" match.
But you can change your approach to this, as the default behavior of $push is to "append" to the end of the array. But MongoDB 2.6 introduced a $position modifier so you can in fact always "pre-pend" to the array meaning your "oldest" item is at the end.
Take this for example:
db.artest.update(
{ "array": { "$in": [5] } },
{ "$push": { "array": { "$each": [5], "$position": 0 } }},
{ "upsert": true }
)
db.artest.update(
{ "array": { "$in": [5] } },
{ "$push": { "array": { "$each": [6], "$position": 0 } }},
{ "upsert": true }
)
This results in a document that is the "reverse" of the normal $push behavior:
{ "_id" : ObjectId("53eaf4517d0dc314962c93f4"), "array" : [ 6, 5 ] }
Alternately you could apply the $sort modifier when updating your documents in order to "order" the elements so they were reversed. But that may not be the best option if duplicate values are stored.
So look into storing your arrays in "reverse" if you intend to match the "newest" items "first". Currently that is your only way of getting your "match from last" behavior.

Related

How do I match an array of sub-documents in MongoDB?

Match documents if a value in an array of sub-documents is greater than some value only if the same document contains a field that is equal to some value
I have a collection that contains documents with an array of sub-documents. This array of sub-documents contains a field that dictates whether or not I can filter the documents in the collection based on another field in the sub-document. This'll make more sense when you see an example of the document.
{
"_id":"ObjectId('XXX')",
"Data":{
"A":"",
"B":"-25.78562 ; 28.35629",
"C":"165"
},
"SubDocuments":[
{
"_id":"ObjectId('XXX')",
"Data":{
"Value":"XXX",
"DataFieldId":"B"
}
},
{
"_id":"ObjectId('XXX')",
"Data":{
"Value":"",
"DataFieldId":"A"
}
},
{
"_id":"ObjectId('XXX')",
"Data":{
"Value":"105",
"DataFieldId":"Z"
}
}
]
}
I only want to match documents that contain sub-documents with a DataFieldId that is equal to Z but also filter for Values that are greater than 105 only if Data Field Id is equal to Z.
Try as below:
db.collection.aggregate([
{
$project: {
_id:1,
Data:1,
filteredSubDocuments: {
$filter: {
input: "$SubDocuments",
as: "subDoc",
cond: {
$and: [
{ $eq: ["$$subDoc.Data.DataFieldId", "Z"] },
{ $gte: ["$$subDoc.Data.Value", 105] }
]
}
}
}
}
}
])
Resulted response will be:
{
"_id" : ObjectId("5cb09659952e3a179190d998"),
"Data" : {
"A" : "",
"B" : "-25.78562 ; 28.35629",
"C" : "165"
},
"filteredSubDocuments" : [
{
"_id" : "ObjectId('XXX')",
"Data" : {
"Value" : 105,
"DataFieldId" : "Z"
}
}
]
}
This can be done by using the $elemMatch operator on sub-documents, for details you can click on provided link. For your problem you can try below query by using $elemMatch which is match simpler than aggregation:
db.collectionName.find({
"SubDocuments": {
$elemMatch: {
"Data.DataFieldId": "Z" ,
"Data.Value" : {$gte: 105}
}
} })
Its working fine, I have verified it locally, one modification you required is that you have to put the value of SubDocuments.Data.Value as Number or Long as per your requirements.

Compare Properties from Array to Single Property in Document

I have a mongoDB orders collection, the documents of which look as follows:
[{
"_id" : ObjectId("59537df80ab10c0001ba8767"),
"shipments" : {
"products" : [
{
"orderDetails" : {
"id" : ObjectId("59537df80ab10c0001ba8767")
}
},
{
"orderDetails" : {
"id" : ObjectId("59537df80ab10c0001ba8767")
}
}
]
},
}
{
"_id" : ObjectId("5953831367ae0c0001bc87e1"),
"shipments" : {
"products" : [
{
"orderDetails" : {
"id" : ObjectId("5953831367ae0c0001bc87e1")
}
}
]
},
}]
Now, from this collection, I want to filter out the elements in which, any of the values at shipments.products.orderDetails.id path is same as value at _id path.
I tried:
db.orders.aggregate([{
"$addFields": {
"same": {
"$eq": ["$shipments.products.orderDetails.id", "$_id"]
}
}
}])
to add a field same as a flag to decide whether the values are equal, but the value of same comes as false for all documents.
EDIT
What I want to do is compare the _id field the the documents with all shipments.products.orderDetails.id values in the array.
If even 1 of the shipments.products.orderDetails.ids match the value of the _id field, I want that document to be present in the final result.
PS I am using MongoDB 3.4, and have to use the aggregation pipeline.
Your current attempt fails because the notation returns an "array" in comparison with a "single value".
So instead either use $in where available, which can compare to see if one value is "in" an array:
db.orders.aggregate([
{ "$addFields": {
"same": {
"$in": [ "$_id", "$shipments.products.orderDetails.id" ]
}
}}
])
Or notate both as arrays using $setIsSubset
db.orders.aggregate([
{ "$addFields": {
"same": {
"$setIsSubset": [ "$shipments.products.orderDetails.id", ["$_id"] ]
}
}}
])
Where in that case it's doing a comparison to see if the "sets" have an "intersection" that makes _id the "subset" of the array of values.
Either case will return true when "any" of the id properties within the array entries at the specified path are a match for the _id property of the document.

Query by field value, not value in field array

The following snippet shows three queries:
find all the documents
find the documents containing a field a containing either the string "x" or an array containing the string "x"
find the documents containing a field a containing an array containing the string "x"
I was not able to find the documents containing a field a containing the string "x", not inside an array.
> db.stuff.find({},{_id:0})
{ "a" : "x" }
{ "a" : [ "x" ] }
> db.stuff.find({a:"x"},{_id:0})
{ "a" : "x" }
{ "a" : [ "x" ] }
> db.stuff.find({a:{$elemMatch:{$eq:"x"}}},{_id:0})
{ "a" : [ "x" ] }
>
MongoDB basically does not care if the data at a "given path" is actually in an array or not. If you want to make the distinction, then you need to "tell it that":
db.stuff.find({ "a": "x", "$where": "return !Array.isArray(this.a)" })
This is what $where adds to the bargain, where you can supply a condition that explicitly asks "is this an array" via Array.isArray() in JavaScript evaluation. And the JavaScript NOT ! assertion reverses the logic.
An alternate approach is to add the $exists check:
db.stuff.find({ "a": "x", "a.0": { "$exists": false } })
Which also essentially asks "is this an array" by looking for the first element index. So the "reverse" false case means "this is not an array".
Or even as you note you can use $elemMatch to select only the array, but "negate" that using $not:
db.stuff.find({ "a": { "$not": { "$elemMatch": { "$eq": "x" } } } })
Though probably "not" the best of options since that also "negates index usage", which the other examples all strive to avoid by at least including "one" positive condition for a match. So it's for the best to include the "implicit AND" by combining arguments:
db.stuff.find({
"a": { "$eq": "x", "$not": { "$elemMatch": { "$eq": "x" } } }
})
Or for "aggregation" which does not support $where, you can test using the $isArray aggregation operator should your MongoDB version ( 3.2 or greater ) support it:
db.stuff.aggregate([
{ "$match": { "a": "x" } },
{ "$redact": {
"$cond": {
"if": { "$not": { "$isArray": "$a" } },
"then": "$$KEEP",
"else": "$$PRUNE"
}
}}
])
Noting that it is good practice to supply "regular" query conditions as well where possible, and in all cases.
Also noting that querying the BSON $type does not typically work in this case, since the "contents" of the array itself are in fact a "string", which is what the $type operator is going to consider, and thus not report that such an array is in fact an array.

Pull array within array

Similar to Find document with array that contains a specific value, but i'm trying to pull it.
db.getCollection('users').find({'favorites':{$elemMatch:{0:5719}}}, {"favorites.$": 1})
returns this:
{
"_id" : "FfEj5chmviLdqWh52",
"favorites" : [
[
5719,
"2016-03-21T17:46:01.441Z",
"a"
]
]
}
even after this returned 1:
Meteor.users.update(this.userId, {$pull: {'favorites':{$elemMatch:{0:movieid}}}})
It doesn't work because $pull is trying to remove a matching element from the "favorites" array. What you want to do is remove from the "array inside the array" of favorites.
For this you need a positional match to point to the nth inner element, then a very careful $pull expression to actually remove that element:
Meteor.users.update(
{ "favorites": { "$elemMatch": { "$elemMatch": { "$eq": 5719 } } } },
{ "$pull": { "favorites.$": 5719 } }
)
The "double" $elemMatch with the $eq operator is a bit more expressive than { 0: 5719 } since it is not "locked" into the first position only and is actually looking at the matching value. But you can write it that way if you must, or if you "really mean" to match that value in the first position only.
Note that the "index" returned from the match in the positional $ argument is actually that of the "outer" array. So to pull from the
Of course if there is only ever one nested array element within, the you might as well just write:
{ "$pull": { "favorites.0": 5719 } }
Using the direct "first index" position, since you know the inner array will always be there.
In either case, your object updates correctly:
{
"_id" : "FfEj5chmviLdqWh52",
"favorites" : [
[
"2016-03-21T17:46:01.441Z",
"a"
]
]
}
If you are trying to $pull the entire array entry from favorites, then the $eleMatch just needs to be dialed back one element:
Meteor.users.update(
{ "_id": this.userId },
{ "$pull": { "favorites": { "$elemMatch": { "$eq": 5719 } } } }
)
Or even:
Meteor.users.update(
{ "_id": this.userId },
{ "$pull": { "favorites": { "$elemMatch": { "0": 5719 } } } }
)
Noting that:
{ "_id": this.userId },
Is the long form that we generally use as a "query" selector, and especially when we want criteria "other than" the _id of the document. MiniMongo statements require at "least" the _id of the document though.
The rest of the statement has one "less" $elemMatch because the $pull already applies to the array.
That removes the whole matched element from the outer array:
{
"_id" : "FfEj5chmviLdqWh52",
"favorites" : []
}
This is the first code i found that actually works:
Meteor.users.update(Meteor.userId(), {$pull: {favorites: {$in: [i]}}})
Apparently $in does partial matching. It seems safer than the working code from this answer:
Meteor.users.update(
{ "_id": this.userId },
{ "$pull": { "favorites": { "$elemMatch": { "$eq": i } } } }
)

Remove duplicate in MongoDB

I have a collection with the field called "contact_id".
In my collection I have duplicate registers with this key.
How can I remove duplicates, resulting in just one register?
I already tried:
db.PersonDuplicate.ensureIndex({"contact_id": 1}, {unique: true, dropDups: true})
But did not work, because the function dropDups is no longer available in MongoDB 3.x
I'm using 3.2
Yes, dropDups is gone for good. But you can definitely achieve your goal with little bit effort.
You need to first find all duplicate rows and then remove all except first.
db.dups.aggregate([{$group:{_id:"$contact_id", dups:{$push:"$_id"}, count: {$sum: 1}}},
{$match:{count: {$gt: 1}}}
]).forEach(function(doc){
doc.dups.shift();
db.dups.remove({_id : {$in: doc.dups}});
});
As you see doc.dups.shift() will remove first _id from array and then remove all documents with remaining _ids in dups array.
script above will remove all duplicate documents.
this is a good pattern for mongod 3+ that also ensures that you will not run our of memory which can happen with really big collections. You can save this to a dedup.js file, customize it, and run it against your desired database with: mongo localhost:27017/YOURDB dedup.js
var duplicates = [];
db.runCommand(
{aggregate: "YOURCOLLECTION",
pipeline: [
{ $group: { _id: { DUPEFIELD: "$DUPEFIELD"}, dups: { "$addToSet": "$_id" }, count: { "$sum": 1 } }},
{ $match: { count: { "$gt": 1 }}}
],
allowDiskUse: true }
)
.result
.forEach(function(doc) {
doc.dups.shift();
doc.dups.forEach(function(dupId){ duplicates.push(dupId); })
})
printjson(duplicates); //optional print the list of duplicates to be removed
db.YOURCOLLECTION.remove({_id:{$in:duplicates}});
We can also use an $out stage to remove duplicates from a collection by replacing the content of the collection with only one occurrence per duplicate.
For instance, to only keep one element per value of x:
// > db.collection.find()
// { "x" : "a", "y" : 27 }
// { "x" : "a", "y" : 4 }
// { "x" : "b", "y" : 12 }
db.collection.aggregate(
{ $group: { _id: "$x", onlyOne: { $first: "$$ROOT" } } },
{ $replaceWith: "$onlyOne" }, // prior to 4.2: { $replaceRoot: { newRoot: "$onlyOne" } }
{ $out: "collection" }
)
// > db.collection.find()
// { "x" : "a", "y" : 27 }
// { "x" : "b", "y" : 12 }
This:
$groups documents by the field defining what a duplicate is (here x) and accumulates grouped documents by only keeping one (the $first found) and giving it the value $$ROOT, which is the document itself. At the end of this stage, we have something like:
{ "_id" : "a", "onlyOne" : { "x" : "a", "y" : 27 } }
{ "_id" : "b", "onlyOne" : { "x" : "b", "y" : 12 } }
$replaceWith all existing fields in the input document with the content of the onlyOne field we've created in the $group stage, in order to find the original format back. At the end of this stage, we have something like:
{ "x" : "a", "y" : 27 }
{ "x" : "b", "y" : 12 }
$replaceWith is only available starting in Mongo 4.2. With prior versions, we can use $replaceRoot instead:
{ $replaceRoot: { newRoot: "$onlyOne" } }
$out inserts the result of the aggregation pipeline in the same collection. Note that $out conveniently replaces the content of the specified collection, making this solution possible.
maybe it be a good try to create a tmpColection, create unique index, then copy data from source, and last step will be swap names?
Other idea, I had is to get doubled indexes into array (using aggregation) and then loop thru calling the remove() method with the justOne parameter set to true or 1.
var itemsToDelete = db.PersonDuplicate.aggregate([
{$group: { _id:"$_id", count:{$sum:1}}},
{$match: {count: {$gt:1}}},
{$group: { _id:1, ids:{$addToSet:"$_id"}}}
])
and make a loop thru ids array
makes this sense for you?
I have used this approach:
Take the mongo dump of the particular collection.
Clear that collection
Add a unique key index
Restore the dump using mongorestore.