Mongo sort by string value that is actually number - mongodb

I have collection that contains objects such as this:
{
"_id" : ObjectId("57f00cf47958af95dca29c0c"),
"id" : "...",
"threadId" : "...",
"ownerEmail" : "...#...",
"labelIds" : [
...
],
"snippet" : "...",
"historyId" : "35699995",
"internalDate" : "1422773000000",
"headers" : {
"from" : "...#...",
"subject" : "....",
"to" : "...#..."
},
"contents" : {
"html" : "...."
}
}
When accessing objects, I want to sort them by iternalDate value, which was supposed to be integer, however it is a string. Is there a way to sort them when fetching even if these are strings? By alphabetic order? Or is there a way to convert them to integer painlessly?

Collation is what you need...
db.collection.find()
.sort({internalDate: 1})
.collation({locale: "en_US", numericOrdering: true})

It seems to me that the best solution here would be to parse it first as an integer. You could do it using a simple script in javascript like this, using the mongodb client for node:
db.collection.find({}, {internalDate: 1}).forEach(function(doc) {
db.collection.update(
{ _id: doc._id },
{ $set: { internalDate: parseInt(doc.internalDate) } }
)
})

you also can use the aggregate method to sort number which is actually a string.
db.collection.aggregate([{
$sort : {
internalDate : 1
}
}], {
collation: {
locale: "en_US",
numericOrdering: true
}
});
if you are using mongoose-paginate package for serverside pagination .. so don't use this package, use only mongoose-paginate-v2 for serverside pagination. this package for nodejs side

I was having this issue. I use string length to sort first and then apply the sort of my numeric value stored like a string. e.g. "1", "100", "20", "3" that should be sorted like 1, 3, 29, 100.
db.AllTours.aggregate([
{
$addFields : {
"MyStringValueSize" : { $strLenCP: "$MyValue" }
}
},
{
$sort : {
"MyStringValueSize" : 1,
"MyValue" : 1
}
}
]);
There is a new feature in version 4.0 called $toInt that can be used to parse your string and then sort. In my case I can't upgrade from 3.6.

With aggregate, this works for me.
db.collection.aggregate([<pipeline>]).collation({locale:"en_US", numericOrdering:true})

This is my solution and it worked for me
db.getCollection('myCollection').aggregate([
{
$project: {
newMonth: {
$cond: { if: {
$and: [
{$ne: ['$month', '10']},
{$ne: ['$month', '11']},
{$ne: ['$month', '12']},
]
}, then: {$concat: ['0', '$month']}, else: '$month' }
}
}
},
{
$sort: {newMonth: -1}
}
])

Related

Overwrite value and create key while update query in mongodb

I have a mongodb collection that looks like this:
{
"_id" : ObjectId("60471bd482c0da3c0e70d26f"),
"owner" : "John",
"propAvailable" : {
"val1" : true
}
},
{
"_id" : ObjectId("60471bd482c0da3c0e76523f"),
"owner" : "Matt",
"propAvailable" : {
"val1" : {
"val2" : true
}
}
I need to run an update query on this collection that will update the value of the 'propAvailable' key such that
db.collection('props').update({'owner' : 'John'} , {$set : {'propAvailable.val1.val2' : true}});
This query works if the document already looks like the second one but gives the error:
Cannot create field 'val2' in element {'val1': true} if the document format is the first one. Is there a way to write this query so that it overwrites the boolean 'true' and replaces it with the object {'val2' : true}
You can use:
db.collection.update({
"owner": "John"
},
{
$set: {
"propAvailable.val1": {
val2: true
}
}
})
To create val2: true inside propAvailable.val1 and replace its current content.
As you can see working on the playground
If you're using Mongo version 4.2+ you can use pipelined updates to achieve this, like so:
db.collection.updateMany({
owner: "John"
},
[
{
$set: {
"propAvailable.val1": {
$mergeObjects: [
{
$cond: [
{
$eq: [
"object",
{
$type: "$propAvailable.val1"
}
]
},
"$propAvailable.val1",
{}
]
},
{
val2: true
}
]
}
}
},
])
Mongo Playground
For older mongo versions this is impossible to do in 1 query if objects potentially have additional fields under val1 you want to preserve. You will have to either read and update, or execute two different updates for each case.

MongoDB - Update a field in an object of an array with multiple filter conditions

If I have the following document in my database :
{
"_id" : MainId,
"subdoc" : [
{
"_id" : SubdocId,
"userid" : "someid",
"toupdate": false,
},
{
"_id" : SubdocId2,
"userid" : "someid2",
"toupdate": false,
}
],
"extra" : [
"extraid1",
"extraid2"
]
}
How can I update the subdocument SubdocId2 where the id (SubdocId2) must match and either SubdocId2's userid is "someid2" OR value "extraid1" exists in "extra"?
The farthest I got is:
db.coll.update({
"subdoc._id":"SubdocId2", {
$or: ["extra":{$in:["extraid1"]}, "subdoc.userid":"someid2"]
}
}, {"subdoc.$.toupdate":true})
Maybe I forgot to quote up something, but I get an error (SyntaxError: invalid property id)
Please try this :
db.coll.update(
{
$and: [{ "subdoc._id": SubdocId2 }, {
$or: [{ "extra": { $in: ["extraid1"] } },
{ "subdoc.userid": "someid2" }]
}] // subdoc._id && (extra || subdoc.userid)
}, { $set: { "subdoc.$.toupdate": true } })
In your query there are couple of syntax issues & also if you don't use $set in your update part - it replace the entire document with "subdoc.$.toupdate": true. Of course if you're using mongoose that's different scenario as mongoose will internally does add $set but when executing in shell or any client you need to specify $set. Also if SubdocId2 is ObjectId() you need to convert string to ObjectId() in code before querying database.

MongoDB How to remove value from array if exist otherwise add

I have document
{
"_id" : ObjectId("5aebf141a805cd28433c414c"),
"forumId" : ObjectId("5ae9f82989f7834df037cc90"),
"userName" : "Name",
"usersLike" : [
"1","2"
],
"comment" : "Comment",
}
I want to remove value from usersLike array if the value exists, or add if the value does not exist.
Eg:
If I try to push 1 into usersLike, it should return
{
"_id" : ObjectId("5aebf141a805cd28433c414c"),
"forumId" : ObjectId("5ae9f82989f7834df037cc90"),
"userName" : "Name",
"usersLike" : [
"2"
],
"comment" : "Comment",
}
How can I query it..??
MongoDB version 4.2+ introduces pipelined update. Which means we can now use aggregation operators while updating. this gives us a lot of power.
db.collection.updateOne(
{
_id: ObjectId("597afd8200758504d314b534")
},
[
{
$set: {
usersLike: {
$cond: [
{
$in: ["1", "$usersLike"]
},
{
$setDifference: ["$usersLike", ["1"]]
},
{
$concatArrays: ["$usersLike", ["1"]]
}
]
}
}
}
]
)
Mongodb doesn't support conditional push or pull update. However you can still do it by using find:
db.collectionName.find({_id:ObjectId("597afd8200758504d314b534"),usersLike:{$in:["1"]}}).pretty()
if id exist in usersLike than pull else push.
Or you use the update query to pull as:
db.collectionName.update({
_id: ObjectId("597afd8200758504d314b534"),
usersLike: {
$in: ["1"]
}
}, {
$pull: { 'usersLike': "1" }
}, { multi: true })
And to push you can use:
db.collectionName.update({
_id:ObjectId("597afd8200758504d314b534"),
usersLike:{$nin:["1"]
}},{
$push:{'usersLike':"1"}
}, {multi: true})
Try this :
db.collectionName.update({_id:ObjectId("597afd8200758504d314b534")},{$pull:{'usersLike':"1"}}, {multi: true})
Try this
if db.collectionName.find({'_id':ObjectId("5aebf141a805cd28433c414c"),'usersLike:{'$in:['1']}}).count() > 0:
db.collectionName.update({'_id':ObjectId("5aebf141a805cd28433c414c")},{'$pull':{'usersLike':'1'}})
else:
db.collectionName.update({'_id':ObjectId("5aebf141a805cd28433c414c")},{'$addToSet':{'usersLike':'1'}})

Filter sub-document array using substring as criteria

My collection:
{
title: 'Computers',
maincategories:[
{
title: 'Monitors',
subcategories:[
{
title: '24 inch',
code: 'AFG'
}
]
}
]
}
I want query the code. The code is just the first part so I want to have all subcategories that contains the given search. So AFG101 would return this subcategories.
My query:
module.exports = (req, res) => {
var q = {
'maincategories.subcategories': {
$elemMatch: {
code: 'AFG101'
}
}
};
var query = mongoose.model('TypeCategory').find(q, {'maincategories.$': 1, 'title': 1});
query.exec((err, docs) => {
res.status(200).send(docs);
});
};
My problem:
How do I search for a part of a string? AFG101 should return all subcategories with property code containing any part of the string. So in this case, AFG would be a hit. Same as in this sql question: MySQL: What is a reverse version of LIKE?
How do I project the subcategories. Current query returns all subcategories. I only want to returns those hitting.
The best way to do this is in MongoDB 3.4 using the $indexOfCP string aggregation operator.
let code = "afg101";
db.collection.aggregate([
{ "$project": {
"title": 1,
"maincategories": {
"$map": {
"input": "$maincategories",
"as": "mc",
"in": {
"$filter": {
"input": "$$mc.subcategories",
"as": "subcat",
"cond": {
"$gt": [
{
"$indexOfCP": [
code,
{ "$toLower": "$$subcat.code" }
]
},
-1
]
}
}
}
}
}
}}
])
which returns:
{
"_id" : ObjectId("582cba57e6f570d40d77b3a8"),
"title" : "Computers",
"maincategories" : [
[
{
"title" : "24 inch",
"code" : "AFG"
}
]
]
}
You can read my other answers to similar question 1, 2 and 3.
From 3.2 backward, the only way to do this is with mapReduce.
db.collection.mapReduce(
function() {
var code = 'AFG101';
var maincategories = this.maincategories.map(function(sdoc) {
return {
"title": sdoc.title,
"subcategories": sdoc.subcategories.filter(function(scat) {
return code.indexOf(scat.code) != -1;
}
)};
});
emit(this._id, maincategories);
},
function(key, value) {},
{ "out": { "inline": 1 }
})
which yields something like this:
{
"results" : [
{
"_id" : ObjectId("582c9a1aa358615b6352c45a"),
"value" : [
{
"title" : "Monitors",
"subcategories" : [
{
"title" : "24 inch",
"code" : "AFG"
}
]
}
]
}
],
"timeMillis" : 15,
"counts" : {
"input" : 1,
"emit" : 1,
"reduce" : 0,
"output" : 1
},
"ok" : 1
}
Well, just like your question has two parts, I could think of two separate solution, however I don't see a way to join them together.
For first part $where can be used to do a reverse regex, but it's dirty, it's an overkill and it can't use any indexes, since $where runs on each documents.
db.TypeCategory.find({$where:function(){for(var i in this.maincategories)
{for(var j in this.maincategories[i].subcategories)
{if("AFG101".indexOf(this.maincategories[i].subcategories[j].code)>=0)
{return true}}}}},{"maincategories.subcategories.code":1})
Even if you use this option, it would need couple of boundary check and it cannot project two level of nested array. MongoDB doesn't support such projection (yet).
For that purpose we might go for aggregation
db.TypeCategory.aggregate([{$unwind:"$maincategories"},
{$unwind:"$maincategories.subcategories"},
{$match:{"maincategories.subcategories.code":"AFG"}},
{$group:{_id:"$_id","maincategories":{$push:"$maincategories"}}}
])
However I don't think there is a way to do reverse regex check in aggregation, but I might be wrong too. Also this aggregation is costly since there are two unwinds which can lead to overflow the memory limit for aggregation for a really large collection.
You can use $substr and do it
db.getCollection('cat').aggregate([
{"$unwind" : "$maincategories"},
{"$unwind" : "$maincategories.subcategories"},
{"$project" :
{"maincategories" : 1,
"title":1,"sub" : {"$substr" :["$maincategories.subcategories.code",0,3]}}},
{"$match" : {"sub" : "AFG"}},
{"$project" :
{"maincategories" : 1,
"title":1}
}
])

MongoDB unwind output, grabbing one string

"_id": "Long_ID_Stuff"
"GFV" : "user001"
"hf": "NA"
"h" : {
"totalSamples" : 16,
"hist" : [
["US",16]]
"newEvent" :[
["US", NumberLong("654654654654")]
]
}
I am trying to pull out just the "US" portion of this document in a query and so far it has been giving me nothing.
My query thus far is:
db.x_collection.aggregate([{$unwind :"$h.hist"},{$match : { m:"TOP_COUNTRIES"}},{$match: {"h.lastUpdate":{$gt:1446336000000}}},{$match: {"h.hist":"US"}}]).pretty()
Do I need to do a $unwind: $h, then $unwind: $h.hist?
Without a little more information this is my best guess at what you are looking for. Given that the collection you are aggregating is the "h" collection.
db.x_collection.aggregate([
{ $unwind : "$h.hist"},
{ $match :
{ h.hist : "US" },
{ lastUpdate: { $gt:1446336000000 }},
{ m: "TOP_COUNTRIES" }
}
]).pretty();
if "h" is an array you will need to add this:
{ $unwind : $h },