MongoDB sorting - mongodb

I want implement a "bump" feature for topics. Once a topic is bumped, it will have a new "bump_date" field. I want to sort it so that when there is a "bump_date" field, it will be sorted as if it was the "created" field. Here's an example of my db.topics:
{
"text" : "test 1",
"created" : "Sun Nov 20 2011 02:03:28 GMT-0800 (PST)"
},
{
"text" : "test 2",
"created" : "Sun Nov 18 2011 02:03:28 GMT-0800 (PST)"
},
{
"text" : "test 3",
"created" : "Sun Nov 17 2011 02:03:28 GMT-0800 (PST)",
"bump_date: : "Sun Nov 19 2011 02:03:28 GMT-0800 (PST)"
}
I want the sort to return in the order of "test 1", "test 3", "test 2"

Sorting in MongoDB is done like so:
db.collection.find({ ... spec ... }).sort({ key: 1 })
where 1 is ascending and -1 is descending.
In your specific example: db.topics.find().sort({ bump_date: 1 }), although it might be better to call it something like "updated_at".
You'll also definitely want to put an index on your "bump_date" field.
sorting: http://www.mongodb.org/display/DOCS/Sorting+and+Natural+Order
indexes: http://www.mongodb.org/display/DOCS/Indexes

As Brian Hicks suggested, creating an additional updated_at field is the way to go. This way, when a document is created you can have created_at and updated_at initially be the same.
{
"created_at": xxx,
"updated_at": xxx
}
If you then "bump" the updated_at field by setting it to the current time when there is a a bump event you can sort on the updated_at field to achieve the ordering you desire.

Also:
db.collection.find( { $query: {}, $orderby: { column : -1 } } )
where 1 is ascending and -1 is descending.

Currently it is not possible in mongodb to do a sort based on user defined criteria over multiple columns.eg. here the function would have been to return bump_date if it is set,else return created
Either you will have to use a server-side or client-side code as mentioned here :
Mongo complex sorting?
or if you want to stay with basic quering and sorting, you shall :
create a key bump_date equivalent to created whenever a new record is created. This will not be a data overhead, as you can expect every topic of yours to be bumped once in a while in future,hence bump_date field will eventually be added. So add it from the start itself.
Whenever the article is bumped,update the field bump_date .
Your example documents will look like this with this change :
{
"text" : "test 1",
"created" : "Sun Nov 20 2011 02:03:28 GMT-0800 (PST)",
"bump_date" : "Sun Nov 20 2011 02:03:28 GMT-0800 (PST)"
},
{
"text" : "test 2",
"created" : "Sun Nov 18 2011 02:03:28 GMT-0800 (PST)",
"bump_date" : "Sun Nov 18 2011 02:03:28 GMT-0800 (PST)"
},
{
"text" : "test 3",
"created" : "Sun Nov 17 2011 02:03:28 GMT-0800 (PST)",
"bump_date: : "Sun Nov 19 2011 02:03:28 GMT-0800 (PST)"
}
You shall ensureIndex on bump_date field. Now you can query the required data easily.
db.topics.find().sort({ bump_date: 1 })

Related

How to avoid duplicate values ? [duplicate]

This question already has answers here:
Return document in each group with max value using MongoDB
(2 answers)
Closed 4 years ago.
I am new to Mongo Db and would appreciate some help with this query.my mongodb data transection same ids number of thre based on create_date how to display each transection id first record
db.collection.aggregate([{
{"$project": {
"RESOURCE_ID": 1,
"TRANSACTION_ID":1,
"CREATE_DATE":1
}}
])
output:
RESOURCE_ID TRANSACTION_ID CREATE_DATE
1 "100-101" "0:ffff0a0a0983UY:-142" "Fri Sep 07 16:51:30IST2018"
2 "100-101" "0:ffff0a0a0983UY:-142" "Fri Sep 07 16:51:29IST2018"
3 "100-101" "0:ffff0a0a0983UY:-142" "Fri Sep 07 16:51:29IST2018"
4 "100-102" "0:ffff0a0a0983UY:-111" "Fri Sep 06 16:51:29IST2018"
5 "100-102" "0:ffff0a0a0983UY:-111" "Fri Sep 06 16:51:28IST2018"
expected output:
RESOURCE_ID TRANSACTION_ID CREATE_DATE
1 "100-101" "0:ffff0a0a0983UY:-142" "Fri Sep 07 16:51:30IST2018"
2 "100-102" "0:ffff0a0a0983UY:-111" "Fri Sep 06 16:51:29IST2018"
This is what you want:
db.collection.aggregate([{"$sort": {CREATE_DATE: 1}},{$group:{"_id": "$RESOURCE_ID", TRANSACTION_ID: {$first: "$TRANSACTION_ID"}, "CREATE_DATE": {$first: "$CREATE_DATE"}}}])
What it does - sorts all the documents by the date, in an ascending order, because we want the oldest document first (notice the sort clause).
Then, it groups the documents by the field "RESOURCE_ID" (which is marked as the new _id), and takes the $first TRANSACTION_ID and $first CREATE_DATE.
Notice that you will have to convert your timestamps to actual timestamps, in order for mongo to really understand the order of the timestamps. Otherwise mongo will sort them as strings, and this isn't what you want
An example:
> db.collection.find().pretty()
{
"_id" : ObjectId("5bd2bf353ca22147747ec212"),
"RESOURCE_ID" : "100-101",
"TRANSACTION_ID" : "0:ffff0a0a0983UY:-142",
"CREATE_DATE" : ISODate("2017-10-13T10:53:53Z")
}
{
"_id" : ObjectId("5bd2bf3c3ca22147747ec213"),
"RESOURCE_ID" : "100-101",
"TRANSACTION_ID" : "0:ffff0a0a0983UY:-142",
"CREATE_DATE" : ISODate("2017-10-14T10:53:53Z")
}
{
"_id" : ObjectId("5bd2bf3c3ca22147747ec214"),
"RESOURCE_ID" : "100-102",
"TRANSACTION_ID" : "0:ffff0a0a0983UY:-111",
"CREATE_DATE" : ISODate("2017-10-13T10:53:53Z")
}
{
"_id" : ObjectId("5bd2bf3c3ca22147747ec215"),
"RESOURCE_ID" : "100-102",
"TRANSACTION_ID" : "0:ffff0a0a0983UY:-111",
"CREATE_DATE" : ISODate("2017-10-14T10:53:53Z")
}
> db.collection.aggregate([{"$sort": {CREATE_DATE: 1}},{$group:{"_id": "$RESOURCE_ID", TRANSACTION_ID: {$first: "$TRANSACTION_ID"}, "CREATE_DATE": {$first: "$CREATE_DATE"}}}])
{ "_id" : "100-102", "TRANSACTION_ID" : "0:ffff0a0a0983UY:-111", "CREATE_DATE" : ISODate("2017-10-13T10:53:53Z")
{ "_id" : "100-101", "TRANSACTION_ID" : "0:ffff0a0a0983UY:-142", "CREATE_DATE" : ISODate("2017-10-13T10:53:53Z")
Also, worth mentioning that you should add indexes for CREATE_DATE because you are sorting by this field, and to RESOURCE_ID because mongo has to sort it in order to group by it

Date query in mongodb isn't working

Type of document
{
"_id" : ObjectId("585232c2bbdfc4243ecf2670"),
"field1" : "value1",
"date" : "Mon Dec 19 2016 14:45:17 GMT+0530 (IST)",
"field2" : "value2",
"field3" : true
}
Query used:
db.myCollection.find({"date":{"$lt":new Date()}})
I want to run this query on 12:05 AM to fetch all past records upto
yesterday 23:59:59
It seems that the value is a string and not a Date.
fields of type Date should appear like this:
"date" : ISODate("2016-12-19T14:45:17.000Z");
and not like what you're seeing.
Make sure you save a Date object into the collection, and not a string representation of it.

Querying a list in MongoDB

I have a document that looks like this:
{
"_id" : ObjectId("570fc2381d4899be8a8ec9d9"),
"statuses" : [
{
"created_at" : "Wed Apr 13 09:56:39 +0000 2016",
"id" : 7.20188946337153e+017,
"id_str" : "720188946337153024",
"text" : "RT #BCC_Assicura: #FormulaAuto la #polizza #Auto e #Moto economica BccPordenonese - #BCC #Assicurazioni #Click2go"
},
{
"created_at" : "Wed Apr 13 09:40:13 +0000 2016",
"id" : 7.20184809658708e+017,
"id_str" : "720184809658707970",
"text" : "Auto e moto storiche, vademecum su assicurazione e bollo - \n#autostoriche #bollo #RCauto #ASI #FMI"
}
]}
How do I query for all the records where the variable text contains the string "assicur"?
Thank you!
One possibility would be to use a regex;
> db.test.find({"statuses.text":{$regex: 'assicur'}})
That said, this will not be possible to index in mongodb, so it's probably best done along with other filters that cut the documents down to a small set before doing the string matching.

Emit a DateTime in the Map function of MongoDb

My map function looks like this:
map = function()
{
day = Date.UTC(this.TimeStamp.getFullYear(), this.TimeStamp.getMonth(), this.TimeStamp.getDate());
emit({day : day, store_id : this.Store_Id}, {count : 1});
}
TimeStamp is stored as date in the database, like this:
{ "TimeStamp" : "Mon Mar 01 2010 11:58:09 GMT+0000 (BST)", ...}
I need the "day" in the result collection to be stored as a date type as well, but it's stored as long (Epoch ticks) like this:
{ "_id" : { "day" : 1265414400000, "store_id" : 10}, "value" : { "count" : 7 } }
I tried changing the emit to something like this but didn't help:
emit({day : {"$date" : day},...)
Any ideas as to how to do that?
Date.utc is going to return miliseconds from epoch. So when you put your data back into the DB, you can use for example:
new Date(dateAsLong)
and it will be stored as the BSON date format.
earlier than mongo 1.7 it will show up in your hash as:
"Mon Mar 01 2010 11:58:09 GMT+0000 (BST)"
1.7+ it will appear as:
ISODate("2010-03-01T11:58:09Z")

finding mongodb entries by joda DateTime range scala/casbah

I'm stumped on casbah find. I'm trying to pull back all documents from a MongoDB between date1 and date2. Here's an example set of mongo docs:
{ "_id" : NumberLong("1285248838000"), "openTime" : "Thu Sep 23 2010 06:33:58 GMT-0700 (PDT)", "closeTime" : "Thu Sep 23 2010 06:36:15 GMT-0700 (PDT)", "timeInTrade" : "00:02:17", "direction" : "Long", "size" : 1, "outcome" : "Loss" }
{ "_id" : NumberLong("1285595711000"), "openTime" : "Mon Sep 27 2010 06:55:11 GMT-0700 (PDT)", "closeTime" : "Mon Sep 27 2010 06:57:37 GMT-0700 (PDT)", "timeInTrade" : "00:02:26", "direction" : "Short", "size" : 1, "outcome" : "Win"}
{ "_id" : NumberLong("1285594773000"), "openTime" : "Mon Sep 27 2010 06:39:33 GMT-0700 (PDT)", "closeTime" : "Mon Sep 27 2010 06:41:47 GMT-0700 (PDT)", "timeInTrade" : "00:02:14", "direction" : "Short", "size" : 1, "outcome" : "Win" }
{ "_id" : NumberLong("1286289026000"), "openTime" : "Tue Oct 05 2010 07:30:26 GMT-0700 (PDT)", "closeTime" : "Tue Oct 05 2010 07:36:23 GMT-0700 (PDT)", "timeInTrade" : "00:05:57", "direction" : "Short", "size" : 2, "outcome" : "Loss"}
So, let's say I want to pull back the documents from Sep 27. How would I go about doing that?
In the casbah documentation, it looks like I could construct a builder like this:
val dt = new DateTime("2010-09-27T00:00:00.000-08:00")
val bldr = MongoDBObject.newBuilder
bldr += "openTime" $gte dt $lt dt.plusDays(1)
val result = coll.find(bldr.result)
In my IDE (Netbeans), this will not compile because "$gte is not a member of java.lang.String". I had similar results with the other documented ways to construct my filter.
I suspect that the next problem I would have is that it doesn't know how to compare the dates because they are stored as joda DateTimes, so if anyone has experience with these problems, I would greatly appreciate some guidance.
Thanks,
John
FOLLOW-UP:
I've got a partial solution, but only because I was using the milliseconds as the _id. Here is some code that works for that case:
val begin = dt.getMillis
val end = dt.plusDays(1).getMillis
val json = "{ '_id' : { '$gte' : " + begin + " , '$lt' : " + end + "}}"
val dbObject = JSON.parse(json).asInstanceOf[DBObject];
for (x <- coll.find(dbObject)) println(x)
I'm still interested in learning about a solution that works on DateTime instead of the Long millis...
opentime is stored as a string on the Mongo side. Your $gte function won't work b/c string comparison won't work.
To make this work, you'll have to use a $where clause and a function that performs the comparison correctly. So you'll basically have to write a javascript function that correctly interprets the JODA time. You'll then have to include that function with your DB call or you'll have to store it server-side and proceed from there.
Here are some details on the where clause. Here are some details on server-side code execution.