Mongodb how to find how much days ago from a timestamp field - mongodb

I was trying to find the number of days ago using the timestamp but i dont know how to do that ?
{
"_id" : ObjectId("5504cc9ddd5af617caae30b3"),
"session_id" : 1,
"Timestamp" : "2014-04-07T10:51:09.277Z",
"Item_ID" : 214536502,
"Category" : 0
}
How I can calculate the number of days ago using the field "Timestamp" ?

You may use aggregate, $project with new Date() on the Timestamp field, then do the calculation, something like this:
pipe = {
"$project" : {
"_id" : 1,
"daySince" : {
"$divide" : [
{
"$subtract" : [
new Date(),
new Date("$Timestamp")
]
},
86400000
]
}
}
}
To calculate:
db.collection.aggregate(pipeline=pipe)
Since Timestamp isn't a ISODate object, you just need to convert it to one, then subtract to current date, and divide the result by 60*60*24*1000, then it will be the number of days since today.
You can also change the new Date() to what you need to be compared.
Updated:
Since I believe the Timestamp format might be malformed, alternatively you may use mapReduce functions to calculate this:
// in your mongo shell using the db
var mapTimestamp = function() {
daySince = parseInt(new Date() - new Date(this.Timestamp) / 86400000);
emit(this._id, daySince);
}
// since you map reduce only on one field, there's really no need for this
var reduceTimestamp = function (key, value) { return value; }
db.collection.mapReduce(mapTimestamp, reduceTimestamp, {out: "sample"})
To show the results:
db.sample.find()

Related

MongoDB TTL/ExpireAfterSeconds is misbehaving and not deleting all data after given time

1) We have put expireAfterSeconds=15 on column of type: date
[
{
"v" : 1,
"key" : {
"_ts" : -1
},
"name" : "AnjaliIndex",
"ns" : "test.sessions",
"expireAfterSeconds" : 15
}
]
It is working fine on yesterdays date but is not working fine on todays date i.e it is removing data when i change document date from current date to yesterdays date where it should delete all data. (Current date which i given is even not future time but previous time)
Why is this happening? Is there any particular cycle or time when mongodb engine collect documents for expiry?
(I have seen related question but in that question use case is different where he was giving future date)
Mongo DB Version: 3.2.22
Sample Document:(not gettinkg deleted)
{
"_id" : ObjectId("5dde452818c87122389bbc09"),
"authorization" : "a0ce0b43-194d-4402-99cb-b660b3365757",
"userNumber" : "gourav#gmail.com",
"_ts" : ISODate("2019-11-27T13:43:04.776Z")
}
I will try to answer and see if that can help you.
db.my_collection.createIndex( { "createdAt": 1 }, { expireAfterSeconds: 3600 } )
After that, every document that you insert in this collection must have the "createdAt" with the current date:
db.myCollection.insert( {
"createdAt": new Date(), // This can be set in UTC
"dataExample": 2,
"Message": "#### My data ####"
} )

How to group dates in mongoDB by first or second half of the month (fortnights)

With the following data structure, using mongoDB's (v3.4) aggregation framework how do you group information every 15 days?
{
"_id" : ObjectId("5cb10a201e20af7503305fea"),
"user" : ObjectId("5b21240c4e71161fdd40b27c"),
"version" : NumberLong(2),
"value" : 42,
"itemRef" : ObjectId("5cb10a201e20af7503305fe9"),
"status" : "ACCEPTED",
"date" : ISODate("2019-04-13T11:00:00.466Z")
}
the required output would be:
[date: 2019/01/01, totalValue:15],
[date: 2019/01/16, totalValue:5],
[date: 2019/02/01, totalValue:25],
[date: 2019/02/16, totalValue:30]
The way I found to resolve this problem with mongoDB 3.4 was using $cond + $dayOfMonth to define in which part of the month this date is.
db.contract.aggregate(
[
{$match:{...queryGoesHere...}},
{$project:
{dateText:
{$cond:
[
{$lte:[{$dayOfMonth:$date},15]},
['$dateToString': ['format': '%Y-%m-01', 'date': '$date']],
['$dateToString': ['format': '%Y-%m-16', 'date': '$date']]
]
}
value:'$value'
}
},
{$group:
{
_id:'$dateText',
total:{'$sum':1}
}
}
]
The solution is in the projection of the "dateText", it first uses the $cond to determine if the date is in the first or second part of the month. It determines this using the '$dayOfMonth' which returs the day in the month. If it is less or equal to 15, it uses the '$dateToString' to format the date by year-month-01 else it formats it to year-month-16.
Hope this can help someone in the future.

How to find all date-ranges overlapping another date-range in MongoDB?

everyone!
I have a record of this type:
{
start: 2015-03-27T15:00:00.000Z,
end: 2015-03-27T17:00:00.000Z
}
and trying to find in the database crossing periods.
{
start: 2015-03-27T15:30:00.000Z,
end: 2015-03-27T16:00:00.000Z
}
I do scheduling system. And I was not there for some time occupied.
I believe you are trying to find documents with overlapping date ranges. In other words any document whose start or end dates falls in between given date range.
You can definitely achieve this with little bit match and logic.
Let's assume I have two documents in my collection
{
"_id" : ObjectId("56f692730c96eddb0a2c287e"),
"start" : "2015-03-27T15:00:00.000Z",
"end" : "2015-03-27T17:00:00.000Z"
}
{
"_id" : ObjectId("56f6928c0c96eddb0a2c287f"),
"start" : "2015-03-27T16:00:00.000Z",
"end" : "2015-03-27T27:00:00.000Z"
}
When I execute following piece of code
var startDate = "2015-03-27T20:00:00.000Z";
var endDate = "2015-03-27T21:00:00.000Z";
var findOverlapingDates = function(startDate, endDate){
return db.collection.find({
$or: [
{$and: [
{start:{$gte: startDate}}, {start:{$lte: endDate}}
]},
{start:{$lte: startDate}, end:{$gte: startDate}}
]
});
};
printjson(findOverlapingDates(startDate, endDate).toArray());
I get
[
{
"_id" : ObjectId("56f6928c0c96eddb0a2c287f"),
"start" : "2015-03-27T16:00:00.000Z",
"end" : "2015-03-27T27:00:00.000Z"
}
]
Which is overlapping document for given date range. Hope it all makes sense. For best performance, I'll recommend having index on both start and end fields.

Get documents which expired before today in MongoDB

I am using MongoDB 2.6 and I'm making a query that filter the documents by expired date. I want to know which documents expired before today. My persisted data represent publications.
I've made some queries but it doesn't return documents. I know there are many documents who satisfy this condition.
I tried two query but no one works:
1.
{
"domain.ApplicationCase.fields.ExpireDate": { $gte : {
$currentDate: {
lastModified: true,
"cancellation.date": { $type: "timestamp" }
}}}
}
2.
{
"domain.ApplicationCase.fields.ExpireDate": { $gte : new Date()}
}
Part of document records:
{
"_id" : "1234546",
"fields" : {
"Orchestration" : "default",
"Segmentation" : "PFI",
"MatchKey" : "1",
"UserID" : "001"
},
"domain" : {
"ApplicationCase" : {
"_id" : null,
"fields" : {
"ExpireDate" : "2015-11-13T13:47:26Z",
....
What's wrong?
If you want to get records which are expired before today then you should use $lte instead of $gte:
db.myCollection.find({
"domain.ApplicationCase.fields.ExpireDate": { $lte : new Date()}
})
Update:
So the core problem with your documents is that, the value of domain.ApplicationCase.fields.ExpireDate are not in the date format instead they are in simple String format.
So you first need to convert them to the date in order for the query to work since you are comparing an String with the Date.
Probably, you can use a code like this to convert the string to the date:
db.myCollection.find({
"domain.ApplicationCase.fields.ExpireDate": {$exists: true}
}).snapshot().forEach(function(record) {
var stringValue = record.domain.ApplicationCase.fields.ExpireDate;
db.myCollection.update({_id: record._id}, {$set: {
"domain.ApplicationCase.fields.ExpireDate": ISODate(stringValue)
}});
})

Find closest date in one query

I'm currently trying to figure out a way to find the closest date of a entry in mongoDB to the on i'm looking for.
Currently i solved the problem by using 2 queries. One using $gte and limit(1) to look for the next larger date and then $lte - limit(1) to see if there is a closer on that might be lower.
I was wondering, if there might be a way to find the closest date in just one query, but was not able to find anything on that matter.
Hope you can help me with this, or at least tell me for sure that this is the only way to do so.
db.collection.find({"time":{$gte: isoDate}}).sort({"time":1}).limit(1)
db.collection.find({"time":{$lte: isoDate}}).sort({"time":-1}).limit(1)
But I am looking for a way to do this in one query so i dont have to subtract the results to find the closest one.
I solved a similar problem using an aggregation.
Sample data:
{
"_id" : ObjectId("5e365a1655c3f0bea76632a0"),
"time" : ISODate("2020-02-01T00:00:00Z"),
"description" : "record 1"
}
{
"_id" : ObjectId("5e365a1655c3f0bea76632a1"),
"time" : ISODate("2020-02-01T00:05:00Z"),
"description" : "record 2"
}
{
"_id" : ObjectId("5e365a1655c3f0bea76632a2"),
"time" : ISODate("2020-02-01T00:10:00Z"),
"description" : "record 3"
}
{
"_id" : ObjectId("5e365a1655c3f0bea76632a3"),
"time" : ISODate("2020-02-01T00:15:00Z"),
"description" : "record 4"
}
{
"_id" : ObjectId("5e365a1655c3f0bea76632a4"),
"time" : ISODate("2020-02-01T00:20:00Z"),
"description" : "record 5"
}
{
"_id" : ObjectId("5e365a1655c3f0bea76632a5"),
"time" : ISODate("2020-02-01T00:25:00Z"),
"description" : "record 6"
}
And I'm looking for the record nearest to ISODate('2020-02-01T00:18:00.000Z').
db.test_collection.aggregate([
{
$match:
{
time:
{
$gte: ISODate('2020-02-01T00:13:00.000Z'),
$lte: ISODate('2020-02-01T00:23:00.000Z')
}
}
},
{
$project:
{
time: 1,
description: 1,
time_dist: {$abs: [{$subtract: ["$time", ISODate('2020-02-01T00:18:00.000Z')]}]}}
},
{
$sort: {time_dist: 1}
},
{
$limit: 1
}])
The $match stage sets up a "time window". I used 5 minutes for this example.
The $project stage adds a time distance field. This is the time in milliseconds each record is from the query time of ISODate('2020-02-01T00:18:00.000Z').
Then I sorted on the time_dist field and limit the results to 1 to return the record with time closest to ISODate('2020-02-01T00:18:00.000Z').
The result of the aggregation:
{
"_id" : ObjectId("5e365a1655c3f0bea76632a4"),
"time" : ISODate("2020-02-01T00:20:00Z"),
"description" : "record 5",
"time_dist" : NumberLong(120000)
}
check this one
db.collection.find({"time":{$gte: isoDate,$lt: isoDate}}).sort({"time":1}).limit(1)
Please use the same format what mongodb support like following
ISODate("2015-10-26T00:00:00.000Z")
In Pymongo, I used the following function. The idea is to take a datetime object, subtract some days from it and add some days to it, then find a date between those two dates. If there are no such records, increase the date span:
import datetime, dateutil
def date_query(table, date, variance=1):
'''Run a date query using closest available date'''
try:
date_a = date - dateutil.relativedelta.relativedelta(days=variance)
date_b = date + dateutil.relativedelta.relativedelta(days=variance)
result = db[table].find({'date': {'$gte': date_a, '$lt': date_b}}).sort([('date', 1)])
result = list(result)
assert len(result) >= 1
return result[len(result)//2] # return the result closest to the center
except:
return date_query(table, date, variance=variance*2)
accourding to https://stackoverflow.com/a/33351918/4885936
don't need ISODate
simple easy solution is:
if you want 1 hour left to due date just simply :
const tasks = await task.find({
time: {
$gt: Date.now(),
$lt: Date.now() + 3600000 // one hour to miliseconds
}
})
this code get tasks from now to upcoming one hour later.