get mongodb records created in a specific month - mongodb

I'm trying to get a specific range of documents, based on when they were created. What I'm trying to do is something like:
/getclaims/2015-01
/getclaims/2015-02
...
that way a user can browse through all records based on the selected month.
In my database I'm not storing a created_at date, but I know mongodb stores this in the objectid.
I found that I can get records like this:
db.claims.find({
$where: function () { return Date.now() - this._id.getTimestamp() < (365 * 24 * 60 * 60 * 1000) }
})
of course that doesn't filter based on a specific month, but only within a certain time limit.
What would be a possible way of limited a query based on a specific month, using the Timestamp from the objectid's?
I'm using mongoose, but it's probably a good idea to start in mongo shell itself.

Based on the function borrowed from the answer to this question - https://stackoverflow.com/a/8753670/131809
function objectIdWithTimestamp(timestamp) {
// Convert date object to hex seconds since Unix epoch
var hexSeconds = Math.floor(timestamp/1000).toString(16);
// Create an ObjectId with that hex timestamp
return ObjectId(hexSeconds + "0000000000000000");
}
Create a start and an end date for the month you're looking for:
var start = objectIdWithTimestamp(new Date(2015, 01, 01));
var end = objectIdWithTimestamp(new Date(2015, 01, 31));
Then, run the query with $gte and $lt:
db.claims.find({_id: {$gte: start, $lt: end}});

Related

Mongodb aggregation over multiple collections

Because the large number of docs, someone decided to create everyday a new collection automatically in our Mongodb with the date as sufix with this format XXXXX_YYYY_MM_DD.
I want to use this DB as datasource in Grafana (with the JSON datasource). I have found aggregation queries that union several collections, but the problem is to dinamycally get the dates to get the collections to union.
The only dates I can get from Grafana is __from and __to dates, so i can lookup the collections of these dates, but not the whole range of dates between from and to.
Any idea how to perform it?
you can have some JS/shell script which can execute in mongo shell to get collection names:
var startDate= new Date("2021-09-01");
var endDate = new Date("2021-09-09");
var collections=[];
var startTime = startDate.getTime(), endTime = endDate.getTime();
//Then you could loop from one to another incrementing loopTime by 86400000 (1000*60*60*24) - number of milliseconds in one day:
for(let loopTime = startTime; loopTime <= endTime; loopTime += 86400000)
{
var loopDay=new Date(loopTime)
//generate collection for each day
collections.push("COLLECTIONXXXX_"+loopDay.toISOString().slice(0, 10).replace(/-/g, '_'))
}
print(JSON.stringify(collections))
Output:
["COLLECTIONXXXX_2021_09_01","COLLECTIONXXXX_2021_09_02","COLLECTIONXXXX_2021_09_03","COLLECTIONXXXX_2021_09_04","COLLECTIONXXXX_2021_09_05","COLLECTIONXXXX_2021_09_06","COLLECTIONXXXX_2021_09_07","COLLECTIONXXXX_2021_09_08","COLLECTIONXXXX_2021_09_09"]
In above example we had start date(2021_09_01) and end date (2021_09_09)and we have generated all collection names and stored in array, now you can use collection names from this collections array and run your aggregation union.

mongodb difference in time

How can i filter database entries that have a datetime less than 60min in the past?
I tried some date operations as follows in mongodb with two fields timestamp and marketstartime that are of type date in all my documents:
{"$subtract": ["$timestamp", "$marketstartime"]}
but it returns always null for that operation. Why?
My timestamp and marketstartime entries in the db are in date type and look as follows, this should be correct:
2017-12-23 12:00:00.000Z
The actual question I’m trying to solve: How can I get all entries that have a timestamp less than 60 min in the past from now?
A query can composed for documents with timestamp value set less than 60 minutes ago.
from datetime import datetime, timedelta
query = {
'$timestamp': {
'$lt': datetime.now() + timedelta(minutes=-60)
}
}
cursor = db.collection.find(query)

Best way to get documents in specific timeframe MongoDB

I am creating a script that would run after every x minutes and needs to gather data from MongoDB by timestamps.
For example, how would I match the documents with aggregation that have a timestamp in the following timeframe:
start_time = current_time - 60 min
end_time = start_time + 30 min
And I would need to get all the documents that stay within that time frame.
The MongoDB objects have ISODate timestamps on them.
Thanks!
You can create date objects in mongo shell like so:
db.getCollection('order').aggregate([
{
$match : {
start_time : {$gte : new Date(ISODate().getTime() - 1000 * 60 * 60)},
end_time : {$lte : new Date(ISODate().getTime() + 1000 * 60 * 30)}
}
}
...
])
You can use this in aggregate but also in normal find queries.
Note I wrote this without testing, so it might have syntax errors..
db.collection.find({"createdAt": { $gt: new Date('2017-04-25')},"updatedAt":{$lt:new Date('2017-06-25')}})
updatedAt and createdAt are the feilds I have taken at the time of designing the schema by timestamp. You could give feilds according to you design.
the find query would be little better than aggregate in this case as no complex functions have to be performed

Mongo DB ISO format

Based on the below query return result I want to filter the month and ther year.
For example I only want data for the month of March.
db.SBM_USER_DETAIL.aggregate([
{
$project: {
join_date: '$JOIN_DATE'
}
}
]).map(
function(d) {
d.join_date = moment(d.join_date).locale('es').tz("Asia/Kolkata").format();
return d
})
How to use the returned formatted value of join_date inside the MongoDB aggregation query?
MongoDB's ISODate is very similar to the javascript Date class. If you have a date range in the Kolkata timezone, and want to filter by that, instantiate a pair of Date objects to define the range, before running the find.
For this instance, to return all join_date values that fall within March 2017, converted to the Kolkata (UTC-07:00) timezone, filter for date greater than or equal to midnight March 1 and less than midnight April 1, then convert the results using moment:
var first = new Date("2017-03-01T00:00:00-07:00");
var last = new Date("2017-04-01T00:00:00-07:00");
db.SBM_USER_DETAIL.find(
{join_date:{$gte: first, $lt: last}}, //filter based on join_date
{join_date:1,_id:0} // only return join_date, omit this if you need all fields
).map(
function(d) {
d.join_date = moment(d.join_date).locale('es').tz("Asia/Kolkata").format();
return d;
}
);

Merge a Date object which has 04:00 as its time with another time string ("8:30am") into a new Date object

I have a mongo db with a document that has a date and a time separately.
Date object was intended to be just a date, so it has the time as
04:00 (time-zone adjusted to -4)
db.logs.findOne().date
ISODate("2014-08-05T04:00:00Z")
And there's a separate time field which is still a string
db.logs.findOne().time
8:30am
How do I merge them both into one field?
Do the combination client-side, using appropriate libraries to handle calendar and clock calculations, and then update/insert the document. MongoDB (as of 2.6, anyway) doesn't have facilities for updating to the combination server side.
db.logs.find().forEach(function(log) {
log.date = new Date(log.date.getFullYear(),
/* date */ log.date.getMonth(),
log.date.getDate(),
/* time */ parseInt(log.time)
/* + 12 if PM */ + (((log.time.charAt(log.time.length - 2) + log.time.charAt(log.time.length - 1)) === 'pm') ? 12 : 0));
db.logs.save(log);
})