So, I need to extract the day-of-week for some objects to make some aggregations. But all my documents have is a timestamp, not a Date. So I'm trying to use $dayOfMonth (and others) with an expression, I can't figure out what it is not working.
Here is my query (along with a helper function to create my date from the timestamp):
db.Logging.aggregate([
{
$match: {
"timestamp": { $gte: dateToTimestamp("2017-04-10") }
}
},
{
$project: {
_id: 0,
timestamp: "$timestamp",
dia: { $dayOfMonth: myDate("$timestamp") }
}
}
])
function dateToTimestamp(str) {
let d = new Date( str );
return d.getTime() + d.getTimezoneOffset()*60*1000;
}
function myDate( ts) {
var d = new ISODate();
d.setTime( ts );
return d;
}
The problem seems to be in passing the value of $timestamp to the myDate function. If I use a literal (e.g. 1492430243) either as the value of ts inside the function or as the value of the parameter passed to myDate it works fine.
In other words: this $dayOfMonth: myDate("1492430243") works.
Although a solution has been shown to work here (Mongodb aggregation by day based on unix timestamp - a pretty ugly solution, if I may add), I want to know why my solution doesn't. As per mongodb docs, $dayOfMonth works with Date types, my function returns a date, so what is wrong?
Related
I'm trying to match the values greater than and less than a datetime parameter which holds the value for the current date time after it's been formatted with momentjs.
The datetime variable
datetime = moment.utc().format('YYYY-MM-DDTHH:mm:ssZ')
The values I'm trying to match exist in a mongo's document as seen here in the picture
The code I used for matching :
if (_params.datetime) {
_mongoParams.push({
$and: [
{
_rewards: {
_endDatetime: { $lte: new Date(datetime) }
},
},
{
_rewards: {
_startDatetime: { $gte: new Date(datetime) }
},
}
],
})
}
The problem is I'm not getting any results when I try to make requests with the datetime parameter. I tried placing the datetime variable inside an ISODate constructor but it didn't work too.
How can I possibly match the dates ?
I can find an item with this search:
db.item.find({"startdate":{$gte: 1485521569000 }})
The date seems to correspond to
> new Date(1485521569000)
ISODate("2017-01-27T12:52:49Z")
But if I search
db.item.find({"startdate":{"$gte" : ISODate("2017-01-01T00:00:00Z")}})
I don't get any results. What am I doing wrong?
PS
the only way I found is
db.item.find({"startdate":{"$gte" : (new Date("2017-01-01")).getTime()}})
is that right or there is a better way?
Below one will work.
db.getCollection('test').find({
"key": {
"$gte": (ISODate("2017-01-01")).getTime()
}
})
Reason:
You have your data is in int64 or long
In the new Date query, you are converting date to time which return int. So get performs integer comparison.
In ISODate you are passing a date, it doesn't convert date to integer i.e milliseconds. So if you convert, both will work.
new Date() returns the current date as a Date object. The mongo shell wraps the Date object with the ISODate helper
var d = ISODate("2017-01-01")
print(d); //Sun Jan 01 2017 05:30:00 GMT+0530 (IST)
Hence, the comparison fails.
If i understand, you have timestamp in startdate field,
This is the option if helps you, Can do it with aggregate() function, using $toDate,
$toDate convert timestamp to ISO date
db.collection.aggregate([
{
$match: {
$expr: {
$gte: [
{
$toDate: "$startdate"
},
ISODate("2017-01-27T12:52:49.000Z")
]
}
}
}
])
Playground: https://mongoplayground.net/p/H7SFHsgOcCu
I am getting started with the Changestream in Mongo. In my current setup a stitch functions inserts the changelog events into a revision collection. However when I read data from the collection, I can't convert the Timestamp fields. I have tried with the following 2 attempts:
1) A pipeline
[
{
$match: {
'documentKey._id': _id,
},
},
{
$sort: { _id: -1 },
},
{
$addFields: {
convertedDate: { $toDate: 'clusterTime' },
},
},
]
But it gives the error: Error parsing date string 'clusterTime'; 0: passing a time zone identifier as part of the string is not allowed 'c'; 6: Double timezone specification 'r'
2) The bson Timestamp class
import { Timestamp } from 'bson';
const asTimestampInstance = new Timestamp(v.clusterTime);
But here typescript gives me the error: Expected 2 arguments, but got 1.ts(2554)
index.d.ts(210, 30): An argument for 'high' was not provided.
In Altas, the clustertime correctly looks like a timestamp:
I hope that I am just missing something simple :)
Unfortunately $toDate doesn't work with timestamps directly. At least not in v4.0.
The argument should be either a number, a string, or an ObjectId.
You need to convert Timestamp to string first:
$addFields: {
convertedDate: { $toDate: {$dateToString:{date:"$clusterTime"}} },
},
2) The bson Timestamp class
You should take first 32-bit value from BSON's Timestamp class instance, it means seconds Epoch time, then multiply seconds on 1000 and make it milliseconds Epoch time, then call JS Date constructor.
If v is document from ChangeStream, v.clusterTime is BSON Timestamp class object. So, you should write:
const date = new Date(v.clusterTime.getHighBits() * 1000);
This example worked for me on MongoDB 4.0, ODM Mongoose 5.12, Node.js 12.
I want to obtain the records that the "FECHA_FIN" field is greater than or equal to today's date.
this is an example of my data:
but with this query:
db.getCollection('susp_programadas').find( {"FECHA_FIN":{ $gte: new Date("YYYY-mm-dd") }} )
I do not get results, what am I doing wrong? Thank you
You can convert the date to an ISO date and query that way. Since you stored the date as a string mongo has no idea how to query it against an ISO date without conversion.
If you stored your date in mongo as the default ISO date then you could have easily done this:
db.getCollection('susp_programadas').find({"FECHA_FIN":{$gte: new Date()}})
So this is how you can do it now:
db.getCollection('susp_programadas').aggregate([
{
$project: {
date: { $dateFromString: { dateString: '$FECHA_FIN' }}
}
},
{ $match: { date: { $gte: new Date() }}}
])
You can use the $dateFromString in an aggregate query with a $match to get the results you want. Note that $dateFromString is only available in MongoDB version 3.6 and up.
If there is no way to convert your data to ISODate or upgrade your DB you could also consider another solution which via $where:
db.getCollection('susp_programadas').find({
$where: function(){
return new Date(this.FECHA_FIN) > Date.now()
}
})
However that solution suffers from the fact that $where can not use indexes so have that in mind.
How can I aggregate my MongoDB results by ObjectId date. Example:
Default cursor results:
cursor = [
{'_id': ObjectId('5220b974a61ad0000746c0d0'),'content': 'Foo'},
{'_id': ObjectId('521f541d4ce02a000752763a'),'content': 'Bar'},
{'_id': ObjectId('521ef350d24a9b00077090a5'),'content': 'Baz'},
]
Projected results:
projected_cursor = [
{'2013-09-08':
{'_id': ObjectId('5220b974a61ad0000746c0d0'),'content': 'Foo'},
{'_id': ObjectId('521f541d4ce02a000752763a'),'content': 'Bar'}
},
{'2013-09-07':
{'_id': ObjectId('521ef350d24a9b00077090a5'),'content': 'Baz'}
}
]
This is what I'm currently using in PyMongo to achieve these results, but it's messy and I'd like to see how I can do it using MongoDB's aggregation framework (or even MapReduce):
cursor = db.find({}, limit=10).sort("_id", pymongo.DESCENDING)
messages = [x for x in cursor]
this_date = lambda x: x['_id'].generation_time.date()
dates = set([this_date(message) for message in messages])
dates_dict = {date: [m for m in messages if this_date(m) == date] for date in dates}
And yes, I know that the easiest way would be to simply add a new date field to each record then aggregate by that, but that's not what I want to do right now.
Thanks!
Update: There is a built in way to do this now, see https://stackoverflow.com/a/51766657/295687
There is no way to accomplish what you're asking with mongodb's
aggregation framework, because there is no aggregation operator that
can turn ObjectId's into something date-like (there is a JIRA
ticket, though). You
should be able to accomplish what you want using map-reduce, however:
// map function
function domap() {
// turn ObjectId --> ISODate
var date = this._id.getTimestamp();
// format the date however you want
var year = date.getFullYear();
var month = date.getMonth();
var day = date.getDate();
// yields date string as key, entire document as value
emit(year+"-"+month+"-"+day, this);
}
// reduce function
function doreduce(datestring, docs) {
return {"date":datestring, "docs":docs};
}
The Jira Ticket pointed out by llovett has been solved, so now you can use date operators like $isoWeek and $year to extract this information from an ObjectId.
Your aggregation would look something like this:
{
"$project":
{
"_id": {
"$dateFromParts" : {
"year": { "$year": "$_id"},
"month": { "$month": "$_id"},
"day": { "$dayOfMonth": "$_id"}
}
}
}
}
So this doesn't answer my question directly, but I did find a better way to replace all that lambda nonsense above using Python's setdefault:
d = {}
for message in messages:
key = message['_id'].generation_time.date()
d.setdefault(key,[]).append(message)
Thanks to #raymondh for the hint in is PyCon talk:
Transforming Code into Beautiful, Idiomatic Python