I'm using metabase and I have native mongodb query. I want to filter documents created yesterday. The problem is that I only have json. Is there any way to compute yesterday?
my json is:
[...
{
"$match": {
"createdAt": { "$ne": "$yesterday" },
}
},
...]
Unfortunately Metabase does not allow to use Date() to get now Date. also you should notice that dateFromString is available in version 3.6 of mongoDB and newer but another problem here is i think dateFromString does not work well with now Date in aggregate and mongoDB return an error that you should pass a string to convert to Date in dateFromString!
so i suggest to you to get yesterday Date in metabase write code something like this :
[{
"$project": {
"user": 1,
"createdAt": 1,
"yesterday": {
"$subtract": [ISODate(), 86400000]
}
}
},
{
"$project": {
"user": 1,
"yesterday": 1,
"createdAt": 1,
"dateComp": { "$cmp": ["$yesterday", "$createdAt"] }
}
},
{
"$match": {
"dateComp": -1
}}]
In Metabase, you can actually use dates relative to today, such as yesterday, by using Date(). This is a workaround currently implemented in the mongo driver for metabase. This workaround internally handles the date as a String, so we have to parse it before trying to use yesterday as a Date.
[
{
"$project": {
"_id": 1,
"todayHandledAsString": Date(),
"todayHandledAsDate": {"$dateFromString": {
"dateString": Date()
}}
}
},
{
"$project": {
"_id": 1,
"todayHandledAsString": 1,
"todayHandledAsDate": 1,
"yesterday": {
"$subtract": ["$todayHandledAsDate", 86400000]
}
}
}
]
Unfortunately, Metabase also disallows operations or comments, so we have to use a hardcoded value of 86400000 instead of 24 * 60 * 60 * 1000, which is a single day in millis.
pay_status is a field of current table. After map them,then you can choose.
[
{$match:{{pay_status}}},
{$match:{{pay_time}}},
{$project:{...}}
]
Related
I have a mongoDB that contains documents like this:
The data types of start_local_datetime and last_update_local_datetime are both Date.
How can I find the documents whose difference between last_update_local_datetime and start_local_datetime is larger than 10 days?
I mean I want to query data like this:
start_local_datetime: 2019-08-23T10:17:42.000+00:00
terminate_local_datetime: 2019-09-19T10:17:42.000+00:00
Documents like this aren't something that I want.
start_local_datetime: 2019-08-23T10:17:42.000+00:00
terminate_local_datetime: 2019-08-25T10:17:42.000+00:00
Because terminate_local_datetime - start_local_datetime is smaller than 10 days.
You can write an aggregation pipeline, using $dateDiff operator, like this:
db.collection.aggregate([
{
"$match": {
$expr: {
"$gt": [
{
"$dateDiff": {
"startDate": "$start_local_datetime",
"unit": "day",
"endDate": "$terminate_local_datetime"
}
},
10
]
}
}
}
])
See it working here. However, this will only work in Mongo 5.0 or above, as the operator was added in that version. For other versions, this will work
db.collection.aggregate([
{
"$addFields": {
"timeDifference": {
"$divide": [
{
"$subtract": [ <-- Returns difference between two dates in milliseconds
"$terminate_local_datetime",
"$start_local_datetime"
]
},
86400000
]
}
}
},
{
"$match": {
$expr: {
"$gt": [
"$timeDifference",
10
]
}
}
},
{
"$project": {
timeDifference: 0
}
}
])
Here, we calculate the time difference manually and then compare it, with 10.
This is the playground link.
In my MongoDB collection I have documents that contain a nested string field, containing a month and year, e.g. '04/2021'. Sample document:
{
"_id": {
"$oid": "608ba45cec43c5b24cda034b"
},
"status": "pass",
"stage": 5,
"priority": 0,
"payload": {
"company_id": "8800",
"company_name": "<MY COMPANY>",
"target_period": "04/2021"
},
"retry_count": 0,
"build_number": "101",
"job_name": "P123",
"createdAt": {
"$date": "2021-04-30T06:31:56.000Z"
},
"updatedAt": {
"$date": "2021-05-10T03:55:44.686Z"
}
}
I am trying to write an aggregation pipeline that will dynamically return documents where said field points to the past month. For example, ran this month (May 2021) I would get documents labeled with '04/2021'. From this post I found the oneliner for getting the comparison string: new Date(new Date().getFullYear(), new Date().getMonth(), 1). (I understand that by the virtue of getMonth returning a zero-based index of month, getting the previous month works by accident and has to be solved somehow.)
This pipeline does not work:
[
{
$addFields: {
previous_month: {
$dateToString: {
'date': new Date(new Date().getFullYear(), new Date().getMonth(), 1),
'format': '%m/%G'
}
}
}
},
{
$match: {
"payload.target_period": "$previous_month"
}
}
]
With MongoDB Compass I can see that the field previous_month is populated just fine by the $addFields stage (above sample document gets value 04/2021), but the $match stage returns 0 documents. I'm running MongoDB version 4.2.12.
You should use $expr operator while trying to self reference another ket in a document inside
$match stage.
[
{
$addFields: {
previous_month: {
$dateToString: {
'date': new Date(new Date().getFullYear(), new Date().getMonth(), 1),
'format': '%m/%G'
}
}
}
},
{
$match: {
$expr: {
$eq: [ "$payload.target_period", "$previous_month" ],
},
}
}
]
Instead of doing whole process in query, I think you can prepare input date in your client language, (js, nodejs) easily,
have prepared a function zeroFill it will return number with concat 0 if its less than 10,
get previous month date and pick previous month
concat both month and year
function zeroFill(i) { return (i < 10 ? '0' : '') + i; }
var date = new Date();
date.setMonth(date.getMonth() - 1);
let searchDate = zeroFill(date.getMonth() + 1) + "/" + date.getFullYear();
console.log(searchDate); // mm/yyyy
Your query would be just:
[{ $match: { "payload.target_period": searchDate } }]
Playground
I would suggest moment.js library, it is much simpler to use:
{ $match: { "payload.target_period": moment().startOf("months").subtract(1, "months").format("MM/YYYY") } }
I have db schema that has string date format("date":"2020-09-01 16:07:45").
I need to search between given date range, I know this is possible if we're using ISO date format but I'm not sure if we can query with date format being string.
I have tried the following query, it doesn't seem to show accurate results.
db.daily_report_latest.find({"date":{$gte: "2021-01-01 00:00:00", $lte:"2021-03-01 00:00:00"}},{"date":1})
Is there any alternative to this? Appreciate your help.
You're right, you can't query a date field with a string, but you can just cast it to date type like so:
Mongo Shell:
db.daily_report_latest.find({
"date": {$gte: ISODate("2021-01-01T00:00:00Z"), $lte: ISODate("2021-03-01T00:00:00Z")}
}, {"date": 1});
For nodejs:
db.daily_report_latest.find({
"date": {$gte: new Date("2021-01-01 00:00:00"), $lte: new Date("2021-03-01 00:00:00")}
}, {"date": 1});
For any other language just check what the mongo driver date type is and do the same.
Note that the mongo shell isn't able to parse the string input in the format you provided, you should read here about the supported formats and transform your string pre-query like I did.
Another thing to consider for the nodejs usecase is timezones, the string will be parsed as the machine current timezone so again you need to adjust to that.
You can use $dateFromString feature of aggregation. (Documentation)
pipeline = []
pipeline.append({"$project": {document: "$$ROOT", "new_date" : { "$dateFromString": { "dateString": '$date', "timezone": 'America/New_York' }}}})
pipeline.append({"$match":{"new_date": {"$gte": ISODate("2021-01-01 00:00:00"), "$lte":ISODate("2021-03-01 00:00:00")}}})
data = db.daily_report_latest.aggregate(pipeline=pipeline)
So in the both the solutions, first typecast the date field in DB to date and then compare it with your input date range.
SOLUTION #1: For MongoDB Version >= 4.0 using $toDate.
db.daily_report_latest.find(
{
$expr: {
$and: [
{ $gte: [{ $toDate: "$date" }, new Date("2021-01-01 00:00:00")] },
{ $lte: [{ $toDate: "$date" }, new Date("2021-03-01 00:00:00")] }
]
}
},
{ "date": 1 }
)
SOLUTION #2: For MongoDb version >= 3.6 using $dateFromString.
db.daily_report_latest.find(
{
$expr: {
$and: [
{ $gte: [{ $dateFromString: { dateString: "$date" }}, new Date("2021-01-01 00:00:00")] },
{ $lte: [{ $dateFromString: { dateString: "$date" }}, new Date("2021-03-01 00:00:00")] }
]
}
},
{ "date": 1 }
)
In mongodb collection i have field called event_startdate Which is in following format
event_startdate:2018-10-14 16:00:00.000 (type:date)
i am storing date and time together is this field and want to serach event which is happening today and using query given bellow
var date= moment().format('YYYY-MM-D')
Event.find({ event_startdate:date}).exec()
Since time also attached to event_startdate i am unable to fetch today's detail.
is there a way to find this?
var date= moment().format('YYYY-MM-D') would give you a string which then you are trying to compare to an ISO date in mongo.
Try something among these lines:
db.getCollection('COLNAME').aggregate([
{
"$match": {
"$expr": {
$eq: [
"2018-10-10", // <--- You string date goes here
{
"$dateToString": {
"date": "$date",
"format": "%Y-%m-%d"
}
}
]
}
}
}
])
This is using the $expr pipeline operator with dateToString.
You can also do using find.
db.getCollection('loginDetail').find({
"$expr": {
$eq:
[ "2018-09-21", { "$dateToString": { "date": "$eventDate", "format": "%Y-%m-%d"
}
}
]
}
})
From a MongoDB collection storing data with time stamps I need to return a single record for each hour.
So far I have selected the set of records between two dates successfully, but I cant figure how to build the hourly record I need in the $group clause.
var myName = "CollectionName"
//schema for mongoose
var mySchema = new Schema({
dt: Date,
value: Number
});
var myDB = mongoose.createConnection('mongodb://localhost:27017/MYDB');
myDBObj = myDB.model(myName, evalSchema, myName);
The match in this aggregate call works fine, and the $hour creates a record for each hour in the day.. but I don't know how to recreate the a full date and get an error "unknown group operator $avg" ...
myDBObj.aggregate([
{
$match: { "dt": { $gt: new Date("October 13, 2010 12:00:00"), $lt: new Date("November 13, 2010 12:00:00") } }
},{
$group: {
"_id": { "dt": { "$hour": "$dt" } , "price": { "$avg": "$price" }}
}], function (err, data) { if (err) { return next(err); } res.json(data); });
I think I need to use $dayOfYear so there is different records for each hour of each day, and include a new Date() somewhere ...
Can someone help me do this correctly? any help is appreciated.
The $group pipeline stage works by "grouping" all data by the "key" specified for _id. Other fields you are actually aggregating are separate from the _id value and are their own field properties.
So your $group becomes this instead:
{ "$group": {
"_id": { "$hour": "$dt" },
"price": { "$avg": "$price" }
}}
Or if you want that broken by day then make a compound key:
{ "$group": {
"_id": {
"day": { "$dayOfYear": "$dt" },
"hour": { "$hour": "$dt" }
},
"price": { "$avg": "$price" }
}}
Or just use date math to produce Date objects rounded by hour:
{ "$group": {
"_id": {
"$add": [
{ "$subtract": [
{ "$subtract": [ "$dt", new Date(0) ] },
{ "$mod": [
{ "$subtract": [ "$dt", new Date(0) ] },
1000 * 60 *60
]}
]},
new Date(0)
]
},
"price": { "$avg": "$price" }
}}
Where subrtacting another date object (epoch date) from another prodces a numeric value you can round ( 1000 milliseconds, 60 seconds, 60 minutes = 1 hour ) with the applied math, and adding a number to a date object produces a date corresponding to that value.
So your problem was you had everything in the _id, where the $avg accumulator is not recognised. All accumulators need to be specified outside of the grouping key. That is the intent.
If you want to make an accumulator value part of a grouping key ( does not seem relevant here though ), you instead follow with another group stage, referencing the field that was produced from the former.