I have a document which looks something like this
{
"_id":ObjectId("5bcef414b4305f4054571305"),
"timestamp": ISODate("2018-10-23T10:12:36.755 Z"),
"config" : {
"expiry_duration" : NumberLong(10000)
}
}
I need to find documents which are expired ,i.e, whose $datediff(time.Now(), $timestamp) > config.expiry_duration
I am not clear if I need to use aggregate or if I can achieve this using find itself
You can do it using .find() method but you need $expr operator (which is available in MongoDB 3.6):
db.collection.find({
$expr: {
$gt: [ { $subtract: [ ISODate("2018-10-23T16:39:06.266Z"), "$timestamp" ] }, "$config.expiry_duration" ]
}
})
To get current date you can type new Date() in Mongo shell
If you need a solution for MongoDB < 3.6 you can use .aggregate() and $redact pipeline stage:
db.col.aggregate({
$redact: {
$cond: {
if: { $gt: [ { $subtract: [ ISODate("2018-10-23T16:39:06.266Z"), "$timestamp" ] }, "$config.expiry_duration" ] },
then: "$$KEEP",
else: "$$DESCEND"
}
}
})
Related
Does anyone who knows how to filter data and only keep those history_doc.data.objectId equals _id?
I have tried so many methods but none of them works
{'history_doc.data.objectId': {$eq: '$_id'}}
{'history_doc.data.objectId': {$eq: {$toString: '$_id'}}}
You can use $expr and $eq with $toObjectId into aggregation query like this:
.aggregate({
$match: {
$expr: {
$eq: [
"$_id",
{
"$toObjectId": "$history_doc.data.objectId"
}
]
}
}
})
Example here.
You can do it with Aggregation Framework:
$match - to filter documents based on some custom criteria
$eq and $toString - To check it _id and history_doc.data.objectId are the same.
db.collection.aggregate([
{
"$match": {
"$expr": {
"$eq": [
{
"$toString": "$_id"
},
"$history_doc.data.objectId"
]
}
}
}
])
Working example
I'm querying through Metabase which is connected to a Mongodb server. The field which I'm querying is nested and is a Unix timestamp. See below
{
room_data: {
"meta": {
"xxx_unrecognized": null,
"xxx_sizecache": 0,
"id": "Hke7owir4oejq3bMf",
"createdat": 1565336450838,
"updatedat": 1565336651548,
}
}
}
The query I have written is as follows
[
{
$match: {
client_id: "{{client_id}}",
"room_data.meta.createdat": {
$gt: "{{start}}",
$lt: "{{end}}",
}
}
},
{
$group: {
id: "$room_data.recipe.id",
count: {
$sum: 1
}
}
}
]
I do not get any result as the field room_data.meta.createdat is not a date (Aug 20, 2020) which I'm passing in. Here start and end are the parameters (Metabase feature) which I'm passing in the Date format. I need some help in converting those dates into unix timestamp which can then be used to filter out the results between the specific dates
If you're using Mongo version 4.0+ you can then use $toDate in you're aggregation like so:
db.collection.aggregate([
{
$match: {
$expr: {
$and: [
{
$eq: [
"$client_id",
{{client_id}}
]
},
{
$lt: [
{
$toDate: "$room_data.meta.createdat"
},
{{end}}
]
},
{
$gt: [
{
$toDate: "$room_data.meta.createdat"
},
{{start}}
]
}
]
}
}
}
])
MongoPlayground
If you're you're on an older Mongo version I recommend you either convert you're database fields to be Date type, or you convert your input into a number timestamp somehow (I'm unfamiliar with metabase).
The last option is to use $subtract as you can subtract a number from a date in Mongo, then check to see whether that date is before or after 1970-01-01T00:00:00Z. the problem with this approach is it does not consider timezones, so if your input's timezone is different than your database one or is dynamic this will be a problem you'll have to account for.
db.collection.aggregate([
{
$match: {
$expr: {
$and: [
{
$eq: [
"$client_id",
{{client_id}}
]
},
{
$gt: [
{
"$subtract": [
{{end}},
"$room_data.meta.createdat"
]
},
ISODate("1970-01-01T00:00:00.000Z")
]
},
{
$lt: [
{
"$subtract": [
{{start}},
"$room_data.meta.createdat"
]
},
ISODate("1970-01-01T00:00:00.000Z")
]
}
]
}
}
}
])
MongoPlayground
db.getCollection('name').update({"bid":"3860"},
{
[
{"$set":
{ "tl" :
{ $add :
[
{
$multiply: ["$intField", 24*60*60*1000]
},
"$dateField"
]
}
}
},
{ "$unset": {'intField' : 1}}
]
}
}
)
I am able to do $set. I am trying to do $unset also in single call.
is it possible? Mongo Version is 4.2. Any suggestions please?
This update with aggregation pipeline will remove the intField field from the document using the REMOVE aggregation system variable:
db.set.update(
{ },
[
{
$set: {
t1: { $add: [ { $multiply: ["$intField", 24*60*60*1000 ] }, "$dateField" ] },
intField: "$$REMOVE"
}
}
]
)
You are almost there. There is a small mistake that $update aggregation operator accepts either a string or an array.
db.getCollection('name').update({
"bid": "3860"
},
[
{
"$set": {
"tl": {
$add: [
{
$multiply: [
"$intField",
24*60*60*1000
]
},
"$dateField"
]
}
}
},
{
"$unset": 'int2Field' -->change here
}
]
})
It is not same as $unset update operator syntax. It is $unset aggregate operator format.
Refer
Why do I have to use this code: { $match: { $expr: { <aggregation expression> } } } to match a document using a document input variable as opposed to doing: { $match: { <query> } } ?
For example:
$lookup: {
from: "comments",
let: { myvar: '$myInputDocVariable'},
pipeline: [
{ $match:
{ $expr:
{ $and:
[
{ $eq: [ "$varFromCommentDocument", "$$myvar" ] },
]
}
}
},
],
as: "returnedValue"
}
The query above works fine but the query below does not work as expected. Why is this? Does this mean that if you are using input variables in a $lookup pipeline you have to use $expr? why is that?
$lookup: {
from: "comments",
let: { myvar: '$myInputDocVariable'},
pipeline: [
{ $match: { "$varFromCommentDocument", "$$myvar" } }
],
as: "returnedValue"
}
When you perform uncorrelated sub-queries for $lookup operator:
If you need to compare parent collection's field within pipeline, MongoDB cannot apply the standard query syntax (field:value) for variable / Aggregation expressions. In this case, you need to use $expr operator.
Example:
{ $match:
{ $expr:
{ $and:[
{ $eq: [ "$varFromCommentDocument", "$$myvar" ] },
]}
}
}
if it matches against "hard-coded" values, you don't need to use $expr operator.
Example:
$lookup: {
from: "comments",
pipeline: [
{ $match:{
"key": "value",
"key2": "value2"
}}
],
as: "returnedValue"
}
Does this mean that if you are using input variables in a $lookup
pipeline you have to use $expr
Yes correct, by default in filters i.e; in filter part of .find() or in $match aggregation stage you can't use an existing field in the document.
If at all if you need to use existing field's value in your query filter then you need to use aggregation pipeline, So in order to use aggregation pipeline in .find() or in $match you need to wrap your filter query with $expr. Same way to access local variables got created using let of $lookup filter in $match needs to be wrapped by $expr.
Let's consider below example :
Sample Docs :
[
{
"key": 1,
"value": 2
},
{
"key": 2,
"value": 4
},
{
"key": 5,
"value": 5
}
]
Query :
db.collection.find({ key: { $gt: 1 }, value: { $gt: 4 } })
Or
db.collection.aggregate([ { $match: { key: { $gt: 1 }, value: { $gt: 4 } } } ])
Test : mongoplayground
If you see the above query both input 1 & 4 are passed into query but it you check below query where you try to match key field == value field - it doesn't work :
db.collection.aggregate([ { $match: { key: { $eq: "$value" } } } ])
Test : mongoplayground
Above as you're comparing two existing fields then you can't do that as it mean you're checking for docs with key field value as string "$value". So to say it's not a string it's actually a reference to value field you need to use $eq aggregation operator rather than $eq query operator like below :
db.collection.aggregate([ { $match: { $expr: { $eq: [ "$key", "$value" ] } } } ])
Test : mongoplayground
I have collection with something similar datastructure
{
id: 1
limit: {
max: 10000,
used: 0
}
}
and I tried running the below query but it is giving 0 results
db.getCollection('promos').aggregate(
[
{ $match: { id: 1} },
{$match: { $expr: {$gt ["limit.max" , "limit.used"]}}}
])
I also used the below query
db.getCollection('promos').aggregate(
[
{ $match: { id: 1} },
{$match: { "$limit.max": {$gt: "limit.used"}}}
])
None of them is giving the result . Any help will be appreciated.
You need to prefix "field expressions" with the $. This also can be simply done in a .find()
db.getCollection('promos').find({
"id": 1,
"$expr": { "$gt": [ "$limit.max" , "$limit.used" ] }
})
Or a single $match stage if you really need to use aggregate instead:
db.getCollection('promos').aggregate([
{ "$match": {
"id": 1,
"$expr": { "$gt": [ "$limit.max" , "$limit.used" ] }
}}
])
That's how $expr works and you can "mix it" with other regular query operators in the same query or pipeline stage.
Also see $gt for general usage examples
Of course if you don't actually even have MongoDB 3.6, then you use $redact instead:
db.getCollection('promos').aggregate([
{ "$match": { "id": 1 } },
{ "$redact": {
"$cond": {
"if": { "$gt": [ "$limit.max" , "$limit.used" ] },
"then": "$$KEEP",
"else": "$$PRUNE"
}
}}
])
Or use $where. Works in all versions:
db.getCollection('promos').find({
"id": 1,
"$where": "this.limit.max > this.limit.used"
})