I have a mongodb database with a data structure like this:
{
"_id" : "123456":,
"Dates" : ["2014-02-05 09:00:15 PM", "2014-02-06 09:00:15 PM", "2014-02-07 09:00:15 PM"],
}
There are other stuff too but what I would like to do is pull data where Dates{$slice: -1} is older than 3 days. I have a query like this but it doesnt work:
db.posts.find({}, {RecoveryPoints:{$slice: -1} $lt: ISODate("2014-02-09 00:00:00 AM")})
I am beginning to think is not possible but I figured I would come here first. My guess is that I'll have to do this logic within my code.
Edit: I wanted to mention too that my goal is to return the entire record and not just the date that mathes it
Your query has all the elements on the projection side and not the query side. Since you do not want to change the appearance of the document, then thee is no need to project.
This is actually a simple comparison as if you are always looking at the first element then you can use the dot "." notation form to look for it:
> db.dates.find({ "Dates.0": {$lt: ISODate("2014-02-09 00:00:00 AM") } }).pretty()
{
"_id" : "123456",
"Dates" : [
ISODate("2014-02-05T09:00:15Z"),
ISODate("2014-02-06T09:00:15Z"),
ISODate("2014-02-07T09:00:15Z")
]
}
> db.dates.find({ "Dates.0": {$lt: ISODate("2014-02-05 00:00:00 AM") } }).pretty()
>
So the dot "." notation works at index value in the case of arrays, and to get other elements just change the position:
> db.dates.find({ "Dates.1": {$lt: ISODate("2014-02-07 00:00:00 AM") } }).pretty()
{
"_id" : "123456",
"Dates" : [
ISODate("2014-02-05T09:00:15Z"),
ISODate("2014-02-06T09:00:15Z"),
ISODate("2014-02-07T09:00:15Z")
]
}
> db.dates.find({ "Dates.3": {$lt: ISODate("2014-02-07 00:00:00 AM") } }).pretty()
>
> db.dates.find({ "Dates.2": {$lt: ISODate("2014-02-08 00:00:00 AM") } }).pretty()
{
"_id" : "123456",
"Dates" : [
ISODate("2014-02-05T09:00:15Z"),
ISODate("2014-02-06T09:00:15Z"),
ISODate("2014-02-07T09:00:15Z")
]
}
Note that this only makes sense if you are sorting the array on changes and can otherwise be sure that the first (or position you are querying ) element is going to be the one you want to compare. If you need to otherwise test against all elements, then use the $elemMatch operator in the query instead.
The only reason you wouldn't get the result you want with this query is where your dates are strings. If so, change them, and change whatever code is causing them to be saved that way.
Related
I am trying to query my database on the date_recorded field, but the query give zero results. I'm using the MongoShell and have also tried it in Compass.
I've tried variations of $eq with ISODate and Date:
db.my_collection.find({ "date_recorded": { "$eq": new ISODate("2017-06-09T01:27:33.967Z") }}).count()
I have also tried variations of $gte with ISODate and Date:
db.my_collection.find({ "date_recorded": { "$gte": new Date("2017-06-09T01:27:33.967Z") }}).count()
Record is in the db, notice the highlighted field
Like #JohnnyHK said, the date_recorded is a string. To make a comparison, we either need to convert date_recorded to a date or compare is with a string.
The following queries can get us the expected output:
db.my_collection.find({
"date_recorded": {
$eq:"2017-06-09T01:27:33.967Z"
}
}).count()
db.my_collection.find({
$expr:{
$eq:[
{
$toDate:"$date_recorded"
},
new ISODate("2017-06-09T01:27:33.967Z")
]
}
}).count()
I have data that looks like this:
[
{
"start_time" : ISODate("2017-08-22T19:43:41.442Z"),
"end_time" : ISODate("2017-08-22T19:44:22.150Z")
},
{
"start_time" : ISODate("2017-08-22T19:44:08.344Z"),
"end_time" : ISODate("2017-08-22T19:46:25.500Z")
}
]
Is there any way to run an aggregation query that will give me a frequency result like:
{
ISODate("2017-08-22T19:43:00.000Z"): 1,
ISODate("2017-08-22T19:44:00.000Z"): 2,
ISODate("2017-08-22T19:45:00.000Z"): 1,
ISODate("2017-08-22T19:46:00.000Z"): 1
}
Essentially I want to group by minute, with a sum, but the trick is that each record might count toward multiple groups. Additionally, as in the case with 19:45, the date is not explicitly mentioned in the data (it is calculated as being between two other dates).
At first I thought I could do this with a function like $minute. I could group by minute and check to see if the data fits in that range. However, I'm stuck on how that would be accomplished, if that's possible at all. I'm not sure how to turn a single entry into multiple date groups.
You can use below aggregation in 3.6 mongo version.
db.col.aggregate([
{"$addFields":{"date":["$start_time","$end_time"]}},
{"$unwind":"$date"},
{"$group":{
"_id":{
"year":{"$year":"$date"},
"month":{"$month":"$date"},
"day":{"$dayOfMonth":"$date"},
"hour":{"$hour":"$date"},
"minute":{"$minute":"$date"}
},
"count":{"$sum":1}
}},
{"$addFields":{
"_id":{
"$dateFromParts":{
"year":"$_id.year",
"month":"$_id.month",
"day":"$_id.day",
"minute":"$_id.minute"
}
}
}}
])
I have a DB on MongoDB, with a collection like this:
{
"_id" : ObjectId("59a64b4cfb80146432aff6ac"),
"name": "Mid Range",
"end" : NumberLong("50000"),
"start" : NumberLong("10000"),
},
{
"_id" : ObjectId("59a64b4cfb80146432aff6ac"),
"name": "Hi Range",
"end" : NumberLong("100000"),
"start" : NumberLong("150000"),
}
The user enters a number to validate: 125000, i need a query to get: "Hi Range" document.
How can i do that? I'm trying to avoid code side, but if there is no choice it's ok.
Thanks!
You could even make do without the $and. You just need to set the query to find a result such that your number is less than or equal to its end and greater than or equal to its start. For example:
db.collection.find({start: {$lte: 125000}, end: {$gte: 125000}})
Note: Careful, if you want that range to include the start and end number, use $lte, $gte. Using $lt or $gt will not include it.
You can do it like this -
db.find({name:/Hi Range/}) // regular Expression.
To avoid logic that determine that 12500 is Hi Range you could do something like this
db.collection.find( { $and: [ { start: { $lte: 125000 } }, { end: { $gt: 12500 } } ] } )
Bear with me, this is not really my question. Just trying to get someone to understand.
Authors note:
The possible duplicate question solution allows $elemMatch to constrain because >all of the elements are an array. This is a little different.
So, in the accepted answer the main point has been brought up. This behavior is well
documented and you should not "compare 'apples'` with 'oranges'". The fields are of
different types, and while there is a workaround for this, the best solution for the real
world is don't do this.
Happy reading :)
I have a collection of documents I am trying to search, the collection contains the following:
{ "_id" : ObjectId("52faa8a695fa10cc7d2b7908"), "x" : 1 }
{ "_id" : ObjectId("52faa8ab95fa10cc7d2b7909"), "x" : 5 }
{ "_id" : ObjectId("52faa8ad95fa10cc7d2b790a"), "x" : 15 }
{ "_id" : ObjectId("52faa8b095fa10cc7d2b790b"), "x" : 25 }
{ "_id" : ObjectId("52faa8b795fa10cc7d2b790c"), "x" : [ 5, 25 ] }
So I want to find the results where x falls between the values of 10 and 20. So this is the query that seemed logical to me:
db.collection.find({ x: {$gt: 10, $lt: 20} })
But the problem is this returns two documents in the result:
{ "_id" : ObjectId("52faa8ad95fa10cc7d2b790a"), "x" : 15 }
{ "_id" : ObjectId("52faa8b795fa10cc7d2b790c"), "x" : [ 5, 25 ] }
I am not expecting to see the second result as none of the values are between 10 and 20.
Can someone explain why I do not get the result I expect? I think { "x": 15 } should be the only document returned.
So furthermore, how can I get what I expect?
This behaviour is expected and explained in mongo documentation here.
Query a Field that Contains an Array
If a field contains an array and your query has multiple conditional
operators, the field as a whole will match if either a single array
element meets the conditions or a combination of array elements
meet the conditions.
Mongo seems to be willing to play "smug", by giving back results when a combination of array elements match all conditions independently.
In our example, 5 matches the $lt:20 condition and 25 matches the $gt:10 condition. So, it's a match.
Both of the following will return the [5,25] result:
db.collection.find({ x: {$gt: 10, $lt: 20} })
db.collection.find({ $and : [{x: {$gt: 10}},{x:{ $lt: 20}} ] })
If this is user expected behaviour, opinions can vary. But it certainly is documented, and should be expected.
Edit, for Neil's sadistic yet highly educational edit to original answer, asking for a solution:
Use of the $elemMatch can make "stricter" element comparisons for arrays only.
db.collection.find({ x: { $elemMatch:{ $gt:10, $lt:20 } } })
Note: this will match both x:[11,12] and x:[11,25]
I believe when a query like this is needed, a combination on two queries is required, and the results combined. Below is a query that returns correct results for documents with x being not an array:
db.collection.find( { $where : "!Array.isArray(this.x)", x: {$gt: 10, $lt: 20} } )
But the best approach in this case is to change the type of x to always be an array, even when it only contains one element. Then, only the $elemMatch query is required to get correct results, with expected behaviour.
You can first check if the subdocument is not and array and provide a filter for the desired values:
db.collection.find(
{
$and :
[
{ $where : "!Array.isArray(this.x)" },
{ x: { $gt: 10, $lt: 20 } }
]
}
)
which returns:
{ "_id" : ObjectId("52fb4ec1cfe34ac4b9bab163"), "x" : 15 }
I am experiencing a very weird issue with MongoDB shell version: 2.4.6. It has to do with creating ISODate objects from strings. See below for a specific example.
Why does this not work.
collection.aggregate({$project: {created_at: 1, ts: {$add: new Date('created_at')}}}, {$limit: 1})
{
"result" : [
{
"_id" : ObjectId("522ff3b075e90018b2e2dfc4"),
"created_at" : "Wed Sep 11 04:38:08 +0000 2013",
"ts" : ISODate("0NaN-NaN-NaNTNaN:NaN:NaNZ")
}
],
"ok" : 1
}
But this does.
collection.aggregate({$project: {created_at: 1, ts: {$add: new Date('Wed Sep 11 04:38:08 +0000 2013')}}}, {$limit: 1})
{
"result" : [
{
"_id" : ObjectId("522ff3b075e90018b2e2dfc4"),
"created_at" : "Wed Sep 11 04:38:08 +0000 2013",
"ts" : ISODate("2013-09-11T04:38:08Z")
}
],
"ok" : 1
}
The short answer is that you're passing the string 'created_at' to the Date constructor. If you pass a malformed date string to the constructor, you get ISODate("0NaN-NaN-NaNTNaN:NaN:NaNZ") in return.
To properly create a new date you'd have to do so by passing in the contents of 'created_at'. Unfortunately, I don't know of a way to run a date constructor on a string using the aggregation framework at this time. If your collection is small enough, you could do this in the client by iterating over your collection and adding a new date field to each document.
Kindof similar problem:
I had documents in MongoDB with NO dates being set in, and the DB is filling up so ultimately I need to go in and delete items that are older than one year.
I did sortof have date in a crappy human readable string format. which I can grab from '$ParentMasterItem.Name';
Example: '20211109 VendorName ProductType Pricing Update Workflow'.
So here's my attempt to pull out the dates (via substring parsing) -- (thankfully I do happen to know that every one of the 100K documents has it set the same way)
db.MyCollectionName.aggregate({$project: {
created_at: 1,
ts: {$add: {
$dateFromString: {
dateString: {
/* 0123 (year)
45 (month)
67 (day)
# '20211109 blahblah string'*/
$concat: [ { $substr: [ "$ParentMasterItem.Name", 0, 4 ]}, "-",
{ $substr: [ "$ParentMasterItem.Name", 4, 2 ]}, "-",
{ $substr: [ "$ParentMasterItem.Name", 6, 2 ]},
'T01:00:00Z']
}}}}}}, {$limit: 10})
output:
{ _id: 8445390, ts: ISODate("2022-12-19T01:00:00.000Z") },