I have a field called file_date, and in the start of the query I've set a new field called start_date which is 10 days back using:
{ $set: { start_date:{ $subtract: [ "$$NOW", 10*24*60*60*1000 ] } } }.
I want to use $match to find all the results where file_date is bigger then start_date.
I've tried using $gte but I can't seem to get the right syntax.
I'm using mongo 4.2 so I cant use SubstructDate
Thanks for any help
Changing the dates to strings using $toString.
using $gte
we're trying to remove documents older than 3 months in a specific collection.
There's no TTL configured on this collection and no single date/time attribute on those documents.
How can I remove those old documents anyway? is there any script I could run to make it automatically?
thanks
Assuming you did not generate your own _id field the ObjectId contains a timestamp, from the docs:
The 12-byte ObjectId value consists of: ...
a 4-byte timestamp value, representing the ObjectId's creation, measured in seconds since the Unix epoch
So if you're using Mongo version 4.0+ you could use $toDate, match the documents and overwrite the current collection using $out
db.collection.aggregate([
{
$addFields: {
shouldKeep: {
$lt: [
{
$subtract: [
"$$NOW",
{
$toDate: "$_id"
}
]
},
7776000// 90 days in seconds
]
}
}
},
{
$match: {
shouldKeep: true
}
},
{
$project: {
shouldKeep: 0
}
},
{
out: "curr_collection"
}
])
Mongo Playground
Mind you this is a POC example however this does not deal with many issues, such as timezones. exact month start and ending ( currently it calculates 90 days back ) and more.
Not mentioning that using $out on a large collection contains a lot of overhead.
My recommendation is to paginate the results and do this in code.
For nodejs for example you can use ObjectId's getTimestamp method, like so: (pseudo code)
const someDocuments = [...];
for each document:
const timestamp = document._id.getTimestamp();
if (timestamp < 3 months ago) delete document.
Now in code you can handle timezones, month start dates and scale issues relatively easily.
I'm doing MongoDB Academy, and for this question:
What is the difference between the number of people born in 1998 and the number of people born after 1998 in the sample_training.trips collection?
The simplest way is to do this is (the way they expect you to answer):
db.trips.find({"birth year":1988}).count()
and:
db.trips.find({"birth year":{$gt:1988}}).count()
then, manually calculate.
I'm not familiar with programming and code syntax but, wondering about a sophisticated method, something like the code bellow can be improved.
db.trips.aggregate({$subtract:[db.trips.find({"pop":1988}).count(),db.trips.find({"pop":{$gt:1988}}).count()]})
Note: Atlas "free subscriber" don't allow to use $subtract in queries, so I even tested if it will work.
Much Simpler way use:
(query 1) - (query 2)
db.trips.find({"birth year":{"$gt":1998}}).count() - db.trips.find({"birth year":1998}).count()
You'll get the result as an integer
6
You can group the number of people born in the year 1998 and after 1988 using the following approach that too with just one aggregate query.
Aggregate pipeline stages:
Select only those records where the birth year is equal or greater than 1988
Group the number of people by the birth year
Add a new field as born_on to every record using the $cond operator in the $project stage
Group record again by the newly added field born_on, which will now give you only two records with count as on_1988 and after_1988
The query is as following
db.trips.aggregate([
{
"$match":{
"birth year":{
"$gte":1988
}
}
},
{
"$group":{
"_id":"$birth year",
"count":{
"$sum":1
}
}
},
{
"$project":{
"count":"$count",
"born_on":{
"$cond":[
{
"$eq":[
"$_id",
1988
]
},
"on_1988",
"after_1988"
]
}
}
},
{
"$group":{
"_id":"$born_on",
"total":{
"$sum":"$count"
}
}
}
])
** There could be another more optimized way of achieving the same result but this also works
you can use the following queries to get this value:
db.trips.find( { "birth year": { "$gt": 1998 } } ).count()
db.trips.find( { "birth year": 1998 } ).count()
Use the $gt instead of $gte operator to exclude all 1998 births, and then see how many people were born in 1998 by using implicit equality, then subtract the two values to get the difference.
I want to obtain the records that the "FECHA_FIN" field is greater than or equal to today's date.
this is an example of my data:
but with this query:
db.getCollection('susp_programadas').find( {"FECHA_FIN":{ $gte: new Date("YYYY-mm-dd") }} )
I do not get results, what am I doing wrong? Thank you
You can convert the date to an ISO date and query that way. Since you stored the date as a string mongo has no idea how to query it against an ISO date without conversion.
If you stored your date in mongo as the default ISO date then you could have easily done this:
db.getCollection('susp_programadas').find({"FECHA_FIN":{$gte: new Date()}})
So this is how you can do it now:
db.getCollection('susp_programadas').aggregate([
{
$project: {
date: { $dateFromString: { dateString: '$FECHA_FIN' }}
}
},
{ $match: { date: { $gte: new Date() }}}
])
You can use the $dateFromString in an aggregate query with a $match to get the results you want. Note that $dateFromString is only available in MongoDB version 3.6 and up.
If there is no way to convert your data to ISODate or upgrade your DB you could also consider another solution which via $where:
db.getCollection('susp_programadas').find({
$where: function(){
return new Date(this.FECHA_FIN) > Date.now()
}
})
However that solution suffers from the fact that $where can not use indexes so have that in mind.
I know that ObjectIds contain the date they were created on. Is there a way to query this aspect of the ObjectId?
Popping Timestamps into ObjectIds covers queries based on dates embedded in the ObjectId in great detail.
Briefly in JavaScript code:
/* This function returns an ObjectId embedded with a given datetime */
/* Accepts both Date object and string input */
function objectIdWithTimestamp(timestamp) {
/* Convert string date to Date object (otherwise assume timestamp is a date) */
if (typeof(timestamp) == 'string') {
timestamp = new Date(timestamp);
}
/* Convert date object to hex seconds since Unix epoch */
var hexSeconds = Math.floor(timestamp/1000).toString(16);
/* Create an ObjectId with that hex timestamp */
var constructedObjectId = ObjectId(hexSeconds + "0000000000000000");
return constructedObjectId
}
/* Find all documents created after midnight on May 25th, 1980 */
db.mycollection.find({ _id: { $gt: objectIdWithTimestamp('1980/05/25') } });
In pymongo, it can be done this way:
import datetime
from bson.objectid import ObjectId
mins = 15
gen_time = datetime.datetime.today() - datetime.timedelta(mins=mins)
dummy_id = ObjectId.from_datetime(gen_time)
result = list(db.coll.find({"_id": {"$gte": dummy_id}}))
Using inbuilt function provided by mongodb drivers in in Node.js lets you query by any timestamp:
var timestamp = Date.now();
var objectId = ObjectID.createFromTime(timestamp / 1000);
Alternatively, to search for records before the current time, you can simply do:
var objectId = new ObjectID(); // or ObjectId in the mongo shell
Source: http://mongodb.github.io/node-mongodb-native/api-bson-generated/objectid.html
You can use $convert function to extract the date from ObjectId starting in 4.0 version.
Something like
$convert: { input: "$_id", to: "date" }
You can query on date comparing between start and end time for date.
db.collectionname.find({
"$expr":{
"$and":[
{"$gte":[{"$convert":{"input":"$_id","to":"date"}}, ISODate("2018-07-03T00:00:00.000Z")]},
{"$lte":[{"$convert":{"input":"$_id","to":"date"}}, ISODate("2018-07-03T11:59:59.999Z")]}
]
}
})
OR
You can use shorthand $toDate to achieve the same.
db.collectionname.find({
"$expr":{
"$and":[
{"$gte":[{"$toDate":"$_id"}, ISODate("2018-07-03T00:00:00.000Z")]},
{"$lte":[{"$toDate":"$_id"},ISODate("2018-07-03T11:59:59.999Z")]}
]
}
})
how to find Find the Command (this date[2015-1-12] to this Date[2015-1-15]):
db.collection.find({
_id: {
$gt: ObjectId(Math.floor((new Date('2015/1/12'))/1000).toString(16) + "0000000000000000"),
$lt: ObjectId(Math.floor((new Date('2015/1/15'))/1000).toString(16) + "0000000000000000")
}
}).pretty()
Count the Command (this date[2015-1-12] to this Date[2015-1-15]):
db.collection.count({
_id: {
$gt: ObjectId(Math.floor((new Date('2015/1/12'))/1000).toString(16) + "0000000000000000"),
$lt: ObjectId(Math.floor((new Date('2015/1/15'))/1000).toString(16) + "0000000000000000")
}
})
Remove the Command (this date[2015-1-12] to this Date[2015-1-15]):
db.collection.remove({
_id: {
$gt: ObjectId(Math.floor((new Date('2015/1/12'))/1000).toString(16) + "0000000000000000"),
$lt: ObjectId(Math.floor((new Date('2015/1/15'))/1000).toString(16) + "0000000000000000")
}
})
Since the first 4 bytes of an ObjectId represent a timestamp, to query your collection chronologically, simply order by id:
# oldest first; use pymongo.DESCENDING for most recent first
items = db.your_collection.find().sort("_id", pymongo.ASCENDING)
After you get the documents, you can get the ObjectId's generation time like so:
id = some_object_id
generation_time = id.generation_time
Yes you can query object by date using MongoDB inserted ID
db.collectionname.find({_id: {$lt: ObjectId.fromDate( new ISODate("TZformat") ) } });
let's suppose users is my collection and I want all users created less than 05 January 2018
db.users.find({_id: {$lt: ObjectId.fromDate( new ISODate("2018-01-05T00:00:00.000Z") ) } });
For running from a query we can use like
db.users.find({_id: {$lt: ObjectId.fromDate(new Date((new Date().getTime() - (1 * 3 * 60 * 60 * 1000))) ) } })
All the users from the current time - 3 hours
To get last 60 days old documents in mongo collection i used below query in shell.
db.collection.find({_id: {$lt:new ObjectId( Math.floor(new Date(new Date()-1000*60*60*24*60).getTime()/1000).toString(16) + "0000000000000000" )}})
If you want to make a range query, you can do it like in this post. For example querying for a specific day (i.e. Apr 4th 2015):
> var objIdMin = ObjectId(Math.floor((new Date('2015/4/4'))/1000).toString(16) + "0000000000000000")
> var objIdMax = ObjectId(Math.floor((new Date('2015/4/5'))/1000).toString(16) + "0000000000000000")
> db.collection.find({_id:{$gt: objIdMin, $lt: objIdMax}}).pretty()
From the documentation:
o = new ObjectId()
date = o.getTimestamp()
this way you have date that is a ISODate.
Look at
http://www.mongodb.org/display/DOCS/Optimizing+Object+IDs#OptimizingObjectIDs-Extractinsertiontimesfromidratherthanhavingaseparatetimestampfield.
for more information
Using MongoObjectID you should also find results as given below
db.mycollection.find({ _id: { $gt: ObjectId("5217a543dd99a6d9e0f74702").getTimestamp().getTime()}});
A Solution Filtering within MongoDB Compass.
Based on versions:
Compass version: 1.25.0
MongoDB version: 4.2.8
Option 1:
#s7vr 's answer worked perfectly for me. You can paste this into the Filter field:
{$expr: { $and: [ {$gte: [{$toDate: "$_id"}, ISODate('2021-01-01')]}, {$lt: [{$toDate: "$_id"}, ISODate('2021-02-01')]} ] } }
Option 2:
I also found this to work (remember that the Date's month parameter is 0-based indexing so January is 0):
{_id: {$gte: ObjectId(Date(2021, 0, 1) / 1000), $lt: ObjectId(Date(2021, 1, 1) / 1000) } }
Option 3:
Equivalent with ISODate:
{_id: {$gte: ObjectId(ISODate('2021-01-01') / 1000), $lt: ObjectId(Date('2021-02-01') / 1000) } }
After writing this post, I decided to run the Explain on these queries. Here's the skinny on performance:
Option 1: 39 ms, 0 indexes used, 30 ms in COLLSCAN
Option 2: 0 ms, _id index used
Option 3: 1 ms, _id index used, 1 ms in FETCH
Based on my rudimentary analysis, it appears that option 2 is the most efficient. I will use Option 3, personally, as it is a little cleaner to use ISODate rather than remembering 0-based month indexing in the Date object.
In rails mongoid you can query using
time = Time.utc(2010, 1, 1)
time_id = ObjectId.from_time(time)
collection.find({'_id' => {'$lt' => time_id}})