I have a collection of user's browsing time.
db.userBrowsing.find()
{ userID: "ed3p6ddk0r", start_time: ISODate('2011-11-03 14:17:00'), time: 75 } );
{ userID: "2bnliaossn", start_time: ISODate('2011-11-03 14:23:00'), time: 10 } );
{ userID: "rhjynj9z8o", start_time: ISODate('2011-11-03 15:02:00'), time: 5 } );
{ userID: "q3wasyu5yp", start_time: ISODate('2011-11-03 16:45:00'), time: 20 } );
By aggregation, I maintained the totalBrowsingTime collection which contains userID and totalTime.
db.userBrowsing.find()
{ userID: "ed3p6ddk0r", totaltTime: 75 } );
{ userID: "2bnliaossn", totalTime: 10 } );
{ userID: "rhjynj9z8o", totalTime: 5 } );
{ userID: "q3wasyu5yp",totalTime: 20 } );
Day by day this collection populate for every user. So I need incremental aggregation to fill userBrowsing collection.
Should I use cronjob for this? or is there any different method for that?
Thanks!
Related
I have a collection in mongodb using mongoose packages like below
[
{
_id: new ObjectId("62ae97b6be08b688f93f2c07"),
reportId: '1',
method: 'A1',
category: 'B2',
date: '2022-06-19',
time: '22:55',
emergency: 'normal',
__v: 0
},
{
_id: new ObjectId("62ae97b6be08b688f93f2c08"),
reportId: '2',
method: 'A3',
category: 'B5',
date: '2022-06-18',
time: '23:05',
emergency: 'normal',
__v: 0
},
{
_id: new ObjectId("62ae97b6be08b688f93f2c09"),
reportId: '3',
method: 'A5',
category: 'B1',
date: '2022-06-19',
time: '23:55',
emergency: 'urgent',
__v: 0
}
]
and I want to filter this data, and here is my find function()
const options = [
{ method: { $in: ['A1','A2'] } },
{ emergency: { $in: data.emergency } },
{ category: { $in: data.category } }
];
const response = await Report.find({ $or: options,});
Until now, it works perfectly, but I still got one more filter: the date and time (They are all int type String).
I want to search for the range date and time between last night after 23 o'clock to 23 o'clock tonight.
But I have no idea how to write the query, please help me figure it out, thanks!!!
Here is my testing query:
date: {
$where: function () {
const yesterday = moment().subtract(1, 'days').format('YYYYMMDD') + '2300';
const date = moment(this.date).format('YYYYMMDD') + this.time.replace(':', '');
const today = moment().format('YYYYMMDD') + '2300';
return yesterday < date && date <= today;
},
},
I would recommend you to first save dates in a format that supports comparators (lt, gt, etc.) such as a Timestamp or even a plain number. That way you can use MongoDB filters like this:
let filter = {
date: { $gt: new Date(..), $lt: new Date(..) }
}
// This can be executed directly in the Mongo shell.
{
created_on: {
$gt: ISODate("1972-12-12"),
$lt: ISODate('2022-06-019')
}
}
But if you insist on using strings, then you would need to filter all your documents, and then parse each one to do your comparison.
I created a web-scraper to store data for a week to find a trend.
I wrote code to delete data from more than week ago every time the script runs.
However the data is still being stored for more than a week ago, is there a reason for this?"
example coin data createdAt field looks like
"createdAt": {
"$date": "2021-08-11T10:55:19.843Z"
},
coinSchema.statics.deleteOldData = async function () {
// delete old data
const today = new Date(Date.now());
today.setHours(0, 0, 0, 0);
const oneWeekAgo = new Date(Date.now());
const pastDate = oneWeekAgo.getDate() - 7;
oneWeekAgo.setDate(pastDate);
await this.deleteMany({
createdAt: {
$gte: today,
}, // 16 < 17 wont delete it prevent duplicates for one day
});
await this.deleteMany({
createdAt: {
$lt: pastDate,
}, // from 1 week ago
});
};
in the script i have the this run
async function main() {
await Coin.deleteOldData();
my coin model looks like :
const coinSchema = mongoose.Schema(
{
specNo: {
type: String,
required: true,
},
coinName: {
type: String,
required: true,
},
fullName: {
type: String,
required: false,
},
category: {
type: String,
},
array: [
{
GradeName: String,
PopulationCount: String,
trend: { type: Number, default: 0 },
},
],
},
{
timestamps: true,
}
);
Did you look at TTL based index? https://docs.mongodb.com/manual/core/index-ttl/
This is a good way to cleanup old data where DB itself takes care of it. In your case 7 days is 604800 seconds so if you create a index on createdAt with ttl 604800 then you should be all good!
db.collection.createIndex( { "createdAt ": 1 }, { expireAfterSeconds: 604800 } )
You can do it like this:
db.collection.deleteMany({
createdAt: { $lte: new Date(Date.now() - 7 * 24 * 60 * 60 * 1000).toISOString() },
})
my model data is like
{
sensor_id: 12345,
timestamp: ISODate("2019-01-31T10:00:00.000Z"),
temperature: 40
}
{
sensor_id: 12345,
timestamp: ISODate("2019-01-31T10:01:00.000Z"),
temperature: 40
}
{
sensor_id: 12345,
timestamp: ISODate("2019-01-31T10:02:00.000Z"),
temperature: 41
}
and expected output is like:
{
sensor_id: 12345,
start_date: ISODate("2019-01-31T10:00:00.000Z"),
end_date: ISODate("2019-01-31T10:59:59.000Z"),
measurements: [
{
timestamp: ISODate("2019-01-31T10:00:00.000Z"),
temperature: 40
},
{
timestamp: ISODate("2019-01-31T10:01:00.000Z"),
temperature: 40
},
…
{
timestamp: ISODate("2019-01-31T10:42:00.000Z"),
temperature: 42
}
],
transaction_count: 42,
sum_temperature: 2413
}
I have some different data set but actual problem is to use $facet with giving the start time and end time for size based bucketting in mongodb.
I'm trying to create a document, that last 120 seconds, and as soon as i call this method i want the TTL to restart.
At the moment i can"t update my document, after 120 sc .. the document get deleted and re-created instead of being always updated.
There is my collection :
LaptopConnections = new Mongo.Collection('laptopConnection');
let LaptopConnectionSchema = new SimpleSchema({
creationDate: {
type: Date,
label: "Date when the laptop was created",
defaultValue: Date.now
},
"state.date": {
type: Date,
label: "time when the laptop was updated",
autoValue: function () {
return new Date;
}
}
}
, { timestamps: true }
)
LaptopConnections.attachSchema(LaptopConnectionSchema)
And there is my method :
Meteor.startup(() => {
LapConnections._ensureIndex({ creationDate: 1 }, { expireAfterSeconds: 120 });// Will delete the collection after ~~ two minutes,
});
Meteor.methods({
create_lapconnection(lap_id) {
check(lap_id, String);
if (!LapConnections.findOne({ _id: lap_id })) {
console.log('workiiing',lap_id)
LaptopConnections.insert({
_id: box_id,
creationDate: Date.now(),
});
} else {
console.log('updated',lap_id)
LaptopConnections.update({ _id: lap_id }, { upsert: true }, {
$set: {
"state.date": Date.now(),
}
}
);
}
}
})
You are updating the state.date field while your index sits on the creationDate field. Once you update creationDate or change your index to use the state.date field instead it should work.
I want to create a publication which should return documents which have a different value for a set of fields. For example, if have a collection called 'User' with below transactions:
{
userId: 1,
date: '12-1-15',
amount: 450
}
{
userId: 1,
date: '12-1-15',
amount: 250
}
{
userId: 2,
date: '12-1-15',
amount: 100
}
{
userId: 2,
date: '11-1-15',
amount: 200
}
Then i need to create a query which will publish only the below documents, i.e, single document for each day and if a day has multiple documents then i want to publish only the document with highest amount.
[{
userId: 1,
date: '12-1-15',
amount: 450
},
{
userId: 2,
date: '12-1-15',
amount: 100
}
{
userId: 2,
date: '11-1-15',
amount: 200
}]
Thanks in advance. :)