Query For Total User Count Day By Day In MongoDB - mongodb

I have "users" collection and i want day by day total user count eg:
01.01.2012 -> 5
02.01.2012 -> 9
03.01.2012 -> 18
04.01.2012 -> 24
05.01.2012 -> 38
06.01.2012 -> 48
I have createdAt attritube for each user. Can you help me about the query ?
{
"_id" : ObjectId( "5076d3e70546c971539d9f8a" ),
"createdAt" : Date( 1339964775466 ),
"points" : 200,
"profile" : null,
"userId" : "10002"
}

here this is works for, day by day count data
output i got:
30/3/2016 4
26/3/2016 4
21/3/2016 4
12/3/2016 12
14/3/2016 18
10/3/2016 10
9/3/2016 11
8/3/2016 19
7/3/2016 21
script:
model.aggregate({
$match: {
createdAt: {
$gte: new Date("2016-01-01")
}
}
}, {
$group: {
_id: {
"year": { "$year": "$createdAt" },
"month": { "$month": "$createdAt" },
"day": { "$dayOfMonth": "$createdAt" }
},
count:{$sum: 1}
}
}).exec(function(err,data){
if (err) {
console.log('Error Fetching model');
console.log(err);
} else {
console.log(data);
}
});

You have a couple of options, in order of performance :
Maintain the count in seperate aggregation documents. Every time you add a user you update the counter for that day (so, each day has its unique counter document in a, say, a users.daycounters collection). This is easily the fastest approach and scales best.
In 2.2 or higher you can use the aggregation framework. Examples close to your use case are documented here. Look for the $group operator : http://docs.mongodb.org/manual/applications/aggregation/
You can use the map/reduce framework : http://www.mongodb.org/display/DOCS/MapReduce. This is sharding compatible but relatively slow due to the JavaScript context use. Also it's not very straightforward for something as simple as this.
You can use the group() operator documented here : http://www.mongodb.org/display/DOCS/Aggregation#Aggregation-Group. Since this does not work in a sharded environment and is generally slow due to the use of the single-threaded JavaScript context this is not recommended.

Related

How to write a single query to count elements above a certain value in MongoDB

I have the following sample collection of movies:
[
{
"title":"Boots and Saddles",
"year":1909,
"cast":[],
"genres":[]
},
{
"title":"The Wooden Leg",
"year":1909,
"cast":[],
"genres":[]
},
{
"title":"The Sanitarium",
"year":1910,
"cast":["Fatty Arbuckle"],
"genres":["Comedy"]
},
{
"title":"Snow White",
"year":1916,
"cast":["Marguerite Clark"],
"genres":["Fantasy"]
},
{
"title":"Haunted Spooks",
"year":1920,
"cast":["Harold Lloyd"],
"genres":["Comedy"]
},
{
"title":"Very Truly Yours",
"year":1922,
"cast":["Shirley Mason", "lan Forrest"],
"genres":["Romance"]
}
]
I want to count number of movies appeared in the last 20 years (from the last movie recorded in this collection).
I have following query to find which year is the most recent movie (result shows 2018):
db.movies.find({},{"_id":0, "year":1}).sort({year:-1}).limit(1)
So to find how many movies appeared in the last 20 years I wrote this:
db.movies.aggregate([{$match:{year:{$gte:1999}}},{$count:"title"}])
However, this is not very optimized, because if the database is modified or updated,I will have to modify that query every time.
Is there a more elegant way to find the result?
Thank you in advance!
You can use mongodb aggregate method.
db.movies.aggregate([
{ $sort: { year: -1 } },
{ $limit: 1 },
{
$project: {
currentYear: { $year: new Date() },
latestMovieYear: "$year",
last20Years: { $subtract: [ "$currentYear", 20 ] }
}
},
{
$match: {
year: { $gte: "$last20Years", $lte: "$latestMovieYear" }
}
},
{ $count: "movies" }
]);
Sort the documents by year in descending order, and limit the number of documents to 1. It will return latest movie present in the collection.
Use the $project operator to create a new field currentYear that returns the current year, latestMovieYear that returns the year of the latest movie, and last20Years that subtracts 20 from the current year.
Use $match operator to filter out the movies that have a year greater than or equal to last20Years and less than or equal to latestMovieYear.
Use the $count operator to count the number of documents that match the above criteria.

MongoDB query is slow and not using Indexes

I have a collection with structure like this:
{
"_id" : ObjectId("59d7cd63dc2c91e740afcdb"),
"enrollment" : [
{ "month":-10, "enroll":'00'},
{ "month":-9, "enroll":'00'},
{ "month":-8, "enroll":'01'},
//other months
{ "month":8, "enroll":'11'},
{ "month":9, "enroll":'11'},
{ "month":10, "enroll":'00'}
]
}
I am trying to run the following query:
db.getCollection('collection').find({
"enrollment": {
"$not": {
"$elemMatch": { "month": { "$gte": -2, "$lte": 9 }, "enroll": "00" }
}
}
}).count()
This query is taking 1.6 to 1.9 seconds. I need to get this down as low as possible, to milli seconds if that is possible.
I tried creating multi key index on month and enrolled fields. I tried various combinations but the query is not using any indexes.
I tried all these combinations:
1. { 'enrollment.month':1 }
2. { 'enrollment.month':1 }, { 'enrollment.enroll':1 } -- two seperate indexes
3. { 'enrollment.month':1, 'enrollment.enroll':1}
4. { 'enrollment.enroll':1, 'enrollment.month':1}
Parsed Query:
Query Plan:
Any suggestions to improve the performance are highly appreciated.
I am fairly confident that the hardware is not an issues but open for any suggestions.
My data size is not huge. Its just under 1GB. Total number of documents are 41K and sub document count is approx. 13 million
Note: I have posted couple of questions on this in last few days, but with this i am trying to narrow down the area. Please do not take this as a duplicate of my earlier questions.
Try to inverse the query:
db.getCollection('collection').find({
"enrollment": {
"$elemMatch": {
"month": { "$lt": -2, "$gt": 9 },
"enroll": {$ne: "00"}
}
}
}).count()

MongoDB Aggregate multiple count and latest date

I'm trying get a Mongo 3.0 query that is beyond my depth. Was hoping for a bit of help. Basically, my database has transcription records whereby there is a given username, project_id, expedition_id and finished_date. Those are the fields I'm interested in. A project will have multiple expeditions, each expedition multiple transcriptions.
I would like to display information for a given user in a stats page for a given project. The display would be User Name, Total Project Transcriptions that user submitted for the whole project, Total Participated Expeditions the number of expeditions the user participated in across the project, and the last date the user actually performed a transcription.
So far, it's easy enough to get the Total Project Transcriptions by using the count on the user_name and matching the project_id
db.transcriptions.aggregate([
{ "$match" : {"projectId" => 13}},
{ "$group": {
"_id": "$user_name",
"transcriptionCount" : {"$sum" : 1 }
}
}
])
Each transcription document has an expeditionId field (4, 7, 9, 10, etc.) and the finished_date. So if a user performed 100 transcriptions, only participating in expedition 7 and 10, the Total Participated Expeditions would = 2
The last finished_date being a date showing the last time a user performed a transcription. Example of returned record:
user_name: john smith
transcriptionCount: 100
expeditionCount: 2
last_date: 2017-08-15
Hope I explained that well enough. Would appreciate any help.
You can try the below aggregation.
db.transcriptions.aggregate([
{
"$match": {
"projectId" => 13
}
},
{
"$sort": {
"finished_date": -1
}
},
{
"$group": {
"_id": "$user_name",
"transcriptionCount": {
"$sum": 1
},
"expedition": {
"$addToSet": "$expedition_id"
},
"last_date": {
"$first": "$finished_date"
}
}
},
{
"$project": {
"_id": 0,
"user_name": "$_id",
"transcriptionCount": 1,
"expeditionCount": {
"$size": "$expedition"
},
"last_date": 1
}
}
])

How to count the number of documents on date field in MongoDB

Scenario: Consider, I have the following collection in the MongoDB:
{
"_id" : "CustomeID_3723",
"IsActive" : "Y",
"CreatedDateTime" : "2013-06-06T14:35:00Z"
}
Now I want to know the count of the created document on the particular day (say on 2013-03-04)
So, I am trying to find the solution using aggregation framework.
Information:
So far I have the following query built:
collection.aggregate([
{ $group: {
_id: '$CreatedDateTime'
}
},
{ $group: {
count: { _id: null, $sum: 1 }
}
},
{ $project: {
_id: 0,
"count" :"$count"
}
}
])
Issue: Now considering above query, its giving me the count. But not based on only date! Its taking time as well into consideration for unique count.
Question: Considering the field has ISO date, Can any one tell me how to count the documents based on only date (i.e excluding time)?
Replace your two groups with
{$project:{day:{$dayOfMonth:'$createdDateTime'},month:{$month:'$createdDateTime'},year:{$year:'$createdDateTime'}}},
{$group:{_id:{day:'$day',month:'$month',year:'$year'}, count: {$sum:1}}}
You can read more about the date operators here: http://docs.mongodb.org/manual/reference/aggregation/#date-operators

Querying documents by fields in an array

In MongoDB I have a collection of documents called 'clients', where each document is structured roughly as follows. Note that TimeStamp is a DateTime:
{
"Sessions": [
{
"SessionId": "pojiu5iprr2xw5ucsudyrkpv",
"LocationId": {
"$oid": "4de4590bfcee1a1b44165e2b"
},
"Timestamp": "Wed, 01 Jun 2011 09:39:26 GMT -04:00"
}
],
"_id": {
"$oid": "4de6410efcee1a1204a3326c"
}
}
I'd like to be able to query for "all clients which have at least one session with a timestamp in the last 24 hours". Is this possible without resorting to a full M/R scan? I will also accept answers with a statically defined Date, e.g. "all clients which have at least one session with a timestamp later than 1/5/2012"
How about adding a field to the main document for "latest session time".
Updated:
Ok, here's how you can do it without adding a new field/index:
db.clients.find(
{ "Sessions" :
{ $elemMatch :
{ Timestamp : { $gt : new Date(new Date() - 86400000) } }
}
}
);
same code on a single line:
db.clients.find({ "Sessions" : { $elemMatch : { Timestamp : { $gt : new Date(new Date() - 86400000) } } } } );
86400000 is the number of milliseconds in a day.