In MongoDB I have a collection of documents called 'clients', where each document is structured roughly as follows. Note that TimeStamp is a DateTime:
{
"Sessions": [
{
"SessionId": "pojiu5iprr2xw5ucsudyrkpv",
"LocationId": {
"$oid": "4de4590bfcee1a1b44165e2b"
},
"Timestamp": "Wed, 01 Jun 2011 09:39:26 GMT -04:00"
}
],
"_id": {
"$oid": "4de6410efcee1a1204a3326c"
}
}
I'd like to be able to query for "all clients which have at least one session with a timestamp in the last 24 hours". Is this possible without resorting to a full M/R scan? I will also accept answers with a statically defined Date, e.g. "all clients which have at least one session with a timestamp later than 1/5/2012"
How about adding a field to the main document for "latest session time".
Updated:
Ok, here's how you can do it without adding a new field/index:
db.clients.find(
{ "Sessions" :
{ $elemMatch :
{ Timestamp : { $gt : new Date(new Date() - 86400000) } }
}
}
);
same code on a single line:
db.clients.find({ "Sessions" : { $elemMatch : { Timestamp : { $gt : new Date(new Date() - 86400000) } } } } );
86400000 is the number of milliseconds in a day.
Related
I have a collection of 100 records.Each record have date field like below
"created_date" : ISODate("2018-11-20T00:00:00.000Z")
I want to update each record by removing the time like below
"created_date" : "2018-11-20"
How to write a query for this kind of update
You can remove the timestamp from the date using Aggregate Query and update collection using forEach function. It may take time but it will update the collection data as per your requirement. Here is Mongo aggregation query:
db.getCollection('demo').aggregate([
{
$addFields: {
DateinString: {
$dateToString: {
format: "%Y-%m-%d",
date: "$created_date"
}
}
}
}
]).forEach(function(myDoc){
db.getCollection("demo").update({
_id: myDoc._id
}, {
$set: {
"created_date": myDoc.DateinString
}
})
})
I am passing in two dates formatted as MM-DD-YYYY which is a date range. I need to query all records within that range and include specified fields. I've had no luck.
Part of a record in Mongo:
{
"_id": "some ID",
"date": {
"$date": "2015-06-26T13:02:12.121Z"
},
Query:
var Start = '09-07-2015'
var End = '09-14-2015'
If I do:
var query = Order.find({
date : {
$lt : End,
$gt : Start
}
});
I get the full document within the week ranges as expected. However, I want to specify the fields to return rather than full document. So I've tried using grouping and project to specify those fields:
var query = Order.aggregate(
{
$match :
{
date: {
$gte: start,
$lt: end
}
},
$group:
{
cust_ID: '$request.headers.customer_id',
wholesaler_ID: '$request.headers.wholesalerID'
}
}
);
Likewise: I've also tried it using project to get the results I want. I thought maybe it won't match on a date string like 09-07-2015, so I included the ISO date directly. Still no luck... the query comes back undefined or empty:
var query = Order.aggregate(
{
$project:
{
date: 'date',
cust_ID: '$request.headers.custID',
wholesaler_ID: '$request.headers.wholesalerID'
}
},
{
$match :
{
date: {
$gte: "2014-12-09T21:02:56.872Z",
$lt: "2015-12-09T21:02:56.872Z"
}
}
}
);
var query = Order.find({
date : {
$lt : End,
$gt : Start
}}, {cust_ID:1, wholeseller_ID:1}
);
This will work.
I just tested this using Robomongo
db.getCollection('offerdb').find({time_posted:{$gt: '2015-10-21T21:40:04+05:30', $lte:'2015-12-14T05:53:14+05:30'}},{_id:1, merchant_id:1})
Works like a charm for me.
Try this command in mongo shell
use dbname
db.collection_name.find({date: {$gte: ISODate('2015-09-07 00:00:00'), $lte: ISODate('2015-09-14 23:59:59.999999')}},{'cust_ID':1,'_id':0,'wholeseller_ID':1})
I've searched the internet long and hard but can't find a solution to this problem. Whilst there are lots of Map reduce examples, i'm getting confused because my document has a property which is an array of objects.
I'm pretty sure this should be easy for someone with experience but i'm a noob at the minute.
I have a document which looks roughly like this
{
_id:guid,
clientId:guid,
reference:'abc123'
items:
[
{ _id:guid, category:'A', length:100, active:true },
{ _id:guid, category:'B', length:150, active:true },
{ _id:guid, category:'A', length:10, active:false },
{ _id:guid, category:'A', length:111, active:true },
]
}
and I want to produce this output
dateFromIdGuid(day) category countOfItems countOfActive sumOfLength
I'd like to keep the data in this format to reduce the number of write operations (there are already over 1000 writes to this collection per second and rising)
This is driving me insane so any help would be very much appreciated.
Thanks.
If you are talking about extracting a timestamp and reducing that to a discrete day from a GUID, then MongoDB is not going to be of much help to you there. You would need an external language implementation that would support such a function and implement an external mapReduce process such as with Hadoop.
It makes me wonder though if we are in fact talking about a GUID or whether you actually mean an ObjectID which would be the default value for the _id field of your document unless this has been specifically overridden to have a GUID in there.
Even if that is not true, you would be helped by adding a "timestamp" field of some sort to your document and using the correct BSON Date object type as shown below:
{
_id:guid,
"timestamp": ISODate("2014-05-27T00:00:00Z")
"clientId":guid,
"reference":'abc123'
"items":
[
{ _id:guid, category:'A', length:100, active:true },
{ _id:guid, category:'B', length:150, active:true },
{ _id:guid, category:'A', length:10, active:false },
{ _id:guid, category:'A', length:111, active:true },
]
}
This allows you to use the MongoDB aggregation framework as it can operate on Date objects of this type in order to break down the results to discrete days:
db.collection.aggregate([
{ "$unwind": "$items" },
{ "$group": {
"_id": {
"day": { "$dayOfYear": "$timestamp" },
"category": "$items.category"
},
"countOfItems": { "$sum": 1 },
"countOfActive": {
"$sum": {
"$cond": [
"$items.active",
1,
0
]
}
},
"sumOfLength": { "$sum": "$items.length" }
}}
])
That not only gives you the results in the fastest way MongoDB can do it but that "timestamp" value is also useful for filtering queries within date ranges which is something you cannot easily do from other values.
Also there is a way in the JavaScript available to MongoDB mapReduce that allows you to get the date from an ObejctId. This runs slower than the aggregation framework though:
db.collection.mapReduce(
function() {
var date = this._id.getTimestamp();
items.forEach(function(item) {
var day =
"" + date.getFullyear() +
"" + ( date.getMonth() + 1 ) +
"" + date.getDate();
emit(
{
day: day,
category: item.category
},
{
countOfItems: 1,
countOfActive: ( item.active ) ? 1 : 0,
sumOfLength: item.length
}
);
});
},
function( key, values ) {
var reduced = {
countOfItems: 0,
countOfActive: 0,
sumOfLength: 0
};
values.forEach(function(value) {
for ( var k in value ) {
reduced[k] += value[k];
}
});
return reduced;
},
{
"out": { "inline": 1 }
}
)
That basically does the same thing where the mapper breaks apart the array and provides grouping keys while the reducer just sums up the values from the mapper. So even if you had to extract from GUID's that gives you a basic layout for a mapper and reducer in a language such as Java when using Hadoop.
Take a look at the aggregate and mapReduce manual pages for more information on options you can apply.
I have a collection
{ "_id" : 1325376000, "value" : 13393}
{ "_id" : 1325462400, "value" : 13393}
ObjectIds are Unix Timestamp and are storing as Number manually.(at insert time).
now I'm searching for a solution that i could calculate sum of values for each month with Aggregation Framework.
Here is a way you can do it by generating the aggregation pipeline programmatically:
numberOfMonths=24; /* number of months you want to go back from today's */
now=new Date();
year=now.getFullYear();
mo=now.getMonth();
months=[];
for (i=0;i<numberOfMonths;i++) {
m1=mo-i+1; m2=m1-1;
d = new Date(year,m1,1);
d2=new Date(year,m2,1);
from= d2.getTime()/1000;
to= d.getTime()/1000;
dt={from:from, to:to, month:d2}; months.push(dt);
}
prev="$nothing";
cond={};
months.forEach(function(m) {
cond={$cond: [{$and :[ {$gte:["$_id",m.from]}, {$lt:["$_id",m.to]} ]}, m.month, prev]};
prev=cond;
} );
/* now you can use "cond" variable in your pipeline to generate month */
db.collection.aggregate( { $project: { month: cond , value:1 } },
{ $group: {_id:"$month", sum:{$sum:"$value"} } }
)
I have "users" collection and i want day by day total user count eg:
01.01.2012 -> 5
02.01.2012 -> 9
03.01.2012 -> 18
04.01.2012 -> 24
05.01.2012 -> 38
06.01.2012 -> 48
I have createdAt attritube for each user. Can you help me about the query ?
{
"_id" : ObjectId( "5076d3e70546c971539d9f8a" ),
"createdAt" : Date( 1339964775466 ),
"points" : 200,
"profile" : null,
"userId" : "10002"
}
here this is works for, day by day count data
output i got:
30/3/2016 4
26/3/2016 4
21/3/2016 4
12/3/2016 12
14/3/2016 18
10/3/2016 10
9/3/2016 11
8/3/2016 19
7/3/2016 21
script:
model.aggregate({
$match: {
createdAt: {
$gte: new Date("2016-01-01")
}
}
}, {
$group: {
_id: {
"year": { "$year": "$createdAt" },
"month": { "$month": "$createdAt" },
"day": { "$dayOfMonth": "$createdAt" }
},
count:{$sum: 1}
}
}).exec(function(err,data){
if (err) {
console.log('Error Fetching model');
console.log(err);
} else {
console.log(data);
}
});
You have a couple of options, in order of performance :
Maintain the count in seperate aggregation documents. Every time you add a user you update the counter for that day (so, each day has its unique counter document in a, say, a users.daycounters collection). This is easily the fastest approach and scales best.
In 2.2 or higher you can use the aggregation framework. Examples close to your use case are documented here. Look for the $group operator : http://docs.mongodb.org/manual/applications/aggregation/
You can use the map/reduce framework : http://www.mongodb.org/display/DOCS/MapReduce. This is sharding compatible but relatively slow due to the JavaScript context use. Also it's not very straightforward for something as simple as this.
You can use the group() operator documented here : http://www.mongodb.org/display/DOCS/Aggregation#Aggregation-Group. Since this does not work in a sharded environment and is generally slow due to the use of the single-threaded JavaScript context this is not recommended.