How do I compute the difference of two queries? - mongodb

Question

I have a MongoDB collection that contains a set of documents. Each document has an ISODate date and an integer id (not _id). id: X is said to exist for date: D if there is a document in the collection with field values { id: X, date: D }. So, for example:
{ id: 1, date: 1/1/2000 }
{ id: 1, date: 1/2/2000 }
{ id: 1, date: 1/3/2000 }
{ id: 1, date: 1/4/2000 }
{ id: 2, date: 1/2/2000 }
{ id: 2, date: 1/3/2000 }
{ id: 3, date: 1/3/2000 }
I would like to track ids over time as they are created and destroyed day-to-day. Using the above data, over the date range 1/1/2000 to 1/4/2000:
1/1/2000: id 1 is created
1/2/2000: id 2 is created
1/3/2000: id 3 is created
1/4/2000: id 2 is destroyed
1/4/2000: id 3 is destroyed
I think the best way to solve this would be to loop day by day, see what ids exist between today and the next day, and perform a set difference. For example, to get the set of ids created and destroyed on 1/2/2000, I need to perform two set differences between arrays for either day:
var A = [ <ids that exist on 1/1/2000> ];
var B = [ <ids that exist on 1/2/2000> ];
var created_set = set_difference(B, A); // Those in B and not in A
var destroyed_set = set_difference(A, B); // Those in A and not in B
I can use a find() command to get cursors for A and B, but I do not know how to perform the set_difference between two cursors.
My other option was to use an aggregation pipeline, but I cannot think about how to formulate the pipeline in such a way that I can use the $setDifference operator.
As a MongoDB novice, I am sure I'm thinking about the problem the wrong way. Surely this is something that can be done? What am I missing?

Answer 1

db.mystuff.aggregate([
{$group: {_id: '$id', created: {$first: '$date'}, destroyed: {$last: '$date'}}}
])

Answer 2

Suppose you have the following sample collection:
db.collection.insert([
{ id: 1, date: ISODate("2000-01-01") },
{ id: 1, date: ISODate("2000-01-02") },
{ id: 1, date: ISODate("2000-01-03") },
{ id: 1, date: ISODate("2000-01-04") },
{ id: 2, date: ISODate("2000-01-02") },
{ id: 2, date: ISODate("2000-01-03") },
{ id: 3, date: ISODate("2000-01-03") }
]);
The following aggregation will give you some direction towards what you are trying to achieve using the $setDifference operator:
var start = new Date(2000, 0, 1);
var end = new Date(2000, 0, 2)
db.collection.aggregate([
{
"$match":{
"date": {
"$gte": start,
"$lte": end
}
}
},
{
$group: {
_id: "$date",
"A": {
"$addToSet": {
"$cond": [
{ "$eq": [ "$date", start ] },
"$id",
false
]
}
},
"B": {
"$addToSet": {
"$cond": [
{ "$eq": [ "$date", end ] },
"$id",
false
]
}
}
}
},
{
"$project": {
"A": {
"$setDifference": [ "$A", [false] ]
},
"B": {
"$setDifference": [ "$B", [false] ]
}
}
},
{
"$project": {
"_id": 0,
"date": "$_id",
"created_set": {
"$setDifference": [ "$B", "$A" ]
},
"destroyed_set": {
"$setDifference": [ "$A", "$B" ]
}
}
}
]);
Output:
{
"result" : [
{
"date" : ISODate("2000-01-02T00:00:00.000Z"),
"created_set" : [2, 1],
"destroyed_set" : []
},
{
"date" : ISODate("2000-01-01T00:00:00.000Z"),
"created_set" : [],
"destroyed_set" : [1]
}
],
"ok" : 1
}