Pymongo Query Filtering a versioned data - mongodb

Question

Here's a sample data from my database:
{'data':
[{'is_active': True, 'printer_name': 'A', 'vid': 14510},
{'is_active': True, 'printer_name': 'Changed A', 'vid': 14511}
]
},
{'data':
[{'is_active': False, 'printer_name': 'B', 'vid': 14512}]
}
The vid field here is the version id. Whenever a record is edited, the modified data is pushed into the list and it therefore has a higher vid than its old version.
Now I want to define a method called get_all_active_printers which returns all printers with is_active :True
Here's my attempt, but it returns both printers when it should not return printer B
def get_all_active_printers():
return printers.find(
{'data.is_active': True}, {"data": {"$slice": -1}, '_id': 0, 'vid': 0})
Whats wrong with my query ?
Edit 1 in response to comment by WanBachtiar
Here's the actual output from using the command print([c for c in get_all_active_printers()])
[{'data': [{'printer_name': 'Changed A', 'vid': 1451906336.6602068, 'is_active': True, 'user_id': ObjectId('566bbf0d680fdc1ac922be4c')}]}, {'data': [{'printer_name': 'B', 'vid': 1451906343.8941162, 'is_active': False, 'user_id': ObjectId('566bbf0d680fdc1ac922be4c')}]}]
As you can see in the actual output - the is_active value for Printer B is False, but get_all_active_printers still returns B
Here's my version details:
Python 3.4.3
pymongo 3.2
mongodb 2.4.9
On Ubuntu 14.04, if that matters.
Edit 2
Noticed yet another issue. The query is returning vid field, even though have clearly mentioned 'vid': 0 in the projection.
* Edit 3*
I am not sure by what you mean when you say
"make sure that there is no other documents for {'printer_name': 'B'}"
. Yes the second data (on printer B) has a second row. That was the first data - when the printer was created when the field is_active was true. Later it becomes false. Here's the snapshot of the database:
But I want to filter on the latest data as old data is only for keeping an audit trail.
If i move 'data.is_active': True to the projections as in the following code:
def get_all_active_printers():
return printers.find(
{}, {'data': {'$slice': -1}, 'data.is_active': True, '_id': 0, 'vid': 0})
I get the following error message:
pymongo.errors.OperationFailure: database error: You cannot currently
mix including and excluding fields. Contact us if this is an issue.
So how do i filter based on the latest data, given the snapshot above ? Sorry if my question did not make it clear earlier.

Answer 1

Thanks for clarifying the question.
So you are wanting to query documents that have only the latest element with is_active: True.
Unfortunately, in your case find({'data.is_active': True}) would find all documents containing any data element with is_active:True, not just the last element in the array. Also, without knowing the length of the array, you cannot reference the last element of the array using the array.i syntax.
However there are other ways/alternatives:
Update using $push, $each and $position to insert new elements to the front of the array. Mongo Shell example:
/* When updating */
db.printers.update(
/* Filter document for printer B */
{"data.printer_name": 'B'},
/* Push a new document to the front of the array */
{"$push": {
"data": {
$each: [{'is_active': false, 'printer_name': "B", 'vid': 14513 }],
$position: 0
}
}
}
);
/* When querying now you know that the latest one is on index 0 */
db.printers.find(
{"data.0.is_active": true},
{"data": { $slice: 1} }
);
Note that $position is new in MongoDB v2.6.
Use MongoDB aggregation to $unwind the array, $group then $match to filter. For example:
db.printers.aggregate([
{$unwind: '$data' },
{$sort: { 'data.vid' : 1 } },
{$group: {
'_id': { 'printer_name' : '$data.printer_name', id:'$_id' },
'vid': { $max: '$data.vid' },
'is_active' : { $last: '$data.is_active' }
}
},
{$match:{"is_active": true}}
]);
It may be beneficial for you to re-consider the document schema. For example, instead of having an array of documents maybe you should consider having them flat for ease of querying.
{'is_active': true, 'printer_name': 'A', 'vid': 14512}
{'is_active': false, 'printer_name': 'B', 'vid': 14513}
For more examples and discussions of different version tracking schema designs, see these blog posts:
How to track versions with MongoDB
Further thought on how to track versions with MongoDB
Also a useful reference on schema designs : Data Modeling Introduction.
The query is returning vid field, even though have clearly mentioned
'vid': 0 in the projection.
You could hide it with "data.vid": 0 instead of vid:0.
If i move 'data.is_active': True to the projections as in the following code... I get the following error message.
There are rules of projections that you have to follow. Please see projecting fields from query results for more information on projections.
If you are starting a new project, I would recommend to use the latest stable release of MongoDB, currently it is v3.2.0.
Regards,
Wan.