I have a collection of documents which contain unique id field. Now I have a list of ids which may contain some ids that do not exist in the collection. What's the best way to find out those ids from the list?
I know I can use $in operator to get the documents which have ids contained in the list then compare with the given id list, but is there better way to do it?
I suppose you have the following documents in your collection:
{ "_id" : ObjectId("55b725fd7279ca22edb618bb"), "id" : 1 }
{ "_id" : ObjectId("55b725fd7279ca22edb618bc"), "id" : 2 }
{ "_id" : ObjectId("55b725fd7279ca22edb618bd"), "id" : 3 }
{ "_id" : ObjectId("55b725fd7279ca22edb618be"), "id" : 4 }
{ "_id" : ObjectId("55b725fd7279ca22edb618bf"), "id" : 5 }
{ "_id" : ObjectId("55b725fd7279ca22edb618c0"), "id" : 6 }
and the following list of id
var listId = [ 1, 3, 7, 9, 8, 35 ];
We can use the .filter method to return the array of ids that is not in your collection.
var result = listId.filter(function(el){
return db.collection.distinct('id').indexOf(el) == -1; });
This yields
[ 7, 9, 8, 35 ]
Now you can also use the aggregation frameworks and the $setDifference operator.
db.collection.aggregate([
{ "$group": { "_id": null, "ids": { "$addToSet": "$id" }}},
{ "$project" : { "missingIds": { "$setDifference": [ listId, "$ids" ]}, "_id": 0 }}
])
This yields:
{ "missingIds" : [ 7, 9, 8, 35 ] }
Unfortunately MongoDB can only use built in functions (otherwise I'd recommend using a set) but you could try and find all distinct id's in your list then just manually pull them out.
Something like (untested):
var your_unique_ids = ["present", "not_present"];
var present_ids = db.getCollection('your_col').distinct('unique_field', {unique_field: {$in: your_unique_ids}});
for (var i=0; i < your_unique_ids.length; i++) {
var some_id = your_unique_ids[i];
if (present_ids.indexOf(some_id) < 0) {
print(some_id);
}
}
Below query will fetch you the result :
var listid = [1,2,3,4];
db.collection.aggregate([
{$project: { uniqueId :
{
"$setDifference":
[ listid , db.collection.distinct( "unique_field" )]} , _id : 0 }
},
{$limit:1}
]);
Related
I have the following JSON collection in MongoDB.
{
"_id": ObjectId("57529381551673386c9150a6"),
"team_code": 3,
"team_id": 2
},
{
"_id": ObjectId("57529381551673386c91514a"),
"team_code": 4,
"team_id": 5
},
{
"_id": ObjectId("57529381551673386c91514b"),
"team_code": 3,
"team_id": 2
},
{
"_id": ObjectId("57529381551673386c91514c"),
"team_code": 4,
"team_id": 5,
}
As it can be seen from the data , there a 2 records each with (team_code=3, team_id =2) and (team_code =4, team_id=5).
Is it possible to get the distinct set of team codes and team ids.
Some thing like ,
{
"team_code" : 3, "team_id" : 2
},
{
"team_code" : 4, "team_id" : 5,
}
You can do this using the following Aggregation Pipeline:
var distinctIdCode = { $group: { _id: { team_code: "$team_code", team_id: "$team_id" } } }
db.foo.aggregate([distinctIdCode])
This will give you:
{ "_id" : { "team_code" : 4, "team_id" : 5 } }
{ "_id" : { "team_code" : 3, "team_id" : 2 } }
This query returns new documents created from the documents in your collection. The documents returned have an _id which is the result of grouping the collection documents on the team_code and team_id fields.
For Java code:
Bson field1 = new Document("team_code", "$team_code").append("team_id", "$team_id");Bson groupField = new Document("_id", field1 );
Bson group = new Document("$group", groupField );List<Bson> bsonList = new ArrayList<Bson>();
bsonList.add(group);
com.mongodb.client.AggregateIterable<Document> res=collection.aggregate( bsonList );
for (Document doc : res) {
Document subDoc=(Document)doc.get("_id");
System.out.println(subDoc.get("team_code")+" : "+subDoc.get("team_id"));
//deptEnvs.put(subDoc.getInteger("dept"), subDoc.getString("smEnv"));
}
I saw few solutions but those are not exact my solution. I have a DB with name results and collection name is marks like below:
db.marks.find();
{ "_id" : ObjectId("54f57522627af4bfdcf79764"), "name" : "John", "scroe1" : 23, "score2" : 21, "score5" : 12 }
{ "_id" : ObjectId("54f5761a627af4bfdcf79765"), "name" : "Mike", "scroe2" : 22, "score3" : 20, "score4" : 22 }
{ "_id" : ObjectId("559d0bc521cb2e056507c3e3"), "name" : "Bush", "score2" : 30 }
I tried with
var doc=db.marks.findOne(); for (var key in doc) print(key);
and i got
_id
name
score1
score2
score5
But i Want all keys in collection like below:
_id, name, score1, score2, score3, score4, score5
name
scroe1
score2
score3here
findOne will only return the first found document. Since the first document you list does not have the score3 and score4 keys, it will not display them. If you want to show all root-level keys across all documents, you would need to iterate through all the documents in the db.
var keys = [];
db.marks.find().forEach(function(doc){
for (var key in doc){
if(keys.indexOf(key) < 0){
keys.push(key);
}
}
});
print(keys);
mr = db.runCommand({
"mapreduce" : "my_collection",
"map" : function() {
for (var key in this) { emit(key, null); }
},
"reduce" : function(key, stuff) { return null; },
"out": "my_collection" + "_keys"
})
Then run distinct on the resulting collection so as to find all the keys:
db[mr.result].distinct("_id")
["foo", "bar", "baz", "_id", ...]
Mongodb find() command has two arguments first one is query and second is projections.
Something like db.collection.find(query,projection).
if the document is db.myCol.find();, then it returns:
{
{
_id:1
name: ''hello',
age: 23
}, {
_id:2
name: ''bollo',
age: 27
}
}
And db.myCol.find({},{_id:1}); returns:
1
2
How to avoid duplicate entries in mongoDb in Meteor application.
On the command: db.products.find({},{"TEMPLATE_NAME": 1},{unique : true})
{ "_id" : ObjectId("5555d0a16ce3b01bb759a771"), "TEMPLATE_NAME" : "B" }
{ "_id" : ObjectId("5555d0b46ce3b01bb759a772"), "TEMPLATE_NAME" : "A" }
{ "_id" : ObjectId("5555d0c86ce3b01bb759a773"), "TEMPLATE_NAME" : "C" }
{ "_id" : ObjectId("5555d0f86ce3b01bb759a774"), "TEMPLATE_NAME" : "C" }
{ "_id" : ObjectId("5555d1026ce3b01bb759a775"), "TEMPLATE_NAME" : "A" }
{ "_id" : ObjectId("5555d1086ce3b01bb759a776"), "TEMPLATE_NAME" : "B" }
I want to retrieve only the unique template names and show them on HTML page.
Use the aggregation framework where your pipeline stages consist of the $group and $project operators respectively. The $group operator step groups the input documents by the given key and thus will return distinct documents in the result. The $project operator then reshapes each document in the stream, such as by adding new fields or removing existing fields:
db.products.aggregate([
{
"$group": {
"_id": "$TEMPLATE_NAME"
}
},
{
"$project": {
"_id": 0,
"TEMPLATE_NAME": "$_id"
}
}
])
Result:
/* 0 */
{
"result" : [
{
"TEMPLATE_NAME" : "C"
},
{
"TEMPLATE_NAME" : "A"
},
{
"TEMPLATE_NAME" : "B"
}
],
"ok" : 1
}
You could then use the meteorhacks:aggregate package to implement the aggregation in Meteor:
Add to your app with
meteor add meteorhacks:aggregate
Then simply use .aggregate function like below.
var products = new Mongo.Collection('products');
var pipeline = [
{
"$group": {
"_id": "$TEMPLATE_NAME"
}
},
{
"$project": {
"_id": 0,
"TEMPLATE_NAME": "$_id"
}
}
];
var result = products.aggregate(pipeline);
-- UPDATE --
An alternative that doesn't use aggregation is using underscore's methods to return distinct field values from the collection's find method as follows:
var distinctTemplateNames = _.uniq(Collection.find({}, {
sort: {"TEMPLATE_NAME": 1}, fields: {"TEMPLATE_NAME": true}
}).fetch().map(function(x) {
return x.TEMPLATE_NAME;
}), true)
;
This will return an array with distinct product template names ["A", "B", "C"]
You can check out some tutorials which explain the above approach in detail: Get unique values from a collection in Meteor and METEOR – DISTINCT MONGODB QUERY.
You can use distinct of mongodb like :
db.collectionName.distinct("TEMPLATE_NAME")
This query will return you array of distinct TEMPLATE_NAME
There are 200 documents in school db. I must remove each document which has "type":"homework" and the lowest score.
{
"_id" : 0,
"name" : "aimee Zank",
"scores" :
[
{
"type" : "exam",
"score" : 1.463179736705023
},
{
"type" : "quiz",
"score" : 11.78273309957772
},
{
"type" : "homework",
"score" : 6.676176060654615
},
{
"type" : "homework",
"score" : 35.8740349954354
}
]
}
For example,here
{
"type" : "homework",
"score" : 6.676176060654615
}
must be removed as score = 6.6 < 35.8
I sorted all the documents like this:
db.students.find({"scores.type":"homework"}).sort({"scores.score":1})
But I do not know how then to remove the doc having the lowest score and type:homework???
NOTE: how to solve it by not using aggregation method? E.g., by sorting and then updating.
This can be done in a couple of steps. The first step is to grab a list of the documents with the minimum score by using the aggregation framework with $match, $unwind and $group operators that streamlines your documents to find the minimum score for each document:
lowest_scores_docs = db.school.aggregate([
{ "$match": {"scores.type": "homework"} },
{ "$unwind": "$scores" }, { "$match": {"scores.type": "homework"} },
{ "$group": { "_id":"$_id", "lowest_score": {"$min": "$scores.score" } } } ] )
The second step is to loop through the dictionary above and use the $pull operator in the update query to remove the element from the array as follows:
for result in lowest_scores_docs["result"]:
db.school.update({ "_id": result["_id"] },
{ "$pull": { "scores": { "score": result["lowest_score"] } } } )
import pymongo
import sys
# connnecto to the db on standard port
connection = pymongo.MongoClient("mongodb://localhost")
db = connection.school # attach to db
students = db.students # specify the colllection
try:
cursor = students.find({})
print(type(cursor))
for doc in cursor:
hw_scores = []
for item in doc["scores"]:
if item["type"] == "homework":
hw_scores.append(item["score"])
hw_scores.sort()
hw_min = hw_scores[0]
#students.update({"_id": doc["_id"]},
# {"$pull":{"scores":{"score":hw_min}}})
except:
print ("Error trying to read collection:" + sys.exc_info()[0])
I am trying to find in a collection all of the documents that have the given key equal to one of the strings in an array.
Heres an example of the collection.
{
roomId = 'room1',
name = 'first'
},
{
roomId = 'room2',
name = 'second'
},
{
roomId = 'room3',
name = 'third'
}
And heres an example of the array to look through.
[ 'room2', 'room3' ]
What i thought would work is...
collection.find({ roomId : { $in : [ 'room2', 'room3' ]}}, function( e, r )
{
// r should return the second and third room
});
How can i achieve this?
One way this could be solve would be to do a for loop...
var roomIds = [ 'room2', 'room3' ];
for ( var i=0; i < roomIds.length; i++ )
{
collection.find({ id : roomIds[ i ]})
}
But this is not ideal....
What you posted should work - no looping required. The $in operator does the job:
> db.Room.insert({ "_id" : 1, name: 'first'});
> db.Room.insert({ "_id" : 2, name: 'second'});
> db.Room.insert({ "_id" : 3, name: 'third'});
> // test w/ int
> db.Room.find({ "_id" : { $in : [1, 2] }});
{ "_id" : 1, "name" : "first" }
{ "_id" : 2, "name" : "second" }
> // test w/ strings
> db.Room.find({ "name" : { $in : ['first', 'third'] }});
{ "_id" : 1, "name" : "first" }
{ "_id" : 3, "name" : "third" }
Isn't that what you expect?
Tested w/ MongoDB 2.1.1