I am trying to make a search on several field where some of them can be left blank.
Given the following Json document in MongoDB (only three fields here, but N fields in reality, with N > 10) :
{
'first_name' : 'value_X',
'last_name' : 'value_Y',
'mail_adress' : 'value_Z'
}
Then let's suppose a form where the user can enter the value of the first name, last name, mail address and phone number.
If all the field are filled the query in MongoDB looks like :
db.collection.find( {
'first_name' : 'value_A',
'last_name' : 'value_B',
'mail_adress' : 'value_C'
}
)
The problem I have is that I might have some fields left blank by the user :
It means that my query can also become :
db.collection.find({ 'first_name' : 'value_A' })
or
db.collection.find( {'last_name' : 'value_B'})
or
db.collection.find( {'mail_adress' : 'value_C'})
or
db.collection.find( {'first_name' : 'value_A','last_name' : 'value_B'})
or
db.collection.find( {'first_name' : 'value_A','last_name' : 'value_C'})
or
db.collection.find( {'last_name' : 'value_B','mail_adress' : 'value_C'})
I would like to know if I need to write a query for each particular instance ?
I also thought to generate query dynamically depending on the field entered by a user, it would be easy as query are Json doucment.
I am also wondering if MongoDB could ignore field if they are null (but I don't want to lose performance in my query)
Anyone can help me ? Thanks
EDIT on 20140404 at 6PM : part of answer
Here is the Python code with Aditya suggestion.
I also added the python code about generating the query in front end as svjn suggested:
'''
Created on 4 Apr 2014
#author: scoulombel
'''
import pymongo
from pymongo import MongoClient
import random
import string
# fill the database
def fillCollection(collection):
doc = {
'first_name' : 'value_X',
'last_name' : 'value_Y',
'mail_adress' : 'value_Z'
}
docID = collection.insert(doc)
for _ in range(0,10):
doc = {
'first_name' : ''.join(random.choice(string.ascii_uppercase) for _ in range(4)) + ''.join(random.choice(string.digits) for _ in range(4)),
'last_name' : ''.join(random.choice(string.ascii_uppercase) for _ in range(4)) + ''.join(random.choice(string.digits) for _ in range(4)),
'mail_adress' : ''.join(random.choice(string.ascii_uppercase) for _ in range(4)) + ''.join(random.choice(string.digits) for _ in range(4))
}
docID = collection.insert(doc)
# read data in a cursor
def readCursor(cursor):
object_list = []
count = 0
for doc in cursor :
del doc["_id"]
count += 1
doc['resOrder'] = count
object_list.append(doc)
print object_list
if __name__ == '__main__':
#client = MongoClient()
client = MongoClient('ip', 'port')
db = client.test_OR_AND_DB
collection = db.collection
fillCollection(collection)
# SEARCH VALUE which can be null or not
value_A = 'value_X'
value_B = 'null' #'value_Y'
value_C = 'null' #'value_Z'
# APPROACH 1 : ADITYA answer for search
cursor1 = db.collection.find(
{
'$or' : [
{ '$and' : [ {'first_name' : { '$ne': 'null' }}, {'first_name' : value_A } ]},
{ '$and' : [ {'last_name' : { '$ne': 'null' }}, {'last_name' : value_B } ]},
{ '$and' : [ {'mail_adress' :{ '$ne' :'null' }}, {'mail_adress':value_C } ]}
]
}
)
readCursor(cursor1)
# APPROACH 2 ; create a JSON representing the query dynamically
path_value_dict = {'first_name':value_A, 'last_name':value_B, 'mail_address':value_C} # if embedded it would be for instance name.first_name
query= {}
for key in path_value_dict.keys():
if path_value_dict[key] != 'null':
query[key] = path_value_dict[key]
print query
cursor2 = db.collection.find({'first_name': 'value_X', 'mail_adress': 'value_Z'})
readCursor(cursor2)
Use Mongodb $or operator for your query.
The $or operator performs a logical OR operation on an array of two or more <expressions> and selects the documents that satisfy at least one of the <expressions>.
Use :
db.collection.find({
$or : [{ $and : [ {'first_name' : { $ne:null }, {'first_name' :'value_A' }]},
{ $and : [ {'last_name' : { $ne:null }, {'last_name' :'value_B' }]},
{ $and : [ {'mail_addres' :{ $ne:null }, {'mail_addres':'value_C'}]}
]}
);
Related
known = [{ system_id : 1234},
{ system_id : 1235},
{ system_id : 1236},
{ system_id : 1237}]
peeps = [
{system_id: 1234, name : bob},
{system_id: 1232, name : jim},
{system_id: 1231, name : dave},
{system_id: 1237, name : jeff}
]
If I have the above two collections, and I want to find documents in the peeps collection that have system_ids that exist in documents in the known collection, how can that be done? I'm currently running an aggregation across peeps where I find all the unique combinations of name and system_id, but I need to exclude anything that is not in the known collection.
db.peeps.aggregate({
$lookup : {
from : "known",
localField : "system_id",
foreignField : "system_id",
as : "someField"
}
},{
$match : {
"someField.0" : {
$exists : true
}
}
},{
$project : {
"someField" : 0
}
})
This will output all those peeps which have system_id matched in knwon.
I want to check if user id exists inside an array field of mongodb (using meteor)
db.posts.find().pretty()
{
"_id" : "hT3ezqEyTaiihoh6Z",
"body" : "hey\n",
"authorId" : "AyJo5nf2Lkdqd6aRh",
"createdAt" : ISODate("2016-05-13T06:19:34.726Z"),
"updatedAt" : ISODate("2016-05-13T06:19:34.726Z"),
"likecount" : 0,
"already_voted" : [ ] }
db.posts.find( { _id:"hT3ezqEyTaiihoh6Z"},{ already_voted: { $in : ["AyJo5nf2Lkdqd6aRh"]} }).count()
1
It gives count value 1 , where as I am expecting it to be 0 .
Your logic is fine. Just the syntax is wrong.
db.posts
.find({
_id: "hT3ezqEyTaiihoh6Z",
already_voted: { $in: ["AyJo5nf2Lkdqd6aRh"] },
})
.count();
This should work.
You can just simply use count method. Don't need to use two operation like Find and then count.
db.posts
.count({
_id: "hT3ezqEyTaiihoh6Z",
already_voted: { $in: ["AyJo5nf2Lkdqd6aRh"] }
});
here is a typical document
{
title : 'someTitle',
places : [{name : 'someName', location : 'someLocation'}, {name ...}]
}
I have the following query
var qs = {title : 'someTitle', places : {$elemMatch : {name : 'someName' } } };
where I select a document which matches the title and which contains a document entry within its 'places' array that has name equal to 'someName'. However the issue is that the entries within the places array are large documents, and I only need a couple of fields from that document. I tried projecting the fields like so but it did not work.
var projection = {'places.$.name': 1, 'places.$.location' : 1};
Which is supposed to return an array with a document containing only the 'name' and 'location' property.
I got the following error
Can't canonicalize query: BadValue Cannot specify more than one positional proj. per query.
to be clear, I would like to accomplish this without the aggregate framework
You are doing it wrong. According to the documentation
Only one positional $ operator may appear in the projection document.
But you still need to use the $ operator to get the expected result:
var qs = { title : 'someTitle', 'places.name' : 'someName' };
var projection = {'places.$': 1 };
db.collection.find(qs, projection);
Which returns:
{
"_id" : ObjectId("564f52d7d9a433df958b5630"),
"places" : [
{
"name" : "someName",
"location" : "someLocation"
}
]
}
Also you don't need the $elemMatch operator here use "dot notation" instead.
Now if what you want is an array of "name" and "location" for each subdocument in the array then aggregation is the way to go.
db.collection.aggregate([
{ '$match': {
'title' : 'someTitle',
'places.name' : 'someName'
}},
{ '$project': {
'places': {
'$map': {
'input': '$places',
'as': 'place',
'in': {
'name': '$$place.name',
'location': '$$place.location'
}
}
}
}}
])
Which yields:
{
"_id" : ObjectId("564f52d7d9a433df958b5630"),
"places" : [
{
"name" : "someName",
"location" : "someLocation"
},
{
"name" : "bar",
"location" : "foo"
}
]
}
For the fields inside an array, you can project them the same as in embedded object
var projection = {'places.name': 1, 'places.location' : 1};
Check this guideline
https://docs.mongodb.com/manual/reference/operator/aggregation/project/#include-specific-fields-from-embedded-documents
We have the following Testsnippet in Ruby
def self.course_overview(course_member=nil)
course_member = CourseMember.last if course_member == nil
group_global = {"$group" =>
{"_id" => { "course_id" => "$course_id",
"title" => "$title",
"place" => "$place",
"description" => "$description",
"choosen_id" => "$choosen_id",
"year" => {"$year" => "$created_at"},
"course_member_ids" => "$course_member_ids"}}
}
match_global = {"$match" => {"_id.course_member_ids" => {"$in" => "#{course_member.id}"} }}
test = CoursePlan.collection.aggregate([group_global, match_global])
return test
end
The problem is the "match_global" statement. We would like to match all Documents where the course_member ID is appearing in the course_member_ids array.
The above statement fails with the error: "...must be an array". This make sense to me but according to other comments on the web this should be possible this way.
Any advice? How is it possible to return the docs where the course_member id is in the array of the course_member ids?
Sample CoursePlan Object:
{
"_id" : ObjectId("5371e70651a53ed5ad000055"),
"course_id" : ObjectId("5371e2e051a53ed5ad000039"),
"course_member_ids" : [
ObjectId("5371e2a751a53ed5ad00002d"),
ObjectId("5371e2b251a53ed5ad000030"),
ObjectId("5371e2bb51a53ed5ad000033")
],
"created_at" : ISODate("2014-05-13T09:33:58.042Z"),
"current_user" : "51b473bf6986aee9c0000002",
"description" : "Schulung 1 / Elektro",
"fill_out" : ISODate("2014-04-30T22:00:00.000Z"),
"place" : "TEST",
"title" : "Schulung 1",
"updated_at" : ISODate("2014-05-13T09:33:58.811Z"),
"user_ids" : [
ObjectId("51b473bf6986aee9c0000002"),
ObjectId("521d7f606986ae4826000002"),
ObjectId("521d8b3f6986aed678000007")
]
}
Since course_member_ids is an array of course members you should test for equality. In shell syntax:
{$match:{"_id.course_member_ids":<valueYouWantToTest>}}
You don't need $in as this query is analogous to a find when you want to select documents that have a particular single value you are looking for.
This is the first of 7 test/example documents, in collection "SoManySins."
{
"_id" : ObjectId("51671bb6a6a02d7812000018"),
"Treats" : "Sin1 = Gluttony",
"Sin1" : "Gluttony",
"Favourited" : "YES",
"RecentActivity" : "YES",
"GoAgain?" : "YeaSure."
}
I would like to be able to query to retrieve any info in any position,
just by referring to the position. The following document,
{
"_id" : ObjectId("51671bb6a6a02d7812000018"),
"Sin1" : "Gluttony",
"?????????" : "??????",
"RecentActivity" : "YES",
"GoAgain?" : "YeaSure."
}
One could retrieve whatever might be in the 3rd key~value
pair. Why should one have to know ahead of time what the
data is, in the key? If one has the same structure for the
collection, who needs to know? This way, you can get
double the efficiency? Like having a whole lot of mailboxes,
and your app's users supply the key and the value; your app
just queries the dbs' documents' arrays' positions.
Clara? finally? I hope?
The sample document you've provided is not saved as an array in BSON:
{
"_id" : ObjectId("51671bb6a6a02d7812000018"),
"Sin1" : "Gluttony",
"?????????" : "??????",
"RecentActivity" : "YES",
"GoAgain?" : "YeaSure."
}
Depending on the MongoDB driver you are using, the fields here are typically represented in your application code as an associative array or hash. These data structures are not order-preserving so you cannot assume that the 3rd field in a given document will correspond to the same field in another document (or even that the same field ordering will be consistent on multiple fetches). You need to reference the field by name.
If you instead use an array for your fields, you can refer by position or select a subset of the array using the $slice projection.
Example document with an array of fields:
{
"_id" : ObjectId("51671bb6a6a02d7812000018"),
"fields": [
{ "Sin1" : "Gluttony" },
{ "?????????" : "??????" },
{ "RecentActivity" : "YES" },
{ "GoAgain?" : "YeaSure." }
]
}
.. and query to find the second element of the fields array (a $slice with skip 1, limit 1):
db.SoManySins.find({}, { fields: { $slice: [1,1]} })
{
"_id" : ObjectId("51671bb6a6a02d7812000018"),
"fields" : [
{
"?????????" : "??????"
}
]
}
This is one way to Query and get back data when you may not
know what the data is, but you know the structure of the data:
examples in Mongo Shell, and in PHP
// the basics, setup:
$dbhost = 'localhost'; $dbname = 'test';
$m = new Mongo("mongodb://$dbhost");
$db = $m->$dbname;
$CursorFerWrites = $db->NEWthang;
// defining a set of data, creating a document with PHP:
$TheFieldGenerator = array( 'FieldxExp' => array(
array('Doc1 K1'=>'Val A1','Doc1 K2'=>'ValA2','Doc1 K3'=>'Val A3'),
array('Doc2 K1'=>'V1','Doc2 K2'=>'V2','Doc2 K3'=>'V3' ) ) ) ;
// then write it to MongoDB:
$CursorFerWrites->save($TheFieldGenerator);
NOTE : In the Shell : This produces the same Document:
> db.NEWthang.insert({"FieldxExp" : [
{"Doc1 K1":"Val A1","Doc1 K2":"Val A2","Doc1 K3":"Val A3"},
{"Doc2 K1":"V1", "Doc2 K2":"V2","Doc2 K3":"V3"}
]
})
#
Now, some mongodb Shell syntax:
> db.NEWthang.find().pretty()
{
"_id" : ObjectId("516c4053baa133464d36e836"),
"FieldxExp" : [
{
"Doc1 K1" : "Val A1",
"Doc1 K2" : "Val A2",
"Doc1 K3" : "Val A3"
},
{
"Doc2 K1" : "V1",
"Doc2 K2" : "V2",
"Doc2 K3" : "V3"
}
]
}
> db.NEWthang.find({}, { "FieldxExp" : { $slice: [1,1]} } ).pretty()
{
"_id" : ObjectId("516c4053baa133464d36e836"),
"FieldxExp" : [
{
"Doc2 K1" : "V1",
"Doc2 K2" : "V2",
"Doc2 K3" : "V3"
}
]
}
> db.NEWthang.find({}, { "FieldxExp" : { $slice: [0,1]} } ).pretty()
{
"_id" : ObjectId("516c4053baa133464d36e836"),
"FieldxExp" : [
{
"Doc1 K1" : "Val A1",
"Doc1 K2" : "Val A2",
"Doc1 K3" : "Val A3"
}
]
}
Finally, how about write the Query in some PHP ::
// these will be for building the MongoCursor:
$myEmptyArray = array();
$TheProjectionCriteria = array('FieldxExp'=> array('$slice' => array(1,1)));
// which gets set up here:
$CursorNEWthang1 = new MongoCollection($db, 'NEWthang');
// and now ready to make the Query/read:
$ReadomgomgPls=$CursorNEWthang1->find($myEmptyArray,$TheProjectionCriteria);
and the second document will be printed out:
foreach ($ReadomgomgPls as $somekey=>$AxMongoDBxDocFromCollection) {
var_dump($AxMongoDBxDocFromCollection);echo '<br />';
}
Hope this is helpful for a few folks.