I'm learning mongodb and java and have the following question, is it possible to chain filters?
So my example document in mongo is as follows
{"_id" : "...."
"name" :"Joe",
"roles" : ["A","B", "C"],
"value" : 1000
}
Can i do an update using a filter which will update depending on whether the document in mongo contain the roles
example my
listCriteria = ["B","D","E"]
so update this document if it has the roles B, D, E, update the value to 2000
In java I know I can use filters
Bson filter = Filters.eq("name", "Joe");
Filters.in("roles", roles);
.....
this.collection.updateOne(filter, updatedDocument...)
How can I chain it so that it updates the document with name "Joe" only if the roles in the documents contains atleast one in the list criteria
Must contain all 3 roles - use $all query operator:
db.people.update(
{
name: "Joe",
roles: {$all: ["B", "D", "E"]}
},
{
$set: {value: 2000}
}
);
Must contain any of 3 roles - use $in query operator:
db.people.update(
{
name: "Joe",
roles: {$in: ["B", "D", "E"]}
},
{
$set: {value: 2000}
}
);
About generally chaining filters, you can use $and query operator:
db.people.update(
{
$and: [
{name: "Joe"},
{roles: "B"},
{roles: "D"},
{roles: "E"}
]
},
{
$set: {value: 2000}
}
);
Related
I have a dataset like this on MongoDB:
{
"_id" : ObjectId("5f7b02a197cca91d3476a3b7"),
arrayHolder: [
{name: 'Peter'},
{name: 'John'}
]
},
{
"_id" : ObjectId("5f7b02a197cca91d3476a3b8"),
arrayHolder: [
{name: 'Harry'},
{name: 'Nik'}
]
}
Now I want to add one more key values to all objects under arrayHolder objects. I am sharing the Expected Output:
{
"_id" : ObjectId("5f7b02a197cca91d3476a3b7"),
arrayHolder: [
{name: 'Peter', valid: true },
{name: 'John', valid: true}
]
},
{
"_id" : ObjectId("5f7b02a197cca91d3476a3b8"),
arrayHolder: [
{name: 'Harry', valid: true},
{name: 'Nik', valid: true}
]
}
For this I wrote the query but that query will add the new object inside the array rather than adding one more key-value under the objects.
My Query is:
db.Collection.updateMany({}, {$push: {arrayHolder: {$each: {valid: true}}}})
This Query will add the new Objects inside an array. I am sharing the output which I am getting after this query run:
{
"_id" : ObjectId("5f7b02a197cca91d3476a3b7"),
arrayHolder: [
{name: 'Peter'},
{name: 'John'},
{valid: true}
]
},
{
"_id" : ObjectId("5f7b02a197cca91d3476a3b8"),
arrayHolder: [
{name: 'Harry'},
{name: 'Nik'},
{valid: true}
]
}
Is there anyone who guide me where I have done the mistakes and how to fire the proper query so that expected output will achieved.
Thanks in advance for the people who interacted with this problem
You can use all positional operator $[] to do that:
db.Collection.updateMany({}, {$set: {'arrayHolder.$[].valid': true}})
For example, I'm having a document like this:
{
name: "Test",
params: {
device: "windows",
gender: "m",
age: 28
}
}
Now I get the following params input:
{
device: "windows",
gender: "m"
}
In this case, I want to find all documents that partially match with the input object. Is there an easy way to solve this issue? I've tried $elemMatch but this seems only to work with arrays.
If I understood your question correctly, you have an input object which may contain some of the fields in the main document's params object, in any order, for example:
{
device: "windows",
gender: "m"
}
{
gender: "m",
device: "windows"
}
{
device: "windows",
age: 28
}
and you want to match only if all the fields in the input object exist in the main document:
{
device: "linux", // NO MATCH
gender: "m"
}
{
gender: "m", // MATCH
device: "windows"
}
{
device: "windows", // NO MATCH
age: 29
}
Correct?
Option 1
Does your param object always contain only those three fields (device, gender and age)?
If so, you can manually project each of them at the root level, do your match, and "unproject" them again:
db.my_collection.aggregate([
{
$project: {
name: 1,
params: 1,
device: "$params.device", // add these three for your match stage
gender: "$params.gender",
age: "$params.age"
}
},
{
$match: input_params
},
{
$project: {name: 1, params: 1} // back to original
}
]);
But I assume you want to do this for any number of fields inside your params object.
Option 2
Do you have a way to manipulate the input object coming in? If so, you could preface all fields with "params.":
let input_params =
{
device: "windows",
gender: "m"
};
let new_input_params =
{
"params.device": "windows",
"params.gender": "m"
};
Then your query would just be:
db.my_collection.find(new_input_params);
Option 3
If you have no way of modifying the input, you can use the $replaceRoot aggregation operator (as of version 3.4, credit to this answer) to flatten your params into the root document. Since this will replace the root document with the embedded one, you need to extract the fields you are interested in first, at least the _id field:
db.my_collection.aggregate([
{
$addFields: {"params.id": "$_id"} // save _id inside the params doc,
// as you will lose the original one
},
{
$replaceRoot: {newRoot: "$params"}
},
{
$match: input_params
},
...
]);
This will match the documents and retain the _id field which you can use to get the rest of the document again, through a $lookup for example:
...
{
$lookup: {
from: "my_collection",
localField: "id",
foreignField: "_id",
as: "doc"
}
},
...
This will get your documents in the following format:
{
"device" : "windows",
"gender" : "m",
"age" : 28,
"id" : ObjectId("XXX"),
"doc" : [
{
"_id" : ObjectId("XXX"),
"name" : "Test",
"params" : {
"device" : "windows",
"gender" : "m",
"age" : 28
}
}
]
}
To go full circle and get your original document format back, you can:
...
{
$unwind: "$doc" // get rid of the [] around the "doc" object
// ($lookup always results in array)
},
{
$replaceRoot: {newRoot: "$doc"} // get your "doc" back to the root
}
...
As I write this, I cannot believe there is no cleaner way of doing this using MongoDB alone, but I cannot think of any.
I hope this helps!
You can use $or operator for partial comparison of the fields from params object. You can use like this:
db.collection.find({
$or: [
{
"params.device": "windows"
},
{
"params.gender": "m"
}
]
})
Using projection
A document may have many fields, but not all of these fields may be necessary and important when requested. In this case, you can only include the required fields in the sample by using the projection.
By default, queries in MongoDB return all fields in matching
documents. To limit the amount of data that MongoDB sends to
applications, you can include a projection document to specify or
restrict fields to return.
For example, you previously added the following documents to the database:
> db.users.insertOne({"name": "Jennifer", "gender": "female", languages: ["english", "spanish"]})
> db.users.insertOne({"name": "Alex", "gender": "male", languages: ["english", "french"]})
> db.users.insertOne({"name": "Maria", "gender": "female", languages: ["english", "german"]})
And now you want to display information that match the field gender and has value female, so you can specify it via query:
db.users.find({gender: "female"}, {name: 1})
The operation returns the following documents:
{ "name" : "Jennifer", "gender" : "female" }
{ "name" : "Maria", "gender" : "female" }
The <value> can be any of the following:
{ field1: <value>, field2: <value> ... }
1 or true to include the field in the return documents.
0 or false to exclude the field.
Expression using a Projection Operators.
Moreover, if you don't want to concretize the selection, but want to display all the documents, then we can leave the first brackets empty:
db.users.find({}, {name: 1, _id: 0})
I have one document like this:
-document: users-
{
"name": "x", password: "x" recipes:
[{title: eggs, dificult: 1},{title: "pizza" dificult: 2}],
name: "y", password: "y" recipes: [{title: "oil", dificult: 2},{title: "eggandpotatoes" dificult: 2}]
}
I want to get all recipes filtering by title and dificult
I have tried some like this
db.users.find({$and: [{"recipes.title": /.*egg.*/},{"recipes.dificult": 2}]},
{"recipes.title": 1,"recipes.dificult": 1}).toArray();
this should return
{title: "eggandpotatoes" dificult: 2}
but return
{title: eggs, dificult: 1}
{title: "eggandpotatoes" dificult: 2}
I would like once the filter works, limit the result with start: 2 and end: 5
returning from the 2 result the next 5.
Thanks in advance.
You can use the $filter aggregation to filter the recipes and $slice to limit the results. It will give you everything you are looking expect but regex search which is not possible as of now.
db.users.aggregate([{
$project: {
_id: 0,
recipes: {
$slice: [{
$filter: {
input: "$recipes",
as: "recipe",
"cond": {
$and: [{
$eq: ["$$recipe.title", "eggandpotatoes"]
}, {
$eq: ["$$recipe.dificult", 2]
}]
}
}
}, 2, 3]
}
}
}]);
If you need to find out regular expression in title then use $elemMatch as below :
db.users.find({"recipes":{"$elemMatch":{"title": /.*egg.*/,"dificult": 2}}}, {"recipes.$":1})
One thing I mentioned here if your recipes array contains some thing like this
"recipes" : [ { "title" : "egg", "dificult" : 2 }, { "title" : "eggand", "dificult" : 2 } ]
then $ projection return only first matched array objcect in above case it will be
"recipes" : [ { "title" : "egg", "dificult" : 2 } ]
If you need this both array then go to #Sagar answer using aggregation
I'm trying to query one of my mongodb collection that looks like this:
> db.collection.find()
{name: "foo", things: [{stuff:"banana", value: 1}, {stuff:"apple", value: 2}, {stuff:"strawberry", value: 3}]}
{name: "bar", things: [{stuff:"banana", value: 4}, {stuff:"pear", value: 5}]}
...
My goal is to list all the object that have the things field containing an element with stuff=banana but no stuff=apple
I tried something like this:
db.transactions.find({
"things": {
$elemMatch: {
"stuff": "banana",
$ne: {
"stuff": "apple"
}
}
}
)
But it's not working. Any ideas?
The below query will get the list of all documents that have the things field containing an element with stuff=banana but no stuff=apple:
db.test.find({"$and":[{"things.stuff":"banana"}, {"things.stuff":{$ne:"apple"}}]})
Use the $not and $and operators:
db.collection.find({
$and:[
{"things": {$elemMatch: { "stuff": "banana" }}},
{"things": {$not: {$elemMatch: { "stuff": "apple"}}}}
]
});
Use $in and $nin to include and exclude
db.transactions.find({
"things.stuff": {
$in: ["banana"],
$nin:["apple"]
}
})
I know the theory of MongoDB and the fact that is doesn't support joins, and that I should use embeded documents or denormalize as much as possible, but here goes:
I have multiple documents, such as:
Users, which embed Suburbs, but also has: first name, last name
Suburbs, which embed States
Child, which embeds School, belongs to a User, but also has: first name, last name
Example:
Users:
{ _id: 1, first_name: 'Bill', last_name: 'Gates', suburb: 1 }
{ _id: 2, first_name: 'Steve', last_name: 'Jobs', suburb: 3 }
Suburb:
{ _id: 1, name: 'Suburb A', state: 1 }
{ _id: 2, name: 'Suburb B', state: 1 }
{ _id: 3, name: 'Suburb C', state: 3 }
State:
{ _id: 1, name: 'LA' }
{ _id: 3, name: 'NY' }
Child:
{ _id: 1, _user_id: 1, first_name: 'Little Billy', last_name: 'Gates' }
{ _id: 2, _user_id: 2, first_name: 'Little Stevie', last_name: 'Jobs' }
The search I need to implement is on:
first name, last name of Users and Child
State from Users
I know that I have to do multiple queries to get it done, but how can that be achieved? With mapReduce or aggregate?
Can you point out a solution please?
I've tried to use mapReduce but that didn't get me to have documents from Users which contained a state_id, so that's why I brought it up here.
This answer is outdated. Since version 3.2, MongoDB has limited support for left outer joins with the $lookup aggregation operator
MongoDB does not do queries which span multiple collections - period. When you need to join data from multiple collections, you have to do it on the application level by doing multiple queries.
Query collection A
Get the secondary keys from the result and put them into an array
Query collection B passing that array as the value of the $in-operator
Join the results of both queries programmatically on the application layer
Having to do this should be rather the exception than the norm. When you frequently need to emulate JOINs like that, it either means that you are still thinking too relational when you design your database schema or that your data is simply not suited for the document-based storage concept of MongoDB.
So now join is possible in mongodb and you can achieve this using $lookup and $facet aggregation here and which is probably the best way to find in multiple collections
db.collection.aggregate([
{ "$limit": 1 },
{ "$facet": {
"c1": [
{ "$lookup": {
"from": Users.collection.name,
"pipeline": [
{ "$match": { "first_name": "your_search_data" } }
],
"as": "collection1"
}}
],
"c2": [
{ "$lookup": {
"from": State.collection.name,
"pipeline": [
{ "$match": { "name": "your_search_data" } }
],
"as": "collection2"
}}
],
"c3": [
{ "$lookup": {
"from": State.collection.name,
"pipeline": [
{ "$match": { "name": "your_search_data" } }
],
"as": "collection3"
}}
]
}},
{ "$project": {
"data": {
"$concatArrays": [ "$c1", "$c2", "$c3" ]
}
}},
{ "$unwind": "$data" },
{ "$replaceRoot": { "newRoot": "$data" } }
])
You'll find MongoDB easier to understand if you take a denormalized approach to schema design. That is, you want to structure your documents the way the requesting client application understands them. Essentially, you are modeling your documents as domain objects with which the applicaiton deals. Joins become less important when you model your data this way. Consider how I've denormalized your data into a single collection:
{
_id: 1,
first_name: 'Bill',
last_name: 'Gates',
suburb: 'Suburb A',
state: 'LA',
child : [ 3 ]
}
{
_id: 2,
first_name: 'Steve',
last_name: 'Jobs',
suburb: 'Suburb C',
state 'NY',
child: [ 4 ]
}
{
_id: 3,
first_name: 'Little Billy',
last_name: 'Gates',
suburb: 'Suburb A',
state: 'LA',
parent : [ 1 ]
}
{
_id: 4,
first_name: 'Little Stevie',
last_name: 'Jobs'
suburb: 'Suburb C',
state 'NY',
parent: [ 2 ]
}
The first advantage is that this schema is far easier to query. Plus, updates to address fields are now consistent with the individual Person entity since the fields are embedded in a single document. Notice also the bidirectional relationship between parent and children? This makes this collection more than just a collection of individual people. The parent-child relationships mean this collection is also a social graph. Here are some resoures which may be helpful to you when thinking about schema design in MongoDB.
Here's a JavaScript function that will return an array of all records matching specified criteria, searching across all collections in the current database:
function searchAll(query,fields,sort) {
var all = db.getCollectionNames();
var results = [];
for (var i in all) {
var coll = all[i];
if (coll == "system.indexes") continue;
db[coll].find(query,fields).sort(sort).forEach(
function (rec) {results.push(rec);} );
}
return results;
}
From the Mongo shell, you can copy/paste the function in, then call it like so:
> var recs = searchAll( {filename: {$regex:'.pdf$'} }, {moddate:1,filename:1,_id:0}, {filename:1} )
> recs
Based on #brian-moquin and others, I made a set of functions to search entire collections with entire keys(fields) by simple keyword.
It's in my gist; https://gist.github.com/fkiller/005dc8a07eaa3321110b3e5753dda71b
For more detail, I first made a function to gather all keys.
function keys(collectionName) {
mr = db.runCommand({
'mapreduce': collectionName,
'map': function () {
for (var key in this) { emit(key, null); }
},
'reduce': function (key, stuff) { return null; },
'out': 'my_collection' + '_keys'
});
return db[mr.result].distinct('_id');
}
Then one more to generate $or query from keys array.
function createOR(fieldNames, keyword) {
var query = [];
fieldNames.forEach(function (item) {
var temp = {};
temp[item] = { $regex: '.*' + keyword + '.*' };
query.push(temp);
});
if (query.length == 0) return false;
return { $or: query };
}
Below is a function to search a single collection.
function findany(collection, keyword) {
var query = createOR(keys(collection.getName()));
if (query) {
return collection.findOne(query, keyword);
} else {
return false;
}
}
And, finally a search function for every collections.
function searchAll(keyword) {
var all = db.getCollectionNames();
var results = [];
all.forEach(function (collectionName) {
print(collectionName);
if (db[collectionName]) results.push(findany(db[collectionName], keyword));
});
return results;
}
You can simply load all functions in Mongo console, and execute searchAll('any keyword')
You can achieve this using $mergeObjects by MongoDB Driver
Example
Create a collection orders with the following documents:
db.orders.insert([
{ "_id" : 1, "item" : "abc", "price" : 12, "ordered" : 2 },
{ "_id" : 2, "item" : "jkl", "price" : 20, "ordered" : 1 }
])
Create another collection items with the following documents:
db.items.insert([
{ "_id" : 1, "item" : "abc", description: "product 1", "instock" : 120 },
{ "_id" : 2, "item" : "def", description: "product 2", "instock" : 80 },
{ "_id" : 3, "item" : "jkl", description: "product 3", "instock" : 60 }
])
The following operation first uses the $lookup stage to join the two collections by the item fields and then uses $mergeObjects in the $replaceRoot to merge the joined documents from items and orders:
db.orders.aggregate([
{
$lookup: {
from: "items",
localField: "item", // field in the orders collection
foreignField: "item", // field in the items collection
as: "fromItems"
}
},
{
$replaceRoot: { newRoot: { $mergeObjects: [ { $arrayElemAt: [ "$fromItems", 0 ] }, "$$ROOT" ] } }
},
{ $project: { fromItems: 0 } }
])
The operation returns the following documents:
{ "_id" : 1, "item" : "abc", "description" : "product 1", "instock" : 120, "price" : 12, "ordered" : 2 }
{ "_id" : 2, "item" : "jkl", "description" : "product 3", "instock" : 60, "price" : 20, "ordered" : 1 }
This Technique merge Object and return the result
Minime solution worked except that it required a fix:
var query = createOR(keys(collection.getName()));
need to add keyword as 2nd parameter to createOR call here.