Seems to be a simple question, but I can't find the answer. There is a collection of posts, documents of which have a field rubric. How to read ALL documents from this collection in mongoose using Post.find ({rubric: value}, function), but keeping rubric. What value has to be for this to work?
To find according if a field exists whatever value it has, you can use $exists in this way:
db.collection.find({
"rubric": {
"$exists": true
}
})
Using mongoose is the same query. Something like this:
yourModel.find({
"rubric": {
"$exists": true
}
})
Example here.
Also, to know about these world in mongo world you can check documentation every time you have an issue.
At the left of the page you see "Reference > Operators" and there are a lof of different operators you can use.
Related
Mongo's documentation on $or operator says:
When evaluating the clauses in the
$or
expression, MongoDB either performs a collection scan or, if all the clauses are supported by indexes, MongoDB performs index scans. That is, for MongoDB to use indexes to evaluate an
$or
expression, all the clauses in the
$or
expression must be supported by indexes. Otherwise, MongoDB will perform a collection scan.
So effectively, if you want the query to be efficient, both of the attributes used in the $or condition should be indexed.
However, I'm not sure if this applies to "findOne" operations as well since I can't use Mongo's explain functionality for a findOne operation. It seems logical to me that if you only care about returning one document, you would check an indexed condition first, since you could bail after just finding one without needing to care about the non-indexed fields.
Example
Let's pretend in the below that "email" is indexed, but username is not. It's a contrived example, but bear with me.
db.users.findOne(
{
$or: [
{ email : 'someUser#gmail.com' },
{ username: 'some-user' }
]
}
)
Would the above use the email index, or would it do a full collection scan until it finds a document that matches the criteria?
I could not find anything documenting what would be expected here. Does someone know the answer?
Well I feel a little silly - it turns out that the docs I found saying you can only run explain on "find" may have been just referring to when you're in MongoDB Compass.
I ran the below (userEmail not indexed)
this.userModel
.findOne()
.where({ $or: [{ _id: id }, { userEmail: email }] })
.explain();
this.userModel
.getModelInstance()
.findOne()
.where({ _id: id })
.explain();
...and found that the first does do a COLLSCAN and the second's plan was IDHACK (basically it does a normal ID lookup).
When running performance tests, I can see that the first is about 4x slower in a collection with 20K+ documents (depends on where in the natural order the document you're finding is)
Consider the following:
I have a MongoDB collection named C_a. It contains a very large number of documents (e.g., more than 50,000,000).
For the sake of simplicity let's assume that each document has the following schema:
{
"username" : "Aventinus"
"text": "I love StackOverflow!",
"tags": [
"programming",
"mongodb"
]
}
Using text index I can return all documents which contain the keyword StackOverflow like this:
db.C_a.find({$text:{$search:"StackOverflow"}})
My question is the following:
Considering that the query above may return hundreds of thousands of documents, what is the easiest/fastest way to directly save the returned results into another collection named C_b?
Note: This post explains how to use aggregate to find exact matches (i.e., specific dates). I'm interested in using Text Index to save all the posts which include a specific keyword.
The referenced answer is correct. The example query from that answer can be updated to use your criteria:
db.C_a.aggregate([
{$match: {$text: {$search:"StackOverflow"}}},
{$out:"C_b"}
]);
From the MongoDB documentation for $text:
If using the $text operator in aggregation, the following restrictions also apply.
The $match stage that includes a $text must be the first stage in the pipeline.
A text operator can only occur once in the stage.
The text operator expression cannot appear in $or or $not expressions.
The text search, by default, does not return the matching documents in order of matching scores. Use the $meta aggregation expression in the $sort stage.
In Meteor I use MongoDB to store a collection of Objects. There is around 500k docs inserted.
I use Objets.find({ "_id": { "$in": objIds } }); Where objIds is an array of _id. This works fine when I have an array length of 1000 but when I try with 13145 _ids the app stops responding.
Obviously there's already an index on the _id field and also this search probably won't ever happen but I'm not sure if this is normal behavior. Is there a max length for the $in operator? Couldn't find one in the documentation.
Here's my publish in Meteor :
Meteor.publish('objetsByIds', function objetsByIdsPublication(objIds) {
return Objets.find({ "_id": { "$in": objIds } });
})
Don't know much about Meteor, BUT, MongoDB uses cursors when retrieving large amounts of data, and depending on the driver implementation is how Meteor handle this.
Though you could take a look at cursors here, other idea that comes to my mind is to divide the query. So if you know 1000 works well, make a loop where, using mod, you make the results be 1000 documents long.
I've done a bit of research on this and haven't come across anything that jumps out at me immediately as what I'm looking for.
Say we have a document (or documents) in a collection that look something like this:
//First example document
{
"_id": "JK",
"letters": ["J", "K"]
}
//Second example document
{
"_id": "LM",
"letters": ["L"]
}
So I run a query like the one below to see if I have any matching documents and of course I don't so I expect to get null.
> db.example.findOne({"_id": "LM", "letters": {"$in": ["M"]}})
null
So I do an update and add "M" to the letters array on the documents (syntax may not be quite right):
> db.example.update({"_id": "LM"}, {"$addToSet": {"letters": "M"}})
I run the possibility of not having a matching _id, so the findOne would would also return null given the example documents in the collection for this query.
> db.example.findOne({"_id": "AB", "letters": {"$in": ["A"]}})
null
Based on the way I've constructed the above query, I get null back when "A" is not found in letters or the _id of "AB" is not found on any document. In this case I know that this document isn't in there because I know what is in the collection.
What I'd like to do is keep my update query from above with $addToSet and modify it to use upsert WHILE ALSO specifying the document to insert in the event that $addToSet fails due to the document not existing to cut down on database transactions. Is this possible? Or will I have to break up my queries a bit in order to accommodate this?
Because this information may influence answers:
I do my querying through mongo shell and pymongo.
mongo version: 2.6.11
pymongo version: 2.8
Thanks for any help!
EDIT: So after a break and a bit more digging, it seems setOnInsert does what I was looking for. I do believe that this probably solves my issue, but I've not had a chance to test yet.
Imagine a collection of movies (stored in a MongoDB collection), with each one looking something like this:
{
_id: 123456,
name: 'Blade Runner',
buyers: [1123, 1237, 1093, 2910]
}
I want to get a list of movies, each one with an indication whether buyer 2910 (for example) bought it.
Any ideas?
I know I can change [1123, 1237, 1093, 2910] to [{id:1123}, {id:1237}, {id:1093}, {id:2910}] to allow the use of $elemMatch in the projection, but would prefer not to touch the structure.
I also know I can perhaps use the $unwind operator (within the aggregation framework), but that seems very wasteful in cases where buyer has thousands of values (basically exploding each document into thousands of copies in memory before matching).
Any other ideas? Am I missing something really simple here?
You can use the $setIsSubset aggregation operator to do this:
var buyer = 2910;
db.movies.aggregate(
{$project: {
name: 1,
buyers: 1,
boughtIt: {$setIsSubset: [[buyer], '$buyers']}
}}
)
That will give you all movie docs with a boughtIt field added that indicates whether buyer is contained in the the movie's buyers array.
This operator was added in MongoDB 2.6.
Not really sure of your intent here, but you don't need to change the structure just to use $elemMatch in projection. You can just issue like this:
db.movies.find({},{ "buyers": { "$elemMatch": { "$eq": 2910 } } })
That would filter the returned array elements to just the "buyer" that was indicated, or nothing where this was not present. It is true to point out that the $eq operator used here is not actually documented, but it does exist. So that may not be immediately clear that you can construct a condition in that way.
It seems a little wasteful to me though as you are returning "everything" regardless of whether the "buyer" is present or not. So a "query" seems more logical than a projection:
db.movies.find({ "buyers": 2910 })
And optionally either just keeping only that matched result:
db.movies.find({ "buyers": 2910 },{ "buyers.$": 1})
Set operators in the aggregation framework give you more options with $project which can do more to alter the document. But if you just want to know if someone "bought" the item, then a "query" seems the be logical and fastest way to do so.