A 5,000 to 10,000 record Mongo collection contains:
{
"_id" : ObjectId("55e16c34c78b04f43f2f55a0"),
"appID" : NumberInt(4830800),
"topics" : {
"test1" : 1.440899998865E12,
"test2" : 1.440899998865E12,
"test3" : 1.440899998865E12,
"test4" : 1.440899998865E12
},
}
I need to query for records that contain a specified property name in the topics field and where the value of the specified name is greater than or equal to a given number.
something like
find({"topics.test1": { $gte: 1440825382535 }})
This query works as expected, returning a set of records that have a test1 property with a test1 value >= 1440825382535
If I create a simple index on the topics field explain() says that no index is used for the query (understandably).
The set of property names that may be searched for is not predefined. The query is dynamically built based on names that are found elsewhere.
Is there a way to index this table to speed up queries? The full scan query takes quite a bit of time to run (on the order of 1.5 seconds).
To make this type of data indexable, you need to change the schema to make topics an array and move the dynamic test1, test2, etc. keys into values.
So something like:
{
"_id" : ObjectId("55e16c34c78b04f43f2f55a0"),
"appID" : NumberInt(4830800),
"topics" : [
{name: "test1", value: 1.440899998865E12},
{name: "test2", value: 1.440899998865E12},
{name: "test3", value: 1.440899998865E12},
{name: "test4", value: 1.440899998865E12}
]
}
Then your query changes to:
find({topics: {$elemMatch: {name: 'test1', value: {$gte: 1440825382535}}}})
Which you can support with an index of:
{'topics.name': 1, 'topics.value': 1}
Was a little confused at what you were trying to do, but maybe something like this?
find({"topics.test1": {$exists: true}, { $gte: 1440825382535 }})
Related
I am new to Mongodb. I need some input on following scenario.
I have a document { "field1" : "abc", "field2": "xyz", "testId" : "test123" }
On every transaction, say we receive 1000 such documents and out of these there may be some new documents, say 100 (means there is no entry for these 100 documents).
Now I have 2 approaches to update these documents.
Approach 1: Query for each document to find if it has a db entry else add it to db. (we can use upsert method to do this).
Approach 2: Remove all documents which matches particular field value and insert all 1000 documents as new entries using SaveAll.
Can some suggest the best approach on this.
Add a unique index, do a bulk insert and ignore any errors:
db.foo.createIndex({ "testId": 1 }, { unique: true })
See https://docs.mongodb.com/manual/core/index-unique/.
db.runCommand({
insert: "foo",
documents: [
{ "field1" : "abc", "field2": "xyz", "testId" : "test123" },
...
],
ordered: false // continue despite errors
})
See https://docs.mongodb.com/manual/reference/command/insert/.
Im trying to get back into Mongodb and Ive come across something that I cant figure out.
I have this data structure
> db.ratings.find().pretty()
{
"_id" : ObjectId("55881e43424cbb1817137b33"),
"e_id" : ObjectId("5565e106cd7a763b2732ad7c"),
"type" : "like",
"time" : 1434984003156,
"u_id" : ObjectId("55817c072e48b4b60cf366a7")
}
{
"_id" : ObjectId("55893be1e6a796c0198e65d3"),
"e_id" : ObjectId("5565e106cd7a763b2732ad7c"),
"type" : "dislike",
"time" : 1435057121808,
"u_id" : ObjectId("55817c072e48b4b60cf366a7")
}
{
"_id" : ObjectId("55893c21e6a796c0198e65d4"),
"e_id" : ObjectId("5565e106cd7a763b2732ad7c"),
"type" : "null",
"time" : 1435057185089,
"u_id" : ObjectId("55817c072e48b4b60cf366a7")
}
What I want to be able to do is count the documents that have either a like or dislike leaving the "null" out of the count. So I should have a count of 2. I tried to go about it like this whereby I set the query to both fields:
db.ratings.find({e_id: ObjectId("5565e106cd7a763b2732ad7c")}, {type: "like", type: "dislike"})
But this just prints out all three documents. Is there any reason?
If its glaringly obvious im sorry pulling out my hair at the moment.
Use the following db.collection.count() method which returns the count of documents that would match a find() query:
db.ratings.count({
"e_id": ObjectId("5565e106cd7a763b2732ad7c"),
type: {
"$in": ["like", "dislike"]
}
})
The db.collection.count() method is equivalent to the db.collection.find(query).count() construct. Your query selection criteria above can be interpreted as:
Get me the count of all documents which have the e_id field values as ObjectId("5565e106cd7a763b2732ad7c") AND the type field which has either value "like" or "dislike", as depicted by the $in operator that selects the documents where the value of a field equals any value in the specified array.
db.ratings.find({e_id: ObjectId("5565e106cd7a763b2732ad7c")},
{type: "like", type: "dislike"})
But this just prints out all three
documents. Is there any reason? If its glaringly obvious im sorry
pulling out my hair at the moment.
The second argument here is the projection used by the find method . It specifies fields that should be included -- regardless of their value. Normally, you specify a boolean value of 1 or true to include the field. Obviously, MongoDB accepts other values as true.
If you only need to count documents, you should issue a count command:
> db.runCommand({count: 'collection',
query: { "e_id" : ObjectId("5565e106cd7a763b2732ad7c"),
type: { $in: ["like", "dislike"]}}
})
{ "n" : 2, "ok" : 1 }
Please note the Mongo Shell provides the count helper for that:
> db.collection.find({ "e_id" : ObjectId("5565e106cd7a763b2732ad7c"),
type: { $in: ["like", "dislike"]}}).count()
2
That being said, to quote the documentation, using the count command "can result in an inaccurate count if orphaned documents exist or if a chunk migration is in progress." To avoid that, you might prefer using the aggregation framework:
> db.collection.aggregate([
{ $match: { "e_id" : ObjectId("5565e106cd7a763b2732ad7c"),
type: { $in: ["like", "dislike"]}}},
{ $group: { _id: null, n: { $sum: 1 }}}
])
{ "_id" : null, "n" : 2 }
This query should solve your problem
db.ratings.find({$or : [{"type": "like"}, {"type": "dislike"}]}).count()
I have mongodb document like
{
"_id" : ObjectId("543d563bde1e58511c264340"),
...some fields ...
"pref" : [
{
"user_id" : 1,
"value" : 0.56
}
]
}
How can I find all the documents where pref does not contain an entry with user_id :1 ?
It's a little unclear what you're looking for here. If you want to find all entries where user_id has any other value than '1', then you'd want:
db.collection.find({"pref.user_id": {'$ne': 1}})
If you're looking for documents where the 'user_id' field doesn't exist at all:
db.collection.find({"pref.user_id": {'$exists': 0}})
Keep in mind, though the behavior of both of these queries on a nested array. What you're actually going to get is all the documents where any of the objects in the 'pref' array matches the specified condition.
I am having following document in mongodb
{
"_id" : ObjectId("517b88decd483543a8bdd95b"),
"studentId" : 23,
"students" : [
{
"id" : 23,
"class" : "a"
},
{
"id" : 55,
"class" : "b"
}
]
}
{
"_id" : ObjectId("517b9d05254e385a07fc4e71"),
"studentId" : 55,
"students" : [
{
"id" : 33,
"class" : "c"
}
]
}
Note: Not an actual data but schema is exactly same.
Requirement: Finding the document which matches the studentId and students.id(id inside the students array using single query.
I have tried the code like below
db.data.aggregate({$match:{"students.id":"$studentId"}},{$group:{_id:"$student"}});
Result: Empty Array, If i replace {"students.id":"$studentId"} to {"students.id":33} it is returning the second document in the above shown json.
Is it possible to get the documents for this scenario using single query?
If possible, I'd suggest that you set the condition while storing the data so that you can do a quick truth check (isInStudentsList). It would be super fast to do that type of query.
Otherwise, there is a relatively complex way of using the Aggregation framework pipeline to do what you want in a single query:
db.students.aggregate(
{$project:
{studentId: 1, studentIdComp: "$students.id"}},
{$unwind: "$studentIdComp"},
{$project : { studentId : 1,
isStudentEqual: { $eq : [ "$studentId", "$studentIdComp" ] }}},
{$match: {isStudentEqual: true}})
Given your input example the output would be:
{
"result" : [
{
"_id" : ObjectId("517b88decd483543a8bdd95b"),
"studentId" : 23,
"isStudentEqual" : true
}
],
"ok" : 1
}
A brief explanation of the steps:
Build a projection of the document with just studentId and a new field with an array containing just the id (so the first document it would contain [23, 55].
Using that structure, $unwind. That creates a new temporary document for each array element in the studentIdComp array.
Now, take those documents, and create a new document projection, which continues to have the studentId and adds a new field called isStudentEqual that compares the equality of two fields, the studentId and studentIdComp. Remember that at this point there is a single temporary document that contains those two fields.
Finally, check that the comparison value isStudentEqual is true and return those documents (which will contain the original document _id and the studentId.
If the student was in the list multiple times, you might need to group the results on studentId or _id to prevent duplicates (but I don't know that you'd need that).
Unfortunately it's impossible ;(
to solve this problem it is necessary to use a $where statement
(example: Finding embeded document in mongodb?),
but $where is restricted from being used with aggregation framework
db.data.find({students: {$elemMatch: {id: 23}} , studentId: 23});
I'm new to mongodb and am trying to query child objects. I have a collection of States, and each State has child Cities. One of the Cities has a Name property that is null, which is causing errors in my app. How would I query the State collections to find child Cities that have a name == null?
If it is exactly null (as opposed to not set):
db.states.find({"cities.name": null})
(but as javierfp points out, it also matches documents that have no cities array at all, I'm assuming that they do).
If it's the case that the property is not set:
db.states.find({"cities.name": {"$exists": false}})
I've tested the above with a collection created with these two inserts:
db.states.insert({"cities": [{name: "New York"}, {name: null}]})
db.states.insert({"cities": [{name: "Austin"}, {color: "blue"}]})
The first query finds the first state, the second query finds the second. If you want to find them both with one query you can make an $or query:
db.states.find({"$or": [
{"cities.name": null},
{"cities.name": {"$exists": false}}
]})
Assuming your "states" collection is like:
{"name" : "Spain", "cities" : [ { "name" : "Madrid" }, { "name" : null } ] }
{"name" : "France" }
The query to find states with null cities would be:
db.states.find({"cities.name" : {"$eq" : null, "$exists" : true}});
It is a common mistake to query for nulls as:
db.states.find({"cities.name" : null});
because this query will return all documents lacking the key (in our example it will return Spain and France). So, unless you are sure the key is always present you must check that the key exists as in the first query.