$and with $nearSphere in mongodb - mongodb

I have a collection having from and to point locations. Now I wish to find documents which have both, to and from locations nearby the given source and destinations.
Here's the setup:
collection: db.t2.find():
{
"_id" : ObjectId("5..4"),
"uid" : "sdrr",
"valid_upto": 122334,
"loc" : {
"from" : {
"type" : "Point",
"coordinates" : [ 77.206672, 28.543347 ]
},
"to" : {
"type" : "Point",
"coordinates" : [ 77.1997687, 28.5567278 ]
}
}
}
Indices: db.t2.getIndices():
{
"v" : 1,
"name" : "_id_",
"key" : {
"_id" : 1
},
"ns" : "mydb.t2"
},
{
"v" : 1,
"name" : "uid_1_loc.from_2dsphere_loc.to_2dsphere_valid_upto_1",
"key" : {
"uid" : 1,
"loc.from" : "2dsphere",
"loc.to" : "2dsphere",
"valid_upto" : 1
},
"ns" : "mydb.t2"
}
Single queries for either to or from work good with the current settings give nice results. However, when I use to and from together in a single query with $and clause:
db.t2.find({
"$and" : [
{
"loc.from" : {
"$nearSphere" : [ 77.5454589,28.4621213 ],
"$maxDistance" : 0.18
}
},
{
"loc.to" : {
"$nearSphere" : [ 77.206672, 28.543347 ],
"$maxDistance" : 0.18
}
}
]
})
it throws the following error:
error: {
"$err" : "can't find any special indices: 2d (needs index), 2dsphere (needs index), for: { $and: [ { loc.from: { $nearSphere: [ 77.5454589, 28.4621213 ], $maxDistance: 0.18 } }, { loc.to: { $nearSphere: [ 77.206672, 28.543347 ], $maxDistance: 0.18 } } ] }",
"code" : 13038
}
I suppose the data has been indexed as evident from getIndices(), but still its unable to find indices! Where is the problem then and how can I fix it to have effect of a $and-ed operation?

The error appears to be present from a MongoDB 2.4 version where there indeed was a bug that would not allow a $near type of query within and $and operation that accessed another field.
But your particular problem here is that you just cannot do this.
The code and comments to test this can be vied on GitHub but essentially:
// There can only be one NEAR. If there is a NEAR, it must be either the root or the root
// must be an AND and its child must be a NEAR.
size_t numGeoNear = countNodes(root, MatchExpression::GEO_NEAR);
if (numGeoNear > 1) {
return Status(ErrorCodes::BadValue, "Too many geoNear expressions");
}
So that is an error that would be emitted from MongoDB 2.6 you tried to do this.
A brief look at all the surrounding code within the method will show you that "geo" queries are not alone in this and the other "special" index type of "text" is included in the same rules.
Part of the reason for this is the $meta "scoring" that is required, as in this case is $maxDistance. There really is no valid way to combine or discern which value would actually apply in combined results such as this.
On a bit more of a technical note, the other issue is with being able to "intersect" indexes in a query such as this. The required fuzzy matching makes this a very different prospect to something like the basic "Btree" index intersection.
For now at least, your best approach is to perform each query by itself and manually "union/intersect" your results in code, with of course your own tagging as to which results are for your origin and which are for your destination.

This was a known issue in version 2.4 and prior of MongoDB, fixed in version 2.5.5:
https://jira.mongodb.org/browse/SERVER-4572
Core ServerSERVER-4572 Geospatial index cannot be used in $and
criteria of a query?
Should be fixed as of 2.6 - if you're running 2.4 or previous I'd upgrade, if you're running 2.6.X I'd report it as a bug.

Related

$near operator on a nested object list item

I have mongo db with a collection of objects with a nested object list (events) like this:
{
"_id" : ObjectId("59db84093f2fba2bf0bcfa90"),
"progressStatus" : "NOT_STARTED",
"events" : [
{
"issueDate" : ISODate("2017-10-09T00:00:00.000Z"),
"eventType" : "xyz",
"location" : {
"point" : {
"type" : "Point",
"coordinates" : [
25.6011977000001,
45.6579755
]
}
},
"cancelled" : false,
}
]
}
Trying to make a query using $near or $nearSphere operator on events.$.location:
{
"events":{
"$elemMatch":{
"eventType":"xyz",
"$and":[
{
"cancelled":false
},
{
"location.point":{
"$nearSphere":{
"$geometry":{
type:"Point",
coordinates:[
25.601198,
45.657976
]
},
"$maxDistance":20.4
}
}
}
]
}
}
}
This query gives me an error:
Error: error: {
"waitedMS" : NumberLong(0),
"ok" : 0,
"errmsg" : "geoNear must be top-level expr",
"code" : 2
}
How should it be done?
neptune, this happens because geospatial queries with $nearSphere uses geoNear command to fetch documents, and the geoNear expression (when the command is run) needs to be at the top level on query. In your example, it happens to be at low levels (within elemMatch), raising an error when constructing the call to geoNear.
As I understand, your 2dsphere index is on "events.location.point", right ?
So, you can make this query like:
db.sample6.find({"events": {$elemMatch: {$and: [{eventType: "xyz"},{cancelled: false}]}}, "events.location.point": {"$nearSphere": {"$geometry": {type: "Point", coordinates: [25.601198, 45.657976]}, "$maxDistance": 20.4}}})
or just
db.sample6.find({"events.eventType": "xyz", "events.cancelled": false, "events.location.point": {"$nearSphere": {"$geometry": {type: "Point", coordinates: [25.601198, 45.657976]}, "$maxDistance": 20.4}}})
This last one I think is more wise, since it has already AND behaviour, without the need of $and use. Also, if it's possible, it would be cleaner and less confusing (since these queries have a lot of brackets and parentheses) to construct the geospatial info right on "location" field (if it only contains the coordinates).

Mongodb $near returning error

I have a basic document, like so:
{
"_id" : ObjectId("5760fe623f6d3ad25e387ffc"),
"type": 5,
"product" : {
"location" : {
"geometry" : [ 153.39999999999998, -28.016667 ],
"name" : "Gold Coast QLD, Australia",
"id" : "ChIJt2BdK0cakWsRcK_e81qjAgM"
}
}
}
I am trying to query the location using the $near method provided by Mongodb.
This is my query:
db.posts.find({
'product.location.geometry': {
$near: [ 153.39999999999998, -28.016667 ]
}
})
Within the Mongodb documentation, it states that:
To specify a point using legacy coordinates, $near requires a 2d index
and has the following syntax:
{
$near: [ <x>, <y> ],
$maxDistance: <distance in radians>
}
It even gives this example on their site:
db.legacy2d.find({
location : { $near : [ -73.9667, 40.78 ], $maxDistance: 0.10 }
})
This is the error it is producing:
Error: error: {
"waitedMS" : NumberLong(0),
"ok" : 0,
"errmsg" : "error processing query: ns=mytestnodedb.postsTree: GEONEAR field=product.location.geometry maxdist=1.79769e+308 isNearSphere=0\nSort: {}\nProj: {}\n planner returned error: unable to find index for $geoNear query",
"code" : 2
}
I am unable to identify anything that is wrong with my query. Mongo states that the $near must be longitude followed by latitude, which I am definitely doing. I am purposefully leaving out $maxDistance since Mongo states that it will return results sorted from nearest to farthest.
Well error is pretty much self explainatory. Query requires 2d index which it can't find.
I'd create index as:
db.collection.createIndex({"product.location.geometry":"2d"})
Now if I run your query on sample data, I get
{
"_id" : ObjectId("5760fe623f6d3ad25e387ffc"),
"type" : 5.0,
"product" : {
"location" : {
"geometry" : [
153.39999999999998,
-28.016667
],
"name" : "Gold Coast QLD, Australia",
"id" : "ChIJt2BdK0cakWsRcK_e81qjAgM"
}
}
}

Is there a way to query an embedded document in an embedded document?

I have a weird mongodb document, but still need to query on it. Is it possible?
For example: I need every player within a certain radius.
{
"_id" : ObjectId("55d89c63c746230c200c528e"),
"speler_id" : 12,
"naam" : "Arjen Robben",
"seconds" : [
[
{
"locatie" : [
8.7173307286181370,
33.2784843816214250
],
"timestamp" : ISODate("1970-01-01T19:00:01.000Z")
},
{
"locatie" : [
-45.8853075448968970,
138.1526615469845800
],
"timestamp" : ISODate("1970-01-01T19:00:02.000Z")
},
{
"locatie" : [
80.5503710377444690,
10.0500048843973580
],
"timestamp" : ISODate("1970-01-01T19:00:03.000Z")
}
]
]
}
Well you can always use $geoWithin with $center or $centerSphere ( depending on whether these are global geometry coordinates or just a flat plane, for distance caluation purposes ) after processing with $unwind in the aggregation framework:
db.collection.aggregate([
{ "$unwind": "$seconds" },
{ "$unwind": "$seconds" },
{ "$match": {
"seconds.locatie": {
"$geoWithin": {
"$centerSphere": [
[
8.7173307286181370,
33.2784843816214250
],
100
]
}
}
}}
])
Which on the presented data would return:
{
"_id" : ObjectId("55d89c63c746230c200c528e"),
"speler_id" : 12,
"naam" : "Arjen Robben",
"seconds" : {
"locatie" : [
8.717330728618137,
33.278484381621425
],
"timestamp" : ISODate("1970-01-01T19:00:01Z")
}
}
{
"_id" : ObjectId("55d89c63c746230c200c528e"),
"speler_id" : 12,
"naam" : "Arjen Robben",
"seconds" : {
"locatie" : [
80.55037103774447,
10.050004884397358
],
"timestamp" : ISODate("1970-01-01T19:00:03Z")
}
}
Since $geoWithin does not "require" a geospatial index, then this is fine to use at later aggregation stages than the initial match. The $centerSphere in this case defines a point to query from and the "radius" extending from that point. This is just really a geometery "shortcut" as you can alternately provdide a GeoJSON polygon of your own definition.
But it's not really great. And mostly because it will not be able to use an index and therefore is pretty much brute force over the whole collection, and in that you cannot do nice things like return the distance from the queried point, like you can do with $geoNear.
Therefore while you can do things like this, most geoSpatial queries with MongoDB are best left to keeping that location data at the top level of the document, rather than embedded within arrays. So such modelling usually means having separate collection objects rather than embedded ones for the best results.
If you want an aggregated array in your response, then it is better to do this in aggregation after the intial geospatial query is made.

Error in mongodb query to get movie based on id

> db.movmodels.findOne()
{
"_id" : ObjectId("55320b0e0e9e0d9d0540593c"),
"username" : "punk",
"favMovies" : [
{
"alternate_ids" : {
"imdb" : "0137523"
},
"abridged_cast" : [
{
"characters" : [
"Tyler"
],
"id" : "162652627",
"name" : "Brad Pitt"
},
{
"characters" : [
"Narrator"
],
"id" : "162660884",
"name" : "Edward Norton"
},
{
"characters" : [
"Robert"
],
"id" : "162676383",
"name" : "Meat Loaf"
},
{
"characters" : [
"Angel Face"
],
"id" : "162653925",
"name" : "Jared Leto"
},
{
"characters" : [
"Boss"
],
"id" : "770706064",
"name" : "Zach Grenier"
}
],
"synopsis" : "",
"ratings" : {
"audience_score" : 96,
"audience_rating" : "Upright",
"critics_score" : 80,
"critics_rating" : "Certified Fresh"
},
"release_dates" : {
"dvd" : "2000-06-06",
"theater" : "1999-10-15"
},
"critics_consensus" : "",
"runtime" : 139,
"mpaa_rating" : "R",
"year" : 1999,
"title" : "Fight Club",
**"id" : "13153"**
}
],
"__v" : 0
}
This is my data in mongodb.
As I am new to mongodb I wanted to know query to get movie with a particular id.
The query that I tried is. I need to get the movie based on id so that I can remove it from my database
db.movmodels.findOne({username:"punk"},{favMovies:{id:13153}})
but this gives me error.
2015-04-18T05:41:26.221-0400 E QUERY Error: error: {
"$err" : "Can't canonicalize query: BadValue ported projection option: favMovies: { id: 13153.0 }",
"code" : 17287
}
at Error (<anonymous>)
at DBQuery.next (src/mongo/shell/query.js:259:15)
at DBCollection.findOne (src/mongo/shell/collection.js:188:22)
at (shell):1:14 at src/mongo/shell/query.js:259
There are several problems with your query:
The second parameter to find() is a projection, not part of the query. What you want is to supply one document for the query that has two properties: {"username" : "punk", favMovies : { ... } }
However, you also don't want to compare the entire sub-document favMovies, but you only want to match on one of its properties, the id, which requires to 'reach into the object' using the dot operator: {username:"punk", "favMovies.id" : 13153}.
However, that will probably not work yet, because 13153 is not the same as "13153", the latter being a string while the former is a number in JSON.
db.movmodels.findOne({username:"punk", "favMovies.id" : "13153"})
Keep in mind, however, that this will find the entire document for the user named "punk". I'm not sure what exactly your data structure should look like, but it appears you'll have to $pull the movie from the user. In general, I'd say you're embedding too much data into the user, but that's hard to tell without knowing the exact use case.
Here you go:
If you just wanted to get first user who has this fav movie:
db.movmodels.findOne({"favMovies.id": 13153});
And, if you want to know if that user has that movie as favorite.
db.movmodels.findOne({"favMovies.id": 13153, username:"punk"});
Second argument in the findOne is used to only return particular field.
You can use also $elemMatch projection operator (not to be confused with the $elemMatch query operator)
db.movmodels.find({username:"punk"},{favMovies:{$elemMatch:{id:"13153"}}});
`
If you want to find a movie that has another movie (with id 13153) in 'favMovies' array, then write the query as below:
db.movmodels.findOne({username:"punk",'favMovies.id':13153})
And if you want to find a movie with _id 55320b0e0e9e0d9d0540593cwrite the following query:
db.movmodels.findOne({username:"punk",'_id':ObjectId("55320b0e0e9e0d9d0540593c")})

Resolving MongoDB DBRef array using Mongo Native Query and working on the resolved documents

My MongoDB collection is made up of 2 main collections :
1) Maps
{
"_id" : ObjectId("542489232436657966204394"),
"fileName" : "importFile1.json",
"territories" : [
{
"$ref" : "territories",
"$id" : ObjectId("5424892224366579662042e9")
},
{
"$ref" : "territories",
"$id" : ObjectId("5424892224366579662042ea")
}
]
},
{
"_id" : ObjectId("542489262436657966204398"),
"fileName" : "importFile2.json",
"territories" : [
{
"$ref" : "territories",
"$id" : ObjectId("542489232436657966204395")
}
],
"uploadDate" : ISODate("2012-08-22T09:06:40.000Z")
}
2) Territories, which are referenced in "Map" objects :
{
"_id" : ObjectId("5424892224366579662042e9"),
"name" : "Afghanistan",
"area" : 653958
},
{
"_id" : ObjectId("5424892224366579662042ea"),
"name" : "Angola",
"area" : 1252651
},
{
"_id" : ObjectId("542489232436657966204395"),
"name" : "Unknown",
"area" : 0
}
My objective is to list every map with their cumulative area and number of territories. I am trying the following query :
db.maps.aggregate(
{'$unwind':'$territories'},
{'$group':{
'_id':'$fileName',
'numberOf': {'$sum': '$territories.name'},
'locatedArea':{'$sum':'$territories.area'}
}
})
However the results show 0 for each of these values :
{
"result" : [
{
"_id" : "importFile2.json",
"numberOf" : 0,
"locatedArea" : 0
},
{
"_id" : "importFile1.json",
"numberOf" : 0,
"locatedArea" : 0
}
],
"ok" : 1
}
I probably did something wrong when trying to access to the member variables of Territory (name and area), but I couldn't find an example of such a case in the Mongo doc. area is stored as an integer, and name as a string.
I probably did something wrong when trying to access to the member variables of Territory (name and area), but I couldn't find an example
of such a case in the Mongo doc. area is stored as an integer, and
name as a string.
Yes indeed, the field "territories" has an array of database references and not the actual documents. DBRefs are objects that contain information with which we can locate the actual documents.
In the above example, you can clearly see this, fire the below mongo query:
db.maps.find({"_id":ObjectId("542489232436657966204394")}).forEach(function(do
c){print(doc.territories[0]);})
it will print the DBRef object rather than the document itself:
o/p: DBRef("territories", ObjectId("5424892224366579662042e9"))
so, '$sum': '$territories.name','$sum': '$territories.area' would show you '0' since there are no fields such as name or area.
So you need to resolve this reference to a document before doing something like $territories.name
To achieve what you want, you can make use of the map() function, since aggregation nor Map-reduce support sub queries, and you already have a self-contained map document, with references to its territories.
Steps to achieve:
a) get each map
b) resolve the `DBRef`.
c) calculate the total area, and the number of territories.
d) make and return the desired structure.
Mongo shell script:
db.maps.find().map(function(doc) {
var territory_refs = doc.territories.map(function(terr_ref) {
refName = terr_ref.$ref;
return terr_ref.$id;
});
var areaSum = 0;
db.refName.find({
"_id" : {
$in : territory_refs
}
}).forEach(function(i) {
areaSum += i.area;
});
return {
"id" : doc.fileName,
"noOfTerritories" : territory_refs.length,
"areaSum" : areaSum
};
})
o/p:
[
{
"id" : "importFile1.json",
"noOfTerritories" : 2,
"areaSum" : 1906609
},
{
"id" : "importFile2.json",
"noOfTerritories" : 1,
"areaSum" : 0
}
]
Map-Reduce functions should not be and cannot be used to resolve DBRefs in the server side.
See what the documentation has to say:
The map function should not access the database for any reason.
The map function should be pure, or have no impact outside of the
function (i.e. side effects.)
The reduce function should not access the database, even to perform
read operations. The reduce function should not affect the outside
system.
Moreover, a reduce function even if used(which can never work anyway) will never be called for your problem, since a group w.r.t "fileName" or "ObjectId" would always have only one document, in your dataset.
MongoDB will not call the reduce function for a key that has only a
single value