Mongo though provides 2dsphere index on legacy co-ordinates, the query requires to present to Point/Shapes in geoJSON format. For e.g., I have inserted the following records to address collection.
{ "city" : "First", "geo" : [ 13.45, 23.46 ] }
{ "city" : "Second", "geo" : [ 13.45, 20.46 ] }
Then I added 2dsphere index using following command as mongodb still allows 2dsphere index on legacy co-ordinates.
db.address.ensureIndex({"geo":"2dsphere"})
Then if I do $near query using legacy format, but got an exception.
> db.address.find({"geo":{$near:{"x":13.45,"y":23.45}}})
error: {
"$err" : "can't parse query (2dsphere): { $near: { x: 13.45, y: 23.45 } }",
"code" : 16535
}
But If do same query with geoJSON format, then I get result.
> db.address.find({"geo":{$near:{"type":"Point",coordinates:[13.45,23.45]}}})
{ "_id" : ObjectId("537306b4b8ac1f134d9efe89"), "city" : "First", "geo" : [ 13.45, 23.46 ] }
{ "_id" : ObjectId("537306c3b8ac1f134d9efe8a"), "city" : "Second", "geo" : [ 13.45, 20.46 ] }
My question is, GeoConverters has all conversion made to legacy format. So, obviously they wont' work if I use 2dsphere index. Are there any converts available for geoJSON format. Is there any workaround?
Currently spring-data-mongo doesn't support the new mongo (> 2.4) 2dsphere indexes. There is a open issue on Jira about it:
https://jira.spring.io/browse/DATAMONGO-1113?jql=project%20%3D%20DATAMONGO%20AND%20text%20~%20%22%24geometry%22
In the link you can find a gist link to example of how create such converters. You can use it or you can overcome this limitation creating a #Query with the query that you want that spring-data-mongo execute.
Regards.
avaz
Related
Is there any way to use MongoDB with Spark for geospatial queries? I cannot see how to do that with Stratio.
There are many ways to query geospatial data from spark. Use magellan https://github.com/harsha2010/magellan or hive esri geospatial toolkit. https://github.com/Esri/spatial-framework-for-hadoop
I've never tried the mongo librairie from stratio, but with the spark data source api, or the mongo connector, I think you can run geo queries with the mongo syntax then convert them into an RDD or a Dataframe.
You can query MongoDB from Spark SQL using this library.
MongoDB allows applications to do the following types of query on geospatial data: inclusion, intersection, proximity.
Obviously, you will be able to use all the other operator in addition to the geospatial ones. Let's now look at some concrete examples.
Here is an example:
Find all the airports in California. For this you need to get the California location (Polygon) and use the command $geoWithin in the query. From the shell it will look like :
use geo
var cal = db.states.findOne( {code : "CA"} );
db.airports.find(
{
loc : { $geoWithin : { $geometry : cal.loc } }
},
{ name : 1 , type : 1, code : 1, _id: 0 }
);
The Result:
{ "name" : "Modesto City - County", "type" : "", "code" : "MOD" }
...
{ "name" : "San Francisco Intl", "type" : "International", "code" : "SFO" }
{ "name" : "San Jose International", "type" : "International", "code" : "SJC" }
If you would like to try other example, check out this blog post here.
I have some documents with a "loc" field that looks like this:
mongos> db.foo.findOne()
{
// ... some other stuff
"loc" : {
"type" : "Point",
"coordinates" : [
-83.362342,
26.687779
]
}
}
It's indexed like so:
mongos> db.foo.getIndices()
[
// ... some other stuff
{
"v" : 1,
"key" : {
"loc" : "2dsphere"
},
"name" : "loc_2dsphere",
"ns" : "amitest.foo",
"2dsphereIndexVersion" : 2
},
]
According to the docs I should be able to query it like the following, but I get the error shown:
mongos> db.foo.find(
{loc :
{$near:
{$geometry:
{type: "Point",
coordinates: [-83.362342, 26.687779]
},
$maxDistance: 100
}
}
}
).limit(5)
error: { "$err" : "use geoNear command rather than $near query", "code" : 13501 }
I can successfully query the field using runCommand(), but that's not ideal because I can't combine it with other criteria:
mongos> db.runCommand(
{geoNear : "foo",
near : {type : "Point",
coordinates : [-83.362342, 26.687779]
},
spherical : true,
maxDistance : 10,
limit : 5
}
)
I've got MongoDB 2.6.0, on a hash-sharded collection with 8 shards.
In the geospatial index documentation, it states that: "For sharded collections, queries using $near are not supported. You can instead use either the geoNear command or the $geoNear aggregation stage."
There is a bug related to this, https://jira.mongodb.org/browse/SERVER-926, which is marked as closed and targets a future release of MongoDB, 1.7.2, which will enable the $near operator to be used with sharded collections. However, note, there is a big but which is that near queries will be routed to all shards via mongos, which is likely to be very inefficient. This is because sharding is not supported on a geo column in general, see the related and still open bug https://jira.mongodb.org/browse/SERVER-1982
This, in turn, is related to to the fact that MongoDB uses geohashing to convert two dimensional geographical objects to something that can be both indexed using a B-tree and something that could, in theory, be used as a shard key. It is a difficult problem to fix, as with geohash indexes, objects very close together, can end up with hash values very far apart, and so it is difficult to design something that is both appropriate to use as a shard key, but also supports efficient geospatial querying such as $near. See the limitations section in the Wikipedia geohash article for more on potential issues with geohashing.
I have a collection having from and to point locations. Now I wish to find documents which have both, to and from locations nearby the given source and destinations.
Here's the setup:
collection: db.t2.find():
{
"_id" : ObjectId("5..4"),
"uid" : "sdrr",
"valid_upto": 122334,
"loc" : {
"from" : {
"type" : "Point",
"coordinates" : [ 77.206672, 28.543347 ]
},
"to" : {
"type" : "Point",
"coordinates" : [ 77.1997687, 28.5567278 ]
}
}
}
Indices: db.t2.getIndices():
{
"v" : 1,
"name" : "_id_",
"key" : {
"_id" : 1
},
"ns" : "mydb.t2"
},
{
"v" : 1,
"name" : "uid_1_loc.from_2dsphere_loc.to_2dsphere_valid_upto_1",
"key" : {
"uid" : 1,
"loc.from" : "2dsphere",
"loc.to" : "2dsphere",
"valid_upto" : 1
},
"ns" : "mydb.t2"
}
Single queries for either to or from work good with the current settings give nice results. However, when I use to and from together in a single query with $and clause:
db.t2.find({
"$and" : [
{
"loc.from" : {
"$nearSphere" : [ 77.5454589,28.4621213 ],
"$maxDistance" : 0.18
}
},
{
"loc.to" : {
"$nearSphere" : [ 77.206672, 28.543347 ],
"$maxDistance" : 0.18
}
}
]
})
it throws the following error:
error: {
"$err" : "can't find any special indices: 2d (needs index), 2dsphere (needs index), for: { $and: [ { loc.from: { $nearSphere: [ 77.5454589, 28.4621213 ], $maxDistance: 0.18 } }, { loc.to: { $nearSphere: [ 77.206672, 28.543347 ], $maxDistance: 0.18 } } ] }",
"code" : 13038
}
I suppose the data has been indexed as evident from getIndices(), but still its unable to find indices! Where is the problem then and how can I fix it to have effect of a $and-ed operation?
The error appears to be present from a MongoDB 2.4 version where there indeed was a bug that would not allow a $near type of query within and $and operation that accessed another field.
But your particular problem here is that you just cannot do this.
The code and comments to test this can be vied on GitHub but essentially:
// There can only be one NEAR. If there is a NEAR, it must be either the root or the root
// must be an AND and its child must be a NEAR.
size_t numGeoNear = countNodes(root, MatchExpression::GEO_NEAR);
if (numGeoNear > 1) {
return Status(ErrorCodes::BadValue, "Too many geoNear expressions");
}
So that is an error that would be emitted from MongoDB 2.6 you tried to do this.
A brief look at all the surrounding code within the method will show you that "geo" queries are not alone in this and the other "special" index type of "text" is included in the same rules.
Part of the reason for this is the $meta "scoring" that is required, as in this case is $maxDistance. There really is no valid way to combine or discern which value would actually apply in combined results such as this.
On a bit more of a technical note, the other issue is with being able to "intersect" indexes in a query such as this. The required fuzzy matching makes this a very different prospect to something like the basic "Btree" index intersection.
For now at least, your best approach is to perform each query by itself and manually "union/intersect" your results in code, with of course your own tagging as to which results are for your origin and which are for your destination.
This was a known issue in version 2.4 and prior of MongoDB, fixed in version 2.5.5:
https://jira.mongodb.org/browse/SERVER-4572
Core ServerSERVER-4572 Geospatial index cannot be used in $and
criteria of a query?
Should be fixed as of 2.6 - if you're running 2.4 or previous I'd upgrade, if you're running 2.6.X I'd report it as a bug.
Here is the output from my mongodb shell of a very simple example of a $geoWithin query. As you can see, I have only a single GeoJson Polygon in my collection, and each of its coordinates lies within the described $box. Furthermore, the GeoJson seems valid, as the 2dsphere index was created without error.
> db.Townships.find()
{ "_id" : ObjectId("5310f13c9f3a313af872530c"), "geometry" : { "type" : "Polygon", "coordinates" : [ [ [ -96.74084500000001, 36.99911500000002 ], [ -96.74975600000002, 36.99916100000001 ], [ -96.74953099999998, 36.99916000000002 ], [ -96.74084500000001, 36.99911500000002 ] ] ] }, "type" : "Feature" }
> db.Townships.ensureIndex( { "geometry" : "2dsphere"})
> db.Townships.find( { "geometry" : { $geoWithin : { "$box" : [[-97, 36], [-96, 37]] } } } ).count()
0
Thanks for any advice.
From documentation:
The $box operator specifies a rectangle for a geospatial $geoWithin query. The query returns documents that are within the bounds of the rectangle, according to their point-based location data. The $box operator returns documents based on grid coordinates and does not query for GeoJSON shapes.
If you insert this document...
db.Townships.insert(
{ "geometry" : [ -96.74084500000001, 36.99911500000002 ],
"type" : "Feature"
})
...your query will found it (but without index support).
I have a collection 'place' ,one documentis as below
{
"_id" : ObjectId("52401a7267778834a23a54a2"),
"userid" : "123",
"loc" : {
"lng" : 77.6166685,
"lat" : 12.9361732
},
"t" : ISODate("2013-04-23T10:39:46.540Z")
}
I want to find out locations of user with userid 234 and loc near to [77.6166685,12.9361732] within 1km
but the below query is not working
db.place.find({{"userid":"234","loc":{"$near":{"$geometry":{"type":"point","coordinates":[77.6166685,12.9361732]},"$maxDistance":1000}}})
it showing error as given below
error: {
"$err" : "point not in interval of [ -180, 180 ] :: caused by :: { 0: 0.0, 1: 250.0 }",
"code" : 16433
}
what is this error and how can I correct it?
I think that the problem is the following:
You document has a form which is applicable for "2d" indexes, (you have not told what kind of indexes do you have) and you are trying to query it with a query which is applicable for "2dsphere" indexes. So you have two options:
convert the document to GeoJSON format (here you are not specifying geometry), put 2dsphere index and to query it in this way
do nothing and to query it with a correct way with "2d" indexes
I have done in the second way:
db.d.find({
"loc":{
"$near": [77.6166685,12.9361732],
"$maxDistance":1000
}
})
and got a correct result.