Error creating index for table 'MSParcels': WriteConcern detected an error 'Can't extract geo keys from object, malformed geometry?:
{ type: "Polygon", coordinates:
[ [ [ -122.118466012, 47.6511409501, 0.0 ],
[ -122.118687874, 47.6508529655, 0.0 ],
[ -122.118817718, 47.650852731, 0.0 ],
[ -122.118890754, 47.650852592, 0.0 ],
[ -122.118891979, 47.651140118, 0.0 ],
[ -122.118703033, 47.6511404878, 0.0 ],
[ -122.118466012, 47.6511409501, 0.0 ] ] ] }
Problem is, I'm copying from SQL Server where identical coordinates pass STIsValid
Using C# driver MongoDB.Driver.Builders.IndexKeys.GeoSpatialSpherical
Mongo version 2.4.4
Any advice?
The geojson isn't valid for mongodb- it only accepts x,y and not z coordinates (altitude). This is because it only has 2D indexing / querying capabilities.
You need to remove the z coordinates from the geojson document to be something like:
{ type: "Polygon", coordinates:
[ [ [ -122.118466012, 47.6511409501],
[ -122.118687874, 47.6508529655],
[ -122.118817718, 47.650852731],
[ -122.118890754, 47.650852592],
[ -122.118891979, 47.651140118],
[ -122.118703033, 47.6511404878],
[ -122.118466012, 47.6511409501] ] ] }
There has been a feature request to improve this - please vote for: SERVER-9220
Your coordinates invalid. A geojson polygon is a array of arrays of arrays with two coordinates not three (the additional 0.0)
Related
I want to store GeoJson data for an area using MongoDB. The data comes from an official website. Each area is represented as MultiPolygon. In the end, I want to find all areas that contain a lng/lat pairs using a $intersect like that:
db.areas.find({
"location.geometry": {
"$geoIntersects": {
"$geometry": {
"type": "Point",
"coordinates": [
<lng>,
<lat>
]
}
}
}
}
In principle, it seems to work just fine. However, I've encountered problems with some areas seemingly with respect to the set of polygons of a MultiPolygon. I could boil down my problem to an individual case:
An area (being a GeoJson MultiPolygon) has six polygons, say [A, B, C, D, E, F]. Also the point <lng>,<lat> I query for lies within polygon A. Now the query above only works if the area does not contain the polygons D and F (A has to be included always, of course) -- that is, I get the expected search result. Otherwise, the query is empty (but no error). In short
What works: [A], [A,B], [A,B,C], [A,B,C,E], [A,C], ... (any combination with A and without D & F)
What doesn't work: [A,D], [A,B,F], ... (any combination that contains D or F)
What is the problem with polygons D and F? Are they not allowed to overlap with other polygons in the MultiPolygon? Are they maybe too small? I've tried the GeoJson definition but couldn't see any issues. Could it be because the GeoJson support of MongoDB.
You are correct that without any special considerations, you can insert a Polygonor MultiPolygon into MongoDB that has deformed GeoJSON structure. This is because unless you specifically create a geo index on the field, MongoDB doesn't know it is GeoJSON at all. The geo engine will silently not match a target intersect geometry, much as it would if you pointed it at a simple scalar field like {"name":"buzz"}.
If you add an index thusly:
db.geo.createIndex({loc:"2dsphere"})
Then this will activate the geo-aware machinery and if you try to insert or update a deformed GeoJSON shape, it will produce an error (scroll to see the Loop not closed part):
{
"nMatched" : 0,
"nUpserted" : 0,
"nModified" : 0,
"writeError" : {
"code" : 16755,
"errmsg" : "Can't extract geo keys: { _id: 0.0, loc: { type: \"MultiPolygon\", coordinates: [ [ [ [ -83.0, 40.0 ], [ -83.0, 41.0 ], [ -82.0, 41.0 ], [ -82.0, 40.0 ], [ -83.0, 40.0 ] ] ], [ [ [ -93.0, 40.0 ], [ -93.0, 41.0 ], [ -92.0, 41.0 ], [ -92.0, 40.0 ], [ -93.0, 40.0 ] ] ], [ [ [ -73.0, 49.0 ], [ -72.0, 41.0 ], [ -72.0, 40.0 ], [ -73.0, 40.0 ], [ -73.0, 41.0 ] ] ] ] } } Loop is not closed: [ [ -73.0, 49.0 ], [ -72.0, 41.0 ], [ -72.0, 40.0 ], [ -73.0, 40.0 ], [ -73.0, 41.0 ] ]"
}
}
In other words, the geo index becomes the guard at the door and ensures that all shapes written are GeoJSON compliant. This is also why it is very useful to ensure the index is created before inserts and updates because trying to create a geo index on many docs with potentially 100s or 1000s of deformed shapes will lead to much tedious work trying to isolate and fix the bad shapes one at a time.
After some more digging, I figured out that the polygons causing the issues contained duplicate coordinates (apart from the first and last coordinate). Online GeoJson validator didn't raise an error, but it seems that MongoDB doesn't handle it.
After removing all duplicates, everything works fine -- at least I hope that removing duplicates alter the shape of the polygons too much (but that's not overly crucial for my case). It's just a bit unfortunate that MongoDB doesn't raise an error but simply returns an empty result.
I have ~400K documents in a mongo collection, all with geometry of type:Polygon. It is not possible to add a 2dsphere index to the data as it currently stands because the geometry apparently has self-intersections.
In the past we had a hacky workaround which was to compute the bounding box of the geometry on a mongoose save hook and then index that rather than the geometry itself, but we would like to simplify things and just use the actual geometry.
So far I have tried using turf as follows (this is the body of a function called fix):
let geom = turf.polygon(geometry.coordinates);
geom = turf.simplify(geom, { tolerance: 1e-7 });
geom = turf.cleanCoords(geom);
geom = turf.unkinkPolygon(geom);
geom = turf.combine(geom);
return geom.features[0].geometry;
The most important function there is the unkinkPolygons which I hoped would do exactly what I wanted, i.e. make the geometry nice enough to be indexed. The simplify is possibly not helpful but I added it in for good measure. The clean is there because unkink complained about its input, and the combine is there to turn an array of Polygons into a single MultiPolygon. Actually, unkink still wasn't happy with it's inputs, so I had to write a hacky function as follows that jitters duplicated vertices, this modifies the geom before passing to unkink:
function jitterDups(geom) {
let coords = geom.geometry.coordinates;
let points = new Set();
for (let ii = 0; ii < coords.length; ii++) {
// last coords is allowed to match first, not sure if it must match.
let endsMatch = coords[ii][0].join(",") === coords[ii][coords[ii].length - 1].join(",");
for (let jj = 0; jj < coords[ii].length - (endsMatch ? 1 : 0); jj++) {
let str = coords[ii][jj].join(",");
while (points.has(str)) {
coords[ii][jj][0] += 1e-8; // if you make this too small it doesn't do the job
if (jj === 0 && endsMatch) {
coords[ii][coords[ii].length - 1][0] = coords[ii][jj][0];
}
str = coords[ii][jj].join(",");
}
points.add(str);
}
}
}
However, even after all of that mongo still complains.
Here is some sample raw Polygon input:
{ type: "Polygon", coordinates: [ [ [ -0.027542009179339, 51.5122867222457 ], [ -0.027535822940572, 51.512281465421 ], [ -0.027535925691804, 51.5122814221859 ], [ -0.027589474043984, 51.5122605515771 ], [ -0.027638484531731, 51.5122996934574 ], [ -0.027682911101528, 51.5123351881505 ], [ -0.027689915350493, 51.5123872384419 ], [ -0.027672409315982, 51.5123868001613 ], [ -0.027667905522642, 51.5123866344944 ], [ -0.027663068941865, 51.5123864992013 ], [ -0.02764931654289, 51.512375566682 ], [ -0.027552504539425, 51.5122983194123 ], [ -0.027542009179339, 51.5122867222457 ] ], [ [ -0.027542009179339, 51.5122867222457 ], [ -0.027557948301911, 51.5122984109658 ], [ -0.027560309178214, 51.5123001412876 ], [ -0.027542009179339, 51.5122867222457 ] ] ] }
And that same data after it has passed through the above fixing pipeline:
{ type: "MultiPolygon", coordinates: [ [ [ [ -0.027560309178214, 51.5123001412876 ], [ -0.02754202882236209, 51.51228674396312 ], [ -0.027542009179339, 51.5122867222457 ], [ -0.027535822940572, 51.512281465421 ], [ -0.027589474043984, 51.5122605515771 ], [ -0.027682911101528, 51.5123351881505 ], [ -0.027689915350493, 51.5123872384419 ], [ -0.027663068941865, 51.5123864992013 ], [ -0.027552504539425, 51.5122983194123 ], [ -0.02754202884162257, 51.51228674398443 ], [ -0.027557948301911, 51.5122984109658 ], [ -0.027560309178214, 51.5123001412876 ] ] ], [ [ [ -0.02754202884162257, 51.51228674398443 ], [ -0.02754202882236209, 51.51228674396312 ], [ -0.027541999179339, 51.5122867222457 ], [ -0.02754202884162257, 51.51228674398443 ] ] ] ] }
And here is the relevant bit of the error that is spat out by the index creation:
Edges 0 and 9 cross.
Edge locations in degrees: [-0.0275603, 51.5123001]-[-0.0275420, 51.5122867] and [-0.0275420, 51.5122867]-[-0.0275579, 51.5122984]
"code" : 16755,
"codeName" : "Location16755"
My question is: is there a bug in turf, or is it not doing what I need here in terms of keeping mongo happy? Also is there any documentation on exactly what the 2dshpere index needs in terms of "fixing"? Also, does anyone have suggestions as to what other tools I might use to fix the data, e.g. mapshaper or PostGIS's ST_MakeValid.
Note that once the existing data is fixed I also need a solution for fixing new data on the fly (ideally something that works nice with node).
Mongo Version: 3.4.14 (or any later 3.x)
The problem here is not that the polygon is intersecting itself, but rather that you have a (tiny) hole in the polygon, composed of 4 points, which shares a point with the exterior. So the hole "touches" the exterior, not intersects with it, but this is not allowed.
You can fix such cases using Shapely buffer with a tiny value, e.g.:
shp = shapely.geometry.shape({ "type": "Polygon", "coordinates": [ [ [ -0.027542009179339, 51.5122867222457 ], [ -0.027535822940572, 51.512281465421 ], [ -0.027535925691804, 51.5122814221859 ], [ -0.027589474043984, 51.5122605515771 ], [ -0.027638484531731, 51.5122996934574 ], [ -0.027682911101528, 51.5123351881505 ], [ -0.027689915350493, 51.5123872384419 ], [ -0.027672409315982, 51.5123868001613 ], [ -0.027667905522642, 51.5123866344944 ], [ -0.027663068941865, 51.5123864992013 ], [ -0.02764931654289, 51.512375566682 ], [ -0.027552504539425, 51.5122983194123 ], [ -0.027542009179339, 51.5122867222457 ] ], [ [ -0.027542009179339, 51.5122867222457 ], [ -0.027557948301911, 51.5122984109658 ], [ -0.027560309178214, 51.5123001412876 ], [ -0.027542009179339, 51.5122867222457 ] ] ] })
shp = shp.buffer(1e-12, resolution=0)
geojson = shapely.geometry.mapping(shp)
Assuming I have the following LineString in any document:
{
type: "LineString",
coordinates: [
[ 53.3477, -6.2597 ], [ 51.5008, -0.1224 ],
[48.8567, 2.3508], [52.5166, 13.3833]
]
}
When rendering this line on a map the result is:
How can I get the total distance of this LineString in kilometers, using the mongodb GeoQuery features?
I've been reading through mongo's docs on geospacial querying, and have things working well for singl Polygon types but am having trouble with MultiPolygon. What I want to do is essentially this:
Given a MultiPolygon outlining areas of exclusion:
{
"type" : "MultiPolygon",
"coordinates" : [
[
[
[
-117.873730659485,
33.6152089844919
],
[
-117.873065471649,
33.615048159758
],
[
-117.873044013977,
33.614690770386
],
[
-117.873666286469,
33.6146729008785
],
[
-117.873730659485,
33.6152089844919
]
]
]
]
}
I simply want to be able to pass in a Point to see if it is excluded. I've tried $geoIntersects just to see if it even can determine if a Point is included or not, but that doesn't work. In the end, I want to check that a point is not included within the exclusion list, but the query is simpler without the additional $not operator... Here's what I've been trying:
var geoPoint = {type: 'Point', coordinates: [-117.8731230, 33.6150696]};
db.myCollection.aggregate([
{$match: {'exclusionsPolygons': {$geoIntersects: {$geometry: geoPoint}}}}
]);
Note that if I do the same exact thing with a GeoJSON type of Polygon then it works just fine:
Given this single polygon:
{
"type" : "Polygon",
"coordinates" : [
[
[
-117.8711744,
33.6129677
],
[
-117.8751744,
33.6129677
],
[
-117.874444839148,
33.6162171973226
],
[
-117.87287399259,
33.6172714730352
],
[
-117.871410434393,
33.6165209730032
],
[
-117.8711744,
33.6129677
]
]
]
}
This query works just find and returns the item(s) whose singular polygon contains the point:
var geoPoint = {type: 'Point', coordinates: [-117.8731230, 33.6150696]};
db.myCollection.aggregate([
{$match: {'singularPolygon': {$geoIntersects: {$geometry: geoPoint}}}}
]);
After some tinkering, it turns out the result set was right and I was wrong...
I was using the areas of interest on the map to get addresses to try to query against. One such place was, I thought, in an exclusion polygon:
However, once I made the polygon larger the result set started coming back as I expected it to... So, I reset the polygon and double-checked the map content, finding that if I zoom in further the area of interest was actually excluded from the polygon as there are multiple areas of interest contained:
Whoops - my bad :)
I'm trying to insert a Polygon on a mongodb document
polygons: {
type: "Polygon",
coordinates: [
[
[ -104.6093679, 50.449794 ],
[ -104.6093679, 50.449794 ],
[ -104.6093863, 50.449794 ],
[ -104.6093863, 50.449794 ],
[ -104.6093679, 50.449794 ]
]
]
}
But mongodb throws this exception: "Can't extract geo keys from object, malformed geometry?" I also checked the correctness of index and verified that with the same structure but different data, insert works well. My question is: are there some limitations on Polygon area that mongodb can manage?