Exterior Shell of polygon is invalid - 2dSphere Polygon - mongodb

Question, trying to index and getting the following error:
"Exterior shell of polygon is invalid"
However, I've tested the JSON on http://geojsonlint.com/ and it works
Here's my JSON
{
"type": "Polygon",
"coordinates": [
[
[
116.306655,
39.984977
],
[
116.30673,
39.984977
],
[
116.306734,
39.98483
],
[
116.30667,
39.98483
],
[
116.306678,
39.984714
],
[
116.306384,
39.984705
],
[
116.30638,
39.984858
],
[
116.306193,
39.984852
],
[
116.306198,
39.984601
],
[
116.306031,
39.984597
],
[
116.306031,
39.984596
],
[
116.306031,
39.984596
],
[
116.306023,
39.984961
],
[
116.306082,
39.984964
],
[
116.306082,
39.985019
],
[
116.306655,
39.985032
],
[
116.306655,
39.984977
]
]
]
}


What version of mongodb are you running. If you are running 2.4, try upgrading to 2.6. I had the same error message with census tiger line data that I had converted to GeoJSON, and which I had also confirmed via jsonlint. I had been running the index on version 2.4. I upgraded to mongodb version 2.6, and that solved my problem. I was able to create a 2dsphere index and run geonear queries just fine after that.

Related

How to parse date pattern using grok

How to parse the below log line using grok
Also how to match the pattern of the date.
I tried %{TIMESTAMP_ISO8601:logtime} but no match
Log Line:
13-Nov-2019 00:00:20.230 DEBUG [[ACTIVE] ExecuteThread: '272' for queue: 'weblogic.kernel.Default (self-tuning)'] [196.157.7.12] 965929132 [wire] >> "[\n]"

The question is a bit unclear as to exactly what fields you want them mapped to.
So, here's what matches for me:
%{MONTHDAY:day}[-]%{MONTH:month}[-]%{YEAR:year} %{TIME:time} %{WORD:logtype} \[\[%{WORD:status}\] ExecuteThread: '%{NUMBER:threadNumber}' for queue: '%{GREEDYDATA:queueData}'\] \[%{IP:ip}\] %{NUMBER:numbers} \[%{WORD:text}\] >> "\[\\n\]"
The first 4 fields, answer your date/time pattern query and the rest is what I have used to fit the rest of the fields. Since, no exact mappings were provided , I have mapped them as per my understanding using
This is the output:
{
"day": [
[
"13"
]
],
"month": [
[
"Nov"
]
],
"year": [
[
"2019"
]
],
"time": [
[
"00:00:20.230"
]
],
"logtype": [
[
"DEBUG"
]
],
"status": [
[
"ACTIVE"
]
],
"threadNumber": [
[
"272"
]
],
"queueData": [
[
"weblogic.kernel.Default (self-tuning)"
]
],
"ip": [
[
"196.157.7.12"
]
],
"numbers": [
[
"965929132"
]
],
"text": [
[
"wire"
]
]
}
You can break 'time' further if you want. For any other combinations of patterns, refer Grok Patterns.

Creating an OpenLayers 5 MultiPolygon feature from a turf.js MultiPolygon

In OpenLayers 5.3.0, I've created a MultiPolygon using the 'difference' tool in turf.js. The turf.js MultiPolygon looks fine when I examine the JSON, but when I try to use that to create a feature in OpenLayers, I get "Uncaught TypeError: t.addEventListener is not a function".
I've tried many combinations of JSON.stringify, JSON.parse, GeoJSON.readFeatures, .getCoordinates()... I tried adding the turf.js MultiPolygon as a feature directly via source.addFeature(multiPolygonGeometry), but then I get 'Uncaught TypeError: e.getId is not a function'. I also tried source.addFeatures(multiPolygonGeometry) (note the plural 'addFeatures'), and that didn't give me any errors, but also didn't appear to add anything to the source.
Relevant lines in my code are as follows:
multiPolygonGeometry = turf.difference(largeArea,maskAreas);
multiPolygonFeature = new ol.Feature({
geometry: multiPolygonGeometry,
id: 'multiPolygonFeature1'
});
multiPolygonGeometry looks like this in the console:
{
"type": "Feature",
"properties": {},
"geometry": {
"type": "MultiPolygon",
"coordinates": [
[
[
[
140.9716711384525,
-36.97645228850101
],
[
140.97418321565786,
-36.97679331852701
],
[
140.9741163253784,
-36.97713531664132
],
[
140.9740304946899,
-36.97903805606076
],
[
140.97437381744382,
-36.98025509866784
],
[
140.97594864874696,
-36.98127512642501
],
[
140.9714880598484,
-36.9804459718428
],
[
140.9714500775476,
-36.97642733756345
],
[
140.9716711384525,
-36.97645228850101
]
]
],
[
[
[
140.97455248763328,
-36.97684309230892
],
[
140.97751071844857,
-36.97723786980259
],
[
140.97749308140382,
-36.977304276099005
],
[
140.97715289421623,
-36.97770848336402
],
[
140.97661807025068,
-36.97969050789806
],
[
140.97628355026242,
-36.97958658471583
],
[
140.97634792327878,
-36.97900377288852
],
[
140.9764981269836,
-36.97866094031662
],
[
140.97510337829587,
-36.97727245260485
],
[
140.97455248763328,
-36.97684309230892
]
]
],
[
[
[
140.97628420893903,
-36.98092777726751
],
[
140.97617893060388,
-36.98131793226549
],
[
140.97596635572492,
-36.98127841787872
],
[
140.97628420893903,
-36.98092777726751
]
]
]
]
},
"ol_lm": {
"change": []
}
}
And then I get this message:
events.js:174 Uncaught TypeError: t.addEventListener is not a function
at v (events.js:174)
at e.handleGeometryChanged_ (Feature.js:210)
at e (events.js:41)
at e.dispatchEvent (Target.js:101)
at e.notify (Object.js:151)
at e.set (Object.js:170)
at e.setProperties (Object.js:186)
at new e (Feature.js:108)
at getPolygon (maskedPolygon.js:319) <-- this is the second line in my code sample above
at <anonymous>:1:1
What am I doing wrong here? I'm sure it's something simple but I just can't seem to crack this.

Turf works with GeoJSON features so your "multiPolygonGeometry" is a GeoJSON feature which can be parsed by OpenLayers then given an Id:
multiPolygonFeature = new ol.format.GeoJSON().readFeature(multiPolygonGeometry);
multiPolygonFeature.setId('multiPolygonFeature1');

Watson Speech To Text, Word timestamps are not in sync with Audio

I am using speech to text with following params -
timestamps=true&max_alternatives=1&model=en-US_NarrowbandModel&smart_formatting=true',
Headers - 'Content-Type' => 'audio/flac', 'Transfer-Encoding' => 'chunked'
And providing an audio/flac file to process, but returned words time boundary is not in sync with audio.
For eg. Response is -:
take a morning I have 2 questions please %HESITATION first how how much of the ability
Timestamps are like these -
[
[
"take",
1409.48,
1409.62
],
[
"a",
1409.62,
1409.67
],
[
"morning",
1409.67,
1410.03
],
[
"I",
1410.06,
1410.17
],
[
"have",
1410.17,
1410.38
],
[
"two",
1410.41,
1410.58
],
[
"questions",
1410.58,
1411.05
],
[
"please",
1411.05,
1411.42
],
[
"%HESITATION",
1411.42,
1411.65
],
[
"first",
1411.65,
1412.17
],
[
"how",
1412.33,
1412.62
],
[
"how",
1412.65,
1412.77
],
[
"much",
1412.77,
1413
],
[
"of",
1413,
1413.1
],
[
"the",
1413.1,
1413.37
],
[
"ability",
1413.37,
1413.82
]
]
But in actual audio these words are at different times. (few seconds difference)
Any Suggestion ??

Is there a better way to write this Postgres JSONB query?

Below is a sample JSONB array. I'm trying to figure out how to write a query that doesn't require a cross product like this.
select b.id from brand b,jsonb_array_elements (b.tree) a where a#>>'{Name}' = 'Skiing';
Bonus points for helping me translate this to SQL Alchemy
[
{
"Name": "Snowboarding",
"Order": 1,
"Categories": {
"Jackets": [
22002,
23224
],
"Helmets": [
24920
],
"Freestyle Boards": [
20164
],
"Goggles": [
23169,
23280
],
"Hats": [
22966,
21727
],
"Bindings": [
19265
],
"Gloves": [
20461
],
"Boots": [
26374,
19079,
21765,
22669
],
"Freeride Boards": [
18395,
25505
],
"Pants": [
24143,
20957
]
}
},
{
"Name": "Skiing",
"Order": 2,
"Categories": {
"Jackets": [
22518,
25791,
19972
],
"Pants": [
17516,
23113
],
"Goggles": [
25066,
20996
],
"Helmets": [
24378
],
"Hats": [
20009,
21245
],
"Cross-country Skiing": [
17464
],
"Gloves": [
25822
],
"Boots": [
16616
],
"Poles": [
19280
]
}
},....]

SQL solution first:
SELECT brand.id
FROM brand
WHERE brand.tree #> '[{"Name": "Skiing"}]'::jsonb;
As for sqlalchemy version, you can simply use contains in order to generate SQL statement above:
q = (session.query(Brand.id)
.filter(Brand.tree.contains([{"Name": "Skiing"}]))
)

MongoDB version 2.6 still fails on 2dsphere geoindexing citing inability to extract geokeys and possible malformed geometry

I had been getting the referenced error under version 2.4 with a different lon/lat data set. I upgraded to 2.6 and was able to use 2dsphere index on that data set. However, with a different data set (excerpt below) I received the old error message on trying to index using 2dshpere. the error message excerpt is as follows:
I am having the same problem receiving the error: can't extract geokeys from object; malformed geometry?. The object is the following, which validated on on GeoJSONLint.com. Also, I am using mongodb version 2.6.1. This happened to me recently when I was using version 2.4, and I upgraded to 2.6, and that solved the problem then, but I am getting the error again with a new geojson data set. Please, if anyone can let me know how to solve that would be very helpful.
{"type":"Feature",
"geometry": {
"type":"Polygon",
"coordinates": [[[-80.341942, 25.935059],[-80.341995, 25.936712],[-80.342026, 25.937851],[-80.342026, 25.938393],[-80.34211, 25.939489],[-80.342178, 25.942131],[-80.342308, 25.942329],[-80.342339, 25.94285],[-80.342438, 25.944914],[-80.342445, 25.945118],[-80.342514, 25.947062],[-80.342537, 25.947479],[-80.342613, 25.949152],[-80.342667, 25.950306],[-80.342712, 25.951366],[-80.342773, 25.952713],[-80.342857, 25.954611],[-80.341438, 25.955782],[-80.340202, 25.956839],[-80.339104, 25.957079],[-80.33831, 25.957081],[-80.336006, 25.957088],[-80.333244, 25.957096],[-80.331787, 25.957102],[-80.326721, 25.957117],[-80.326714, 25.957117],[-80.32444, 25.957127],[-80.323616, 25.957129],[-80.322365, 25.957132],[-80.318619, 25.957146],[-80.317375, 25.95715],[-80.316597, 25.957151],[-80.314278, 25.957161],[-80.313507, 25.957163],[-80.3134, 25.957163],[-80.31308, 25.957165],[-80.312973, 25.957167],[-80.312798, 25.957167],[-80.312263, 25.957167],[-80.312088, 25.957167],[-80.311996, 25.957167],[-80.311737, 25.957167],[-80.311646, 25.957169],[-80.311531, 25.957169],[-80.311272, 25.95717],[-80.311188, 25.95717],[-80.311073, 25.95717],[-80.311028, 25.95717],[-80.310913, 25.95717],[-80.310883, 25.95717],[-80.310837, 25.95717],[-80.310829, 25.95717],[-80.310783, 25.95717],[-80.310776, 25.95717],[-80.310738, 25.957169],[-80.310684, 25.957167],[-80.310646, 25.957165],[-80.310616, 25.957165],[-80.310547, 25.957163],[-80.310356, 25.957161],[-80.310287, 25.957159],[-80.310249, 25.957157],[-80.310143, 25.957155],[-80.310104, 25.957155],[-80.302231, 25.95697],[-80.301704, 25.956972],[-80.298981, 25.956896],[-80.301468, 25.956968],[-80.298668, 25.956886],[-80.29528, 25.956772],[-80.295052, 25.95677],[-80.294975, 25.95677],[-80.294975, 25.956905],[-80.294975, 25.956928],[-80.294762, 25.956938],[-80.29454, 25.956947],[-80.294548, 25.956835],[-80.294579, 25.954868],[-80.294579, 25.954702],[-80.294601, 25.953625],[-80.294601, 25.953461],[-80.294571, 25.953079],[-80.294502, 25.95186],[-80.294487, 25.95163],[-80.294472, 25.951212],[-80.294418, 25.949945],[-80.294296, 25.946936],[-80.294281, 25.946581],[-80.294197, 25.944429],[-80.294128, 25.943003],[-80.294098, 25.942413],[-80.29409, 25.94228],[-80.293854, 25.939285],[-80.293846, 25.939144],[-80.293694, 25.936041],[-80.293625, 25.934875],[-80.293549, 25.933006],[-80.293541, 25.932341],[-80.293533, 25.931826],[-80.293472, 25.931252],[-80.293495, 25.93054],[-80.293373, 25.928652],[-80.293343, 25.928129],[-80.293335, 25.927803],[-80.29332, 25.92738],[-80.293228, 25.925432],[-80.295456, 25.925182],[-80.296936, 25.925102],[-80.297676, 25.925114],[-80.297974, 25.925117],[-80.298866, 25.925131],[-80.30027, 25.925121],[-80.300957, 25.925116],[-80.305336, 25.925138],[-80.306831, 25.925156],[-80.307472, 25.925163],[-80.307686, 25.925158],[-80.308235, 25.92514],[-80.30941, 25.925127],[-80.309402, 25.924763],[-80.313126, 25.924776],[-80.318581, 25.924747],[-80.319275, 25.924746],[-80.319557, 25.924723],[-80.320023, 25.924686],[-80.320137, 25.924664],[-80.320526, 25.924587],[-80.320992, 25.924469],[-80.321259, 25.9244],[-80.321419, 25.924341],[-80.321869, 25.92417],[-80.322388, 25.923891],[-80.32254, 25.923805],[-80.322746, 25.923674],[-80.323059, 25.923479],[-80.323067, 25.923475],[-80.323181, 25.923376],[-80.323235, 25.92333],[-80.32373, 25.922907],[-80.324226, 25.922354],[-80.324783, 25.921375],[-80.324966, 25.920954],[-80.32531, 25.921684],[-80.325325, 25.922615],[-80.32534, 25.923359],[-80.32534, 25.923492],[-80.325348, 25.923607],[-80.325356, 25.923817],[-80.325371, 25.924597],[-80.325401, 25.925245],[-80.325432, 25.925819],[-80.325439, 25.92606],[-80.325462, 25.926447],[-80.325447, 25.926756],[-80.325409, 25.927439],[-80.325562, 25.92762],[-80.325706, 25.927801],[-80.325928, 25.92808],[-80.326218, 25.927971],[-80.326492, 25.927883],[-80.327003, 25.927832],[-80.33033, 25.927818],[-80.333618, 25.927853],[-80.334419, 25.927872],[-80.336128, 25.927896],[-80.336777, 25.927883],[-80.338219, 25.927912],[-80.339294, 25.927912],[-80.33963, 25.927931],[-80.340416, 25.927938],[-80.340919, 25.927923],[-80.341103, 25.927925],[-80.341606, 25.927925],[-80.34166, 25.929146],[-80.341721, 25.930458],[-80.341728, 25.930592],[-80.341766, 25.931606],[-80.341858, 25.933409],[-80.341942, 25.935059]
]]},
"properties":{
"id": "ZIP33015","census_year": 2010,"exposure_year": 2014, "commercial_exposure": 652653581, "residential_exposure": 2299303204, "mobile_home": 0, "tenants": 8631577, "condo_owners": 49223035, "total_exposure": 3009811397}}
When I run db.exposureByZip.ensureIndex({"geometry" : "2dsphere"}) on the above, I get the following error:
{
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 1,
"ok" : 0,
"errmsg" : "Can't extract geo keys from object, malformed geometry?: { _id: ObjectId('53a6be43beeb0a0246ae75bb'), geometry: { type: \"Polygon\", coordinates: [ [ [ -80.341942, 25.935059 ], [ -80.341995, 25.936712 ], [ -80.342026, 25.937851 ], [ -80.342026, 25.938393 ], [ -80.34211000000001, 25.939489 ], [ -80.342178, 25.942131 ], [ -80.342308, 25.942329 ], [ -80.342339, 25.94285 ], [ -80.342438, 25.944914 ], [ -80.342445, 25.945118 ], [ -80.34251399999999, 25.947062 ], [ -80.34253699999999, 25.947479 ], [ -80.342613, 25.949152 ], [ -80.34266700000001, 25.950306 ], [ -80.34271200000001, 25.951366 ], [ -80.34277299999999, 25.952713 ], [ -80.342857, 25.954611 ], [ -80.341438, 25.955782 ], [ -80.34020200000001, 25.956839 ], [ -80.33910400000001, 25.957079 ], [ -80.33831000000001, 25.957081 ], [ -80.336006, 25.957088 ], [ -80.33324399999999, 25.957096 ], [ -80.33178700000001, 25.957102 ], [ -80.32672100000001, 25.957117 ], [ -80.326714, 25.957117 ], [ -80.32444, 25.957127 ], [ -80.323616, 25.957129 ], [ -80.322365, 25.957132 ], [ -80.318619, 25.957146 ], [ -80.317375, 25.95715 ], [ -80.316597, 25.957151 ], [ -80.314278, 25.957161 ], [ -80.313507, 25.957163 ], [ -80.3134, 25.957163 ], [ -80.31308, 25.957165 ], [ -80.312973, 25.957167 ], [ -80.312798, 25.957167 ], [ -80.312263, 25.957167 ], [ -80.312088, 25.957167 ], [ -80.31199599999999, 25.957167 ], [ -80.31173699999999, 25.957167 ], [ -80.311646, 25.957169 ], [ -80.311531, 25.957169 ], [ -80.311272, 25.95717 ], [ -80.311188, 25.95717 ], [ -80.31107299999999, 25.95717 ], [ -80.31102799999999, 25.95717 ], [ -80.310913, 25.95717 ], [ -80.310883, 25.95717 ], [ -80.31083700000001, 25.95717 ], [ -80.310829, 25.95717 ], [ -80.310783, 25.95717 ], [ -80.310776, 25.95717 ], [ -80.310738, 25.957169 ], [ -80.31068399999999, 25.957167 ], [ -80.31064600000001, 25.957165 ], [ -80.310616, 25.957165 ], [ -80.310547, 25.957163 ], [ -80.310356, 25.957161 ], [ -80.310287, 25.957159 ], [ -80.310249, 25.957157 ], [ -80.310143, 25.957155 ], [ -80.310104, 25.957155 ], [ -80.30223100000001, 25.95697 ], [ -80.301704, 25.956972 ], [ -80.298981, 25.956896 ], [ -80.301468, 25.956968 ], [ -80.29866800000001, 25.956886 ], [ -80.29528000000001, 25.956772 ], [ -80.295052, 25.95677 ], [ -80.29497499999999, 25.95677 ], [ -80.29497499999999, 25.956905 ], [ -80.29497499999999, 25.956928 ], [ -80.29476200000001, 25.956938 ], [ -80.29454, 25.956947 ], [ -80.29454800000001, 25.956835 ], [ -80.294579, 25.954868 ], [ -80.294579, 25.954702 ], [ -80.294601, 25.953625 ], [ -80.294601, 25.953461 ], [ -80.294571, 25.953079 ], [ -80.29450199999999, 25.95186 ], [ -80.294487, 25.95163 ], [ -80.294472, 25.951212 ], [ -80.29441799999999, 25.949945 ], [ -80.294296, 25.946936 ], [ -80.294281, 25.946581 ], [ -80.294197, 25.944429 ], [ -80.294128, 25.943003 ], [ -80.29409800000001, 25.942413 ], [ -80.29409, 25.94228 ], [ -80.293854, 25.939285 ], [ -80.293846, 25.939144 ], [ -80.293694, 25.936041 ], [ -80.29362500000001, 25.934875 ], [ -80.293549, 25.933006 ], [ -80.293541, 25.932341 ], [ -80.293533, 25.931826 ], [ -80.29347199999999, 25.931252 ], [ -80.29349499999999, 25.93054 ], [ -80.293373, 25.928652 ], [ -80.29334299999999, 25.928129 ], [ -80.293335, 25.927803 ], [ -80.29331999999999, 25.92738 ], [ -80.293228, 25.925432 ], [ -80.295456, 25.925182 ], [ -80.296936, 25.925102 ], [ -80.297676, 25.925114 ], [ -80.297974, 25.925117 ], [ -80.298866, 25.925131 ], [ -80.30027, 25.925121 ], [ -80.300957, 25.925116 ], [ -80.305336, 25.925138 ], [ -80.306831, 25.925156 ], [ -80.307472, 25.925163 ], [ -80.307686, 25.925158 ], [ -80.308235, 25.92514 ], [ -80.30941, 25.925127 ], [ -80.30940200000001, 25.924763 ], [ -80.313126, 25.924776 ], [ -80.31858099999999, 25.924747 ], [ -80.319275, 25.924746 ], [ -80.319557, 25.924723 ], [ -80.32002300000001, 25.924686 ], [ -80.320137, 25.924664 ], [ -80.320526, 25.924587 ], [ -80.320992, 25.924469 ], [ -80.321259, 25.9244 ], [ -80.32141900000001, 25.924341 ], [ -80.32186900000001, 25.92417 ], [ -80.322388, 25.923891 ], [ -80.32254, 25.923805 ], [ -80.322746, 25.923674 ], [ -80.323059, 25.923479 ], [ -80.32306699999999, 25.923475 ], [ -80.32318100000001, 25.923376 ], [ -80.323235, 25.92333 ], [ -80.32373, 25.922907 ], [ -80.324226, 25.922354 ], [ -80.324783, 25.921375 ], [ -80.324966, 25.920954 ], [ -80.32531, 25.921684 ], [ -80.32532500000001, 25.922615 ], [ -80.32534, 25.923359 ], [ -80.32534, 25.923492 ], [ -80.32534800000001, 25.923607 ], [ -80.325356, 25.923817 ], [ -80.325371, 25.924597 ], [ -80.325401, 25.925245 ], [ -80.32543200000001, 25.925819 ], [ -80.325439, 25.92606 ], [ -80.325462, 25.926447 ], [ -80.325447, 25.926756 ], [ -80.32540899999999, 25.927439 ], [ -80.32556200000001, 25.92762 ], [ -80.325706, 25.927801 ], [ -80.325928, 25.92808 ], [ -80.326218, 25.927971 ], [ -80.326492, 25.927883 ], [ -80.327003, 25.927832 ], [ -80.33033, 25.927818 ], [ -80.333618, 25.927853 ], [ -80.334419, 25.927872 ], [ -80.336128, 25.927896 ], [ -80.336777, 25.927883 ], [ -80.338219, 25.927912 ], [ -80.339294, 25.927912 ], [ -80.33963, 25.927931 ], [ -80.340416, 25.927938 ], [ -80.340919, 25.927923 ], [ -80.341103, 25.927925 ], [ -80.341606, 25.927925 ], [ -80.34166, 25.929146 ], [ -80.34172100000001, 25.930458 ], [ -80.341728, 25.930592 ], [ -80.34176600000001, 25.931606 ], [ -80.341858, 25.933409 ], [ -80.341942, 25.935059 ] ] ] }, type: \"Feature\", properties: { commercial_exposure: 652653581, total_exposure: 3009811397, condo_owners: 49223035, mobile_home: 0, census_year: 2010, tenants: 8631577, residential_exposure: 2299303204, exposure_year: 2014, id: \"ZIP33015\" } }",
"code" : 16755
}

There is a bug posted for this problem: https://jira.mongodb.org/browse/SERVER-13735
“Malformed geometry” error when creating a geosphere index for a valid
MultiPolygon
Similar issue, a valid GeoJSON is being submitted for a 2dsphere index and is causing the "Can't extract geo keys from object, malformed geometry?" error.
There have been a series of previous bugs where valid GeoJSON was causing this error message. They've fixed those in 2.6.x but this appears to be a new one. I'd chime in on JIRA as it will help get it prioritized for a fix.

Maybe the problem is because your polygon coordinates has the first coordinate and last coordinate as the same value. Try to remove the last coordinate pooint from your polygons.