My Schema is as follows:
mongoose = require 'mongoose'
ObjectId = mongoose.Schema.ObjectId
CheckinSchema = new mongoose.Schema
text:
type: String, required: true
location_latLong:
lat:
type: Number
lon:
type: Number
location_country:
type: String
addedOn:
type: Date
default: Date.now
CheckinSchema.index
location_latLong: '2d'
exports.CheckinSchema = CheckinSchema
The Model is generated separately. I get an error however when running a query. The error is:
count fails:{ errmsg: "exception: can't find special index: 2d for: { location_latLong: { $wi...", code: 13038, ok: 0.0 }
My query is:
{ location_latLong: { '$within': { '$box': [[ '-100', '-100' ],[ '100', '100' ]] } } }
So my question is.... what gives? How can I properly do Geospatial indexing in Mongoose (using Node.js)
THis is because you have specified the wrong order in your geo index, Since mongodb is based on GeoJSON, its recommented to have the longitude field first
instead this
location_latLong:
lat:
type: Number
lon:
type: Number
use this
location_latLong:
lon:
type: Number
lat:
type: Number
The names you assign to a location object (lon,lat keys) are completely ignored, only the ordering is detected.
In mongodb geospatial page, its recommended in multiple places
By default, the index assumes you are indexing longitude/latitude and
is thus configured for a [-180..180) value range
and
The code assumes that you are using decimal degrees in (longitude,
latitude) order. This is the same order used for the GeoJSON spec.
Using (latitude, longitude) will result in very incorrect results, but
is often the ordering used elsewhere, so it is good to double-check.
The names you assign to a location object (if using an object and not
an array) are completely ignored, only the ordering is detected.
I have already answered this before.
Cheer
Related
I'm kinda new to mongoose, and I'm not sure if it's a right term.
what I'm building is a community site (like redit), and I have a schema like below
const postSchema = new mongoose.Schema({
content: {
type: String,
required: true,
},
title: {
type: String,
required: true,
},
userId: {
type: mongoose.Schema.Types.ObjectId,
required: true,
ref: 'User',
},
board: {
type: String,
required: true,
enum: ['board1','board2'],
},
created_at: {
type: Date,
default: Date.now,
},
updated_at: {
type: Date,
},
})
there are many kinds of 'board'
and I'm not sure if it can be 'indexed'.
purpose of it is for getting posts faster
for example in sql (assume that board column is indexed)
--> select * from post where board = 'board1' ;
I'm confusing about the terms, need some direction..
Short answer:
You need to create an index on the field board by doing:
db.post.createIndex(
{ board: 1 } ,
{ name: "borad index" }
)
Long answer:
Indexing in mongodb uses memory in order to save running time.
Let's take an example: say you have all words in English in your DB. And you are reading a book and from time to time you need to search for a word to check its meaning.
How would you do that? A dictionary. You'll sort the words alphabetically and then you could easily search for every word you wanted.
Indexing apply the same concept. When you create an index on the field board it takes all its values, sort them and save it in a table (and reference for each entry the full document from your collection).
Now when you search for select * from post where board = 'board1' it first use the memorized table of sorted boards, finds the ones that equal to board1 and then by the reference gives you the full documents that belongs to it. You can continue reading here.
All of the examples and implementations of geo search in Mongo, that I've been able to find, use an array property [lng, lat] to perform the search on:
An example record would be:
{
name: 'foo bar',
locations: [
{
name: 'franks house',
geo: [-0.12, 34.51]
},
{
name: 'jennifers house',
geo: [-0.09, 32.17]
}
]
}
And the resulting query:
db.events.find({ 'locations.geo': { $nearSphere: [ -0.12, 34.51 ], $maxDistance: 0.02 }})
This works fine, but the format of the record is not great for users to read or write because it's not self-documenting where lat and lng go in that array. There's room for gotchas. I'd like to make the records more human friendly:
{
name: 'foo bar',
locations: [
{
name: 'franks house',
lat: 34.51,
lng: -0.12
},
{
name: 'jennifers house',
lat: 32.17,
lng: -0.09
}
]
}
What would the resulting mongo query look like for this type of record? I haven't been able to find any examples of this so am wondering if it's even possible.
It's not recommended to use separate fields for latitude and longitude. The 2dsphere index is used to index geospatial data which requires an array of coordinates, see documentation. This is the reason you can't find examples for separate coordinate fields.
You should separate representation from data storage. Just because coordinates are stored in an array, you don't necessarily need to present them as an array to the user. For example you could use a pre call on save to store separate parameters in an array, something like:
var schema = new Schema(..);
schema.pre('save', function (next) {
this.coordinates = [this.longitude, this.latitude]
next();
});
The document I am working on is extremely large. It collects user input from an extremely long survey (like survey monkey) and stores the answers in a mongodb database.
I am unsurprisingly getting the following error
Error: Document exceeds maximal allowed bson size of 16777216 bytes
If I cannot change the fields in my document is there anything I can do? Is there some way to compress down the document, by removing white space or something like that?
Edit
Here is the structure of the document
Schema({
id : { type: Number, required: true },
created: { type: Date, default: Date.now },
last_modified: { type: Date, default: Date.now },
data : { type: Schema.Types.Mixed, required: true }
});
An example of the data field:
{
id: 65,
question: {
test: "some questions",
answers: [2,5,6]
}
// there could be thousands of these question objects
}
One thing you can do is to build your own mongoDB :-). Mongodb is an open source and the limitation about the size of a document is rather arbitrary to enforce a better schema design. You can just modify this line and build it for yourself. Be careful with this.
The most straight forward idea is to have each small question in a different document with a field which reference to its parent.
Another idea is to limit number of documents in the parent. Lets say you limit is N elements then the parent looks like this:
{
_id : ObjectId(),
id : { type: Number, required: true },
created: { type: Date, default: Date.now }, // you can store it only for the first element
last_modified: { type: Date, default: Date.now }, // the same here
data : [{
id: 65,
question: {
test: "some questions",
answers: [2,5,6]
}
}, ... up to N of such things {}
]
}
This way modifying number N you can make sure that you will be in 16 MB of BSON. And in order to read the whole survey you can select
db.coll.find({id: the Id you need}) and then combine the whole survey on the application level. Also do not forget to ensureIndex on id.
Try different things, do a benchmark on your data and see what works for you.
You should be using gridfs. It allows you to store documents in chunks. Here's the link: http://docs.mongodb.org/manual/reference/gridfs/
I am attempting to use MongoDB's Geospatial Indexing by querying for latitude and longitude points around a certain point using MongoDB's find method. I keep getting the error:
MongoError: can't find any special indices: 2d (needs index), 2dsphere (needs index)
I'm not sure where the documentation is for this after Googling for about an hour. Also I can't find any good explanations. Here is the Schema that I created using Mongoose:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var EventSchema = new Schema ({
name: String,
description: String,
location: {type: [String], index: '2dsphere'},
time: Date,
photo: Buffer,
activity: String
});
mongoose.model('Event', EventSchema);
I then use the find method to find other Event documents around the point that the user provides.
var maxDistance = 0.09;
var lonLat = {$geometry: {type: 'Point', coordinates: [34.09,124.34] }};
Event.find({
'geo': {
$near: lonLat,
$maxDistance: maxDistance
}
}).exec(function(err, events) {
if(err) {
throw err;
} else {
return events;
}
});
What syntax do I have wrong here? Am I missing something huge? Any urls to any documentation will be great as well besides this link.
Did you try to create the index with this sentence?
db.event.ensureIndex({ "location" : "2d" } )
It's a little confusing your example. I don't get if 'location' are your coordinates or 'geo', because you create the schema with the former and you query with the latter.
In both cases, a good idea is to execute
db.event.findOne().pretty()
So you can check the format of your collection.
Also with this command you can check your current indexes.
db.event.getIndexes()
All of this works if you are working on 'event' collection.
I've written an application that finds local establishments and delivers them via a RESTful API. My stack is: express, express-resource and mongoose. Here is a sample of my model:
var PlaceSchema = new mongoose.Schema(
{
name: {
type: String,
required: true
},
geolocation: {
lat: Number,
lng: Number
},
address: {
type: String
}
}
);
PlaceSchema.index({
geolocation: '2d'
});
I've checked a few times and my lon/lat values are being correctly stored in the db, so there aren't any data errors.
Then I run the query to grab the data with the specific query values:
mongoose.connection.db.executeDbCommand(
{
geoNear: 'places',
near: [
parseFloat(req.body.lat),
parseFloat(req.body.lng)
],
num: limit,
query: {
// Some additional queries that
// don't impact the geo results.
},
spherical: true,
distanceMultiplier: 6371, // converting results to km
maxDistance: distance / 6371,
},
function(err, result)
{
res.send(200, {
status: true,
data: theaters
});
}
);
There are a few issues with the results that it's returning: a) the calculation in km is really wrong. In most cases there's a 6-7km difference, but it varies, b) places that are closer are appearing farther than other places.
I'm using the direct mongoose query because it will return the calculated distance (which I require for my API returns). Switching to the mongoose find method apparently wouldn't let me gain access to this data.
Wondering if my index is perhaps wrong? Should it be 2dsphere instead? The documentation in that regard is slightly confusing, but most of the examples I see use just 2d.
Thanks a bunch!
No matter what type of geospatial indexing you use in MongoDB, you always must store longitude first and then latitude.
From http://docs.mongodb.org/manual/core/2d/#store-points-on-a-2d-plane and multiple other places in the docs:
Whether as an array or document, if you use longitude and latitude,
store coordinates in this order: longitude, latitude.