Parse Query where near geopoint error - mongodb

So I have posted my issue here. In short, I can't query on Geopoint on my parse-server I get a strange error:
{"code":1,"message":"Internal server error."} (Code: 1, Version: 1.14.4)
Optional("{"code":1,"message":"Internal server error."}")
On the dashboard logs it says something a bit more complex:
2017-05-08T04:06:47.283Z - Uncaught internal server error. { MongoError: error processing query: ns=heroku_r6dg7cvc.CommentaryTree: $and
_created_at $gt new Date(1494130007253)
_rperm $in [ null "*" "a81FrGp2R3" ]
GEONEAR field=location maxdist=7.17619e-07 isNearSphere=1
Sort: {}
Proj: {}
planner returned error: unable to find index for $geoNear query
at Function.MongoError.create (/app/node_modules/mongodb-core/lib/error.js:31:11)
at /app/node_modules/mongodb-core/lib/connection/pool.js:489:72
at authenticateStragglers (/app/node_modules/mongodb-core/lib/connection/pool.js:435:16)
at Connection.messageHandler (/app/node_modules/mongodb-core/lib/connection/pool.js:469:5)
at Socket.<anonymous> (/app/node_modules/mongodb-core/lib/connection/connection.js:321:22)
at emitOne (events.js:96:13)
at Socket.emit (events.js:191:7)
at readableAddChunk (_stream_readable.js:178:18)
at Socket.Readable.push (_stream_readable.js:136:10)
at TCP.onread (net.js:561:20)
name: 'MongoError',
message: 'error processing query: ns=heroku_r6dg7cvc.CommentaryTree: $and\n _created_at $gt new Date(1494130007253)\n _rperm $in [ null "*" "a81FrGp2R3" ]\n GEONEAR field=location maxdist=7.17619e-07 isNearSphere=1\nSort: {}\nProj: {}\n planner returned error: unable to find index for $geoNear query',
waitedMS: 0,
ok: 0,
errmsg: 'error processing query: ns=heroku_r6dg7cvc.CommentaryTree: $and\n _created_at $gt new Date(1494130007253)\n _rperm $in [ null "*" "a81FrGp2R3" ]\n GEONEAR field=location maxdist=7.17619e-07 isNearSphere=1\nSort: {}\nProj: {}\n planner returned error: unable to find index for $geoNear query',
code: 2 } MongoError: error processing query: ns=heroku_r6dg7cvc.CommentaryTree: $and
_created_at $gt new Date(1494130007253)
_rperm $in [ null "*" "a81FrGp2R3" ]
GEONEAR field=location maxdist=7.17619e-07 isNearSphere=1
Sort: {}
Proj: {}
planner returned error: unable to find index for $geoNear query
at Function.MongoError.create (/app/node_modules/mongodb-core/lib/error.js:31:11)
at /app/node_modules/mongodb-core/lib/connection/pool.js:489:72
at authenticateStragglers (/app/node_modules/mongodb-core/lib/connection/pool.js:435:16)
at Connection.messageHandler (/app/node_modules/mongodb-core/lib/connection/pool.js:469:5)
at Socket.<anonymous> (/app/node_modules/mongodb-core/lib/connection/connection.js:321:22)
at emitOne (events.js:96:13)
at Socket.emit (events.js:191:7)
at readableAddChunk (_stream_readable.js:178:18)
at Socket.Readable.push (_stream_readable.js:136:10)
at TCP.onread (net.js:561:20)
After searching for a while, its possible it might be related to this.
Where the solution would be the same to the second link, but I don't know how to apply the solution to a parse-server on heroku.

You using the mlab extension on Heroku? Go to MLab and add an index for the geo field on your collection. You don't need to use an API for this, use the MLab dashboard.

Related

GitLab pipeline - postgres connection termined unexpectedly

I am having a little trouble with postgres in docker .. I am running this pipeline in gitlab ... and everything starts fine.. I start a postgres:14-alpine docker image and run unit tests of the application ... and when it's about to start (create the test database) ...
I tried increasing the timeouts
pool: {
max: 50,
min: 5,
idle: 600000, // 10min
acquire: 600000,
createTimeoutMillis: 600000,
acquireTimeoutMillis: 600000,
idleTimeoutMillis: 600000,
reapIntervalMillis: 600000,
createRetryIntervalMillis: 600000
}
but it keeps ending up with this "Connection terminated unexpectedly"
Creating database db_unittests, owner will be postgres.
Executing (default): CREATE DATABASE db_unittests WITH OWNER = postgres;
/builds/project/node_modules/sequelize/lib/dialects/postgres/query.js:386
return new sequelizeErrors.DatabaseError(err);
^
DatabaseError [SequelizeDatabaseError]: Connection terminated unexpectedly
at Query.formatError (/builds/project/node_modules/sequelize/lib/dialects/postgres/query.js:386:16)
at Query.run (/builds/project/node_modules/sequelize/lib/dialects/postgres/query.js:87:18)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async /builds/project/node_modules/sequelize/lib/sequelize.js:619:16
at async run (/builds/project/tests/scripts/database-recreate.js:50:3) {
parent: Error: Connection terminated unexpectedly
at Connection.<anonymous> (/builds/project/node_modules/pg/lib/client.js:132:73)
at Object.onceWrapper (node:events:627:28)
at Connection.emit (node:events:513:28)
at Socket.<anonymous> (/builds/project/node_modules/pg/lib/connection.js:107:12)
at Socket.emit (node:events:525:35)
at endReadableNT (node:internal/streams/readable:1358:12)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
sql: 'CREATE DATABASE db_unittests WITH OWNER = postgres;',
parameters: undefined
},
original: Error: Connection terminated unexpectedly
at Connection.<anonymous> (/builds/project/node_modules/pg/lib/client.js:132:73)
at Object.onceWrapper (node:events:627:28)
at Connection.emit (node:events:513:28)
at Socket.<anonymous> (/builds/project/node_modules/pg/lib/connection.js:107:12)
at Socket.emit (node:events:525:35)
at endReadableNT (node:internal/streams/readable:1358:12)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
sql: 'CREATE DATABASE db_unittests WITH OWNER = postgres;',
parameters: undefined
},
sql: 'CREATE DATABASE db_unittests WITH OWNER = postgres;',
parameters: undefined
}
Locally, it all works fine .. only in the pipeline - it acting up :disappointed:
Any idea why is that so? Thanks a lot

Heroku Tails suggesting postgres database column doesn't exist, but it does and works in localhost

I have an app deployed on heroku. My production and dev environments both use PostgreSQL and I use knex for my database querying. I am getting an issue where no resources load onto a page and heroku log --tail says this:
2022-08-23T21:34:19.565970+00:00 app[web.1]: There is an error here: select *, "authors"."name" as "author_name", "authors"."id" as "author_id" from "books" inner join "authors" on "books"."author_id" = "authors"."id" where "books"."id" = $1 limit $2 - column books.author_id does not exist
However, the call works fine on localhost - I have checked my database schema on pgadmin and it appears that the column does exist.
Here is the call to the database:
function getBookById(id) {
console.log(id)
return db('books')
.join('authors', 'books.author_id', 'authors.id')
.where('books.id', id)
.select('*', 'authors.name as author_name', 'authors.id as author_id')
.first()
}
I have done a bit more diving in and looks like it is breaking on a number of database hits and returning 503s.
Here is some more error messages I am getting when I run heroku log --tail
2022-08-24T02:25:36.434109+00:00 app[web.1]: at Parser.parseErrorMessage (/app/node_modules/pg-protocol/dist/parser.js:287:98)
2022-08-24T02:25:36.434109+00:00 app[web.1]: at Parser.handlePacket (/app/node_modules/pg-protocol/dist/parser.js:126:29)
2022-08-24T02:25:36.434110+00:00 app[web.1]: at Parser.parse (/app/node_modules/pg-protocol/dist/parser.js:39:38)
2022-08-24T02:25:36.434110+00:00 app[web.1]: at TLSSocket.<anonymous> (/app/node_modules/pg-protocol/dist/index.js:11:42)
2022-08-24T02:25:36.434111+00:00 app[web.1]: at TLSSocket.emit (node:events:513:28)
2022-08-24T02:25:36.434112+00:00 app[web.1]: at addChunk (node:internal/streams/readable:315:12)
2022-08-24T02:25:36.434112+00:00 app[web.1]: at readableAddChunk (node:internal/streams/readable:289:9)
2022-08-24T02:25:36.434112+00:00 app[web.1]: at TLSSocket.Readable.push (node:internal/streams/readable:228:10)
2022-08-24T02:25:36.434113+00:00 app[web.1]: at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23) {
2022-08-24T02:25:36.434113+00:00 app[web.1]: length: 127,
2022-08-24T02:25:36.434114+00:00 app[web.1]: severity: 'ERROR',
2022-08-24T02:25:36.434114+00:00 app[web.1]: code: '42703',
2022-08-24T02:25:36.434114+00:00 app[web.1]: detail: undefined,
2022-08-24T02:25:36.434115+00:00 app[web.1]: hint: undefined,
2022-08-24T02:25:36.434115+00:00 app[web.1]: position: '22',
2022-08-24T02:25:36.434115+00:00 app[web.1]: internalPosition: undefined,
2022-08-24T02:25:36.434115+00:00 app[web.1]: internalQuery: undefined,
2022-08-24T02:25:36.434116+00:00 app[web.1]: where: undefined,
2022-08-24T02:25:36.434116+00:00 app[web.1]: schema: undefined,
2022-08-24T02:25:36.434116+00:00 app[web.1]: table: undefined,
2022-08-24T02:25:36.434116+00:00 app[web.1]: column: undefined,
2022-08-24T02:25:36.434117+00:00 app[web.1]: dataType: undefined,
2022-08-24T02:25:36.434117+00:00 app[web.1]: constraint: undefined,
2022-08-24T02:25:36.434117+00:00 app[web.1]: file: 'parse_target.c',
2022-08-24T02:25:36.434118+00:00 app[web.1]: line: '1061',
2022-08-24T02:25:36.434118+00:00 app[web.1]: routine: 'checkInsertTargets'
2022-08-24T02:25:36.434118+00:00 app[web.1]: }
2022-08-24T02:25:36.434240+00:00 app[web.1]: There is an error here: insert into "books" ("author_id", "blurb", "cover_image", "genre", "pub_year", "title") values ($1, $2, $3, $4, $5, $6) returning "id" - column "author_id" of relation "books" does not exist
I'm not sure if this is something weird with heroku's postgres - or something I'm doing wrong (more likely). It is unclear to me how to further trouble shoot this issue.
Thanks for an help

What is this Mongoose CursorNotFound Error?

I have mongoose throwing this error from time to time, it's not stopping the node application from running but I can't find any information about whats causing the error and I would like to fix it.
The link to the repo is here https://github.com/bkawk/divergence2
I have found I can stop the error by adding a limit to the records here
https://github.com/bkawk/divergence2/blob/master/src/db.js#L94
{ MongoError: cursor id 31871209567 not found
at /Users/bkawk/Documents/divergence2/node_modules/mongoose/node_modules/mongodb-core/lib/connection/pool.js:598:61
at authenticateStragglers (/Users/bkawk/Documents/divergence2/node_modules/mongoose/node_modules/mongodb-core/lib/connection/pool.js:516:16)
at Connection.messageHandler (/Users/bkawk/Documents/divergence2/node_modules/mongoose/node_modules/mongodb-core/lib/connection/pool.js:552:5)
at emitMessageHandler (/Users/bkawk/Documents/divergence2/node_modules/mongoose/node_modules/mongodb-core/lib/connection/connection.js:309:10)
at Socket.<anonymous> (/Users/bkawk/Documents/divergence2/node_modules/mongoose/node_modules/mongodb-core/lib/connection/connection.js:488:15)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at addChunk (_stream_readable.js:263:12)
at readableAddChunk (_stream_readable.js:250:11)
at Socket.Readable.push (_stream_readable.js:208:10)
at TCP.onread (net.js:607:20)
name: 'MongoError',
message: 'cursor id 31871209567 not found',
ok: 0,
errmsg: 'cursor id 31871209567 not found',
code: 43,
codeName: 'CursorNotFound' }

Error: connection timeout - expressjs mongodb mongoose

I'm new to express/node and web programming in general. What is the best way to deal with this error when mongoose's mongodb connection times out, this is how I connect:
mongoose.connect(config.mongoUrl);
const db = mongoose.connection;
db.on('error', console.error.bind(console, 'error connecting with mongodb database:'));
db.once('open', function() {
console.log('connected to mongodb database');
});
This is the error when it times out while my server is running:
error connecting with mongodb database: Error: connection timeout
at Db. (C:\Users\Sean\OneDrive\webpages\000\lasttry\node_modules\mongoose\lib\drivers\node-mongodb-native\connection.js:169:17)
at emitTwo (events.js:106:13)
at Db.emit (events.js:191:7)
at Server.listener (C:\Users\Sean\OneDrive\webpages\000\lasttry\node_modules\mongodb\lib\db.js:1798:14)
at emitOne (events.js:96:13)
at Server.emit (events.js:188:7)
at Server. (C:\Users\Sean\OneDrive\webpages\000\lasttry\node_modules\mongodb\lib\server.js:274:14)
at emitOne (events.js:96:13)
at Server.emit (events.js:188:7)
at Pool. (C:\Users\Sean\OneDrive\webpages\000\lasttry\node_modules\mongodb-core\lib\topologies\server.js:335:12)
at emitOne (events.js:96:13)
at Pool.emit (events.js:188:7)
at Connection. (C:\Users\Sean\OneDrive\webpages\000\lasttry\node_modules\mongodb-core\lib\connection\pool.js:270:12)
at Connection.g (events.js:291:16)
at emitTwo (events.js:106:13)
at Connection.emit (events.js:191:7)
How about on disconnect just reconnect to mongo. See below:
mongoose.connect(config.mongoUrl);
var db = mongoose.connection;
db.on('error', console.error.bind(console, 'error connecting with mongodb database:'));
db.once('open', function() {
console.log('connected to mongodb database');
});
db.on('disconnected', function () {
//Reconnect on timeout
mongoose.connect(config.mongoUrl);
db = mongoose.connection;
});
You can also set a timeout value on the connection.
mongoose.connect(url, { server: { socketOptions: { connectTimeoutMS: 1000 }}}, function(err) { ... });
Also, make sure that mongo is still running on your machine. A connection timeout could mean mongo isn't running.
Reference: Another stack overflow question
Check mongod is running.
typing mongo in shell.
Add connectTimeoutMS=300000 paramter for you uri.
uri looks like mongodb://localhost/collectionName?connectTimeoutMS=300000

Can't do basic find with Waterline in sails console

I have a Job model that searches a table of jobs. The table was created using rails, and I set 'migrate' to 'safe' in the Job model. I have written out all the attributes in the model. I cannot get basic functions to work in the sails console. I have three jobs in the table, with ids 1,2, and 3. When I execute:
sails> Job.findOneById(1).done(function(err,job){console.log(j.id)})
It poops out on me. The sails console crashes (super annoying). The output looks like this:
sails> Job.findOneById(1).done(function(err,job){console.log(j.id)})
undefined
sails>
/home/redmage/tac/node_modules/sails-postgresql/lib/query.js:544
var type = self._schema[key].type;
^
TypeError: Cannot read property 'type' of undefined
at /home/redmage/tac/node_modules/sails-postgresql/lib/query.js:544:33
at Array.forEach (native)
at Query.cast (/home/redmage/tac/node_modules/sails-postgresql/lib/query.js:541:23)
at /home/redmage/tac/node_modules/sails-postgresql/lib/adapter.js:372:32
at Array.forEach (native)
at __FIND__ (/home/redmage/tac/node_modules/sails-postgresql/lib/adapter.js:371:23)
at Query.handleReadyForQuery (/home/redmage/tac/node_modules/sails-postgresql/node_modules/pg/lib/query.js:84:10)
at null.<anonymous> (/home/redmage/tac/node_modules/sails-postgresql/node_modules/pg/lib/client.js:159:19)
at EventEmitter.emit (events.js:117:20)
at null.<anonymous> (/home/redmage/tac/node_modules/sails-postgresql/node_modules/pg/lib/connection.js:97:12)
at Socket.EventEmitter.emit (events.js:95:17)
at Socket.<anonymous> (_stream_readable.js:746:14)
at Socket.EventEmitter.emit (events.js:92:17)
at emitReadable_ (_stream_readable.js:408:10)
at emitReadable (_stream_readable.js:404:5)
at readableAddChunk (_stream_readable.js:165:9)
at Socket.Readable.push (_stream_readable.js:127:10)
at TCP.onread (net.js:528:21)
Here is the ORM for Job. It is very basic:
module.exports = {
//VERY IMPORTANT:
migrate: 'safe',
tableName: 'jobs',
attributes: {
}
}