We're using HashiCorp's Vault to store database connection credentials, then constructing the connection string for pg-promise from those. The 'catch' is that the Vault details are provided from a Promise wrapper, due to request callbacks to the Vault API.
Example database.js module:
const pgp = require('pg-promise')(/* options obj */);
const getDbo = () => {
return new Promise( (resolve, reject) => {
vault.init().then(secrets => {
let credentials = secrets.dbUser + ':' + secrets.dbPass
let connStr = 'postgres://' + credentials + '<#endpoint/db>'
let dbo = pgp(connStr, (err) => {
reject(err)
})
resolve(dbo);
})
}
module.exports = { get: getDbo }
This is being imported in multiple routes. With this we are seeing the warning "WARNING: Creating a duplicate database object for the same connection." Is there a better way to resolve this so there is only one object per connection details?
Creating and initializing a connection for pg-promise is a completely synchronous operation, as per the API, so there is no point using promises for that.
For initializing the library see Where should I initialize pg-promise.
See also:
Verify database connection with pg-promise when starting an app.
Related
I'm making a multi tenant app using mongo db and would like to know what the proper procedure between switching between databases is. I know I can get a new reference to a database using the db() command:
const client = await MongoClient.connect(url);
client.mainDb = client.db('main');
app.set('mongoClient', client);
On bootup I get and store a reference to my main for all my global app data. Then each request also passes in a tenant id. I'm using Feathersjs which provides me with a hook for every request before and after.
In my before hook, I get a reference to the clients data and store it to be used during that singular request:
app.hooks({
before: {
all: [(context) => {
// Run before all API requests
const tenant = context.params?.query?.$tenant;
const tenantDbName = ... // some logic to query the tenant db name
const client = context.app.get('mongoClient');
context.params.tenantDb = client.db(tenantDbName);
}]
}
}
After the request, I'm unclear on if I should do anything to cleanup the connection. Do I just let the garbage collector clean it up since its request that was made which has ended? Or is there a function in Mongo to clean it up?
app.hooks({
after: {
all: [(context) => {
// Cleanup DB or reset connection?
context.params.tenantDb = null;
}]
}
}
I just need to ensure that the next request doesn't use a previous requests database as this could serve them other users data.
I have a use case that many connections to the database are created dynamically using pg-promise. Sometimes I need to connect again to the same database and user however the password changed.
Is there a way to update an existing connection so I dont get the "WARNING: Creating a duplicate database object for the same connection."?
Editing for better explanation:
Context
I have a non-traditional application that is a node service that handles geospatial data aquisition in the software QGIS, with Postgres + PostGIS.
This application creates temporary users in the PostgreSQL server and manage permissions on the tables and columns based on the type of work the user needs to do.
Code
const dbs = {} //global variable that stores all connections
const getConnection = async (user, password, server, port, dbname) => {
const connString = `postgres://${user}:${password}#${server}:${port}/${dbname}`
if (connString in dbs) {
return dbs[connString] //if connection already exists returns the connection
}
dbs[connString] = db.pgp(connString) //create new connection
await dbs[connString] //tests if connections is correct
.connect()
.then(obj => {
obj.done() // success, release connection;
})
.catch(e => {
errorHandler.critical(e)
})
return dbs[connString]
}
What I want is add another case, that if the connection already exists but the password changed it updates the existing connection password (or destroy it and create a new one).
The issue in your case is that you are using password as part of the connection-string key, which isn't used within the library's unique-connection check, hence the side effect.
For the key, you need to use a unique connection string that does not contain the password. And when the request is made, you need to update the connection details.
Example below makes use of the connection object, not the connection string, because it is simpler that way. But if you want, you can use a connection string too, you would just need to generate a separate connection string, with the password, and update $pool.options.connectionString, not $pool.options.password.
const dbs = {}; // global variable that stores all connections
const getConnection = async (user, password, host, port, database) => {
const key = `${user}#${host}:${port}/${database}`; // unique connection key
const cn = { host, port, database, user, password }; // actual connection
let db; // resulting database object
if (key in dbs) {
db = dbs[key];
db.$pool.options.password = password; // updating the password
} else {
db = pgp(cn); // creating new connection
dbs[key] = db;
await db // test if can connect
.connect()
.then(obj => {
obj.done(); // success, release connection;
})
.catch(e => {
errorHandler.critical(e);
throw e;
})
}
return db;
}
I'm trying to set session level variable on a connection in pg-promise, the variable value will be read by trigger in the database level with current_setting('var_name'). This session variable will be different for each user, while I'm also sharing the same database connection for all users.
I found this somewhat related question utilizing the connect event, but I have concern that since I'm sharing the same connection for all users that the session var will not be set correctly when different users call the query method.
Is there another way to safely set this session var and make sure that it's isolated for each user while still sharing the same database object?
const pgPromise = pgp({
promiseLib: Promise, // overriding the default (ES6 Promise)
async connect(client, dc, useCount) {
// const cp = client.connectionParameters;
// console.log('Connected to database:', cp.database, dc);
if (dc && dc.email) {
console.log(useCount);
const email = encodeURI(dc.email);
await client.query(`SET SESSION "app.user" = '${email}'`);
}
}
});
get tenantDb() {
const state = this.request.sessionState();
const config = { host, database, port };
// pass state as database context,
// we'll get warning of duplicate database object for the same connection
const connection = pgPromise(config, state);
return connection;
}
Update
It turns out that I need to upgrade pg-promise version to the latest version, I was using v7 which doesn't differentiate the connection object based on the context, once I upgrade pg-promise to v10 the warning disappear, a more optimized solution would be if we can somehow set the session settings along with the schema callback in the initOptions parameter when we initiate the database, that way we only need one extra query execution together with the schema.
I am working on a SANDBOX Cluster & a new app created by me in MongoDB Stitch.
I need to understand "Payload Signature Verification" in MongoDB Stitch App. Lets say, I need to make a REST GET API, which will fetch me a list of products, but this API call must be authenticated ie. only registered/authenticated users will be able to make this call. MongoDB Stitch suggests below to do that:
https://docs.mongodb.com/stitch/services/webhook-requests-and-responses/#webhook-verify-payload-signature
But, i need to understand:
(1) Where to add this BODY & SECRET ? As per my knowledge, it must be kept in the stitch app, as you must not expose any of your secret keys in client side javascript.
(2) { "message":"MESSAGE" } is this configurable? if yes, what value should we add here?
This function must be coded in MongoDB Stitch App. That is clear. This function returns "hash" based on the "body" & "secret" you pass in earlier step.
And now, you must pass this hash in your API Request:
Now, the question is:
You can easily see any request which is being passed to server in developer tools, anybody can easily copy it & pass it same through POSTMAN. So:
-> How do i secure my requests? (FYI: I have also added "RULES", saying this request must execute only if the domain name contains lets say, www.mysite.com. But i am able to execute the request successfully from localhost.)
-> If, anybody can copy & paste my request in POSTMAN & run it. SO, what is the use of generating that HASH ?
-> How do i keep my request(s) tokens alive/valid for limited period of time, lets say request is valid only for next 5 minutes ? (i mean how do i do this in Stitch APP ? Where is that Option ?)
-> How do i get the refresh token ? & even if i get it somehow, how do i re-pass it to the request ?
All such queries are UN_ANSWERED in MongoDB Stich Documentation : https://docs.mongodb.com/stitch/
Basically i want to understand the full life-cycle of any GET/POST/PUT/PATCH/DELETE request of MongoDB Stitch App / Stitch REST APIs.
If anybody have used MongoDB Stich, please explain me.
I don't know your specific use-case, though I also had issues with creating an Authenticated HTTP REST API. My idea was: I already have all security rules and schemas defined in Stitch, now I want to access the data over HTTP still using the logic defined in Stitch and not rewriting everything.
I wasn't able to create such API with Stitch functions and Webhooks, though I created an API server in (literally) 1 hour with NodeJS Koa (express or any other framework would do) and Stitch server SDK:
// app.js
const Koa = require('koa')
const app = module.exports = new Koa()
const auth = require('./auth')
const router = require('./router')
app.use(auth())
app.use(router.routes())
app.use(router.allowedMethods())
// listen
if (!module.parent) {
app.listen(3000)
}
// auth.js
const { loginWithApiKey } = require('./stitch')
function auth () {
return async function auth (ctx, next) {
const apiKey = ctx.query.api_key
try {
await loginWithApiKey(apiKey)
} catch (e) {
ctx.throw(401, 'Not Authorized')
}
await next()
}
}
module.exports = auth
// router.js
const router = require('koa-router')()
const { BSON } = require('mongodb-stitch-server-sdk')
const { db } = require('./stitch')
router.get('/', async (ctx) => {
ctx.body = { message: 'Nothing to see, but you\'re good!' }
})
const COLLECTIONS_WHITELIST = [
'activities',
'expenses',
'projects',
'resources'
]
// List
router.get('/:collection', async (ctx) => {
const collection = ctx.params.collection
isValidCollection(ctx, collection)
ctx.body = await db
.collection(collection)
.find()
.toArray()
})
function isValidCollection (ctx, collection) {
// check if the collection is allowed in the API
if (!COLLECTIONS_WHITELIST.includes(collection)) {
ctx.throw(404, `Unknown API entity ${collection}`)
}
}
module.exports = router
I hope it helps
Connects without a hitch, but on insert() throws me this error.
var MongoClient = require('mongodb').MongoClient;
const assert = require('assert');
var url = 'mongodb://____:____#ds125565.mlab.com:25565/heroku_w268n9pj';
MongoClient.connect(url, function(err, client) {
assert.equal(null, err);
db = client.db('temp');
console.log("connected!");
const collection = db.collection('temp');
collection.insert([{
something: please
}
});
I saw some other answers regarding mLab accounts and credentials, but I just created a new admin account for this. Frustrating because it was working previously with v2.3.
When attempting to connect to an mlab database, you have to correctly specify the client. It's located at the end of your connection string, just after the final forward slash.
mlab_url = "mongodb://db_user_name:db_password#ds139725.mlab.com:39725/heroku_sd1fp182?retryWrites=false"
client = MongoClient(url)
db = client["heroku_sd1fp182"]
collection = db["coinHack"]
You may also get the error:
This MongoDB deployment does not support retryable writes. Please add retryWrites=false to your connection string.
Just add "?retryWrites=false" to your connection string, as shown above.