Vercel creates new DB connection for every request - mongodb

Question

I'm working on a new website, and although things were working well as we developed locally we've run into an issue when we tried to deploy on Vercel. The app uses the Sapper framework for both the pages and an API, and a database in MongoDB Atlas that we access through Mongoose. The behavior we have locally is that we npm run dev and a single DB connection is made which persists until we shut the app down.
When it gets deployed to Vercel though, the code which makes the DB connection and prints that "DB connection successful" message and is only supposed to run once is instead run on every API request
This seems to quickly get out of hand, reaching our database's limit of 500 connections:
As a result, after the website is used briefly even by a single user some of our API requests start failing with this error (We have the db accepting any connection rather than an IP whitelist, so the suggestion the error gives isn't helpful):
Our implementation is that we have a call to mongoose.connect in a .js file:
mongoose.connect(DB, {
useNewUrlParser: true,
useCreateIndex: true,
useFindAndModify: false,
useUnifiedTopology: true
}).then(() => console.log("DB connection successful!")).catch(console.error);
and then we import that file in Sapper's server.js. The recommendation we've been able to find is "just cache the connection", but that hasn't been successful and seems to be more of a node-mongodb-native thing. Regardless, this is what we tried which didn't work better or worse locally, but also didn't fix the problems on Vercel:
let cachedDb = {};
exports.connection = async () => {
if (cachedDb.isConnected)
return;
try {
const db = await mongoose.connect(DB, {
useNewUrlParser: true,
useCreateIndex: true,
useFindAndModify: false,
useUnifiedTopology: true
});
cachedDb.isConnected = db.connections[0].readyState;
console.log("DB connection successful!");
return cachedDb;
} catch(error) {console.log("Couldn't connect to DB", error);}
So... is there a way to make this work without replacing at least one of the pieces? The website isn't live yet so replacing something isn't the end of the world, but "just change a setting" is definitely preferred to starting from scratch.

Answer 1

Summary
Serverless functions on Vercel work like a self-contained process. While it is possible to cache the connection "per function," it is not a good idea to deploy a serverful-ready library to a serverless environment. Here are a few questions that you need to answer:
Is your framework or DB library caching the connection?
Is your code prepared for Serverless?
What type of workload is Vercel optimized for?
Further Context
Vercel is an excellent platform for your frontend that would use Serverless Functions as helpers. The CDN available in conjunction with the workflow makes the deployment process very quick and allows you to move faster. Deploying a full-blown API or serverful workload will never be a good idea. Let's suppose I need to use MySQL with Vercel. Instead of mysql, you should use mysql-serverless, which is optimized for the serverless primitives. Even with that in mind, it will be probably cheaper to just use a VM/Container for the API depending on the level of requests you are expecting. Therefore, we would end up with the following ideal solution:
Frontend (Vercel - Serverless) --> Backend (Serverful - External provider) --> DB
Disclaimer: At the moment, I work for Vercel.

Answer 2

If you are using the cloud database of MongoDB Atlas, then you can use the mongodb-data-api library, which is encapsulated based on the Data API of MongoDB Atlas. All data operations are performed through the HTTPS interface, and there is no connection problem.
import { MongoDBDataAPI, Region } from 'mongodb-data-api'
const api = new MongoDBDataAPI({
apiKey: '<your_mongodb_api_key>',
appId: '<your_mongodb_app_id>'
})
api
.findOne({
dataSource: '<target_cluster_name>',
database: '<target_database_name>',
collection: '<target_collection_name>',
filter: { name: 'Surmon' }
})
.then((result) => {
console.log(result.document)
})

Answer 3

The example codes provided by NextJS say to cache the database connection yet this is the issue that happens with myself as well.
Both here
https://github.com/vercel/next.js/blob/canary/examples/with-mongodb-mongoose/utils/dbConnect.js
And here
https://github.com/vercel/next.js/blob/canary/examples/with-mongodb/util/mongodb.js
are caching the connection and if I copy the example i get the same issue as the OP.
It also says here
https://nextjs.org/docs/basic-features/data-fetching#getstaticprops-static-generation
that i can interact directly with my database. Massively conflicting information where I'm told on one hand to cache the connection, while a member of the team tells me its not suitable for this approach despite the docs & examples telling me otherwise. Is this a bug report type situtation?

Answer 4

I was struggling with the similar issue but I came across an example here:
https://github.com/vercel/next.js/blob/canary/examples/with-mongodb/util/mongodb.js
Apparently the trick is to use the global variable:
let cached = global.mongo
if (!cached) {
cached = global.mongo = { conn: null, promise: null }
}