Connecting to a remote MongoDB using Meteor - mongodb

Apologies in advance for any failing with my terminology and understanding with Meteor/Mongo, I've just started learning and developing with it.
I am trying to connect my local meteor app to a remote mongodb which is hosted elsewhere.
My code looks like this:
Bills = new Mongo.Collection("bills");
if (Meteor.isClient) {
Meteor.subscribe("bills");
// This code only runs on the client
Template.body.helpers({
documentContent: function () {
return Bills.find();
}
});
Template.documentBody.helpers({
documentContent: function ()
{
var thingy = Bills.find();
console.log(thingy);
return Bills.find({_id: "784576346gf874"});
}
});
}
I have connected to the DB via the shell using the following:
$ MONGO_URL="mongodb://mysite.net:27017/legislation" meteor
In my browser I receive no errors and within my defined template I see [object Object]. The console shows a local miniCollection but doesn't return any of my documents from the subscribed collection.
I guess what I am asking is; if you were connecting to a remote MongoDB within your local app, how would you do it?
Thank you for taking the time to read, any helps is massively appreciated.

Rex, If you're not seeing errors in the output on the browser, or in the console where you're running the server then you may be setup ok. That's exactly how I'm doing it.
Run meteor list in server directory and look for insecure and autopublish
You should understand these two packages They are for rapid prototyping. If they are present, then keep digging into MongoDB and the connection.
I recommend Robomongo for viewing documents directly in MongoDB.
If they are absent, then you need to go about publishing data (getting it from the server to the client) and securing it (letting clients only modify their data).
I recommend these two packages for that.
reywood:publish-composite
ongoworks:security
If you haven't read an introduction to meteor book, it's really worth the time. I've been developing for some time and learned meteor recently. It was invaluable.

Related

Vercel creates new DB connection for every request

I'm working on a new website, and although things were working well as we developed locally we've run into an issue when we tried to deploy on Vercel. The app uses the Sapper framework for both the pages and an API, and a database in MongoDB Atlas that we access through Mongoose. The behavior we have locally is that we npm run dev and a single DB connection is made which persists until we shut the app down.
When it gets deployed to Vercel though, the code which makes the DB connection and prints that "DB connection successful" message and is only supposed to run once is instead run on every API request
This seems to quickly get out of hand, reaching our database's limit of 500 connections:
As a result, after the website is used briefly even by a single user some of our API requests start failing with this error (We have the db accepting any connection rather than an IP whitelist, so the suggestion the error gives isn't helpful):
Our implementation is that we have a call to mongoose.connect in a .js file:
mongoose.connect(DB, {
useNewUrlParser: true,
useCreateIndex: true,
useFindAndModify: false,
useUnifiedTopology: true
}).then(() => console.log("DB connection successful!")).catch(console.error);
and then we import that file in Sapper's server.js. The recommendation we've been able to find is "just cache the connection", but that hasn't been successful and seems to be more of a node-mongodb-native thing. Regardless, this is what we tried which didn't work better or worse locally, but also didn't fix the problems on Vercel:
let cachedDb = {};
exports.connection = async () => {
if (cachedDb.isConnected)
return;
try {
const db = await mongoose.connect(DB, {
useNewUrlParser: true,
useCreateIndex: true,
useFindAndModify: false,
useUnifiedTopology: true
});
cachedDb.isConnected = db.connections[0].readyState;
console.log("DB connection successful!");
return cachedDb;
} catch(error) {console.log("Couldn't connect to DB", error);}
So... is there a way to make this work without replacing at least one of the pieces? The website isn't live yet so replacing something isn't the end of the world, but "just change a setting" is definitely preferred to starting from scratch.
Summary
Serverless functions on Vercel work like a self-contained process. While it is possible to cache the connection "per function," it is not a good idea to deploy a serverful-ready library to a serverless environment. Here are a few questions that you need to answer:
Is your framework or DB library caching the connection?
Is your code prepared for Serverless?
What type of workload is Vercel optimized for?
Further Context
Vercel is an excellent platform for your frontend that would use Serverless Functions as helpers. The CDN available in conjunction with the workflow makes the deployment process very quick and allows you to move faster. Deploying a full-blown API or serverful workload will never be a good idea. Let's suppose I need to use MySQL with Vercel. Instead of mysql, you should use mysql-serverless, which is optimized for the serverless primitives. Even with that in mind, it will be probably cheaper to just use a VM/Container for the API depending on the level of requests you are expecting. Therefore, we would end up with the following ideal solution:
Frontend (Vercel - Serverless) --> Backend (Serverful - External provider) --> DB
Disclaimer: At the moment, I work for Vercel.
If you are using the cloud database of MongoDB Atlas, then you can use the mongodb-data-api library, which is encapsulated based on the Data API of MongoDB Atlas. All data operations are performed through the HTTPS interface, and there is no connection problem.
import { MongoDBDataAPI, Region } from 'mongodb-data-api'
const api = new MongoDBDataAPI({
apiKey: '<your_mongodb_api_key>',
appId: '<your_mongodb_app_id>'
})
api
.findOne({
dataSource: '<target_cluster_name>',
database: '<target_database_name>',
collection: '<target_collection_name>',
filter: { name: 'Surmon' }
})
.then((result) => {
console.log(result.document)
})
The example codes provided by NextJS say to cache the database connection yet this is the issue that happens with myself as well.
Both here
https://github.com/vercel/next.js/blob/canary/examples/with-mongodb-mongoose/utils/dbConnect.js
And here
https://github.com/vercel/next.js/blob/canary/examples/with-mongodb/util/mongodb.js
are caching the connection and if I copy the example i get the same issue as the OP.
It also says here
https://nextjs.org/docs/basic-features/data-fetching#getstaticprops-static-generation
that i can interact directly with my database. Massively conflicting information where I'm told on one hand to cache the connection, while a member of the team tells me its not suitable for this approach despite the docs & examples telling me otherwise. Is this a bug report type situtation?
I was struggling with the similar issue but I came across an example here:
https://github.com/vercel/next.js/blob/canary/examples/with-mongodb/util/mongodb.js
Apparently the trick is to use the global variable:
let cached = global.mongo
if (!cached) {
cached = global.mongo = { conn: null, promise: null }
}

Use AWS Amplify and App Sync with existing Node Server using Mongodb

Currently, I'm developing a native application using React-Native. I've decided to go with AWS Amplify because of it's real time updates as well as its authentication.
I also have a Web Application that runs on a Node.js with Epxress server. This web application connects to a Mongo database.
My big problem is that I would like to have all of my aws amplify queries run to my existing MongoDb instead of a new dynamoDb database which is provided with AWS AppSync, but unfortunately I dont know where to start. This is especially helpful in adding authentication easily in my existing web application as well.
My first idea was to just create all my API endpoints in a new node js server and have app sync call to these API end points, but I'm not sure how to implement calling end points on an existing server (and this seems kind of counter intuitive to the 'serverless' idea)
My other idea came from this: Can AWS App-Sync be used without dynamoDB
This states to use AWS Lambda to 'pipeline' my data to the existing mongodb, but I'm not really sure what that entails.
TL;DR - I would like to be able to query an existing Mongodb instead of using DynamoDb when using AWS Amplify with AppSync.
I hope this is clear enough and doesn't sound like I'm rambling. Thanks in advance!
I would suggest using either an HTTP datasource to connect to your MongoDB backend or a Lambda function. Here are a couple getting started tutorials for both:
https://docs.aws.amazon.com/appsync/latest/devguide/tutorial-http-resolvers.html
https://docs.aws.amazon.com/appsync/latest/devguide/tutorial-lambda-resolvers.html
If you go the Lambda route, then you can leverage the new #function feature of the GraphQL Transformer in the Amplify CLI: https://aws-amplify.github.io/docs/cli/graphql#function

Import test-users into Meteor app with use of npm script?

We can easily create a user from meteor shell like this
Accounts.createUser({username: 'john', password: '12345'})
Similarly, I just want to add multiple users via npm script. Any ideas?
In other words, I want to use fixtures functionality via npm command and not on the initial run.
Thank you.
For normal collections (i.e. different than Meteor.users), you can directly tap into your MongoDB collection. Open a Meteor Mongo shell while your project is running in development mode, then directly type Mongo shell commands.
For Meteor.users collection, you want to leverage the accounts-base and accounts-password packages automatic management, so instead of directly fiddling the MongoDB, you want to insert documents / users through your Meteor app.
Unfortunately, your app source files (like your UsersFixtures.js file) are absolutely not suitable for CLI usage.
The usual solution is to embed a dedicated method within your app server:
// On your server.
// Make sure this Method is not available on production.
// When started with `meteor run`, NODE_ENV will be `development` unless set otherwise previously in your environment variables.
if (process.env.NODE_ENV !== 'production') {
Meteor.methods({
addTestUser(username, password) {
Accounts.createUser({
username,
password // If you do not want to transmit the clear password even in dev environment, you can call the method with 2nd arg: {algorithm: "sha-256", digest: sha256function(password)}
})
}
});
}
Then start your Meteor project in development mode (meteor run), access your app in your browser, open your browser console, and directly call the method from there:
Meteor.call('addTestUser', myUsername, myPassword)
You could also use Accounts.createUser directly in your browser console, but it will automatically log you in as the new user.

Hosting the database separately for Meteor apps

It seems to be a common and safer practice to host the database separately from Meteor apps. That is to say, have an EC2 instance for your Meteor app, and an EC2 instance for your MongoDB, and make them talk to one another.
From what I understand, people do this because it's more secure, and it allows them to deploy newer versions of their app without touching the database.
I'd like to do this with Amazon EC2 alone, as opposed to using another 3rd party service, like Compose.io.
How can I host a Meteor app and its database separately on two EC2 instances, and have them communicate with one another?
It is common practice, and people mostly do it because it offers you the ability to scale them both independently.
As to the how, you'll want to obviously configure each of your Amazon EC2 instances, installing meteor on one, and MongoDB on the other. You'll also need to configure your VPC (Amazon Virtual Private Cloud) so that your MongoDB instance accepts incoming connections on whatever port you specify (default is 27017), so that your Meteor Application can connect.
After that it's just a matter of telling your meteor app where to go to get the database connection. The most secure way of doing this will be to set a couple Environment Variables, named MONGODBSERVER and MONGODBPORT, DBUSER, DBPASSWORD, etc.
You'll then want to set some variables in your server Meteor code, using something like:
Meteor.startup(function() {
var DbUser = process.env.DBUSER;
var DbPassword = process.env.DBPASSWORD;
var MongoDBServer = process.env.MONGODBSERVER;
var MongoDBPort = process.env.MONGODBPORT;
});
And if you're using the native MongoDB Driver, connecting becomes trivial:
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect('mongodb://DbUser:DbPassword#MongoDBServer:MongoDBPort/databasename', function(err, db) {
...
});
Then it's just a matter of constructing your Mongo models using something like:
Temperatures = new Mongo.Collection('temperatures');
Temperatures._ensureIndex({temp: 1, time: 1});
And then taking action on those models in regard to the database:
Temperatures.insert({temp: ftemp, time: Math.floor(Date.now() / 1000)});
I'll also mention that http://modulus.io is a really decent Meteor hosting solution. I'd recommend them, unless you are stuck on using Amazon EC2 instances, which is fine, but more complicated for a simple application.
You need to set an Environment Variable for Mongo where it is hosted
MONGO_URL
mongodb://:#hostingproviderurl:port/xxx?autoReconnect=true&connectTimeoutMS=60000
the correct mongodb:// url string would be provided by the mongodb hosting provider.

Scala Lift - Connect to remote MongoDB

I currently have my app running on my local machine, in Boot.scala I have:
MongoDB.defineDb(
DefaultMongoIdentifier,
MongoAddress(MongoHost("127.0.0.1", 27017), "platform")
)
I've successfully deployed the app to a cloud provider, and am in the process of setting up a database # mongohq.com
What would I need to change to enable the app to connect? I've taken a look here:
https://www.assembla.com/wiki/show/liftweb/Mongo_Configuration
But am a little confused by the connection details provided by mongohq, all they provide is:
Mongo URI
mongodb://<user>:<password>#<host>:<port>/<my_account_name>
Thanks in advance for any help, much appreciated :)
I am not familiar with MongoHQ in particular, but you should be able to put something in Boot like this:
MongoDB.defineDbAuth(
DefaultMongoIdentifier,
new Mongo(new ServerAddress("<host>", <port>)),
<my_account_name>,
<user>,
<pass>
)
Where the <*> variables are the particular part of the connection URI that were provided to you when you signed up for MongoHQ.