Why is the database empty, even after a successful import? - mongodb

I attempted to import a Json file today on my macbook, but I got an error:
Davids-MacBook-Pro:server Droberts$ node import.js
/filepath/A_to_B.json
development
mongodb://localhost/somefoldername
{ db: { safe: true } }
A
B
/Users/Droberts/Desktop/folderA/folderB/node_modules/mongoose/node_modules/mongodb/lib/server.js:236
process.nextTick(function() { throw err; })
^
Error: connect ECONNREFUSED
at exports._errnoException (util.js:746:11)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1010:19)
However, after running mongod in a new tab, I was able to rerun the command successfully:
Davids-MacBook-Pro:server Droberts$ node import.js
/filepath/A_to_B.json
development
mongodb://localhost/somefoldername
{ db: { safe: true } }
A
B
adding data to mongodb
finished import
However, the most startling thing is, my database remains empty!:
Davids-MacBook-Pro:server Droberts$ mongo
MongoDB shell version: 3.2.1
connecting to: test
Server has startup warnings:
2016-01-15T17:50:09.158-0500 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2016-01-15T17:50:09.158-0500 I CONTROL [initandlisten]
2016-01-15T17:50:09.158-0500 I CONTROL [initandlisten]
2016-01-15T17:50:09.158-0500 I CONTROL [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
> show dbs
somefoldername 0.000GB
local 0.000GB
What on earth is going on?!!! Here is the source for node.js:
'use strict';
// Set default node environment to development
process.env.NODE_ENV = process.env.NODE_ENV || 'development';
console.log(process.env.NODE_ENV);
var mongoose = require('mongoose');
var config = require('./config/environment');
var CityPair = require('./api/citypair/citypair.model');
console.log(config.mongo.uri)
console.log(config.mongo.options)
// Connect to database
mongoose.connect(config.mongo.uri, config.mongo.options);
var fs = require('fs');
var file = JSON.parse(fs.readFileSync(process.argv[2], 'utf8'));
console.log(file.startCity.name)
console.log(file.endCity.name)
//CityPair.find({}).remove(
CityPair.find({
$or: [{startCity : file.startCity, endCity : file.endCity},
{startCity : file.endCity, endCity : file.startCity}]
}).remove(
function() {
console.log('adding data to mongodb');
CityPair.create(file, function(err){
console.log('finished import');
process.exit();
});
}
);

Related

Deno Uncaught Error: No such host is known. (os error 11001)

I tried to connect the MongoDB Atlas to my Deno Application using https://deno.land/x/mongo#v0.21.2 framework. I tried the below code to run my application. but I get an error No such host is known. (os error 11001) What went wrong here
Error
error: Uncaught Error: No such host is known. (os error 11001)
at unwrapResponse (rt\10_dispatch_json.js:24:13)
at sendAsync (rt\10_dispatch_json.js:75:12)
at async Object.connect (rt\30_net.js:221:13)
at async MongoClient.connect (client.ts:41:14)
at async mongodb.ts:33:1
Mongodb.ts File
import { MongoClient } from "https://deno.land/x/mongo#v0.21.0/mod.ts";
const client1 = new MongoClient();
await client1.connect("mongodb+srv://user1:MYPASSWORD#cluster0.hmdnu.mongodb.net/TestingDB?retryWrites=true&w=majority");
const db = client1.database("TestingDB");
export default db;
I used this command to run my server
deno run --allow-net --allow-write --allow-read --allow-plugin --unstable server.ts
I fixed this using https://www.youtube.com/watch?v=hhdhydffKKE this video reference
Follow these steps to fix this
import { MongoClient } from "https://deno.land/x/mongo#v0.21.0/mod.ts";
const client1 = new MongoClient();
await client.connect({
db: "<db>",
tls: true,
servers: [
{
host: "<host>",
port: 27017,
},
],
credential: {
username: "<user>",
password: "<password>",
db: "<db>",
mechanism: "SCRAM-SHA-1",
},
});
const db = client1.database("TestingDB");
export default db;
This is not mentions in the document, but this will help to fix the issue
<db> is the database name, you can get the database name by following these steps
Step-1
Step-2
To find the <host> follow these steps
Step -1
Step -2
Step -3

Unknown auth message code 1397113172 when connect to Heroku postgres

Thanks reading my issue.
Currently, I am using postgres (hobby-dev) on Heroku and facing this issue every time that I connect to the database.
error: Uncaught (in promise) Error: Unknown auth message code 1397113172
throw new Error(`Unknown auth message code ${code}`);
^
at Connection.handleAuth (connection.ts:197:15)
at Connection.startup (connection.ts:155:16)
at async Pool._createConnection (pool.ts:32:5)
at async pool.ts:61:7
at async Promise.all (index 0)
at async Pool._startup (pool.ts:63:25)
My application using Deno now
import { Pool } from "https://deno.land/x/postgres/mod.ts";
import { config } from "./config.ts";
const port = config.DB_PORT ? parseInt(config.DB_PORT || "") : undefined;
const POOL_CONNECTIONS = 20;
const dbPool = new Pool({
port,
hostname: config.DB_HOST,
user: config.DB_USER,
database: config.DB_NAME,
password: config.DB_PASS
}, POOL_CONNECTIONS);
export { dbPool };
Here is debug screen.
I have found this issue post and it mentioned about lacking ssl. Not sure how to do it on heroku.
I have tried some solutions, even change lib to pg and it still not work. I am very appreciated if any clue or help to fix this issue.
Note:
I read a document on heroku about "Heroku Postgres Connection Pooling is not available for Hobby-tier databases.". Then I switched to use Client with syntax similar like this to connect to Heroku postgres this:
import { Client } from "https://deno.land/x/postgres/mod.ts";
let config;
config = {
hostname: "localhost",
port: 5432,
user: "user",
database: "test",
applicationName: "my_custom_app"
};
// alternatively
config = "postgres://user#localhost:5432/test?application_name=my_custom_app";
const client = new Client(config);
await client.connect();
await client.end();
ref: https://deno-postgres.com/#/

Connect with mongodb server on digital ocean

I followed
DigitalOcean Mongodb Install
sudo ufw allow from your_other_server_ip/32 to any
I set your other server as 127.0.01 as I will be connecting it with express local device as localhost:3000
sudo ufw allow from 127.0.0.1/32 to any
and created admin user.
I have also updated mongodb.conf to
logappend=true
bind_ip = 127.0.0.1,139.**.*.**
port = 27017
How can I make connection with mongoose now.
I tried with a gui with ssh connection and it worked.
How can I connect it with HTTP URL.
EDIT - 1
I installed mongodb
https://www.digitalocean.com/community/tutorials/how-to-install-mongodb-on-ubuntu-18-04
And with
https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-mongodb-on-ubuntu-16-04#part-two-securing-mongodb
Step 3 — Testing the Remote Connection
I am getting
MongoDB shell version v3.6.3
Enter password:
connecting to: mongodb://139.**.*.***:27017/
MongoDB server version: 3.6.3
And running
> show users
{
"_id" : "testdb.testusr",
"user" : "testusr",
"db" : "testdb",
"roles" : [
{
"role" : "readWrite",
"db" : "testdb"
}
]
}
If i try to connect with mongoose with below code
var connectionString = "mongodb://testusr:testpwd#139.**.*.***:27017/testdb";
mongoose
.connect(connectionString, {
keepAlive: 1,
useUnifiedTopology: true,
useNewUrlParser: true,
})
.then(() => console.log('DB Connected!'))
.catch(err => {
console.log(`DB Connection Error: ${err.message}`);
});
I am getting below output
DB Connection Error: Server selection timed out after 30000 ms

How to connect to mongodb with authorization in meanjs?

I'm new to meanjs.
My problem is: my local mongodb has username/password, and I can't run meanjs demo successfully.
I git clone https://github.com/Azure-Samples/meanjs.git
And try to run it(npm run on Windows 10), but got this error:
Could not connect to MongoDB!
{ MongoError: Authentication failed.
at Function.MongoError.create (D:\Cordova\AzureMeanDemo\meanjs\node_modules\mongoose\node_modules\mongodb-core\lib\error.js:31:11)
then I set username/password in meanjs\config\env\development.js like:
db: {
uri: process.env.MONGOHQ_URL || process.env.MONGODB_URI || 'mongodb://' + (process.env.DB_1_PORT_27017_TCP_ADDR || 'localhost') + '/mean-dev',
options: {
user: 'superAdmin', //here
pass: '*****' //here
},
// Enable mongoose debug mode
debug: process.env.MONGODB_DEBUG || false
},
...
but I still got the same error.
I've searched on google for a whole day, but got nothing about this.
Could anyone help me on this problem?
-------------- Solved -------------------
I changed uri: 'mongodb://username:passwd#localhost/mean-dev?authSource=adm‌​in'. It worked for me.
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/projectName,{user:'',pass:''});

Node and Mongoose - Not reconnecting if mongod was not running when first tried to connect

Im using docker-composer and Im finding issues with execution order of services. The main issue happens when my express app tries to connect to mongod but this is not yet ready.
The issue can be reproduced easily by running first the nodejs application but not mongod (manually forcing this case).
My app uses mongoose and try to establish connection to mongod. Because mongod is not up and running, the app throws an error about it.
$ nodemon server/app.js
24 Apr 21:42:05 - [nodemon] v1.7.0
24 Apr 21:42:05 - [nodemon] to restart at any time, enter `rs`
24 Apr 21:42:05 - [nodemon] watching: *.*
24 Apr 21:42:05 - [nodemon] starting `node server/app.js`
Listening on port 8000
disconnected
connection error: { [MongoError: connect ECONNREFUSED] name: 'MongoError', message: 'connect ECONNREFUSED' }
Starting mongod later seems to reconnect.
24 Apr 21:51:28 - [nodemon] v1.7.0
24 Apr 21:51:28 - [nodemon] to restart at any time, enter `rs`
24 Apr 21:51:28 - [nodemon] watching: *.*
24 Apr 21:51:28 - [nodemon] starting `node server/app.js`
Listening on port 8000
disconnected
connection error: { [MongoError: connect ECONNREFUSED] name: 'MongoError', message: 'connect ECONNREFUSED' }
connected
reconnected
Despite of that, operations that require access to mongo will not come through... neither error is shown
This is the code to connect to mongo using mongoose:
// Starting mongo
mongoose.connect(config.database, {
server:{
auto_reconnect:true,
reconnectTries: 10,
reconnectInterval: 5000,
}
});
// Listening for connection
var mongo = {};
var db = mongoose.connection;
db.on('connected', console.error.bind(console, 'connected'));
db.on('error', console.error.bind(console, 'connection error:'));
db.on('close', console.error.bind(console, 'connection close.'));
db.once('open', function() {
console.log("We are alive");
});
db.on('reconnected', function(){
console.error('reconnected');
});
db.on('disconnected', console.error.bind(console, 'disconnected'));
And here is the route that will try to get data from mongo but fail.
router.post('/auth', function(req, res){
User.findOne({name: req.body.name})
.then(function(user){
if(!user)
{
res.status(401).send({ success: false, message: 'Authentication failed. User not found.' });
}
...
How can I recover from running nodejs before mongo is ready?.
In my case, I created separate function only for mongoose connect method:
const connect = () => {
mongoose.connect('mongodb://localhost:27017/myapp', {
useNewUrlParser: true,
reconnectTries: Number.MAX_VALUE,
reconnectInterval: 500,
poolSize: 10,
});
};
I'm calling it at the same start. I also added Event Handler for error event:
mongoose.connection.on('error', (e) => {
console.log('[MongoDB] Something went super wrong!', e);
setTimeout(() => {
connect();
}, 10000);
});
If mongoose fails to connect because MongoDB is not running, error event handler is fired and setTimeout schedules "custom" reconnect.
Hope it helps.
How long does it take before mongod is ready? Because it seems like this is an edge case issue, where mongod might take a couple of seconds to get ready; and when mongoose is connected it serves requests as expected. Just trying to understand why the slight delay (probably a only a few seconds) is necessary to resolve?
But here is a solution anyway:
You could set up an express middleware to check if mongoose is ready and throw an error if not:
app.use(function(req,res,next){
if (mongoose.Connection.STATES.connected === mongoose.connection.readyState){
next();
} else {
res.status(503).send({success:false, message: 'DB not ready' });
}
});
This should go before you inject your router.
I had the same issue with Mongoose 5+. I was able to get this working by creating a retry function using set timeout.
const mongoose = require('mongoose');
const {
MONGO_USERNAME,
MONGO_PASSWORD,
MONGO_HOSTNAME,
MONGO_PORT,
MONGO_DB,
MONGO_DEBUG,
MONGO_RECONNECT_TRIES,
MONGO_RECONNECT_INTERVAL,
MONGO_TIMEOUT_MS,
} = process.env;
if (MONGO_DEBUG) {
console.log(`********* MongoDB DEBUG MODE *********`);
mongoose.set('debug', true);
}
const DB_OPTIONS = {
useNewUrlParser: true,
reconnectTries: MONGO_RECONNECT_TRIES,
reconnectInterval: MONGO_RECONNECT_INTERVAL,
connectTimeoutMS: MONGO_TIMEOUT_MS,
};
const DB_URL = `mongodb://${MONGO_USERNAME}:${MONGO_PASSWORD}#${MONGO_HOSTNAME}:${MONGO_PORT}/${MONGO_DB}?authSource=admin`;
// Initialize conenction retry counter
let reconnectTriesAlready = 1;
// Connect to database with timeout and retry
const connectWithRetry = () => {
mongoose.connect(DB_URL, DB_OPTIONS).then(() => {
// Connected successfully
console.log('********* MongoDB connected successfully *********');
// Reset retry counter
reconnectTriesAlready = 1;
}).catch(err => {
// Connection failed
console.error(`********* ERROR: MongoDB connection failed ${err} *********`)
// Compare retries made already to maximum retry count
if (reconnectTriesAlready <= DB_OPTIONS.reconnectTries) {
// Increment retry counter
reconnectTriesAlready = reconnectTriesAlready + 1;
// Reconnect retries made already has not exceeded maximum retry count
console.log(`********* MongoDB connection retry after ${MONGO_RECONNECT_INTERVAL / 1000} seconds *********`)
// Connection retry
setTimeout(connectWithRetry, MONGO_RECONNECT_INTERVAL)
} else {
// Reconnect retries made already has exceeded maximum retry count
console.error(`********* ERROR: MongoDB maximum connection retry attempts have been made already ${DB_OPTIONS.reconnectTries} stopping *********`)
}
})
}
connectWithRetry();