Mongo database disconnects when using docker - mongodb

I am using docker compose with my app and are trying to connect mongodb to the server. When i run my app locally outside of docker i get this as output(works as intended)
[nodemon] 2.0.15
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node index.js`
Server running
Mongoose connected to db...
Mongodb connected....
When i run the docker-compose up command and the server runs in the container i get this output
[nodemon] 2.0.15
docker-server | [nodemon] to restart at any time, enter `rs`
docker-server | [nodemon] watching path(s): *.*
docker-server | [nodemon] watching extensions: js,mjs,json
docker-server | [nodemon] starting `node index.js`
docker-server | Works
docker-server | Works
docker-server | Mongoose connection is disconnected...
After a while the mongoose disconnects.
My package.json is
{
"name": "make-me-a-sandwich",
"version": "1.1.0",
"description": "This is the Swagger 2.0 API for Web Architectures course group project work. ",
"main": "index.js",
"scripts": {
"prestart": "npm install",
"start": "nodemon index.js"
},
"keywords": [
"swagger"
],
"license": "Unlicense",
"private": true,
"dependencies": {
"connect": "^3.2.0",
"js-yaml": "^3.3.0",
"swagger-tools": "0.10.1",
"mongoose": "^6.1.5",
"nodemon": "^2.0.15"
}
}
My index.js file is
const http = require('http');
const connect = require('./models/db');
const PORT = 80;
const server = http.createServer(function (request, response) {
const { url, method, headers } = request;
const filePath = new URL(url, `http://${headers.host}`).pathname;
if (filePath === '/' && method.toUpperCase() === 'GET') {
console.log("Works")
response.statusCode = 200;
response.setHeader('Content-Type', 'text/plain');
response.end('Hello, World! GET\n');
} else {
response.statusCode = 200;
response.setHeader('Content-Type', 'text/plain');
response.end('Hello, World! Teemu\n');
}
});
server.on('error', err => {
console.error(err);
server.close();
});
server.on('close', () => console.log('Server closed.'));
server.listen(PORT, () => {
console.log("Server running");
});
connect.connectDB();
model/db.js
const mongoose = require('mongoose');
function connectDB() {
mongoose
.connect('mongodb://mongo_db:27017', {
useNewUrlParser: true,
})
.then(() => {
console.log('Mongodb connected....');
})
.catch(err => console.log(err.message));
mongoose.connection.on('connected', () => {
console.log('Mongoose connected to db...');
});
mongoose.connection.on('error', err => {
console.log(err.message);
});
mongoose.connection.on('disconnected', () => {
console.log('Mongoose connection is disconnected...');
});
};
function disconnectDB() {
mongoose.disconnect();
}
module.exports = { connectDB, disconnectDB };
Dockerfile
FROM node:17.3.0
WORKDIR /server
COPY package.json .
RUN npm install
COPY . .
EXPOSE 80
CMD ["npm", "start"]
docker-compose file
version: "3"
services:
server-a:
container_name: docker-server
build:
dockerfile: Dockerfile
context: ./backend/server-a
ports:
- "3000:80"
links:
- mongo_db
networks:
- backend
mongo_db:
container_name: mongo
image: mongo:latest
ports:
- '27017:27017'
networks:
backend:
Help would be really appreciated. Let me also know if i can offer any other information.

Different containers need to be on the same Compose network to communicate. If a service doesn't have a networks: block, Compose automatically attaches it to a default network. So in your example, the server-a container is only on the backend network, but the mongo_db container is only on the default network, and that's why they can't communicate.
The easiest way to resolve this is to delete all of the networks: blocks in the file. Then Compose will attach all of the containers to the default network. Removing other unnecessary options, you could reduce this Compose file to just
version: "3.8"
services:
server-a:
build: ./backend/server-a
ports:
- "3000:80"
mongo_db:
image: mongo:latest
ports:
- '27017:27017'
In a comment you suggest that it's important to keep a second named network. If that's the case, then you need to make sure the database container also has a networks: block that names a network in common with the application container.

Related

Cannot create a mongo database with docker

I'm having trouble creating a mongo database using the docker-compose command. Docker desktop tells me that everything is up and running including the db, but all I get is the standard 'admin, config, local' not the db I want to create. Here's my docker-compose.yaml
version: '3'
services:
app:
build: ./
entrypoint: ./.docker/entrypoint.sh
ports:
- 3000:3000
volumes:
- .:/home/node/app
depends_on:
- db
db:
image: mongo:4.4.4
restart: always
volumes:
- ./.docker/dbdata:/data/db
- ./.docker/mongo:/docker-entrypoint-initdb.d
environment:
- MONGO_INITDB_ROOT_USERNAME=root
- MONGO_INITDB_ROOT_PASSWORD=root
- MONGO_INITDB_DATABASE=nest
mongo-express:
image: mongo-express
restart: always
ports:
- 8081:8081
environment:
- ME_CONFIG_MONGODB_SERVER=db
- ME_CONFIG_MONGODB_AUTH_USERNAME=root
- ME_CONFIG_MONGODB_AUTH_PASSWORD=root
- ME_CONFIG_MONGODB_ADMINUSERNAME=root
- ME_CONFIG_MONGODB_ADMINPASSWORD=root
depends_on:
- db
my init.js inside .docker/mongo
db.routes.insertMany([
{
_id: "1",
title: "Primeiro",
startPosition: {lat: -15.82594, lng: -47.92923},
endPosition: {lat: -15.82942, lng: -47.92765},
},
{
_id: "2",
title: "Segundo",
startPosition: {lat: -15.82449, lng: -47.92756},
endPosition: {lat: -15.82776, lng: -47.92621},
},
{
_id: "3",
title: "Terceiro",
startPosition: {lat: -15.82331, lng: -47.92588},
endPosition: {lat: -15.82758, lng: -47.92532},
}
]);
and my dockerfile
FROM node:14.18.1-alpine
RUN apk add --no-cache bash
RUN npm install -g #nestjs/cli
USER node
WORKDIR /home/node/app
and this is the 'error' log I get from docker when I run the nest container with mongodb, nest app and mongo express(there is actually a lot more but SO keeps thinking that it is spam for some reason.
about to fork child process, waiting until server is ready for connections.
Successfully added user: {
"user" : "root",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
Error saving history file: FileOpenFailed Unable to open() file /home/mongodb/.dbshell: No such file or directory
{"t":{"$date":"2022-06-01T19:39:15.542+00:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn2","msg":"Connection ended","attr":{"remote":"127.0.0.1:39304","connectionId":2,"connectionCount":0}}
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.js
{"t":{"$date":"2022-06-01T19:39:15.683+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"127.0.0.1:39310","connectionId":3,"connectionCount":1}}
{"t":{"$date":"2022-06-01T19:39:15.684+00:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn3","msg":"client metadata","attr":{"remote":"127.0.0.1:39310","client":"conn3","doc":{"application":{"name":"MongoDB Shell"},"driver":{"name":"MongoDB Internal Client","version":"4.4.4"},"os":{"type":"Linux","name":"Ubuntu","architecture":"x86_64","version":"18.04"}}}}
{"t":{"$date":"2022-06-01T19:39:15.701+00:00"},"s":"I", "c":"STORAGE", "id":20320, "ctx":"conn3","msg":"createCollection","attr":{"namespace":"nest.routes","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"f689868e-af6d-4ec6-b555-dcf520f24788"}},"options":{}}}
{"t":{"$date":"2022-06-01T19:39:15.761+00:00"},"s":"I", "c":"INDEX", "id":20345, "ctx":"conn3","msg":"Index build: done building","attr":{"buildUUID":null,"namespace":"nest.routes","index":"_id_","commitTimestamp":{"$timestamp":{"t":0,"i":0}}}}
uncaught exception: ReferenceError: colection is not defined :
#/docker-entrypoint-initdb.d/init.js:23:1
failed to load: /docker-entrypoint-initdb.d/init.js
exiting with code -3
this is what running docker-compose ps shows
NAME COMMAND SERVICE STATUS PORTS
nest-api-app-1 "./.docker/entrypoin…" app running 0.0.0.0:3000->3000/tcp
nest-api-db-1 "docker-entrypoint.s…" db running 27017/tcp
nest-api-mongo-express-1 "tini -- /docker-ent…" mongo-express running 0.0.0.0:8081->8081/tcp
this what my docker desktop shows
The MongoDB container only creates a database if no database already exists. You probably already have one, which is why a new database isn't created and your initialization script isn't run.
Delete the contents of ./.docker/dbdata on the host. Then start the containers with docker-compose and Mongo should create your database for you.

Express app won't connect to docker postgres instance in production

I am building a webapp using express and postgres. For building my app I use the following files:
docker-compose.yml:
version: '3'
services:
postgres:
image: postgres:12
volumes:
- postgres-volume:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=${ADMIN_DB_PASSWORD}
networks:
- db
restart: unless-stopped
api:
build:
context: ./backEnd/
dockerfile: Dockerfile.debug
volumes:
- ./backEnd/index.js:/app/index.js
ports:
- "80:3002"
networks:
- db
depends_on:
- postgres
environment:
- DB_HOST=postgres
- DB_PORT=5432
- DB_DATABASE=hive
- DB_USER=${DB_API_USER}
- DB_PASSWORD=${DB_API_PASSWORD}
restart: unless-stopped
networks:
db:
The container runs the following code:
const express = require('express');
const bodyParser = require('body-parser')
const pgp = require('pg-promise')();
const connection = {
host: process.env.DB_HOST,
port: process.env.DB_PORT,
database: process.env.DB_DATABASE,
user: process.env.DB_USER,
password: process.env.DB_PASSWORD,
max: 30,
}
const db = pgp(connection);
function getUser(email){
return(
db.oneOrNone("SELECT * FROM users WHERE email = ${email}", {
email:email.toLowerCase()
})
)
}
const app = express();
app.use(bodyParser.json());
app.post('/login', async (req, res)=>{
const email = req.body.email;
console.log(email)
const user = await getUser(email);
console.log("made it past")
});
app.listen(3002, function () {
console.log('Listening on port 3002!');
});
When I call the login endpoint, it faithfully logs the email, but never gets past the
await getUser(email)
It does not throw and error or returns null, it just stays there. Interestingly it is working on my local machine and gets past the await, just not on my linux box.
I have also noticed, that if I change the db host to something nonsensical, pg-promise throws an error on my local machine. It does not however throw an error on my remote linux machine.
Also, if I run the script on the Linux box, without docker, it seamlessly connects to postgres. It appears to be something that only happens in conjunction with docker on Linux.
I am completely stumped by this error, as I have no indication of what is going wrong. The database also appears to be set up correctly, as I can connect to it and use it from my local machine with the same code.
Thank you in advance for your help.

Failed to connect to a mongodb docker container from mongoose on host

I'm trying to connect to mongodb running in docker from the app running on host using mongoose but it failed.
I can't use the port 27017 for the new mongodb container because it is used by other container. So I followed the guide here for setting it up using the compose.
Below are the snippets:
docker-compose.yml
version: '3'
services:
db:
image: mongo:latest
restart: always
ports:
- '8081:8081'
environment:
MONGO_INITDB_ROOT_USERNAME: root1
MONGO_INITDB_ROOT_PASSWORD: password1
But when I do docker-ps, port 27017 still there but I'm not sure if that causes an issue.
PORTS
0.0.0.0:8081->8081/tcp, 27017/tcp
Then I created a new user in admin database.
use admin
db.createUser(
{
user: "admin1",
pwd: "password2",
roles: [ { role: "userAdminAnyDatabase", db: "admin" }, "readWriteAnyDatabase" ]
}
)
server.js
const connectOption = {
useNewUrlParser: true,
user: 'admin1',
pass: 'password2',
authSource: 'admin',
}
const mongoURL = 'mongodb://localhost:8081/app1';
mongoose.connect(mongoURL, connectOption)
.then(() => console.log('MongoDB Connected'))
.catch(error => console.log(error));
And the error I received is
{
MongoNetworkError: failed to connect to server [localhost:8081] on first connect [MongoNetworkError: write EPIPE]
...
...
...
name: 'MongoNetworkError',
errorLabels: [ 'TransientTransactionError' ],
[Symbol(mongoErrorContextSymbol)]: {}
}
Assuming you're running nodejs application as a docker-compose service, in db service remove ports section (including - '8081:8081'
line). In server.js, change const mongoURL = 'mongodb://localhost:8081/app1'; to const mongoURL = 'mongodb://db:27017/app1';.
If you want to access the db from host machine, change ports 8081:8081 to <give-a-port-number>:27017.

Using a separate testing mongo database on mongo and Heroku

In my project I am using https://www.npmjs.com/package/dotenv-safe in order to declare environment variables needed for configuration. For example:
NODE_ENV=development
JWT_SECRET=xxxxxxx
JWT_EXPIRATION_MINUTES=15
MONGO_URI=mongodb://mongodb:27017/proddb
BASE_URI=http://localhost:3000/
MONGO_URI_TESTS=mongodb://mongodb:27017/testdb
PORT=3000
Then I use those files in a config file :
module.exports = {
env: process.env.NODE_ENV,
port: process.env.PORT,
jwtSecret: process.env.JWT_SECRET,
jwtExpirationInterval: process.env.JWT_EXPIRATION_MINUTES,
mongo: {
uri: process.env.NODE_ENV === 'test'
? process.env.MONGO_URI_TESTS
: process.env.MONGO_URI,
},
logs: process.env.NODE_ENV === 'production' ? 'combined' : 'dev',
};
and in my package.json file, I've got:
"scripts": {
"start": "NODE_ENV=production node ./src/index.js",
"dev": "LOG_LEVEL=debug nodemon --inspect=0.0.0.0 ./src/index.js",
"test": "NODE_ENV=test nyc --reporter=html --reporter=text mocha --timeout 20000 --recursive src/tests"
}
The problem? Everything works fine but when tests are run on Heroku (prod) , they run on the main database and not on the testdb...

MongoError: failed to connect to server [mongo:27017] on first connect

Docker config:
FROM node:8
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8080
CMD [ "npm", "start" ]
Docker compose:
version: "2"
services:
web:
build: .
depends_on:
- mongo
mongo:
image: mongo
ports:
- "27017:27017"
package.json:
{
"name": "docker_web_app",
"version": "1.0.0",
"description": "",
"author": "Stepan Yakovenko <stiv.yakovenko#gmail.com>",
"main": "server.js",
"scripts": {
"start": "node server.js"
},
"dependencies": {"express": "^4.16.1","mongodb": "~3.0.1","monk": "~6.0.5" }
}
server.js:
var mongo = require('mongodb');
var monk = require('monk');
var db = monk('mongo:27017/nodetest1');
db.then(function(){console.log("hello"});
Most of the time it works, but if I purge docker cache, usually it doesn't work and gives me this:
web_1 | (node:15) UnhandledPromiseRejectionWarning: MongoError: failed to connect to server [mongo:27017] on first connect [MongoErr
or: connect ECONNREFUSED 172.18.0.2:27017]
The root cause I think that docker's depends_on doesn't guarantee me that mongo has started listening, because in this case I get listening message from mongo after this error. How can I fix this? Does docker has any fix for this situation? Or how can I ask mongo to try connecting forever?
Thanx
This is sample reconnect code, which worked for me:
var connect = function () {
var db = monk('mongo:27017/nodetest1');
db.then(function () {
console.log("connected");
}).catch(function () {
// sometimes node starts before mongo, so we have to reconnect in case of error
connect();
});
};
connect();