Express app won't connect to docker postgres instance in production - postgresql

I am building a webapp using express and postgres. For building my app I use the following files:
docker-compose.yml:
version: '3'
services:
postgres:
image: postgres:12
volumes:
- postgres-volume:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=${ADMIN_DB_PASSWORD}
networks:
- db
restart: unless-stopped
api:
build:
context: ./backEnd/
dockerfile: Dockerfile.debug
volumes:
- ./backEnd/index.js:/app/index.js
ports:
- "80:3002"
networks:
- db
depends_on:
- postgres
environment:
- DB_HOST=postgres
- DB_PORT=5432
- DB_DATABASE=hive
- DB_USER=${DB_API_USER}
- DB_PASSWORD=${DB_API_PASSWORD}
restart: unless-stopped
networks:
db:
The container runs the following code:
const express = require('express');
const bodyParser = require('body-parser')
const pgp = require('pg-promise')();
const connection = {
host: process.env.DB_HOST,
port: process.env.DB_PORT,
database: process.env.DB_DATABASE,
user: process.env.DB_USER,
password: process.env.DB_PASSWORD,
max: 30,
}
const db = pgp(connection);
function getUser(email){
return(
db.oneOrNone("SELECT * FROM users WHERE email = ${email}", {
email:email.toLowerCase()
})
)
}
const app = express();
app.use(bodyParser.json());
app.post('/login', async (req, res)=>{
const email = req.body.email;
console.log(email)
const user = await getUser(email);
console.log("made it past")
});
app.listen(3002, function () {
console.log('Listening on port 3002!');
});
When I call the login endpoint, it faithfully logs the email, but never gets past the
await getUser(email)
It does not throw and error or returns null, it just stays there. Interestingly it is working on my local machine and gets past the await, just not on my linux box.
I have also noticed, that if I change the db host to something nonsensical, pg-promise throws an error on my local machine. It does not however throw an error on my remote linux machine.
Also, if I run the script on the Linux box, without docker, it seamlessly connects to postgres. It appears to be something that only happens in conjunction with docker on Linux.
I am completely stumped by this error, as I have no indication of what is going wrong. The database also appears to be set up correctly, as I can connect to it and use it from my local machine with the same code.
Thank you in advance for your help.

Related

Cannot connect to MongoDB v6 in docker in ec2 after enabling SSL/TLS in .NET core app

I think I am bit confused on MongoDB v& in docker with TLS enabled. Everything seems to be working fine on the EC2 instance and also with those public key on my local MongoDB Compass tool.
But come to my .NET application, it is not connecting.
I have tried everything and I need some help with connection string.
Here is the code:
string conn = #"mongodb://username:Password#ec2instance.compute.amazonaws.com:27017/?tls=true&authMechanism=SCRAM-SHA-1";
var clientSettings = MongoClientSettings.FromUrl(new MongoUrl(conn));
clientSettings.AllowInsecureTls = true;
clientSettings.DirectConnection = true;
clientSettings.Scheme = ConnectionStringScheme.MongoDB;
clientSettings.UseTls = true;
clientSettings.ServerApi = new ServerApi(ServerApiVersion.V1);
SslSettings sslSettings = new SslSettings
{
ServerCertificateValidationCallback = (sender, certificate, chain, errors) => true,
CheckCertificateRevocation = false,
EnabledSslProtocols = SslProtocols.Tls12,
ClientCertificates = new List<X509Certificate>() {
new X509Certificate2(AppDomain.CurrentDomain.BaseDirectory + #"Files\intermediate_pfx.pfx", "Phaseuser")
},
ClientCertificateSelectionCallback = (sender, host, certificates, certificate, issuers) => clientSettings.SslSettings.ClientCertificates.ToList()[0],
};
clientSettings.SslSettings = sslSettings;
MongoClient client = new MongoClient(clientSettings);
var dbList = client.ListDatabases().ToList();
And here is my compose.yaml file:
services:
mongo:
image: mongo
command: ["--bind_ip_all", "--tlsMode", "requireTLS", "--tlsCAFile", "/usr/local/ca-certificates/root_ca.crt", "--tlsCertificateKeyFile", "/run/secrets/server-certificate"]
environment:
MONGO_INITDB_ROOT_USERNAME: username
MONGO_INITDB_ROOT_PASSWORD: Password
restart: always
volumes:
- $PWD/ca-certs:/usr/local/ca-certificates
- $PWD/db:/data/db
secrets:
- server-certificate
ports:
- '27017-27019:27017-27019'
secrets:
server-certificate:
file: mongo.pem
Everytime when I tried to connect to it, its says timeout.
Any suggestions would be really helpful.

Can't connection DB use ktor application

I try start ktor application with postgresql database. For it i used docker compose. My docker-compose.yml.
version: '3.0'
services:
ktor-sample:
build: ./
command: ./ktor-sample
ports:
- "8080:8080"
depends_on:
- db
db:
restart: unless-stopped
image: postgres:9.6.10-alpine
volumes:
- ./test:/var/lib/postgresql/data
ports:
- "5433:5433"
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD:
POSTGRES_DB: AutoHistory
I every one receive error:
WARN Exposed - Transaction attempt #2 failed: Connection to localhost:5433 refused.
Check that the hostname and port are correct and that the postmaster is accepting
TCP/IP connections.. Statement(s): null
ktor-sample_1 | org.postgresql.util.PSQLException: Connection to localhost:5433
refused. Check that the hostname and port are correct and that the postmaster is
accepting TCP/IP connections.
This database is created and can connection to it.
As for me I do next I create an object for initializing of db
it can look like this
object DbFactory {
fun init() {
val pool = hikari()
val db = Database.connect(pool)
transaction(db) {
SchemaUtils.create(Creators, Visitors, Places, Roles, UserRoles, Users)
}
}
private fun hikari(): HikariDataSource {
val hikariConfig = HikariConfig("db.properties")
return HikariDataSource(hikariConfig)
}
}
and in Application.kt, on the first place initialize your db
fun main() {
embeddedServer(Netty, port = 8080, host = "0.0.0.0") {
DbFactory.init()
...
the file which is describing your data should live in root and be called db.properties look something like
jdbcUrl=jdbc:postgresql://localhost:5433/AutoHistory
dataSource.driverClass = org.postresql.Driver
dataSource.driver=postgresql
dataSource.database= AutoHistory
dataSource.user= postgres
dataSource.password=
I have no such rows in docker-compose.yml
ktor-sample:
build: ./
command: ./ktor-sample
ports:
- "8080:8080"
depends_on:
- db
and do start my app manually but anyway I think it should help you

Flutter can not connect to the docker localhost parse-server

I am trying to run a parse-server and parse-dashboard via the following docker-compose.yml
docker-compose:
version: '3.9'
services:
database:
image: mongo:5.0
environment:
MONGO_INITDB_ROOT_USERNAME: admin
MONGO_INITDB_ROOT_PASSWORD: admin
volumes:
- data_volume:/data/mongodb
server:
restart: always
image: parseplatform/parse-server:4.10.4
ports:
- 1337:1337
environment:
- PARSE_SERVER_APPLICATION_ID=COOK_APP
- PARSE_SERVER_MASTER_KEY=MASTER_KEY_1
- PARSE_SERVER_CLIENT_KEY=CLIENT_KEY_1
- PARSE_SERVER_DATABASE_URI=mongodb://admin:admin#mongo/parse_server?authSource=admin
- PARSE_ENABLE_CLOUD_CODE=yes
links:
- database:mongo
volumes:
- data_volume:/data/server
- ./../lib/core/database/parse_server/cloud:/parse-server/cloud
dashboard:
image: parseplatform/parse-dashboard:4.0.0
ports:
- "4040:4040"
depends_on:
- server
restart: always
environment:
- PARSE_DASHBOARD_APP_ID=COOK_APP
- PARSE_DASHBOARD_MASTER_KEY=MASTER_KEY_1
- PARSE_DASHBOARD_USER_ID=admin
- PARSE_DASHBOARD_USER_PASSWORD=admin
- PARSE_DASHBOARD_ALLOW_INSECURE_HTTP=true
- PARSE_DASHBOARD_SERVER_URL=http://localhost:1337/parse
volumes:
- data_volume:/data/dashboard
volumes:
data_volume:
driver: local
After the container is running via docker-compose up I am trying to connect to it using Flutter and write a new class to my server:
Flutter code:
import 'package:flutter/material.dart';
import 'package:parse_server_sdk_flutter/parse_server_sdk.dart';
void main() async {
WidgetsFlutterBinding.ensureInitialized();
const keyApplicationId = 'COOK_APP';
const keyClientKey = 'CLIENT_KEY_1';
const keyParseServerUrl = 'http://localhost:1337/parse';
var res = await Parse().initialize(keyApplicationId, keyParseServerUrl,
clientKey: keyClientKey, autoSendSessionId: true);
var connRes = await res.healthCheck();
var s = connRes.error?.message ?? "";
print("ERROR:" + s);
var firstObject = ParseObject('FirstClass')
..set(
'message', 'Hey ! First message from Flutter. Parse is now connected');
await firstObject.save();
print('done');
}
My error message:
SocketException: Connection refused (OS Error: Connection refused, errno = 111), address = localhost, port = 35262
But for some unknown reason, I can not connect to my local server even if I can access with no problem my dashboard.
Your application running on the mobile device/emulator will treat localhost as own machine. In order to access the parse server running on your host machine (docker image exposing 0.0.0.0 global) you need to specify the host IP address.
Just replace const keyParseServerUrl = 'http://localhost:1337/parse'; with const keyParseServerUrl = 'http://YOUR_HOST_IP_ADDRESS:1337/parse'; provided in the docker parse server 0.0.0.0 should be exposed.

ConnectionTimeoutException: No suitable servers found while inserting data into mongodb database

I am new in Docker and Mongodb.
I have following in my docker-compose.yml file.
version: '3.3'
services:
web:
build:
context: ./
dockerfile: Dockerfile
container_name: php73
volumes:
- ./src:/var/www/html/
ports:
- 8000:80
depends_on:
- db
networks:
- my-network
db:
image: mongo:latest
container_name: mymongo
restart: always
ports:
- '27017-27019:27017-27019'
networks:
- my-network
networks:
my-network:
Following file is run in php container. it just creates a database and insert some collectins into the database.
<?php
require 'vendor/autoload.php';
$myClient = new MongoDB\Client('mongodb://127.0.0.1:27017');
$mydb = $myClient->my_db;
$mycollection = $mydb->my_collection;
$insertData = $mycollection->insertOne([
'doc1' => 'abc',
'doc2' => 'def'
]);
?>
But, it shows following error:
PHP Fatal error: Uncaught MongoDB\\Driver\\Exception\\ConnectionTimeoutException: No suitable servers found (`serverSelectionTryOnce` set): [connection refused calling ismaster on '127.0.0.1:27017'] in /var/www/html/vendor/mongodb/mongodb/src/functions.php:431\nStack trace:\n#0 /var/www/html/vendor/mongodb/mongodb/src/functions.php(431): MongoDB\\Driver\\Manager->selectServer(Object(MongoDB\\Driver\\ReadPreference))\n#1 /var/www/html/vendor/mongodb/mongodb/src/Collection.php(929): MongoDB\\select_server(Object(MongoDB\\Driver\\Manager), Array)\n#2 /var/www/html/mycode.php(16): MongoDB\\Collection->insertOne(Array)\n#3 {main}\n thrown in /var/www/html/vendor/mongodb/mongodb/src/functions.php on line 431, referer: http://localhost:8000/index.php
I could not figure out why it is showing ConnectionTimeoutException.
Could anyone give any hints ?
Update your connection string to
<?php
require 'vendor/autoload.php';
$myClient = new MongoDB\Client('mongodb://db:27017');
// or new MongoDB\Client('mongodb://db:27017');
$mydb = $myClient->my_db;
$mycollection = $mydb->my_collection;
$insertData = $mycollection->insertOne([
'firstname' => 'abc',
'lastname' => 'def'
]);
?>
docker-compose create network by default, you can access other container using container name, where 127.0.0.1 refer to localhost of the php container, not DB container.

Failed to connect to a mongodb docker container from mongoose on host

I'm trying to connect to mongodb running in docker from the app running on host using mongoose but it failed.
I can't use the port 27017 for the new mongodb container because it is used by other container. So I followed the guide here for setting it up using the compose.
Below are the snippets:
docker-compose.yml
version: '3'
services:
db:
image: mongo:latest
restart: always
ports:
- '8081:8081'
environment:
MONGO_INITDB_ROOT_USERNAME: root1
MONGO_INITDB_ROOT_PASSWORD: password1
But when I do docker-ps, port 27017 still there but I'm not sure if that causes an issue.
PORTS
0.0.0.0:8081->8081/tcp, 27017/tcp
Then I created a new user in admin database.
use admin
db.createUser(
{
user: "admin1",
pwd: "password2",
roles: [ { role: "userAdminAnyDatabase", db: "admin" }, "readWriteAnyDatabase" ]
}
)
server.js
const connectOption = {
useNewUrlParser: true,
user: 'admin1',
pass: 'password2',
authSource: 'admin',
}
const mongoURL = 'mongodb://localhost:8081/app1';
mongoose.connect(mongoURL, connectOption)
.then(() => console.log('MongoDB Connected'))
.catch(error => console.log(error));
And the error I received is
{
MongoNetworkError: failed to connect to server [localhost:8081] on first connect [MongoNetworkError: write EPIPE]
...
...
...
name: 'MongoNetworkError',
errorLabels: [ 'TransientTransactionError' ],
[Symbol(mongoErrorContextSymbol)]: {}
}
Assuming you're running nodejs application as a docker-compose service, in db service remove ports section (including - '8081:8081'
line). In server.js, change const mongoURL = 'mongodb://localhost:8081/app1'; to const mongoURL = 'mongodb://db:27017/app1';.
If you want to access the db from host machine, change ports 8081:8081 to <give-a-port-number>:27017.