Im on a mac(OS 10.14) using nodejs 14 and PostgresSQL 12.
I just installed Loopback4 and after following this tutorial Im not able to use any of the enpoints that use Models, ie that connect to Postgres, I constantly get a timeout.
It seems like its not even reaching the Postgres Server, but the error gives no information, just that the request times out.
There are no issues with the Postgres server since I can connect and request information with other nodejs applications to the same database.
I also tried to set this as the host host: '/var/run/postgresql/', same result.
I now tried the approach with a Docker container, setting the datasource files as follows:
import {inject, lifeCycleObserver, LifeCycleObserver} from '#loopback/core';
import {juggler} from '#loopback/repository';
const config = {
name: 'mydb',
connector: 'postgresql',
url: 'postgres://postgres:mysecretpassword#localhost:5434/test',
ssl: false,
};
// Observe application's life cycle to disconnect the datasource when
// application is stopped. This allows the application to be shut down
// gracefully. The `stop()` method is inherited from `juggler.DataSource`.
// Learn more at https://loopback.io/doc/en/lb4/Life-cycle.html
#lifeCycleObserver('datasource')
export class PostgresSqlDataSource extends juggler.DataSource
implements LifeCycleObserver {
static dataSourceName = 'PostgresSQL';
static readonly defaultConfig = config;
constructor(
#inject('datasources.config.PostgresSQL', {optional: true})
dsConfig: object = config,
) {
super(dsConfig);
}
}
With that same url I can log on my command line from my mac.
Is there a way to add logging and print any connection error? Other ways to debug it?
[UPDATE]
As of today Loopback4 Postgres connector does not work properly with Nodejs 14.
When starting the application, instead of running
npm start, you can set the debug string by running:
DEBUG=loopback:connector:postgresql npm start
If you want it to be more generic, you can use:
DEBUG=loopback:* npm start
Related
I am using NextJS to build an app. I am using MongoDB via mongoosejs to connect to my database hosted in mongoAtlas.
My database connection file looks like below
import mongoose from "mongoose";
const MONGO_URI =
process.env.NODE_ENV === "development"
? process.env.MONGO_URI_DEVELOPMENT
: process.env.MONGO_URI_PRODUCTION;
console.log(`Connecting to ${MONGO_URI}`);
const database_connection = async () => {
if (global.connection?.isConnected) {
console.log("reusing database connection")
return;
}
const database = await mongoose.connect(MONGO_URI, {
authSource: "admin",
useNewUrlParser: true
});
global.connection = { isConnected: database.connections[0].readyState }
console.log("new database connection created")
};
export default database_connection;
I have seen this MongoDB developer community thread and this GitHub thread.
The problem seems to happen only in dev mode(when you run yarn run dev). In the production version hosted on Vercel there seems to be no issue. I understand that in dev mode the server is restarted every time a change is saved so to cache a connection you need to use as global variable. As you can see above, I have done exactly that. The server even logs: reusing database connection, then in mongoAtlas it shows like 10 more connections opened.
How can I solve this issue or what am I doing wrong?
From the typeorm docs:
Generally, you call initialize method of the DataSource instance on application bootstrap, and destroy it after you completely finished working with the database. In practice, if you are building a backend for your site and your backend server always stays running - you never destroy a DataSource.
But, for implementing postgres's schema based multitenancy, I'm scoping connections per request, because, each request has to be sent to a different schema. So, in my getConnection option, I'm doing this:
async function getTenantConnection(
tenantName: string,
connectionOptions?: PostgresConnectionOptions,
) {
if (!connectionOptions) {
connectionOptions = baseConnection;
}
const options: PostgresConnectionOptions = {
...connectionOptions,
schema: tenantName,
entities: [__dirname + '/../**/*.entity.js'],
synchronize: false,
};
const dataSource = new DataSource(options);
return await dataSource.initialize();
}
so on each request, I'm doing getTenantConnection which sort of initialized the database. The previously available getConnection() seems deprecated, and now doing stress tests on the app, I'm getting TCP connection issues, which I simply cannot debug:
5;3m[ExceptionsHandler] connect ETIMEDOUT 20.119.245.111:5432
2023-01-31T10:40:40.581303183Z Error: connect ETIMEDOUT 20.119.245.111:5432
2023-01-31T10:40:40.581370686Z at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1278:16)
2023-01-31T10:40:40.581381886Z at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17)
2023-01-31T10:40:40.589395960Z [Nest] 53 - 01/31/2023, 10:40:40 AM ERROR [ExceptionsHandler] connect ETIMEDOUT 20.119.245.111:5432
I'm just speculating that the database pool has something to do with this. I don't understand the code fully, but the source code for typeorm doesn't seem to contain any pooling done in the initialize() method section as well. I had tried to take reference from this article which demonstrates schema based multitenancy in postgres using typeorm , but methods available there aren't available anymore so I had to resolve to using .initialize(). Please let me know how I can go about implementing this.
I'm trying to connect a Cloud Run service to Cloud SQL postgres instance. I believe I'm nearly there, but am having some trouble getting the deployed instance to connect properly. My local environment can connect (via SSL) to the database intended for production, but the deployed version can't...
I'm using TypeORM, and have everything setup properly in the configuration...
#Module({
imports: [
TypeOrmModule.forRootAsync({
imports: [ConfigModule],
inject: [ConfigService],
useFactory: (configService: ConfigService) => {
const socketPath = configService.get('DB_SOCKET_PATH');
const extra = socketPath ? {
socketPath: socketPath,
ssl: {
rejectUnauthorized: false,
ca: Buffer.from(process.env.DB_SSL_CA, 'base64').toString('ascii'),
cert: Buffer.from(process.env.DB_SSL_CERT, 'base64').toString('ascii'),
key: Buffer.from(process.env.DB_SSL_KEY, 'base64').toString('ascii'),
}
} : { };
return ({
type: 'postgres',
host: socketPath || configService.get('DB_HOST'),
port: configService.get('DB_PORT'),
username: configService.get('DB_USER'),
password: configService.get('DB_PASS'),
database: configService.get('DB_NAME'),
extra: extra,
entities: [__dirname + '/../../modules/**/*.entity{.ts,.js}'],
namingStrategy: new SnakeNamingStrategy(),
synchronize: true,
});
}
})
]
})
export class DatabaseModule { }
Despite that I'm getting an error when I try to use the socketPath as the host rather than the actual host variable (necessary for GCP). It seems that TypeORM is adding extra characters, /.s.PGSQL.5432, at the end of my connection string that I don't want. And just to clarify, the socket path is in the form of /cloudsql/<PROJECT_ID>:<REGION>:<INSTANCE>.
[Nest] 28532 - 02/15/2021, 2:25:07 PM [ExceptionHandler] connect ENOENT <DB_SOCKET_PATH>/.s.PGSQL.5432 +3ms
Error: connect ENOENT <DB_SOCKET_PATH>/.s.PGSQL.5432
at PipeConnectWrap.afterConnect [as oncomplete] (net.js:1141:16)
At an older point in time, this used to work for me but I guess something changed in the TypeORM library. Does anybody have any ideas on this? Thanks!
EDIT: As of now I've gotten it to connect to the server correctly, but it's now giving me an error that says the server doesn't support SSL connections, which makes no sense given that I can connect via SSL fine on my local machine...?
SOLUTION: The issue does not seem to any code's fault, but rather some networking stuff on the GCP side. I configured the service and database to run through a VPC then just used a private IP address for the host.
It seems that TypeORM is adding extra characters, /.s.PGSQL.5432
This is actually intended - the Postgres spec requires that the unix sockets end with this suffix.
[Nest] 28532 - 02/15/2021, 2:25:07 PM [ExceptionHandler] connect ENOENT <DB_SOCKET_PATH>/.s.PGSQL.5432 +3ms
The error means that the socket wasn't found - usually because there was a misconfiguration and the Cloud SQL proxy couldn't start. You can check your logs at the instance start up to see if the proxy left any errors, but generally it'll come down to the following:
The Cloud SQL Admin API needs to be enabled
Your service account needs to have Cloud SQL Connect IAM role (or equivalent)
The service needs to be configured for Cloud SQL.
For a full list of instructions, see the Connecting from Cloud Run to Cloud SQL page.
Is there anything in the modern PostgreSQL connection protocol that would indicate the server version?
And if not, is there a special low-level request that an endpoint can execute against an open connection to pull the server details that would contain the version?
I'm looking at a possible extension of node-postgres that would automatically provide the server version upon every fresh connection. And I want to know if this is at all possible.
Having to execute SELECT version() upon every new connection and then parsing it is too high-level for the base driver that manages the connection. It should be done on the protocol level.
After a bit of research, I found that PostgreSQL does provide server version during connection, within the start-up message.
And specifically within node-postgres driver, we can make Pool provide a custom Client that handles event parameterStatus on the connection, and exposes the server version:
const {Client, Pool} = require('pg');
class MyClient extends Client {
constructor(config) {
super(config);
this.connection.on('parameterStatus', msg => {
if (msg.parameterName === 'server_version') {
this.version = msg.parameterValue;
}
});
}
}
const cn = {
database: 'my-db',
user: 'postgres',
password: 'bla-bla',
Client: MyClient // here's our custom Client type
};
const pool = new Pool(cn);
pool.connect()
.then(client => {
console.log('Server Version:', client.version);
client.release(true);
})
.catch(console.error);
On my test PC, I use PostgreSQL v11.2, so this test outputs:
Server Version: 11.2
UPDATE - 1
Library pg-promise has been updated to support the same functionality in TypeScript. And you can find a complete example in this ticket.
UPDATE - 2
See example here:
// tests connection and returns Postgres server version,
// if successful; or else rejects with connection error:
async function testConnection() {
const c = await db.connect(); // try to connect
c.done(); // success, release connection
return c.client.serverVersion; // return server version
}
I've come across a node chat example on github, When I try to run it, I see the following error:
Error connecting to mongo perhaps it isn't running ?
I've installed mongo 0.9.2, nodejs 5.2 pre, npm 3.0 and other dependencies. The example can be found here: https://github.com/gregstewart/chat.io
I can not determine whether if the example not really works or I didn't run it right. Please help.
Did you install and start mongo-db on your system? This error is mostly because of a missing mongo instance running on the local machine.
Check out the follwing code excerpts from chat.io.
main.js:
/**
* Configure the user provider (mongodB connection for user data storage)
*/
var userProvider = new UserProvider('localhost', 27017);
Creates a new UserProvider object using host and port for database (localhost:27017, mongo-db default).
UserProvider.js:
UserProvider = function(host, port) {
this.db = new mongo.Db('node-mongo-chat', new Server(host, port, {auto_reconnect: true}, {}));
this.db.addListener('error', function(error) {
console.log('Error connecting to mongo -- perhaps it isn\'t running?');
});
this.db.open(function() {
});
};
Opening the connection to the server, printing out an error on failure (the error you mentioned above).
Consider reading up on the mongo-db docs concerning installation and setup here