node-mysql pool experiences ETIMEDOUT - google-cloud-sql

I have a node-mysql pool configuration of
var db_init={
host : 'ip_address_of_GCS_SQL',
user : 'user_name_of_GCS_SQL',
password : 'password here',
database : 'db here',
supportBigNumbers: true,
connectionLimit:100
};
Pool was created using
GLOBAL.db_foobar = mysql.createPool(db_init);
I basically just left the connection on for a couple of hours and I saw this error reported by my connection.query Request (after getConnection of course):
prodAPI-104 (out): { status: 'Error',
prodAPI-104 (out): details: '[foobar_function]Error in query',
prodAPI-104 (out): err: '{ [Error: read ETIMEDOUT]\n code: \'ETIMEDOUT\',\n errno: \'ETIMEDOUT\',\n syscall: \'read\',\n fatal: true }',
prodAPI-104 (out): query: 'SELECT * FROM `foobar_table`;' }
Why is this happening? The MySQL in Google-Cloud-SQL didn't report a query taking too long to create so I dunno why this happened.

I suspect the reason is that keepalive is not enabled on the connection to the MySQL server.
node-mysql does not have an option to enable keepalive and neither does node-mysql2, but node-mysql2 provides a way to supply a custom function for creating sockets which we can use to enable keepalive:
var mysql = require('mysql2');
var net = require('net');
var pool = mysql.createPool({
connectionLimit : 100,
host : '123.123.123.123',
user : 'foo',
password : 'bar',
database : 'baz',
stream : function(opts) {
var socket = net.connect(opts.config.port, opts.config.host);
socket.setKeepAlive(true);
return socket;
}
});

Related

MongoServerError: bad auth : Authentication failed. despite all solutions

Hi all so I went through the MERN Stack tutorial: https://www.mongodb.com/languages/mern-stack-tutorial.
Am not able to connect successfully to MongoDB.
Server is running on port: 8080
MongoServerError: bad auth : Authentication failed.
at Connection.onMessage (/Users/username/Documents/Projects/projectname/server/node_modules/mongodb/lib/cmap/connection.js:227:30)
at MessageStream.<anonymous> (/Users/username/Documents/Projects/projectname/server/node_modules/mongodb/lib/cmap/connection.js:60:60)
at MessageStream.emit (node:events:513:28)
at processIncomingData (/Users/username/Documents/Projects/projectname/server/node_modules/mongodb/lib/cmap/message_stream.js:125:16)
at MessageStream._write (/Users/username/Documents/Projects/projectname/server/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
at writeOrBuffer (node:internal/streams/writable:392:12)
at _write (node:internal/streams/writable:333:10)
at Writable.write (node:internal/streams/writable:337:10)
at TLSSocket.ondata (node:internal/streams/readable:766:22)
at TLSSocket.emit (node:events:513:28) {
ok: 0,
code: 8000,
codeName: 'AtlasError',
connectionGeneration: 0,
[Symbol(errorLabels)]: Set(2) { 'HandshakeError', 'ResetPool' }
The 'config.env' file contains this:
ATLAS_URI=mongodb+srv://correctusername:correctpassword#atlascluster.zgubjgl.mongodb.net/employees?retryWrites=true&w=majority
PORT=8080
The 'conn.js' contains:
const { MongoClient } = require("mongodb");
const Db = process.env.ATLAS_URI;
const client = new MongoClient(Db, {
useNewUrlParser: true,
useUnifiedTopology: true,
});
var _db;
module.exports = {
connectToServer: function (callback) {
client.connect(function (err, db) {
// Verify we got a good "db" object
if (db)
{
_db = db.db("employees");
console.log("Successfully connected to MongoDB.");
}
return callback(err);
});
},
getDb: function () {
return _db;
},
};
Solutions attempted:
Ensured that the username and password do not have '<>' enclosing them.
Created a new user in 'Database Access' in MongoDB itself with 'readWriteAnyDatabase#admin' access to mirror the ending of the ATLAS_URI connection string, same error.
Removed 'employees' in connection string and tried again, same error.
Tried another database name in both 'config.env' and 'conn.js' files, same error.
Ensured that my 'Network Access' in MongoDB itself is '0.0.0.0/0 (includes your current IP address)'.
Ensured that cluster name matches.
All of the solutions above are things I tried based off what I have researched on Stack. Is the problem that I don't have an employees database set up on MongoDB itself? Or is it a security setting on my Mac that is causing an issue?
Please help - feeling very downtrodden that I have reached this obstacle as I have a really good platform idea to start a business based on.

Unknown auth message code 1397113172 when connect to Heroku postgres

Thanks reading my issue.
Currently, I am using postgres (hobby-dev) on Heroku and facing this issue every time that I connect to the database.
error: Uncaught (in promise) Error: Unknown auth message code 1397113172
throw new Error(`Unknown auth message code ${code}`);
^
at Connection.handleAuth (connection.ts:197:15)
at Connection.startup (connection.ts:155:16)
at async Pool._createConnection (pool.ts:32:5)
at async pool.ts:61:7
at async Promise.all (index 0)
at async Pool._startup (pool.ts:63:25)
My application using Deno now
import { Pool } from "https://deno.land/x/postgres/mod.ts";
import { config } from "./config.ts";
const port = config.DB_PORT ? parseInt(config.DB_PORT || "") : undefined;
const POOL_CONNECTIONS = 20;
const dbPool = new Pool({
port,
hostname: config.DB_HOST,
user: config.DB_USER,
database: config.DB_NAME,
password: config.DB_PASS
}, POOL_CONNECTIONS);
export { dbPool };
Here is debug screen.
I have found this issue post and it mentioned about lacking ssl. Not sure how to do it on heroku.
I have tried some solutions, even change lib to pg and it still not work. I am very appreciated if any clue or help to fix this issue.
Note:
I read a document on heroku about "Heroku Postgres Connection Pooling is not available for Hobby-tier databases.". Then I switched to use Client with syntax similar like this to connect to Heroku postgres this:
import { Client } from "https://deno.land/x/postgres/mod.ts";
let config;
config = {
hostname: "localhost",
port: 5432,
user: "user",
database: "test",
applicationName: "my_custom_app"
};
// alternatively
config = "postgres://user#localhost:5432/test?application_name=my_custom_app";
const client = new Client(config);
await client.connect();
await client.end();
ref: https://deno-postgres.com/#/

Knex is not reading connection string from knexfile

I have been given a knexfile like this:
require('dotenv').config()
module.exports = {
client: 'pg',
connection: process.env.DB_CONNECTION,
pool: {
min: 2,
max: 10
},
migrations: {
tableName: 'knex_migrations'
}
};
The connection string I supply is:
Host=localhost;Database=heypay;Username=postgres;Password=1234
However, Knex keeps issuing the error:
password authentication failed for user "user"
Apparently, the username I have given is not user. Moreover, I have tried to hardcore the connection string into the connection filed under module.exports. This still ended up in vain.
The trick is, the connection property can either be a string or an object. That's why you were able to supply an environment variable (it's a string).
The reason your original string was failing is not a Knex problem: Postgres connection strings have a slightly different format. You can use a similar approach as your first attempt, but pay attention to the key names:
host=localhost port=5432 dbname=mydb connect_timeout=10
Also note spaces, not semicolons. However in my experience most people use a Postgres URI:
postgresql://[user[:password]#][netloc][:port][,...][/dbname][?param1=value1&...]
So in your example, you'd use:
module.exports = {
client: 'pg',
connection: 'postgresql://your_database_user:password#localhost/myapp_test',
pool: {
min: 2,
max: 10
},
migrations: {
tableName: 'knex_migrations'
}
};
I was using a .NET style connection string, the correct one would be in the following format:
{
host : '127.0.0.1',
user : 'your_database_user',
password : 'your_database_password',
database : 'myapp_test'
}

Mocha tests can't connect to postgres database, using knex

I am trying to implement some integration tests using mocha for functions that interact with a postgres database through knex, in a nodejs express app.
The functions work outside of mocha - I can start the app in node or nodemon, submit requests through Postman, retrieve records from the database, add new records, etc. But when I try to test the code through mocha, I get errors like the following for any functions that try to access the database:
select * from "item" where "user_id" = $1 - relation "item" does not exist
The environment variable for the database connection is set to connect to the right database; when I manually test the app end-to-end, everything works; I get data back from the database.
I've included what I think are the relevant code snippets below: the test script for one of the tests that won't work, the function I'm trying to test, and the modules that that function relies on.
TEST SCRIPT
const Item = require('../db/item');
const chai = require('chai');
const chaiAsPromised = require('chai-as-promised');
// set up the middleware
chai.use(chaiAsPromised);
var should = require('chai').should()
describe('Item.getByUser', function() {
contex`enter code here`t('With valid id', function() {
const item_id = 1;
const expectedResult = "Canoe";
it('should return items', function() {
return Item
.getByUser(item_id)
.then(items => {
items[0].name.should.equal(expectedResult);
});
});
});
SNIPPET FROM THE ITEM.GETBYUSER FUNCTION:
const knex = require('./connection');
module.exports = {
getByUser: function(id) {
return knex('item').where('user_id', id);
},
SNIPPET FROM THE CONNECTION MODULE:
require('dotenv-safe').config();
const environment = process.env.NODE_ENV || 'development';
const config = require('../knexfile')[environment];
module.exports = require('knex')(config);
SNIPPET FROM THE KNEXFILE MODULE:
module.exports = {
development: {
client: 'pg',
connection: process.env.DATABASE_URL
},
production: {
client: 'pg',
connection: process.env.DATABASE_URL
}
};
The error message I get for the above test is:
1) Item.getByUser
With valid id
should return items:
select * from "item" where "user_id" = $1 - relation "item" does not exist
error: relation "item" does not exist
at Connection.parseE (node_modules\pg\lib\connection.js:567:11)
at Connection.parseMessage (node_modules\pg\lib\connection.js:391:17)
at Socket.<anonymous> (node_modules\pg\lib\connection.js:129:22)
at addChunk (_stream_readable.js:284:12)
at readableAddChunk (_stream_readable.js:265:11)
at Socket.Readable.push (_stream_readable.js:220:10)
at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)
Okay, it's looking like this was in fact related to the environment variable for the database in some way I can't quite figure out. Although I had the database connection set to 'postgres://localhost/mydatabase' the database actually being used when I tested the db live, including migrating and seeding the database through knex commands, was 'postgres://localhost/username' - a database with the same name as the Owner of 'mydatabase'. But I think the mocha tests were trying to connect to mydatabase, which was still empty at that point, since the migrate and seed affected the the database 'username.'
So, I think this can be closed. I'll try to replicate the problem where I was connected to the wrong db; not sure how that could have happened, as I never intentionally set the connection, or any environment variable to 'postgres://localhost/username'.

issues connecting to postgres using pg client

Trying to connect to postgres using the pg client (following these instructions).
Here is my connection string var connectionString = "postgres://postgres:pass#localhost/ip:5432/chat";
Here's the error I'm getting trying to connect:
UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): error: database "ip:5432/twitchchat" does not exist
However, when I run pg_isready I get the response /tmp:5432 - accepting connections Which I read as telling me postgres is running on port 5432.
The database is very definitely existing.
Here's the simple connection code:
var pg = require('pg');
var connectionString =
"postgres://postgres:pass#localhost/ip:5432/chat";
var connection = new pg.Client(connectionString);
connection.connect();
What's happening here? How do I fix this?
Your connection string is malformed:
Change it to:
`"postgres://postgres:pass#localhost:5432/chat"`
The correct format is:
postgresql://[user]:[password]#[address]:[port]/[dbname]
Ok I fixed it with the following:
const { Client } = require('pg');
const connection = new Client({
user: 'postgres',
host: 'localhost',
database: 'chat',
password: 'pass',
port: 5432,
});
connection.connect();