How to handle idle_in_transaction_session_timeout? - pg-promise

When we set idle_in_transaction_session_timeout, the database will terminate connections that are idle for some time.
This works as expected, but I wonder how we should deal with this situation in the aplication code.
We are using pg-promise 10.3.1.
Details of the test:
we set the connection pool size to 1, so that we only have a single session
we set the for the idle-transaction-session-timeout to 2.5 sec:
SET idle_in_transaction_session_timeout TO 2500
now the active transaction will be in state idle in transaction:
see What can cause “idle in transaction” for “BEGIN” statements
now we start a transaction and sleep for 5 seconds
after 2.5sec the database will terminate the session and send an error to the client:
pgp-error error: terminating connection due to idle-in-transaction timeout
after another 2.5sec the transactional code tries to send a query (via the already terminated session), and this fails as expected:
dbIdle failed Error: Client has encountered a connection error and is not queryable
then pg-promise will try to rollback the transaction which will also fail (also expected, I guess)
But now we start a new query and also this query fails with
dbCall failed Client has encountered a connection error and is not queryable
is this expected? I was hoping that pg-promise can somehow remove the "broken" connection from the pool and that we could get a new one
obvously this is not the case, so how should we deal with this situation: i.e. how to recover, so that we can send new queries to the database?
Code example:
import pgPromise, { IMain } from "pg-promise";
import * as dbConfig from "./db-config.json";
import { IConnectionParameters } from "pg-promise/typescript/pg-subset";
const cll = "pg";
console.time(cll);
const pgp: IMain = pgPromise({
query(e) {
console.timeLog(cll,`> ${e.query}`);
},
error(e, ctx) {
console.timeLog(cll,"pgp-error", e);
}
});
const connectParams: IConnectionParameters = {
...dbConfig,
application_name: "pg-test",
max: 1
};
const db = pgp(connectParams);
/**
* #param timeoutMs 0 is no timeout
*/
async function setDbIdleInTxTimeout(timeoutMs: number = 0) {
await db.any("SET idle_in_transaction_session_timeout TO $1;", timeoutMs);
}
async function dbIdle(sleepTimeSec: number) {
console.timeLog(cll, `starting db idle ${sleepTimeSec}`);
const result = await db.tx(async t => {
await new Promise(resolve => setTimeout(resolve, sleepTimeSec * 1000));
return t.one("Select $1 as sleep_sec", sleepTimeSec);
});
console.timeLog(cll, result);
}
async function main() {
await setDbIdleInTxTimeout(2500);
try {
await dbIdle(5);
} catch (e) {
console.timeLog(cll, "dbIdle failed", e);
}
try {
await db.one("Select 1+1 as res");
} catch (e) {
console.timeLog(cll, "dbCall failed", e);
}
}
main().finally(() => {
pgp.end();
});
Console output (removed some useless lines):
"C:\Program Files\nodejs\node.exe" D:\dev_no_backup\pg-promise-tx\dist\index.js
pg: 23.959ms > SET idle_in_transaction_session_timeout TO 2500;
pg: 28.696ms starting db idle 5
pg: 29.705ms > begin
pg: 2531.247ms pgp-error error: terminating connection due to idle-in-transaction timeout
at TCP.onStreamRead (internal/stream_base_commons.js:182:23) {
name: 'error',
severity: 'FATAL',
code: '25P03',
}
pg: 2533.569ms pgp-error Error: Connection terminated unexpectedly
pg: 5031.091ms > Select 5 as sleep_sec
pg: 5031.323ms pgp-error Error: Client has encountered a connection error and is not queryable
pg: 5031.489ms > rollback
pg: 5031.570ms pgp-error Error: Client has encountered a connection error and is not queryable
pg: 5031.953ms dbIdle failed Error: Client has encountered a connection error and is not queryable
pg: 5032.094ms > Select 1+1 as res
pg: 5032.164ms pgp-error Error: Client has encountered a connection error and is not queryable
pg: 5032.303ms dbCall failed Error: Client has encountered a connection error and is not queryable
Process finished with exit code 0

This issue #680 has been fixed in pg-promise 10.3.5

Related

for a grpc client `listen` function to the stream not working same as `await for (...)`

I have a golang server which streams data and a dart client. I put the following code which seems to be working fine
var response = stub.streamMusic(RequestMusicId(musicId: musicId));
await for(var v in response) {
print(v);
}
but when i tried to listen to the stream using this
var response = stub.streamMusic(RequestMusicId(musicId: musicId));
response.listen((value) {
print(value);
});
i get the following error, even though the server is running.
Connecting to VM Service at http://127.0.0.1:55936/l3_uHzU_qIw=/
Unhandled exception:
gRPC Error (code: 14, codeName: UNAVAILABLE, message: Error connecting: Connection shutting down., details: null, rawResponse: null, trailers: {})
Exited (255)

Future not completed on PostgreSQL + Dart

I have got follow code:
void main() async {
connection = PostgreSQLConnection('localhost', 5432, 'db', username: 'postgres', password: '12345');
await connection.open();
//...
// some http handler that have next code inside:
List<List<dynamic>> result = await connection.query(body['sql']).timeout(Duration(seconds: 90));
If I run a lot of queries to http handler I am getting error:
TimeoutException after 0:00:30.000000: Future not completed
It's very strange, because on query I have timeout 90 seconds. So it's happens on connection. But I am opening connection only once in main.
Could anybody explain how it's happens? And how to fix it?
PostgreSQLConnection has parameters: timeoutInSeconds and queryTimeoutInSeconds and they both default to 30. I suggest you use them instead of timeout of Future.

How to catch/Capture "javax.net.ssl.SSLHandshakeException: Failed to create SSL connection" while sending message over java vert.x Eventbus

I am trying to use SSL over eventbus. To test the failure case I tried sending message to the eventbus from another verticle in same cluster by passing some different keystore.
I am getting below exception on console but it is not failing the replyHandler hence my code is not able to detect the SSL exception.
my code:
eb.request("ping-address", "ping!", new DeliveryOptions(), reply -> {
try {
if (reply.succeeded()) {
System.out.println("Received reply " + reply.result().body());
} else {
System.out.println("An exception " + reply.cause().getMessage());
}
} catch (Exception e) {
System.out.println("An error occured" + e.getCause());
}
});
Exception on console:
**javax.net.ssl.SSLHandshakeException: Failed to create SSL connection**
at io.vertx.core.net.impl.ChannelProvider$1.userEventTriggered(ChannelProvider.java:109)
at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:341)
at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:327)
at io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:319)
at io.netty.handler.ssl.SslHandler.handleUnwrapThrowable(SslHandler.java:1249)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1230)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514)
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:813)
Caused by: javax.net.ssl.SSLException: Received fatal alert: bad_certificate
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1647)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1615)
at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1781)
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1070)
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896)
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:282)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1329)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
... 20 more
But handler is failing for timeout after 30 sec.
Timed out after waiting 30000(ms) for a reply. address: __vertx.reply.8419a431-d633-4ba8-a12e-c41fd5a4f37a, repliedAddress: ping-address
I want to capture the SSL exception immediately and handle it. Please guide me how can I Capture/catch this exception.
I tried with below code. Below one is able to handle the exception and I am not getting reply result from called event-bus. Reply result is always null. (value is always null)
MessageProducer<Object> ms = eb.sender("ping-address");
ms.write("ping!", reply -> {
if (reply.succeeded()) {
reply.map(value -> {
System.out.println("Received reply " + value);
return reply;
});
} else {
System.out.println("No reply");
System.out.println("An exception : " + reply.cause().getMessage());
}
});
You can't catch this exception because the Vert.x clustered EventBus implementation buffers messages when the nodes are not connected together. The message could be sent later if the problem is only temporary.
If you want to be notified earlier, you could set a lower timeout in DeliveryOptions.

Error # Database Connection Initialization : Error: database name must be a string

i just try to connect mongodb. but it show
Error # Database Connection Initialization : Error: database name
must be a string
Following is code:
try {
mongoDb.connect("localhost:27017/sample");
var db = mongoDb
}
catch (err) {
console.log('Error # Database Connection Initialization : ' + err);
}
Please try this
// Initialize connection once
MongoClient.connect("mongodb://localhost:27017/sample", function(err, database) {
if(err) return console.error(err);
db = database;
// the Mongo driver recommends starting the server here because most apps *should* fail to start if they have no DB. If yours is the exception, move the server startup elsewhere.
});

Protractor 5.1.1- Chai assertion failure is causing process to exit with error code: 199. No reports are generating after that

cucumber framework. When a chai assertion is failed, the process is exiting with error code:199. No reports are generated after that.
Protractor - 5.1.1
Here is my updated code,
this.Then(/^I should see process is saved in db$/, function (next) {
var sql = "select * from process where name = ?";
sql = mysql.format(sql, params.flow.procName);
console.log(sql);
dbConn.query(sql, function(err, rows, fields){
if(!err) {
procDbObj = rows;
var procName = procDbObj[0].name;
console.log(rows);
expect(procDbObj[0].name).to.equal(params.flow.procName);
expect(procDbObj[0].description).to.equal(params.flow.procDesc);
expect(procName).to.equal("AABBDCD").and.notify(next);
}
});
});
Below is the error I am seeing when an assertion is failed,
[11:23:59] E/launcher - expected 'Auto_proc_2h5c83' to equal 'AABBDCD'
[11:23:59] E/launcher - AssertionError: expected 'Auto_proc_2h5c83' to equal 'AABBDCD'
at Query._callback (C:\Users\panubrolu\workspace\ProtractorCucumber\features\step_definitions\E2E_step_definition.js:64:34)
at Query.Sequence.end (C:\Users\panubrolu\workspace\ProtractorCucumber\node_modules\mysql\lib\protocol\sequences\Sequence.js:86:24)
at Query._handleFinalResultPacket (C:\Users\panubrolu\workspace\ProtractorCucumber\node_modules\mysql\lib\protocol\sequences\Query.js:137:8)
at Query.EofPacket (C:\Users\panubrolu\workspace\ProtractorCucumber\node_modules\mysql\lib\protocol\sequences\Query.js:121:8)
at Protocol._parsePacket (C:\Users\panubrolu\workspace\ProtractorCucumber\node_modules\mysql\lib\protocol\Protocol.js:280:23)
at Parser.write (C:\Users\panubrolu\workspace\ProtractorCucumber\node_modules\mysql\lib\protocol\Parser.js:75:12)
at Protocol.write (C:\Users\panubrolu\workspace\ProtractorCucumber\node_modules\mysql\lib\protocol\Protocol.js:39:16)
at Socket. (C:\Users\panubrolu\workspace\ProtractorCucumber\node_modules\mysql\lib\Connection.js:103:28)
at emitOne (events.js:96:13)
at Socket.emit (events.js:191:7)
[11:23:59] E/launcher - Process exited with error code 199
Any help is really appreciated. Thanks in advance.
I think there are a few problems in your code.
1: It the db-query a Promise? You are using Callback (next) and Promises ('return') at the same time. If it is a Promise then the code should be something like this
this.Then(/^I should see process is saved in db$/, function () {
var sql = "select * from process where name = ?";
sql = mysql.format(sql, params.flow.procName);
console.log(sql);
return dbConn.query(sql, function(err, rows, fields){
if(!err) {
procDbObj = rows;
var procName = procDbObj[0].name;
console.log(rows);
expect(procDbObj[0].name).to.eventually.equal(params.flow.procName);
expect(procDbObj[0].description).to.eventually.equal(params.flow.procDesc);
} else {
return Promise.reject('Query Failed, error = ' + err);
}
});
});
2: If it is a promise, did you also use chai and chai-as-promised?
3: If you are using a callback, then you should 'notify' Cucumber that the scenario is done, you can do that with expect('foo').to.equal('bar').and.notify(next);
4: You don't have a promise / callback when the query fails, see the Promise.reject('message') in the first point
Hope this will give you some info for debugging you problem.