How to set the ThreadPool Size in Slick? - scala

How do I set the threadPool Size in slick ?
Here is my config:
pp {
profile = "slick.jdbc.OracleProfile$"
db {
url = "jdbc:oracle:thin:#52.4.90.244:1521:pp"
driver = oracle.jdbc.OracleDriver
keepAliveConnection = true
connectionPool = disabled
user = "xxx"
password = "xxx"
}
}
I am actually doing only one query, in streaming, do I really need a threadPool ? My goal is simply to retrieve all the rows from a table.
So far i am having the following error:
*** (s.basic.BasicBackend.stream) Signaling onSubscribe(slick.jdbc.JdbcBackend$JdbcStreamingActionContext#76f2bbc1)
*** (s.basic.BasicBackend.action) #1: [fused] cleanUp
try: andThen
1: PushStatementParameters StatementParameters(ForwardOnly,ReadOnly,null,null,100)
2: StreamingResultAction [select * FROM DRUG]
*** (s.basic.BasicBackend.action) #2: SynchronousDatabaseAction.Pin
*** (s.basic.BasicBackend.action) #3: [fused] andThen
1: PushStatementParameters StatementParameters(ForwardOnly,ReadOnly,null,null,100)
2: StreamingResultAction [select * FROM DRUG]
*** (s.basic.BasicBackend.action) #4: SynchronousDatabaseAction.Pin
*** (s.basic.BasicBackend.action) #5: SynchronousDatabaseAction.Unpin
*** (s.basic.BasicBackend.stream) Signaling onError(java.util.concurrent.RejectedExecutionException: Task slick.basic.BasicBackend$DatabaseDef$$anon$2#3061964c rejected from slick.util.AsyncExecutor$$anon$2$$anon$1#55d260cc[Shutting down, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 0])

Related

Transactions are getting stuck because of tperrno issues

After migrating the OS from RHEL-7.9 to RHEL-8 we are continuously receiving the below errors.
A system exception has occurred : Commit transaction failed: tperrno = 1, error text = <TPEABORT - transaction cannot commit>
A system exception has occurred : Enqueuing to QSpace [PPQ_SPACE1], Q [CPA] failed: tperrno = 13, [TPETIME - timeout occured]
A system exception has occurred : Enqueuing to QSpace [PPQ_SPACE1], Q [CPA] failed: tperrno = 12, [TPESYSTEM - internal system error]
Failed to open Resource Manager: tperrno = 16 error text = <TPERMERR - resource manager error
A system exception has occurred : Begin transaction failed: tperrno = 0 error text = <>
Tried to raise this issue with Tuxedo support and applied patch but still receiving this issues.

Powershell and selenium chromedriver getting SSL error opening website [duplicate]

I have added
ChromeOptions options = new ChromeOptions();
options.AddArgument("--ignore-certificate-errors-spki-list");
options.AddArgument("--ignore-ssl-errors");
options.AddArgument("test-type");
options.AddArguments("-incognito");
options.AddArgument("no-sandbox");
options.AddArgument("--start-maximized");
driver = new ChromeDriver(options);
But still getting:
ssl_client_socket_impl.cc(1061)] handshake failed error
How to suppress this error from console?
This error message...
[ERROR:ssl_client_socket_openssl.cc(855)] handshake failed; returned -1, SSL error code 1, net_error -2
...implies that the handshake failed between ChromeDriver and Chrome Browser failed at some point.
Root Cause
This error is generated due to net::SSLClientSocketImpl::DoHandshake and net::SSLClientSocketImpl implemented in ssl_client_socket_impl.cc
net::SSLClientSocketImpl::DoHandshake as follows:
int SSLClientSocketImpl::DoHandshake() {
crypto::OpenSSLErrStackTracer err_tracer(FROM_HERE);
int rv = SSL_do_handshake(ssl_.get());
int net_error = OK;
if (rv <= 0) {
int ssl_error = SSL_get_error(ssl_.get(), rv);
if (ssl_error == SSL_ERROR_WANT_CHANNEL_ID_LOOKUP) {
// The server supports channel ID. Stop to look one up before returning to
// the handshake.
next_handshake_state_ = STATE_CHANNEL_ID_LOOKUP;
return OK;
}
if (ssl_error == SSL_ERROR_WANT_X509_LOOKUP &&
!ssl_config_.send_client_cert) {
return ERR_SSL_CLIENT_AUTH_CERT_NEEDED;
}
if (ssl_error == SSL_ERROR_WANT_PRIVATE_KEY_OPERATION) {
DCHECK(ssl_config_.client_private_key);
DCHECK_NE(kSSLClientSocketNoPendingResult, signature_result_);
next_handshake_state_ = STATE_HANDSHAKE;
return ERR_IO_PENDING;
}
OpenSSLErrorInfo error_info;
net_error = MapLastOpenSSLError(ssl_error, err_tracer, &error_info);
if (net_error == ERR_IO_PENDING) {
// If not done, stay in this state
next_handshake_state_ = STATE_HANDSHAKE;
return ERR_IO_PENDING;
}
LOG(ERROR) << "handshake failed; returned " << rv << ", SSL error code "
<< ssl_error << ", net_error " << net_error;
net_log_.AddEvent(
NetLogEventType::SSL_HANDSHAKE_ERROR,
CreateNetLogOpenSSLErrorCallback(net_error, ssl_error, error_info));
}
next_handshake_state_ = STATE_HANDSHAKE_COMPLETE;
return net_error;
}
As per ERROR:ssl_client_socket_openssl.cc handshake failed the main issue is the failure of handshake when ChromeDriver handshakes with SSL pages in Chrome. Though Chromium team conducts test for SSL handshake through net_unittests, content_tests, and browser_tests but were not exhaustive. Some usecases are left out relying on the upstream tests.
Conclusion
This error won't interupt the execution of your Test Suite and you can ignore this issue for the time being till it is fixed by the Chromium Team.
You can restrict Chromium's log level to 3, so that only fatal errors are logged. Please bear in mind that you won't see any other error-related messages which might cause mayhem in production! The code looks like this:
var chromeOptions = new ChromeOptions();
chromeOptions.AddArgument("log-level=3");
var driver = new ChromeDriver(options : chromeOptions);
Chromium log levels are:
Description
Value
INFO
0
WARNING
1
LOG_ERROR
2
LOG_FATAL
3

How to handle idle_in_transaction_session_timeout?

When we set idle_in_transaction_session_timeout, the database will terminate connections that are idle for some time.
This works as expected, but I wonder how we should deal with this situation in the aplication code.
We are using pg-promise 10.3.1.
Details of the test:
we set the connection pool size to 1, so that we only have a single session
we set the for the idle-transaction-session-timeout to 2.5 sec:
SET idle_in_transaction_session_timeout TO 2500
now the active transaction will be in state idle in transaction:
see What can cause “idle in transaction” for “BEGIN” statements
now we start a transaction and sleep for 5 seconds
after 2.5sec the database will terminate the session and send an error to the client:
pgp-error error: terminating connection due to idle-in-transaction timeout
after another 2.5sec the transactional code tries to send a query (via the already terminated session), and this fails as expected:
dbIdle failed Error: Client has encountered a connection error and is not queryable
then pg-promise will try to rollback the transaction which will also fail (also expected, I guess)
But now we start a new query and also this query fails with
dbCall failed Client has encountered a connection error and is not queryable
is this expected? I was hoping that pg-promise can somehow remove the "broken" connection from the pool and that we could get a new one
obvously this is not the case, so how should we deal with this situation: i.e. how to recover, so that we can send new queries to the database?
Code example:
import pgPromise, { IMain } from "pg-promise";
import * as dbConfig from "./db-config.json";
import { IConnectionParameters } from "pg-promise/typescript/pg-subset";
const cll = "pg";
console.time(cll);
const pgp: IMain = pgPromise({
query(e) {
console.timeLog(cll,`> ${e.query}`);
},
error(e, ctx) {
console.timeLog(cll,"pgp-error", e);
}
});
const connectParams: IConnectionParameters = {
...dbConfig,
application_name: "pg-test",
max: 1
};
const db = pgp(connectParams);
/**
* #param timeoutMs 0 is no timeout
*/
async function setDbIdleInTxTimeout(timeoutMs: number = 0) {
await db.any("SET idle_in_transaction_session_timeout TO $1;", timeoutMs);
}
async function dbIdle(sleepTimeSec: number) {
console.timeLog(cll, `starting db idle ${sleepTimeSec}`);
const result = await db.tx(async t => {
await new Promise(resolve => setTimeout(resolve, sleepTimeSec * 1000));
return t.one("Select $1 as sleep_sec", sleepTimeSec);
});
console.timeLog(cll, result);
}
async function main() {
await setDbIdleInTxTimeout(2500);
try {
await dbIdle(5);
} catch (e) {
console.timeLog(cll, "dbIdle failed", e);
}
try {
await db.one("Select 1+1 as res");
} catch (e) {
console.timeLog(cll, "dbCall failed", e);
}
}
main().finally(() => {
pgp.end();
});
Console output (removed some useless lines):
"C:\Program Files\nodejs\node.exe" D:\dev_no_backup\pg-promise-tx\dist\index.js
pg: 23.959ms > SET idle_in_transaction_session_timeout TO 2500;
pg: 28.696ms starting db idle 5
pg: 29.705ms > begin
pg: 2531.247ms pgp-error error: terminating connection due to idle-in-transaction timeout
at TCP.onStreamRead (internal/stream_base_commons.js:182:23) {
name: 'error',
severity: 'FATAL',
code: '25P03',
}
pg: 2533.569ms pgp-error Error: Connection terminated unexpectedly
pg: 5031.091ms > Select 5 as sleep_sec
pg: 5031.323ms pgp-error Error: Client has encountered a connection error and is not queryable
pg: 5031.489ms > rollback
pg: 5031.570ms pgp-error Error: Client has encountered a connection error and is not queryable
pg: 5031.953ms dbIdle failed Error: Client has encountered a connection error and is not queryable
pg: 5032.094ms > Select 1+1 as res
pg: 5032.164ms pgp-error Error: Client has encountered a connection error and is not queryable
pg: 5032.303ms dbCall failed Error: Client has encountered a connection error and is not queryable
Process finished with exit code 0
This issue #680 has been fixed in pg-promise 10.3.5

Https proxy to URLSession

My code:
init() {
let configuration = URLSessionConfiguration.default
configuration.httpCookieStorage = HTTPCookieStorage.shared
configuration.requestCachePolicy = .reloadIgnoringLocalCacheData
configuration.urlCache = nil
var proxy = [AnyHashable: Any]()
proxy[AnyHashable(kCFNetworkProxiesHTTPSEnable)] = 1
proxy[AnyHashable(kCFNetworkProxiesHTTPSProxy)] = "..."
proxy[AnyHashable(kCFNetworkProxiesHTTPSPort)] = 14283
proxy[AnyHashable(kCFProxyUsernameKey)] = "..."
proxy[AnyHashable(kCFProxyPasswordKey)] = "..."
configuration.connectionProxyDictionary = proxy
When I run the application, I see this:
2017-10-25 00:45:44.768947+0500 ... Api[58675:12224660] *** WARNING: CFMachPortSetInvalidationCallBack() called on a CFMachPort with a Mach port (0x3613) which does not have any send rights. This is not going to work. Callback function: 0x7fff389e734f
2017-10-25 00:45:44.769328+0500 ... Api[58675:12224660] TIC Read Status [1:0x100f91800]: 1:57
2017-10-25 00:45:44.798125+0500 ... Api[58675:12224660] Task <4EFD78A7-1055-4D1C-9322-D0954B2B3760>.<1> HTTP load failed (error code: 310 [4:-2096])
2017-10-25 00:45:44.798300+0500 ... Api[58675:12224656] Task <4EFD78A7-1055-4D1C-9322-D0954B2B3760>.<1> finished with error - code: 310
URL Session Task Failed: The operation couldn’t be completed. (kCFErrorDomainCFNetwork error 310.)
Program ended with exit code: 0
What could be the problem?
Either the username and password is not correct or the port mentioned is used by any other program. Restart the phone and run this app and check it again.

Failing spec in Scala/Play during insert into DB(slick)(json request)

I just try to write a simple spec like this:
"saves the record on create" in {
val request = FakeRequest(POST, "/countries").withJsonBody(Json.parse("""{ "country": {"title":"Germany", "abbreviation":"GER"} }"""))
val create = route(app, request).get
status(create) mustBe OK
contentType(create) mustBe Some("application/json")
contentAsString(create) must include("country")
}
But on execution it throws such an error:
java.util.concurrent.RejectedExecutionException: Task slick.backend.DatabaseComponent$DatabaseDef$$anon$2#f456097 rejected from java.util.concurrent.ThreadPoolExecutor#6265d40c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1]
It works good for get request test for index page, any ideas how to workaround this ?
The problem was OneAppPerTest since problems with DB connections: just replacing it to OneAppPerSuitesolves the problem