Kafka Connect Debezium postgres - postgresql

I am trying to connect a Postgres SQL db hosted in Amazon RDS with a Kafka topic using Debezium.
I am following the following tutorial :
http://debezium.io/docs/tutorial/
My kafka and kafka connect services start up fine and the kafka connect service also picks up my debezium postgres connector jars in /usr/share/java dir.
However on trying to attach the postgres config json via the kafka connect API using the following curl command :
curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d #connector1.json
I end up getting the following error:
[2018-06-13 23:45:44,749] ERROR Uncaught exception in REST call to /connectors/ (org.apache.kafka.connect.runtime.rest.errors.ConnectExceptionMapper:61)
org.apache.kafka.connect.errors.ConnectException: Could not intialize type registry
at io.debezium.connector.postgresql.connection.PostgresConnection.<init>(PostgresConnection.java:68)
at io.debezium.connector.postgresql.PostgresConnector.validate(PostgresConnector.java:95)
at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:277)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$6.call(DistributedHerder.java:534)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$6.call(DistributedHerder.java:531)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.tick(DistributedHerder.java:267)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:216)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:275)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
at org.postgresql.Driver.makeConnection(Driver.java:431)
at org.postgresql.Driver.connect(Driver.java:247)
at io.debezium.jdbc.JdbcConnection.lambda$patternBasedFactory$1(JdbcConnection.java:161)
at io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:585)
at io.debezium.connector.postgresql.connection.PostgresConnection.<init>(PostgresConnection.java:65)
... 11 more
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.postgresql.core.PGStream.<init>(PGStream.java:62)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:144)
... 18 more
Could some please advice on the exception and how to resolve it?
Is there a property/configuratuion that I may be missing here?

Do you have postgres database port configured to be accessible from outside?
IIRC RDS database uses SSL for connection so it might be necessary to configure SSL also in postgres connector.

Related

MongoDB Debezium Fails to connect due to ssl handshake failure

I'm running a MongoDB Debezium Kafka Connector on AWS MSK, and the connector goes to the failed status with this error on the MongoDB server Error receiving request from client: SSLHandshakeFailed: The server is configured to only allow SSL connections and com.mongodb.MongoSocketReadException: Prematurely reached end of stream in the debezium logs.
Below is my debezium configuration, and I have enabled mongodb.ssl.enabled=true.
Does anybody know if I'm missing something from the configuration?
I also enabled the mongodb.ssl.invalid.hostname.allowed but that didn't fix the issue
connector.class=io.debezium.connector.mongodb.MongoDbConnector
mongodb.ssl.enabled=true
collection.include.list=***
mongodb.password=***
tasks.max=2
mongodb.user=***
mongodb.ssl.invalid.hostname.allowed=true
mongodb.hosts=***
database.include.list=***
Debezium stack trace:
at
com.mongodb.connection.BaseCluster.getDescription(BaseCluster.java:160)
at com.mongodb.Mongo.getClusterDescription(Mongo.java:378) at
com.mongodb.Mongo.getReplicaSetStatus(Mongo.java:414) at
io.debezium.connector.mongodb.ConnectionContext.clientForPrimary(ConnectionContext.java:335)
at
io.debezium.connector.mongodb.ConnectionContext.lambda$primaryClientFor$1(ConnectionContext.java:179)
at
io.debezium.connector.mongodb.ConnectionContext.lambda$primaryClientFor$2(ConnectionContext.java:188)
at
io.debezium.connector.mongodb.ConnectionContext$MongoPrimary.execute(ConnectionContext.java:258)
at
io.debezium.connector.mongodb.ConnectionContext$MongoPrimary.databaseNames(ConnectionContext.java:296)
at
io.debezium.connector.mongodb.MongoDbConnectorConfig$DatabaseRecommender.lambda$validValues$1(MongoDbConnectorConfig.java:239)
at java.base/java.util.HashMap$Values.forEach(HashMap.java:977) at
io.debezium.connector.mongodb.ReplicaSets.onEachReplicaSet(ReplicaSets.java:102)
at
io.debezium.connector.mongodb.MongoDbConnectorConfig$DatabaseRecommender.validValues(MongoDbConnectorConfig.java:236)
at io.debezium.config.Field.validate(Field.java:567) at
io.debezium.config.Field.lambda$validate$7(Field.java:583) at
java.base/java.util.Arrays$ArrayList.forEach(Arrays.java:4390) at
io.debezium.config.Field.validate(Field.java:580) at
io.debezium.config.Configuration.lambda$validate$25(Configuration.java:1653)
at
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) at
java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at
java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at
java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at
java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
at io.debezium.config.Field$Set.forEachTopLevelField(Field.java:127)
at io.debezium.config.Configuration.validate(Configuration.java:1652)
at
io.debezium.connector.mongodb.MongoDbConnector.validate(MongoDbConnector.java:194)
at
org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:375)
at
org.apache.kafka.connect.runtime.AbstractHerder.lambda$validateConnectorConfig$1(AbstractHerder.java:326)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829) [2022-04-14
03:41:56,279] INFO Closing all connections to
(io.debezium.connector.mongodb.ConnectionContext:75) [2022-04-14 03:41:56,280] ERROR Uncaught exception in REST call to /connectors
(org.apache.kafka.connect.runtime.rest.errors.ConnectExceptionMapper:61)
org.apache.kafka.connect.errors.ConnectException: Unable to connect to
primary node of 'atlas-:27017' after 2 failed attempts

Spring boot connect to MongoDB replica set of mongo container running

I have set 2 mongo container which having 27017:27017 and 27018:27017 as port. mongo1 is set as primary while mongo2 set as secondary and used replica set.
mongo1 as primary has both write, read access and mongo2 has only read access
spring.data.mongodb.uri = mongodb://abc.xyz:27017/user_demo
//monog1, abc.xyz is the hostname
spring.data.mongodb.uri = mongodb://abc.xyz:27018/user_demo
//mongo2
If I connect only monogo1 in my normal spring boot program, it works(read, write) and if I set mongo2 it works(only read), but when I set both in one line it gives error
spring.data.mongodb.uri = mongodb://abc.xyz:27017,abc.xyz:27018/user_demo?replicaSet=idea-mongo-set
I checked replica set is working by storing in mongo1, in mongo2 I can read
So, what is wrong with the last line. Is there any other way to store uri for replica set
In official spring site it is given
spring.data.mongodb.uri=mongodb://user:secret#mongo1.example.com:12345,mongo2.example.com:23456/test
Error I am getting is this
2019-01-08 15:56:35.245 INFO 29804 --- [ main] c.j.bootifulmongodb.BootMongoDBApp : Started BootMongoDBApp in 3.603 seconds (JVM running for 4.373)
2019-01-08 15:56:54.466 INFO 29804 --- [azure.com:27018] org.mongodb.driver.cluster : Exception in monitor thread while connecting to server abc.xyz:27018
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.connection.SocketStream.open(SocketStream.java:63) ~[mongodb-driver-core-3.4.3.jar:na]
at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:115) ~[mongodb-driver-core-3.4.3.jar:na]
at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:113) ~[mongodb-driver-core-3.4.3.jar:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[na:1.8.0_181]
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85) ~[na:1.8.0_181]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_181]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_181]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_181]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[na:1.8.0_181]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_181]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_181]
at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:57) ~[mongodb-driver-core-3.4.3.jar:na]
at com.mongodb.connection.SocketStream.open(SocketStream.java:58) ~[mongodb-driver-core-3.4.3.jar:na]
... 3 common frames omitted
It looks like your uri is on the right track for connecting to a replica set.
spring.data.mongodb.uri = mongodb://abc.xyz:27017,abc.xyz:27018/user_demo?replicaSet=idea-mongo-set
Is it just a typo that credentials (user, password) are omitted here?
This exact syntax is what we use for our replica set connection uri:
mongodb://user:pwd#server1:27017,server2:27017,server3:27017/our_collection?replicaSet=replica_set_name&authSource=authagainstthiscollection&authMechanism=SCRAM-SHA-1
Because you set abc.xyz:27018 hostname in mongo rs.initiate, so client will try to access this hostname directly. But this hostname is unreachable for client, it's just a FQDN in your azure cluster.
I think this document will help you clients use the hostnames listed in the replica set config not the seed list

Jaspersoft report won't run on container of a jasper report server

When running a report in jaspersoft studio it succesfully generates a report, with data adapted from a container of a postgres database on a ssh. the server is running as a jaspersoft for docker server on the same ssh.
however, after publishing the report to the jasperserver, when running the report on the server I get the following error trace:
com.jaspersoft.jasperserver.api.JSException: jsexception.error.creating.connection Arguments: at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.JdbcDataSourceService.createConnection(JdbcDataSourceService.java:64) at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.BaseJdbcDataSource.setReportParameterValues(BaseJdbcDataSource.java:49) at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.JdbcDataSourceService.setReportParameterValues(JdbcDataSourceService.java:69) at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.EngineServiceImpl.fillReport(EngineServiceImpl.java:1823) at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.EngineServiceImpl$ReportFill.runWithDataSource(EngineServiceImpl.java:1169) at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.EngineServiceImpl$ReportFill.runReport(EngineServiceImpl.java:1051) at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.EngineServiceImpl$ReportRunnable.run(EngineServiceImpl.java:946) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:207) at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64) at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:136) at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29) at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21) at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:31) at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) at org.postgresql.Driver.makeConnection(Driver.java:393) at org.postgresql.Driver.connect(Driver.java:267) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:75) at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.JdbcDriverManagerConnectionFactory.createConnection(JdbcDriverManagerConnectionFactory.java:36) at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582) at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1148) at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106) at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.TibcoDriverManagerImpl.unlockConnection(TibcoDriverManagerImpl.java:68) at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.JdbcDataSourceService.createConnection(JdbcDataSourceService.java:56) ... 9 more Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at java.net.Socket.connect(Socket.java:538) at org.postgresql.core.PGStream.<init>(PGStream.java:60) at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:101) ... 26 more
Any ideas what the problem is? Is there a way to see what the remaining 26 errors are?
the connection of the containers in the ssh
My machine is windows, but the ssh environment is Linux. I've used the jaspersoft for docker from https://github.com/TIBCOSoftware/JS-Docker
and the datasource is from: https://github.com/docker-library/postgres
Make sure you have a data source defined on the server. When you deploy, you have to manually select this datasource during the deploy.

SSH tunnelling failure between Metabase and Postgresql

Trying to add a Postgres database on Metabase via SSH tunneling I ran into an error giving me the following error message: "Server error encountered"
My logs are very similar to #williamjacksn 's logs on the post:
http://discourse.metabase.com/t/ssh-tunnel-failure/2469
I am certain that my credentials are OK since I use the same for a Postico SSH connection as well as a DBeaver connection.
I already checked the following fix: https://github.com/metabase/metabase/pull/6970
Howerver I have no idea how to use it as i cannot find the ssh.clj file on my machine.
Would there be a fix for this ?
Extra info:
Install via the .jar file. Working on Firefox 58.0.2 | Mac OSX 10.10.5
Thanks in advance
Log:
03-12 10:16:48 INFO util.ssh :: creating ssh tunnel metabase#192.168.1.2:22 -L 56334:localhost:5432
03-12 10:16:53 ERROR metabase.driver :: Failed to connect to database: Timed out after 5000 milliseconds.
03-12 10:16:53 DEBUG metabase.middleware :: POST /api/setup/validate 400 (5 s) (0 DB calls)
{:errors {:dbname "Timed out after 5000 milliseconds."}}
Mar 12, 2018 10:16:53 AM org.postgresql.Driver connect
SEVERE: Connection error:
org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:275)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.(PgConnection.java:194)
at org.postgresql.Driver.makeConnection(Driver.java:450)
at org.postgresql.Driver.connect(Driver.java:252)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:678)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:190)
at clojure.java.jdbc$get_connection.invokeStatic(jdbc.clj:364)
at clojure.java.jdbc$get_connection.invoke(jdbc.clj:226)
at clojure.java.jdbc$db_query_with_resultset_STAR_.invokeStatic(jdbc.clj:1014)
at clojure.java.jdbc$db_query_with_resultset_STAR_.invoke(jdbc.clj:996)
at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1090)
at clojure.java.jdbc$query.invoke(jdbc.clj:1047)
at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1063)
at clojure.java.jdbc$query.invoke(jdbc.clj:1047)
at metabase.driver.generic_sql$can_connect_QMARK_.invokeStatic(generic_sql.clj:220)
at metabase.driver.generic_sql$can_connect_QMARK_.invoke(generic_sql.clj:217)
at metabase.driver$fn__25577$G__25354__25584.invoke(driver.clj:51)
at metabase.driver$can_connect_with_details_QMARK_$fn__25710.invoke(driver.clj:451)
at clojure.core$binding_conveyor_fn$fn__4676.invoke(core.clj:1938)
at clojure.lang.AFn.call(AFn.java:18)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:400)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:243)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:225)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:402)
at java.base/java.net.Socket.connect(Socket.java:591)
at org.postgresql.core.PGStream.(PGStream.java:68)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:144)
... 24 more
Mar 12, 2018 10:16:58 AM org.postgresql.Driver connect
SEVERE: Connection error:
org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:275)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.(PgConnection.java:194)
at org.postgresql.Driver.makeConnection(Driver.java:450)
at org.postgresql.Driver.connect(Driver.java:252)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:678)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:190)
at clojure.java.jdbc$get_connection.invokeStatic(jdbc.clj:364)
at clojure.java.jdbc$get_connection.invoke(jdbc.clj:226)
at clojure.java.jdbc$db_query_with_resultset_STAR_.invokeStatic(jdbc.clj:1014)
at clojure.java.jdbc$db_query_with_resultset_STAR_.invoke(jdbc.clj:996)
at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1090)
at clojure.java.jdbc$query.invoke(jdbc.clj:1047)
at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1063)
at clojure.java.jdbc$query.invoke(jdbc.clj:1047)
at metabase.driver.generic_sql$can_connect_QMARK_.invokeStatic(generic_sql.clj:220)
at metabase.driver.generic_sql$can_connect_QMARK_.invoke(generic_sql.clj:217)
at metabase.driver$fn__25577$G__25354__25584.invoke(driver.clj:51)
at metabase.driver$can_connect_with_details_QMARK_$fn__25710.invoke(driver.clj:451)
at clojure.core$binding_conveyor_fn$fn__4676.invoke(core.clj:1938)
at clojure.lang.AFn.call(AFn.java:18)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:400)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:243)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:225)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:402)
at java.base/java.net.Socket.connect(Socket.java:591)
at org.postgresql.core.PGStream.(PGStream.java:68)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:144)
... 24 more`
I already checked the following fix: https://github.com/metabase/metabase/pull/6970 Howerver I have no idea how to use it as i cannot find the ssh.clj file on my machine.
To try out the ssh.clj changes proposed in the pull request you need to get the source code from GitHub, then build Metabase yourself from source code. I just happen to have described the steps over on the Metabase discussion forum for someone in a similar situation (though needing a different PR for a different purpose):
http://discourse.metabase.com/t/snowflake-driver-test/3110

SQLTransientConnectionException after upgrading MariaDB connector

I've upgraded the "org.mariadb.jdbc" % "mariadb-java-client" connector from 1.5.9 to 1.6.0 and I'd started to fail while connecting to the DB due to timeout exceptions.
I'm using it with HikariCP 2.5.1 and Slick 3.2.0. If I rollback the change again to MariaDB connector 1.5.9 it successfully connects, and if I try to upgrade directly to 2.0.1, it fails with the very same error.
The thing is that based on the 1.6.0 changelog, we shouldn't experiment any breaking change. But according to the differences in the GitHub repository, it could have some more modifications than the ones specified in the changelog :/
Exception with a local DB:
java.sql.SQLTransientConnectionException: xxx.db - Connection is not available, request timed out after 5006ms.
at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:548)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:186)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:145)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:83)
at slick.jdbc.hikaricp.HikariCPJdbcDataSource.createConnection(HikariCPJdbcDataSource.scala:18)
at slick.jdbc.JdbcBackend$BaseSession.<init>(JdbcBackend.scala:439)
at slick.jdbc.JdbcBackend$DatabaseDef.createSession(JdbcBackend.scala:47)
at slick.jdbc.JdbcBackend$DatabaseDef.createSession(JdbcBackend.scala:38)
at slick.basic.BasicBackend$DatabaseDef$class.acquireSession(BasicBackend.scala:218)
at slick.jdbc.JdbcBackend$DatabaseDef.acquireSession(JdbcBackend.scala:38)
at slick.basic.BasicBackend$DatabaseDef$$anon$2.run(BasicBackend.scala:239)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLNonTransientConnectionException: Could not connect to address=(host=localhost)(port=3306)(type=master) : Connection refused
at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.get(ExceptionMapper.java:156)
at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.getException(ExceptionMapper.java:118)
at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.throwException(ExceptionMapper.java:92)
at org.mariadb.jdbc.Driver.connect(Driver.java:108)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:95)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:101)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:341)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:193)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:430)
at com.zaxxer.hikari.pool.HikariPool.access$500(HikariPool.java:64)
at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:570)
at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:563)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: java.sql.SQLException: Could not connect to address=(host=localhost)(port=3306)(type=master) : Connection refused
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:1020)
at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:481)
at org.mariadb.jdbc.Driver.connect(Driver.java:103)
... 12 common frames omitted
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(AbstractConnectProtocol.java:392)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:1013)
... 14 common frames omitted
Exception with a remote DB:
java.sql.SQLTransientConnectionException: xxx.db - Connection is not available, request timed out after 5003ms.
at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:548)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:186)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:145)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:83)
at slick.jdbc.hikaricp.HikariCPJdbcDataSource.createConnection(HikariCPJdbcDataSource.scala:18)
at slick.jdbc.JdbcBackend$BaseSession.<init>(JdbcBackend.scala:439)
at slick.jdbc.JdbcBackend$DatabaseDef.createSession(JdbcBackend.scala:47)
at slick.jdbc.JdbcBackend$DatabaseDef.createSession(JdbcBackend.scala:38)
at slick.basic.BasicBackend$DatabaseDef$class.acquireSession(BasicBackend.scala:218)
at slick.jdbc.JdbcBackend$DatabaseDef.acquireSession(JdbcBackend.scala:38)
at slick.basic.BasicBackend$DatabaseDef$$anon$2.run(BasicBackend.scala:239)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
It certainly looks like, at least in the local case, that:
The port is mis-configured. Or,
The pool size exceeds the database maximum. Or,
The username/password is incorrect. Or,
The user does not have permission to connect to the server or the specified database.
java.net.ConnectException: Connection refused is a socket level error. Typically, this indicates that either there was no server running on the specified port, or the server is rejecting the connection for some other reason (security, etc).
You might double check all of the driver/datasource properties to verify that they are correct. It might be useful if you could post your HikariCP configuration.
Disclaimer: I am one of the mariadb developper
In 1.6.0 version, "usePipelineAuth" option has changed the connection implementation.
During connection, different queries are executed. When option is active those queries are send using pipeline (all queries are send, then only all results are reads), permitting faster connection creation.
That permit to saving network latency.
Disabling this option will probably solved your issue.
At the same time, i've create an issue on mariadb tracker.