I have created a job which is writing data into a table in PostgreSQL DB, this is the staging job. After the staging job is completed, another job writes the staging data into the final table which is in the same database. However, the staging job works fine but the final job fails with the below error. I have worked with Talend and Database team and everything works fine at the server level. Not sure why the final job is failing when there is no issue with the staging job. Could anyone please help with this issue.
Talend version : 7.3
PostgreSQL version : 14.2
[INFO ]: aws_dev_engagebh.job_stg_view_to_ods_orders_view_0_3.Job_STG_VIEW_TO_ODS_ORDERS_VIEW - TalendJob: 'Job_STG_VIEW_TO_ODS_ORDERS_VIEW' - Start.
[statistics] connecting to socket on port 10633
[statistics] connected
[FATAL]: aws_dev_engagebh.job_stg_view_to_ods_orders_view_0_3.Job_STG_VIEW_TO_ODS_ORDERS_VIEW - tDBOutput_2 SSL error: java.net.SocketException: Connection reset
org.postgresql.util.PSQLException: SSL error: java.net.SocketException: Connection reset
at org.postgresql.ssl.MakeSSL.convert(MakeSSL.java:43)
at org.postgresql.core.v3.ConnectionFactoryImpl.enableSSL(ConnectionFactoryImpl.java:446)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:140)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:197)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:217)
at org.postgresql.Driver.makeConnection(Driver.java:458)
at org.postgresql.Driver.connect(Driver.java:260)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at aws_dev_engagebh.job_stg_view_to_ods_orders_view_0_3.Job_STG_VIEW_TO_ODS_ORDERS_VIEW.tDBInput_1Process(Job_STG_VIEW_TO_ODS_ORDERS_VIEW.java:5543)
at aws_dev_engagebh.job_stg_view_to_ods_orders_view_0_3.Job_STG_VIEW_TO_ODS_ORDERS_VIEW.runJobInTOS(Job_STG_VIEW_TO_ODS_ORDERS_VIEW.java:7971)
at aws_dev_engagebh.job_stg_view_to_ods_orders_view_0_3.Job_STG_VIEW_TO_ODS_ORDERS_VIEW.main(Job_STG_VIEW_TO_ODS_ORDERS_VIEW.java:7316)
Caused by: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
at sun.security.ssl.Alert.createSSLException(Alert.java:127)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:370)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:313)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:308)
at sun.security.ssl.SSLTransport.decode(SSLTransport.java:141)
at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1290)
at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1199)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:401)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:373)
at org.postgresql.ssl.MakeSSL.convert(MakeSSL.java:41)
Related
I have a PostGIS database and a client built on top of HikariCP to read the data from a database. My client can read the data without any problem on some machines. However, on some other machines client gets stuck and is not able to read any data throwing socket timeout exceptions.
MyClass:120 - Failed to execute HikariProxyPreparedStatement#2091541230 wrapping <my-query>.
org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:332)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:155)
at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:118)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)
...
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:140)
at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:109)
at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:67)
at org.postgresql.core.PGStream.receiveChar(PGStream.java:293)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1947)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
... 32 more
ProxyConnection:161 - HikariPool-1 - Connection org.postgresql.jdbc.PgConnection#1aafd32f marked as broken because of SQLSTATE(08006), ErrorCode(0)
org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:332)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:155)
at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:118)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)
...
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:140)
at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:109)
at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:67)
at org.postgresql.core.PGStream.receiveChar(PGStream.java:293)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1947)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
... 31 more
Before client throws SocketTimeoutException on the database side, I monitored pg_stat_activity table. The corresponding row for the query above had wait_event_type=Client and wait_event=ClientWrite. In addition, database server logged messages indicating connection is lost.
LOG: unexpected EOF on client connection with an open transaction
LOG: could not send data to client: Connection timed out
FATAL: connection to client lost
Versions
PostGIS-jdbc: 2.2.1 (postgresql jdbc: 9.4.1208.jre7)
HikariCP: 3.1.0
Postgres server: 10.3
PostGIS server: 2.4.4
If I don't set socketTimeout through jdbc connection string, then connection would get stuck forever. Once the connection reaches it's max life time, it would be dropped and connected again. However, it still cannot read the data. When I set socketTimeout, then exception would be thrown.
UPDATE
If socketTimeout is not set, then pg_stat_activity table would have a row for the connection with the following values: state=idle in transaction, wait_event_type=Client and wait_event=ClientRead.
My guess is that some sort of network setting is blocking the read from the server on the client side. How can I further debug this and find the root cause?
We found out that this was caused by the database server's MTU setting. MTU was set to 9000 by default and resulted in packet loss. Changing it to 1500 resolved the issue.
Trying to add a Postgres database on Metabase via SSH tunneling I ran into an error giving me the following error message: "Server error encountered"
My logs are very similar to #williamjacksn 's logs on the post:
http://discourse.metabase.com/t/ssh-tunnel-failure/2469
I am certain that my credentials are OK since I use the same for a Postico SSH connection as well as a DBeaver connection.
I already checked the following fix: https://github.com/metabase/metabase/pull/6970
Howerver I have no idea how to use it as i cannot find the ssh.clj file on my machine.
Would there be a fix for this ?
Extra info:
Install via the .jar file. Working on Firefox 58.0.2 | Mac OSX 10.10.5
Thanks in advance
Log:
03-12 10:16:48 INFO util.ssh :: creating ssh tunnel metabase#192.168.1.2:22 -L 56334:localhost:5432
03-12 10:16:53 ERROR metabase.driver :: Failed to connect to database: Timed out after 5000 milliseconds.
03-12 10:16:53 DEBUG metabase.middleware :: POST /api/setup/validate 400 (5 s) (0 DB calls)
{:errors {:dbname "Timed out after 5000 milliseconds."}}
Mar 12, 2018 10:16:53 AM org.postgresql.Driver connect
SEVERE: Connection error:
org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:275)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.(PgConnection.java:194)
at org.postgresql.Driver.makeConnection(Driver.java:450)
at org.postgresql.Driver.connect(Driver.java:252)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:678)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:190)
at clojure.java.jdbc$get_connection.invokeStatic(jdbc.clj:364)
at clojure.java.jdbc$get_connection.invoke(jdbc.clj:226)
at clojure.java.jdbc$db_query_with_resultset_STAR_.invokeStatic(jdbc.clj:1014)
at clojure.java.jdbc$db_query_with_resultset_STAR_.invoke(jdbc.clj:996)
at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1090)
at clojure.java.jdbc$query.invoke(jdbc.clj:1047)
at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1063)
at clojure.java.jdbc$query.invoke(jdbc.clj:1047)
at metabase.driver.generic_sql$can_connect_QMARK_.invokeStatic(generic_sql.clj:220)
at metabase.driver.generic_sql$can_connect_QMARK_.invoke(generic_sql.clj:217)
at metabase.driver$fn__25577$G__25354__25584.invoke(driver.clj:51)
at metabase.driver$can_connect_with_details_QMARK_$fn__25710.invoke(driver.clj:451)
at clojure.core$binding_conveyor_fn$fn__4676.invoke(core.clj:1938)
at clojure.lang.AFn.call(AFn.java:18)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:400)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:243)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:225)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:402)
at java.base/java.net.Socket.connect(Socket.java:591)
at org.postgresql.core.PGStream.(PGStream.java:68)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:144)
... 24 more
Mar 12, 2018 10:16:58 AM org.postgresql.Driver connect
SEVERE: Connection error:
org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:275)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.(PgConnection.java:194)
at org.postgresql.Driver.makeConnection(Driver.java:450)
at org.postgresql.Driver.connect(Driver.java:252)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:678)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:190)
at clojure.java.jdbc$get_connection.invokeStatic(jdbc.clj:364)
at clojure.java.jdbc$get_connection.invoke(jdbc.clj:226)
at clojure.java.jdbc$db_query_with_resultset_STAR_.invokeStatic(jdbc.clj:1014)
at clojure.java.jdbc$db_query_with_resultset_STAR_.invoke(jdbc.clj:996)
at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1090)
at clojure.java.jdbc$query.invoke(jdbc.clj:1047)
at clojure.java.jdbc$query.invokeStatic(jdbc.clj:1063)
at clojure.java.jdbc$query.invoke(jdbc.clj:1047)
at metabase.driver.generic_sql$can_connect_QMARK_.invokeStatic(generic_sql.clj:220)
at metabase.driver.generic_sql$can_connect_QMARK_.invoke(generic_sql.clj:217)
at metabase.driver$fn__25577$G__25354__25584.invoke(driver.clj:51)
at metabase.driver$can_connect_with_details_QMARK_$fn__25710.invoke(driver.clj:451)
at clojure.core$binding_conveyor_fn$fn__4676.invoke(core.clj:1938)
at clojure.lang.AFn.call(AFn.java:18)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:400)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:243)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:225)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:402)
at java.base/java.net.Socket.connect(Socket.java:591)
at org.postgresql.core.PGStream.(PGStream.java:68)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:144)
... 24 more`
I already checked the following fix: https://github.com/metabase/metabase/pull/6970 Howerver I have no idea how to use it as i cannot find the ssh.clj file on my machine.
To try out the ssh.clj changes proposed in the pull request you need to get the source code from GitHub, then build Metabase yourself from source code. I just happen to have described the steps over on the Metabase discussion forum for someone in a similar situation (though needing a different PR for a different purpose):
http://discourse.metabase.com/t/snowflake-driver-test/3110
I am not able to shutdown jboss server with the below command.
Command :
JBOSS_HOME/bin/jboss-cli.sh --connect controller=$SERVER_IP_ADDRESS:$SERVER_PORT command=:shutdown
Each time I was killing the server to restart which is not a good process to do. as we are moving to PROD environment we should use shutdown command to stop the server instead of killing
I am getting the below error. Please help.
Server Log :
jboss#devkopmdmh01.corp.ybusa.net::/usr/local/prod/jboss/jboss-eap-6.1/jboss-as/bin > ./shutdownMDM.sh
org.jboss.as.cli.CliInitializationException: Failed to connect to the controller
at org.jboss.as.cli.impl.CliLauncher.initCommandContext(CliLauncher.java:280)
Caused by: org.jboss.as.cli.CommandLineException: The controller is not available at 10.0.15.162:8080
at org.jboss.as.cli.impl.CommandContextImpl.tryConnection(CommandContextImpl.java:951)
... 8 more
Caused by: java.io.IOException: java.net.ConnectException: JBAS012144: Could not connect to remote://10.0.15.162:8080. The connection timed out
at org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:129)
... 11 more
Caused by: java.net.ConnectException: JBAS012144: Could not connect to remote://10.0.15.162:8080. The connection timed out
at org.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:131)
... 13 more
The JBoss CLI is attempting to connect to the native management endpoint for the running JBoss and send a shutdown command. It looks like it's trying to send to 10.0.15.162:8080 which is not the right port (most likely).
Take a look in your bin/jboss-cli.xml file which should contain the host and port to connect to. For example:
<default-controller>
<host>localhost</host>
<port>9999</port>
</default-controller>
I am not able to establish connection to Compose for PostgreSQL DB service using eclipse. Getting below error while Testing the connection.
org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:122)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:65)
at org.postgresql.jdbc2.AbstractJdbc2Connection.(AbstractJdbc2Connection.java:116)
at org.postgresql.jdbc3.AbstractJdbc3Connection.(AbstractJdbc3Connection.java:30)
at org.postgresql.jdbc3.Jdbc3Connection.(Jdbc3Connection.java:24)
at org.postgresql.Driver.makeConnection(Driver.java:369)
at org.postgresql.Driver.connect(Driver.java:245)
at org.eclipse.datatools.enablement.internal.postgresql.PostgreSQLJDBCConnection.createConnection(PostgreSQLJDBCConnection.java:87)
at org.eclipse.datatools.connectivity.DriverConnectionBase.internalCreateConnection(DriverConnectionBase.java:105)
at org.eclipse.datatools.connectivity.DriverConnectionBase.open(DriverConnectionBase.java:54)
at org.eclipse.datatools.enablement.internal.postgresql.PostgreSQLJDBCConnection.(PostgreSQLJDBCConnection.java:47)
at org.eclipse.datatools.enablement.internal.postgresql.PostgreSQLConnectionFactory.createConnection(PostgreSQLConnectionFactory.java:51)
at org.eclipse.datatools.connectivity.internal.ConnectionFactoryProvider.createConnection(ConnectionFactoryProvider.java:83)
at org.eclipse.datatools.connectivity.internal.ConnectionProfile.createConnection(ConnectionProfile.java:359)
at org.eclipse.datatools.connectivity.ui.PingJob.createTestConnection(PingJob.java:76)
at org.eclipse.datatools.connectivity.ui.PingJob.run(PingJob.java:59)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Caused by: java.net.ConnectException: Connection refused: connect at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
I am able to connect to the DB through the application deployed in BlueMix.
How are you populating the connection credentials? The application will have access the VCAP_SERVICES when run on Bluemix, but unless you have specifically copied that environment variable to your Eclipse environment, that information will not be available to your application.
The error indicates that the connection was refused. This most likely means that the host and port combination you've entered is incorrect. You'll want to ensure that you can connect to the host.
You can check that your local system can connect by doing nc -vz xxxxx.dblayer.com 10000 where xxxxx.dblayer.com is the full hostname provided to you within the variable and 10000 is the port, also provided.
It seems as though your syntax/configuration may not be correct as a connection being refused would indicate that it cannot connect however it's not clear from your post what hostname/port it's actually trying to connect to.
I'm trying to start a play application in a console on heroku using this line of code:
new play.core.StaticApplication(new java.io.File("."))
Unfortunately it seems to be having a problem connecting to my database. I'm using postgres. Here's my configuration from application.conf
db.default.driver=org.postgresql.Driver
db.default.url=${?DATABASE_URL}
db.default.user=myusername
db.default.password=mypassword
My app is able to access the database through regular use. Here's the error i'm getting:
c.j.b.h.AbstractConnectionHook - Failed to obtain initial connection Sleeping for 0ms and trying again. Attempts left: 0. Exception: java.net.ConnectException: Connection refused.Message:Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
Configuration error: Configuration error[Cannot connect to database [default]]