I'm setting up a new database server, and want to manipulating through IDEA database tool. However, I can't connect to my remote server, whereas it's ok with local database. I'm pretty sure I filled up every field correctly.
I've downloaded driver file and even added it to project library.
pg_hba.conf is configured to be accept connection from everywhere with md5 auth. I've edited postgresql.conf to listen on "*" too.
What's even weird is that I can connect to my server on another machine!
On successful scenarios, idea.log would have this line:
INFO - ution.rmi.RemoteProcessSupport - Driver: PostgreSQL JDBC Driver 42.2.5
while on my machine, here is what I found (connecting to remote):
INFO - ution.rmi.RemoteProcessSupport -
Driver is absent here. (tested with MySQL, the same: missing Driver, but have correct Driver field when connecting to localhost.)
IDEA didn't provide anything useful when connections failed:
Connection to postgres#servername failed.
[08001] The connection attempt failed.
It seems that IDEA didn't use Driver file while connecting? How can I revolve this? I've spent tons of hours on this. It's frustrating.
Below is the full idea.log:
2019-04-10 13:53:58,200 [3956389] INFO - ution.rmi.RemoteProcessSupport - Port/ID: 45554/RemoteDriverImpl3db749c2
2019-04-10 13:53:58,224 [3956413] INFO - urce.DatabaseConnectionManager - Connecting as: postgres
2019-04-10 13:53:58,224 [3956413] INFO - urce.DatabaseConnectionManager - Connecting to: jdbc:postgresql://106.15.190.104:5432/postgres
2019-04-10 13:53:58,228 [3956417] INFO - ution.rmi.RemoteProcessSupport -
2019-04-10 13:53:59,964 [3958153] INFO - g.FileBasedIndexProjectHandler - Reindexing refreshed files: 372 to update, calculated in 44ms
2019-04-10 13:54:00,135 [3958324] INFO - .diagnostic.PerformanceWatcher - Reindexing refreshed files took 171ms; general responsiveness: ok; EDT responsiveness: ok
2019-04-10 13:54:00,745 [3958934] INFO - mponents.impl.stores.StoreUtil - saveProjectsAndApp took 49 ms
2019-04-10 13:54:26,475 [3984664] INFO - mponents.impl.stores.StoreUtil - saveProjectsAndApp took 36 ms
2019-04-10 13:55:00,583 [4018772] INFO - pi.util.registry.RegistryValue - Registry value 'ide.mac.allowDarkWindowDecorations' has changed to 'true'
2019-04-10 13:55:00,597 [4018786] INFO - tellij.openapi.util.IconLoader - replace '/general/notificationError.svg' with '/icons/general/notificationError.svg'
2019-04-10 13:59:23,296 [4281485] INFO - urce.DatabaseConnectionManager - Connecting as: postgres
2019-04-10 13:59:23,296 [4281485] INFO - urce.DatabaseConnectionManager - Connecting to: jdbc:postgresql://106.15.190.104:5432/postgres
2019-04-10 13:59:23,458 [4281647] INFO - mponents.impl.stores.StoreUtil - saveProjectsAndApp took 39 ms
2019-04-10 13:59:28,092 [4286281] INFO - mponents.impl.stores.StoreUtil - saveProjectsAndApp took 43 ms
2019-04-10 13:59:58,251 [4316440] INFO - pi.util.registry.RegistryValue - Registry value 'ide.mac.allowDarkWindowDecorations' has changed to 'true'
2019-04-10 14:00:01,087 [4319276] INFO - pi.util.registry.RegistryValue - Registry value 'ide.mac.allowDarkWindowDecorations' has changed to 'true'
2019-04-10 14:00:05,540 [4323729] INFO - ution.rmi.RemoteProcessSupport - "/Applications/IntelliJ IDEA.app/Contents/jdk/Contents/Home/jre/bin/java" -Djava.net.preferIPv4Stack=true -Djava.rmi.server.hostname=127.0.0.1 -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath "/Applications/IntelliJ IDEA.app/Contents/lib/util.jar:/Applications/IntelliJ IDEA.app/Contents/lib/trove4j.jar:/Applications/IntelliJ IDEA.app/Contents/lib/groovy-all-2.4.15.jar:/Applications/IntelliJ IDEA.app/Contents/plugins/DatabaseTools/lib/jdbc-console.jar:/Applications/IntelliJ IDEA.app/Contents/plugins/DatabaseTools/lib/dekaf-single-2.0.0.387.jar:/Users/kevin/Library/Preferences/IntelliJIdea2019.1/jdbc-drivers/PostgreSQL/42.2.5/postgresql-42.2.5.jar" com.intellij.database.remote.RemoteJdbcServer org.postgresql.Driver
2019-04-10 14:00:05,745 [4323934] INFO - ution.rmi.RemoteProcessSupport - Port/ID: 42900/RemoteDriverImpla5fe3908
2019-04-10 14:00:05,767 [4323956] INFO - urce.DatabaseConnectionManager - No user name provided (may be in url)
2019-04-10 14:00:05,767 [4323956] INFO - urce.DatabaseConnectionManager - Connecting to: jdbc:postgresql://localhost:5432/postgres
2019-04-10 14:00:05,806 [4323995] INFO - pi.util.registry.RegistryValue - Registry value 'ide.mac.allowDarkWindowDecorations' has changed to 'true'
2019-04-10 14:00:15,474 [4333663] INFO - ution.rmi.RemoteProcessSupport - "/Applications/IntelliJ IDEA.app/Contents/jdk/Contents/Home/jre/bin/java" -Djava.net.preferIPv4Stack=true -Djava.rmi.server.hostname=127.0.0.1 -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath "/Applications/IntelliJ IDEA.app/Contents/lib/util.jar:/Applications/IntelliJ IDEA.app/Contents/lib/trove4j.jar:/Applications/IntelliJ IDEA.app/Contents/lib/groovy-all-2.4.15.jar:/Applications/IntelliJ IDEA.app/Contents/plugins/DatabaseTools/lib/jdbc-console.jar:/Applications/IntelliJ IDEA.app/Contents/plugins/DatabaseTools/lib/dekaf-single-2.0.0.387.jar:/Users/kevin/Library/Preferences/IntelliJIdea2019.1/jdbc-drivers/PostgreSQL/42.2.5/postgresql-42.2.5.jar" com.intellij.database.remote.RemoteJdbcServer org.postgresql.Driver
2019-04-10 14:00:15,690 [4333879] INFO - ution.rmi.RemoteProcessSupport - Port/ID: 45818/RemoteDriverImpl78e32888
2019-04-10 14:00:15,713 [4333902] INFO - urce.DatabaseConnectionManager - Connecting as: postgre
2019-04-10 14:00:15,714 [4333903] INFO - urce.DatabaseConnectionManager - Connecting to: jdbc:postgresql://localhost:5432/postgres
2019-04-10 14:00:15,747 [4333936] WARN - urce.DatabaseConnectionManager - Connecting to: jdbc:postgresql://localhost:5432/postgres
2019-04-10 14:00:15,747 [4333936] WARN - urce.DatabaseConnectionManager - [08001] Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
2019-04-10 14:00:15,748 [4333937] WARN - lij.database.util.ErrorHandler - Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
java.sql.SQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:280)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
at org.postgresql.Driver.makeConnection(Driver.java:454)
at org.postgresql.Driver.connect(Driver.java:256)
at com.intellij.database.remote.jdbc.impl.RemoteDriverImpl.connect(RemoteDriverImpl.java:41)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283)
at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:227)
at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:179)
at com.sun.proxy.$Proxy157.connect(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.execution.rmi.RemoteUtil.invokeRemote(RemoteUtil.java:175)
at com.intellij.execution.rmi.RemoteUtil.access$200(RemoteUtil.java:38)
at com.intellij.execution.rmi.RemoteUtil$RemoteInvocationHandler.invoke(RemoteUtil.java:271)
at com.sun.proxy.$Proxy158.connect(Unknown Source)
at com.intellij.database.dataSource.DatabaseConnectionEstablisher.connect(DatabaseConnectionEstablisher.java:199)
at com.intellij.database.dataSource.DatabaseConnectionEstablisher.tryConnect(DatabaseConnectionEstablisher.java:140)
at com.intellij.database.dataSource.DatabaseConnectionEstablisher.lambda$establishConnection$1(DatabaseConnectionEstablisher.java:90)
at com.intellij.database.dataSource.AsyncUtil.lambda$null$4(AsyncUtil.java:53)
at com.intellij.database.dataSource.AsyncUtil.lambda$underProgress$14(AsyncUtil.java:140)
at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:586)
at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:532)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:86)
at com.intellij.database.dataSource.AsyncUtil.underProgress(AsyncUtil.java:146)
at com.intellij.database.dataSource.AsyncUtil.underProgress(AsyncUtil.java:140)
at com.intellij.database.dataSource.AsyncUtil.lambda$captureIndicator$5(AsyncUtil.java:53)
at com.intellij.database.dataSource.AsyncUtil.lambda$null$11(AsyncUtil.java:109)
at com.intellij.database.dataSource.AsyncUtil.lambda$static$2(AsyncUtil.java:37)
at com.intellij.database.dataSource.AsyncUtil.lambda$thenComposeAsync$12(AsyncUtil.java:107)
at com.intellij.database.dataSource.AsyncUtil.lambda$processWhenComplete$13(AsyncUtil.java:125)
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at com.intellij.database.dataSource.AsyncUtil.lambda$processWhenComplete$13(AsyncUtil.java:125)
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:778)
at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2140)
at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:110)
at com.intellij.database.dataSource.AsyncUtil.processWhenComplete(AsyncUtil.java:121)
at com.intellij.database.dataSource.AsyncUtil.lambda$null$11(AsyncUtil.java:109)
at com.intellij.openapi.application.impl.ApplicationImpl$1.run(ApplicationImpl.java:311)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.postgresql.core.PGStream.<init>(PGStream.java:70)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:91)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:192)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
at org.postgresql.Driver.makeConnection(Driver.java:454)
at org.postgresql.Driver.connect(Driver.java:256)
at com.intellij.database.remote.jdbc.impl.RemoteDriverImpl.connect(RemoteDriverImpl.java:41)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
... 3 more
Related
2023-01-13 11:02:36.291 INFO 1 --- [ main]
org.mongodb.driver.cluster: Cluster created with settings {hosts=mongo.apps.lab.pacosta.com:27017], mode=SINGLE, requiredClusterType=UNKNOWN, derverSelectionTimeout='30000 ms'}
2023-01-13 11:02:36.429 INFO 1 --- [costa.com:27017] org.mongodb.driver.cluster: Exception in monitor thread
while connecting to server mongo.apps.lab.pacosta.com:27017
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:70) ~[mongodb-driver-core-4.1.1.jar!/:na]
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:143) ~[mongodb-driver-core-4.1.1.jar!/:na]
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:188) ~[mongodb-driver-core-4.1.1.jar!/:na]
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:144) ~[mongodb-driver-core-4.1.1.jar!/:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212] Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_212]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_212]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_212]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_212]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_212]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_212]`
Here are screenshots of OCP routes and environment variables.
We are doing the same thing in our spring boot service using OCP version 4.11 but instead of using service ip we are taking it as a domain in mongo uri which is unable to create the connection
I get "connection refused error" when I try to write the results of a Dataframe to an RDS (MySQL). I am using PySpark 3 on EMR cluster v6.x (1 master node, 1 slave node). The table does not exist yet. But the data base exist.
spark-submit --jars s3://{some s3 folder}/mysql-connector-java-8.0.25.jar s3://{some s3 folder}/pyspark_script.py
The part of the script that writes to mysql is here (after testing, its the only part of the script that delivers error/is not working): * I have changed the name of my db, user, and password here below
df.write\
.mode("overwrite")\
.format("jdbc")\
.option("url","jdbc:mysql://localhost:3306/{my database name}?useSSL=false")\
.option("driver","com.mysql.cj.jdbc.Driver")\
.option("dbtable","mydb_table")\
.option("user","myuser")\
.option("password","mypassword")\
.save()
This is the error I get: It is about connection refused.
I have already given the EMR Role access RDS and its data!
Traceback (most recent call last):
File "/mnt/tmp/spark-93919f38-ea4d-44d6-be7d-0416be972753/pyspark_script.py", line 57, in <module>
.option("password","assignment")\
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 1107, in save
File "/usr/lib/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py", line 1305, in __call__
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 111, in deco
File "/usr/lib/spark/python/lib/py4j-0.10.9-src.zip/py4j/protocol.py", line 328, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o163.save.
: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:833)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:453)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:246)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198)
at org.apache.spark.sql.execution.datasources.jdbc.connection.BasicConnectionProvider.getConnection(BasicConnectionProvider.scala:49)
at org.apache.spark.sql.execution.datasources.jdbc.connection.ConnectionProvider$.create(ConnectionProvider.scala:68)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.$anonfun$createConnectionFactory$1(JdbcUtils.scala:62)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:48)
at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:46)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:90)
at org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:194)
at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:232)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:229)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:190)
at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:134)
at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:133)
at org.apache.spark.sql.DataFrameWriter.$anonfun$runCommand$1(DataFrameWriter.scala:989)
at org.apache.spark.sql.catalyst.QueryPlanningTracker$.withTracker(QueryPlanningTracker.scala:107)
at org.apache.spark.sql.execution.SQLExecution$.withTracker(SQLExecution.scala:232)
at org.apache.spark.sql.execution.SQLExecution$.executeQuery$1(SQLExecution.scala:110)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:135)
at org.apache.spark.sql.catalyst.QueryPlanningTracker$.withTracker(QueryPlanningTracker.scala:107)
at org.apache.spark.sql.execution.SQLExecution$.withTracker(SQLExecution.scala:232)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:135)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:253)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:134)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:68)
at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:989)
at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:438)
at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:415)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:301)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:89)
at com.mysql.cj.NativeSession.connect(NativeSession.java:144)
at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:953)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:823)
... 45 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:607)
at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:155)
at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:63)
... 48 more
21/12/19 11:40:04 INFO SparkContext: Invoking stop() from shutdown hook
21/12/19 11:40:04 INFO AbstractConnector: Stopped Spark#74d96709{HTTP/1.1, (http/1.1)}{0.0.0.0:4040}
21/12/19 11:40:04 INFO SparkUI: Stopped Spark web UI at http://{ip}.eu-central-1.compute.internal:4040
21/12/19 11:40:04 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
21/12/19 11:40:04 INFO MemoryStore: MemoryStore cleared
21/12/19 11:40:04 INFO BlockManager: BlockManager stopped
21/12/19 11:40:04 INFO BlockManagerMaster: BlockManagerMaster stopped
21/12/19 11:40:04 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
21/12/19 11:40:04 INFO SparkContext: Successfully stopped SparkContext
21/12/19 11:40:04 INFO ShutdownHookManager: Shutdown hook called
21/12/19 11:40:04 INFO ShutdownHookManager: Deleting directory /mnt/tmp/spark-fd1b8e7c-7b4c-424d-a451-743a6e075fbd
21/12/19 11:40:04 INFO ShutdownHookManager: Deleting directory /mnt/tmp/spark-93919f38-ea4d-44d6-be7d-0416be972753
21/12/19 11:40:04 INFO ShutdownHookManager: Deleting directory /mnt/tmp/spark-fd1b8e7c-7b4c-424d-a451-743a6e075fbd/pyspark-40fbaaf5-2e34-44ba-875f-88308084546d
Try this thread: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
EMR with localhost RDS? seems odd, did you missed set the IP correctly ?
The jar file can be run successfully with java, but the image is run incorrectly.
docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
backend latest bad78d214712 16 minutes ago 735MB
command :docker run -p 8000:8000 backend
a springboot jar,mysql 5.7
Dockerfile and jar package in the same directory
dockfile
FROM aurorasystem/openjdk-zeromq:latest
MAINTAINER Aurora System <it#aurora-system.com>
VOLUME /tmp
COPY backend-2.0.0.jar app.jar
RUN bash -c "touch /app.jar"
EXPOSE 8888
ENTRYPOINT ["java","-jar","/app.jar"]
error
19-06-06 08:19:37.173 [main] WARN : c.z.hikari.util.DriverDataSource.<init> - Registered driver with driverClassName=com.mysql.jdbc.Driver was not found, trying direct instantiation.
19-06-06 08:19:38.380 [main] ERROR : com.zaxxer.hikari.pool.HikariPool.throwPoolInitializationException - HikariPool-1 - Exception during pool initialization.
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:835)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:455)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:240)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:207)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:136)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:369)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:198)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:467)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:541)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:302)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1804)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1741)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:576)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:498)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:307)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1083)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:853)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:546)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:142)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:316)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248)
at com.aurora.backend.Application.main(Application.java:19)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:91)
at com.mysql.cj.NativeSession.connect(NativeSession.java:152)
at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:955)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:825)
... 39 common frames omitted
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:155)
at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:65)
... 42 common frames omitted
19-06-06 08:19:38.382 [main] WARN : o.s.b.w.s.c.AnnotationConfigServletWebServerApplicationContext.refresh - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [org/springframework/boot/autoconfigure/liquibase/LiquibaseAutoConfiguration$LiquibaseConfiguration.class]: Invocation of init method failed; nested exception is liquibase.exception.DatabaseException: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
19-06-06 08:19:38.407 [main] INFO : o.a.catalina.core.StandardService.log - Stopping service [Tomcat]
19-06-06 08:19:38.434 [main] INFO : o.s.b.a.l.ConditionEvaluationReportLoggingListener.logMessage -
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
19-06-06 08:19:38.439 [main] ERROR : o.s.boot.SpringApplication.reportFailure - Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [org/springframework/boot/autoconfigure/liquibase/LiquibaseAutoConfiguration$LiquibaseConfiguration.class]: Invocation of init method failed; nested exception is liquibase.exception.DatabaseException: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1745)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:576)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:498)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:307)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1083)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:853)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:546)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:142)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:316)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248)
at com.aurora.backend.Application.main(Application.java:19)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
Caused by: liquibase.exception.DatabaseException: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:307)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1804)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1741)
... 26 common frames omitted
Caused by: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:835)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:455)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:240)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:207)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:136)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:369)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:198)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:467)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:541)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:302)
... 28 common frames omitted
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:91)
at com.mysql.cj.NativeSession.connect(NativeSession.java:152)
at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:955)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:825)
... 39 common frames omitted
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:155)
at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:65)
... 42 common frames omitted
Seems you connect to you mysql database on you local machine, and put the host name as local host. Please try docker run --net=host -p 8000:8000 backend to use host network, or use the ip address directly.
I have set 2 mongo container which having 27017:27017 and 27018:27017 as port. mongo1 is set as primary while mongo2 set as secondary and used replica set.
mongo1 as primary has both write, read access and mongo2 has only read access
spring.data.mongodb.uri = mongodb://abc.xyz:27017/user_demo
//monog1, abc.xyz is the hostname
spring.data.mongodb.uri = mongodb://abc.xyz:27018/user_demo
//mongo2
If I connect only monogo1 in my normal spring boot program, it works(read, write) and if I set mongo2 it works(only read), but when I set both in one line it gives error
spring.data.mongodb.uri = mongodb://abc.xyz:27017,abc.xyz:27018/user_demo?replicaSet=idea-mongo-set
I checked replica set is working by storing in mongo1, in mongo2 I can read
So, what is wrong with the last line. Is there any other way to store uri for replica set
In official spring site it is given
spring.data.mongodb.uri=mongodb://user:secret#mongo1.example.com:12345,mongo2.example.com:23456/test
Error I am getting is this
2019-01-08 15:56:35.245 INFO 29804 --- [ main] c.j.bootifulmongodb.BootMongoDBApp : Started BootMongoDBApp in 3.603 seconds (JVM running for 4.373)
2019-01-08 15:56:54.466 INFO 29804 --- [azure.com:27018] org.mongodb.driver.cluster : Exception in monitor thread while connecting to server abc.xyz:27018
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.connection.SocketStream.open(SocketStream.java:63) ~[mongodb-driver-core-3.4.3.jar:na]
at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:115) ~[mongodb-driver-core-3.4.3.jar:na]
at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:113) ~[mongodb-driver-core-3.4.3.jar:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[na:1.8.0_181]
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85) ~[na:1.8.0_181]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_181]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_181]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_181]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[na:1.8.0_181]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_181]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_181]
at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:57) ~[mongodb-driver-core-3.4.3.jar:na]
at com.mongodb.connection.SocketStream.open(SocketStream.java:58) ~[mongodb-driver-core-3.4.3.jar:na]
... 3 common frames omitted
It looks like your uri is on the right track for connecting to a replica set.
spring.data.mongodb.uri = mongodb://abc.xyz:27017,abc.xyz:27018/user_demo?replicaSet=idea-mongo-set
Is it just a typo that credentials (user, password) are omitted here?
This exact syntax is what we use for our replica set connection uri:
mongodb://user:pwd#server1:27017,server2:27017,server3:27017/our_collection?replicaSet=replica_set_name&authSource=authagainstthiscollection&authMechanism=SCRAM-SHA-1
Because you set abc.xyz:27018 hostname in mongo rs.initiate, so client will try to access this hostname directly. But this hostname is unreachable for client, it's just a FQDN in your azure cluster.
I think this document will help you clients use the hostnames listed in the replica set config not the seed list
I have very basic storm topology which contains 1 unreliable spout + 1 bolt. Spout is fed by Kafka broker via zookeeper. The Spout sends tuples to a bolt witch in its role do nothing ( has empty execute method).
The topology run in local cluster mode and it crashes after a while from run.
I tried to increase the app memory by -Xmx parameter many times, but the problem still appears.
please help!
Here is the code snippet that build the topology:
builder.setSpout("kafka_forwarding_spout", new KafkaForwardingSpout(),1);
builder.setBolt("fake_bolt", new FakeBolt(),1).localOrShuffleGrouping("kafka_forwarding_spout");
Here is a snippet from storm log:
1318482 [Thread-37-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
1319574 [CuratorFramework-9] INFO com.netflix.curator.framework.state.ConnectionStateManager - State change: LOST
1312307 [Thread-39-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
1307441 [Thread-25-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
1227216 [Thread-29-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
1191153 [Thread-31-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
1771836 [Thread-39-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
1688630 [ConnectionStateManager-0] WARN com.netflix.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered.
1650422 [Thread-35-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
1635500 [Thread-19-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
1616369 [CuratorFramework-11] ERROR com.netflix.curator.framework.imps.CuratorFrameworkImpl - Background exception was not retry-able or retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
at com.netflix.curator.ConnectionState.getZooKeeper(ConnectionState.java:72) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:74) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:353) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.BackgroundSyncImpl.performBackgroundOperation(BackgroundSyncImpl.java:39) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.OperationAndData.callPerformBackgroundOperation(OperationAndData.java:40) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:547) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl.access$200(CuratorFrameworkImpl.java:50) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl$2.call(CuratorFrameworkImpl.java:177) [curator-framework-1.0.1.jar:na]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_05]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_05]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_05]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_05]
1617025 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2001] WARN org.apache.zookeeper.server.NIOServerCnxn - EndOfStreamException: Unable to read additional data from client sessionid 0x149dd7eb6a2000b, likely client has closed socket
1600212 [Thread-23-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
1584086 [Thread-21-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
1570224 [Thread-9] INFO backtype.storm.daemon.worker - Shutting down receive thread
1569786 [ConnectionStateManager-0] WARN com.netflix.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered.
2158302 [Thread-5] ERROR backtype.storm.event - Error when processing event
java.lang.RuntimeException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
at backtype.storm.util$wrap_in_runtime.invoke(util.clj:44) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at backtype.storm.zookeeper$get_children.invoke(zookeeper.clj:133) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at backtype.storm.cluster$mk_distributed_cluster_state$reify__2385.get_children(cluster.clj:101) ~[na:na]
at backtype.storm.cluster$mk_storm_cluster_state$reify__2814.assignments(cluster.clj:243) ~[na:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_05]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_05]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_05]
at java.lang.reflect.Method.invoke(Method.java:483) ~[na:1.8.0_05]
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) ~[clojure-1.4.0.jar:na]
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.supervisor$assignments_snapshot.invoke(supervisor.clj:39) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at backtype.storm.daemon.supervisor$mk_synchronize_supervisor$this__6251.invoke(supervisor.clj:298) ~[na:na]
at backtype.storm.event$event_manager$fn__2921.invoke(event.clj:39) ~[na:na]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_05]
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
at com.netflix.curator.ConnectionState.getZooKeeper(ConnectionState.java:72) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:74) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:353) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:180) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:173) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.RetryLoop.callWithRetry(RetryLoop.java:85) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:169) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:161) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:36) ~[curator-framework-1.0.1.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_05]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_05]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_05]
at java.lang.reflect.Method.invoke(Method.java:483) ~[na:1.8.0_05]
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) ~[clojure-1.4.0.jar:na]
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) ~[clojure-1.4.0.jar:na]
at backtype.storm.zookeeper$get_children.invoke(zookeeper.clj:131) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
... 13 common frames omitted
2117747 [Thread-4] ERROR com.netflix.curator.ConnectionState - Connection timed out
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
at com.netflix.curator.ConnectionState.getZooKeeper(ConnectionState.java:72) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:74) [curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:353) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:149) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:138) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.RetryLoop.callWithRetry(RetryLoop.java:85) [curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:134) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:125) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:34) [curator-framework-1.0.1.jar:na]
at backtype.storm.zookeeper$exists_node_QMARK_$fn__2170.invoke(zookeeper.clj:96) [na:0.9.1-incubating]
at backtype.storm.zookeeper$exists_node_QMARK_.invoke(zookeeper.clj:93) [na:0.9.1-incubating]
at backtype.storm.zookeeper$mkdirs.invoke(zookeeper.clj:107) [na:0.9.1-incubating]
at backtype.storm.cluster$mk_distributed_cluster_state$reify__2385.set_ephemeral_node(cluster.clj:69) [na:0.9.1-incubating]
at backtype.storm.cluster$mk_storm_cluster_state$reify__2814.supervisor_heartbeat_BANG_(cluster.clj:315) [na:0.9.1-incubating]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_05]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_05]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_05]
at java.lang.reflect.Method.invoke(Method.java:483) ~[na:1.8.0_05]
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) [clojure-1.4.0.jar:na]
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) [clojure-1.4.0.jar:na]
at backtype.storm.daemon.supervisor$eval6298$exec_fn__1459__auto____6299$heartbeat_fn__6301.invoke(supervisor.clj:378) [na:0.9.1-incubating]
at backtype.storm.timer$schedule_recurring$this__2141.invoke(timer.clj:92) [na:0.9.1-incubating]
at backtype.storm.timer$mk_timer$fn__2124$fn__2125.invoke(timer.clj:48) [na:0.9.1-incubating]
at backtype.storm.timer$mk_timer$fn__2124.invoke(timer.clj:41) [na:0.9.1-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_05]
2078727 [Thread-16] ERROR com.netflix.curator.ConnectionState - Connection timed out
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
at com.netflix.curator.ConnectionState.getZooKeeper(ConnectionState.java:72) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:74) [curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:353) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:149) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:138) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.RetryLoop.callWithRetry(RetryLoop.java:85) [curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:134) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:125) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:34) [curator-framework-1.0.1.jar:na]
at backtype.storm.zookeeper$exists_node_QMARK_$fn__2170.invoke(zookeeper.clj:96) [na:0.9.1-incubating]
at backtype.storm.zookeeper$exists_node_QMARK_.invoke(zookeeper.clj:93) [na:0.9.1-incubating]
at backtype.storm.zookeeper$exists.invoke(zookeeper.clj:141) [na:0.9.1-incubating]
at backtype.storm.cluster$mk_distributed_cluster_state$reify__2385.set_data(cluster.clj:85) [na:0.9.1-incubating]
at backtype.storm.cluster$mk_storm_cluster_state$reify__2814.worker_heartbeat_BANG_(cluster.clj:291) [na:0.9.1-incubating]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_05]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_05]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_05]
at java.lang.reflect.Method.invoke(Method.java:483) ~[na:1.8.0_05]
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) [clojure-1.4.0.jar:na]
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) [clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker$do_executor_heartbeats.doInvoke(worker.clj:53) [na:0.9.1-incubating]
at clojure.lang.RestFn.invoke(RestFn.java:439) [clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker$eval5665$exec_fn__1459__auto____5666$fn__5669.invoke(worker.clj:367) [na:0.9.1-incubating]
at backtype.storm.timer$schedule_recurring$this__2141.invoke(timer.clj:92) [na:0.9.1-incubating]
at backtype.storm.timer$mk_timer$fn__2124$fn__2125.invoke(timer.clj:48) [na:0.9.1-incubating]
at backtype.storm.timer$mk_timer$fn__2124.invoke(timer.clj:41) [na:0.9.1-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_05]
2079838 [CuratorFramework-11] INFO com.netflix.curator.framework.state.ConnectionStateManager - State change: LOST
2052508 [Thread-37-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
1948998 [yassin-long-d_localhost.localdomain-1416760311632-aa3d627e_watcher_executor] INFO kafka.consumer.ZookeeperConsumerConnector - [yassin-long-d_localhost.localdomain-1416760311632-aa3d627e], exception during rebalance
java.lang.NullPointerException: null
at org.I0Itec.zkclient.ZkConnection.readData(ZkConnection.java:103) ~[zkclient-0.3.jar:0.3]
at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:770) ~[zkclient-0.3.jar:0.3]
at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:766) ~[zkclient-0.3.jar:0.3]
at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) ~[zkclient-0.3.jar:0.3]
at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:766) ~[zkclient-0.3.jar:0.3]
at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:761) ~[zkclient-0.3.jar:0.3]
at kafka.utils.ZkUtils$.readData(ZkUtils.scala:461) ~[kafka_2.10-0.8.1.jar:na]
at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:505) ~[kafka_2.10-0.8.1.jar:na]
at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:504) ~[kafka_2.10-0.8.1.jar:na]
at scala.collection.Iterator$class.foreach(Iterator.scala:727) ~[scala-library-2.10.1.jar:na]
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) ~[scala-library-2.10.1.jar:na]
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) ~[scala-library-2.10.1.jar:na]
at scala.collection.AbstractIterable.foreach(Iterable.scala:54) ~[scala-library-2.10.1.jar:na]
at kafka.utils.ZkUtils$.getCluster(ZkUtils.scala:504) ~[kafka_2.10-0.8.1.jar:na]
at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:407) ~[kafka_2.10-0.8.1.jar:na]
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141) [scala-library-2.10.1.jar:na]
at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:402) [kafka_2.10-0.8.1.jar:na]
at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run(ZookeeperConsumerConnector.scala:355) [kafka_2.10-0.8.1.jar:na]
1872056 [Thread-39-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
1853312 [Thread-35-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
1840380 [Thread-31-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
1808599 [Thread-41-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
1799622 [Thread-25-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
2287292 [Thread-41-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
2287075 [Thread-37-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
2286862 [yassin-long-d_localhost.localdomain-1416760311632-aa3d627e_watcher_executor] INFO kafka.consumer.ZookeeperConsumerConnector - [yassin-long-d_localhost.localdomain-1416760311632-aa3d627e], end rebalancing consumer yassin-long-d_localhost.localdomain-1416760311632-aa3d627e try #0
2280400 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2001] WARN org.apache.zookeeper.server.NIOServerCnxn - EndOfStreamException: Unable to read additional data from client sessionid 0x149dd7eb6a20005, likely client has closed socket
2280180 [ConnectionStateManager-0] WARN com.netflix.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered.
2267345 [Thread-9] INFO backtype.storm.messaging.loader - Shutting down receiving-thread: [events_topology-1-1416760309, 1027]
2242225 [Thread-19-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
2224569 [Thread-29-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
2226559 [Thread-21-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
2209276 [Thread-23-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
2296607 [Thread-41-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
2293718 [Thread-37-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
2292830 [Thread-25-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
2498572 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2001] WARN org.apache.zookeeper.server.NIOServerCnxn - EndOfStreamException: Unable to read additional data from client sessionid 0x149dd7eb6a20003, likely client has closed socket
2540517 [Thread-41-kafka_forwarding_spout-EventThread] ERROR org.apache.zookeeper.ClientCnxn - Error while calling watcher
java.lang.OutOfMemoryError: Java heap space
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main-SendThread(localhost:2001)"
2637439 [Thread-35-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
2647739 [Thread-39-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
2672605 [main-SendThread(localhost:2001)] ERROR org.apache.zookeeper.ClientCnxn - from main-SendThread(localhost:2001)
java.lang.OutOfMemoryError: Java heap space
2764243 [Thread-25-kafka_forwarding_spout-SendThread(zoo2:2182)] ERROR org.apache.zookeeper.ClientCnxn - from Thread-25-kafka_forwarding_spout-SendThread(zoo2:2182)
java.lang.OutOfMemoryError: Java heap space
2720032 [main-SendThread(localhost:2001)] ERROR org.apache.zookeeper.ClientCnxn - from main-SendThread(localhost:2001)
java.lang.OutOfMemoryError: Java heap space
2731157 [Thread-8] ERROR org.apache.zookeeper.server.NIOServerCnxn - Thread Thread[Thread-8,5,main] died
java.lang.OutOfMemoryError: Java heap space
2800693 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2001] ERROR org.apache.zookeeper.server.NIOServerCnxn - Thread Thread[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2001,5,main] died
Seems like a java heap size problem. Check there are many other configurable memory related parameters.
This issue is being caused due to the Java Virtual Machine running out of available Heap memory in the system. Depending on on which underlying operating system that you are using, you can use the -Xmx parameter followed by the amount of memory you want to allocate. For example, if you want to allocate 2GB of memory then you need to mention -Xms2048m and the maximum heap size to be set as -Xmx2048m. However, remember that increasing the heap size after getting an OutofMemoryError is generally a short-term solution and it is highly probable to occur again in the future. It is generally an indicator for optimising your code to be memory efficient.