How to enable automatic retries for Mongo Socket exception? - mongodb

I'm running a spring boot gradle project. Spring boot version is 2.3.2.RELEASE, MongoDB version is 4.0.5.
Per below site, retryable writes are enabled by default from MongoDB 3.6+ https://docs.mongodb.com/manual/core/retryable-writes/
I saw the following exceptions in the project, MongoReadException and MongoWriteException while reading/saving to the mongo database. I would like to know if these exceptions are automatically retried starting Spring boot 2.3.2.RELEASE.
Below are the exceptions I wanted to retry,
Caused by: org.springframework.data.mongodb.UncategorizedMongoDbException: Exception sending message; nested exception is com.mongodb.MongoSocketWriteException: Exception sending message
Caused by: org.springframework.data.mongodb.UncategorizedMongoDbException: Prematurely reached end of stream; nested exception is com.mongodb.MongoSocketReadException: Prematurely reached end of stream
To try it out, I created a spring boot demo app in my local that tries to save a document to a database whose uri is provided with invalid value in the properties file. I get MongoTimeoutException on save document which is as expected. However, I don't see any retries on the save operation.
I also tried to add the property retryWrites=true to the connection uri, it didn't work
spring.data.mongodb.uri=mongodb://XXX:XXXX#invalid-server:5555/dummyDB?&retrywrites=true
Does anyone know how to enable the retries ? How do I test it locally ?
Please help. Thanks in advance.
Below is the exception stacktrace with MongoSocketOpenException. I would like to test if this exception is auto retried, however, it's not.
nested exception is org.springframework.dao.DataAccessResourceFailureException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused}}]; nested exception is com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused}}]] with root cause
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused}}]
at com.mongodb.internal.connection.BaseCluster.getDescription(BaseCluster.java:177) ~[mongodb-driver-core-4.0.5.jar:na]
at com.mongodb.internal.connection.SingleServerCluster.getDescription(SingleServerCluster.java:41) ~[mongodb-driver-core-4.0.5.jar:na]
at com.mongodb.client.internal.MongoClientDelegate.getConnectedClusterDescription(MongoClientDelegate.java:147) ~[mongodb-driver-sync-4.0.5.jar:na]
at com.mongodb.client.internal.MongoClientDelegate.createClientSession(MongoClientDelegate.java:98) ~[mongodb-driver-sync-4.0.5.jar:na]
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.getClientSession(MongoClientDelegate.java:278) ~[mongodb-driver-sync-4.0.5.jar:na]
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:202) ~[mongodb-driver-sync-4.0.5.jar:na]
at com.mongodb.client.internal.MongoCollectionImpl.executeSingleWriteRequest(MongoCollectionImpl.java:1008) ~[mongodb-driver-sync-4.0.5.jar:na]
at com.mongodb.client.internal.MongoCollectionImpl.executeInsertOne(MongoCollectionImpl.java:469) ~[mongodb-driver-sync-4.0.5.jar:na]
at com.mongodb.client.internal.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:452) ~[mongodb-driver-sync-4.0.5.jar:na]
at com.mongodb.client.internal.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:446) ~[mongodb-driver-sync-4.0.5.jar:na]
at org.springframework.data.mongodb.core.MongoTemplate.lambda$insertDocument$15(MongoTemplate.java:1442) ~[spring-data-mongodb-3.0.2.RELEASE.jar:3.0.2.RELEASE]
at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:566) ~[spring-data-mongodb-3.0.2.RELEASE.jar:3.0.2.RELEASE]
at org.springframework.data.mongodb.core.MongoTemplate.insertDocument(MongoTemplate.java:1436) ~[spring-data-mongodb-3.0.2.RELEASE.jar:3.0.2.RELEASE]
at org.springframework.data.mongodb.core.MongoTemplate.doInsert(MongoTemplate.java:1236) ~[spring-data-mongodb-3.0.2.RELEASE.jar:3.0.2.RELEASE]
at org.springframework.data.mongodb.core.MongoTemplate.insert(MongoTemplate.java:1168) ~[spring-data-mongodb-3.0.2.RELEASE.jar:3.0.2.RELEASE]
at org.springframework.data.mongodb.repository.support.SimpleMongoRepository.save(SimpleMongoRepository.java:85) ~[spring-data-mongodb-3.0.2.RELEASE.jar:3.0.2.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_101]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_101]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_101]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_101]
at org.springframework.data.repository.core.support.ImplementationInvocationMetadata.invoke(ImplementationInvocationMetadata.java:72) ~[spring-data-commons-2.3.2.RELEASE.jar:2.3.2.RELEASE]
at org.springframework.data.repository.core.support.RepositoryComposition$RepositoryFragments.invoke(RepositoryComposition.java:382) ~[spring-data-commons-2.3.2.RELEASE.jar:2.3.2.RELEASE]
at org.springframework.data.repository.core.support.RepositoryComposition.invoke(RepositoryComposition.java:205) ~[spring-data-commons-2.3.2.RELEASE.jar:2.3.2.RELEASE]
at org.springframework.data.repository.core.support.RepositoryFactorySupport$ImplementationMethodExecutionInterceptor.invoke(RepositoryFactorySupport.java:549) ~[spring-data-commons-2.3.2.RELEASE.jar:2.3.2.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.8.RELEASE.jar:5.2.8.RELEASE]
at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java:155) ~[spring-data-commons-2.3.2.RELEASE.jar:2.3.2.RELEASE]
at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:130) ~[spring-data-commons-2.3.2.RELEASE.jar:2.3.2.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.8.RELEASE.jar:5.2.8.RELEASE]
at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:80) ~[spring-data-commons-2.3.2.RELEASE.jar:2.3.2.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.8.RELEASE.jar:5.2.8.RELEASE]
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) ~[spring-aop-5.2.8.RELEASE.jar:5.2.8.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.8.RELEASE.jar:5.2.8.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.8.RELEASE.jar:5.2.8.RELEASE]
at com.sun.proxy.$Proxy52.save(Unknown Source) ~[na:na]

Retries apply to operations like reads and writes.
Operations are only sent once the driver successfully connects to the server(s) you indicate in connection string/client configuration.
Your server is not accessible by the client therefore that connection isn't working therefore no operations can be performed therefore the concept of operation retries doesn't apply.
The monitoring is retried automatically, once you fix the connectivity issue between the client and the server (or start the server) the client will recognize this automatically.

Related

Spring Boot Lambda Mongo Configuration Issue

I have deployed a Spring boot lambda function and got the following error upon API invocation. But it works on the local machine. I'm wondering about the missing AWS configuration
Please refer to the error.
2022-12-31T01:42:59.669-06:00 07:42:59.669 INFO 8 --- [ main] org.mongodb.driver.cluster : No server chosen by com.mongodb.client.internal.MongoClientDelegate$1#515fff35 from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=cluster0-shard-00-02.gu1ji.mongodb.net:27017, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=cluster0-shard-00-01.gu1ji.mongodb.net:27017, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=cluster0-shard-00-00.gu1ji.mongodb.net:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
2022-12-31T01:43:07.405-06:00 07:43:07.405 INFO 8 --- [.net:27017] org.mongodb.driver.cluster : Exception in monitor thread while connecting to server cluster0-shard-00-02.gu1ji.mongodb.net:27017
2022-12-31T01:43:07.405-06:00 com.mongodb.MongoSocketOpenException: Exception opening socket
2022-12-31T01:43:07.405-06:00 at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:70)
2022-12-31T01:43:07.405-06:00 at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:180)
2022-12-31T01:43:07.405-06:00 at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:193)
2022-12-31T01:43:07.405-06:00 at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:157)
2022-12-31T01:43:07.405-06:00 at java.base/java.lang.Thread.run(Unknown Source)
2022-12-31T01:43:07.405-06:00 Caused by: java.net.SocketTimeoutException: connect timed out
2022-12-31T01:43:07.405-06:00 at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
2022-12-31T01:43:07.405-06:00 at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
2022-12-31T01:43:07.405-06:00 at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
2022-12-31T01:43:07.405-06:00 at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source)
2022-12-31T01:43:07.405-06:00 at java.base/java.net.SocksSocketImpl.connect(Unknown Source)
2022-12-31T01:43:07.405-06:00 at java.base/java.net.Socket.connect(Unknown Source)
2022-12-31T01:43:07.405-06:00 at java.base/sun.security.ssl.SSLSocketImpl.connect(Unknown Source)
2022-12-31T01:43:07.405-06:00 at com.mongodb.internal.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:107)
2022-12-31T01:43:07.405-06:00 at com.mongodb.internal.connection.SocketStream.initializeSocket(SocketStream.java:79)
2022-12-31T01:43:07.405-06:00 at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:65)
2022-12-31T01:43:07.405-06:00 ... 4 common frames omitted
2022-12-31T01:43:07.405-06:00
Copy
07:43:07.405 INFO 8 --- [.net:27017] org.mongodb.driver.cluster : Exception in monitor thread while connecting to server cluster0-shard-00-01.gu1ji.mongodb.net:27017
07:43:07.405 INFO 8 --- [.net:27017] org.mongodb.driver.cluster : Exception in monitor thread while connecting to server cluster0-shard-00-01.gu1ji.mongodb.net:27017
2022-12-31T01:43:07.405-06:00 com.mongodb.MongoSocketOpenException: Exception opening socket
I have deployed my lambda within the VPC without proper configurations. VPC configurations are not mandatory for the cloud function, hence I removed them.

ActiveMQ Artemis: javax.jms.IllegalStateRuntimeException: Session is closed after interruption in PostgreSQL DB connection

Recently we have upgraded ActiveMQ Artemis version from 1.5.3 to 2.21.0 and now using PostgreSQL to store the messages. Earlier we were storing the messages in file system.
ActiveMQ Artemis is used as a embedded server inside a Spring-based Java web application.
We are getting following error if connection to the PostgreSQL is temporarily interrupted:
javax.jms.IllegalStateRuntimeException: Session is closed
at org.apache.activemq.artemis.jms.client.JmsExceptionUtils.convertToRuntimeException(JmsExceptionUtils.java:59)
at org.apache.activemq.artemis.jms.client.ActiveMQJMSContext.createMapMessage(ActiveMQJMSContext.java:257)
at com..data.core.messaging.dataload.BaseDataLoaderMessageSender.getMapMessage(BaseDataLoaderMessageSender.java:220)
at com.test.data.core.messaging.dataload.BaseDataLoaderMessageSender.send(BaseDataLoaderMessageSender.java:140)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: javax.jms.IllegalStateException: Session is closed
at org.apache.activemq.artemis.jms.client.ActiveMQSession.checkClosed(ActiveMQSession.java:1255)
at org.apache.activemq.artemis.jms.client.ActiveMQSession.createMapMessage(ActiveMQSession.java:169)
at org.apache.activemq.artemis.jms.client.ActiveMQJMSContext.createMapMessage(ActiveMQJMSContext.java:255)
... 132 more
The sequeunce of events are
Start the server
Shutdown DB
Perform some action on JMS queue like send a message/read messages from queue - It will fail. This is expected
Start DB
Perform some action on JMS queue like send a message/read messages from queue
In the step 5 as the connection is restored then I expect it to be successful.
ConnectionFactory is created as a Spring bean:
<bean id="queueConnectionFactory"
class="org.apache.activemq.artemis.jms.client.ActiveMQJMSConnectionFactory">
<constructor-arg value="false"/>
<constructor-arg>
<bean class="org.apache.activemq.artemis.api.core.TransportConfiguration">
<constructor-arg
value="org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnectorFactory"/>
</bean>
</constructor-arg>
<property name="ConsumerWindowSize" value="0"/>
</bean>
And the JMSContext and MessageProducer are being created as
jmsContext = connectionFactory.createContext();
producer = jmsContext.createProducer();
Note: I tried to recreate ConnectionFactory programatically in case of exception but still the same issue.
Can anyone suggest what can be wrong here?
Edit 1:
Error log in case of DB connection is lost
[INFO ] 2022-06-24 10:28:53.353 [MessageBroker-1] WebSocketMessageBrokerStats - WebSocketSession[0 current WS(0)-HttpStream(0)-HttpPoll(0), 0 total, 0 closed abnormally (0 connect failure, 0 send limit, 0 transport error)], stompSubProtocol[processed CONNECT(0)-CONNECTED(0)-DISCONNECT(0)], stompBrokerRelay[null], inboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], outboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], sockJsScheduler[pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 0]
[ERROR] 2022-06-24 10:29:11.737 [Thread-6 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6#4d16a1c0)] JDBCConnectionProvider -
SQL EXCEPTIONS:
SQLState: 08001 ErrorCode: 0 Message: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
[WARN ] 2022-06-24 10:29:11.737 [Thread-6 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6#4d16a1c0)] JDBCJournalImpl - Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:280) ~[postgresql-42.2.5.jar:42.2.5]
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49) ~[postgresql-42.2.5.jar:42.2.5]
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195) ~[postgresql-42.2.5.jar:42.2.5]
at org.postgresql.Driver.makeConnection(Driver.java:454) ~[postgresql-42.2.5.jar:42.2.5]
at org.postgresql.Driver.connect(Driver.java:256) ~[postgresql-42.2.5.jar:42.2.5]
at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:319) ~[tomcat-jdbc.jar:?]
at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:212) ~[tomcat-jdbc.jar:?]
at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:739) ~[tomcat-jdbc.jar:?]
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:671) ~[tomcat-jdbc.jar:?]
at org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:198) ~[tomcat-jdbc.jar:?]
at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:132) ~[tomcat-jdbc.jar:?]
at jdk.internal.reflect.GeneratedMethodAccessor55.invoke(Unknown Source) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.18.jar:5.3.18]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) ~[spring-aop-5.3.18.jar:5.3.18]
at com.sun.proxy.$Proxy38.getConnection(Unknown Source) ~[?:?]
at org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource.getConnection(AbstractRoutingDataSource.java:194) ~[spring-jdbc-5.3.18.jar:5.3.18]
at org.apache.activemq.artemis.jdbc.store.drivers.JDBCConnectionProvider.getConnection(JDBCConnectionProvider.java:61) ~[artemis-jdbc-store-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:213) ~[artemis-jdbc-store-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:1049) ~[artemis-jdbc-store-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent.runForExecutor(ActiveMQScheduledComponent.java:313) ~[artemis-commons-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent.lambda$bookedRunForScheduler$2(ActiveMQScheduledComponent.java:320) ~[artemis-commons-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) ~[artemis-commons-2.21.0.jar:?]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) ~[artemis-commons-2.21.0.jar:?]
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) ~[artemis-commons-2.21.0.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.21.0.jar:?]
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.waitForConnect(Native Method) ~[?:?]
at java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:107) ~[?:?]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) ~[?:?]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) ~[?:?]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) ~[?:?]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:?]
at java.net.Socket.connect(Socket.java:609) ~[?:?]
at org.postgresql.core.PGStream.<init>(PGStream.java:70) ~[postgresql-42.2.5.jar:42.2.5]
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:91) ~[postgresql-42.2.5.jar:42.2.5]
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:192) ~[postgresql-42.2.5.jar:42.2.5]
... 28 more
[WARN ] 2022-06-24 10:29:11.833 [Thread-6 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6#4d16a1c0)] server - AMQ222010: Critical IO Error, shutting down the server. file=NULL, message=Critical IO Error. Failed to process JDBC Record statements
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:280) ~[postgresql-42.2.5.jar:42.2.5]
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49) ~[postgresql-42.2.5.jar:42.2.5]
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195) ~[postgresql-42.2.5.jar:42.2.5]
at org.postgresql.Driver.makeConnection(Driver.java:454) ~[postgresql-42.2.5.jar:42.2.5]
at org.postgresql.Driver.connect(Driver.java:256) ~[postgresql-42.2.5.jar:42.2.5]
at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:319) ~[tomcat-jdbc.jar:?]
at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:212) ~[tomcat-jdbc.jar:?]
at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:739) ~[tomcat-jdbc.jar:?]
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:671) ~[tomcat-jdbc.jar:?]
at org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:198) ~[tomcat-jdbc.jar:?]
at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:132) ~[tomcat-jdbc.jar:?]
at jdk.internal.reflect.GeneratedMethodAccessor55.invoke(Unknown Source) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.18.jar:5.3.18]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) ~[spring-aop-5.3.18.jar:5.3.18]
at com.sun.proxy.$Proxy38.getConnection(Unknown Source) ~[?:?]
at org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource.getConnection(AbstractRoutingDataSource.java:194) ~[spring-jdbc-5.3.18.jar:5.3.18]
at org.apache.activemq.artemis.jdbc.store.drivers.JDBCConnectionProvider.getConnection(JDBCConnectionProvider.java:61) ~[artemis-jdbc-store-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:213) ~[artemis-jdbc-store-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:1049) ~[artemis-jdbc-store-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent.runForExecutor(ActiveMQScheduledComponent.java:313) ~[artemis-commons-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent.lambda$bookedRunForScheduler$2(ActiveMQScheduledComponent.java:320) ~[artemis-commons-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) ~[artemis-commons-2.21.0.jar:?]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) ~[artemis-commons-2.21.0.jar:?]
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) ~[artemis-commons-2.21.0.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.21.0.jar:?]
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.waitForConnect(Native Method) ~[?:?]
at java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:107) ~[?:?]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) ~[?:?]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) ~[?:?]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) ~[?:?]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:?]
at java.net.Socket.connect(Socket.java:609) ~[?:?]
at org.postgresql.core.PGStream.<init>(PGStream.java:70) ~[postgresql-42.2.5.jar:42.2.5]
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:91) ~[postgresql-42.2.5.jar:42.2.5]
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:192) ~[postgresql-42.2.5.jar:42.2.5]
... 28 more
[INFO ] 2022-06-24 10:29:11.845 [Thread-12] load - preDeActivate called
[WARN ] 2022-06-24 10:29:11.923 [Thread-6 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6#4d16a1c0)] server - AMQ222057: It was not possible to add references due to an IO error code 6 message = JDBC Transaction failed.
[WARN ] 2022-06-24 10:29:11.923 [Thread-6 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6#4d16a1c0)] server - AMQ222063: Error processing IOCallback code = 6 message = JDBC Transaction failed.
[INFO ] 2022-06-24 10:30:04.019 [Thread-12] load - deActivate called
[WARN ] 2022-06-24 10:30:04.112 [Thread-4 (ActiveMQ-client-global-threads)] client - AMQ212037: Connection failure to invm:0 has been detected: AMQ219015: The connection was disconnected because of server shutdown [code=DISCONNECTED]
[WARN ] 2022-06-24 10:30:04.112 [Thread-3 (ActiveMQ-client-global-threads)] client - AMQ212037: Connection failure to invm:0 has been detected: AMQ219015: The connection was disconnected because of server shutdown [code=DISCONNECTED]
[WARN ] 2022-06-24 10:30:04.112 [Thread-1 (ActiveMQ-client-global-threads)] client - AMQ212037: Connection failure to invm:0 has been detected: AMQ219015: The connection was disconnected because of server shutdown [code=DISCONNECTED]
[WARN ] 2022-06-24 10:30:04.112 [Thread-2 (ActiveMQ-client-global-threads)] client - AMQ212037: Connection failure to invm:0 has been detected: AMQ219015: The connection was disconnected because of server shutdown [code=DISCONNECTED]
[INFO ] 2022-06-24 10:30:04.147 [Thread-12] load - stop called
[ERROR] 2022-06-24 10:30:04.575 [DataLoad-EventLoadMessageProcessorThread] load - AMQ219017: Consumer is closed
javax.jms.IllegalStateRuntimeException: AMQ219017: Consumer is closed
at org.apache.activemq.artemis.jms.client.JmsExceptionUtils.convertToRuntimeException(JmsExceptionUtils.java:59) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQJMSConsumer.receive(ActiveMQJMSConsumer.java:68) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at com.test.data.core.messaging.dataload.BaseDataLoaderEventLoadMessageProcessor.processMessagesInBatch(BaseDataLoaderEventLoadMessageProcessor.java:108) [test-core-base-data-loader-22.8.0-SNAPSHOT.jar:?]
at com.test.data.core.messaging.dataload.BaseDataLoaderEventLoadMessageProcessor.run(BaseDataLoaderEventLoadMessageProcessor.java:80) [test-core-base-data-loader-22.8.0-SNAPSHOT.jar:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: javax.jms.IllegalStateException: AMQ219017: Consumer is closed
at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.checkClosed(ClientConsumerImpl.java:951) ~[artemis-core-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.receive(ClientConsumerImpl.java:198) ~[artemis-core-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.receive(ClientConsumerImpl.java:382) ~[artemis-core-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.getMessage(ActiveMQMessageConsumer.java:209) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:129) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQJMSConsumer.receive(ActiveMQJMSConsumer.java:66) ~[artemis-jms-client-2.21.0.jar:2.21.0]
... 3 more
Caused by: org.apache.activemq.artemis.api.core.ActiveMQObjectClosedException: AMQ219017: Consumer is closed
at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.checkClosed(ClientConsumerImpl.java:951) ~[artemis-core-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.receive(ClientConsumerImpl.java:198) ~[artemis-core-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.receive(ClientConsumerImpl.java:382) ~[artemis-core-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.getMessage(ActiveMQMessageConsumer.java:209) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:129) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQJMSConsumer.receive(ActiveMQJMSConsumer.java:66) ~[artemis-jms-client-2.21.0.jar:2.21.0]
... 3 more
[ERROR] 2022-06-24 10:30:04.575 [GS-EventLoadMessageProcessorThread] load - AMQ219017: Consumer is closed
javax.jms.IllegalStateRuntimeException: AMQ219017: Consumer is closed
at org.apache.activemq.artemis.jms.client.JmsExceptionUtils.convertToRuntimeException(JmsExceptionUtils.java:59) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQJMSConsumer.receive(ActiveMQJMSConsumer.java:68) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at com.test.data.core.messaging.dataload.BaseDataLoaderEventLoadMessageProcessor.processMessagesInBatch(BaseDataLoaderEventLoadMessageProcessor.java:108) [test-core-base-data-loader-22.8.0-SNAPSHOT.jar:?]
at com.test.data.core.messaging.dataload.BaseDataLoaderEventLoadMessageProcessor.run(BaseDataLoaderEventLoadMessageProcessor.java:80) [test-core-base-data-loader-22.8.0-SNAPSHOT.jar:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: javax.jms.IllegalStateException: AMQ219017: Consumer is closed
at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.checkClosed(ClientConsumerImpl.java:951) ~[artemis-core-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.receive(ClientConsumerImpl.java:198) ~[artemis-core-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.receive(ClientConsumerImpl.java:382) ~[artemis-core-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.getMessage(ActiveMQMessageConsumer.java:209) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:129) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQJMSConsumer.receive(ActiveMQJMSConsumer.java:66) ~[artemis-jms-client-2.21.0.jar:2.21.0]
... 3 more
Caused by: org.apache.activemq.artemis.api.core.ActiveMQObjectClosedException: AMQ219017: Consumer is closed
at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.checkClosed(ClientConsumerImpl.java:951) ~[artemis-core-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.receive(ClientConsumerImpl.java:198) ~[artemis-core-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.receive(ClientConsumerImpl.java:382) ~[artemis-core-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.getMessage(ActiveMQMessageConsumer.java:209) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:129) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQJMSConsumer.receive(ActiveMQJMSConsumer.java:66) ~[artemis-jms-client-2.21.0.jar:2.21.0]
... 3 more
Exception in thread "DataLoad-EventLoadMessageProcessorThread" Exception in thread "GS-EventLoadMessageProcessorThread" javax.jms.IllegalStateRuntimeException: AMQ219019: Session is closed
at org.apache.activemq.artemis.jms.client.JmsExceptionUtils.convertToRuntimeException(JmsExceptionUtils.java:59)
at org.apache.activemq.artemis.jms.client.ActiveMQJMSContext.recover(ActiveMQJMSContext.java:364)
at com.test.data.core.messaging.dataload.BaseDataLoaderEventLoadMessageProcessor.processMessagesInBatch(BaseDataLoaderEventLoadMessageProcessor.java:149)
at com.test.data.core.messaging.dataload.BaseDataLoaderEventLoadMessageProcessor.run(BaseDataLoaderEventLoadMessageProcessor.java:80)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: javax.jms.IllegalStateException: AMQ219019: Session is closed
at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.checkClosed(ClientSessionImpl.java:2069)
at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.rollback(ClientSessionImpl.java:1007)
at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.rollback(ClientSessionImpl.java:1000)
at org.apache.activemq.artemis.jms.client.ActiveMQSession.recover(ActiveMQSession.java:341)
at org.apache.activemq.artemis.jms.client.ActiveMQJMSContext.recover(ActiveMQJMSContext.java:362)
... 3 more
Caused by: ActiveMQObjectClosedException[errorType=OBJECT_CLOSED message=AMQ219019: Session is closed]
... 8 more
javax.jms.IllegalStateRuntimeException: AMQ219019: Session is closed
at org.apache.activemq.artemis.jms.client.JmsExceptionUtils.convertToRuntimeException(JmsExceptionUtils.java:59)
at org.apache.activemq.artemis.jms.client.ActiveMQJMSContext.recover(ActiveMQJMSContext.java:364)
at com.test.data.core.messaging.dataload.BaseDataLoaderEventLoadMessageProcessor.processMessagesInBatch(BaseDataLoaderEventLoadMessageProcessor.java:149)
at com.test.data.core.messaging.dataload.BaseDataLoaderEventLoadMessageProcessor.run(BaseDataLoaderEventLoadMessageProcessor.java:80)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: javax.jms.IllegalStateException: AMQ219019: Session is closed
at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.checkClosed(ClientSessionImpl.java:2069)
at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.rollback(ClientSessionImpl.java:1007)
at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.rollback(ClientSessionImpl.java:1000)
at org.apache.activemq.artemis.jms.client.ActiveMQSession.recover(ActiveMQSession.java:341)
at org.apache.activemq.artemis.jms.client.ActiveMQJMSContext.recover(ActiveMQJMSContext.java:362)
... 3 more
Caused by: ActiveMQObjectClosedException[errorType=OBJECT_CLOSED message=AMQ219019: Session is closed]
... 8 more
Logs is start is called without calling stop
Caused by: javax.jms.JMSException: Failed to create session factory
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:867) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createContext(ActiveMQConnectionFactory.java:306) ~[artemis-jms-client-2.21.0.jar:2.21.0]
... 139 more
Caused by: org.apache.activemq.artemis.api.core.ActiveMQNotConnectedException: AMQ219007: Cannot connect to server(s). Tried with all available servers.
at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:703) ~[artemis-core-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:865) ~[artemis-jms-client-2.21.0.jar:2.21.0]
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createContext(ActiveMQConnectionFactory.java:306) ~[artemis-jms-client-2.21.0.jar:2.21.0]
... 139 more
EDIT 2: Now sometimes threads are going into BLOCKED state while stopping the server
Thread-6
Stack Trace is:
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(java.base#11.0.15/Native Method)
at java.lang.Thread.sleep(java.base#11.0.15/Thread.java:334)
at java.util.concurrent.TimeUnit.sleep(java.base#11.0.15/TimeUnit.java:446)
at org.apache.activemq.artemis.core.server.impl.FileLockNodeManager.lock(FileLockNodeManager.java:442)
at org.apache.activemq.artemis.core.server.impl.FileLockNodeManager.writeFileLockStatus(FileLockNodeManager.java:329)
at org.apache.activemq.artemis.core.server.impl.FileLockNodeManager.setPaused(FileLockNodeManager.java:309)
at org.apache.activemq.artemis.core.server.impl.FileLockNodeManager.pauseLiveServer(FileLockNodeManager.java:269)
at org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.close(LiveOnlyActivation.java:104)
at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.stop(ActiveMQServerImpl.java:1393)
at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.stop(ActiveMQServerImpl.java:1192)
at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.stop(ActiveMQServerImpl.java:1176)
at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.stop(ActiveMQServerImpl.java:987)
at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.stop(ActiveMQServerImpl.java:981)
at org.apache.activemq.artemis.core.server.embedded.EmbeddedActiveMQ.stop(EmbeddedActiveMQ.java:168)
It's hard to say what exactly be happening without more details (e.g. logging from the broker), but in general behavior you're seeing is what I would expect. When the broker is unable to perform operations on the message store (e.g. file-system, database, etc.) this is considered a "critical" error and the broker will shut itself down. Once you restore connectivity you need to restart the broker, and there's no evidence that you're doing that.
You can register a callback with your embedded broker to be alerted when certain life-cycle events occur. Use org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl#registerActivateCallback and pass it an org.apache.activemq.artemis.core.server.ActivateCallback implementation. You could probably use this callback to restart the broker automatically in case of a failure of this kind.
Generally speaking I would recommend you use the file-based journal rather than JDBC as performance will be much better and it will be more robust.

Timeout using Mule 4 MongoDb connector "list collections" operation

I am working in Mule 4. I want to connect to Azure cosmos MongoDB API. I am successfully in saving documents using the "insert document" operation of the MongoDB connector, but all the other MongoDb operations are throwing an exception. Below is the exception I am getting while using the "list collecton" operation.
Root Exception stack trace:
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=dev-mule-est.documents.azure.com:10255, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: connect timed out}}]
at com.mongodb.internal.connection.BaseCluster.getDescription(BaseCluster.java:182)
at com.mongodb.internal.connection.AbstractMultiServerCluster.getDescription(AbstractMultiServerCluster.java:54)
at com.mongodb.client.internal.MongoClientDelegate.getConnectedClusterDescription(MongoClientDelegate.java:145)
at com.mongodb.client.internal.MongoClientDelegate.createClientSession(MongoClientDelegate.java:100)
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.getClientSession(MongoClientDelegate.java:277)
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:181)
at com.mongodb.client.internal.MongoIterableImpl.execute(MongoIterableImpl.java:143)
at com.mongodb.client.internal.MongoIterableImpl.iterator(MongoIterableImpl.java:92)
at com.mongodb.client.internal.MappingIterable.iterator(MappingIterable.java:39)
at com.mongodb.client.internal.MappingIterable.iterator(MappingIterable.java:27)
at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:226)
at org.mule.modules.mongo.internal.operation.CollectionOperations.listCollections(CollectionOperations.java:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282)
at org.mule.runtime.module.extension.internal.runtime.execution.ReflectiveMethodComponentExecutor.lambda$execute$1(ReflectiveMethodComponentExecutor.java:80)
at org.mule.runtime.core.api.util.ExceptionUtils.tryExpecting(ExceptionUtils.java:227)
at org.mule.runtime.core.api.util.ClassUtils.withContextClassLoader(ClassUtils.java:915)
at org.mule.runtime.core.api.util.ClassUtils.withContextClassLoader(ClassUtils.java:879)
at org.mule.runtime.module.extension.internal.runtime.execution.ReflectiveMethodComponentExecutor.execute(ReflectiveMethodComponentExecutor.java:79)
at org.mule.runtime.module.extension.internal.runtime.operation.ReflectiveMethodOperationExecutor.execute(ReflectiveMethodOperationExecutor.java:64)
at org.mule.runtime.module.extension.internal.runtime.execution.InterceptableOperationExecutorWrapper.execute(InterceptableOperationExecutorWrapper.java:60)
at org.mule.runtime.module.extension.internal.runtime.operation.DefaultExecutionMediator.lambda$null$3(DefaultExecutionMediator.java:146)
at org.mule.runtime.core.api.util.ExceptionUtils.tryExpecting(ExceptionUtils.java:227)
at org.mule.runtime.core.api.util.ClassUtils.withContextClassLoader(ClassUtils.java:915)
at org.mule.runtime.core.api.util.ClassUtils.withContextClassLoader(ClassUtils.java:879)
at org.mule.runtime.module.extension.internal.runtime.operation.DefaultExecutionMediator.lambda$executeWithInterceptors$9(DefaultExecutionMediator.java:146)
at reactor.core.* (20 elements filtered from stack; set debug level logging or '-Dmule.verbose.exceptions=true' for everything)(Unknown Source)
at org.mule.runtime.core.privileged.processor.chain.* (2 elements filtered from stack; set debug level logging or '-Dmule.verbose.exceptions=true' for everything)(Unknown Source)
at reactor.core.* (4 elements filtered from stack; set debug level logging or '-Dmule.verbose.exceptions=true' for everything)(Unknown Source)
at org.mule.runtime.core.privileged.processor.chain.* (1 elements filtered from stack; set debug level logging or '-Dmule.verbose.exceptions=true' for everything)(Unknown Source)
at reactor.core.* (11 elements filtered from stack; set debug level logging or '-Dmule.verbose.exceptions=true' for everything)(Unknown Source)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.mule.service.scheduler.internal.AbstractRunnableFutureDecorator.doRun(AbstractRunnableFutureDecorator.java:111)
at org.mule.service.scheduler.internal.RunnableFutureDecorator.run(RunnableFutureDecorator.java:54)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
That is a time out on the query. Probably the query takes too long to return. Try increasing the socket timeout or simplyfying/optimizing the query.

spring cloud stream kafka client excetion

I used spring-cloud-starter stream-kafka to connect to kafka, version 2.1.2. When I started the service in the k8s cluster, the first few could connect successfully. The following errors occurred in subsequent projects:
org.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry'; nested exception is org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:185)
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:53)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:360)
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:158)
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:122)
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:893)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.finishRefresh(ServletWebServerApplicationContext.java:163)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:552)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:142)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:316)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248)
at com.zynn.service.module.user.ZynnServiceModuleUserApplication.main(ZynnServiceModuleUserApplication.java:15)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
I need some help. Any Suggestions are welcome. Thanks
I have found the reason for the error. I must configure item ”spring.kafka.bootstrap-servers” and item “spring.cloud.stream.kafka.binder.brokers”, otherwise I will create a connection pointing to localhost:9092,This connection is impossible to connect to, so after I configure it simultaneously, the project works

How to start spring-boot app without depending on MongoDB?

Question How to start spring-boot app without depending on Database? solves how to start a spring-boot app when mysql is down.
Is it possible to do the same with mongoDB?
Couldn't find any property in Spring boot application properties that would support it.
Spring-boot config
spring.data.mongodb.uri: mongodb://localhost:27017/measurements-db
I get the following stacktrace
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'diseaseRepository': Invocation of init method failed; nested exception is org.springframework.dao.DataAccessResourceFailureException: Timed out after 10000 ms while waiting for a server that matches AnyServerSelector{}. Client view of cluster state is {type=Unknown, servers=[{address=docker.me:27017, type=Unknown, state=Connecting, exception={com.mongodb.MongoException$Network: Exception opening the socket}, caused by {java.net.ConnectException: Connection refused}}]; nested exception is com.mongodb.MongoTimeoutException: Timed out after 10000 ms while waiting for a server that matches AnyServerSelector{}. Client view of cluster state is {type=Unknown, servers=[{address=docker.me:27017, type=Unknown, state=Connecting, exception={com.mongodb.MongoException$Network: Exception opening the socket}, caused by {java.net.ConnectException: Connection refused}}]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1578) ~[spring-beans-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545) ~[spring-beans-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482) ~[spring-beans-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) ~[spring-beans-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) ~[spring-beans-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) ~[spring-beans-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) ~[spring-beans-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1192) ~[spring-beans-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1116) ~[spring-beans-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1014) ~[spring-beans-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:545) ~[spring-beans-4.2.6.RELEASE.jar:4.2.6.RELEASE]
... 24 common frames omitted
Caused by: org.springframework.dao.DataAccessResourceFailureException: Timed out after 10000 ms while waiting for a server that matches AnyServerSelector{}. Client view of cluster state is {type=Unknown, servers=[{address=docker.me:27017, type=Unknown, state=Connecting, exception={com.mongodb.MongoException$Network: Exception opening the socket}, caused by {java.net.ConnectException: Connection refused}}]; nested exception is com.mongodb.MongoTimeoutException: Timed out after 10000 ms while waiting for a server that matches AnyServerSelector{}. Client view of cluster state is {type=Unknown, servers=[{address=docker.me:27017, type=Unknown, state=Connecting, exception={com.mongodb.MongoException$Network: Exception opening the socket}, caused by {java.net.ConnectException: Connection refused}}]
at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:75) ~[spring-data-mongodb-1.8.4.RELEASE.jar:na]
at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.createIndex(MongoPersistentEntityIndexCreator.java:162) ~[spring-data-mongodb-1.8.4.RELEASE.jar:na]
at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.checkForAndCreateIndexes(MongoPersistentEntityIndexCreator.java:133) ~[spring-data-mongodb-1.8.4.RELEASE.jar:na]
at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.checkForIndexes(MongoPersistentEntityIndexCreator.java:125) ~[spring-data-mongodb-1.8.4.RELEASE.jar:na]
at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.onApplicationEvent(MongoPersistentEntityIndexCreator.java:109) ~[spring-data-mongodb-1.8.4.RELEASE.jar:na]
at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.onApplicationEvent(MongoPersistentEntityIndexCreator.java:51) ~[spring-data-mongodb-1.8.4.RELEASE.jar:na]
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:166) ~[spring-context-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:138) ~[spring-context-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:381) ~[spring-context-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:335) ~[spring-context-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.data.mapping.context.AbstractMappingContext.addPersistentEntity(AbstractMappingContext.java:326) ~[spring-data-commons-1.11.4.RELEASE.jar:na]
at org.springframework.data.mapping.context.AbstractMappingContext.getPersistentEntity(AbstractMappingContext.java:179) ~[spring-data-commons-1.11.4.RELEASE.jar:na]
at org.springframework.data.mapping.context.AbstractMappingContext.getPersistentEntity(AbstractMappingContext.java:139) ~[spring-data-commons-1.11.4.RELEASE.jar:na]
at org.springframework.data.mapping.context.AbstractMappingContext.getPersistentEntity(AbstractMappingContext.java:66) ~[spring-data-commons-1.11.4.RELEASE.jar:na]
at org.springframework.data.mongodb.repository.support.MongoRepositoryFactory.getEntityInformation(MongoRepositoryFactory.java:118) ~[spring-data-mongodb-1.8.4.RELEASE.jar:na]
at org.springframework.data.mongodb.repository.support.MongoRepositoryFactory.getTargetRepository(MongoRepositoryFactory.java:92) ~[spring-data-mongodb-1.8.4.RELEASE.jar:na]
at org.springframework.data.repository.core.support.RepositoryFactorySupport.getRepository(RepositoryFactorySupport.java:185) ~[spring-data-commons-1.11.4.RELEASE.jar:na]
at org.springframework.data.repository.core.support.RepositoryFactoryBeanSupport.initAndReturn(RepositoryFactoryBeanSupport.java:251) ~[spring-data-commons-1.11.4.RELEASE.jar:na]
at org.springframework.data.repository.core.support.RepositoryFactoryBeanSupport.afterPropertiesSet(RepositoryFactoryBeanSupport.java:237) ~[spring-data-commons-1.11.4.RELEASE.jar:na]
at org.springframework.data.mongodb.repository.support.MongoRepositoryFactoryBean.afterPropertiesSet(MongoRepositoryFactoryBean.java:108) ~[spring-data-mongodb-1.8.4.RELEASE.jar:na]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1637) ~[spring-beans-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1574) ~[spring-beans-4.2.6.RELEASE.jar:4.2.6.RELEASE]
... 34 common frames omitted
Caused by: com.mongodb.MongoTimeoutException: Timed out after 10000 ms while waiting for a server that matches AnyServerSelector{}. Client view of cluster state is {type=Unknown, servers=[{address=docker.me:27017, type=Unknown, state=Connecting, exception={com.mongodb.MongoException$Network: Exception opening the socket}, caused by {java.net.ConnectException: Connection refused}}]
at com.mongodb.BaseCluster.getServer(BaseCluster.java:82) ~[mongo-java-driver-2.13.3.jar:na]
at com.mongodb.DBTCPConnector.getServer(DBTCPConnector.java:664) ~[mongo-java-driver-2.13.3.jar:na]
at com.mongodb.DBTCPConnector.access$500(DBTCPConnector.java:40) ~[mongo-java-driver-2.13.3.jar:na]
at com.mongodb.DBTCPConnector$MyPort.getConnection(DBTCPConnector.java:513) ~[mongo-java-driver-2.13.3.jar:na]
at com.mongodb.DBTCPConnector$MyPort.get(DBTCPConnector.java:456) ~[mongo-java-driver-2.13.3.jar:na]
at com.mongodb.DBTCPConnector.getPrimaryPort(DBTCPConnector.java:415) ~[mongo-java-driver-2.13.3.jar:na]
at com.mongodb.DBCollectionImpl.createIndex(DBCollectionImpl.java:378) ~[mongo-java-driver-2.13.3.jar:na]
at com.mongodb.DBCollection.createIndex(DBCollection.java:597) ~[mongo-java-driver-2.13.3.jar:na]
at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.createIndex(MongoPersistentEntityIndexCreator.java:142) ~[spring-data-mongodb-1.8.4.RELEASE.jar:na]
... 54 common frames omitted
This is 3 years later... but this is now possible.
The problem is that auto index creation is enabled. With Spring Boot 2.2 this is now possible to disable and defer any necessary index creation to an ApplicationReadyEvent listener on startup.
The specific property you need to change is spring.data.mongodb.auto-index-creation and set
that to false (which will be the default value in a future version).
Below is a sample of the ApplicationReadyEvent listener that can (and should!) take its place.
#EventListener(ApplicationReadyEvent.class)
public void initIndicesAfterStartup() {
IndexOperations indexOps = mongoTemplate.indexOps(DomainType.class);
IndexResolver resolver = new MongoPersistentEntityIndexResolver(mongoMappingContext);
resolver.resolveIndexFor(DomainType.class).forEach(indexOps::ensureIndex);
}