Spring boot and mongo db gitlab CI/CD - mongodb

I have set up a spring boot API that does CRUD operations with MongoDB running.
I need to run the commands on CI with hosted MongoDB automatically using GitLab
mvn clean install -B
and
mvn clean test
from .gitlab-ci.yml.
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:70) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:128) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:117) ~[mongodb-driver-core-3.11.2.jar:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:na]
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) ~[na:na]
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) ~[na:na]
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) ~[na:na]
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403) ~[na:na]
at java.base/java.net.Socket.connect(Socket.java:609) ~[na:na]
at com.mongodb.internal.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:64) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.SocketStream.initializeSocket(SocketStream.java:79) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:65) ~[mongodb-driver-core-3.11.2.jar:na]
... 3 common frames omitted
Looking for a .gitlab-ci.yml file that does Mongo server instantiation and DB creation before running mvn clean install
current .gitlab-ci.yml
image: maven:3.6-jdk-11
stages:
- build
cache:
paths:
- target/
services:
- mongo:latest
build:
stage: build
script:
- "mvn clean install -B"
artifacts:
paths:
- ./target/*************

Thanks for the prominent answers/ comments
I was able to solve the problem using embed.mongo in such a way that while the normal API run's it uses actual Mongo store hosted. The tests will be run on embed.mongo
Dependency added :
<dependency>
<groupId>de.flapdoodle.embed</groupId>
<artifactId>de.flapdoodle.embed.mongo</artifactId>
<version>1.48.0</version>
<scope>test</scope>
</dependency>
Common Mongo config
#Configuration
#EnableAutoConfiguration(exclude = { EmbeddedMongoAutoConfiguration.class })
public class MongoConfig {
}
Annotations used for the test class
#ContextConfiguration(classes = {MongoConfig.class})
#RunWith(SpringJUnit4ClassRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
#AutoConfigureDataMongo

Related

Spring Boot Integration test error of CIRCULAR REFERENCE :io.zonky.test.db.provider.ProviderException

I am working on Spring Boot Application that contains unit and integration test cases to check functionalities.
Docker is running with Podman and it working fine.
I am using POSTGRES database server that configured on my machine. But while running Integration test, the test are getting ignored with the error : CIRCULAR REFERENCE:io.zonky.test.db.provider.ProviderException: Unexpected error when preparing a database cluster; nested exception is java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
Detailed Stack Trace :
java.lang.IllegalStateException: Failed to load ApplicationContext
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:132)
at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:123)
at io.zonky.test.db.EmbeddedDatabaseTestExecutionListener.forEachDatabase(EmbeddedDatabaseTestExecutionListener.java:90)
at io.zonky.test.db.EmbeddedDatabaseTestExecutionListener.resetDatabases(EmbeddedDatabaseTestExecutionListener.java:58)
at io.zonky.test.db.EmbeddedDatabaseTestExecutionListener.beforeTestClass(EmbeddedDatabaseTestExecutionListener.java:34)
at org.springframework.test.context.TestContextManager.beforeTestClass(TestContextManager.java:213)
at org.springframework.test.context.junit.jupiter.SpringExtension.beforeAll(SpringExtension.java:77)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeBeforeAllCallbacks$7(ClassBasedTestDescriptor.java:355)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeBeforeAllCallbacks(ClassBasedTestDescriptor.java:355)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:189)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:77)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:132)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:197)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Suppressed: java.lang.IllegalStateException: Failed to load ApplicationContext
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:132)
at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:123)
at io.zonky.test.db.EmbeddedDatabaseTestExecutionListener.forEachDatabase(EmbeddedDatabaseTestExecutionListener.java:90)
at io.zonky.test.db.EmbeddedDatabaseTestExecutionListener.resetDatabases(EmbeddedDatabaseTestExecutionListener.java:58)
at io.zonky.test.db.EmbeddedDatabaseTestExecutionListener.afterTestClass(EmbeddedDatabaseTestExecutionListener.java:54)
at org.springframework.test.context.TestContextManager.afterTestClass(TestContextManager.java:488)
at org.springframework.test.context.junit.jupiter.SpringExtension.afterAll(SpringExtension.java:86)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeAfterAllCallbacks$13(ClassBasedTestDescriptor.java:421)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeAfterAllCallbacks$14(ClassBasedTestDescriptor.java:421)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeAfterAllCallbacks(ClassBasedTestDescriptor.java:421)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.after(ClassBasedTestDescriptor.java:213)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.after(ClassBasedTestDescriptor.java:77)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:145)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:145)
... 27 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Invocation of init method failed; nested exception is io.zonky.test.db.shaded.com.google.common.util.concurrent.UncheckedExecutionException: io.zonky.test.db.provider.ProviderException: Unexpected error when preparing a database cluster; nested exception is java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:595)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1108)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:868)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:125)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:124)
... 43 more
Caused by: io.zonky.test.db.shaded.com.google.common.util.concurrent.UncheckedExecutionException: io.zonky.test.db.provider.ProviderException: Unexpected error when preparing a database cluster; nested exception is java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
at io.zonky.test.db.shaded.com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1571)
at io.zonky.test.db.shaded.com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1556)
at io.zonky.test.db.context.DefaultDatabaseContext.awaitDatabase(DefaultDatabaseContext.java:282)
at io.zonky.test.db.context.DefaultDatabaseContext.getDatabase(DefaultDatabaseContext.java:126)
at io.zonky.test.db.context.DatabaseTargetSource.getTarget(DatabaseTargetSource.java:32)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:192)
at com.sun.proxy.$Proxy206.toString(Unknown Source)
at java.base/java.lang.String.valueOf(String.java:2951)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:168)
at org.hibernate.jpa.internal.util.LogHelper.logPersistenceUnitInformation(LogHelper.java:53)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.<init>(EntityManagerFactoryBuilderImpl.java:191)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.<init>(EntityManagerFactoryBuilderImpl.java:168)
at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:52)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:365)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.buildNativeEntityManagerFactory(AbstractEntityManagerFactoryBean.java:391)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:378)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.afterPropertiesSet(LocalContainerEntityManagerFactoryBean.java:341)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1855)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1792)
... 59 more
Caused by: io.zonky.test.db.provider.ProviderException: Unexpected error when preparing a database cluster; nested exception is java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
at io.zonky.test.db.provider.postgres.DockerPostgresDatabaseProvider.createDatabase(DockerPostgresDatabaseProvider.java:117)
at io.zonky.test.db.provider.postgres.DockerPostgresDatabaseProvider.createTemplate(DockerPostgresDatabaseProvider.java:102)
at io.zonky.test.db.provider.common.TemplatingDatabaseProvider.createTemplate(TemplatingDatabaseProvider.java:129)
at io.zonky.test.db.provider.common.TemplatingDatabaseProvider.lambda$createTemplateIfPossible$1(TemplatingDatabaseProvider.java:171)
at io.zonky.test.db.provider.common.TemplatingDatabaseProvider$TemplateWrapper.loadTemplate(TemplatingDatabaseProvider.java:294)
at io.zonky.test.db.provider.common.TemplatingDatabaseProvider$TemplateWrapper.access$200(TemplatingDatabaseProvider.java:224)
at io.zonky.test.db.provider.common.TemplatingDatabaseProvider.createTemplateIfPossible(TemplatingDatabaseProvider.java:171)
at io.zonky.test.db.provider.common.TemplatingDatabaseProvider.createDatabase(TemplatingDatabaseProvider.java:112)
at io.zonky.test.db.provider.common.TemplatingDatabaseProvider.createDatabase(TemplatingDatabaseProvider.java:104)
at io.zonky.test.db.provider.common.PrefetchingDatabaseProvider$PrefetchingTask.lambda$forPreparer$0(PrefetchingDatabaseProvider.java:391)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at io.zonky.test.db.provider.common.PrefetchingDatabaseProvider$PrefetchingTask.run(PrefetchingDatabaseProvider.java:420)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$7(DockerClientProviderStrategy.java:215)
at java.base/java.util.Optional.orElseThrow(Optional.java:408)
at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:207)
at org.testcontainers.DockerClientFactory.getOrInitializeStrategy(DockerClientFactory.java:136)
at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:178)
at org.testcontainers.LazyDockerClient.getDockerClient(LazyDockerClient.java:14)
at org.testcontainers.LazyDockerClient.authConfig(LazyDockerClient.java:12)
at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:310)
at io.zonky.test.db.provider.postgres.DockerPostgresDatabaseProvider$DatabaseInstance.<init>(DockerPostgresDatabaseProvider.java:173)
at io.zonky.test.db.provider.postgres.DockerPostgresDatabaseProvider$DatabaseInstance.<init>(DockerPostgresDatabaseProvider.java:137)
at io.zonky.test.db.provider.postgres.DockerPostgresDatabaseProvider$1.load(DockerPostgresDatabaseProvider.java:77)
at io.zonky.test.db.provider.postgres.DockerPostgresDatabaseProvider$1.load(DockerPostgresDatabaseProvider.java:75)
at io.zonky.test.db.shaded.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708)
at io.zonky.test.db.shaded.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416)
at io.zonky.test.db.shaded.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299)
at io.zonky.test.db.shaded.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212)
at io.zonky.test.db.shaded.com.google.common.cache.LocalCache.get(LocalCache.java:4147)
at io.zonky.test.db.shaded.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151)
at io.zonky.test.db.shaded.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140)
at io.zonky.test.db.provider.postgres.DockerPostgresDatabaseProvider.createDatabase(DockerPostgresDatabaseProvider.java:113)
... 14 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Invocation of init method failed; nested exception is io.zonky.test.db.shaded.com.google.common.util.concurrent.UncheckedExecutionException: io.zonky.test.db.provider.ProviderException: Unexpected error when preparing a database cluster; nested exception is java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:595)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1108)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:868)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:125)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:124)
... 41 more
Caused by: io.zonky.test.db.shaded.com.google.common.util.concurrent.UncheckedExecutionException: io.zonky.test.db.provider.ProviderException: Unexpected error when preparing a database cluster; nested exception is java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
at io.zonky.test.db.shaded.com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1571)
at io.zonky.test.db.shaded.com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1556)
at io.zonky.test.db.context.DefaultDatabaseContext.awaitDatabase(DefaultDatabaseContext.java:282)
at io.zonky.test.db.context.DefaultDatabaseContext.getDatabase(DefaultDatabaseContext.java:126)
at io.zonky.test.db.context.DatabaseTargetSource.getTarget(DatabaseTargetSource.java:32)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:192)
at com.sun.proxy.$Proxy187.toString(Unknown Source)
at java.base/java.lang.String.valueOf(String.java:2951)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:168)
at org.hibernate.jpa.internal.util.LogHelper.logPersistenceUnitInformation(LogHelper.java:53)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.<init>(EntityManagerFactoryBuilderImpl.java:191)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.<init>(EntityManagerFactoryBuilderImpl.java:168)
at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:52)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:365)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.buildNativeEntityManagerFactory(AbstractEntityManagerFactoryBean.java:391)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:378)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.afterPropertiesSet(LocalContainerEntityManagerFactoryBean.java:341)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1855)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1792)
... 57 more
[CIRCULAR REFERENCE:io.zonky.test.db.provider.ProviderException: Unexpected error when preparing a database cluster; nested exception is java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration]
I have tried many solution like updated to latest version on zonky :
<dependency>
<groupId>io.zonky.test</groupId>
<artifactId>embedded-database-spring-test</artifactId>
<version>2.1.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.zonky.test</groupId>
<artifactId>embedded-postgres</artifactId>
<version>1.2.6</version>
<scope>test</scope>
</dependency>
But it didn't worked.
Tried #AutoConfigureEmbeddedDatabase(type = AutoConfigureEmbeddedDatabase.DatabaseType.POSTGRES, provider = ZONKY)
Then also it throwing same error : CIRCULAR REFERENCE:io.zonky.test.db.provider.ProviderException: Unexpected error when preparing a database cluster
Detailed Stack trace
Caused by: io.zonky.test.db.shaded.com.google.common.util.concurrent.UncheckedExecutionException: io.zonky.test.db.provider.ProviderException: Unexpected error when preparing a database cluster; nested exception is java.lang.IllegalStateException: Process [C:\Users\NEELAM~1.SHA\AppData\Local\Temp\embedded-pg\PG-aaa7d03a0c0eb51f9510c27dc059d022\bin\initdb.exe, -A, trust, -U, postgres, -D, C:\Users\NEELAM~1.SHA\AppData\Local\Temp\epg14265419788948759198, -E, UTF-8] failed
at io.zonky.test.db.shaded.com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1571)
at io.zonky.test.db.shaded.com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1556)
at io.zonky.test.db.context.DefaultDatabaseContext.awaitDatabase(DefaultDatabaseContext.java:282)
at io.zonky.test.db.context.DefaultDatabaseContext.getDatabase(DefaultDatabaseContext.java:126)
at io.zonky.test.db.context.DatabaseTargetSource.getTarget(DatabaseTargetSource.java:32)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:192)
at com.sun.proxy.$Proxy187.toString(Unknown Source)
at java.base/java.lang.String.valueOf(String.java:2951)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:168)
at org.hibernate.jpa.internal.util.LogHelper.logPersistenceUnitInformation(LogHelper.java:53)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.<init>(EntityManagerFactoryBuilderImpl.java:191)
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.<init>(EntityManagerFactoryBuilderImpl.java:168)
at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:52)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:365)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.buildNativeEntityManagerFactory(AbstractEntityManagerFactoryBean.java:391)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:378)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.afterPropertiesSet(LocalContainerEntityManagerFactoryBean.java:341)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1855)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1792)
... 57 more
[CIRCULAR REFERENCE:io.zonky.test.db.provider.ProviderException: Unexpected error when preparing a database cluster; nested exception is java.lang.IllegalStateException: Process [C:\Users\NEELAM~1.SHA\AppData\Local\Temp\embedded-pg\PG-aaa7d03a0c0eb51f9510c27dc059d022\bin\initdb.exe, -A, trust, -U, postgres, -D, C:\Users\NEELAM~1.SHA\AppData\Local\Temp\epg14265419788948759198, -E, UTF-8] failed]
What can be more possible way to resolve the issue of tests?
upgrading embedded-postgres to 2.0.0 should fix the issue
<dependency>
<groupId>io.zonky.test</groupId>
<artifactId>embedded-postgres</artifactId>
<version>2.0.0</version>
<scope>test</scope>
</dependency>

Spring Boot MongoDB Connectivity Issue

My Spring Boot application is trying to connect to MongoDB Database. The configuration is added on application.yml file. The application was working completely fine locally till yesterday.
But when today I am running it locally on localhost, the following error I am receiving:
2020-12-03 14:59:03.763[0;39m [32m INFO[0;39m [35m10692[0;39m [2m---[0;39m [2m[ngodb.net:27017][0;39m [36morg.mongodb.driver.cluster [0;39m [2m:[0;39m Exception in monitor thread while connecting to server edugyanamcluster-shard-00-00.l4au7.mongodb.net:27017
com.mongodb.MongoSocketWriteException: Exception sending message
at com.mongodb.internal.connection.InternalStreamConnection.translateWriteException(InternalStreamConnection.java:551) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:433) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.InternalStreamConnection.sendCommandMessage(InternalStreamConnection.java:273) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:257) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:83) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:33) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:105) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:62) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:129) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:117) ~[mongodb-driver-core-3.11.2.jar:na]
at java.base/java.lang.Thread.run(Thread.java:835) ~[na:na]
Caused by: javax.net.ssl.SSLHandshakeException: extension (5) should not be presented in certificate_request
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131) ~[na:na]
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[na:na]
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:307) ~[na:na]
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:263) ~[na:na]
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:254) ~[na:na]
at java.base/sun.security.ssl.SSLExtensions.<init>(SSLExtensions.java:90) ~[na:na]
at java.base/sun.security.ssl.CertificateRequest$T13CertificateRequestMessage.<init>(CertificateRequest.java:818) ~[na:na]
at java.base/sun.security.ssl.CertificateRequest$T13CertificateRequestConsumer.consume(CertificateRequest.java:922) ~[na:na]
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392) ~[na:na]
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443) ~[na:na]
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421) ~[na:na]
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:177) ~[na:na]
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1180) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1091) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:721) ~[na:na]
at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:998) ~[na:na]
at com.mongodb.internal.connection.SocketStream.write(SocketStream.java:99) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:430) ~[mongodb-driver-core-3.11.2.jar:na]
... 9 common frames omitted
[2m2020-12-03 14:59:05.112[0;39m [33m WARN[0;39m [35m10692[0;39m [2m---[0;39m [2m[ restartedMain][0;39m [36mJpaBaseConfiguration$JpaWebConfiguration[0;39m [2m:[0;39m spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
[2m2020-12-03 14:59:07.468[0;39m [32m INFO[0;39m [35m10692[0;39m [2m---[0;39m [2m[ restartedMain][0;39m [36mpertySourcedRequestMappingHandlerMapping[0;39m [2m:[0;39m Mapped URL path [/v2/api-docs] onto method [springfox.documentation.swagger2.web.Swagger2ControllerWebMvc#getDocumentation(String, HttpServletRequest)]
[2m2020-12-03 14:59:07.822[0;39m [32m INFO[0;39m [35m10692[0;39m [2m---[0;39m [2m[ restartedMain][0;39m [36m.s.s.UserDetailsServiceAutoConfiguration[0;39m [2m:[0;39m
Need advice on what is the issue all about? As the code was working completely fine few hours before.
MongoDB has updated their server actually, might have updated their cipher.
The solution is actually change the TLS Version to 1.2 in JVM params.
One can add in Eclipse (if using it as an IDE) ->
Project -> Run As -> Run Configurations -> Arguments -> VM Arguments, add ->
-Djdk.tls.client.protocols=TLSv1.2
I could fix this issue using the solution from this other question. I changed the TLS version to 1.2 in JVM params:
-Djdk.tls.client.protocols=TLSv1.2
Other than downgrading to TLSV1.2, Upgrading your JDK fixes this too.
For JDK version 11.0.0 - 11.0.6, upgrading your JDK to a patched version.
JDK-8236039
JDK version 13.0.0 - 13.0.02, upgrading to 13.0.03 should resolve this issue
JDK-8241515
I had literally tried everything like upgrading the jdk and all but the best solution and also the working one is changing the TLS version.
Right click on the project in IDE
Run As -> Run Configurations...
And goto Arguments tab
and in VM Arguments add this
-Djdk.tls.client.protocols=TLSv1.2
Run the application now and it should work fine.
You can refer for images attached for more clarity.
enter image description here
enter image description here
I updated from JDK 11.0.7 to 11.0.10 and now it works
I had this problem and solved first adding this parameter to the JVM:
-Djdk.tls.client.protocols=TLSv1.2
But later, I changed the openjdk:12 docker image to the latest openjdk:11 (LTS) and it solved the issue without the -D parameter anymore =)
I had the same issue https://bugs.openjdk.java.net/browse/JDK-8236039 and I change my JDK to JDK-15 zulu: https://www.azul.com/downloads/zulu-community/?version=java-15-mts and it fixed the problem, I was previously using a version of Open JDK.

Apache Storm Starter 2.2.0 in Eclipse in Windows - Exception while trying to get leader nimbus info from localhost NimbusLeaderNotFound

I have downloaded the zip file for Apache Storm 2.2.0.
I imported the storm-starter maven project into eclipse in Windows 10. Did a Maven--> Update Project
My project did not have any errors. I did not do any updates to pom.xml.
I executed the WordCountTopology.java
I get the error NimbusLeaderNotFoundException.
I tried after stopping firewall also.
00:54:00.584 [main] INFO o.a.s.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest: -xxxxxxxxxxxxxx
00:54:00.668 [main] WARN o.a.s.v.ConfigValidation - task.heartbeat.frequency.secs is a deprecated config please see class org.apache.storm.Config.TASK_HEARTBEAT_FREQUENCY_SECS for more information.
00:54:16.923 [main] WARN o.a.s.u.NimbusClient - Ignoring exception while trying to get leader nimbus info from localhost. will retry with a different seed host.
java.lang.RuntimeException: java.lang.RuntimeException: org.apache.storm.thrift.transport.TTransportException: java.net.ConnectException: Connection refused: connect
at org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:108) ~[storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.security.auth.ThriftClient.<init>(ThriftClient.java:69) ~[storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.utils.NimbusClient.<init>(NimbusClient.java:80) ~[storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:221) [storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:179) [storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:138) [storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.blobstore.NimbusBlobStore.prepare(NimbusBlobStore.java:47) [storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.utils.Utils.validateTopologyBlobStoreMap(Utils.java:1178) [storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.StormSubmitter.validateConfs(StormSubmitter.java:530) [storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:236) [storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:210) [storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:173) [storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.topology.ConfigurableTopology.submit(ConfigurableTopology.java:119) [storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.starter.WordCountTopology.run(WordCountTopology.java:58) [classes/:?]
at org.apache.storm.topology.ConfigurableTopology.start(ConfigurableTopology.java:68) [storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.starter.WordCountTopology.main(WordCountTopology.java:36) [classes/:?]
Caused by: java.lang.RuntimeException: org.apache.storm.thrift.transport.TTransportException: java.net.ConnectException: Connection refused: connect
at org.apache.storm.security.auth.TBackoffConnect.retryNext(TBackoffConnect.java:59) ~[storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:51) ~[storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:98) ~[storm-client-2.2.0.jar:2.2.0]
... 15 more
Caused by: org.apache.storm.thrift.transport.TTransportException: java.net.ConnectException: Connection refused: connect
at org.apache.storm.thrift.transport.TSocket.open(TSocket.java:226) ~[storm-shaded-deps-2.2.0.jar:2.2.0]
at org.apache.storm.thrift.transport.TFramedTransport.open(TFramedTransport.java:91) ~[storm-shaded-deps-2.2.0.jar:2.2.0]
at org.apache.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:101) ~[storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48) ~[storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:98) ~[storm-client-2.2.0.jar:2.2.0]
... 15 more
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[?:1.8.0_241]
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85) ~[?:1.8.0_241]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[?:1.8.0_241]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[?:1.8.0_241]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[?:1.8.0_241]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[?:1.8.0_241]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_241]
at java.net.Socket.connect(Socket.java:606) ~[?:1.8.0_241]
at org.apache.storm.thrift.transport.TSocket.open(TSocket.java:221) ~[storm-shaded-deps-2.2.0.jar:2.2.0]
at org.apache.storm.thrift.transport.TFramedTransport.open(TFramedTransport.java:91) ~[storm-shaded-deps-2.2.0.jar:2.2.0]
at org.apache.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:101) ~[storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48) ~[storm-client-2.2.0.jar:2.2.0]
at org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:98) ~[storm-client-2.2.0.jar:2.2.0]
... 15 more
org.apache.storm.utils.NimbusLeaderNotFoundException: Could not find leader nimbus from seed hosts [localhost]. Did you specify a valid list of nimbus hosts for config nimbus.seeds?
at org.apache.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:250)
at org.apache.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:179)
at org.apache.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:138)
at org.apache.storm.blobstore.NimbusBlobStore.prepare(NimbusBlobStore.java:47)
at org.apache.storm.utils.Utils.validateTopologyBlobStoreMap(Utils.java:1178)
at org.apache.storm.StormSubmitter.validateConfs(StormSubmitter.java:530)
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:236)
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:210)
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:173)
at org.apache.storm.topology.ConfigurableTopology.submit(ConfigurableTopology.java:119)
at org.apache.storm.starter.WordCountTopology.run(WordCountTopology.java:58)
at org.apache.storm.topology.ConfigurableTopology.start(ConfigurableTopology.java:68)
at org.apache.storm.starter.WordCountTopology.main(WordCountTopology.java:36)
Same Eclipse in Windows 10, Under a different eclipse project, For previous storm versions' code (Version 0.0.1) Zookeeper starts, nimbus starts fine. I am able to get the output.
I did not do any cluster configuration in local/eclipse.Success Topology with version 0.0.1

Data Fusion Provisioning of Dataproc Cluster Fails

I've created a simple pipeline which reads from a SQL Server table and writes to a BigQuery table. Then I configure it to use Spark and deploy and run. It starts by provisioning the dataproc cluster and I can see that it relatively quickly creates 3 VM's, one master and two workers. The main cluster creation job stays as "provisioning" though, both in the dataproc UI and in the Data Fusion UI. After about 17 minutes it fails.
I've tried both in an enterprise instance and a basic instance. I've made sure that the instance service account has the "Cloud Data Fusion API Service Agent" role. I've run the preview, which runs in around 20 seconds and succeeds.
This is the log:
2019-06-21 10:59:37,011 - DEBUG [provisioning-service-3:i.c.c.i.p.t.ProvisioningTask#121] - Executing PROVISION subtask REQUESTING_CREATE for program run program_run:default.Load_From_BIQ_v1.-SNAPSHOT.workflow.DataPipelineWorkflow.a7999324-9413-11e9-a296-564a3b7813c8.
2019-06-21 10:59:42,087 - INFO [provisioning-service-3:i.c.c.r.s.p.d.DataprocProvisioner#171] - Creating Dataproc cluster cdap-loadfromb-a7999324-9413-11e9-a296-564a3b7813c8 with system labels {goog-datafusion-version=6_0, cdap-version=6_0_1-1559673739218, goog-datafusion-edition=basic}
2019-06-21 10:59:45,446 - DEBUG [provisioning-service-3:i.c.c.i.p.t.ProvisioningTask#125] - Completed PROVISION subtask REQUESTING_CREATE for program run program_run:default.Load_From_BIQ_v1.-SNAPSHOT.workflow.DataPipelineWorkflow.a7999324-9413-11e9-a296-564a3b7813c8.
2019-06-21 10:59:45,461 - DEBUG [provisioning-service-3:i.c.c.i.p.t.ProvisioningTask#121] - Executing PROVISION subtask POLLING_CREATE for program run program_run:default.Load_From_BIQ_v1.-SNAPSHOT.workflow.DataPipelineWorkflow.a7999324-9413-11e9-a296-564a3b7813c8.
2019-06-21 10:59:46,402 - DEBUG [provisioning-service-3:i.c.c.i.p.t.ProvisioningTask#125] - Completed PROVISION subtask POLLING_CREATE for program run program_run:default.Load_From_BIQ_v1.-SNAPSHOT.workflow.DataPipelineWorkflow.a7999324-9413-11e9-a296-564a3b7813c8.
(...)
2019-06-21 11:17:31,345 - DEBUG [provisioning-service-3:i.c.c.i.p.t.ProvisioningTask#121] - Executing PROVISION subtask REQUESTING_DELETE for program run program_run:default.Load_From_BIQ_v1.-SNAPSHOT.workflow.DataPipelineWorkflow.a7999324-9413-11e9-a296-564a3b7813c8.
2019-06-21 11:17:32,753 - DEBUG [provisioning-service-3:i.c.c.i.p.t.ProvisioningTask#125] - Completed PROVISION subtask REQUESTING_DELETE for program run program_run:default.Load_From_BIQ_v1.-SNAPSHOT.workflow.DataPipelineWorkflow.a7999324-9413-11e9-a296-564a3b7813c8.
2019-06-21 11:17:32,769 - DEBUG [provisioning-service-3:i.c.c.i.p.t.ProvisioningTask#121] - Executing PROVISION subtask POLLING_DELETE for program run program_run:default.Load_From_BIQ_v1.-SNAPSHOT.workflow.DataPipelineWorkflow.a7999324-9413-11e9-a296-564a3b7813c8.
2019-06-21 11:17:33,588 - DEBUG [provisioning-service-3:i.c.c.i.p.t.ProvisioningTask#125] - Completed PROVISION subtask POLLING_DELETE for program run program_run:default.Load_From_BIQ_v1.-SNAPSHOT.workflow.DataPipelineWorkflow.a7999324-9413-11e9-a296-564a3b7813c8.
2019-06-21 11:17:33,601 - DEBUG [provisioning-service-3:i.c.c.i.p.t.ProvisioningTask#112] - Completed PROVISION task for program run program_run:default.Load_From_BIQ_v1.-SNAPSHOT.workflow.DataPipelineWorkflow.a7999324-9413-11e9-a296-564a3b7813c8.
2019-06-21 11:17:35,946 - DEBUG [provisioning-service-4:i.c.c.i.p.t.ProvisioningTask#121] - Executing DEPROVISION subtask REQUESTING_DELETE for program run program_run:default.Load_From_BIQ_v1.-SNAPSHOT.workflow.DataPipelineWorkflow.a7999324-9413-11e9-a296-564a3b7813c8.
2019-06-21 11:17:37,219 - ERROR [provisioning-service-4:i.c.c.i.p.t.ProvisioningTask#151] - DEPROVISION task failed in REQUESTING_DELETE state for program run program_run:default.Load_From_BIQ_v1.-SNAPSHOT.workflow.DataPipelineWorkflow.a7999324-9413-11e9-a296-564a3b7813c8.
com.google.api.gax.rpc.FailedPreconditionException: io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Cannot delete cluster 'cdap-loadfromb-a7999324-9413-11e9-a296-564a3b7813c8' while it has other pending delete operations.
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:59) ~[na:na]
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72) ~[na:na]
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60) ~[na:na]
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:95) ~[na:na]
at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:61) ~[na:na]
at com.google.common.util.concurrent.Futures$4.run(Futures.java:1123) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:435) ~[na:na]
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:900) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:811) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:675) ~[com.google.guava.guava-13.0.1.jar:na]
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:492) ~[na:na]
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:467) ~[na:na]
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41) ~[na:na]
at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684) ~[na:na]
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41) ~[na:na]
at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:392) ~[na:na]
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:475) ~[na:na]
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63) ~[na:na]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:557) ~[na:na]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:478) ~[na:na]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:590) ~[na:na]
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[na:na]
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) ~[na:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_212]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_212]
Caused by: io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Cannot delete cluster 'cdap-loadfromb-a7999324-9413-11e9-a296-564a3b7813c8' while it has other pending delete operations.
at io.grpc.Status.asRuntimeException(Status.java:526) ~[na:na]
... 19 common frames omitted
2019-06-21 11:17:37,235 - DEBUG [provisioning-service-4:i.c.c.i.p.t.ProvisioningTask#159] - Terminated DEPROVISION task for program run program_run:default.Load_From_BIQ_v1.-SNAPSHOT.workflow.DataPipelineWorkflow.a7999324-9413-11e9-a296-564a3b7813c8 due to exception.
Because the Dataproc cluster remains in "provisioning", my suspicion is that the network being used for the Dataproc cluster is not configured such that the nodes of the Dataproc cluster can communicate with each other.
For more information on this, see https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/network#overview.
Make sure the Data Fusion has default network accesses. If you have a new VPC without default network firewall rules, you might face with this problem. Basically give it a try to run Data Fusion on default VPC network with following properties.
"system.profile.properties.network=default"

Spring boot connect to MongoDB replica set of mongo container running

I have set 2 mongo container which having 27017:27017 and 27018:27017 as port. mongo1 is set as primary while mongo2 set as secondary and used replica set.
mongo1 as primary has both write, read access and mongo2 has only read access
spring.data.mongodb.uri = mongodb://abc.xyz:27017/user_demo
//monog1, abc.xyz is the hostname
spring.data.mongodb.uri = mongodb://abc.xyz:27018/user_demo
//mongo2
If I connect only monogo1 in my normal spring boot program, it works(read, write) and if I set mongo2 it works(only read), but when I set both in one line it gives error
spring.data.mongodb.uri = mongodb://abc.xyz:27017,abc.xyz:27018/user_demo?replicaSet=idea-mongo-set
I checked replica set is working by storing in mongo1, in mongo2 I can read
So, what is wrong with the last line. Is there any other way to store uri for replica set
In official spring site it is given
spring.data.mongodb.uri=mongodb://user:secret#mongo1.example.com:12345,mongo2.example.com:23456/test
Error I am getting is this
2019-01-08 15:56:35.245 INFO 29804 --- [ main] c.j.bootifulmongodb.BootMongoDBApp : Started BootMongoDBApp in 3.603 seconds (JVM running for 4.373)
2019-01-08 15:56:54.466 INFO 29804 --- [azure.com:27018] org.mongodb.driver.cluster : Exception in monitor thread while connecting to server abc.xyz:27018
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.connection.SocketStream.open(SocketStream.java:63) ~[mongodb-driver-core-3.4.3.jar:na]
at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:115) ~[mongodb-driver-core-3.4.3.jar:na]
at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:113) ~[mongodb-driver-core-3.4.3.jar:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[na:1.8.0_181]
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85) ~[na:1.8.0_181]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_181]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_181]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_181]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[na:1.8.0_181]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_181]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_181]
at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:57) ~[mongodb-driver-core-3.4.3.jar:na]
at com.mongodb.connection.SocketStream.open(SocketStream.java:58) ~[mongodb-driver-core-3.4.3.jar:na]
... 3 common frames omitted
It looks like your uri is on the right track for connecting to a replica set.
spring.data.mongodb.uri = mongodb://abc.xyz:27017,abc.xyz:27018/user_demo?replicaSet=idea-mongo-set
Is it just a typo that credentials (user, password) are omitted here?
This exact syntax is what we use for our replica set connection uri:
mongodb://user:pwd#server1:27017,server2:27017,server3:27017/our_collection?replicaSet=replica_set_name&authSource=authagainstthiscollection&authMechanism=SCRAM-SHA-1
Because you set abc.xyz:27018 hostname in mongo rs.initiate, so client will try to access this hostname directly. But this hostname is unreachable for client, it's just a FQDN in your azure cluster.
I think this document will help you clients use the hostnames listed in the replica set config not the seed list