I am trying to connect MongoDB as source to Kafka server but when I run Debezium Mongo source connector, I get error. I don't understand why timed out?
[2019-08-22 13:28:58,194] INFO Cluster description not yet available. Waiting for 30000 ms before timing out (org.mongodb.driver.cluster:71)
[2019-08-22 13:28:58,648] INFO Exception in monitor thread while connecting to server morgan-shard-00-00-ayfai.mongodb.net:27017 (org.mongodb.driver.cluster:76)
com.mongodb.MongoSocketReadException: Prematurely reached end of stream
at com.mongodb.internal.connection.SocketStream.read(SocketStream.java:112)
at com.mongodb.internal.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:554)
at com.mongodb.internal.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:425)
at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:289)
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:255)
at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:83)
at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:33)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:106)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:63)
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:127)
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:117)
at java.lang.Thread.run(Thread.java:748)
[2019-08-22 13:29:08,196] INFO Created connector mongodb-source-connector (org.apache.kafka.connect.cli.ConnectStandalone:112)
[2019-08-22 13:29:28,195] ERROR Error while reading the 'shards' collection in the 'config' database: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=morgan-shard-00-00-ayfai.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}] (io.debezium.connector.mongodb.ReplicaSetDiscovery:80)
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=morgan-shard-00-00-ayfai.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}]
at com.mongodb.internal.connection.BaseCluster.getDescription(BaseCluster.java:179)
at com.mongodb.internal.connection.SingleServerCluster.getDescription(SingleServerCluster.java:41)
at com.mongodb.client.internal.MongoClientDelegate.getConnectedClusterDescription(MongoClientDelegate.java:136)
at com.mongodb.client.internal.MongoClientDelegate.createClientSession(MongoClientDelegate.java:94)
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.getClientSession(MongoClientDelegate.java:249)
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:172)
at com.mongodb.client.internal.MongoIterableImpl.execute(MongoIterableImpl.java:132)
at com.mongodb.client.internal.MongoIterableImpl.iterator(MongoIterableImpl.java:86)
at com.mongodb.client.internal.MappingIterable.iterator(MappingIterable.java:39)
at io.debezium.connector.mongodb.MongoUtil.contains(MongoUtil.java:183)
at io.debezium.connector.mongodb.MongoUtil.contains(MongoUtil.java:172)
at io.debezium.connector.mongodb.MongoUtil.onDatabase(MongoUtil.java:116)
at io.debezium.connector.mongodb.MongoUtil.onCollection(MongoUtil.java:131)
at io.debezium.connector.mongodb.MongoUtil.onCollectionDocuments(MongoUtil.java:150)
at io.debezium.connector.mongodb.ReplicaSetDiscovery.getReplicaSets(ReplicaSetDiscovery.java:67)
at io.debezium.connector.mongodb.ReplicaSetMonitorThread.run(ReplicaSetMonitorThread.java:65)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2019-08-22 13:29:28,195] INFO Cluster description not yet available. Waiting for 30000 ms before timing out (org.mongodb.driver.cluster:71)
[2019-08-22 13:29:58,196] ERROR Error while trying to get information about the replica sets (io.debezium.connector.mongodb.ReplicaSetMonitorThread:87)
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=morgan-shard-00-00-ayfai.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}]
at com.mongodb.internal.connection.BaseCluster.getDescription(BaseCluster.java:179)
at com.mongodb.internal.connection.SingleServerCluster.getDescription(SingleServerCluster.java:41)
at com.mongodb.Mongo.getClusterDescription(Mongo.java:412)
at com.mongodb.Mongo.getReplicaSetStatus(Mongo.java:455)
at io.debezium.connector.mongodb.ReplicaSetDiscovery.getReplicaSets(ReplicaSetDiscovery.java:85)
at io.debezium.connector.mongodb.ReplicaSetMonitorThread.run(ReplicaSetMonitorThread.java:65)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Without looking at the connector configuration it would be impossible to know the exact issue but from my experience with MongoDB Atlas and Debezium, it can be an issue with SSL.
Try enabling SSL
mongodb.ssl.enabled: true
https://debezium.io/documentation/reference/1.2/connectors/mongodb.html#mongodb-property-mongodb-ssl-enabled
Related
mongodb is running on my local machine and i want to connect this mongodb from drone.io. but i am getting error message as below
com.mongodb.com.mongodb.MongoTimeoutException:
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while
waiting for a server that matches
ReadPreferenceServerSelector{readPreference=primary}. Client view of
cluster state is {type=UNKNOWN, servers=[{address=localhost:27017,
type=UNKNOWN, state=CONNECTING,
exception={com.mongodb.MongoSocketOpenException: Exception opening
socket}, caused by {java.net.ConnectException: Connection refused}}]
Can any one help on this how can i connect my local mongodb from drone.io
The connection from applications in the Kubernetes cluster to the Mongo DB (Atlas) basically works. However, sometimes the connection drops and I can't figure out why. There are two different errors in my logs that seem to be related:
Timed out after 30000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=REPLICA_SET, servers=[{address=xxxxx-tedsb.gcp.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection timed out}}, {address=xxxxx-shard-00-01-tedsb.gcp.mongodb.net:27017, type=REPLICA_SET_SECONDARY, TagSet{[Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='GCP'}, Tag{name='region', value='EUROPE_NORTH_1'}]}, roundTripTime=34.3 ms, state=CONNECTED}, {address=xxxxx.gcp.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection timed out}}]
and
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.connection.TlsChannelStreamFactoryFactory$TlsChannelStream.lambda$openAsync$0(TlsChannelStreamFactoryFactory.java:246)
at com.mongodb.connection.TlsChannelStreamFactoryFactory$SelectorMonitor.lambda$start$0(TlsChannelStreamFactoryFactory.java:141)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.net.ConnectException: Connection timed out
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source)
at com.mongodb.connection.TlsChannelStreamFactoryFactory$TlsChannelStream.lambda$openAsync$0(TlsChannelStreamFactoryFactory.java:218)
... 2 more
I'm not hitting the limit of number of connections and the pod itself is also not under pressure (memory, cpu, disk space). Same goes for the JVM. I'm using the official drivers ("org.mongodb.scala" %% "mongo-scala-driver" % "4.1.0") and Mongo 4.2.8.
The configuration I'm using is pretty basic:
private val uri = configuration.get[String]("mongodb.uri")
private val clientSettings = MongoClientSettings
.builder()
.uuidRepresentation(UuidRepresentation.STANDARD)
.applyConnectionString(new ConnectionString(uri))
.codecRegistry(Registry)
.build()
val client = MongoClient(clientSettings)
I found this (issue)[https://jira.mongodb.org/browse/JAVA-3274] that got resolved in mongo-scala-driver: 4.1.0 and I was hoping that this would fix my issue, but it didn't.
Any idea what I could check next?
Apparently there is a connectivity issue between your pod and two of the three nodes of the replica set. Since the node that is visible is currently a secondary, the driver cannot satisfy the readPreference=primary that is set by default and therefore times out after a while.
I would review your firewall configuration, make sure your pod has access to all three nodes and not just one. Also, ensure that your pod's IP address is whitelisted in Atlas.
Can you help how to configure the connect-mongo-source.properties?
below are the connect-mongo-source.properties
name=mongodb-source-connector
connector.class=io.debezium.connector.mongodb.MongoDbConnector
mongodb.hosts=mongodb://admin:password#ipaddress:27017/test
mongodb.name=mongo_conn
initial.sync.max.threads=1
tasks.max=1
I am getting this error
[2020-04-22 16:26:58,723] ERROR Error while reading the 'shards' collection in the 'config' database: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN,
servers=
[{address=test?authsource=admin&replicaset=quickfoodcluster-shard-0&readpreference=primary&appname=mongodb%20compass%20community&ssl=true:27017, type=UNKNOWN, state=CONNECTING,
exception={com.mongodb.MongoSocketException: test?authsource=admin&replicaset=quickfoodcluster-shard-0&readpreference=primary&appname=mongodb%20compass%20community&ssl=true},
caused by {java.net.UnknownHostException: test?authsource=admin&replicaset=quickfoodcluster-shard-0&readpreference=primary&appname=mongodb%20compass%20community&ssl=true}},
{address=quickfoodcluster-shard-00-01-zxbxk.azure.mongodb.net:27017,
type=UNKNOWN,
state=CONNECTING,
exception={com.mongodb.MongoSocketException: quickfoodcluster-shard-00-01-zxbxk.azure.mongodb.net},
caused by {java.net.UnknownHostException: quickfoodcluster-shard-00-01-zxbxk.azure.mongodb.net}}] (io.debezium.connector.mongodb.ReplicaSetDiscovery)
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN,
servers=[{address=test?authsource=admin&replicaset=quickfoodcluster-shard-0&readpreference=primary&appname=mongodb%20compass%20community&ssl=true:27017,
type=UNKNOWN,
state=CONNECTING,
exception={com.mongodb.MongoSocketException: test?authsource=admin&replicaset=quickfoodcluster-shard-0&readpreference=primary&appname=mongodb%20compass%20community&ssl=true},
caused by {java.net.UnknownHostException: test?authsource=admin&replicaset=quickfoodcluster-shard-0&readpreference=primary&appname=mongodb%20compass%20community&ssl=true}},
{address=quickfoodcluster-shard-00-01-zxbxk.azure.mongodb.net:27017,
type=UNKNOWN, state=CON
NECTING, exception={com.mongodb.MongoSocketException: quickfoodcluster-shard-00-01-zxbxk.azure.mongodb.net}, caused by {java.net.UnknownHostException: quickfoodcluster-shard-00-01-zxbxk.azure.mongodb.net}}]
I think you can't use DSN string in mongodb.hosts, it should be host:port and user and password be provided in a separate field as mentioned in document here.
name=mongodb-source-connector
connector.class=io.debezium.connector.mongodb.MongoDbConnector
mongodb.hosts=ipaddress:27017
mongodb.user=admin
mongodb.password=password
mongodb.name=mongo_conn
initial.sync.max.threads=1
tasks.max=1
mongodb.hosts needs to be like this, with separate entries for username and password:
mongodb.hosts=replicasetname/IPADDRESS:PORT
mongodb.user=USERNAME
mongodb.password=PASSWORD
I have two pods in my Kubernetes cluster - one is connecting just fine to my MongoDB Atlas cluster, the other is failing due to the following exception:
Caused by: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches WritableServerSelector.
Client view of cluster state is {type=REPLICA_SET, servers=[{address=MY_MONGO_SERVER1:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: connect timed out}}, {address=MY_MONGO_SERVER2:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: connect timed out}}, {address=MY_MONGO_SERVER3:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: connect timed out}}]
Now I'm not sure why one pod is able to connect, while the second one fails. We have another environment (production) that has similar configuration and all is fine with that environment's pods. Also, that environment runs on same Atlas cluster-sized DB, with similarly configured Kubernetes deployments.
Hi enables ssl in mongo server and trying to connect from vertx MongoClient.
Step followed : "ssl":true in configuration and set System.setProperty("org.mongodb.async.type", "netty") before intializing the clinet. When I try running my vertx application i am getting following error. can some one please help?
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}. Client view of cluster state is {type=UNKNOWN, servers=[{address=testssl#cv.com:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLException: SSLEngine closed already}}]
at com.mongodb.connection.BaseCluster.createTimeoutException(BaseCluster.java:377)
at com.mongodb.connection.BaseCluster.handleServerSelectionRequest(BaseCluster.java:292)
at com.mongodb.connection.BaseCluster.access$800(BaseCluster.java:54)
at com.mongodb.connection.BaseCluster$WaitQueueHandler.run(BaseCluster.java:458)
at java.lang.Thread.run(Thread.java:745)