Can you help how to configure the connect-mongo-source.properties?
below are the connect-mongo-source.properties
name=mongodb-source-connector
connector.class=io.debezium.connector.mongodb.MongoDbConnector
mongodb.hosts=mongodb://admin:password#ipaddress:27017/test
mongodb.name=mongo_conn
initial.sync.max.threads=1
tasks.max=1
I am getting this error
[2020-04-22 16:26:58,723] ERROR Error while reading the 'shards' collection in the 'config' database: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN,
servers=
[{address=test?authsource=admin&replicaset=quickfoodcluster-shard-0&readpreference=primary&appname=mongodb%20compass%20community&ssl=true:27017, type=UNKNOWN, state=CONNECTING,
exception={com.mongodb.MongoSocketException: test?authsource=admin&replicaset=quickfoodcluster-shard-0&readpreference=primary&appname=mongodb%20compass%20community&ssl=true},
caused by {java.net.UnknownHostException: test?authsource=admin&replicaset=quickfoodcluster-shard-0&readpreference=primary&appname=mongodb%20compass%20community&ssl=true}},
{address=quickfoodcluster-shard-00-01-zxbxk.azure.mongodb.net:27017,
type=UNKNOWN,
state=CONNECTING,
exception={com.mongodb.MongoSocketException: quickfoodcluster-shard-00-01-zxbxk.azure.mongodb.net},
caused by {java.net.UnknownHostException: quickfoodcluster-shard-00-01-zxbxk.azure.mongodb.net}}] (io.debezium.connector.mongodb.ReplicaSetDiscovery)
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN,
servers=[{address=test?authsource=admin&replicaset=quickfoodcluster-shard-0&readpreference=primary&appname=mongodb%20compass%20community&ssl=true:27017,
type=UNKNOWN,
state=CONNECTING,
exception={com.mongodb.MongoSocketException: test?authsource=admin&replicaset=quickfoodcluster-shard-0&readpreference=primary&appname=mongodb%20compass%20community&ssl=true},
caused by {java.net.UnknownHostException: test?authsource=admin&replicaset=quickfoodcluster-shard-0&readpreference=primary&appname=mongodb%20compass%20community&ssl=true}},
{address=quickfoodcluster-shard-00-01-zxbxk.azure.mongodb.net:27017,
type=UNKNOWN, state=CON
NECTING, exception={com.mongodb.MongoSocketException: quickfoodcluster-shard-00-01-zxbxk.azure.mongodb.net}, caused by {java.net.UnknownHostException: quickfoodcluster-shard-00-01-zxbxk.azure.mongodb.net}}]
I think you can't use DSN string in mongodb.hosts, it should be host:port and user and password be provided in a separate field as mentioned in document here.
name=mongodb-source-connector
connector.class=io.debezium.connector.mongodb.MongoDbConnector
mongodb.hosts=ipaddress:27017
mongodb.user=admin
mongodb.password=password
mongodb.name=mongo_conn
initial.sync.max.threads=1
tasks.max=1
mongodb.hosts needs to be like this, with separate entries for username and password:
mongodb.hosts=replicasetname/IPADDRESS:PORT
mongodb.user=USERNAME
mongodb.password=PASSWORD
Related
This is my connection URI
"mongodb+srv://myUserName:myPassWord#myHost/myDb?retryWrites=true&w=majority"
My user has readAnyDatabase role. But when I try to find a document I get this error.
Error printing return value (MongoTimeoutException) at com.mongodb.internal.connection.BaseCluster/createTimeoutException (BaseCluster.java:401).
Timed out after 30000 ms while waiting for a server that matches com.mongodb.client.internal.MongoClientDelegate$1#63d3bb68. Client view of cluster state is {type=REPLICA_SET,
servers=[{address=myDb-shard-00-00.qwert.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: Connect timed out}},
{address=myDb-shard-00-02.qwert.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: Connect timed out}},
{address=myDb-shard-00-01.qwert.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: Connect timed out}}]
If you see it's able to connect to the Mongo DB (I am infering this from the fact that it is able to discover all the nodes from myDb-shard-00-00 to myDb-shard-00-02), but gives this error when trying to read some data.
I have connected to other mongo dbs (different uri) with the same code and successfully read documents. So, I am guessing that there should be something wrong with my configuration? Or is it something else? What is the direction that I should be looking at to debug this?
I created a test DB in Mongo Atlas.
Below is the uri that I am using in the application properties of the spring boot project.
spring.data.mongodb.uri=mongodb+srv://user:pwd#cluster.mongodb.net/test?retryWrites=true&w=majority&ssl=true&sslInvalidHostNameAllowed=true
I tried this one as well.
spring.data.mongodb.uri=mongodb+srv://user:pwd#cluster.mongodb.net/test?retryWrites=true&w=majority
I have whitelisted my IP in the Atlas
My password contains only the alphanumeric character, no special character.
Still, I get the below exception.
Caused by: org.springframework.dao.DataAccessResourceFailureException: Timed out after
30000 ms while waiting for a server that matches com.mongodb.Mongo$4#707865bd. Client
view of cluster state is {type=REPLICA_SET, servers=[{address=cluster1-shard-00-
01.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception=
{com.mongodb.MongoSocketWriteException: Exception sending message}, caused by
{javax.net.ssl.SSLHandshakeException: extension (5) should not be presented in
certificate_request}}, {address=cluster1-shard-00-02.mongodb.net:27017,
type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException:
Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: extension
(5) should not be presented in certificate_request}}, {address=cluster1-shard-00-
00.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception=
{com.mongodb.MongoSocketWriteException: Exception sending message}, caused by
{javax.net.ssl.SSLHandshakeException: extension (5) should not be presented in
certificate_request}}]; nested exception is com.mongodb.MongoTimeoutException: Timed out
after 30000 ms while waiting for a server that matches com.mongodb.Mongo$4#707865bd.
Client view of cluster state is {type=REPLICA_SET, servers=[{address=cluster1-shard-00-
01.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception=
{com.mongodb.MongoSocketWriteException: Exception sending message}, caused by
{javax.net.ssl.SSLHandshakeException: extension (5) should not be presented in
certificate_request}}, {address=cluster1-shard-00-02.mongodb.net:27017,
type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException:
Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: extension (5)
should not be presented in certificate_request}}, {address=cluster1-shard-00-
00.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception=
{com.mongodb.MongoSocketWriteException: Exception sending message}, caused by
{javax.net.ssl.SSLHandshakeException: extension (5) should not be presented in
certificate_request}}]
mongodb is running on my local machine and i want to connect this mongodb from drone.io. but i am getting error message as below
com.mongodb.com.mongodb.MongoTimeoutException:
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while
waiting for a server that matches
ReadPreferenceServerSelector{readPreference=primary}. Client view of
cluster state is {type=UNKNOWN, servers=[{address=localhost:27017,
type=UNKNOWN, state=CONNECTING,
exception={com.mongodb.MongoSocketOpenException: Exception opening
socket}, caused by {java.net.ConnectException: Connection refused}}]
Can any one help on this how can i connect my local mongodb from drone.io
I am trying to connect MongoDB as source to Kafka server but when I run Debezium Mongo source connector, I get error. I don't understand why timed out?
[2019-08-22 13:28:58,194] INFO Cluster description not yet available. Waiting for 30000 ms before timing out (org.mongodb.driver.cluster:71)
[2019-08-22 13:28:58,648] INFO Exception in monitor thread while connecting to server morgan-shard-00-00-ayfai.mongodb.net:27017 (org.mongodb.driver.cluster:76)
com.mongodb.MongoSocketReadException: Prematurely reached end of stream
at com.mongodb.internal.connection.SocketStream.read(SocketStream.java:112)
at com.mongodb.internal.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:554)
at com.mongodb.internal.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:425)
at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:289)
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:255)
at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:83)
at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:33)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:106)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:63)
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:127)
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:117)
at java.lang.Thread.run(Thread.java:748)
[2019-08-22 13:29:08,196] INFO Created connector mongodb-source-connector (org.apache.kafka.connect.cli.ConnectStandalone:112)
[2019-08-22 13:29:28,195] ERROR Error while reading the 'shards' collection in the 'config' database: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=morgan-shard-00-00-ayfai.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}] (io.debezium.connector.mongodb.ReplicaSetDiscovery:80)
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=morgan-shard-00-00-ayfai.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}]
at com.mongodb.internal.connection.BaseCluster.getDescription(BaseCluster.java:179)
at com.mongodb.internal.connection.SingleServerCluster.getDescription(SingleServerCluster.java:41)
at com.mongodb.client.internal.MongoClientDelegate.getConnectedClusterDescription(MongoClientDelegate.java:136)
at com.mongodb.client.internal.MongoClientDelegate.createClientSession(MongoClientDelegate.java:94)
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.getClientSession(MongoClientDelegate.java:249)
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:172)
at com.mongodb.client.internal.MongoIterableImpl.execute(MongoIterableImpl.java:132)
at com.mongodb.client.internal.MongoIterableImpl.iterator(MongoIterableImpl.java:86)
at com.mongodb.client.internal.MappingIterable.iterator(MappingIterable.java:39)
at io.debezium.connector.mongodb.MongoUtil.contains(MongoUtil.java:183)
at io.debezium.connector.mongodb.MongoUtil.contains(MongoUtil.java:172)
at io.debezium.connector.mongodb.MongoUtil.onDatabase(MongoUtil.java:116)
at io.debezium.connector.mongodb.MongoUtil.onCollection(MongoUtil.java:131)
at io.debezium.connector.mongodb.MongoUtil.onCollectionDocuments(MongoUtil.java:150)
at io.debezium.connector.mongodb.ReplicaSetDiscovery.getReplicaSets(ReplicaSetDiscovery.java:67)
at io.debezium.connector.mongodb.ReplicaSetMonitorThread.run(ReplicaSetMonitorThread.java:65)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2019-08-22 13:29:28,195] INFO Cluster description not yet available. Waiting for 30000 ms before timing out (org.mongodb.driver.cluster:71)
[2019-08-22 13:29:58,196] ERROR Error while trying to get information about the replica sets (io.debezium.connector.mongodb.ReplicaSetMonitorThread:87)
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=morgan-shard-00-00-ayfai.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}]
at com.mongodb.internal.connection.BaseCluster.getDescription(BaseCluster.java:179)
at com.mongodb.internal.connection.SingleServerCluster.getDescription(SingleServerCluster.java:41)
at com.mongodb.Mongo.getClusterDescription(Mongo.java:412)
at com.mongodb.Mongo.getReplicaSetStatus(Mongo.java:455)
at io.debezium.connector.mongodb.ReplicaSetDiscovery.getReplicaSets(ReplicaSetDiscovery.java:85)
at io.debezium.connector.mongodb.ReplicaSetMonitorThread.run(ReplicaSetMonitorThread.java:65)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Without looking at the connector configuration it would be impossible to know the exact issue but from my experience with MongoDB Atlas and Debezium, it can be an issue with SSL.
Try enabling SSL
mongodb.ssl.enabled: true
https://debezium.io/documentation/reference/1.2/connectors/mongodb.html#mongodb-property-mongodb-ssl-enabled
I have two pods in my Kubernetes cluster - one is connecting just fine to my MongoDB Atlas cluster, the other is failing due to the following exception:
Caused by: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches WritableServerSelector.
Client view of cluster state is {type=REPLICA_SET, servers=[{address=MY_MONGO_SERVER1:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: connect timed out}}, {address=MY_MONGO_SERVER2:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: connect timed out}}, {address=MY_MONGO_SERVER3:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: connect timed out}}]
Now I'm not sure why one pod is able to connect, while the second one fails. We have another environment (production) that has similar configuration and all is fine with that environment's pods. Also, that environment runs on same Atlas cluster-sized DB, with similarly configured Kubernetes deployments.