I am connecting my spring boot application with mongo db.
The code is working fine on local environment where the db has a single node and not running in cluster mode.
But when the db is running on staging environment with cluster mode and a single node and 1 replica set, the spring application shows the following error:
INFO | | | 2019-04-18 12:43:25 | [cluster-ClusterId{value='5cb823951a734cea7f7eebe1', description='null'}-10.202.41.254:27017] com.mongodb.diagnostics.logging.SLF4JLogger - Canonical address educationelasticsearch-digital-v1-41-254:27017 does not match server address. Removing 10.20.41.254:27017 from client view of cluster
INFO | | | 2019-04-18 12:43:25 | [cluster-ClusterId{value='5cb823951a734cea7f7eebe1', description='null'}-educationelasticsearch-digital-v1-41-254:27017] com.mongodb.diagnostics.logging.SLF4JLogger - Exception in monitor thread while connecting to server educationelasticsearch-digital-v1-41-254:27017
com.mongodb.MongoSocketException: educationelasticsearch-digital-v1-41-254: nodename nor servname provided, or not known
at com.mongodb.ServerAddress.getSocketAddress(ServerAddress.java:188) ~[mongodb-driver-core-3.8.2.jar:?]
at com.mongodb.internal.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:64) ~[mongodb-driver-core-3.8.2.jar:?]
at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:62) ~[mongodb-driver-core-3.8.2.jar:?]
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:126) ~[mongodb-driver-core-3.8.2.jar:?]
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:117) [mongodb-driver-core-3.8.2.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
Caused by: java.net.UnknownHostException: educationelasticsearch-digital-v1-41-254: nodename nor servname provided, or not known
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[?:1.8.0_171]
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) ~[?:1.8.0_171]
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) ~[?:1.8.0_171]
at java.net.InetAddress.getAllByName0(InetAddress.java:1276) ~[?:1.8.0_171]
at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[?:1.8.0_171]
at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[?:1.8.0_171]
at java.net.InetAddress.getByName(InetAddress.java:1076) ~[?:1.8.0_171]
at com.mongodb.ServerAddress.getSocketAddress(ServerAddress.java:186) ~[mongodb-driver-core-3.8.2.jar:?]
... 5 more
In both the case, application properties are same except the change in IP of staging server in place of localhost:
spring.elasticsearch.education.hostname=localhost // staging server IP
spring.elasticsearch.education.scheme=http
spring.elasticsearch.education.port=9200
###### Mongo properties ########
spring.data.mongodb.host=localhost //staging server ip
spring.data.mongodb.port=27017
spring.data.mongodb.database=ddddd
spring.data.mongodb.authentication-database=admin
spring.data.mongodb.username=root
spring.data.mongodb.password=root
spring.data.mongodb.repositories.enabled=true
logging.level.org.springframework.data.mongodb.core.MongoTemplate=DEBUG
I checked this question but not much resolution in this.
Please suggest what's wrong here?
Look here https://stackoverflow.com/a/34373673/5712629.
Maybe spring.data.mongodb.host and spring.data.mongodb.port are not supported in this version of mongodb driver.
You could try:
spring.data.mongodb.uri=mongodb://root:root#localhost:27017
instead.
Related
I'm trying to connect to a remote PostgreSQL database using JDBC, however the connection times out.
To be able to access the remote server I'm connected to a tunnel. There is no problem when I'm trying to connect from the same network with JDBC, only remotely.
I'm able to connect to the database both remotely and locally using other tools (e.g. pgAdmin, HeidiSQL, python sqlalchemy/psycopg2), but not with JDBC. I do not need to add other options for remote connection with the other tools, simply host:port and user/pass.
Searching for solutions I have tried the following without success:
add property "ssl" with value "true" to the driver connection
add property "sslfactory" with value "org.postgresql.ssl.NonValidatingFactory"
add property "loginTimeout" with value 20
set listen_addresses = '*' in postgresql.conf on remote server
add host all all 0.0.0.0/0 md5 to pg_hba.conf on remote server
Using driver version postgresql-42.2.14. I'm attempting to connect using SQuirreL and MATLAB.
As other tools have no issue connecting I'm guessing this is about tweaking the JDBC settings.
What am I missing here?
Edit:
Trying to ping the IP of the remote server (192.168.x.x.) I get:
Reply from 10.x.x.x: Destination net unreachable.
At the same time I'm able to connect to the database on the same server IP. Is this a clue to the problem?
Connection url:
jdbc:postgresql://192.168.x.x:5432/<dbname>
Results of Test-NetConnection
PS > Test-NetConnection 192.168.x.x
WARNING: Ping to 192.168.x.x failed with status:
DestinationNetworkUnreachable
PS > Test-NetConnection 192.168.x.x -port 5432
ComputerName : 192.168.x.x
RemoteAddress : 192.168.x.x
RemotePort : 5432
InterfaceAlias : Local Area Connection 3
SourceAddress : 192.168.x.x
TcpTestSucceeded : True
Running tracert 192.168.x.x
Tracing route to 192.168.x.x over a maximum of 30 hops
1 <1 ms <1 ms <1 ms 192.168.0.1
2 10.x.x.x reports: Destination net unreachable.
Trace complete.
Stacktrace (SQuirreL):
java.net.SocketTimeoutException: connect timed out
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at org.postgresql.core.PGStream.<init>(PGStream.java:81)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:93)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:197)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:217)
at org.postgresql.Driver.makeConnection(Driver.java:458)
at org.postgresql.Driver.connect(Driver.java:260)
at net.sourceforge.squirrel_sql.fw.sql.SQLDriverManager.getConnection(SQLDriverManager.java:147)
at net.sourceforge.squirrel_sql.client.mainframe.action.OpenConnectionCommand.executeConnect(OpenConnectionCommand.java:136)
at net.sourceforge.squirrel_sql.client.mainframe.action.OpenConnectionCommand.lambda$execute$0(OpenConnectionCommand.java:93)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
You are pinging IP address in the range of 192.168.x.x and getting reply from 10.x.x.x. Those are two different sub-nets.
As odd as it seems, it does not go well with your initial statement about being able to connect via other means. If network was issue then other tools would have failed as well. Not getting a reply for ping "Request time out" is expected result if there is blockage.
"Destination unreachable" means there is no proper route available to the IP you are trying to get to. With that issue, no other connections to that host should be reachable from your host.
Please confirm if all connections you attempted were from same client host. Also have you tried ODBC connector to see if that works? That allows System DSNs and you do not need to do much in attempt to connect.
Please make sure you do tests either from same hosts or two machines on same network to simplify things.
With "Destination unreachable" error your client should not be talking to server in anyway if same destination IP is used.
I have been reading all sort of answers to connect my local MongoDB from Spring boot app which is deployed in Docker Container.
But I am getting below error:
2020-04-24 06:39:29.425 INFO 1 --- [ main] org.mongodb.driver.cluster : Adding discovered server 127.0.0.1:27017 to client view of cluster
2020-04-24 06:39:29.542 INFO 1 --- [127.0.0.1:27017] org.mongodb.driver.cluster : Exception in monitor thread while connecting to server 127.0.0.1:27017
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:67) ~[mongodb-driver-core-3.8.2.jar!/:na]
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:126) ~[mongodb-driver-core-3.8.2.jar!/:na]
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:117) ~[mongodb-driver-core-3.8.2.jar!/:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_252]
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_252]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_252]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_252]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_252]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_252]
at java.net.Socket.connect(Socket.java:607) ~[na:1.8.0_252]
at com.mongodb.internal.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:64) ~[mongodb-driver-core-3.8.2.jar!/:na]
at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:62) ~[mongodb-driver-core-3.8.2.jar!/:na]
... 3 common frames omitted
2020-04-24 06:39:30.833 INFO 1 --- [ main] pertySourcedRequestMappingHandlerMapping : Mapped URL path [/v2/api-docs] onto method [public org.springframework.http.ResponseEntity<springfox.documentation.spring.web.json.Json> springfox.documentation.swagger2.web.Swagger2Controller.getDocumentation(java.lang.String,javax.servlet.http.HttpServletRequest)]
If I run this spring boot app out of docker container, it works fine.
But from inside docker container its not able to access my mongoDB which is outside of docker container.
This how I am running docker container:
docker build -f .\Dockerfile -t dockerspringboot .
docker run -p 8085:8080 dockerspringboot
This is my Dockerfile:
FROM openjdk:8
ADD build/libs/SpringBoot-1.0.jar SpringBoot-1.0.jar
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "SpringBoot-1.0.jar"]
-I tried changing ipaddress to 0.0.0.0 and 127.0.0.1, but of no use.
-I tried changing mondod.conf with below commented/uncommented options:
# network interfaces
net:
#port: 27017
bindIp: 0.0.0.0
# bindIpAll: true
# bindIp: 127.0.0.1
-I tried changing firewall settings to allow all programs access to docker backend process as well.
Also tried changing local ipaddresses by using ipconfig command but still no luck.
You should specify the authorizations to access the database if they exist, try this :
spring.data.mongodb.authentication-database=admin
spring.data.mongodb.database=app1
spring.data.mongodb.host=host.docker.internal
spring.data.mongodb.port=27017
spring.data.mongodb.password=PASSWORD
spring.data.mongodb.username=USERNAME
Add --network="host" to your docker run command. Then your docker container is able to access localhost:27017
I have set 2 mongo container which having 27017:27017 and 27018:27017 as port. mongo1 is set as primary while mongo2 set as secondary and used replica set.
mongo1 as primary has both write, read access and mongo2 has only read access
spring.data.mongodb.uri = mongodb://abc.xyz:27017/user_demo
//monog1, abc.xyz is the hostname
spring.data.mongodb.uri = mongodb://abc.xyz:27018/user_demo
//mongo2
If I connect only monogo1 in my normal spring boot program, it works(read, write) and if I set mongo2 it works(only read), but when I set both in one line it gives error
spring.data.mongodb.uri = mongodb://abc.xyz:27017,abc.xyz:27018/user_demo?replicaSet=idea-mongo-set
I checked replica set is working by storing in mongo1, in mongo2 I can read
So, what is wrong with the last line. Is there any other way to store uri for replica set
In official spring site it is given
spring.data.mongodb.uri=mongodb://user:secret#mongo1.example.com:12345,mongo2.example.com:23456/test
Error I am getting is this
2019-01-08 15:56:35.245 INFO 29804 --- [ main] c.j.bootifulmongodb.BootMongoDBApp : Started BootMongoDBApp in 3.603 seconds (JVM running for 4.373)
2019-01-08 15:56:54.466 INFO 29804 --- [azure.com:27018] org.mongodb.driver.cluster : Exception in monitor thread while connecting to server abc.xyz:27018
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.connection.SocketStream.open(SocketStream.java:63) ~[mongodb-driver-core-3.4.3.jar:na]
at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:115) ~[mongodb-driver-core-3.4.3.jar:na]
at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:113) ~[mongodb-driver-core-3.4.3.jar:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[na:1.8.0_181]
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85) ~[na:1.8.0_181]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_181]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_181]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_181]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[na:1.8.0_181]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_181]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_181]
at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:57) ~[mongodb-driver-core-3.4.3.jar:na]
at com.mongodb.connection.SocketStream.open(SocketStream.java:58) ~[mongodb-driver-core-3.4.3.jar:na]
... 3 common frames omitted
It looks like your uri is on the right track for connecting to a replica set.
spring.data.mongodb.uri = mongodb://abc.xyz:27017,abc.xyz:27018/user_demo?replicaSet=idea-mongo-set
Is it just a typo that credentials (user, password) are omitted here?
This exact syntax is what we use for our replica set connection uri:
mongodb://user:pwd#server1:27017,server2:27017,server3:27017/our_collection?replicaSet=replica_set_name&authSource=authagainstthiscollection&authMechanism=SCRAM-SHA-1
Because you set abc.xyz:27018 hostname in mongo rs.initiate, so client will try to access this hostname directly. But this hostname is unreachable for client, it's just a FQDN in your azure cluster.
I think this document will help you clients use the hostnames listed in the replica set config not the seed list
I have virtual machine centOS(v7.4) running on Openstack. In virtual machine, I deployed kubernetes and 7 pods, virtual machine could ping the external server. But one pod uaa could NOT connect database on external network (app.xxx.com 11521), and logs say:
... 49 common frames omitted
Caused by: java.sql.SQLRecoverableException: IO Error: The Network Adapter could not establish the connection
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:419) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:536) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:228) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:39) ~[commons-dbcp2-2.1.1.jar!/:2.1.1]
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:256) ~[commons-dbcp2-2.1.1.jar!/:2.1.1]
at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2304) ~[commons-dbcp2-2.1.1.jar!/:2.1.1]
at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2290) ~[commons-dbcp2-2.1.1.jar!/:2.1.1]
... 53 common frames omitted
Caused by: oracle.net.ns.NetException: The Network Adapter could not establish the connection
at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:375) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
at oracle.net.resolver.AddrResolution.resolveAndExecute(AddrResolution.java:422) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
at oracle.net.ns.NSProtocol.establishConnection(NSProtocol.java:678) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:238) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1054) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:308) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
... 61 common frames omitted
Caused by: java.net.UnknownHostException: app.xxx.com: Name or service not known
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[na:1.8.0_111]
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) ~[na:1.8.0_111]
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) ~[na:1.8.0_111]
at java.net.InetAddress.getAllByName0(InetAddress.java:1276) ~[na:1.8.0_111]
at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[na:1.8.0_111]
at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[na:1.8.0_111]
at oracle.net.nt.TcpNTAdapter.connect(TcpNTAdapter.java:171) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
at oracle.net.nt.ConnOption.connect(ConnOption.java:123) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:353) ~[ojdbc6-11.2.0.1.0.jar!/:11.2.0.2.0]
... 66 common frames omitted
I ssh into the pod,
[centos#kube-master ingress2]$ sudo kubectl exec -it gearbox-rack-uaa-service -5b5fd58d87-5zsln -- /bin/bash
root#gearbox-rack-uaa-service-5b5fd58d87-5zsln:/# ping app.xxx.com
ping: unknown host
My questions is how to let pod connects to external database?
in your spec file try setting "spec.template.spec.hostNetwork: true"
There are other things this affects but should get over your ping issue
I am trying to put a file into hdfs using ssh from my client pc to the NameNode server. There are 2 machines: One NameNode and one DataNode. Here is the command I am trying;
$ bin/hadoop fs -fs hdfs://MY_IP:MY_PORT -put example.txt example.txt
But it throws an exception. The logs say;
2013-05-23 09:25:31,808 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:_my_user_name_ cause:java.io.IOException: Unknown protocol to DataNode: org.apache.hadoop.hdfs.protocol.ClientProtocol
2013-05-23 09:25:31,808 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on _my_port_, call getProtocolVersion(org.apache.hadoop.hdfs.protocol.ClientProtocol, 61) from _ip_:_port_: error: java.io.IOException: Unknown protocol to DataNode: org.apache.hadoop.hdfs.protocol.ClientProtocol
java.io.IOException: Unknown protocol to DataNode: org.apache.hadoop.hdfs.protocol.ClientProtocol
at org.apache.hadoop.hdfs.server.datanode.DataNode.getProtocolVersion(DataNode.java:1759)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
What can be the problem? Thanks a lot
Check your core-site.xml file and verify the hdfs url.
Check the version of your client's hadoop and cluster's hadoop, my guess is your client version is less than the cluster version.
java.io.IOException: Unknown protocol to DataNode: org.apache.hadoop.hdfs.protocol.ClientProtocol
at org.apache.hadoop.hdfs.server.datanode.DataNode.getProtocolVersion(DataNode.java:1759)