Connecting to MongoDB from Camel-K - mongodb

How do I connect to MongoDB with Camel-K (version 1.6.0) ?
I've tried setting the host name directly on the MongoDB Camel component URI:
...
.to("mongodb:mongoBean?database=[redacted]&collection=[redacted]&hosts=[redacted]&username=[redacted]&password=[redacted]&operation=getDbStats")
but it insists on calling 127.0.0.1:27017:
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=127.0.0.1:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]
at com.mongodb.internal.connection.BaseCluster.getDescription(BaseCluster.java:177)
Probably I should be setting a class reference in a properties file, like so:
camel.beans.mongoBean = #class:...
but then I'm not sure what class that should be? - and what other property names to set?
The full Camel-K integration Java file I'm deploying to the Kubernetes cluster (OpenShift CodeReady containers, with the command kamel run MongoDBTest.java --dev), looks like this:
// camel-k: language=java property=file:integration.properties
import org.apache.camel.builder.RouteBuilder;
public class MongoDBTest extends RouteBuilder {
#Override
public void configure() throws Exception {
from("timer:java?period=1000")
.routeId("java")
.to("mongodb:mongoBean?database=[redacted]&collection=[redacted]&operation=getDbStats&hosts=[redacted]&username=[redacted]&password=[redacted]")
.to("log:info");
}
}
and in the integration.properties file I have:
camel.beans.mongoBean = #class:com.mongodb.MongoClient
should there be some other class reference?
if not setting the host name on the MongoDB Camel component URI, what properties should come in there under camel.beans.mongoBean?

The solution was to set the quarkus.mongodb... Camel Quarkus MongoDB properties, as pointed out here: https://github.com/apache/camel-k/issues/2684#issuecomment-939798018

Related

com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[]

We are trying to connect to Mongo Atlas from Java code using
URI "mongodb+srv::#serveraddress/dbname?retryWrites=true&w=1" but getting MongoTimeoutException exception.
Strange thing here is in exception trace servers=[] list is empty.
What could be the possible reasons for this error?
MongoDB Server and Java Driver Details:
MongoDB Server: 4.2
MongoDB java Driver: 3.12.5
spring-data-mongodb: 2.1.5.RELEASE
Please let me know if any further information needed.
Thank you for the help.
com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[]
at com.mongodb.internal.connection.BaseCluster.getDescription(BaseCluster.java:182)
at com.mongodb.internal.connection.AbstractMultiServerCluster.getDescription(AbstractMultiServerCluster.java:54)
at com.mongodb.client.internal.MongoClientDelegate.getConnectedClusterDescription(MongoClientDelegate.java:152)
at com.mongodb.client.internal.MongoClientDelegate.createClientSession(MongoClientDelegate.java:103)
at com.mongodb.Mongo.createClientSession(Mongo.java:832)
at com.mongodb.MongoClient.startSession(MongoClient.java:582)
at com.mongodb.MongoClient.startSession(MongoClient.java:569)
at com.shutterfly.services.project.InitMongoDBCollection.onApplicationEvent(InitMongoDBCollection.java:36)
at com.shutterfly.services.project.InitMongoDBCollection.onApplicationEvent(InitMongoDBCollection.java:20)
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:402)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:359)
at org.springframework.boot.context.event.EventPublishingRunListener.running(EventPublishingRunListener.java:105)
at org.springframework.boot.SpringApplicationRunListeners.running(SpringApplicationRunListeners.java:78)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:332)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248)
at com.shutterfly.services.project.Application.main(Application.java:18)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53)
at java.lang.Thread.run(Thread.java:748)
Adding the answer so that it should be helpful for others, who might see similar issue in future.
We have resolved this issue, The problem was it was taking time to find the list of clusters and connect to that with default time, we can use "MongoClientOptionsFactoryBean" to customised the parameters and can override the default configuration.
#Bean
public MongoClientOptionsFactoryBean getMongoClientOptionsFactoryBean() {
MongoClientOptionsFactoryBean mongoClientOptionsFactoryBean = new MongoClientOptionsFactoryBean();
mongoClientOptionsFactoryBean.setConnectTimeout(timeout);
mongoClientOptionsFactoryBean.setConnectionsPerHost(connectionsPerHost);
mongoClientOptionsFactoryBean.setMaxWaitTime(maxWaitTime);
mongoClientOptionsFactoryBean.setServerSelectionTimeout(ServerSelectionTimeout);
mongoClientOptionsFactoryBean.setThreadsAllowedToBlockForConnectionMultiplier(connMultiplier);
return mongoClientOptionsFactoryBean;
}
in my case mongo was simply not running :)
please, check it with:
systemctl status mongod.service
and make sure it is enabled before rebooting the server:
systemctl enable mongod.service
so I have same problem
I fix it by this code
use this import
import com.mongodb.client.MongoClient;
and this is the code
String web="mongodb://";
MongoClient mongoClient = MongoClients.create(web+ENVIRONMENT);
database = mongoClient.getDatabase("PUT_HERE_YOUR_DB_NAME");
MAYBE in some place you will need change the web
String web="mongodb+srv://"

java connection refused error with spring boot data geode remote locator

Per my question Apache Geode Web framework I've checked through various spring guides from here and spring data geode samples from here and written a short spring data geode application but it cannot connect to the remote GFSH started Geode locator. The Application class is:
package cm;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.data.gemfire.config.annotation.ClientCacheApplication;
import org.springframework.data.gemfire.config.annotation.ClientCacheApplication.Locator;
import org.springframework.data.gemfire.config.annotation.EnablePdx;
import org.springframework.data.gemfire.repository.config.EnableGemfireRepositories;
#SpringBootApplication
#ClientCacheApplication(name = "CmWeb", locators = #Locator, subscriptionEnabled = true)
#EnableGemfireRepositories(basePackageClasses= {CmRequest.class})
#EnablePdx
public class CmWeb {
public static void main(String[] args) {
SpringApplication.run(CmWeb.class, args);
}
}
and in the resources directory application.properties I've set up the remote locator:
# Configure the client's connection Pool to the servers in the cluster
spring.data.gemfire.pool.locators=1.2.3.4[10334]
Build and run the application and it discovers the locator (which it returns as the server name)
[Timer-DEFAULT-2] o.a.g.c.c.i.AutoConnectionSourceImpl : AutoConnectionSource discovered new locators [UAT:10334]
A couple of seconds later it throws the error:
[Timer-DEFAULT-2] o.a.g.c.c.i.AutoConnectionSourceImpl : locator UAT:10334 is not running.
and
java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[na:1.8.0_232]
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85) ~[na:1.8.0_232]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_232]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:204) ~[na:1.8.0_232]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_232]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[na:1.8.0_232]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_232]
at java.net.Socket.connect(Socket.java:607) ~[na:1.8.0_232]
at org.apache.geode.internal.net.SocketCreator.connect(SocketCreator.java:958) ~[geode-core-1.9.2.jar:na]
at org.apache.geode.internal.net.SocketCreator.connect(SocketCreator.java:899) ~[geode-core-1.9.2.jar:na]
at org.apache.geode.internal.net.SocketCreator.connect(SocketCreator.java:888) ~[geode-core-1.9.2.jar:na]
at org.apache.geode.distributed.internal.tcpserver.TcpClient.getServerVersion(TcpClient.java:290) ~[geode-core-1.9.2.jar:na]
at org.apache.geode.distributed.internal.tcpserver.TcpClient.requestToServer(TcpClient.java:184) ~[geode-core-1.9.2.jar:na]
at org.apache.geode.cache.client.internal.AutoConnectionSourceImpl.queryOneLocatorUsingConnection(AutoConnectionSourceImpl.java:209) [geode-core-1.9.2.jar:na]
at org.apache.geode.cache.client.internal.AutoConnectionSourceImpl.queryOneLocator(AutoConnectionSourceImpl.java:199) [geode-core-1.9.2.jar:na]
at org.apache.geode.cache.client.internal.AutoConnectionSourceImpl.queryLocators(AutoConnectionSourceImpl.java:287) [geode-core-1.9.2.jar:na]
at org.apache.geode.cache.client.internal.AutoConnectionSourceImpl$UpdateLocatorListTask.run2(AutoConnectionSourceImpl.java:500) [geode-core-1.9.2.jar:na]
at org.apache.geode.cache.client.internal.PoolImpl$PoolTask.run(PoolImpl.java:1371) [geode-core-1.9.2.jar:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_232]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_232]
at org.apache.geode.internal.ScheduledThreadPoolExecutorWithKeepAlive$DelegatingScheduledFuture.run(ScheduledThreadPoolExecutorWithKeepAlive.java:276) [geode-core-1.9.2.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_232]
After a lot of investigation I thought it was that the spring data geode client expects a spring boot geode server according to Connecting GemFire using Spring Boot and Spring Data GemFire and so I downloaded the ListRegionsOnServerFunction jar and deployed it on the GFSH server get the same result (have not yet restarted the server...) but that causes the same error condition.
If by Spring-Data-Gemfire - Unable to contact a Locator service. Operation either timed out or Locator does not exist I try and change the application.properties from
spring.data.gemfire.pool.locators=1.2.3.4[10334]
to
spring.gemfire.locators=1.2.3.4[10334]
or other variations then the app can't find the remote locator and throws:
[Timer-DEFAULT-3] o.a.g.c.c.i.AutoConnectionSourceImpl : locator localhost/127.0.0.1:10334 is not running.
Writing this question I've finally found How to connect a remote-locator in Geode and also can't PING the GFSH server from the SPRING app. However, the server bind address is setup properly for remote locator clients and various other services and UI using a locally built Geode Native Client for Geode v 1.10 can connect. I suspect PING may be disabled across this (semi-internal) network by default. I also disabled the firewall rules for ports 10334, 1099, 40404 to allow all traffic but still get the same error condition.
It turns out that from repeated INFO messages in the Spring Boot app after the connection refused:
[Timer-DEFAULT-2] o.a.g.c.c.i.AutoConnectionSourceImpl : updateLocatorInLocatorList changing locator list: loc form: LocatorAddress [socketInetAddress=UAT:10334, hostname=UAT, isIpString=false] ,loc to: UAT:10334
[Timer-DEFAULT-2] o.a.g.c.c.i.AutoConnectionSourceImpl : updateLocatorInLocatorList locator list from:[UAT:10334, /1.2.3.4:10334] to: [LocatorAddress [socketInetAddress=UAT:10334, hostname=UAT, isIpString=false], LocatorAddress [socketInetAddress=/1.2.3.4:10334, hostname=1.2.3.4, isIpString=true]]
and then running list clients on the server, the connection from the Spring Boot app to the Geode server v 1.10 is in fact established. Arrrgh!
It means the locator logic is working but this doesn't explain why after the first connection there's a java.net.ConnectException: Connection refused: connect error. Any ideas?
1 quick note about your Spring Boot application class...
#SpringBootApplication
#ClientCacheApplication(name = "CmWeb", locators = #Locator, subscriptionEnabled = true)
#EnableGemfireRepositories(basePackageClasses= {CmRequest.class})
#EnablePdx
public class CmWeb {
public static void main(String[] args) {
SpringApplication.run(CmWeb.class, args);
}
}
The following statements are true iff you are using Spring Boot for Apache Geode (or Pivotal GemFire), which is highly recommended.
When using SBDG (by declaring the correct org.springframework.geode:spring-geode-starter dependency on your application classpath), then you do not need to explicitly declare the #ClientCacheApplication, #EnableGemfireRepositories or the #EnablePdx annotations since SBDG auto-configures a ClientCache instance by default, auto-configures SD Repositories particularly when all entity classes are in the same package or sub-package as the Spring Boot app and SBDG auto-configures PDX by default, as well.
The locator = #Locator just specifies that the "DEFAULT" GemFire/Geode Pool when configured via the ClientCacheFactory should connect to the cluster via Locators, on localhost using the default Locator port, 10334. Therefore, this attribute is mostly useless and I would recommend the new #EnableClusterAware annotation from SBDG (see here).
The other attributes can be configured via Spring Boot application.properties, like so:
spring.application.name=CmWeb
spring.data.gemfire.pool.subscription-enabled=true
TIP: You can configure subscription on individually "named" Pools, even via properties, if you are using more than 1 Pool (of connections) in your application, perhaps to route different payloads based on workflows to different "grouped" servers in your cluster, etc.
You started to configure the "DEFAULT" Pool in application.properties already with...
# Configure the client's connection Pool to the servers in the cluster
spring.data.gemfire.pool.locators=1.2.3.4[10334]
Regarding...
After a lot of investigation I thought it was that the spring data geode client expects a spring boot geode server
No, SDG does not expect the cluster (of servers) to be configured or bootstrapped with Spring at all. Using Gfsh is perfectly valid. For instance. If the ListRegionsOnServerFunction is not available, SDG falls back to other means (provided by GemFire/Geode itself, which Gfsh knows and uses).
All the messages you are seeing in the Spring Boot app logs are coming from Geode itself, i.e. nothing to do with Spring. In a nutshell, and FWIW, SDG/SBDG is a facade around the Apache Geode (Pivotal GemFire) API and Java client driver. SDG/SBDG is at the mercy of this client doing the right thing, which of course, is partially dependent on proper configuration. Still... I am really just thinking out loud now since I suspect you are already well aware of (or have discovered) all of this.
I would also say the Java client and Native Client are not exactly an apple to apple comparison either. Meaning, if you developed a client using purely the Apache Geode (Pivotal GemFire) API without Spring, you'd have the exact same problem.
I have never seen a case where the first connection is establish but subsequent connections get a "Connection refused", o.O #argh
Have you tried this same configuration/arrangement with older Geode versions, e.g. 1.9?
Sorry for your troubles. I will think on this more.

spring-mail-starter ignoring spring.mail.* config

I have a project using spring-boot and added
the following dependency in my build.gradle:
compile group: 'org.springframework.boot', name: 'spring-boot-starter-mail', version: '1.5.7.RELEASE'
My application.properties contains
SMTP Configuration
spring.mail.host=mail.xxx
spring.mail.port=587
spring.mail.username=donotreply#xxx
spring.mail.password=xxx
spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.from=donotreply#xxx
According to multiple answers here and documentations i found,that should be enought to create the bean with this settings.
I now want to use a mailer in a Service class
#Autowired
private JavaMailSender emailSender;
[...]
#Value("${spring.mail.host}")
private String mailHost;
When i try to send a message using emailSender i get the following error:
Mail server connection failed; nested exception is
com.sun.mail.util.MailConnectException: Couldn't connect to host,
port: localhost, 25; timeout -1; nested exception is:
java.net.ConnectException: Connection refused: connect. Failed
messages: com.sun.mail.util.MailConnectException: Couldn't connect to
host, port: localhost, 25; timeout -1; nested exception is:
java.net.ConnectException: Connection refused: connect
Obviously it is trying to connect to localhost instead of the configured host.
The variable mailHost does contain the correct value, so the properties are read correct.
EDIT:
The autowired class for emailSender is org.springframework.mail.javamail.JavaMailSenderImpl, which seems to be correct to me.
In my case, the reason was that I instantiated the bean myself like this:
#Bean
public JavaMailSender mailSender() throws IOException {
JavaMailSenderImpl mailSender = new JavaMailSenderImpl();
mailSender.setDefaultEncoding("UTF-8");
return mailSender;
}
Not sure why, but that way, it ignores the application.properties. Maybe the reason was, that I explicitly instantiated JavaMailSenderImpl to set a default encoding and possibly that implementation isn't application.properties aware?
However, removing that bean solved the problem.
[FOUND]
A co-worker was working on the same and had commited an incomplete bean definition. That bean was autowired.

MongoDB: PLAIN method not suported

I'm getting this message when I'm trying to deploy my war artifact. My app is using hibernate ogm and it's trying to build persistence context when it's deployed. The message I'm getting is:
org.hibernate.service.spi.ServiceException: OGM000071: Unable to start datatore provider Caused by: org.hibernate.service.spi.ServiceException: OGM000071: Unable to start datatore provider Caused by: org.hibernate.HibernateException: OGM001214: Unable to connect to MongoDB instance: Timed out after 30000 ms while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}. Client view of cluster state is {type=UNKNOWN, servers=[{address=mongo:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSecurityException: Exception authenticating MongoCredential{mechanism=PLAIN, userName='living', source='lvdb', password=, mechanismProperties={}}}, caused by {com.mongodb.MongoCommandException: Command failed with error 2: 'Unsupported mechanism PLAIN' on server mongo:27017. The full response is { \"supportedMechanisms\" : [\"MONGODB-CR\", \"MONGODB-X509\", \"SCRAM-SHA-1\"], \"ok\" : 0.0, \"errmsg\" : \"Unsupported mechanism PLAIN\", \"code\" : 2, \"codeName\" : \"BadValue\" }}}] Caused by: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}. Client view of cluster state is {type=UNKNOWN, servers=[{address=mongo:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSecurityException: Exception authenticating MongoCredential{mechanism=PLAIN, userName='living', source='lvdb', password=, mechanismProperties={}}}, caused by {com.mongodb.MongoCommandException: Command failed with error 2: 'Unsupported mechanism PLAIN' on server mongo:27017. The full response is { \"supportedMechanisms\" : [\"MONGODB-CR\", \"MONGODB-X509\", \"SCRAM-SHA-1\"], \"ok\" : 0.0, \"errmsg\" : \"Unsupported mechanism PLAIN\", \"code\" : 2, \"codeName\" : \"BadValue\" }}}]"}}
What do I need to do in order to use the other mechanisms?
You'd have to specificy the property hibernate.ogm.mongodb.authentication_mechanism (see here).
Which is the authentication mechanism you'd want to use? We may not support it yet, but if you let us know about your preference, we can try and look into making it work quickly.
For MongoDB, default supported mechanism is 'SCRAM-SHA-1'. So, while connecting, you need to specify this mechanism. I saw the same error when connecting to MongoDB through spring-data. While I didn't make any special configuration for mongo, first and obvious option for me to try was PLAIN mechanism for authentication. Finally, resolved it using SCRAM-SHA-1.
For Spring-data, code looked something like this:
MongoCredential mongoCredential = MongoCredential.createScramSha1Credential(mongoUser, mongoDB, mongoPass.toCharArray());
Hope this helps.

When Connected to Network Mongodb Uses my NIC Address?

I have a Grails app that I am hooking up to a MongoDB instance. In my Datasource.groovy, I have the following declared:
grails {
mongo {
host = "localhost"
port = 27017
databaseName = "mmib-mongo"
}
}
Mongo is installed and starts just fine.
The problem is that when I am connected to a network, using grails run-app logs the following exception:
Packaging Grails application.....
| Running Grails application
Configuring Spring Security Core ...
... finished configuring Spring Security Core
Configuring Spring Security LDAP ...
... finished configuring Spring Security LDAP
| Error 2012-11-19 15:42:16,428 [localhost-startStop-1] ERROR context.GrailsContextLoader - Error executing bootstraps: Error creating bean with name 'mongoDatastore': FactoryBean threw exception on object creation; nested exception is org.springframework.dao.DataAccessResourceFailureException: can't call something : MBP-C02J91TWDKQ5.local/192.168.0.131:27017/mmib-mongo; nested exception is com.mongodb.MongoException$Network: can't call something : MBP-C02J91TWDKQ5.local/192.168.0.131:27017/mmib-mongo
Message: Error creating bean with name 'mongoDatastore': FactoryBean threw exception on object creation; nested exception is org.springframework.dao.DataAccessResourceFailureException: can't call something : MBP-C02J91TWDKQ5.local*/192.168.0.131:27017/mmib-mongo*; nested exception is com.mongodb.MongoException$Network: can't call something : MBP-C02J91TWDKQ5.local/192.168.0.131:27017/mmib-mongo
Any clue why when I'm connected to a network, Grails/Mongo wants to use that IP Address instead of localhost, which I verified is 127.0.0.1 in /etc/hosts
Homebrew tells MongoDB to listen on only on 127.0.0.1, not your public interface or a ".local" hostname:
https://github.com/mxcl/homebrew/blob/master/Library/Formula/mongodb.rb#L95
Try deleting the bind_ip line from your local copy of mongod.conf and simply protecting port 27017 with a firewall, or else change the "bind_ip" entry in mongod.conf.