Quarkus Datasource using Unix Socket is ignored - postgresql

I am trying to connect quarkus reactive datasource using (Google Cloud SQLproxy).
You are able to run the cloud_sql_proxy as unix socket available connection.In this way:
./cloud_sql_proxy -dir=/tmp/cloudsql -instances=proyectA:europe-west1:cloudsql-sandbox-ephemeral -credential_file=../security/proyectA-credentials.json &
2020/06/30 12:39:27 Listening on /tmp/cloudsql/proyectA:europe-west1:cloudsql-sandbox-ephemeral/.s.PGSQL.5432 for proyectA:europe-west1:cloudsql-sandbox-ephemeral
2020/06/30 12:39:27 Ready for new connections
Then try to connect with this application.properties:
quarkus.datasource.db-kind=postgresql
quarkus.datasource.username=postgres
quarkus.datasource.password=123456
quarkus.datasource.reactive.url=postgresql:///postgres?cloudSqlInstance=/tmp/cloudsql/proyectA:europe-west1:cloudsql-sandbox-ephemeral/.s.PGSQL.5432
quarkus.datasource.jdbc=false
quarkus.datasource.reactive=true
running the Quarkus APP, get the error:
ion: Conexión refused: localhost/127.0.0.1:5432
Caused by: java.net.ConnectException: Conection refused
But, the message appears to try connect using TCP instead Unix socket. It is posible connect quarkus datasource via unix socket?
Java code is so simple
#Inject
private io.vertx.mutiny.pgclient.PgPool client;
(I can connect to cloud_sql_proxy using TCP without problems, but I need to configure unix socket to be able to deploy my quarkus app to cloud run with the cloud_sql_proxy)

To connect using a UNIX domain socket, the Reactive Pg client needs Netty native support.
First add the native transport jars to your project:
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-transport-native-epoll</artifactId>
<classifier>linux-x86_64</classifier>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-transport-native-kqueue</artifactId>
<classifier>osx-x86_64</classifier>
</dependency>
Then enable native transport in the Quarkus config file:
quarkus.vertx.prefer-native-transport=true
Finally, fix the reactive db url:
quarkus.datasource.reactive.url=postgresql://:5432/postgres?host=/tmp/cloudsql/proyectA:europe-west1:cloudsql-sandbox-ephemeral

Related

How to connect Mongodb Atlas to Spring

I have a web application which is using Spring Boot to handle the backend logics. I'm trying to integrate mongodb to track some information about the users of this webapp.
I created a database on mongodb Atlas and through the Mongo Shell the connection goes fine. The problem comes when I try to connect with Spring. Let me show you all the details
Inside Atlas, I added this IP Address (0.0.0.0/0 (includes your current IP address)) into Security > Network Address. In theory this should allow me to connect to the database from any IP address.
I then created a collection called "test".
If I click on my cluster and then on the connect button, it ask me with which modality I want to connect. I choose "Connect your application", and then I have to select the Driver and the Version. I choose respectively "Java" and "3.6 or later" (I'm not sure if it's the correct version, the alternatives are 3.4 or 3.3). And finally it shows me the connection string which is:
mongodb+srv://admin:<password>#umadit-obxpb.mongodb.net/test?retryWrites=true&w=majority
To connect to Atlas with Spring I'm using this dependency
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-mongodb</artifactId>
</dependency>
Inside the application.properties file I have these two lines to configure mongo.
spring.data.mongodb.host=mongodb+srv://admin:<password>#umadit-obxpb.mongodb.net/test?retryWrites=true&w=majority
spring.data.mongodb.port=27017
Instead of the password I put for obvious reasons.
The only problem is that when I start Spring Boot I continue to receive this error message:
2020-02-25 16:31:25.605 INFO 41162 --- [=majority:27017] org.mongodb.driver.cluster : Exception in monitor thread while connecting to server mongodb+srv://admin:<password>#umadit-obxpb.mongodb.net/test?retrywrites=true&w=majority:27017
com.mongodb.MongoSocketException: mongodb+srv://admin:<password>#umadit-obxpb.mongodb.net/test?retrywrites=true&w=majority: nodename nor servname provided, or not known
at com.mongodb.ServerAddress.getSocketAddress(ServerAddress.java:188) ~[mongo-java-driver-3.6.4.jar:na]
at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:59) ~[mongo-java-driver-3.6.4.jar:na]
at com.mongodb.connection.SocketStream.open(SocketStream.java:57) ~[mongo-java-driver-3.6.4.jar:na]
at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:126) ~[mongo-java-driver-3.6.4.jar:na]
at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:114) ~[mongo-java-driver-3.6.4.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
Caused by: java.net.UnknownHostException: mongodb+srv://admin:<password>#umadit-obxpb.mongodb.net/test?retrywrites=true&w=majority: nodename nor servname provided, or not known
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[na:1.8.0_111]
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) ~[na:1.8.0_111]
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) ~[na:1.8.0_111]
at java.net.InetAddress.getAllByName0(InetAddress.java:1276) ~[na:1.8.0_111]
at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[na:1.8.0_111]
at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[na:1.8.0_111]
at java.net.InetAddress.getByName(InetAddress.java:1076) ~[na:1.8.0_111]
at com.mongodb.ServerAddress.getSocketAddress(ServerAddress.java:186) ~[mongo-java-driver-3.6.4.jar:na]
... 5 common frames omitted
I don't know what to do in order to make it work. Am I missing something?
SOLUTION
As #barrypicker suggested, the problem was inside the properties file. Instead of using spring.data.mongodb.host I used spring.data.mongodb.uri. Now it works perfectly.
spring.data.mongodb.uri=mongodb+srv://admin:<password>#umadit-obxpb.mongodb.net/test?retryWrites=true&w=majority
even without spring.data.mongodb.port
Well, I think your to connect to Mongo Atlas your application.properties should have spring.data.mongodb.uri instead of spring.data.mongodb.host.spring.data.mongodb.uri: mongodb://<user>:<passwd>#<host>:<port>/<dbname>
I think this may work
`
Another issue with the atlas is that your IP is not allowed, make sure you add your IP at the network access tab.
Use this dependency inside "pom.xml" file
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>
Inside the "application.properties"
use "spring.data.mongodb.uri"
spring.data.mongodb.uri = mongodb+srv://<user>:<password>#<cluster_name>.mogodb.net/<dbname>

JMS 2.0 durable subscriptions topic best practice in Kubernetes

We are creating a Mule application which will be running in a container on Kubernetes and will be in a replica set that will be connecting to JMS 2.0 Red Hat AMQ 7 (based on ActiveMQ Artemis).
The pom.xml has been configured to get the jms client:
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>artemis-jms-client-all</artifactId>
<version>2.10.1</version>
</dependency>
And the JMS config is configured as:
<jms:config name="JMS_Config" doc:name="JMS Config" doc:id="8621b07d-b203-463e-bbbe-76eb03741a61" >
<jms:generic-connection specification="JMS_2_0" username="${mq.user}" password="${mq.password}" clientId="${mq.client.id}">
<reconnection >
<reconnect-forever frequency="${mq.reconnection.frequency}" />
</reconnection>
<jms:connection-factory >
<jms:jndi-connection-factory connectionFactoryJndiName="ConnectionFactory" >
<jms:name-resolver-builder jndiInitialContextFactory="org.apache.activemq.artemis.jndi.ActiveMQInitialContextFactory" jndiProviderUrl="${mq.brokerurl}"/>
</jms:jndi-connection-factory>
</jms:connection-factory>
</jms:generic-connection>
<jms:consumer-config>
<jms:consumer-type >
<jms:topic-consumer shared="true" durable="true"/>
</jms:consumer-type>
</jms:consumer-config>
<jms:producer-config persistentDelivery="true"/>
</jms:config>
Then in the JMS listener component:
<jms:listener doc:name="EMS JMS Listener" doc:id="318b4f08-daf6-41f4-944b-3ec1420d5c12" config-ref="JMS_Config" destination="${mq.incoming.queue}" ackMode="AUTO" >
<jms:consumer-type >
<jms:topic-consumer shared="true" subscriptionName="${mq.sub.name}" durable="true"/>
</jms:consumer-type>
<jms:response sendCorrelationId="ALWAYS" />
</jms:listener>
The variables are set as:
mq.client.id=client-id-135a9514-d4d5-4f52-b01c-f6ca34a76b40
mq.sub.name=my-sub
mq.incoming.queue=my-queue
Is this the best way to configure the client? As we have seen errors in the logs when deployed to K8s regarding connections to the AMQ server:
javax.jms.InvalidClientIDException: client-id-135a9514-d4d5-4f52-b01c-f6ca34a76b40 was already set into another connection
In JMS 2.0 you don't have to set the client identifier when creating a shared durable subscription. However, if you do set the client identifier then it must be unique per connection. For whatever reason (e.g. due to Mule or perhaps K8s) multiple connections are being created and since each connection is using the same client identifier you're receiving the javax.jms.InvalidClientIDException.
Remove clientId="${mq.client.id}" from your configuration and the javax.jms.InvalidClientIDException should go away.

java connection refused error with spring boot data geode remote locator

Per my question Apache Geode Web framework I've checked through various spring guides from here and spring data geode samples from here and written a short spring data geode application but it cannot connect to the remote GFSH started Geode locator. The Application class is:
package cm;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.data.gemfire.config.annotation.ClientCacheApplication;
import org.springframework.data.gemfire.config.annotation.ClientCacheApplication.Locator;
import org.springframework.data.gemfire.config.annotation.EnablePdx;
import org.springframework.data.gemfire.repository.config.EnableGemfireRepositories;
#SpringBootApplication
#ClientCacheApplication(name = "CmWeb", locators = #Locator, subscriptionEnabled = true)
#EnableGemfireRepositories(basePackageClasses= {CmRequest.class})
#EnablePdx
public class CmWeb {
public static void main(String[] args) {
SpringApplication.run(CmWeb.class, args);
}
}
and in the resources directory application.properties I've set up the remote locator:
# Configure the client's connection Pool to the servers in the cluster
spring.data.gemfire.pool.locators=1.2.3.4[10334]
Build and run the application and it discovers the locator (which it returns as the server name)
[Timer-DEFAULT-2] o.a.g.c.c.i.AutoConnectionSourceImpl : AutoConnectionSource discovered new locators [UAT:10334]
A couple of seconds later it throws the error:
[Timer-DEFAULT-2] o.a.g.c.c.i.AutoConnectionSourceImpl : locator UAT:10334 is not running.
and
java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[na:1.8.0_232]
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85) ~[na:1.8.0_232]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_232]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:204) ~[na:1.8.0_232]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_232]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[na:1.8.0_232]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_232]
at java.net.Socket.connect(Socket.java:607) ~[na:1.8.0_232]
at org.apache.geode.internal.net.SocketCreator.connect(SocketCreator.java:958) ~[geode-core-1.9.2.jar:na]
at org.apache.geode.internal.net.SocketCreator.connect(SocketCreator.java:899) ~[geode-core-1.9.2.jar:na]
at org.apache.geode.internal.net.SocketCreator.connect(SocketCreator.java:888) ~[geode-core-1.9.2.jar:na]
at org.apache.geode.distributed.internal.tcpserver.TcpClient.getServerVersion(TcpClient.java:290) ~[geode-core-1.9.2.jar:na]
at org.apache.geode.distributed.internal.tcpserver.TcpClient.requestToServer(TcpClient.java:184) ~[geode-core-1.9.2.jar:na]
at org.apache.geode.cache.client.internal.AutoConnectionSourceImpl.queryOneLocatorUsingConnection(AutoConnectionSourceImpl.java:209) [geode-core-1.9.2.jar:na]
at org.apache.geode.cache.client.internal.AutoConnectionSourceImpl.queryOneLocator(AutoConnectionSourceImpl.java:199) [geode-core-1.9.2.jar:na]
at org.apache.geode.cache.client.internal.AutoConnectionSourceImpl.queryLocators(AutoConnectionSourceImpl.java:287) [geode-core-1.9.2.jar:na]
at org.apache.geode.cache.client.internal.AutoConnectionSourceImpl$UpdateLocatorListTask.run2(AutoConnectionSourceImpl.java:500) [geode-core-1.9.2.jar:na]
at org.apache.geode.cache.client.internal.PoolImpl$PoolTask.run(PoolImpl.java:1371) [geode-core-1.9.2.jar:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_232]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_232]
at org.apache.geode.internal.ScheduledThreadPoolExecutorWithKeepAlive$DelegatingScheduledFuture.run(ScheduledThreadPoolExecutorWithKeepAlive.java:276) [geode-core-1.9.2.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_232]
After a lot of investigation I thought it was that the spring data geode client expects a spring boot geode server according to Connecting GemFire using Spring Boot and Spring Data GemFire and so I downloaded the ListRegionsOnServerFunction jar and deployed it on the GFSH server get the same result (have not yet restarted the server...) but that causes the same error condition.
If by Spring-Data-Gemfire - Unable to contact a Locator service. Operation either timed out or Locator does not exist I try and change the application.properties from
spring.data.gemfire.pool.locators=1.2.3.4[10334]
to
spring.gemfire.locators=1.2.3.4[10334]
or other variations then the app can't find the remote locator and throws:
[Timer-DEFAULT-3] o.a.g.c.c.i.AutoConnectionSourceImpl : locator localhost/127.0.0.1:10334 is not running.
Writing this question I've finally found How to connect a remote-locator in Geode and also can't PING the GFSH server from the SPRING app. However, the server bind address is setup properly for remote locator clients and various other services and UI using a locally built Geode Native Client for Geode v 1.10 can connect. I suspect PING may be disabled across this (semi-internal) network by default. I also disabled the firewall rules for ports 10334, 1099, 40404 to allow all traffic but still get the same error condition.
It turns out that from repeated INFO messages in the Spring Boot app after the connection refused:
[Timer-DEFAULT-2] o.a.g.c.c.i.AutoConnectionSourceImpl : updateLocatorInLocatorList changing locator list: loc form: LocatorAddress [socketInetAddress=UAT:10334, hostname=UAT, isIpString=false] ,loc to: UAT:10334
[Timer-DEFAULT-2] o.a.g.c.c.i.AutoConnectionSourceImpl : updateLocatorInLocatorList locator list from:[UAT:10334, /1.2.3.4:10334] to: [LocatorAddress [socketInetAddress=UAT:10334, hostname=UAT, isIpString=false], LocatorAddress [socketInetAddress=/1.2.3.4:10334, hostname=1.2.3.4, isIpString=true]]
and then running list clients on the server, the connection from the Spring Boot app to the Geode server v 1.10 is in fact established. Arrrgh!
It means the locator logic is working but this doesn't explain why after the first connection there's a java.net.ConnectException: Connection refused: connect error. Any ideas?
1 quick note about your Spring Boot application class...
#SpringBootApplication
#ClientCacheApplication(name = "CmWeb", locators = #Locator, subscriptionEnabled = true)
#EnableGemfireRepositories(basePackageClasses= {CmRequest.class})
#EnablePdx
public class CmWeb {
public static void main(String[] args) {
SpringApplication.run(CmWeb.class, args);
}
}
The following statements are true iff you are using Spring Boot for Apache Geode (or Pivotal GemFire), which is highly recommended.
When using SBDG (by declaring the correct org.springframework.geode:spring-geode-starter dependency on your application classpath), then you do not need to explicitly declare the #ClientCacheApplication, #EnableGemfireRepositories or the #EnablePdx annotations since SBDG auto-configures a ClientCache instance by default, auto-configures SD Repositories particularly when all entity classes are in the same package or sub-package as the Spring Boot app and SBDG auto-configures PDX by default, as well.
The locator = #Locator just specifies that the "DEFAULT" GemFire/Geode Pool when configured via the ClientCacheFactory should connect to the cluster via Locators, on localhost using the default Locator port, 10334. Therefore, this attribute is mostly useless and I would recommend the new #EnableClusterAware annotation from SBDG (see here).
The other attributes can be configured via Spring Boot application.properties, like so:
spring.application.name=CmWeb
spring.data.gemfire.pool.subscription-enabled=true
TIP: You can configure subscription on individually "named" Pools, even via properties, if you are using more than 1 Pool (of connections) in your application, perhaps to route different payloads based on workflows to different "grouped" servers in your cluster, etc.
You started to configure the "DEFAULT" Pool in application.properties already with...
# Configure the client's connection Pool to the servers in the cluster
spring.data.gemfire.pool.locators=1.2.3.4[10334]
Regarding...
After a lot of investigation I thought it was that the spring data geode client expects a spring boot geode server
No, SDG does not expect the cluster (of servers) to be configured or bootstrapped with Spring at all. Using Gfsh is perfectly valid. For instance. If the ListRegionsOnServerFunction is not available, SDG falls back to other means (provided by GemFire/Geode itself, which Gfsh knows and uses).
All the messages you are seeing in the Spring Boot app logs are coming from Geode itself, i.e. nothing to do with Spring. In a nutshell, and FWIW, SDG/SBDG is a facade around the Apache Geode (Pivotal GemFire) API and Java client driver. SDG/SBDG is at the mercy of this client doing the right thing, which of course, is partially dependent on proper configuration. Still... I am really just thinking out loud now since I suspect you are already well aware of (or have discovered) all of this.
I would also say the Java client and Native Client are not exactly an apple to apple comparison either. Meaning, if you developed a client using purely the Apache Geode (Pivotal GemFire) API without Spring, you'd have the exact same problem.
I have never seen a case where the first connection is establish but subsequent connections get a "Connection refused", o.O #argh
Have you tried this same configuration/arrangement with older Geode versions, e.g. 1.9?
Sorry for your troubles. I will think on this more.

JNDI port configuration for JBoss eap 6.4

We are migrating an application from JBoss AS 4.2 to JBoss eap 6.4. While deploying the application getting the below exception.
javax.naming.CommunicationException: Could not obtain connection to any of these urls: localhost:10099 and discovery failed with error: javax.naming.CommunicationException: Receive timed out [Root exception is java.net.SocketTimeoutException: Receive timed out] [Root exception is javax.naming.CommunicationException: Failed to connect to server localhost:10099 [Root exception is javax.naming.ServiceUnavailableException: Failed to connect to server localhost:10099 [Root exception is java.net.ConnectException: Connection refused: connect]]]
at org.jnp.interfaces.NamingContext.checkRef(NamingContext.java:1562)
at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:634)
at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:627)
at javax.naming.InitialContext.lookup(InitialContext.java:411)
The connection to localhost:10099 is failed. When I have checked in the jboss-service.xml file of AS 4.2 the configuration for this port is present.
<mbean code="org.jboss.naming.NamingService"
name="jboss:service=Naming"
xmbean-dd="resource:xmdesc/NamingService-xmbean.xml">
<!-- The call by value mode. true if all lookups are unmarshalled using
the caller's TCL, false if in VM lookups return the value by reference.
-->
<attribute name="CallByValue">false</attribute>
<!-- The listening port for the bootstrap JNP service. Set this to -1
to run the NamingService without the JNP invoker listening port.
-->
<attribute name="Port">10099</attribute>
Can anyone please tell me where can we do the similar configuration in EAP 6. Tried adding in socket-binding-group in standalone.xml but did not work.
Check the boot.log to see what the value of jboss.bind.address is.
There should be a line similar to:
DEBUG [ServerInfo] jboss.bind.address: 127.0.0.1
Telnet to the server on the JNDI port to confirm there is a service listening:1.telnet HOSTNAME/IP JNDI_PORT 2. JNDI by default would be on port 1099
Check the firewall rules at the host machine and make sure ports 10099 are opened in order for twiddle to work.

Post Deployment Configuration of MDB using JCA Inbound Socket Resource

I have a message driven bean that is being invoked from data received on a TCP socket via a JCA Resource Adapter in a GlassFish 3.1.1 application server. The deployment descriptor in the sun-ejb-jar.xml file, specifies the URL and port of the socket. When the MDB is deployed, I see the app server execute the endpointActivation() method in the resource adapter with the MessageEndpointFactory and ActivationSpec as parameters.
This works fine when the URL and port are known before deployment and can be specified in the XML file, but now I need the ability to change the TCP socket during runtime in my EJB code. I couldn't find any references in the JCA spec about how to do this during runtime.
For reference, I downloaded the specification from Oracle's website.
J2EE Connector Architecture Specification Version 1.5
Basically, I'd like to move the following XML code that populates the ActivationSpec into my EJB code.
<sun-ejb-jar>
<enterprise-beans>
<ejb>
<ejb-name>MyInboundSocketMDB</ejb-name>
<mdb-resource-adapter>
<resource-adapter-mid>jca-sockets-rar</resource-adapter-mid>
<activation-config>
<activation-config-property>
<activation-config-property-name>url</activation-config-property-name>
<activation-config-property-value>localhost</activation-config-property-value>
</activation-config-property>
<activation-config-property>
<activation-config-property-name>port</activation-config-property-name>
<activation-config-property-value>5006</activation-config-property-value>
</activation-config-property>
</activation-config>
</mdb-resource-adapter>
</ejb>
</enterprise-beans>
</sun-ejb-jar>
Thanks for any suggestions.
You can change your activation-config-property-value in the glassfish web admin.