spring data geode pool is not resolvable as a Pool in the application context - spring-data-gemfire

I've come back to a #SpringBootApplication project that uses spring-geode-starter with version 1.2.4 although the same error happens with upgrades to 1.5.6 version.
It sets up a Geode client using
#Component
#EnableClusterDefinedRegions(clientRegionShortcut=ClientRegionShortcut.PROXY)
and in order to register interest subscriptions over HTTP, also
#Configuration
#EnableGemFireHttpSession
with a bean
#Bean
public ReactiveSessionRepository<?> reactiveSessionRepository() {
return new ReactiveMapSessionRepository(new ConcurrentHashMap<>());
}
On starting the application the spring data geode client connects to the server (Geode version 1.14) and auto copies regions back to the client, which is great.
However, after all the region handles are copied over, there's an error with the #EnableGemFireHttpSession which is
Error creating bean with name 'ClusteredSpringSessions' defined in class path resource [org/springframework/session/data/gemfire/config/annotation/web/http/GemFireHttpSessionConfiguration.class] and [gemfirePool] is not resolvable as a Pool in the application context
The first info message in the logs is:
org.springframework.session.data.gemfire.config.annotation.web.http.GemFireHttpSessionConfiguration 1109 sessionRegionAttributes: Expiration is not allowed on Regions with a data management policy of PROXY
org.springframework.data.gemfire.support.AbstractFactoryBeanSupport 277 lambda$logInfo$3: Falling back to creating Region [ClusteredSpringSessions] in Cache [Web]
So the client is trying to create a region ClusteredSpringSessions but it can't. The problem appears to resolve itself if I define a connection pool for HTTP, with a pool connection bean like this
#Configuration
#EnableGemFireHttpSession(poolName="devPool")
public class SessionConfig {
#Bean
public ReactiveSessionRepository<?> reactiveSessionRepository() {
return new ReactiveMapSessionRepository(new ConcurrentHashMap<>());
}
#Bean("devPool")
PoolFactoryBean sessionPool() {
PoolFactoryBean pool = new PoolFactoryBean();
ConnectionEndpoint ce = new ConnectionEndpoint("1.2.3.4", 10334);
pool.setSubscriptionEnabled(true);
pool.addLocators(ce);
return pool;
}
}
There is still the Expiration is not allowed on Regions with a data management policy of PROXY info message in the log, but this time the Falling back to creating Region [ClusteredSpringSessions] in Cache [Web] appears to work.
I don't understand why a default pool can't connect.
If a pool is defined then in version 1.2.4 that can cause this issue.

Since you are using Spring Boot for Apache Geode (SBDG), which is an excellent choice (thank you), then you can simply include the spring-geode-starter-session dependency on your #SpringBootApplication classpath, which removes the need to explicitly annotate your Spring Boot application with SSDG's #EnableGemFireHttpSession annotation.
See here for more details. I also have a Sample application demonstrating the use of SSDG, here. The guide and source code for this example, along with other examples, can be found here).
Also, I would generally advise that users drive the GemFire/Geode cluster configuration from the application and not let the cluster dictate the Regions (and/or other components/configuration) that the client gets. However, SDG's #EnableClusterDefinedRegions annotation is provided and generally useful in the case you do not have control over the GemFire/Geode cluster your application is using. Still, in the (HTTP) Session UC, the GemFire/Geode cluster would need a Session Region (which defaults to "ClusteredSpringSessions" as determined by Spring Session for Apache Geode (SSDG) itself) anyway.
OK, now to the problem at hand...
I think what is happening here is, due to backwards compatibility and legacy reasons, Spring Data for Apache Geode (SDG), on which both SSDG and SBDG are based; SBDG additionally pulls in SSDG as well, defined a GemFire/Geode Pool by the name of "gemfirePool", specifically when using the SDG XML space and using/defining a DataSource configuration.
So, it is somewhat naively assumed users would be explicitly defining a Pool and calling it "gemfirePool", and not simply relying on a "default" Pool connection to the GemFire/Geode cache server (namely "localhost", 40404, or if using Locators (recommended), "localhost" and 10334).
However, for development purposes, and in SBDG specifically, I rely on the fact that GemFire/Geode creates a "DEFAULT" Pool anyway (when no explicit Pool is defined), and forgo the strict requirement that a "gemfirePool" should exist. But, SBDG builds on SSDG and SDG and they still rely on the legacy arrangement (for example).
I have filed an Issue ticket in SSDG to change this and better align SSDG with what SBDG prefers going forward, but I simply have not gotten around to it yet. My apologies for your inconvenience.
Anyway, it is a simple change you can make externally from your Spring Boot application, in application.properties like so (see here from the HTTP Session Sample I referenced from SBDG above). This will allow you to configure the Session Region Pool "name".
Also note, it is possible to change the name of the Session Region used by the client if what comes down from the cluster when you are using SDG's #EnableClusterDefinedRegions and the Region definition pulled down from the cluster is named differently on the server-side using this property.
Additionally, you can also configure the client Session Region data policy using properties as well (for example).
Regarding the Expiration "info" message you are seeing in the logs...
Since the client Session Region is a PROXY by default, then Expiration, Eviction and other Region data management policies (e.g. Compression, etc), do not actually make much sense.
In fact, SSDG is smart about whether to apply additional Region data management policies locally or not (see here, and specifically, this logic).
The message you are seeing in your application logs is in fact coming from SSDG, specifically. This message really serves as a reminder that your Session state management is actually "managed" on the server-side (when the application client is using a PROXY or even a CACHING_PROXY Region for that matter) and that the corresponding server-side, or cluster Sessions Region should be configured manually and appropriately, with Expiration policies as well as other things if necessary. Otherwise, no Session expiration would actually happen!
I hope all this makes since.
If you continue to have problems, feel free to file an Issue ticket and provide an example test or small application replicating your problem.

Related

Wildfly Elytron: Principal not available in SimpleSecurityManager

I implemented an authentication mechanism similar to CustomHeaderHttpAuthenticationMechanism in https://github.com/wildfly-security-incubator/elytron-examples/tree/master/simple-http-mechanism, using PasswordGuessEvidence and also the other Callbacks mentioned in the example. Reason for the custom mechanism is that beside a simple credential check we need also to validate more constraints to check if a user is validated.
Stepping through this authentication mechanism looks quite good, the authenticationComplete method is called and also the authorizeCallback is successful. However, when accessing an EJB via a resteasy endpoint (EJB is annotated with #SecurityDomain and #RolesAllowed...) the SimpleSecurityManager.authorize method fails because the securityContext.getUtil method neither provides a principal nor something else. If accessing a method annotated by #PermitAll it is successful.
I guess the principal should be created by the ServerAuthenticationContext when working through the different callbacks, right?
How do I manage that the SimpleSecurityManager can recognize the principal, would I need to create it in my authentication mechanism, and how?
In this case it sounds like your EJB deployment has not been mapped to the WildFly Elytron security domain so is still making use of PicketBox security in the EJB tier which is why you are not seeing the identity already established.
Within the EJB subsystem you can also add an application-security-domain mapping to map from the security domain specified in the deployment to the WildFly Elytron security domain.
FYI at some point in the future when we are ready to remove PicketBox from the server these additional mappings will no longer be required, they are just unfortunately needed at the moment whilst we have both solutions in parallel.

How to dynamically configure security for Artemis MQ addresses

Trying to dynamically create and provide security metadata for artemis mq topics (as opposed to defining them statically in broker.xml).
For that purpose I've implemented (as described here) the SecuritySettingPlugin interface.
Now, the issue is the getSecurityRoles/populateSecurityRoles of the implementation are called only at server startup.
So, at some point in time after the mq server has been started, a topic will be created :
org.apache.activemq.artemis.api.jms.management.JMSServerControl.createTopic("newTopic")
Now I would like artemis to call again my SecuritySettingPlugin implementation to get the updated security roles (which will include configuration for the newly created newTopic).
Is that possible ?
P.S. security-invalidation-interval does not invalidate roles configuration cache.
Seems there is a way to customize an address security by API :
ActiveMQServerControl.addSecuritySettings()

Implementation of Proxy on Liberty for Java

I use "Liberty for Java" app and Statica service(Proxy) on Bluemix.
We set http.proxyHost/http.proxyPort/https.proxyHost/https.proxyPort as system properties in Java code every transactions.
for example:
URL url = new URL(xxx);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
........
System.setProperty("http.proxyHost", host);
System.setProperty("http.proxyPort", port);
System.setProperty("https.proxyHost", host);
System.setProperty("https.proxyPort", port);
........
DataOutputStream out = new DataOutputStream(connection.getOutputStream());
I have an issue that one transaction go from the app to a target server directly in spite of tens of thousands of transactions passed the proxy.
Question 1:
Do "Liberty for Java" app on Bluemix clear or update system properties, http.proxyHost/http.proxyPort/https.proxyHost/https.proxyPort?
I wonder "Liberty for Java" app updated with null to access outer servers in multi-thread environment.
Question 2:
Do "Liberty for Java" app on Bluemix communicate with outer servers?
I found the following log in Statica.
https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.agents.na.apm.ibmserviceengage.com
https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.gateway.prd.na.ca.ibmserviceengage.com
( I masked a part of URL.)
P.S. We will change java code with ProxySelector class or Proxy class.
Re #1: No.
Re #2: Potentially yes. In your case, it seems your app is bound with a Monitoring & Analytics service? If so, a data collector will be installed and will send collected data to remote servers.
What's the reason that you need to set the proxy system properties in your code? Is it because you want some connections to go through the proxy and others not?
If so, then the way you do this is not right because the system proxy setting is a global setting, not a thread-scoped setting. This means if one thread sets the proxy setting, all threads will then use that proxy; if one thread unsets it, all threads will then do direct connections. That may explain why you are intermittently seeing some direct connections. The right way is to use a http client lib that supports proxy as parameters, like https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/config/RequestConfig.Builder.html#setProxy%28org.apache.http.HttpHost%29
If you want all connections to go through the http proxy, then you should simply set the JAVA_OPTS environment variable to pass in those system properties, e.g., "-Dhttp.proxyHost=x.x.x.x -Dhttp.proxyPort=xx".

How to use Apache-Commons DBCP with EclipseLink JPA and Tomcat 7.x

I've been working on a web application, deployed on Tomcat 7, which use EclipseLink JPA to handle the persistence layer.
Everything works fine in a test environment but we're having serious issues in the production environment due to a firewall cutting killing inactive connections. Basically if a connection is inactive for a while a firewall the sits between the Tomcat server and the DB server kill it, with the result of leaving "stale" connections in the pool.
The next time that connection is used the code never returns, until it gets a "Connection timed out" SQLException (full ex.getMessage() below).
EL Fine]: 2012-07-13
18:24:39.479--ServerSession(309463268)--Connection(69352859)--Thread(Thread[http-bio-8080-exec-5,5,main])--
MY QUERY REPLACED TO POST IT TO SO [EL Config]: 2012-07-13
18:40:10.229--ServerSession(309463268)--Connection(69352859)--Thread(Thread[http-bio-8080-exec-5,5,main])--disconnect
[EL Info]: 2012-07-13
18:40:10.23--UnitOfWork(1062365884)--Thread(Thread[http-bio-8080-exec-5,5,main])--Communication
failure detected when attempting to perform read query outside of a
transaction. Attempting to retry query. Error was: Exception
[EclipseLink-4002] (Eclipse Persistence Services -
2.3.0.v20110604-r9504): org.eclipse.persistence.exceptions.DatabaseException Internal
Exception: java.sql.SQLException: Eccezione IO: Connection timed out
I already tried several configuration in the persistence.xml, but since I have no access to the firewall configuration I had no luck with these methods. I also tried to use setCheckConnections()
ConnectionPool cp = ((JpaEntityManager)em).getServerSession().getDefaultConnectionPool();
cp.setCheckConnections();
cp.releaseConnection(cp.acquireConnection());
I managed to solve the issue in a test script using testOnBorrow, testWhileIdle and other features that are avalaible from DBCP Apache Commons. I'd like to know how to override the EclipseLink internal connection pool to use a custom connection pool so that I can provide an already configured pool, based on DBCP rather than just configuring the internal one using persistence.xml.
I know I should provide a SessionCustomizer, I'm uncertain which one is the correct pattern to use. Basically I would like to preserve the performance of DBCP in a JPA-like way.
I'm deploying on Tomcat 7, I know that if I switch to GF I won't have this problem, but for a matter of consistency with other webapp on the same server I'd prefere to stay on Tomcat.
What you want is definitely possible, but you might be hitting the limits of the "do it yourself" approach.
This is one of the more difficult things to explain, but there are effectively two ways to configure your EntityManagerFactory. The "do it yourself" approach and the "container" approach.
When you call Persistence.createEntityManagerFactory it eventually delegates to this method of the PersistenceProvider interface implemented by EclipseLink:
EntityManagerFactory createEntityManagerFactory(String emName, Map map)
The deal here is EclipseLink will then take it upon itself to do all the work, including its own connection creation and handling. This is the "do it yourself" approach. I don't know EclipseLink well enough to know if there is a way to feed it connections using this approach. After two days on Stackoverflow it doesn't seem like anyone else has that info either.
So here is why this "works in GF". When you let the container create the EntityManagerFactory for you by having it injected or looking it up, the container uses a different method on the PersistenceProvider interface implemented by EclipseLink:
EntityManagerFactory createContainerEntityManagerFactory(PersistenceUnitInfo info, Map map)
The long and short of it is that this PersistenceUnitInfo is an interface that the container implements and has these two very key methods on it:
public DataSource getJtaDataSource();
public DataSource getNonJtaDataSource();
With this mode EclipseLink will not try to do its own connection handling and will simply call these methods to get the DataSource from the container. This is really what you need.
There are two possible approaches you could take to solving this:
You could attempt to instantiate the EclipseLink PersistenceProvider implementation yourself and call the createContainerEntityManagerFactory method passing in your own implementation of the PersistenceUnitInfo interface and feed the DBCP configured DataSource instances into EclipseLink that way. You would need to parse the persistence.xml file yourself and feed that data in through the PersistenceUnitInfo. As well EclipseLink might also expect a TransactionManager, in which case you'll be stuck unless you hunt down a TransactionManager you can add to Tomcat.
You could use the Java EE 6 certified version of Tomcat, TomEE. DataSources are configured in the tomee.xml, created using DBCP with full support for all the options you need, and passed to the PersistenceProvider using the described createContainerEntityManagerFactory call. You then get the EntityManagerFactory injected via #PersistenceUnit or look it up.
If you do attempt to use TomEE, make sure your persistence.xml is updated to explicitly set transaction-type="RESOURCE_LOCAL" because the default is JTA. Even though it's non-compliant to use JTA with the Persistence.createEntityManagerFactory approach, there aren't any persistence providers that will complain and let you know you're doing something wrong, they treat it as RESOURCE_LOCAL ignoring the schema. So when you go to port your app to an actual certified server, it blows up.
Another note on TomEE is that in the current release, you'll have to put your EclipseLink libs in the <tomcat>/lib/ directory. This is fixed in trunk, just not released yet.
I'm not sure how useful these slides will be without the explanation that goes along with them, but the second part of this presentation is a deep dive into how container-managed EntityManager's work, specifically with regards to connection handling and transactions. You can ignore the transaction part as you aren't using them and already have an in production you're not likely to dramatically change, but it might be interesting for future development.
Best of luck!

How to programmatically un/register POJOs as services in JBoss 4.2.3.GA

I need to be able to circumvent the whole deployer malarkey and programmatically register/unregister (dependency-less) POJOs as services in JBoss.
Currently I'm dynamically creating an MBean interface and registering this with the JBoss MBeanServer, and then registering local/remote with Jndi.
This works ok (I can have a standard service from a vanilla SAR reference one of these service POJOs with the #EJB annotation) - however the container seems to leaves stale references behind as after calling unbind() and unregisterMBean().
Obviously I'm missing something by not dealing with the container in a way it expects, but what am I missing? Or is there an easier way (can't see much in the way of an API)?
thanks.