We are in the process of changing some of the applications that use a relation database, to OrientDB (2.1.9).
This transaction will be made in small steps, we will first start with the JDBC driver approach, and maybe we will switch to a native implementation later.
We have an application that is based on Spring Integration, where we changed from an Oracle Database to OrientDB, using the JDBC driver.
The database connections are managed by Tomcat 8 Connection Pool, and the transactional behavior is handled by Spring's Transaction Manager:
<bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource" destroy-method="close">
<property name="driverClassName" value="com.orientechnologies.orient.jdbc.OrientJdbcDriver" />
...
</bean>
<bean id="txManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
After changing the database to OrientDB, we encountered the following error:
com.orientechnologies.orient.core.exception.ODatabaseException: Database instance is not set in current thread. Assure to set it with: ODatabaseRecordThreadLocal.INSTANCE.set(db);
The OrientDB documentation, on Multi threading, states:
OrientDB supports multi-threads access to the database. ODatabase* and OrientGraph* instances are not thread-safe, so you've to get an instance per thread and each database instance can be used only in one thread per time. For more information about how concurrency is managed by OrientDB look at Concurrency.
Since v2.1 OrientDB doesn't allow implicit usage of multiple database instances from the same thread. Any attempt to manage multiple instances in the same thread must explicitly call the method db.activateOnCurrentThread() against the database instance BFORE you use it.
This posed a problem, as all the database connection management is handled by the connection pool (as it should be).
So, in order to implement the solution proposed by OrientDB, I had to have access to the underlying ODatabaseDocumentTx database, in order to apply the activateOnCurrentThread() method.
To be able to do it, I had to create a wrapper from both the JDBC driver class, and the OrientJdbcConnection.
In the OrientJdbcConnection, I obtained the underlying database by the use of Java Reflection:
public class OrientJdbcConnectionWrapper extends OrientJdbcConnection {
#Override
public void commit() throws SQLException {
final ODatabaseDocumentTx database = getPrivateDatabaseFieldByReflection();
database.activateOnCurrentThread();
super.commit();
}
And the JDBC Driver class wrapper, now returns a wrapped connection:
public class OrientJdbcDriverWrapper extends OrientJdbcDriver {
public Connection connect(final String url, final Properties info) throws SQLException {
if (!acceptsURL(url)) {
throw new SQLException(DO_NOT_ACCEPT_URL_ERROR + url);
}
return new OrientJdbcConnectionWrapper(url, info);
}
}
Finally, the configuration of the Connection pool now uses the wrapped driver:
<bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource" destroy-method="close">
<property name="driverClassName" value="x.y.z.driver.orientdb.OrientJdbcDriverWrapper" />
</bean>
As expected, this works perfectly, but i'm not entirely sure that this is the correct way to do it as i haven't see anyone with this issue.
On the other hand, the OrientDB documentation does suggest the use of activateOnCurrentThread() on a multi-threading environment.
OrientDB has its own database pool, that can be easily configured:
<bean id="dataSource" class="com.orientechnologies.orient.jdbc.OrientDataSource">
<constructor-arg name="url" value="jdbc:orient:plocal:xx\\databases\\test" />
<constructor-arg name="username" value="xx"/>
<constructor-arg name="password" value="xx"/>
<constructor-arg name="info" ref="databaseProperties" />
</bean>
<util:map id="databaseProperties" value-type="java.lang.String">
<entry key="db.usePool" value="true" />
<entry key="db.pool.min" value="1" />
<entry key="db.pool.max" value="10" />
Note: I'm using OrientDB through its orient-jdbc driver.
Related
I am connecting to an external database with a class that extends JdbcTemplate. My problem is that I can't use the globalProperties of the Groovy API because of the Jdbc.
I added these properties I needed in the server-config.properties:
studio.db.driverClassName
studio.db.url
studio.db.username
studio.db.password
I am trying to access them in my application-context.xml with this:
<bean id="jdbc" class="com.dbJdbcTemplate">
<constructor-arg ref="datasource"/>
</bean>
<bean id="datasource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="url" value="${studio.db.url}"/>
<property name="driverClassName" value="${studio.db.driverClassName}"/>
<property name="username" value="${studio.db.username}"/>
<property name="password" value="${studio.db.password}"/>
</bean>
I receive this error:
Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not
get JDBC Connection; nested exception
org.apache.commons.dbcp.SQLNestedException: Cannot load JDBC driver class '${studio.db.driverClassName}'
How do I access the properties from my bean correctly?
Add a <bean class="org.springframework.context.support.PropertySourcesPlaceholderConfigurer" parent="crafter.properties"/> in your site application-context.xml, like is shown in here https://docs.craftercms.org/en/3.0/site-administrators/engine/engine-site-configuration.html#id3. That lines gives you access to Engine's global properties.
I'm maintenance a old system it run in a jboss container,and it use ibatis and spring access an oracle db. Now this system's db related functions are discarded and the db will be closed. How should i do to safety disconnect this system with db (assuming the application code can deal with all exceptions except SqlException)
the key configuration is as follows:
xxx-ds.xml:
<datasources>
...
</datasources>
daoContext.xml:
<jee:jndi-lookup id="oracleSource" jndi-name="java:/DefaultDS"/>
<!-- SqlMap setup for iBATIS Database Layer -->
<bean id="sqlMapClient" class="org.springframework.orm.ibatis.SqlMapClientFactoryBean">
<property name="configLocation" value="META-INF/sql-map-config.xml"/>
</bean>
<bean id="xxxx" class="path.to.class.xxxxDao">
<property name="dataSource" ref="oracleSource"/>
<property name="sqlMapClient" ref="sqlMapClient"/>
</bean>
path.to.class.xxxxDao is extends org.springframework.orm.ibatis.support.SqlMapClientDaoSupport and implements db access methods.
Replace jndi data source with a mock db
I'm working on an existing Spring application that uses JDBC (DAO's extend NamedParameterJdbcDaoSupport). There were four datasources configured, each with it's own DataSourceTransactionManager. (though only one was registered with tx:annotation-driven for some reason)
I've recently added JPA (Spring-data-JPA) into the application and configured two entityManagerFactories (for now I don't need the other two datasources). I also configured two JpaTransactionManagers and removed the corresponding DataSourceTransactionManagers for these dataSources, since the JpaTransactionManagers can also be used for JDBC transactions. (correct me if I'm wrong)
It appears I need to be able to have distributed transactions, since the two datasources (to two different databases) need to be accessed (through JPA) in one service method. Since I did not have all I need to set up JTA (missing XA-driver for one of the databases) I've decided to give the Spring ChainedTransactionManager a try. Sadly this didn't work out as expected. All works fine if I just call a service method that only uses JPA.
Though when I call an existing service method that uses a JDBC find that has a class level #transactional annotation with it's propagation set to NOT_SUPPORTED and call another service method after that with a JPA call and a #transactional, I get an exception:
Caused by: java.lang.IllegalStateException: Already value [org.springframework.jdbc.datasource.ConnectionHolder#462cf9d9] for key [org.jboss.jca.adapters.jdbc.WrapperDataSource#3fbb4c32] bound to thread [http-/127.0.0.1:8080-5]
at org.springframework.transaction.support.TransactionSynchronizationManager.bindResource(TransactionSynchronizationManager.java:189) [spring-tx-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:403) [spring-orm-3.2.5.RELEASE.jar:3.2.5.RELEASE]
After some debugging, I found out that the transactions in Spring get added to a map on a ThreadLocal in the "TransactionSynchronizationManager.bindResource" method. The problem is that when using a JDBC call with #transactional and propogation NOT_SUPPORTED, a transaction is made anyway and registered through that method. When the JpaTransactionManager tries to bind it's resource, it is already on the map (and not marked as void) which causes the error to occur.
Changing the propagation to the default "REQUIRED" for the service call that encapsulates the JDBC call fixes the problem.
I have no idea why Spring is still creating that transaction when the transactional annotation is NOT_SUPPORTED. And if it creates that transaction, it should not bypass the JpaTransactionManager.
So what I'd like to know is if there is some way to tell Spring to use the JpaTransactionManager also when it creates a transaction itself inside the NamedParameterJdbcDaoSupport. (Well actually the JdbcDaoSupport... Well actually the DataSourceUtils)
We're using Spring 3.2.5, spring-data-jpa 1.6.0 and I've used Hibernate 4.2.0 as JpaVendor.
This problem doesn't occur without the ChainedTransactionManager.
Datasources:
<bean id="dataSourceCompta" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:jboss/datasources/comptaDS"/>
</bean>
<bean id="dataSourceUnisys" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:jboss/datasources/insoverDS"/>
</bean>
<bean id="dataSourceInsoverwebMysql" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:jboss/datasources/insoverWebDS"/>
</bean>
<bean id="dataSourceBatch" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:jboss/datasources/batchDS"/>
</bean>
Single remaining JDBC transaction manager (no JPA counterpart):
<bean id="transactionManagerBatch" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSourceBatch"/>
</bean>
JPA Transaction Managers:
<bean id="jpaUnisysTransactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactoryUnisys"/>
<qualifier value="unisys" />
</bean>
<bean id="jpaMysqlTransactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactoryMysql"/>
<qualifier value="mysql" />
</bean>
My ChainedTransactionManager:
<bean id="chainedTransactionManager" class="org.springframework.data.transaction.ChainedTransactionManager">
<constructor-arg>
<list>
<ref bean="jpaUnisysTransactionManager" />
<ref bean="jpaMysqlTransactionManager" />
</list>
</constructor-arg>
</bean>
JPA Entity manager factories:
<bean name="jpaVendorAdapter" class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"/>
<bean id="entityManagerFactoryUnisys" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="persistenceXmlLocation" value="classpath:META-INF/some-persistence.xml"/>
<property name="dataSource" ref="dataSourceUnisys"/>
<property name="persistenceUnitName" value="unisysPU"/>
<property name="jpaVendorAdapter" ref="jpaVendorAdapter"/>
<property name="jpaProperties">
<!-- properties -->
</property>
</bean>
<bean id="entityManagerFactoryMysql" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="persistenceXmlLocation" value="classpath:META-INF/some-persistence.xml"/>
<property name="dataSource" ref="dataSourceCompta"/>
<property name="persistenceUnitName" value="mysqlPU"/>
<property name="jpaVendorAdapter" ref="jpaVendorAdapter"/>
<property name="jpaProperties">
<!-- properties -->
</property>
</bean>
For now I've "fixed" this, by changing all the class-level transactional annotations to have propagation.REQUIRED (default) instead of NOT_SUPPORTED. Though I do not really like this solutions, since it might be somebody set those propagations to NOT_SUPPORTED with a good reason. I've also tried SUPPORTED, but using that had the same issue as NOT_SUPPORTED: a transaction was being made anyway by the Spring DataSourceUtils when the query was being executed by the NamedParameterJdbcDaoSupport DAO.
When no transactional annotation is set on the service, all works well too.
Our Camel Mongo DB component doesn't support multiple mongo db connections. By default it uses single Mongodb bean in all the mongo endpoint.
Issue:
I have created two different mongo bean instance for two database with respective authentication. And two mongodb beans instance are created in the spring.
When initializing/creating mongo endpoints(multiple endpoints with different DB) with DB its taking first mongo endpoint db connection to all the endpoint. So all the mongo endpoints are pointing to same database which was added first when creating first mongo endpoint.
Code below:
<bean id="DbLocal" name="DbLocal" class="com.mongodb.MongoClient">
<constructor-arg index="0">
<bean class="com.mongodb.MongoClientURI">
<constructor-arg index="0" value="mongodb://<user1>:<pwd1>#<host>:<port>/DB1" />
</bean>
</constructor-arg>
</bean>
<bean id="appDb" name="appDb" class="com.mongodb.MongoClient">
<constructor-arg index="0">
<bean class="com.mongodb.MongoClientURI">
<constructor-arg index="0" value="mongodb://<host>:<port>/DB2" />
</bean>
</constructor-arg>
</bean>
<route id="jobStart" startupOrder="1">
<from uri="direct:jobStart" />
<to uri="bean:jobMonitorInsertQueryBuilder" />
<to uri="mongodb:appDb?database={{requestDb}}&collection={{jobmonitorCollection}}&operation=insert"/>
</route
<route id="jobEnd" startupOrder="2">
<from uri="direct:jobEnd" />
<to uri="bean:jobMonitorUpdateQueryBuilder" />
<to uri="mongodb:DbLocal?database={{requestDb}}&collection={{jobmonitorCollection}}&operation=update" />
</route>
I tried by commenting out the if check in the camel mongo component thus allowing mongo connection look up for each mongo endpoint created in MongoDbComponent.java . This way it is working for multiple DB connections.
Any idea why mongo component is allowing only a single mongoDB connection in camel , is it because any existing issue?
protected Endpoint createEndpoint(String uri, String remaining, Map parameters) throws Exception {
// TODO: this only supports one mongodb
//if (db == null) {
db = CamelContextHelper.mandatoryLookup(getCamelContext(), remaining, Mongo.class);
LOG.debug("Resolved the connection with the name {} as {}", remaining, db);
//}
https://github.com/apache/camel/blob/master/components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbComponent.java
Appreciate any help in this regard.
Camel forum link : http://camel.465427.n5.nabble.com/Camel-mongo-db-does-not-support-multiple-mongo-DB-tp5765468.html
-Sree
This feature has been implemented in camel 2.18.2, 2.19.0. Kindly use these version to implement multiple DB's in your Project .
FYI :
https://issues.jboss.org/browse/ENTESB-5884
Jira Ticket Link - https://issues.apache.org/jira/browse/CAMEL-10644
Thanks,
Panneer
I use restlet in camel route in from("restlet:http/myLink") clause. When user's requests more then ten per second, I begin recieve errors processing request like a "org.restlet.engine.connector.Controller run
INFO: Connector overload detected. Stop accepting new work"
I think, that error is caused by number of threads,request query's size or number,or something like that. I try set to maxThreads param different values in spring config
<bean id="restlet" class="org.apache.camel.component.restlet.RestletComponent">
<property name="maxThreads" value="15"/>
</bean>
but I am not succeed. In documentation http://camel.apache.org/restlet.html I ddin't find ant param for setting size\number of request queue. I need help :(
P.S. camel-restlet version is 2.12.2
Update
I try to set big numbers to maxThreads,maxConnectionsPerHost,maxTotalConnections, but it's useless. If inject org.restlet.Component to camel's config like that:
<bean id="restletComponent" class="org.restlet.Component" />
<bean id="restlet" class="org.apache.camel.component.restlet.RestletComponent">
<constructor-arg index="0">
<ref bean="restletComponent" />
</constructor-arg>
<property name="maxThreads" value="255"/>
<property name="maxConnectionsPerHost" value="1000"/>
<property name="maxTotalConnections" value="1000" />
</bean>
How I can override properties, that use BaseHelper params?
After go through the options of lowThread as well.
But I found current released camel doesn't support it.