Alfresco API to run faceted search - rest

I'm using Alfresco community 5.x version and I'm wondering if There are REST or any other remote alfresco apis to be able to run Faceted search.
I've seen some Restful apis to administer/manage some aspects of Faceted search viz : http://docs.alfresco.com/community5.0/references/RESTful-Facet.html
However no public APIs to run a faceted search.
I did notice that alfresco share fires the following against the core alfresco service to run its faceted search; but could not find any notes/docs related to that -
http://alfresco.mycompany.com/alfresco/s/slingshot/search
?facetFields={http://www.alfresco.org/model/content/1.0}creator,
{http://www.alfresco.org/model/content/1.0}content.mimetype,
{http://www.alfresco.org/model/content/1.0}created,
{http://www.alfresco.org/model/content/1.0}content.size,
{http://www.alfresco.org/model/content/1.0}modifier,
{http://www.alfresco.org/model/content/1.0}modified
&filters=
&term=wal
&tag=
&startIndex=0
&sort=
&site=
&rootNode=alfresco://company/home
&repo=false
&query=
&pageSize=25
&maxResults=0
&noCache=1455504682131
&spellcheck=true&
We have API based integration with Alfresco in our custom internal applications and don't use Alfresco Share.
I'm not sure if I should be using the above url or not.
Any suggestions on this?
Thanks!
Alfresco Version: 5.0.d

You can use the default search webscript:
webscripts\org\alfresco\slingshot\search\search.get.js
If you look at the code:
var params =
{
siteId: args.site,
containerId: args.container,
repo: (args.repo !== null) ? (args.repo == "true") : false,
term: args.term,
tag: args.tag,
query: args.query,
rootNode: args.rootNode,
sort: args.sort,
maxResults: (args.maxResults !== null) ? parseInt(args.maxResults, 10) : DEFAULT_MAX_RESULTS,
pageSize: (args.pageSize !== null) ? parseInt(args.pageSize, 10) : DEFAULT_PAGE_SIZE,
startIndex: (args.startIndex !== null) ? parseInt(args.startIndex, 10) : 0,
facetFields: args.facetFields,
filters: args.filters,
spell: (args.spellcheck !== null) ? (args.spellcheck == "true") : false
};
So if you present the right arguments with the facets to look for, then Alfresco will return the right faceted results.

I finally figured out how to implement and integrate the faceted search into a custom UI. the same also works with share.
create model in model manager (no hyphens)
Indexing attribute:
String: list of values whole match
Date, Number: enhanced search
For each type define the layout design - w/o this you wont be able to change type in share at least.
In share/search manager create filters/facets for fields you're interested in
Add a custom *context.xml to define a bean with your custom FacetQueryProvider implementation. inject that into the facet.solrFacetHelper bean
The custom FacetQueryProvider e.g. DollarAmountDisplayHandler basically provides facet queries based on the dollar amount buckets bean in the *context.xml, those will then be passed to solr.
Jar up the FacetQueryProvider implementation and copy to tomcat/lib directory.
Custom-solr-facets-context.xml
<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE beans PUBLIC '-//SPRING//DTD BEAN//EN' 'http://www.springframework.org/dtd/spring-beans.dtd'>
<beans>
<bean id="facet.dateFacetFields" class="org.springframework.beans.factory.config.SetFactoryBean">
<property name="sourceSet">
<set>
<value>#{http://www.alfresco.org/model/content/1.0}created</value>
<value>#{http://www.alfresco.org/model/content/1.0}modified</value>
<value>#{http://www.mycomp.com/model/hono/1.0}invoiceDate</value>
</set>
</property>
</bean>
<bean id="facet.dollarAmountBuckets" class="org.springframework.beans.factory.config.MapFactoryBean">
<property name="sourceMap">
<map>
<entry key="[0 TO 1000]" value="$0-$1K" />
<entry key="[1000 TO 10000]" value="$1K-$10K" />
<entry key="[10000 TO 100000]" value="$10K-$100K" />
<entry key="[100000 TO MAX]" value="Above.$100K" />
</map>
</property>
</bean>
<bean id="facet.dollarAmountDisplayHandler" class="com.mycomp.edm.alfresco.extensions.search.solr.facets.handlers.DollarAmountDisplayHandler" parent="baseFacetLabelDisplayHandler" >
<constructor-arg index="0">
<set>
<value>#{http://www.mycomp.com/model/hono/1.0}invoiceAmount</value>
</set>
</constructor-arg>
<constructor-arg index="1">
<ref bean="facet.dollarAmountBuckets" />
</constructor-arg>
</bean>
<bean id="facet.solrFacetHelper" class="org.alfresco.repo.search.impl.solr.facet.SolrFacetHelper" >
<constructor-arg>
<list>
<ref bean="facet.contentSizeBucketsDisplayHandler" />
<ref bean="facet.dateBucketsDisplayHandler" />
<ref bean="facet.dollarAmountDisplayHandler" />
</list>
</constructor-arg>
<property name="specialFacetIds">
<set>
<value>SITE</value>
<value>TAG</value>
<value>ANCESTOR</value>
<value>PARENT</value>
<value>ASPECT</value>
<value>TYPE</value>
<value>OWNER</value>
</set>
</property>
</bean>
</beans>
The model:
<?xml version="1.0" encoding="UTF-8"?>
<model xmlns="http://www.alfresco.org/model/dictionary/1.0" name="hon:hono">
<description>hono model</description>
<author>amit</author>
<imports>
<import uri="http://www.alfresco.org/model/content/1.0" prefix="cm"/>
<import uri="http://www.alfresco.org/model/dictionary/1.0" prefix="d"/>
</imports>
<namespaces>
<namespace uri="http://www.mycomp.com/model/hono/1.0" prefix="hon"/>
</namespaces>
<data-types/>
<constraints/>
<types>
<type name="hon:invoice">
<title>Invoice</title>
<description>invoice model</description>
<parent>cm:content</parent>
<properties>
<property name="hon:invoiceNumber">
<title>Invoice Number</title>
<type>d:int</type>
<mandatory>false</mandatory>
<index enabled="true">
<tokenised>TRUE</tokenised>
<facetable>true</facetable>
</index>
</property>
<property name="hon:invoiceAmount">
<title>Invoice Amount</title>
<type>d:int</type>
<mandatory>false</mandatory>
<index enabled="true">
<tokenised>TRUE</tokenised>
<facetable>true</facetable>
</index>
</property>
<property name="hon:invoiceDate">
<title>Invoice Date</title>
<type>d:date</type>
<mandatory>false</mandatory>
<index enabled="true">
<tokenised>TRUE</tokenised>
<facetable>true</facetable>
</index>
</property>
<property name="hon:organizationName">
<title>Organization Name</title>
<type>d:text</type>
<mandatory>false</mandatory>
<index enabled="true">
<tokenised>FALSE</tokenised>
<facetable>true</facetable>
</index>
</property>
<property name="hon:customerName">
<title>Customer Name</title>
<type>d:text</type>
<mandatory>false</mandatory>
<index enabled="true">
<tokenised>FALSE</tokenised>
<facetable>true</facetable>
</index>
</property>
</properties>
<associations/>
<overrides/>
<mandatory-aspects/>
</type>
</types>
<aspects/>
</model>
Facet Query Provider
package com.mycomp.edm.alfresco.extensions.search.solr.facets.handlers;
import org.alfresco.repo.search.impl.solr.facet.FacetQueryProvider;
import org.alfresco.repo.search.impl.solr.facet.SolrFacetConfigException;
import org.alfresco.repo.search.impl.solr.facet.handler.AbstractFacetLabelDisplayHandler;
import org.alfresco.repo.search.impl.solr.facet.handler.FacetLabel;
import org.springframework.extensions.surf.util.ParameterCheck;
import java.util.*;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
/**
* Created by Amit on 2/24/16.
*/
public class DollarAmountDisplayHandler extends AbstractFacetLabelDisplayHandler implements FacetQueryProvider {
private static final Pattern SIZE_RANGE_PATTERN = Pattern.compile("(\\[\\d+\\sTO\\s(\\d+|MAX)\\])");
private final Map<String, FacetLabel> facetLabelMap;
private final Map<String, List<String>> facetQueriesMap;
public DollarAmountDisplayHandler(Set<String> facetQueryFields, LinkedHashMap<String, String> dollarValueBucketMap)
{
System.out.println("instantiating bean DollarAmountDisplayHandler");
ParameterCheck.mandatory("facetQueryFields", facetQueryFields);
ParameterCheck.mandatory("dollarValueBucketMap", dollarValueBucketMap);
this.supportedFieldFacets = Collections.unmodifiableSet(facetQueryFields);
facetLabelMap = new HashMap<>(dollarValueBucketMap.size());
Map<String, List<String>> facetQueries = new LinkedHashMap<>(facetQueryFields.size());
for (String facetQueryField : facetQueryFields)
{
List<String> queries = new ArrayList<>();
int index = 0;
for (Map.Entry<String, String> bucket : dollarValueBucketMap.entrySet())
{
String sizeRange = bucket.getKey().trim();
Matcher matcher = SIZE_RANGE_PATTERN.matcher(sizeRange);
if (!matcher.find())
{
throw new SolrFacetConfigException(
"Invalid dollar value range. Example of a valid size range is: [0 TO 1000]");
}
// build the facet query. e.g. {http://www.mycomp.com/model/hono/1.0}invoiceAmount:[0 TO 1000]
String facetQuery = facetQueryField + ':' + sizeRange;
queries.add(facetQuery);
// indexOf('[') => 1
String sizeRangeQuery = sizeRange.substring(1, sizeRange.length() - 1);
sizeRangeQuery = sizeRangeQuery.replaceFirst("\\sTO\\s", "\"..\"");
facetLabelMap.put(facetQuery, new FacetLabel(sizeRangeQuery, bucket.getValue(), index++));
}
facetQueries.put(facetQueryField, queries);
}
this.facetQueriesMap = Collections.unmodifiableMap(facetQueries);
System.out.println("Bean DollarAmountDisplayHandler instantiated");
}
#Override
public FacetLabel getDisplayLabel(String value)
{
FacetLabel facetLabel = facetLabelMap.get(value);
return (facetLabel == null) ? new FacetLabel(value, value, -1) : facetLabel;
}
#Override
public Map<String, List<String>> getFacetQueries()
{
return this.facetQueriesMap;
}
}

Related

Alfresco with custom authentication webservice

We have a custom authentication service which accepts a parameter over a REST service and provides a json response. Based on the response we redirect the user. I wish to integrate this webservice with our new Alfresco Box.
Is there a way to do custom base authentication instead of using LDAP or inbuilt SSO plugin?
If I understood correctly...
Implement your custom remote user mapper:
package best.package.ever;
import org.alfresco.repo.security.authentication.external.DefaultRemoteUserMapper;
public class CustomRemoteUserMapper extends DefaultRemoteUserMapper {
public boolean canHandle(HttpServletRequest request) {
return true; // TODO
}
public String getTrustedUserId(HttpServletRequest request) {
// TODO: validate & authorize... e.g. based on trusted Json Web Token
return "trusted.user.id"; // TODO
}
public String getRemoteUser(HttpServletRequest request) {
if (canHandle(request)) {
return getTrustedUserId(request);
} else {
return super.getRemoteUser(request);
}
}
}
Put your library best-package-ever.jar into tomcat\webapps\alfresco\WEB-INF\lib
Declare your custom bean and inject it into RemoteUserMapper bean: tomcat\shared\classes\alfresco\extension\authentication-custom-context.xml
<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE beans PUBLIC '-//SPRING//DTD BEAN//EN' 'http://www.springframework.org/dtd/spring-beans.dtd'>
<beans>
<bean id="customRemoteUserMapper" class="best.package.ever.CustomRemoteUserMapper">
<property name="personService" ref="PersonService"/>
<property name="authorityService" ref="AuthorityService" />
</bean>
<bean id="RemoteUserMapper" class="org.alfresco.repo.management.subsystems.ChainingSubsystemProxyFactory">
<property name="applicationContextManager">
<ref bean="Authentication" />
</property>
<property name="interfaces">
<list>
<value>org.alfresco.repo.security.authentication.external.RemoteUserMapper</value>
<value>org.alfresco.repo.management.subsystems.ActivateableBean</value>
</list>
</property>
<property name="sourceBeanName">
<value>customRemoteUserMapper</value> <!-- extending remoteUserMapper -->
</property>
</bean>
</beans>

Spring-batch Entity Manager becomes null after init

I'm currently implementing a Spring-batch that reads and writes to files BUT also needs to do CRUD operations on a database.
I've tried to simply define an Entity manager in my xml configuration, and use it in my DAO class. However, right after the init, the EntityManager becomes null.
Can anyone help me with this ? (a solution or a link via something usable would be perfect).
My batchContext.xml
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="${batch.datasource.driverClassName}"/>
<property name="url" value="${batch.datasource.url}"/>
<property name="username" value="${batch.datasource.username}"/>
<property name="password" value="${batch.datasource.password}"/>
<property name="testOnBorrow" value="true"/>
<property name="testOnReturn" value="true"/>
<property name="testWhileIdle" value="true"/>
<property name="timeBetweenEvictionRunsMillis" value="1800000"/>
<property name="numTestsPerEvictionRun" value="3"/>
<property name="minEvictableIdleTimeMillis" value="1800000"/>
</bean>
<bean id="entityManagerFactory" name="entTransactionMgr" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<!-- <property name="persistenceXmlLocation" value="classpath:/META-INF/spring/persistence.xml" /> -->
<property name="persistenceUnitName" value="persistenceUnit"/>
<property name="packagesToScan" value="${jpa.scan.packages}"/>
<property name="dataSource" ref="dataSource"/>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"/>
</property>
<property name="loadTimeWeaver">
<bean class="org.springframework.instrument.classloading.InstrumentationLoadTimeWeaver" />
</property>
<!-- Custom jpaDialect pour le deuxieme batch:job-repository-->
<property name="jpaDialect">
<bean class="fr.mma.soecm.batchfacade.util.CustomHibernateJpaDialect" />
</property>
<property name="jpaProperties">
<props>
<!-- multiple props here-->
</props>
</property>
</bean>
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory"/>
</bean>
<!-- mode="aspectj" -->
<tx:annotation-driven transaction-manager="transactionManager"/>
<batch:job-repository id="jobRepository" data-source="dataSource" transaction-manager="transactionManager"/>
<!-- Jobs held separatelly -->
<import resource="batchContext-job.xml"/>
My DAO
#Repository("batchJobDao")
#Transactional
public class BatchJobDaoImpl implements BatchJobDao{
#PersistenceContext(unitName="persistenceUnit")
#Autowired
private EntityManager entityManager;
// #PersistenceContext(unitName="persistenceUnit", type=PersistenceContextType.EXTENDED)
// public void setEntityManager(EntityManager entityManager) {
// System.out.println("Setting Entity Manager :" + entityManager);
// this. entityManager = entityManager;
// }
#PostConstruct
public void init(){
if (entityManager == null){
System.out.println(" Entity Manager is null");
} else {
System.out.println(" Entity Manager is not null");
getAllJobExecutions();
}
}
private static final String RECHERCHER_JOB_EXECUTION = "Select bje from BatchJobExecution bje";
#SuppressWarnings("unchecked")
#Override
#Transactional("transactionManager")
public List<BatchJobExecution> getAllJobExecutions(){
// EntityManagerFactory emf=Persistence.createEntityManagerFactory("entTransactionMgr");
// EntityManager em=emf.createEntityManager();
Query query = entityManager.createQuery(RECHERCHER_JOB_EXECUTION, BatchJobExecution.class);
List<BatchJobExecution> executions = (List<BatchJobExecution>) query.getResultList();
System.out.println("EXES : " + executions);
return executions;
}
}
There is some code commented out because I've tried multiple aproaches (changing the persistence context type, recovering the entity manager "manually", having a persistence.xml file for the entityManager) without succes.
My output when running the job is (without all the extra lines..):
Entity Manager is not null
EXES : [fr.mma.soecm.batchfacade.domain.BatchJobExecution#10e6c33,...]
[BatchService] - Synchronous job launch
[AbstractStep] - Encountered an error executing the step
Caused by: java.lang.NullPointerException
And the null pointer, on debug, is throws by the EntityManager being null when I call the "createQuery" in my DAO.
Thanks for your help.
I'll keep searching on my end.. God Speed!
As mentioned in the comment above, my aparent problem was due to the fact that I was trying to call the Service or the DAO in the Constructor of the Step.
After moving the Service call in the "doRead()" method, I could perform all CRUD operations with the EntityManager.
Please let me know if anyone has questions about this/and how to make it work otherwise, as I've not found any explanation on the internet, since I've began searching last week.

Spring Batch: DAO call from processor slows down batch process -- can I reuse DB connection?

When I add a DAO call in the MdwValidatingItemProcessor below, I get a serious hit in performance. Prior to adding the additional DAO call, 2500 records where being read/processed/written in 40ish seconds. When I add the one additional DAO call (lobPolicyMapper.getPolicyOrigin(item.getLob(),item.getRgn_cd())
), it becomes 120 seconds. And the lob_policy table only has one record in it. I plan to have many such DAO calls in my processor to do additional validation of the item. Can I reuse the DAO's connection so for each processor that is validating an item, the connection doesn't have to be opened and closed constantly? The AsyncItemProcessor delegates to the MdwValidatingItemProcessor:
public class MdwValidatingItemProcessor implements
ItemProcessor<MecMdw, MecMdw> {
#Autowired
DataSourceTransactionManager txManager;
#Autowired
LobPolicyMapper lobPolicyMapper;
#Autowired
MecUtils mecUtils;
#Autowired
LobShopMapper lobShopMapper;
#Autowired
KaiserIssuerMapper kaiserIssuerMapper;
private Validator validator;
public void setValidator(Validator validator) {
this.validator = validator;
}
private List<Map> validationReasons;
#BeforeStep
public void beforeStep(StepExecution stepExecution) {
validationReasons = (List<Map>) stepExecution.getJobExecution().getExecutionContext().get("validationReasons");
}
public MecMdw process(MecMdw item) throws Exception {
DefaultTransactionDefinition def = new DefaultTransactionDefinition();
def.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRED);
TransactionStatus status = txManager.getTransaction(def);
BindingResult results = BindAndValidate(item);
try {
if (results.hasErrors()) {
buildValidationException(results, item);
return setAsKickOut(item);
}
mecUtils.checkForPaddingAndValidateSSN(item);
//if both SSN and DOB are valid, set Rep_DOB_or_SSN to be SSN
if (item.getInvalid_ssn().equals(MecConstants.Y_FLAG) && mecUtils.isDOBValid(item)) {
item.setRep_doborssn("SSN");
}
else if(mecUtils.isDOBValid(item)){
item.setRep_doborssn("DOB");
}
else{
List<String> listOfErrors = new ArrayList<String>();
listOfErrors.add("BothSSNAndDOBInvalid");
item.setValidationErrors(listOfErrors);
return setAsKickOut(item);
}
// Policy Origin code not found in MEC database based on Region /
// LOB.
LobPolicy lobPolicy = lobPolicyMapper.getPolicyOrigin(
item.getLob(), item.getRgn_cd());
if (lobPolicy == null || ("").equals(lobPolicy.getLob())
|| lobPolicy.getLob() == null) {
return setAsKickOut(item);
}
//set the Rep_PolicyOgnCd
item.setRep_policy_ogn_cd(lobPolicy.getPolicyOrigin());
//If origin of policy = SHOP, look for Shop Identifier.(mec_lob_shop table)
if(("SHOP").equals(lobPolicy.getPolicyOrigin())){
if(lobShopMapper.getValidationShopIdentifier(item) == null){
return setAsKickOut(item);
}
}
//Validation Shop Identifier not found in MEC database based on Region/LOB/PID
if(lobShopMapper.getValidationShopIdentifier(item) == null){
return setAsKickOut(item);
}
//Kaiser Issuer EIN not found in MEC database based on Region/LOB.
Integer kaiserIssuer = kaiserIssuerMapper.getKaiserIssuerIdWithLobAndRegion(item);
if(kaiserIssuer == 0){
return setAsKickOut(item);
}
else{
item.setRep_kaiser_issuer_id(kaiserIssuer.toString() );
}
} catch (Exception ex) {
ex.printStackTrace();
} finally {
txManager.commit(status);
}
return item;
}
private MecMdw setAsKickOut(MecMdw item) {
item.setKick_out_fl('Y');
return item;
}
private BindingResult BindAndValidate(MecMdw item) {
DataBinder binder = new DataBinder(item);
binder.setValidator(validator);
binder.validate();
return binder.getBindingResult();
}
private void buildValidationException(BindingResult results, MecMdw item) {
List<String> listOfErrors = new ArrayList<String>();
for (ObjectError error : results.getAllErrors()) {
String[] codes = error.getCodes();
listOfErrors.add(codes[1]);
}
item.setValidationErrors(listOfErrors);
// throw new ValidationException(msg.toString());
}
I have a batch job using AsyncItemProcessor and AsyncItemWriter as follows:
<job id="mecmdwvalidatorJob" xmlns="http://www.springframework.org/schema/batch">
<step id="mdwvalidatorStep1">
<tasklet>
<chunk reader="pageItemReader" processor="asyncItemProcessor"
writer="asynchItemWriter" commit-interval="1000" skip-limit="2147483647">
<skippable-exception-classes> <!-- TODO -->
<include class="java.lang.Exception" />
</skippable-exception-classes>
</chunk>
</tasklet>
</step>
</job>
<bean id="pageItemReader"
class="org.springframework.batch.item.database.JdbcPagingItemReader">
<property name="dataSource" ref="dataSource" />
<property name="queryProvider">
<bean
class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="selectClause"
value="select MDW_ID,FK_LOG_FILE_ID,TAX_YEAR,SUBS_TYPE_CD,SUB_FIRST_NM,SUB_MIDDLE_NM,SUB_LAST_NM,SUB_SUFFIX,SUB_DOB,SUB_ADDR1,SUB_ADDR2,SUB_CITY,SUB_STATE,SUB_PROVINCE,SUB_ZIP,SUB_ZIP4,SUB_COUNTRY_CD,SUB_COUNTRY,SUB_F_POSTAL_CD,LOB,SUB_SSN,GRP_EMP_NAME1,GRP_EMP_NAME2,GRP_EIN,GRP_ADDR1,GRP_ADDR2,GRP_CITY,GRP_STATE,GRP_PROVINCE,GRP_ZIP,GRP_ZIP4,GRP_COUNTRY_CD,GRP_COUNTRY,GRP_F_POSTAL_CD,ISSUER_NAME1,ISSUER_NAME2,ISSUER_PHONE,ISSUER_ADDR1,ISSUER_ADDR2,ISSUER_CITY,ISSUER_PROVINCE,ISSUER_ZIP,ISSUER_ZIP4,ISSUER_COUNTRY_CD,ISSUER_COUNTRY,ISSUER_F_POSTAL_CD,MEM_FIRST_NM,MEM_MIDDLE_NM,MEM_LAST_NM,MEM_SUFFIX,MEM_SSN,MEM_DOB,MEM_START_DATE,MEM_END_DATE,REGION_CD,SUB_MRN,SUB_MRN_PREFIX,MEM_MRN,MRN_PREFIX,PID,SUB_GRP_ID,SUB_GRP_NAME,INVALID_ADDR_FL" />
<property name="fromClause"
value="from MEC_MDW JOIN MEC_FILE_LOG on MEC_FILE_LOG.LOG_FILE_ID=MEC_MDW.FK_LOG_FILE_ID " />
<property name="whereClause" value="where MEC_FILE_LOG.STATUS=:status" />
<property name="sortKey" value="MDW_ID" />
</bean>
</property>
<property name="parameterValues">
<map>
<entry key="status" value="READY TO VALIDATE" />
</map>
</property>
<property name="pageSize" value="1000" />
<property name="rowMapper" ref="mdwRowMapper" />
</bean>
<bean id="mdwRowMapper" class="org.my.rowmapper.MdwRowMapper" />
<bean id="asyncItemProcessor"
class="org.springframework.batch.integration.async.AsyncItemProcessor">
<property name="delegate">
<bean
class="org.my.itemprocessor.MdwValidatingItemProcessor">
<property name="validator">
<bean
class="org.springframework.validation.beanvalidation.LocalValidatorFactoryBean" />
</property>
</bean>
</property>
<property name="taskExecutor" ref="taskExecutor" />
</bean>
<task:executor id="taskExecutor" pool-size="10" />
<bean id="asynchItemWriter"
class="org.springframework.batch.integration.async.AsyncItemWriter">
<property name="delegate" ref="customerCompositeWriter">
</property>
</bean>
<bean id="customerCompositeWriter"
class="org.springframework.batch.item.support.CompositeItemWriter">
<property name="delegates">
<list>
<ref bean="itemWriter1" />
<ref bean="itemWriter2" />
</list>
</property>
</bean>
<bean id="itemWriter1" class="org.my.writer.MdwWriter" />
<bean id="itemWriter2" class="org.my.writer.KickoutWriter" />
The transaction manager and MyBatix SqlSessionTemplate:
<!-- transaction manager, use JtaTransactionManager for global tx -->
<bean id="transactionManager"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
<!-- define SqlSessionFactory as BATCH execution -->
<bean id="sqlSession" class="org.mybatis.spring.SqlSessionTemplate">
<constructor-arg index="0" ref="sqlSessionFactory" />
<constructor-arg index="1" value="BATCH" />
</bean>
<!-- stored job-meta in database -->
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="transactionManager" ref="batchTransactionManager" />
<property name="databaseType" value="sqlserver" />
</bean>
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
The DAO making the call:
public interface LobPolicyMapper {
public abstract LobPolicy getPolicyOrigin(#Param("lob") String lob, #Param("regionCd") String regionCd);
UPDATE I added ehcache to the the MyBatis XML. This will help with the repetitive queries. But again, I am looking for a way to share this same DAO's connection across the AsyncItemProcessors:
<mapper namespace="org.mybatis.LobPolicyMapper">
<cache type="org.mybatis.caches.ehcache.EhcacheCache"/>
<select id="getPolicyOrigin" parameterType="hashmap" resultType='org.my.domain.LobPolicy'>
SELECT
l.lob_id as lobId,
l.lob as lob,
l.policy_origin as policyOrigin,
l.region_cd as regionCd
from mec_lob_policy l
where lob= #{lob} and region_cd=#{regionCd}
</select>
I added ehcache to the project
<dependency>
<groupId>org.mybatis</groupId>
<artifactId>mybatis-ehcache</artifactId>
<version>1.0.0</version>
</dependency>
I think I need a database pool. Here is what I am seeing in my logs on exiting the processor:
2015-08-27 17:09:54,194 [taskExecutor-3] DEBUG org.mybatis.spring.SqlSessionUtils - Releasing transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession#77b027f]
2015-08-27 17:10:06,674 [taskExecutor-3] DEBUG org.mybatis.spring.SqlSessionUtils - Transaction synchronization committing SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession#77b027f]
2015-08-27 17:10:06,675 [taskExecutor-3] DEBUG org.mybatis.spring.SqlSessionUtils - Transaction synchronization deregistering SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession#77b027f]
2015-08-27 17:10:06,675 [taskExecutor-3] DEBUG org.mybatis.spring.SqlSessionUtils - Transaction synchronization closing SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession#77b027f]
2015-08-27 17:10:06,675 [taskExecutor-3] DEBUG org.springframework.jdbc.datasource.DataSourceTransactionManager - Initiating transaction commit
2015-08-27 17:10:06,675 [taskExecutor-3] DEBUG org.springframework.jdbc.datasource.DataSourceTransactionManager - Committing JDBC transaction on Connection [ConnectionID:14 ClientConnectionId: 1b95dc0e-83c0-487b-af59-f5be52931818]
2015-08-27 17:10:06,805 [taskExecutor-3] DEBUG org.springframework.jdbc.datasource.DataSourceTransactionManager - Releasing JDBC Connection [ConnectionID:14 ClientConnectionId: 1b95dc0e-83c0-487b-af59-f5be52931818] after transaction
2015-08-27 17:10:06,805 [taskExecutor-3] DEBUG org.springframework.jdbc.datasource.DataSourceUtils - Returning JDBC Connection to DataSource
Here is what I am seeing in my logs on entering the processor. New transaction is created and a new database connection is fetched:
2015-08-27 17:10:06,805 [taskExecutor-3] DEBUG org.springframework.jdbc.datasource.DataSourceTransactionManager - Creating new transaction with name [null]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
2015-08-27 17:10:06,805 [taskExecutor-3] DEBUG org.springframework.jdbc.datasource.DriverManagerDataSource - Creating new JDBC DriverManager Connection to [jdbc:sqlserver://blah]
2015-08-27 17:10:07,115 [taskExecutor-3] DEBUG org.springframework.jdbc.datasource.DataSourceTransactionManager - Acquired Connection [ConnectionID:23 ClientConnectionId: d1f016e6-3e9d-4b0e-a34d-14298c292a65] for JDBC transaction
2015-08-27 17:10:07,115 [taskExecutor-3] DEBUG org.springframework.jdbc.datasource.DataSourceTransactionManager - Switching JDBC Connection [ConnectionID:23 ClientConnectionId: d1f016e6-3e9d-4b0e-a34d-14298c292a65] to manual commit

Spring Batch FlatFileItemWriter - How to use stepExecution.jobId to generate file name

I have this FileWriter where I'm trying to append the current Job Id to the filename that is generated.
<bean id="csvFileWriter" class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">
<property name="resource">
<bean class="org.springframework.core.io.FileSystemResource">
<constructor-arg type="java.lang.String">
<value>${csv.file}_#{stepExecution.jobExecution.jobId}</value>
</constructor-arg>
</bean>
</property>
<property name="lineAggregator">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="delimiter">
<util:constant
static-field="org.springframework.batch.item.file.transform.DelimitedLineTokenizer.DELIMITER_COMMA"/>
</property>
<property name="fieldExtractor">
<bean class="org.springframework.batch.item.file.transform.PassThroughFieldExtractor" />
</property>
</bean>
</property>
....
....
but it's bombing out with
Caused by: org.springframework.expression.spel.SpelEvaluationException: EL1008E:(pos 0): Field or property 'stepExecution' cannot be found on object of type 'org.springframework.beans.factory.config.BeanExpressionContext'
at org.springframework.expression.spel.ast.PropertyOrFieldReference.readProperty(PropertyOrFieldReference.java:208)
at org.springframework.expression.spel.ast.PropertyOrFieldReference.getValueInternal(PropertyOrFieldReference.java:72)
at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:52)
at org.springframework.expression.spel.ast.SpelNodeImpl.getTypedValue(SpelNodeImpl.java:102)
at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:97)
at org.springframework.expression.common.CompositeStringExpression.getValue(CompositeStringExpression.java:82)
at org.springframework.expression.common.CompositeStringExpression.getValue(CompositeStringExpression.java:1)
at org.springframework.context.expression.StandardBeanExpressionResolver.evaluate(StandardBeanExpressionResolver.java:139)
... 45 more
Any idea how i can correctly reference the jobId in this case?
Update: Adding worked solution
I implemented the JobExecutionListener which adds the jobId to the ExecutionContext
public class MyExecutionListener implements JobExecutionListener {
public void beforeJob(JobExecution jobExecution) {
long jobId = jobExecution.getJobId();
jobExecution.getExecutionContext().put("jobId",jobId);
jobExecution.getExecutionContext().put("date",date);
}
public void afterJob(JobExecution jobExecution) {
Register the listener to the batch job
<batch:job id="batchJob">
<batch:listeners>
<batch:listener ref="myExecutionListener"/>
</batch:listeners>
And finally the CSV writer gets updated to
<bean id="fundAssetCsvFileWriter"
class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">
<property name="resource">
<bean class="org.springframework.core.io.FileSystemResource">
<constructor-arg value="${csv.file.name}_#{jobExecutionContext['date']}_#{jobExecutionContext['jobId']}.csv" type="java.lang.String"/>
</bean>
The supported names for late-bindig are:
#{jobParameters}
#{jobExecutionContext}
#{stepExecutionContext}
If jobId is not directly accessible, look this question.
Also, resource can be injected directly as
<property name="resource">
<value>file://${csv.file}_#{jobExecutionContext['jobId']}</value>
</property>
because the right resource type is created using a converter.
#{stepExecution.jobExecution.id} or #{stepExecution.jobExecutionId} should work though.
The StepContext does provide access to the StepExecution for late binding via SpEL expressions.

junit 4 testing with spring 3.0 and Hibernate 3 in Eclipse - LazyInitializationException

I'm getting a LazyInitializationException trying to test my DAO methods using the tool stack defined in the title. My understanding is that my test must be running outside the hibernate session, or it has been closed before I try to read children objects from my DAO. From reading the documentation, I understood that using the #TransactionConfiguration tag would allow me to define the transaction manager in which to run the tests.
I've read the documentation multiple times and a zillion forum posts. Still slamming my head into my keyboard... What am I missing? Thanks for your help!
my unit test class:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = {
"classpath:/WEB-INF/applicationContext-db.xml",
"classpath:/WEB-INF/applicationContext-hibernate.xml",
"classpath:/WEB-INF/applicationContext.xml" })
#TestExecutionListeners({DependencyInjectionTestExecutionListener.class, CleanInsertTestExecutionListener.class})
#DataSetLocation("test/java/com/yada/yada/dao/dbunit-general.xml")
#TransactionConfiguration(transactionManager="transactionManager", defaultRollback = true)
#Transactional
public class RealmDAOJU4Test {
#Autowired
private DbUnitInitializer dbUnitInitializer;
#Autowired
private RealmDAO realmDAO;
#Test
public void testGetById() {
Integer id = 2204;
Realm realm = realmDAO.get(id);
assertEquals(realm.getName().compareToIgnoreCase(
"South Technical Realm"), 0);
assertEquals(8, realm.getRealmRelationships().size());
}
}
my applicationContext-hibernate.xml:
<bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource"></property>
<property name="useTransactionAwareDataSource" value="true" />
... other properties removed ...
</bean>
<bean id="transactionManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager">
<property name="sessionFactory">
<ref bean="sessionFactory" />
</property>
</bean>
my dao definition in applicationContext.xml
<bean id="realmDAOTarget" class="com.yada.yada.dao.hibernate.RealmDAOImpl">
<property name="sessionFactory" ref="sessionFactory" />
</bean>
<bean id="realmDAO" class="org.springframework.aop.framework.ProxyFactoryBean">
<property name="proxyInterfaces">
<value>com.yada.yada.dao.RealmDAO</value>
</property>
<property name="interceptorNames">
<list>
<value>hibernateInterceptor</value>
<value>realmDAOTarget</value>
</list>
</property>
</bean>
well, for anyone following along at home, here's what I missed:
TransactionalTestExecutionListener
it is required in the #TestExecutionListeners list for the #Transactional annotation to have any effect.