I have two database configuration javaconfig classes - one for JPA purposes with transactionManager() and entityManagerFactory() #Bean methods and one config class for non-JPA JDBCTemplate based query submission to access data from that database. The overall idea is to read using JDBCTemplate to read data and persist the data, after transformation, into the JPA based datasource. I am using Spring Boot to enable auto configuration. My test fails with:
java.lang.IllegalArgumentException: Not an managed type:
I have both spring-boot-starter-jdbc and spring-boot-starter-data-jpa in my build.gradle. My gut feeling is that the two data sources are in collision course with each other. How do I enforce the use of each of these datasources for the two use-cases that I mentioned earlier - one for JPA and another for JDBCTemplate purposes?
Details (Added after Dave's reply):
My service classes have been annotated with #Service and my repository classes have #Repository. Service uses repository objects using #Autowired though there are some services that are JDBCTemplate-based for data retrieval.
More complex depiction of my environment goes as follows (logically): JDBCTemplate(DataSource(Database(DB2)))--> Spring Batch Item Reader;Processors; Writer --> Service(Repository(JPADataSource(Database(H2)))). Spring batch item processors connect to both databases using services. For spring batch, I am using a H2 Job repo database (remote) to hold job execution details. Does this make sense? For Spring batch, I am using de.codecentric:spring-boot-starter-batch-web:1.0.0.RELEASE. After getting past the entityManagerFactory bean not found errors, I want to have control over the wiring of the above components.
I don't think it's anything to do with the data sources. The log says you have a JPA repository for a type that is not an #Entity. Repositories are automatically scanned from the package you define #EnableAutoConfiguration by default. So one way to control it is to move the class that has that annotation to a different package. In Boot 1.1 you can also set "spring.data.jpa.repositories.enabled=false" to switch off the scan if you don't want it. Or you can use #EnableJpaRepositories as normal.
Related
I have a simple Spring Boot (2.6.4) app that uses Spring Batch and Spring Integration to read data from a file and emit Kafka events. I have configured Spring Integration to poll a configurable directory and trigger the Spring Batch Job as soon as a new file arrives.
As soon as the Job is "kicked" I get this error:
Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove #Transactional annotations from client).
I assume this is triggered by the following Spring Integration configuration:
return IntegrationFlows.from(fileReadingMessageSource,
c -> c.poller(Pollers.fixedDelay(period)
.taskExecutor(taskExecutor)
.maxMessagesPerPoll(maxMessagesPerPoll)
.transactionSynchronizationFactory(transactionSynchronizationFactory())
.transactional(new PseudoTransactionManager())))
The TransactionSynchronizationFactory is configured to move the incoming file into an error or success directory, based on the outcome of the job.
My understanding is that a Spring Batch Job does not like to be executed within the context of a Transaction, since it needs to manage its own transaction to deal with multiple steps failures.
I also understand that I could set the value of setValidateTransactionState of JobRepositoryFactoryBean to false, but I would rather not mess around with the Spring Batch internals.
As suggested elsewhere, I have also tried to use an Async Job Launcher, but no luck.
#Bean
public JobLauncher getJobLauncher() {
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor());
jobLauncher.setJobRepository(this.jobRepository);
return jobLauncher;
}
The app is using an in-memory H2 data-source, but that doesn't seem to affect the error.
I'm unable to find a solution to this problem, any recommendation?
Edit - full project here: https://github.com/luciano-fiandesio/spring-batch-and-integration-demo
this is my first post on this forum so please go easy on me if a made mistake or else. I'm using spring-boot batch with mongdb and everything works fine but I running in trouble when I have to test my batch. My Batchconfiguration is based on multiple jobs (4 jobs), so
1st trouble
when I used JobLauncherTestUtils spring doesn't know which job to inject and I don't know how to specify that to test the jobs one by one.
2nd trouble when I separate my configuration to testing only one job, JobLauncherTestUtils couldn't been creating because it's need dataSource, as I used MongoDb with mongotemplate I don't use dataSource.
the stack error look like :
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'jobRepositoryTestUtils': Unsatisfied dependency expressed through method 'setDataSource' parameter 0; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'javax.sql.DataSource' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {}
Hope anybody will help me to find a solution. Thanks in advance.
1st trouble when i used JobLauncherTestUtils spring don't know wich job to inject and i don't know how to specify that to test the jobs one by one.
You can specify which job to launch using JobLauncherTestUtils#setJob. Please see JobLauncherTestUtils throws NoUniqueBeanDefinitionException while trying to test spring batch steps
2nd trouble when I separate my configuration to testing only one job, JobLauncherTestUtils couldn't been creating because it's need dataSource, as I used MongoDb with mongotemplate I don't use dataSource.
According to the error Error creating bean with name 'jobRepositoryTestUtils', it is not JobLauncherTestUtils that could not be created but jobRepositoryTestUtils. The repository test utils is a convenience class to create and remove job executions from a database (See its javadoc). So it requires a datasource. If you don't have a data source in your test context, you need to remove this bean declaration.
My spring batch application is not inserting relationship between task and job in TASK_TASK_BATCH table.
Spring doc says :
Associating A Job Execution To The Task In Which It Was Executed
Spring Boot provides facilities for the execution of batch jobs easily
within an über-jar. Spring Boot’s support of this functionality allows
for a developer to execute multiple batch jobs within that execution.
Spring Cloud Task provides the ability to associate the execution of a
job (a job execution) with a task’s execution so that one can be
traced back to the other.
This functionality is accomplished by using the TaskBatchExecutionListener. By default, this listener is auto configured in any context that has both a Spring Batch Job configured (via having a bean of type Job defined in the context) and the spring-cloud-task-batch jar is available within the classpath. The listener will be injected into all jobs."
I have all the required jars in my classpath.It's just that I am creating jobs and tasklets dynamically so not using any annotation. As per the doc TaskBatchExecutionListener is responsible for creating mapping in TASK_TASK_BATCH table by calling taskBatchDao's saveRelationship method.
I am just not able to figure out how to configure TaskBatchExecutionListener explicitly in my spring batch application.
If you have the org.springframework.cloud:spring-cloud-task-batch dependency, and the annotation #EnableTask is present, then your application context contains a TaskBatchExecutionListener bean that you can inject into your class that dynamically creates the jobs and tasklets.
That might look similar to this:
#Autowired
JobBuilderFactory jobBuilderFactory;
#Autowired
TaskBatchExecutionListener taskBatchExecutionListener;
Job createJob() throws Exception {
return jobBuilderFactory
.get("myJob")
.start(createStep())
.listener(taskBatchExecutionListener)
.build();
}
I hope that helps. Otherwise please share some minimal code example to demonstrate what you're trying to do.
How to configure Javers with Spring Boot 2.x and Postgresql such that javers tables are created in a specific schema (named dbo) rather than the default "public" schema of the database?
I have created the issue for that:
https://github.com/javers/javers/issues/690
Consider contributing a PR, it's really easy to implement this props.
I want to add some lookup before the Spring context is loaded, which is ideally on the bootstrap phase of Spring Cloud (When it lookup for Configuration Server, Cloud connectors etc). How can I make my code to be executed on that phase ?
What I want to do is query Vault to get all my databases secrets and api keys and set the properties, I know I can encrypt with Spring Cloud Config, but I liked the strong box of Vault. (The integration with Vault part I can handle)
As I saw in the code of Spring Cloud Config, the bootstrap configuration is auto-configured by using the class org.springframework.cloud.bootstrap.BootstrapConfiguration on the resources/META-INF/spring.factories file, the same which you can use to register new auto configuration classes for Spring Boot, as reference you can refer to the file on the project here. This will make your configuration be started and registered before the "normal" application context.