is it possible to implement batch insert using spring-data-jdbc somehow? Or can i get access to JDBCTemplate using this spring-data realization?
There is currently no support for batch operations.
There are two issues requesting that one might want to follow if one is interested in that feature: https://jira.spring.io/browse/DATAJDBC-328 and https://jira.spring.io/browse/DATAJDBC-314
If one is working with Spring Data JDBC there will always be a NamedParameterJdbcTemplate in the application context so one can get that injected in order to perform batch operations without any additional configuration.
Related
I am new to multi-tenancy with mongodb using spring-data-mongodb, we need to use spring-data-mongodb for rest APIs and scheduling tasks( we have more than one schedulers in our application) in the same code with thread-safe. Is autowiring mongoTemplate will make application thread safe as the same mongoTemplate will be accessed from Schedulers and APIs both. Please get me the good practice in such a situation.
Regards
Kris
MongoTemplate itself is thread-safe, i.e. you can call it from multiple threads at the same time, and it will work correctly, i.e. send the different requests correctly to MongoDB.
But that doesn't guarantee consistency: if the scheduler is running and executes multiple updates in the same task, an API call can possibly get some updated records and some other records that aren't updated yet.
By the way: multi-tenancy is having data from multiple organisational entities in the same database. I'm not sure how that links to your question, did you mean multi-threading?
If you use different databases, then you can't use an autowired MongoTemplate.
For autowiring, there must be a single instance, but since the database connection string is a dependency of a MongoTemplate, there must be a single database as well.
You could go for an approach where you do not auto-wire the MongoTemplate directly, but use some sort of factory pattern to create the correct MongoTemplate for the current tenant. See Making spring-data-mongodb multi-tenant for some examples. (It's an old question, but its answers get updated every now and then).
Or you could go with an infrastructural solution, and deploy separate instances of your application, one for each tenant, e.g. on Kubernetes.
We have a requirement where we have to read a batch of a entitytype from the database, submit info about each entity to a service which will callback later with some data to update in the caller entity, save all the caller entities with the updated data. We thought of using spring-batch however we use Couchbase as our database which is eventually consistent and has no support for transactions.
I was going through the spring-batch documentation and I came across the Spring Batch Meta-Data ERD diagram here :
https://docs.spring.io/spring-batch/4.1.x/reference/html/index-single.html#metaDataSchema
With the above information in mind, my question is:
Can Couchbase be used as the underlying job-repository for spring-batch? What are the things I should keep in mind if its possible to use it? Any links to example implementations would be welcome.
The JobRepository needs to be transactional in order for Spring Batch to work properly. Here is an excerpt from the Transaction Configuration for the JobRepository section of the reference documentation:
The behavior of the framework is not well defined if the repository methods are not transactional.
Since Couchbase has no support for transactions as you mentioned, it is not possible to use it as an underlying datasource for the JobRepository.
I am currently developing a Spring Boot app, which uses mybatis for its persistence layer. I want to optimize the batch insertion of entities in the following scenario:
// flightSerieMapper and legMapper are used to create a series of flights.
// legMapper needs to use batch insertion.
#Transactional
public FlightSerie add(FlightSerie flightSerie) {
Integer flightSerieId = flightSeriesSequenceGenerator.getNext();
flightSerie.setFlightSerieId(flightSerieId);
flightSerieMapper.create(flightSerie);
// create legs in batch mode
for (Leg leg : flightSerie.getFlightLegs()) {
Integer flightLegId = flightLegsSequenceGenerator.getNext();
leg.setLegId(flightLegId);
legMapper.create(leg);
}
return flightSerie;
}
mybatis is configured as follows in application.properties:
# this can be externalized if necessary
mybatis.config-location=classpath:mybatis-config.xml
mybatis.executor-type=BATCH
This means that mybatis will execute all statements in batch mode by default, including single insert/update/delete statements. Is this OK? Are there any issues I should be aware of?
Another approach would be to use a dedicated SQLSession specifically for the LegMapper. Which approach is the best (dedicated SQLSession vs global setting in application.properties)?
Note: I have seen other examples where "batch inserts" are created using a <foreach/> loop directly in the mybatis xml mapper file. I don't want to use this approach because it does not actually provide a batch insert.
As #Ian Lim said, make sure you annotate mapper methods with inserts and updates with #Flush annotation if you globally set executor type to BATCH.
Another approach would be to use a dedicated SQLSession specifically
for the LegMapper. Which approach is the best (dedicated SQLSession vs
global setting in application.properties)?
Keep in mind that if you are using different SQL sessions for different mappers there will be different transactions for each SQL session. If a service or service method annotated with #Transactional uses several mappers that use different SQL sessions it will allocate different SQL transactions. So it's impossible to do atomic data operation that involves mappers with different SQL sessions.
I need to read data from existing database is it possible using
compile "org.grails.plugins:db-reverse-engineer:4.0.0"?
My operations are: user should read data from existing table, create new record, create new coulmn, edit coulmn name, edit records.
View format will be in grid like xml grid.
Which technology is the best for these operations in grails, I have plan to work on javascript using jaxrs, is it good to do?
DB Reverse Engineering Plugin (org.grails.plugins:db-reverse-engineer:4.0.0) allows you to generate domain classes using existing DB. After you generate them - just use GORM to perform CRUD operations. You can read about GORM here
You can implement REST api in Grails using standart ways, check this answer to get high level understanding. If you need jaxrs- there is a plugin for that.
I have a requirement to implement in Spring batch,I need to read from a file and from a DB ,the data needs to be processed and written to an email
I have gone through the spring batch documentation but was unable to find a CHUNKtasklet which would read data from multiple readers
SO essentially I have to read from 2 different sources of data(one from file and another from DB,each will need to have its own mapper)
Regards
Tar
I see two options depending on how the data is structured:
Spring Batch relies heavily on composition when building batch components. One option would be to create a custom composite ItemReader that delegates to a other readers (ones Spring Batch provides or otherwise) and provides the logic to assemble a single object based on the results of those delegated ItemReaders.
You could use an ItemReader to provide the base information (say from a database) and use and ItemProcessor to enrich the item (say reading from a file).
Either of the above are normal ways to handle this type of input scenario.