I have a SQL script which performs delete operation from multiple tables based on say employee ids:
DELETE FROM EMP_ADDRESS where EMP_ID in (EMP_IDS);
DELETE FROM EMP_DETAILS where EMP_ID in (EMP_IDS);
DELETE FROM EMPLOYEE where EMP_ID in (EMP_IDS);
Is there a way to call the sql script from Spring batch by passing the employee ids? I tried an alternate approach where in the writer i get the ids and delete from the tables as below:
public class DeleteEmployeeData implements ItemWriter<EmployeeData>{
#Autowired
private JdbcTemplate jdbcTemplate;
#Override
public void write(List<? extends EmployeeData> items) throws Exception {
for(EmployeeData item : items){
jdbcTemplate.update(SQLConstants.DELETE_EMP_ADDRESS,item.getEmployeeId());
jdbcTemplate.update(SQLConstants.DELETE_EMP_DETAILS,item.getEmployeeId());
jdbcTemplate.update(SQLConstants.DELETE_EMPLOYEES,item.getEmployeeId());
}
}
}
This works. But i wanted to know if there is a better approach than this?
Your current approach with a chunk oriented step looks good to me:
The reader reads IDs
A processor filters IDs
And a composite writer with two writers: one to write xml and another one to delete items
Related
I am having two microservices: ms A and ms B
In ms A i am trying to insert a record, while inserting i am auto generating a value for example AWB number which is inserted in the table. The same AWB number which i am passing to ms B to fetch the record from the table but jpa returns no record. How to handle this situation? Below is the code snippet which i used for ms A and B.
microservice A
#Transaction
public class ARepository implements JpaRepository<Product, Long> {
public void save(Product product);
}
microservice B
#Repository
public class BRepository implements JpaRepository<Product, Long> {
#Query("SELECT * FROM PRODUCT WHERE awbNumber=?1");
public Product getProductByAwbNumber(String awbNumber);
}
But from the microservice A, i think #Transaction is not required. So i have removed #Transaction also but still the result is empty. Can someone help me on this what are the steps required to achieve by inserting the record in ms A and retrieve it from ms B.
I've a usecase where I need to connect to two different databases(Postgres and Oracle). Postgres is already configured with jpa. I need to add one more databases(Oracle). In the oracle database i need to choose tables at runtime for insertion and deletion(since tables are not fixed). Currently im passing the tables in my properties file as a list
oracle:
deletion:
table:
-
tableName: user
primaryKey: userId
emailField: emailId
deleteTableName: user_delete
-
tableName: customer
primaryKey: customerId
emailField: emailAddress
deleteTableName: customer_delete
I've created a bean that reads all these properties and puts them in a list
#Bean("oracleTables")
#ConfigurationProperties("oracle.deletion.table")
List<Table> getAllTAbles(){
return new ArrayList<>();
}
I have a list of emailAddresses with me. For each of these tables i need to fetch primary key based on emailAddress from parent table(value in tableName) and insert data into corresponding deleteTable(value in deleteTableName). Once that is done i need to delete data from the actual table(value in tableName) based on email address.
I'm planning to loop through the list of tables I have in my bean and perform fetch, insert and delete.
sample snippet
#Autowired
#Qualifier("oracleTables")
List<Table> tables;
public boolean processDelete(List<String> emails){
for(Table table:tables){
//fetch all the primary keys for given emails from main table(value in tableName)
//insert into corresponding delete table
//delete from main table
}
}
But the question i have is , should i go with jdbcTemplate or jpaRepository/hibernate. And some help with implementation as well with a small sample/link.
The reason for this question is
1)Tables in my case are not fixed
2)I need transaction management to rollback in case of failure in either fetching or inserting or deletion.
3)I need to configure two databases
should i go with jdbcTemplate or jpaRepository/hibernate
Most definitely JdbcTemplate. JPA does not easily allow dynamic tables.
I need transaction management to rollback in case of failure in either fetching or inserting or deletion
If you need transactions, you'll also need to define two separate transaction managers:
#Bean
public TransactionManager oracleTransactionManager() {
var result = new DataSourceTransactionManager();
...
result.setDataSource(oracleDataSource());
return result;
}
#Bean
public TransactionManager postgresTransactionManager() {
...
}
Then, if you want declarative transactions, you need to specify the manager with which to run a given method:
#Transactional(transactionManager = "oracleTransactionManager")
public void doWorkInOracleDb() {
...
}
I need to configure two databases
Just configure two separate DataSource beans. Of course, you will actually need two separate JdbcTemplate beans as well.
I have a table with 30 columns.
I fill the object within my java code. Now I want to look up in my database, if the row is already inserted. I can do this primitive like:
SELECT *
FROM tablename
WHERE table.name=object.name
AND table.street=object.street
AND ...
AND ...
AND ...
I think you get it. It works, but in my opinion this is not the best solution.
Is there any kind of a generic solution (eg: I do not need to change the code, if the table changes), where I can give the where-clause my object and it can match itself? Also the where-clause is not that massive.
The closest thing that comes to mind is the Spring Data JPA Specifications.
You can isolate the where clauses in an instance for a particular entity.
Afterwards, you just pass it to any of the #Repository methods:
public interface UserRepository extends CrudRepository<User, Long>,
JpaSpecificationExecutor<User> {
}
Then in your service:
#Autowired
private UrerRepository repo;
public void findMatching() {
List<User> users = repo.findAll(new MyUserSpecification());
Then, whenever db changes you simply alter one place, which is the Specification implementation.
I'm using spring data (jpaRepository) + Oracle 11g Database.
Here's the code of my JUnit test:
#Test
public void testAjoutUtilisateur() {
Utilisateur utilisateur = new Utilisateur();
(...)
utilisateur=repository.save(utilisateur);
Utilisateur dbutilisateur = repository.findOne(utilisateur.getIdutilisateur());
assertNotNull(dbutilisateur);
When I debug I find that "utilisateur" object returned by repository.save method has an id like "2100" while the corresponding inserted line in the database have an id like "43".
I have an Oracle database with a sequence and a trigger to have the auto incremented property for the id for my "Utilisateur" table.
Here is the id definition in my Utilisateur entity:
#Entity
#NamedQuery(name="Utilisateur.findAll", query="SELECT u FROM Utilisateur u")
#SequenceGenerator(sequenceName="ID_UTILISATEUR_SEQ", name="ID_UTILISATEUR_SEQ")
public class Utilisateur implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy=GenerationType.SEQUENCE, generator="ID_UTILISATEUR_SEQ")
private Long idutilisateur;
Where is the problem? Is it within the save method?
Thank you.
Edit:
I figured out that the problem was already solved by the solution of #jhadesdev and the data lines I was talking about were inserted when the triggers were actives.
Finally, I have to mention that by default the JUnit test seems to not insert data in the database (it inserts then rollback). In order to invalidate this behaviour we have to specify the #TransactionConfiguration(defaultRollback=false) annotation in the test class.
For example (in my case):
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = { "classpath:context/dao-context.xml" })
#TransactionConfiguration(defaultRollback=false)
#Transactional
public class UtilisateurRepositoryTest {
Hope it can help someone.
The problem is that two separate mechanisms are in place to generate the key:
one at Hibernate level which is to call a sequence and use the value to populate an Id column and send it to the database as the insert key
and another mechanism at the database that Hibernate does not know about: the column is incremented via a trigger.
Hibernate thinks that the insert was made with the value of the sequence, but in the database something else occurred. The simplest solution would probably be to remove the trigger mechanism, and let Hibernate populate the key based on the sequence only.
I just jumped on a feature written by someone else that seems slightly inefficient, but my knowledge of JPA isn't that good to find a portable solution that's not Hibernate specific.
In a nutshell the Dao method called within a loop to insert each one of the new entities does a "entityManager.merge(object);".
Isnt' there a way defined in the JPA specs to pass a list of entities to the Dao method and do a bulk / batch insert instead of calling merge for every single object?
Plus since the Dao method is annotated w/ "#Transactional" I'm wondering if every single merge call is happening within its own transaction... which would not help performance.
Any idea?
No there is no batch insert operation in vanilla JPA.
Yes, each insert will be done within its own transaction. The #Transactional attribute (with no qualifiers) means a propagation level of REQUIRED (create a transaction if it doesn't exist already). Assuming you have:
public class Dao {
#Transactional
public void insert(SomeEntity entity) {
...
}
}
you do this:
public class Batch {
private Dao dao;
#Transactional
public void insert(List<SomeEntity> entities) {
for (SomeEntity entity : entities) {
dao.insert(entity);
}
}
public void setDao(Dao dao) {
this.dao = dao;
}
}
That way the entire group of inserts gets wrapped in a single transaction. If you're talking about a very large number of inserts you may want to split it into groups of 1000, 10000 or whatever works as a sufficiently large uncommitted transaction may starve the database of resources and possibly fail due to size alone.
Note: #Transactional is a Spring annotation. See Transactional Management from the Spring Reference.
What you could do, if you were in a crafty mood, is:
#Entity
public class SomeEntityBatch {
#Id
#GeneratedValue
private int batchID;
#OneToMany(cascade = {PERSIST, MERGE})
private List<SomeEntity> entities;
public SomeEntityBatch(List<SomeEntity> entities) {
this.entities = entities;
}
}
List<SomeEntity> entitiesToPersist;
em.persist(new SomeEntityBatch(entitiesToPersist));
// remove the SomeEntityBatch object later
Because of the cascade, that will cause the entities to be inserted in a single operation.
I doubt there is any practical advantage to doing this over simply persisting individual objects in a loop. It would be an interesting to look at the SQL that the JPA implementation emitted, and to benchmark.