I'm setting up a new version of my application in a demo server and would love to find a way of resetting the database daily. I guess I can always have a cron job executing drop and create queries but I'm looking for a cleaner approach. I tried using a special persistence unit with drop-create approach but it doesn't work as the system connects and disconnects from the server frequently (on demand).
Is there a better approach?
H2 supports a special SQL statement to drop all objects:
DROP ALL OBJECTS [DELETE FILES]
If you don't want to drop all tables, you might want to use truncate table:
TRUNCATE TABLE
As this response is the first Google result for "reset H2 database", I post my solution below :
After each JUnit #tests :
Disable integrity constraint
List all tables in the (default) PUBLIC schema
Truncate all tables
List all sequences in the (default) PUBLIC schema
Reset all sequences
Reenable the constraints.
#After
public void tearDown() {
try {
clearDatabase();
} catch (Exception e) {
Fail.fail(e.getMessage());
}
}
public void clearDatabase() throws SQLException {
Connection c = datasource.getConnection();
Statement s = c.createStatement();
// Disable FK
s.execute("SET REFERENTIAL_INTEGRITY FALSE");
// Find all tables and truncate them
Set<String> tables = new HashSet<String>();
ResultSet rs = s.executeQuery("SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA='PUBLIC'");
while (rs.next()) {
tables.add(rs.getString(1));
}
rs.close();
for (String table : tables) {
s.executeUpdate("TRUNCATE TABLE " + table);
}
// Idem for sequences
Set<String> sequences = new HashSet<String>();
rs = s.executeQuery("SELECT SEQUENCE_NAME FROM INFORMATION_SCHEMA.SEQUENCES WHERE SEQUENCE_SCHEMA='PUBLIC'");
while (rs.next()) {
sequences.add(rs.getString(1));
}
rs.close();
for (String seq : sequences) {
s.executeUpdate("ALTER SEQUENCE " + seq + " RESTART WITH 1");
}
// Enable FK
s.execute("SET REFERENTIAL_INTEGRITY TRUE");
s.close();
c.close();
}
The other solution would be to recreatethe database at the begining of each tests. But that might be too long in case of big DB.
Thre is special syntax in Spring for database manipulation within unit tests
#Sql(scripts = "classpath:drop_all.sql", executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD)
#Sql(scripts = {"classpath:create.sql", "classpath:init.sql"}, executionPhase = Sql.ExecutionPhase.BEFORE_TEST_METHOD)
public class UnitTest {}
In this example we execute drop_all.sql script (where we dropp all required tables) after every test method.
In this example we execute create.sql script (where we create all required tables) and init.sql script (where we init all required tables before each test method.
The command: SHUTDOWN
You can execute it using
RunScript.execute(jdbc_url, user, password, "classpath:shutdown.sql", "UTF8", false);
I do run it every time when the Suite of tests is finished using #AfterClass
If you are using spring boot see this stackoverflow question
Setup your data source. I don't have any special close on exit.
datasource:
driverClassName: org.h2.Driver
url: "jdbc:h2:mem:psptrx"
Spring boot #DirtiesContext annotation
#DirtiesContext(classMode = DirtiesContext.ClassMode.BEFORE_EACH_TEST_METHOD)
Use #Before to initialise on each test case.
The #DirtiesContext will cause the h2 context to be dropped between each test.
you can write in the application.properties the following code to reset your tables which are loaded by JPA:
spring.jpa.hibernate.ddl-auto=create
Related
This is the mysql table.
create table Customer
(
id int auto_increment primary key,
birth date null,
createdTime time null,
updateTime datetime(6) null
);
This my java code
#Before
public void init() {
this.entityManagerFactory=Persistence.createEntityManagerFactory("jpaLearn");
this.entityManager=this.entityManagerFactory.createEntityManager();
this.entityTransaction=this.entityManager.getTransaction();
this.entityTransaction.begin();
}
#Test
public void persistentTest() {
this.entityManager.setFlushMode(FlushModeType.COMMIT); //don't work.
for (int i = 0; i < 1000; i++) {
Customer customer = new Customer();
customer.setBirth(new Date());
customer.setCreatedTime(new Date());
customer.setUpdateTime(new Date());
this.entityManager.persist(customer);
}
}
#After
public void destroy(){
this.entityTransaction.commit();
this.entityManager.close();
this.entityManagerFactory.close();
}
When I reading the wikibooks of JPA, it said "This means that when you call persist, merge, or remove the database DML INSERT, UPDATE, DELETE is not executed, until commit, or until a flush is triggered."
But at same time my code runing, I read the mysql log, I find each time the persist execution, mysql will execute the sql. And I also read the wireShark, each time will cause the request to Database.
I remember jpa saveAll method can send SQL statements to the database in batches? If I wanna to insert 10000 records, how to improve the efficiency?
My answer below supposes that you use Hibernate as jpa implementation. Hibernate doesn't enable batching by default. This means that it'll send a separate SQL statement for each insert/update operation.
You should set hibernate.jdbc.batch_size property to a number bigger than 0.
It is better to set this property in your persistence.xml file where you have your jpa configuration but since you have not posted it in the question, below it is set directly on the EntityManagerFactory.
#Before
public void init() {
Properties properties = new Properties();
properties.put("hibernate.jdbc.batch_size", "5");
this.entityManagerFactory = Persistence.createEntityManagerFactory("jpaLearn", properties);
this.entityManager = this.entityManagerFactory.createEntityManager();
this.entityTransaction = this.entityManager.getTransaction();
this.entityTransaction.begin();
}
Then by observing your logs you should see that the Customer records are persisted in the database in batches of 5.
For further reading please check: https://www.baeldung.com/jpa-hibernate-batch-insert-update
you should enable batch for hibernate
spring.jpa.properties.hibernate.order_inserts=true
spring.jpa.properties.hibernate.order_updates=true
spring.jpa.properties.hibernate.jdbc.batch_size=20
and use
reWriteBatchedInserts&reWriteBatchedStatements=true
end of your connectionString
Yestody,I got this question, how jpa run DDL sql with dynamic tableName?
usually,I just used DQL and DML like 'select,insert,update,delete'.
such as :
public interface UserRepository extends JpaRepository<User, Integer> {
#Query(value = "select a.* from user a where a.username = ? and a.password = ?", nativeQuery = true)
List<User> loginCheck(String username, String password);
}
but when I required run DDL sql below
String sql = "create table " + tableName + " as select * from user where login_flag = '1'";
I don't find a way to solve this with Jpa (or EntityManager).
Finally I used JDBC to run the DDL sql,but I think it's ugly...
Connection conn = null;
PreparedStatement ps = null;
String sql=" create table " + tableName + " as select * from user where login_flag = '1' ";
try {
Class.forName(drive);
conn = DriverManager.getConnection(url, username, password);
ps = conn.prepareStatement(sql);
ps.executeUpdate();
ps.close();
conn.close();
} catch (Exception e) {
e.printStackTrace();
}
So,can jpa run DDL sql(such as CREATE/DROP/ALTER) wiht dynamic tableName in an easy way?
Your question seems to consist of two parts
The first part
can jpa run DDL sql
Sure, just use entityManager.createNativeQuery("CREATE TABLE ...").executeUpdate(). This is probably not the best idea (you should be using a database migration tool like Flyway or Liquibase for DB creation), but it will work.
Note that you might run into some issues, e.g. different RDBMSes have different requirements regarding transactions around DDL statements, but they can be solved quite easily most of the time.
You're probably wondering how to get hold of an EntityManager when using Spring Data. See here for an explanation on how to create custom repository fragments where you can inject virtually anything you need.
The second part
with dynamic tableName
JPA only supports parameters in certain clauses within the query, and identifiers are not one of them. You'll need to use string concatenation, I'm afraid.
Why dynamic table names, though? It's not like your entity definitions are going to change at runtime. Static DDL scripts are generally less error-prone.
I am implementing a feature where if there is any exception while writing data into DB, we should retry it for 5 times before failing. I have implemented the feature but not able to test it using arquillian test.
We are using JPA and Versant as database. Till now, I am debbuging the the arquillian test and once my flow reaches DB handler code, I am stopping the database. But this is worst way of testing.
Do you have any suggestion how to achieve the same ?
With JPA in mind, the easiest way is to add method to your data access layer, with which you are able to run native queries. Then you run query against nonexisting table or something similar. So in my DAO utilities I found method like this:
public List findByNativeQuery(String nativeQuery, Map<String, Object> args) {
try{
final EntityManager em = getEntityManager();
final Query query = em.createNativeQuery(nativeQuery);
if (args!=null && args.entrySet().size()>0) {
final Iterator it = args.entrySet().iterator();
while (it.hasNext()) {
final Map.Entry pairs = (Map.Entry)it.next();
query.setParameter(pairs.getKey().toString(), pairs.getValue());
}
}
return query.getResultList();
}
catch (RuntimeException e) {
// throw some new Exception(e.getMessage()); // the best is to throw checked exception
}
}
Native solutions
There is the old trick by dividing by zero in the database. At the time of selection you could try:
select 1/0 from dual;
Insertion time (you need a table):
insert into test_table (test_number_field) values (1/0);
pure JPA solution
You can try to utilize the #Version annotation and decrement it to throw OptimisticLockException. This is not thrown in the database, but in the Java layer, but fullfills your need.
Those all will result in DB fail.
I want to implement some DB cleanup at each startup (full schema deletion and recreation while in dev-enviroment).
I'm doing it in Global.beforeStart. And because it's literally before start I need to load DB drivers myself.
The code is:
#Override
public void beforeStart(Application app){
System.out.println("IN beforeStart");
try{
Class.forName("org.postgresql.Driver");
System.out.println("org.postgresql.Driver LOADED");
} catch (ClassNotFoundException cnfe){
System.out.println("NOT LOADED org.postgresql.Driver");
cnfe.printStackTrace();
}
ServerConfig config = new ServerConfig();
config.setName("pgtest");
DataSourceConfig postgresDb = new DataSourceConfig ();
postgresDb.setDriver("org.postgresql.Driver");
postgresDb.setUsername("postgres");
postgresDb.setPassword("postgrespassword");
postgresDb.setUrl("postgres://postgres:postgrespassword#localhost:5432/TotoIntegration2");
config.setDataSourceConfig(postgresDb);
config.setDefaultServer(true);
EbeanServer server = EbeanServerFactory.create(config);
SqlQuery countTables = Ebean.createSqlQuery("select count(*) from pg_stat_user_tables;");
Integer numTables = countTables.findUnique().getInteger("count");
System.out.println("numTables = " + numTables);
if(numTables>2){
DbHelper.cleanSchema();
}
System.out.println("beforeStart EXECUTED");
//DbHelper.cleanSchema();
}
Class.forName("org.postgresql.Driver") passed without exceptions, but then I'm getting:
com.avaje.ebeaninternal.server.lib.sql.DataSourceException: java.sql.SQLException: No suitable driver found for postgres
on the line EbeanServer server = EbeanServerFactory.create(config);
Why?
Use onStart instead, it's performed right after beforeStart but it's natural candidate for operating on database (in production mode it doesn't wait for first request), javadoc for them:
/**
* Executed before any plugin - you can set-up your database schema here, for instance.
*/
public void beforeStart(Application app) {
}
/**
* Executed after all plugins, including the database set-up with Evolutions and the EBean wrapper.
* This is a good place to execute some of your application code to create entries, for instance.
*/
public void onStart(Application app) {
}
Note, that you don't need include DB config additionally here, you can use your models here the same way as you do in controller.
I was thinking it would be nice to create a base class for NUnit test fixtures that opens a TransactionScope during the SetUp phase, then rolls back the transaction during tear down.
Something like this:
public abstract class TestFixtureBase
{
private TransactionScope _transaction;
[TestFixtureSetUp]
public void TestFixtureSetup()
{
_transaction = new TransactionScope();
}
[TestFixtureTearDown]
public void TestFixtureTearDown()
{
if (_transaction != null)
{
_transaction.Dispose();
}
}
}
Do you think this is a good idea?
Obviously the database is just a test database, not a live database, but it would still be annoying if it filled up with junk data from the unit tests.
What do other people do when running unit tests that involve a lot of data access?
You want to be careful here. TransactionScope is going to promote the transaction to a distributed transaction if you open up more than one connection to the database. I find that it is easier just to write some simple SQL that clears out the tables of interest to my test class before I start running the test.
EDIT: Normally I would call any test that touches the database an integration test since it involves another system. Typically, I will mock out the database when unit testing my code.
[TestSetup]
public void Setup()
{
foreach (string table in new string[] { "table1", "table2" })
{
ClearTable( table );
}
}
private void ClearTable( string table )
{
...standard stuff to set up connection...
SqlCommand command = connection.CreateCommand() );
command.CommandText = "delete from " + table;
command.ExecuteNonQuery();
... stuff to clean up connection...
}
I've used XtUnit
It automatically rolls back at the end of a unit test. You can simply add a [Rollback] attribute to the test. It's an extension to NUnit or MbUnit