I have a default Java EE 7 application with JPA/Eclipselink.
I want to write some UI tests with Arquillian Drone/Graphene.
My testsuite is "working". I can click through the application and make some asserts.
But: I want to create multiple tests. Every test class should reset the database, to make sure, that the conditions are always the same.
I'm using flyway to reset my database.
#Before
public void setup() {
Flyway flyway = new Flyway();
flyway.setDataSource(...);
flyway.clean();
flyway.init();
flyway.migrate();
}
The reset is working. And the first execution of the test also (at this situation, nothing is in any cache).
When I try to execute the test again, the database is inconsistent with strange JPA errors.
First I thought ok: Reset the cache
getEntityManager().getEntityManagerFactory().getCache().evictAll();
Is not enough. Same problems.
Next idea was to destroy the sessions (Some JPA data could be saved in old sessions). I haven't found a good way to destroy all sessions. I made a workaround, but this doesn't work also.
I think I have a default problem, but I can not found any solutions for this problem.
I also tried dbunit and the arquillian-persistence-extension. But it is like flyway, just another way.
In theory the problem is, that the database has been reset manually over SQL and Java/JPA/EclipseLink/The Sessions/deployed applaction have no idea about the changed data.
How can I reset everything (all Caches?)?
I also thought about "redeploy before start testclass". But this is a little bit to much (takes more time and is no fine solution)?
One more information: I'm also doing normal arquillian-tests (without UI/Selenium), here is my flyway database reset working.
Thanks for help :).
You didn't say what errors you really got, you just said:
When I try to execute the test again, the database is inconsistent with strange JPA errors (...)
It is hard to believe that your problem lies in JPA cache. I think that your problem has totally different source.
Your approach to cleaning the database has a fundamental flaw.
Code that you've presented: should be run only once, before all tests. Because Flyway is meant to prepare the database structure, not to setup it into known state.
So conceptually DbUnit & Arquillian Persistence Extension and Flyway do two really different things.
They are not a replacement for each other.
So your code that uses flyway:
#Before
public void setup() {
...
flyway.clean();
flyway.init();
flyway.migrate();
}
is wrong, because it should be run only once before all tests. To do that you can use some container features:
If using EJB Container, then #Singleton + #Startup + #PostConstruct combination could be used to launch flyway tool.
If using Spring Container, then init-method="migrate" would do the trick.
Or use maven and its pre-integration-test phase to launch flyway tool.
By the way: to avoid maintenance effort associated with DbUnit's xml datasets, personally I would recommend DbSetup tool. Nice and simple solution.
Edit
Besides the fact, that changing the DB structure during tests isn't a good practice - your problem may also be caused that both Flyway and JPA are using different datasources (even pointing to the same DB). You should double check that you're not creating DataSource on your own - just inject for the Flyway the same one that PersistenceUnit is using.
Related
Using xUnit and the TestServer from Microsoft.AspNet.TestHost, how can I wrap each test in a database transaction that can be rolled back after the test?
Here's how I create the TestServer:
TestServer = new TestServer(TestServer.CreateBuilder()
.UseStartup<Startup>());
The Startup that's referenced there is the Startup from the web app project. In the ConfigureServices method in that Startup class I add EF like this:
services.AddEntityFramework()
.AddSqlServer()
.AddDbContext<TrailsDbContext>(options => options.UseSqlServer(Configuration["Data:DefaultConnection:ConnectionString"]));
I could pull the DbContext back of services and store a static reference on the Startup class, but that seems pretty hacky. Is there any way I can instantiate the DbContext where I create the TestServer and somehow have the web app use that instead of the one in the Startup class?
Edit: I have tried instantiating another instance of the DbContext where I create the TestServer and using that context to delete and recreate the database before each test, but that adds about 10 seconds to each test's run time.
Some advice: the simplest approach would be to destroy the test database at the end and recreate for each test run. This ensures no lingering test-to-test contamination.
But since you asked how, this can be done by extending Xunit. Xunit allows you to define custom test cases and test runners. A complete answer is hard to include in a SO answer. The simplest solution uses ambient transactions. (Danger! Ambient transactions can be tricky.) Xunit has a sample for a custom BeforeAfterTestAttribute that rolls back a transaction. https://github.com/xunit/samples.xunit/tree/master/AutoRollbackExample. To use ambient transactions, turn off the default EF setting that throws if ambient transactions are present.(optionsBuilder.UseSqlServer().SuppressAmbientTransactionWarning()).
A more complicated, but better solution is to override XunitTestCaseRunner and inject a transaction into each test case, ensuring to rollback at the conclusion of each test.
Also, EF docs provides a sample of using the InMemory provider to test. You may find this useful.
"Testing In Memory : EF Core Docs"
When an application is Live an iterative approach to database changes is obviously required. In the db first world I would change the object (eg. column added to table) in the databaae project, then deploy (recreate) to my local instance, then replace the old table with the new in my edmx - when it was go Live time a delta script is generated out of the database project compared to a copy of the Live database schema. Sounds long winded but at the end of the day I only made the change once (the object in the db project) - everything else is generated
Flip over to code first (EF6) and Im expecting a similar one change experience - i.e. I add the property to the class - however do I additionally need to add a migration script ?
I've been reading and it seems many advise to disable migrations to have more control - I'm confused - I had visions of simply deploying the app and the changes automatically reflected in the target database the next time the app runs - one thing is for sure I don't want to manually write separate deployment scripts (or migration code). As mentioned I'm confused about this final part - can anyone clarify - point out the options
Many thanks
Short answer is there's no need to create a 'migration script', you're correct as EF will handle it for you if you want. I think when you read about disabling migrations, you were probably actually reading 'disable automatic migrations'; EF will still generate migrations regardless.
As you pointed out it IS a two-change process when developing: First you change your class, then you open up the Package Manager console and call Add-Migration. Usually, that's all you have to do, and EF will generate the change code for you. Then, you call Update-Database and it does it's work. When you go to deploy, you will connect to your target database and call Update-Database once and it will apply all migrations that are pending.
You can also enable auto-migrations which skips the Add-Migration step, but I always like to review the generated code. Call me old-fashioned ;)
It gets more complicated when you need support for views, SPROCS, and UDFs, but there are ways to do most anything you want to do. And, even though it's a 2 (3?) step process to get changes out to the DB, it's still much easier than changing the DB and code separately, by yourself.
Then, you can follow the steps here to set your deployment up so that once your EF is initialized on a connection to your production DB, it automatically applies the updates. Again, I would advise to do it yourself (via the package manager console) just to be safe but it's not necessary.
I have written an application using EF 6.0 in combination with an SQL Server Compact 4.0 Database. When a customer uses this application for the first time, it (the application) should create a database-file in a given path with some initital values. Also migrations should be allowed, for it is quite possible that the object model might change with future versions of the app.
Now I´m wondering what would be the best way to to deploy the DB on the users productive system. I could think of three ways:
I could create a DB-file with initial values and just copy it to the right place during installation process and use MigrateDatabaseToLatestVersionInitializer in the app.
In the DbContext-Constructors (I have two contexts) I could check for an existing DB-file and use different Database-Initializers accordingly. Like a CreateDatabaseIfNotExistsInitializer with a seed method that creates initial data if no fiel is found and a MigrateDatabaseToLatestVersionInitializer if the DB-file exists.
I could use the MigrateDatabaseToLatestVersionInitializer always and in its "Seed"-method check for existing table entries and create them if they are not present.
Which of these ways is to be preferred or is there a better way I didn´t think of?
It sounds like this is a desktop application so you might want to catch permissions errors about creating the database file at installation time (i.e. option 1) rather than run time, especially as in option 2 the database initialization is not an imperative command you're giving that you can put a try...catch around.
I don't think option 3 would work as the Seed method gets run after all the migrations, so surely the migrations will either have successfully run, in which case the tables don't need creating, or they will have failed as the DB doesn't exist and therefore your Seed method won't get run.
My database model (sometimes referred to as "context") is dynamically assembled at startup based on which services and/or plugins are installed. Plugins and services export their model definition fragments through my IoC container and the application core picks them up and runs them when the DbContext.OnModelCreating method is called.
The question is: Can I (and how do I) use Code First Migrations with this setup?
(below is more information on what I've tried and what particular problems are)
In my previous project, the database was inherited from some old code so we couldn't use any of the Code First database generation stuff anyway. We simply kept a long line of delta scripts and executed them manually on deploy (it was a single-host kind of project).
Now I'm starting a new project, and this time, the database is brand new, ready for Code First to play with. Initially, I was all excited about Code First Migrations, seemed like the way to go. Until I actually tried it. The initial attempt, quite obviously, failed due to the absence of an explicitly defined DbContext in my project.
So far, it looks like the only viable option is to manually code migrations, with which I am perfectly fine. However, it turns out that this is not as simple as just creating a few classes inherited from DbMigration.
After some experimentation on a small test project, I was able to find out that the migration autogenerator adds an implementation of IMigrationMetadata, which, among other things, contains a hash of my model as the values of the Source and Target properties. Presumably, this hash is then used to identify a path across migrations from the "current" state of the database (as recorded in the __MigrationHistory table) to the newest state as defined by the model in code. This totally makes sense, but...
Naturally, I have no idea where to get that hash for my model, which makes me unable to implement IMigrationMetadata on my migrations.
On the other hand, I see that the metadata interface is not included in the DbMigration class itself, which makes me think that it might be optional. It then follows that migrations can actually work without the hash values, but the question is - how?
All the information I could find on the internet is just simple, very basic tutorials. No information on how to create migrations manually (and whether it's even supported). No documentation on how it actually works and how to extend it. And it is not quite obvious from outside.
I am ready to resort to ILSpy at this point, but the whole EF is so complex that I fear I may not be able to find what I need in reasonable time.
Here are a few ideas that you could pull together to find a solution that works for you. I realize I mentioned some of these in our other thread, but I'm including them here for others reading this question.
Automatic migrations allow Code First to automatically calculate and apply changes to the database.
You can write your own code to generate and apply migrations. I've written a post about applying migrations and the MigrationsScaffolder class will help you create migrations.
When you run the project , an extra table is created in the database.
EdmMetadata table
The hash is always created with the help of EdmMetadata Entity and the current code first model. It is SHA-256 hash stored in the EdmMetadata table of the database. You can get it from that table.
Methodology to be followed will be:
Get the hash of the current model using
var hash=GetModelHash(OldContext);
Check whether the model in the code (new model) is compatible with the model in database(old model) using
CompatibleWithModel(hash,CurrentContext,ObjectContext)
This method returns bool.
If it is not compatible, then delete the existing tables in the database.
Create new tables
Save the current hash to the databse
Seed the data.
The code may look like:
{
var objectContext = ((IObjectContextAdapter)context).ObjectContext;
var modelHash = GetModelHash(objectContext);
if (CompatibleWithModel(modelHash, context, objectContext))
return;
DeleteExistingTables(objectContext);
CreateTables(objectContext);
SaveModelHashToDatabase(context, modelHash, objectContext);
SeedData(context);
}
Be sure to make the class inherited from
IDatabaseInitializer<T> where T:DbContext
I have a Spring-based system that uses Hibernate Search 3.4 (on top of Hibernate 3.5.4). Integration tests are managed by Spring, with #Transactional annotation. At the moment test data (entities that are to be indexed) is loaded by Liquibase script, we use it's Spring integration. It's very inconvenient to manage.
My new solution is to have test data defined as Spring beans and wire them as Resources, by name. This part works.
I tried to have these beans persisted and indexed in setUp method of my test cases (and in test methods themselves) but I failed. They get into DB fine but I can't get them indexed. I tried calling index() on FullTextEntityManager (with flushToIndexes), I tried createIndexer().startAndWait().
What else can I do?
Or may be there is some better option of testing HS?
Thank You in advance
My new solution is to have test data defined as Spring beans and wire
them as Resources, by name. This part works.
sounds like a strange setup for a unit test. To be honest I am not quote sure how you do this.
In Search itself an in memory database (H2) is used together with a Lucene RAM directory. The benefits of such a setup is that it is fast and easy to avoid dependencies between tests.
I tried to have these beans persisted and indexed in setUp method of
my test cases (and in test methods themselves) but I failed. They get
into DB fine but I can't get them indexed.
If automatic indexing is enabled and the persisting of the test data is occurring within an transaction, it should work. A common mistake in combination with Spring is to use the wrong transaction manager. The Hibernate Search forum has a lot of threads around this, for example this one - https://forum.hibernate.org/viewtopic.php?f=9&t=998155. Since you are not giving any concrete configuration and code examples it is hard to give more specific advice.
I tried createIndexer().startAndWait()
that is also a good approach. I would recommend this approach if you want to insert not such a couple of test entities, but a whole set of data. In this case it can make sense to use a framework like dbunit to insert the testdata and then manually index the data. createIndexer().startAndWait() is the right tool for that. Extracting all this loading/persisting/indexing functionality into a common test base class is the way to go. The base class can also be responsible to do all the Spring bootstrapping.
Again, to give more specific feedback you have to refine your question.
I have a complete different approach, when I write any queries, i want to write a complete test suite, but data creation has always been pain(special mention to when test customer gets corrupt and all your test suite breaks.
To solve this I created Random-JPA. It's simple and easy to integrate. The whole idea is you create fresh data and test.
You Can find the full documentation here