I have a Spring-based system that uses Hibernate Search 3.4 (on top of Hibernate 3.5.4). Integration tests are managed by Spring, with #Transactional annotation. At the moment test data (entities that are to be indexed) is loaded by Liquibase script, we use it's Spring integration. It's very inconvenient to manage.
My new solution is to have test data defined as Spring beans and wire them as Resources, by name. This part works.
I tried to have these beans persisted and indexed in setUp method of my test cases (and in test methods themselves) but I failed. They get into DB fine but I can't get them indexed. I tried calling index() on FullTextEntityManager (with flushToIndexes), I tried createIndexer().startAndWait().
What else can I do?
Or may be there is some better option of testing HS?
Thank You in advance
My new solution is to have test data defined as Spring beans and wire
them as Resources, by name. This part works.
sounds like a strange setup for a unit test. To be honest I am not quote sure how you do this.
In Search itself an in memory database (H2) is used together with a Lucene RAM directory. The benefits of such a setup is that it is fast and easy to avoid dependencies between tests.
I tried to have these beans persisted and indexed in setUp method of
my test cases (and in test methods themselves) but I failed. They get
into DB fine but I can't get them indexed.
If automatic indexing is enabled and the persisting of the test data is occurring within an transaction, it should work. A common mistake in combination with Spring is to use the wrong transaction manager. The Hibernate Search forum has a lot of threads around this, for example this one - https://forum.hibernate.org/viewtopic.php?f=9&t=998155. Since you are not giving any concrete configuration and code examples it is hard to give more specific advice.
I tried createIndexer().startAndWait()
that is also a good approach. I would recommend this approach if you want to insert not such a couple of test entities, but a whole set of data. In this case it can make sense to use a framework like dbunit to insert the testdata and then manually index the data. createIndexer().startAndWait() is the right tool for that. Extracting all this loading/persisting/indexing functionality into a common test base class is the way to go. The base class can also be responsible to do all the Spring bootstrapping.
Again, to give more specific feedback you have to refine your question.
I have a complete different approach, when I write any queries, i want to write a complete test suite, but data creation has always been pain(special mention to when test customer gets corrupt and all your test suite breaks.
To solve this I created Random-JPA. It's simple and easy to integrate. The whole idea is you create fresh data and test.
You Can find the full documentation here
Related
We have the need to create and persist rules at runtime. The goal is to create the rules, persist them and then reload them at a later point in time. Using bits and pieces of code cobbled together from drools unit tests, I can successfully create rules from DRL strings and then persist them to a kjar. And using the new KieBuilder.buildAll overload, the kjar (presumably) is built using the new executable model. All of that seems to work.
But what I really want to do is eliminate the DRL strings entirely and create my rules at runtime using the flow or pattern DSL. Again, using example code, I can create those rules at runtime, and execute them in a session. What I can’t seem to do is actually persist them as a kjar (or any other form that I can devise). It seems that the end result of building a rule using flow or pattern DSL is a KieBase. And there seems to be no way to serialize or persist a KieBase. At some point in the process, I need to be able to getBytes() in order to persist the KieBase.
For example, I can create the KieBase like this:
Rule rule = getRule();
ModelImpl model = new ModelImpl().addRule( rule );
KieBase kieBase = KieBaseBuilder.createKieBaseFromModel( model );
But I then need to be able to persist that newly created kieBase so it can be reloaded later. And there doesn't seem to be a workable way to do that.
Any suggestions? I’m using 7.7.0 for my testing.
UPDATE 2018-07-23
Let me clarify my original question with additional information. There are really two use cases where I’d like to use the new executable model to author rules in Java: 1) at design time; 2) at run time. Each use case has slightly different requirements, and so far I’ve been unsuccessful in getting either one to work completely.
For the 1st use case, at design time I need the ability to write rules in Java (using the new pattern DSL) and then save those rules to a kjar. Once there, they can be loaded into a KieServer instance and executed. Purportedly the Kie Maven Plugin can do this, and I’ve attempted to follow the instructions given in the drools doc (for example section 2.2.1.4 of the 7.8.0 doc). But those instructions appear to be incomplete, and there just aren’t any examples of how to accomplish this. What file or files need to be added to the resources\META-INF folder to identify the rules? How are the rules actually exposed in the Java code? Do they need to be in a particular type of class? Are the rules returned from public methods? How are those methods identified as having rules? Are any Java annotations needed to make this work?
All of those questions would be answered for me if there was just one simple end-to-end example that demonstrated how to author a rule in Java, AND create the kjar containing that rule.
For the 2nd use case (actually the more important of the two for me), I need the ability to dynamically create rules at runtime. Based on configuration data within our application, multiple rules need to be programmatically created and ultimately loaded into a KieServer instance. My assumption was that the process would be similar to use case #1 where a kjar could be programmatically created and then loaded into the KieServer. And remember that in this case, the Maven Plugin isn’t in the picture since this is all being done at runtime, not design time. Using the examples for the executable model (primarily the unit tests), I can author the rules in Java, and I can execute them. But I’ve found no way to actually build a kjar from them, or to directly load them into a KieServer.
To execute the rules, they have to be in a specific Java file and the kjar needs to have a file into the META-INF folder stating where the rules actually are.
Take a look at what's the maven plugin doing here
https://github.com/kiegroup/droolsjbpm-integration/blob/master/kie-maven-plugin/src/main/java/org/kie/maven/plugin/GenerateModelMojo.java#L165
There will be probably an easier way in the future, but I can't tell you when.
Thank you for using the bleeding edge features, and good luck with that.
I have a default Java EE 7 application with JPA/Eclipselink.
I want to write some UI tests with Arquillian Drone/Graphene.
My testsuite is "working". I can click through the application and make some asserts.
But: I want to create multiple tests. Every test class should reset the database, to make sure, that the conditions are always the same.
I'm using flyway to reset my database.
#Before
public void setup() {
Flyway flyway = new Flyway();
flyway.setDataSource(...);
flyway.clean();
flyway.init();
flyway.migrate();
}
The reset is working. And the first execution of the test also (at this situation, nothing is in any cache).
When I try to execute the test again, the database is inconsistent with strange JPA errors.
First I thought ok: Reset the cache
getEntityManager().getEntityManagerFactory().getCache().evictAll();
Is not enough. Same problems.
Next idea was to destroy the sessions (Some JPA data could be saved in old sessions). I haven't found a good way to destroy all sessions. I made a workaround, but this doesn't work also.
I think I have a default problem, but I can not found any solutions for this problem.
I also tried dbunit and the arquillian-persistence-extension. But it is like flyway, just another way.
In theory the problem is, that the database has been reset manually over SQL and Java/JPA/EclipseLink/The Sessions/deployed applaction have no idea about the changed data.
How can I reset everything (all Caches?)?
I also thought about "redeploy before start testclass". But this is a little bit to much (takes more time and is no fine solution)?
One more information: I'm also doing normal arquillian-tests (without UI/Selenium), here is my flyway database reset working.
Thanks for help :).
You didn't say what errors you really got, you just said:
When I try to execute the test again, the database is inconsistent with strange JPA errors (...)
It is hard to believe that your problem lies in JPA cache. I think that your problem has totally different source.
Your approach to cleaning the database has a fundamental flaw.
Code that you've presented: should be run only once, before all tests. Because Flyway is meant to prepare the database structure, not to setup it into known state.
So conceptually DbUnit & Arquillian Persistence Extension and Flyway do two really different things.
They are not a replacement for each other.
So your code that uses flyway:
#Before
public void setup() {
...
flyway.clean();
flyway.init();
flyway.migrate();
}
is wrong, because it should be run only once before all tests. To do that you can use some container features:
If using EJB Container, then #Singleton + #Startup + #PostConstruct combination could be used to launch flyway tool.
If using Spring Container, then init-method="migrate" would do the trick.
Or use maven and its pre-integration-test phase to launch flyway tool.
By the way: to avoid maintenance effort associated with DbUnit's xml datasets, personally I would recommend DbSetup tool. Nice and simple solution.
Edit
Besides the fact, that changing the DB structure during tests isn't a good practice - your problem may also be caused that both Flyway and JPA are using different datasources (even pointing to the same DB). You should double check that you're not creating DataSource on your own - just inject for the Flyway the same one that PersistenceUnit is using.
Working on a brand new project from the ground up. That means the data model is in a constant flux, doubly so because things are, inevitably, not as well planned as they should be. Model classes are being created and changed fairly regularly.
The plan was to use the latest version of EF with all the neat code-first stuff in it. But we're constantly tripping over the limitations the framework has in terms of adding or updating tables. The initialization options seem to allow only the complete deletion and re-creation of the database, which isn't really ideal.
I've had a look at the migrations. But this seems a sledgehammer to crack a nut: we don't need to detail every single small change and update with a new migration scaffold.
Are there some better strategies to deal with this? For instance, I started writing some unit tests to pre-populate one of the contexts with some test data, but because this causes the whole Db to drop and re-create, it causes problems with all the other contexts. Or perhaps making use of a custom initialiser to seed the data for us? How can we easily exclude these in production code?
We're also wondering about perhaps abandoning code-first and going back to EDMX diagrams. At least that way changes result in updated SQL commands which can be run directly against the database.
Any suggestions gratefully received.
I think, imho, that:
as the database schema must at least match your model you should/must detail every single change, and code first migration allows that and trace the changes over time
code first migration also allows to migrate the database schema for you
code first migration also allows you to produce sql that allows you to migrate the schema
For these reasons code first is as good (if not better) as the edmx approach
Please take few minutes to implement http://msdn.microsoft.com/en-us/data/jj591621.aspx
One other point, always imho and in a perfect world, if you unit test the business of you model you should not need the DAL, use generic collection. Be aware of different comportement of linq to object vs linq to entities, for example concerning the case sensitivity.
Please help me answer this question: should i write unit testing for data access that interact with local database of iOS app, in this case is SQLite database. If should, how can i write them? use mock up or use db file.
Assuming you want to test your program's logic and not just the ability to access SQLite, a test double (either a mock object or a dummy object) will give you test that's somewhat easier to maintain than a separate db file. A separate db file has to have the right data in the right rows, and if you modify it in one test you have to reset it before the next. If your test data gets out of sync, your tests will start to fail. A mock object with literal test values will never get out of sync.
Using a mock will pretty much force you to use dependency injection so you can substitute it in for the real data object. Using a db file will not force you to use dependency injection. So if you're working with a lot of existing code that doesn't follow the DI pattern, a db file would be the "easy" choice, although not the best choice from an object oriented perspective.
In the past, I've done this by creating a new SQLite DB file for each test case. In the test cases I would test that my code writes to the DB and reads the exact same thing that was written to it. This way all the test data is in the code so the test cases are clearer.
This approach sacrifices speed but my unit tests still ran pretty fast.
Let's say I have an entity called Product and this entity is loaded every time user hits the product information page. Usually I'd save the object in Zend_Cache (memcache) for an hour to avoid hitting the db for each request but as far as I understand that's not possible with Doctrine2 entities because of the Proxy objects.
So my question is, how can I avoid loading the same entity from the database for each request?
[EDIT]
I tried using Doctrine Cache like this
$categoryService = App_Service_Container::getService('\App\Service\Category');
$cache = $categoryService->getEm()->getConfiguration()->getResultCacheImpl();
$apple = $cache->fetch('apple');
But I get the following error
Warning: require(App/Entity/Proxy/_CG_/App/Entity/Category.php)
[function.require]: failed to open stream: No such file or directory
in /opt/vhosts/app/price/library/Doctrine/Common/ClassLoader.php on
line 163
This is same for Zend Cache as well as you can't serialize the entity because of the Proxy class
You've got several options:
Use Doctrine's built-in result caching
Try just sticking entity in memcache via Zend_Cache. When you pull it out, you may need to merge() the Product back into the EM so proxies can be dereferenced. If you fetch-join any associations you need to display the product info, and you're only doing reads, this shoudl work fine.
Don't cache the entity at all. Cache whatever output you generate instead.
EDIT: If you don't care about the hydration overhead, you're using mysql, and your Products and associated tables don't change very often, you might prefer to just rely on the mySQL query cache. It's a fairly blunt object, but useful enough to mention.
You might want to try implementing __sleep or __wakeup methods for your entity class, as Doctrine 2 has special requirements and limitations concerning serialization/deserialization of entities (which is what happens when storing them in Zend_Cache).
There is this guidance.
General information about limitations including serialization.
I find this extremely strange since i just messed around with this myself and didn't have any issues with the proxy object being stored in the database. So im guessing your configuration is not setup 100% ?
If you find the issue with your configuration then be very aware of what timdev said you MUST merge the object back into the EntityManager else you will have weird bugs down the line.
A fourth solution available for you is also to retrieve the data as an array instead of an object, but then of course you lose all the functionality connected to your module which might not be exactly want you wanted.
It seems to me more like a configuration error. Either Proxies have not been generated or there is something wrong with the proxy directory and namespace.
Depending on your configuration, proxies can be either generated automatically or manually. Does your proxies have been indeed generated under App/Entity/Proxy ? Is this indeed the right directory?
FYI proxies can be manually generated by executing doctrine orm:generate-proxies <dest-dir>
Seconding what timdev says: Doctrine has built-in caching, you want to use it.
I also wonder from your question if you are experiencing any performance issues or if you are a victim of overly eager optimisation.