Currently, I'm working on a Java EE project with some non-trivial requirements regarding persistence management. Changes to entities by users first need to be applied to some working copy before being validated, after which they are applied to the "live data". Any changes on that live data also need to have some record of them, to allow auditing.
The entities are managed via JPA, and Hibernate will be used as provider. That is a given, so we don't shy away from Hibernate-specific stuff. For the first requirement, two persistence units are used. One maps the entities to the "live data" tables, the other to the "working copy" tables. For the second requirement, we're going to use Hibernate Envers, a good fit for our use-case.
So far so good. Now, when users view the data on the (web-based) front-end, it would be very useful to be able to indicate which fields were changed in the working copy compared to the live data. A different colour would suffice. For this, we need some way of knowing which properties were altered. My question is, what would be a good way to go about this?
Using the JavaBeans API, a PropertyChangeListener could suffice to be notified of any changes in an entity of the working copy and keep a set of them. But the set would also need to be persisted, since the application could be restarted and changes can be long-lived before they're validated and applied to the live data. And applying the changes on the live data to obtain the working copy every time it is needed isn't feasible (hence the two persistence units).
We could also compare the working copy to the live data and find fields that are different. Some introspection and reflection code would suffice, but again that seems rather processing-intensive, not to mention the live data would need to be fetched.
Maybe I'm missing something simple, or someone know of a wonderful JPA/Hibernate feature I can use. Even if I can't avoid making (a) separate database table(s) for storing such information until it is applied to the live data, some best-practices or real-life experience with this scenario could be very useful.
I realize it's a semi-open question but surely other people must have encountered a requirement like this. Any good suggestion is appreciated, and any pointer to a ready-made solution would be a good candidate as accepted answer.
Maybe you can use the Hibernate flush entity event listener. The dirty properties are calculated before the flush. You can store them somewhere in your database.
A sample code of using the dirty properties feature of Hibernate which may give you an idea.
Related
Let's say we have design our system using akka-persistence. Now, we have events stored in the event-store. While, the system in production, a new feature was requested. As a result, we find the best way to go about it is to add or modify a field in an event. Let's say by changing the field name or type.
Now, we have two versions of the event, the one we have in production, and the one in the new deployment, which are not compatible. We will fail if we try to recover data from the old version.
What is the best way to go about that, other than data migration?
This is definitely one of the bigger issues with using akka persistence in production. There has been a lot of discussion about this on the akka-user mailing list.
I would say that as long as the new feature requires just additional information, going with a serialization format that allows limited schema evolution such as google protocol buffers or json would be a solution.
If the new feature requires changing the existing data, there is nothing you can do but to do data migration.
OK. I know that Entity Framework is ORM. We use it for mapping data from database to object model, and from objects to relational data. But where it fits in a context of persistance layer? Can we say that persistance layer is also Entity Framework?
I would say - No! There are a lot of articles about this topic. But in general you don't want your object-relational mapper to be data-persistent. In fact exactly the opposite, keeping it persistent ignorant you can benefit by using your data classes with different types of data providers such as relational databases, web services, XML files and what not.
To keep data persistence you may take advantage of different design patterns like Repository pattern and Unit Of Work so you can really decouple you business layer from your data layer.
Ok, to make myself clear since it's very difficult through comments, here's an update to what I wanted to explain. Please have in mind that this is just my interpretation, and the way I'm using EF, I've been using it in different projects (desktop and web) but it's not universal, but still covers a lot of the most common scenarios.
So since I'm a big fan of Code First I'll write from this prespective. The Database Model is where your entities lies. Later on based on those entities the EF will generate your database. So what is important on this stage of development - you want to have you database normalized and you want all navigation properties set correctly. Not so trivial tasks as it may seems but that it's, you just care about how efficient your database will be.
Now comes the tricky moment somehow you should deliver you data to the business layer and it's true - as far as we are talking only about data from a database using repository is very arguable. However even then the one advantage that you get when having this Repository between the data and the business logic is that you don't have to take in mind the business needs while creating the data model, and after that this doesn't make it any harder to use your data from inside the business layer even though what exactly will your front end looks like at the time you create the database model.
So at this point let's consider again the example case where in you Database Model you have those two entities - Customers and Orders. When a user log in into your application and wants to see his orders you need to join two tables in order to provide the front end the information that it needs. Option 1 - you don't have a Repository and you are using the DbContext directly from the method that returns the data. That means two things - you gonna have to write the same code everywhere you need to get this specific piece of information and 2 - if the business requirements change and in the same view that since now was used to show a customer and his orders now you have to show some additional info which is taken, let's say from a third table, then what happens - you have to go to each place where you use this view and change the way you retrieve the data. And option 2 - you have Repository, all your methods for accessing data are stored there and the Business Layer is completely ignorant about the way it get's the data, the Database Model is also ignorant about the needs of the business model which lead to loose coupling and only one place where you gonna have to make changes if you have to. In the scenario above, if you indeed use Repository and in your repository you have method called GetUserOrders() and inside this method you make the database call, the joins and so on, and all that the Business layer needs to do to get the data in the proper way is call this method when the requirements change and you have to include one more table, this time you don't have to look for all the places where you are using this data, you just have to modify one method and that's all.
It's pretty much the same logic on the way back. When you have some complex data returned from your front end and you want to save/update the old data with the new one, again - you can do it from the business layer but it leads to the same problem as when you have to get data, instead - you just pass the complex data to another Repository method which knows how to deal with it (say maybe some of the data should be saved directly into database and other should be used to feed a web service or whatever scenario comes to your mind) and here again - when something change, like - you want to use more heavily web services or the opposite, you want to migrate to more database centric design, all you have to do is change the method that takes care about the data the is concerned with this changes and nothing more.
So even though when I'm writing this I can see that DbContext can very well act as a repository and in this regard also as a data persistent layer, there are still some valid reason to not let this happen. Especially right now when the web services are more and more popular, WebAPI2 is out and RESTFull services are frequently used I think that leaving the EF as persistent ignorant as possible is the way to go.
But yet again, this is my opinion. There are a lot of articles on this topic so I urge you to google and read about it, since I think this is very important part form the architecture of every application.
P.S
In response to your comment which was written while I was writing my edited answer:
If I change data source I need to make changes in DAL anyway or in my example in repostitory. - the answer is yes. But there is no way tho change the data source without changing the DAL. The question is how easy will be to do that. I think the with what I've written already you can decide for yourself which way is better but just because I really think this is one of the few really strong arguments of leaving the EF persistent ignorant all write it again. When you have Repository and there are methods which take care for data manipulation, every time something related with the way the data is fetched affects only those methods and nothing else. If you use the context freely, in your business layer even a little change may cause you a lot of trouble just because it always possible to miss something, you have to go through the entire code to make sure that you have fixed all places and it's just not as efficient as having all in one place.
I like working with the entity framework for many reasons- the ease of use of the entity designer, the power of linq, and the ease of binding.
Occasionally I want to build a simple app that doesnt need to use a database, but still needs to work with data and display it on screen, in grids etc, so I'd like to just create a quick EF model and use it for this, but it doesnt seem to work very will with just using it for local data.
My question is- is there a correct usage of the EF for working with local data, and perhaps then just serialize/deserialize the whole context to a file? Or is this just too much effort to make work properyly? I used to use Datasets in this way, along with Linq to Dataset, and it works well... So perhaps those are still the better way to go for this scenario?
Yes you can use entity framework as local, and also access the data that is currently in-memory, read details as link below:
http://msdn.microsoft.com/en-us/data/jj592872.aspx
I don't know what you mean by "local data" exactly (sounds like it's not a database), but I think the Datasets vs. EF portion of your post is (for me) the real question.
EF is great when you need to model robust business logic, are implementing a Domain Model pattern, using Domain Driven Design, etc: basically any scenario where a Table Module or Active Record pattern is inappropriate.
When you just need to display some grids of data, and the business logic is very simple, Datasets are definitely the way to go (in my experience).
I am planning to create an iphone app which uses CoreData. There might be enhancements added later as new versions of the app.
My question is;
When using CoreData, what are the factors to keep in mind to ensure if the user upgrades the version, his previous data remains intact ? Like I heard we should keep the.sqlite file name same. What are other factors to keep in mind while releasing Core Data apps?
Thank you.
Data migration concepts are important to understand if you're going to maintain it over time, since you're likely to want to change at least some things eventually.
The ideal is Lightweight Migration, where minor conversion from your old data model to your new one is automatic. As noted in the document, it can take care of itself if your changes are:
Simple addition of a new attribute
A non-optional attribute becoming optional
An optional attribute becoming non-optional, and defining a default value
Renaming an entity or an attribute is also easy and nearly automatic.
Everything beyond that -- new or removed entities, new or removed or changed relationships -- is hairier. It's not incredibly difficult, but it's definitely more work, with more room for failure.
As such, a little speculation about likely potential changes may make it easier and more efficient to provide a little wiggle room in advance. Obviously if you do too much, especially with theoretical-but-currently-unused relationships, you're likely slowing down the current system and potentially for no reason.
Worth consideration.
One thing we have done is to manage two separate core data databases.
First, a "read-only" core data database that gets supplied with app updates (assuming you want to be sending data with the app, if not then don't bother with this part).
Second, a local core data database (data store) that's stored on the phone that is initially populated with the data from the first, and then added to by the user or with updates from a server that you control. This second core data store can stay persistent between updates.
For later modification and updates you have two options. You can add additional features in a new core data store as long as you don't need to get at the new data at the same time as the old data. The other option is to use apple's core data migration stuff which you can read more about here.
Here are also some additional resources for gearing up with core data, there are plenty of more specific core data examples on SO.
Finally, if you plan on significantly adding/modifying your core data store I'd suggest looking into SQLite. That's easier to change with updates (in my experience) than migrating an existing core data store to a new schema, especially if the schema changes often.
I wonder in which cases it would be good to make an NSManagedObjectModel completely programmatically, with NSEntityDescription instances and all this stuff.
I'm that kind of person who prefers to code programmatically, rejecting Interface Builder. But when it comes to Core Data, I have a hard time figuring out why I should kill my time NOT using the nice Xcode Data Modeler tool.
And since data models are stuck to a given state (except when you want to do some ugly migration operations where thinks probably go wrong and users get mad, really mad), I see no big sense in a data model that's made programmatically for the purpose of changing it all the time.
Did I miss something?
I dont think you missed anything. The only reason I can see to create your model programatically would be if the objects you are modeling are themselves dynamic: you could for instance build a coredata entity (or graph of entities) in response to a web service which changed over time, or was selected by the user. However, I think if you had that or a similar use case, you wouldn't need to write this question (and you'd probably solve it a different way anyway)
So, if your application is dealing with resources that are dynamic, as #Andiih mentioned, then this programatic is the only way to do it. I don't know what my core data entities are until runtime, I don't know what the attributes are, or what the data looks like. So, I ask the server to give me the kinds of resources I should support and what their attributes look like. I build the model, the entities, the properties, the relationships - at runtime. I still want to use Core Data because I'm dealing with a lot of data and I need the benefit of efficient memory management with NSFetchedResultsController, etc. I can only do this programmatically.
The trouble is how to handle migration to try and preserve as much of the persistent store as possible, to reduce the size of the networked data payload after the model changes. Right now I blast the whole model and the persistent store if there's a conflict. I haven't yet figured out a way to create an .xcdatamodel from a programmatically generated model, thus I can't yet create a version mapping to do the migration.
Everything is a trade off. Basically, I think IB and the visual Core Data modeler are the right tool if you're building a simple application. You'll need to make the determination when your application becomes large/complex enough that you prefer to have direct control over all aspects of the code.
Regarding Interface Builder, if you have an application with a variety of complex interactions between view controllers, and multiple custom controls, I find code more appropriate.
For Core Data, the question is pretty much the same. Does your project have a defined scope? Can you foresee everything in that scope being done within the visual modeler? If so, it's probably fine. For other projects, where you may be asked to add features on an ongoing basis, perhaps it's better to spend a little more time writing it out so you have more flexibility later.
One other thing to consider, that doesn't get mentioned much, is it's MUCH harder to ask for help with IB or any hybrid visual design/code system. When something does go wrong, or you need help, it's way easier to post your code, than try to explain what's going on in a visual modeler.
In general, there's no reason to build the managed object model in code. There's nothing you can do in code that can't be done in the model editor. There are some fancy tricks you can do in code, however, to work with multiple models. For example, you can merge two models, establishing cross-model relationships between entities in those models at load time (see Cross-model relationships in NSManagedObjectModel from merged models?).
Regarding whether it's a good idea to code or use the graphical editor, I think the balance tips heavily towards the graphical editor in this case. Being able to verify the model by visual inspection instead of (rather convoluted) code is a win. The model can still be verified by unit test, if you desire.
I have one use case that might be valid, what if you load some data from the internet whether it is XML from an RSS Feed or WSDL response, then flatten those responses into a tabular from generating an in memory data table and finally mash it all up into a single coherent data model, then you can create the entities for those in memory data tables and create master/detail relationships. That's one case I think Core Data data model generated programmatically could become handy and a powerful feature.
I've changed models programmatically in unit tests. For example, I wrote a class that is designed to work with Core Data models that have a particular protocol attached. Instead of testing against a random implementation, I mutated the default model by adding one just in the unit tests programmatically, and tested against that test-only model.