We need to implement a comprehensive audit log of changes for an aggregate entity called Project, where users can see what changed with each save and the before and after values of those changes. The entity contains a few child collections (for example a collection of TeamMember entities).
There are options like EF Plus (http://entityframework-plus.net/) which manages the audit for you or we could override EFs savechanges() and write to our own audit table manually.
Where things get tricky are where you have a navigation property. Any child collections such as TeamMembers are seen as a separate entry in the change tracker and therefore would be logged as an update independently of project. We need to be able to track all changes for a specific project i.e. anything that has changed within that object graph.
Having searched quite exhaustively online, we are at a bit of loss as to what to try. The option we are looking at now is serialising into JSON the whole Project entity, prior to and after save, into an audit log. We then need to write some custom comparison logic to see whats changed and return that to the UI. This seems like unnecessary complexity.
If anyone in the community has done this before and has a good approach or any ideas here it would be of great help.
Related
Currently, I'm working on a Java EE project with some non-trivial requirements regarding persistence management. Changes to entities by users first need to be applied to some working copy before being validated, after which they are applied to the "live data". Any changes on that live data also need to have some record of them, to allow auditing.
The entities are managed via JPA, and Hibernate will be used as provider. That is a given, so we don't shy away from Hibernate-specific stuff. For the first requirement, two persistence units are used. One maps the entities to the "live data" tables, the other to the "working copy" tables. For the second requirement, we're going to use Hibernate Envers, a good fit for our use-case.
So far so good. Now, when users view the data on the (web-based) front-end, it would be very useful to be able to indicate which fields were changed in the working copy compared to the live data. A different colour would suffice. For this, we need some way of knowing which properties were altered. My question is, what would be a good way to go about this?
Using the JavaBeans API, a PropertyChangeListener could suffice to be notified of any changes in an entity of the working copy and keep a set of them. But the set would also need to be persisted, since the application could be restarted and changes can be long-lived before they're validated and applied to the live data. And applying the changes on the live data to obtain the working copy every time it is needed isn't feasible (hence the two persistence units).
We could also compare the working copy to the live data and find fields that are different. Some introspection and reflection code would suffice, but again that seems rather processing-intensive, not to mention the live data would need to be fetched.
Maybe I'm missing something simple, or someone know of a wonderful JPA/Hibernate feature I can use. Even if I can't avoid making (a) separate database table(s) for storing such information until it is applied to the live data, some best-practices or real-life experience with this scenario could be very useful.
I realize it's a semi-open question but surely other people must have encountered a requirement like this. Any good suggestion is appreciated, and any pointer to a ready-made solution would be a good candidate as accepted answer.
Maybe you can use the Hibernate flush entity event listener. The dirty properties are calculated before the flush. You can store them somewhere in your database.
A sample code of using the dirty properties feature of Hibernate which may give you an idea.
OK. I know that Entity Framework is ORM. We use it for mapping data from database to object model, and from objects to relational data. But where it fits in a context of persistance layer? Can we say that persistance layer is also Entity Framework?
I would say - No! There are a lot of articles about this topic. But in general you don't want your object-relational mapper to be data-persistent. In fact exactly the opposite, keeping it persistent ignorant you can benefit by using your data classes with different types of data providers such as relational databases, web services, XML files and what not.
To keep data persistence you may take advantage of different design patterns like Repository pattern and Unit Of Work so you can really decouple you business layer from your data layer.
Ok, to make myself clear since it's very difficult through comments, here's an update to what I wanted to explain. Please have in mind that this is just my interpretation, and the way I'm using EF, I've been using it in different projects (desktop and web) but it's not universal, but still covers a lot of the most common scenarios.
So since I'm a big fan of Code First I'll write from this prespective. The Database Model is where your entities lies. Later on based on those entities the EF will generate your database. So what is important on this stage of development - you want to have you database normalized and you want all navigation properties set correctly. Not so trivial tasks as it may seems but that it's, you just care about how efficient your database will be.
Now comes the tricky moment somehow you should deliver you data to the business layer and it's true - as far as we are talking only about data from a database using repository is very arguable. However even then the one advantage that you get when having this Repository between the data and the business logic is that you don't have to take in mind the business needs while creating the data model, and after that this doesn't make it any harder to use your data from inside the business layer even though what exactly will your front end looks like at the time you create the database model.
So at this point let's consider again the example case where in you Database Model you have those two entities - Customers and Orders. When a user log in into your application and wants to see his orders you need to join two tables in order to provide the front end the information that it needs. Option 1 - you don't have a Repository and you are using the DbContext directly from the method that returns the data. That means two things - you gonna have to write the same code everywhere you need to get this specific piece of information and 2 - if the business requirements change and in the same view that since now was used to show a customer and his orders now you have to show some additional info which is taken, let's say from a third table, then what happens - you have to go to each place where you use this view and change the way you retrieve the data. And option 2 - you have Repository, all your methods for accessing data are stored there and the Business Layer is completely ignorant about the way it get's the data, the Database Model is also ignorant about the needs of the business model which lead to loose coupling and only one place where you gonna have to make changes if you have to. In the scenario above, if you indeed use Repository and in your repository you have method called GetUserOrders() and inside this method you make the database call, the joins and so on, and all that the Business layer needs to do to get the data in the proper way is call this method when the requirements change and you have to include one more table, this time you don't have to look for all the places where you are using this data, you just have to modify one method and that's all.
It's pretty much the same logic on the way back. When you have some complex data returned from your front end and you want to save/update the old data with the new one, again - you can do it from the business layer but it leads to the same problem as when you have to get data, instead - you just pass the complex data to another Repository method which knows how to deal with it (say maybe some of the data should be saved directly into database and other should be used to feed a web service or whatever scenario comes to your mind) and here again - when something change, like - you want to use more heavily web services or the opposite, you want to migrate to more database centric design, all you have to do is change the method that takes care about the data the is concerned with this changes and nothing more.
So even though when I'm writing this I can see that DbContext can very well act as a repository and in this regard also as a data persistent layer, there are still some valid reason to not let this happen. Especially right now when the web services are more and more popular, WebAPI2 is out and RESTFull services are frequently used I think that leaving the EF as persistent ignorant as possible is the way to go.
But yet again, this is my opinion. There are a lot of articles on this topic so I urge you to google and read about it, since I think this is very important part form the architecture of every application.
P.S
In response to your comment which was written while I was writing my edited answer:
If I change data source I need to make changes in DAL anyway or in my example in repostitory. - the answer is yes. But there is no way tho change the data source without changing the DAL. The question is how easy will be to do that. I think the with what I've written already you can decide for yourself which way is better but just because I really think this is one of the few really strong arguments of leaving the EF persistent ignorant all write it again. When you have Repository and there are methods which take care for data manipulation, every time something related with the way the data is fetched affects only those methods and nothing else. If you use the context freely, in your business layer even a little change may cause you a lot of trouble just because it always possible to miss something, you have to go through the entire code to make sure that you have fixed all places and it's just not as efficient as having all in one place.
I have a web application that allows the clerk to edit information. I copy an entity before the edit starts in case the user decides to cancel the changes. The problem is that any changes made on the copy is applied to the original object.
In C# I would create a deep copy to avoid that issue, but this application is using Entity Framework... I am not sure how to do a deep copy of an entity.
Here is more details on my problem... I am still trying to resolve.
I have a xaml screen with a grid binded to a list of inventory items. The items are an EntitySet. When I want the user edits one of the items, I copy the values of the current entity in an object "EntityToEdit" of the same type. The user makes a change, saves, and the list is automatically refreshed with the changes.
The problem occurs when the user selects another item to edit. That second item is somehow changed with the changes made on the first item....
How can I break the "link"?!?
There is a lot of stuff going on in your question, I barely know where to start.
First, I do not recommend binding the Entities directly to your UI. Instead I recommend using some form of abstraction between your database and the presentation layer of your application.
There are a couple of established best-practices patterns that could be used, mostly depending on what technical environment you are working in. Have a look at the MVC pattern (Model-View-Controller) or at MVVM (Model-View-ViewModel).
Regarding your deep copy, there are lots of possible solutions. You can find some here at StackOverflow, like this one: "How do you do a deep copy an object in .Net?", where the objects are serialized into a Stream. Altough I, too, was using this kind of deep copy, but I try to avoid this. I see only very rare cases where a deep copy is really needed.
You should also consider implementing the [IEditableObject][4] interface on those objects that you want to edit. This allows you for an easy way to structure when and how your edited values are commited and to optionally or cancel or reset your edits by implementing straight forward methods like BeginEdit(), EndEdit(), and CancelEdit(). Some .NET controls that support this interface out-of-the-box and call these interface methods automatically if they exist on your objects.
By implementing IEditableObject you have full control of how the values are committed to which object. This will help you to avoid changing the original object accidentally.
I wonder in which cases it would be good to make an NSManagedObjectModel completely programmatically, with NSEntityDescription instances and all this stuff.
I'm that kind of person who prefers to code programmatically, rejecting Interface Builder. But when it comes to Core Data, I have a hard time figuring out why I should kill my time NOT using the nice Xcode Data Modeler tool.
And since data models are stuck to a given state (except when you want to do some ugly migration operations where thinks probably go wrong and users get mad, really mad), I see no big sense in a data model that's made programmatically for the purpose of changing it all the time.
Did I miss something?
I dont think you missed anything. The only reason I can see to create your model programatically would be if the objects you are modeling are themselves dynamic: you could for instance build a coredata entity (or graph of entities) in response to a web service which changed over time, or was selected by the user. However, I think if you had that or a similar use case, you wouldn't need to write this question (and you'd probably solve it a different way anyway)
So, if your application is dealing with resources that are dynamic, as #Andiih mentioned, then this programatic is the only way to do it. I don't know what my core data entities are until runtime, I don't know what the attributes are, or what the data looks like. So, I ask the server to give me the kinds of resources I should support and what their attributes look like. I build the model, the entities, the properties, the relationships - at runtime. I still want to use Core Data because I'm dealing with a lot of data and I need the benefit of efficient memory management with NSFetchedResultsController, etc. I can only do this programmatically.
The trouble is how to handle migration to try and preserve as much of the persistent store as possible, to reduce the size of the networked data payload after the model changes. Right now I blast the whole model and the persistent store if there's a conflict. I haven't yet figured out a way to create an .xcdatamodel from a programmatically generated model, thus I can't yet create a version mapping to do the migration.
Everything is a trade off. Basically, I think IB and the visual Core Data modeler are the right tool if you're building a simple application. You'll need to make the determination when your application becomes large/complex enough that you prefer to have direct control over all aspects of the code.
Regarding Interface Builder, if you have an application with a variety of complex interactions between view controllers, and multiple custom controls, I find code more appropriate.
For Core Data, the question is pretty much the same. Does your project have a defined scope? Can you foresee everything in that scope being done within the visual modeler? If so, it's probably fine. For other projects, where you may be asked to add features on an ongoing basis, perhaps it's better to spend a little more time writing it out so you have more flexibility later.
One other thing to consider, that doesn't get mentioned much, is it's MUCH harder to ask for help with IB or any hybrid visual design/code system. When something does go wrong, or you need help, it's way easier to post your code, than try to explain what's going on in a visual modeler.
In general, there's no reason to build the managed object model in code. There's nothing you can do in code that can't be done in the model editor. There are some fancy tricks you can do in code, however, to work with multiple models. For example, you can merge two models, establishing cross-model relationships between entities in those models at load time (see Cross-model relationships in NSManagedObjectModel from merged models?).
Regarding whether it's a good idea to code or use the graphical editor, I think the balance tips heavily towards the graphical editor in this case. Being able to verify the model by visual inspection instead of (rather convoluted) code is a win. The model can still be verified by unit test, if you desire.
I have one use case that might be valid, what if you load some data from the internet whether it is XML from an RSS Feed or WSDL response, then flatten those responses into a tabular from generating an in memory data table and finally mash it all up into a single coherent data model, then you can create the entities for those in memory data tables and create master/detail relationships. That's one case I think Core Data data model generated programmatically could become handy and a powerful feature.
I've changed models programmatically in unit tests. For example, I wrote a class that is designed to work with Core Data models that have a particular protocol attached. Instead of testing against a random implementation, I mutated the default model by adding one just in the unit tests programmatically, and tested against that test-only model.
So, I've been searching for the answer to this, but I can't find anything
I have an Entity Framework Model (MyModel1) - for now, we'll say this contains a "Users" table
It's part of a big app, that has a references to an "Addresses" project
The addresses project contains an Entity Framework Model (MyModel2), this contains a Users table, and an Addresses table (pointing to the same database.
The main app has a control that edits the user, and in that control it has an "addresses" control which actually exists in the "Addresses" project.
To make this work, the User control passes the User object down to the addresses control, however, as the User that's been passed belongs to MyModel1 and not MyModel2, another User object has to be loaded up, then it can be used.
This isn't ideal as I've had to load up the User twice. Is there a way of say, MyModel2 extending MyModel1, which effectively just adds a relationship to "User". Or is there an ORM that would handle this better? Or even a design pattern that would handle this better?
I discovered Fluent NHibernate which seems to give me a load more control over how the data layer is put together, through some seriously crazy code I was able to extend the entities in a plugin kind of way, very cool
It sounds like today you have a projects that are a mixture of UI, business logic and data access logic.
A better approach would be to put your Data Access layer into a single project separate from the business logic and the UI. Create an EDMX that includes both Users and Addresses and provide a single ObjectContext that can be used to handle the whole process.
Take a look at the Repository pattern too.