I have an architectural question about EF and WCF.
We are developing a three-tier application using Entity Framework (with an Oracle database), and a GUI based on WPF. The GUI communicates with the server through WCF.
Our data model is quite complex (more than a hundred tables), with lots of relations. We are currently using the default EF code generation template, and we are having a lot of trouble with tracking the state of our entities.
The user interfaces on the client are also fairly complex, sometimes an object graph with more than 50 objects are sent down to a single user interface, with several layers of aggregation between the entities. It is an important goal to be able to easily decide in the BLL layer, which of the objects have been modified on the client, and which objects have been newly created.
What would be the clearest approach to manage entities and entity states between the two layers? Self tracking entities? What are the most common pitfalls in this scenario?
Could those who have used STEs in a real production environment tell their experiences?
STEs are supposed to solve this scenario but they are not silver bullet. I have never used them in real project (I don't like them) but I spent some time playing with them. The main pitfalls I found are:
Coupling your data layer with your client application - you must share entity assembly between projects (it also means it is .NET only solution but it should not be a problem in your case)
Large data transfers - you pass 50 entities to clients, client change single entity and you will pass 50 entities back. It will require some fighting with STEs to avoid passing unnecessary data
Unnecessary updates to database - normally when EF works with attached entities it track changes on property level but with STEs it track changes on entity level. So if user modify single property in entity with 100 properties it will generate update with setting all of them. It will require modifying template and adding property level change tracking to avoid this.
Client application should use STEs directly (binding STEs to UI) to get most of its self tracking ability. Otherwise you will have to implement code which will move data from UI back to self tracking entity and modify its state.
They are not proxied = they don't support lazy loading (in case of WCF service it is good behavior)
I described today the way to solve this without STEs. There is also related question about tracking over web services (check #Richard's answer and provided links).
We have developed a layered application with STE's. A user interface layer with ASP.NET and ModelViewPresenter, a business layer, a WCF service layer and the data layer with Entity Framework.
When I first read about STE's the documentation said that they are easier then using custom DTO's. They should be the 'quick and easy way' and that only on really big projects you should use hand written DTO's.
But we've run in a lot of problems using STE's. One of the main problems is that if your entities come from multiple service calls (for example in a master detail view) and so from different contexts you will run into problems when composing the graphs on the server and trying to save them. So our server function still have to check manually which data has changed and then recompose the object graph on the server. A lot has been written about this topic but it's still not easy to fix.
Another problem we ran into was that the STE's wouldn't work without WCF. The change tracking is activated when the entities are serialized. We've originally designed an architecture where WCF could be disabled and the service calls would just be in process (this was a requirement for our unit tests, which would run a lot faster without wcf and be easier to setup). It turned out that STE's are not the right choice for this.
I've also noticed that developers sometimes included a lot of data in their query and just send it to the client instead of really thinking about which data they needed.
After this project we've decided to use custom DTO's with automapper from server to client and use the POCO template in our data layer in a new project.
So since you already state that your project is big I would opt for custom DTO's and service functions that are a specifically created for one goal instead of 'Update(Person person)' functions that send a lot of data
Hope this helps :)
Related
We are currently rewriting an existing internal ASP.NET Web Forms application. Our application consists of a Web Api back end which uses Entity Framework 6 for data access and an front end which uses AngularJS.
We have an existing large database that I've created EF models using the Code-First Using Existing Database method and we are using data transfer object classes as inputs/outputs to our API methods so we aren't directly exposing our model classes. So basically, I'm trying to become proficient with EF, Web Api and AngularJS all at the same time. For the most part I'm fairly comfortable with the latter two, but for EF I haven't completely gotten comfortable with. I've watched a lot of the videos on Microsoft Virtual Academy but this is the first time I've had some hands-on experience with it.
We've been working on this application for a few months and so far we've only had to work with CRUD operations on our entities (POCO DTO's) which are flat objects with simple properties. However, we've finally come across some situations where we need to deal not only with our classes, but properties which are classes themselves; a parent-child relationship.
Therefore, I have the following questions:
I see that when we have a proper foreign key relationship in our DB, that virtual properties are created in EF, which from what I recall are to support lazy-loading. However, lazy-loading isn't really feasible in this environment where we are using web services (Web Api). Our object model does allow for some really large hierarchy of classes where a fully populated object and its children would mean a large amount of data would be passed around when that really isn't necessary, so in most cases a first level object is all we need. In some cases however, we do want to populate child classes, so my question is how do we do that, and where do we do that? I've looked at the automatically-generated code in the DB Context but we have also used scaffolded code to create our controllers. Which place do we need to do this? I've seen code samples showing how do to this but it hasn't said specifically where this code lies. It appears to be within a controller but I could be wrong.
If we do allow for 2- or more level hierarchy of objects, does EF automatically handle operations (updates, deletes, etc.) -- for example, if we have a "Company" object which has a collection of "Customer" objects, and we delete the "Company" object, do the related "Customer" objects get deleted too? Also, is a multi-step operation like that automatically performed within a transaction or do we need to explicitly set that up?
If I modify a model class or the DB context, seeing as this code is automatically-generated, that's generally bad practice as my changes could be overwritten, so I am assuming the controller code is where I want to make my changes. I am aware of database migrations but I have no experience with them and I am sure I'll need to use them at some point because I am fairly confident that our database may not have all the foreign key relationships necessary for EF to do everything we need at the moment.
I know this is a long post, but if anyone can give some guidance on how to do some of these things because it's not only me that's having to deal with this but I have two other developers on my team who are working on this project and we are all as inexperienced with this as the others are. Thanks
For the purpose of sending data across a web service, I'd suggest creating a DTO to hold the data you want to send and mapping your entities to the DTO instead of trying to send the entities themselves in your payload. It also protects your API from changes to your entity.
Cascading deletes are configurable, iirc, but I'm not 100% sure what the default is. Transactions are generally not implicit, so you will want to use those where you require them.
Not exactly sure what you are asking here. In general, how your entities/tables change depends on if you are using database-first or code-first. If you are using database-first (you will have a .edmx file in your solution that has the model matching your schema), you just update the SQL directly and update your entity model via the .edmx. If you use code-first, you will change the entities how you want them and run a database migration to update your database to match.
MSDN article about code-first migration: https://msdn.microsoft.com/en-us/data/jj591621.aspx
We are currently looking at migrating from WCF Data Services to WebApiOData. The problem that I see is that we have to create a method within a webApi controller for every single table/view that we have that we need to query from the client. The entity model is defined in an edmx with quite a few tables. Having one standard odata.svc that just allowed us to add a table to the edmx and just query it straight through the odata.svc was gold, having to add a method every time we add a view or table to the edmx will be a nightmare and if we are talking about > 50 tables/views it will just turn into a mess of methods everywhere.
Is there a simpler strategy for just having one controller that defines a global get method that we can just query on for all tables/views in the edmx? Or am I needing to create methods for everything?
RESTier is a layer of abstraction over Web API OData that should feel similar in ways to WCF Data Services, but still allow the flexibility of Web API OData. It already has an EF provider so ideally you should be up and running within minutes with a very small amount of code. Whether it works or doesn't work, we'd love to hear about your experience either in the comments on the GitHub pages or at odatafeedback#microsoft.com.
I need to retrieve a large graph of entities, manipulate it in the UI (adds, updates, deletes), then persist it all back to the database. After various SO questions and experiments, I'm finding this mass "detached graph update" approach to be very problematic, so I'm now rethinking my approach.
It's only a 2-tier WPF app, so I'm now thinking of having a long-running context that exists for the duration of the UI used to manipulate the entity graph - that way it can track changes automatically. However I'm not sure how to approach this architecturally.
The application currently has three projects - the UI, business tier, and one for the edmx & generated entities. My business tier has a CustomerManager class that exposes a method to retrieve a Customer graph (orders, order lines, etc.), and a method to persist the Customer graph. Assuming that the UI holds on to the same instance of the CustomerManager class, and therefore the same context, changes to the graph (adding and changing entities) will be tracked.
Deleting an entity is a bit more tricky, as the context must be used to do this, i.e.:-
context.Set<Order>().Remove(orderToDelete);
Looking for some architectural advice really. Do I just expose a DeleteOrder method in my CustomerManager class that does this? Given that I have a dozen other entity types, I would presumably need to expose similar methods to delete orders, products, etc.
Is it a sensible approach for the UI to hold on to the same CustomerManager instance, or is there a better way to manage a long-running context? A logical place for the DeleteOrder method would be in my Customer entity (partial) class, but as these classes are in a separate project from the business tier (which is where the context resides), I guess I can't do this (unless I pass the context to the DeleteOrder method)?
Your long living context idea will work only if your context lives in UI and UI talks to database directly to get and persist data. Involving WCF between your UI and context always result in serialization and it causes entity detaching = not tracking changes (unless you use STEs). Having long living context in WCF service is too problematic and in general bad practice.
Have you considered WCF Data Services? They provide client side tracking to some extend by using special client side context.
I have a EF domain model to pass around information between my ASP.NET MVC controllers and my service layer. The MVC Controller is disconnected from the data source and relies completely on the service for loading.
I am using EF 4.3 and Code First.
Because of this, I do not use the EF lazy load functionality in the MVC layer of my app. So, many objects are "partially loaded" as I do not need the data and it's ineffcient to load it.
However, I constantly run into Null Reference Exceptions during development, since there's no way to statically knowing this, short of making a ton of DTOs that represent only the data that is being passed.
I am not fond of creating a lot of DTOs, as that creates a lot more code and mapping issues, and doesn't necessarily reduce the amount of bugs that might be caused. but I suspect that is the only real way to make my code cleaner here.
Is there a design pattern to account for disconnected, n-tier Entity Framework applications other than DTOs?
I really like OData (WCF Data Services). In past projects I have coded up so many Web-Services just to allow different ways to read my data.
OData gives great flexibility for the clients to have the data as they need it.
However, in a discussion today, a co-worker pointed out that how we are doing OData is little more than giving the client application a connection to the database.
Here is how we are setting up our WCF Data Service (Note: this is the traditional way)
Create an Entity Framework (E)F Data Model of our database
Publish that model with WCF Data Services
Add Security to the OData feed
(This is where it is better than a direct connection to the SQL Server)
My co-worker (correctly) pointed out that all our clients will be coupled to the database now. (If a table or column is refactored then the clients will have to change too)
EF offers a bit of flexibility on how your data is presented and could be used to hide some minor database changes that don't affect the client apps. But I have found it to be quite limited. (See this post for an example) I have found that the POCO templates (while nice for allowing separation of the model and the entities) also does not offer very much flexibility.
So, the question: What do I tell my co-worker? How do I setup my WCF Data Services so they are using business oriented contracts (like they would be if every read operation used a standard WCF Soap based service)?
Just to be clear, let me ask this a different way. How can I decouple EF from WCF Data Services. I am fine to make up my own contracts and use AutoMapper to convert between them. But I would like to not go directly from EF to OData.
NOTE: I still want to use EF as my ORM. Rolling my own ORM is not really a solution...
If you use your custom classes instead of using classes generated directly by EF you will also change a provide for WCF Data Services. It means you will no more pass EF context as generic parameter to DataService base class. This will be OK if you have read only services but once you expect any data modifications from clients you will have a lot of work to do.
Data services based on EF context supports data modifications. All other data services use reflection provider which is read only by default until you implement IUpdatable on your custom "service context class".
Data services are technology for creating quickly services exposing your data. They are coupled with their context and it is responsibility of the context to provide abstraction. If you want to make quick and easy services you are dependent on features supported by EF mapping. You can make some abstractions in EDMX, you can make projections (DefiningQuery, QueryView) etc. but all these features have some limitations (for example projections are readonly unless you use stored procedures for modifications).
Data services are not the same as providing connection to database. There is one very big difference - connection to database will ensure only access and execution permissions but it will not ensure data security. WCF Data Services offer data security because you can create interceptors which will add filters to queries to retrieve only data the user is allowed to see or check if he is allowed to modify the data. That is the difference you can tell your colleague.
In case of abstraction - do you want a quick easy solution or not? You can inject abstraction layer between service and ORM but you need to implement mentioned method and you have to test it.
Most simple approach:
DO NOT PUBLISH YOUR TABLES ;)
Make a separate schema
Add views to this
Put those views to EF and publish them.
The views are decoupled from the tables and thus can be simplified and refactored separately.
Standard approach, also for reporting.
Apart from achieving more granular data authorisation (based of certain field values etc) OData also allows your data to be accessible via open standards like JSON/Xml over Http using OAuth. This is very useful for the web/mobile applications. Now you could create a web service to expose your data but that will warrant a change every time your client needs change in the data requirements (e.g. extra fields needed) whereas OData allows this via OData queries. In a big enterprise this is also useful for designing security at infrastructure level as it will only allow the text based (http) calls which can be inspected/verified for security threats via network firewalls'.
You have some other options for your OData client. Have a look at Simple.OData.Client, described in this article: http://www.codeproject.com/Articles/686240/reasons-to-consume-OData-feeds-using-Simple-ODa
And in case you are familiar with Simple.Data microORM, there is an OData adapter for it:
https://github.com/simplefx/Simple.OData/wiki
UPDATE. My recommendations go for client choice while your question is about setting up your server side. Then of course they are not what you are asking. I will leave however my answer so you aware of client alternatives.