I'm developing a data access layer for a database with over 700 tables. I created the model including all the tables, which generated a huge model. I then changed the model to use DBContext from 4.1 which seemed to improve how it compiled and worked. The designer didnt seem to work at all.
I then created a test app which just added two records to the table, but the processor went 100% in the db.SaveChanges method. Being a black box it was difficult to accertain what went wrong.
So my questions are
Is the entity framework the best approach to a large database
If so, should the model be broken down into logical areas. I did note that you cant have the same sql table in multiple models
I have read that the code only approach is best in these large cases. What is that.
Any guidance would be truly appreciated
Thanks
Large database is always something special. Any technology has some pros and cons when working with a large database.
The problem you have encountered is the most probably related to building the model. When you start the application and use EF related stuff for the first time EF must build the model description and compile it - this is the most time consuming operation you can find in EF. Complexity of this operation grows with number of entities in the model. Once the model is compiled it is reused for the whole lifetime of the application (if you restart the application or unload application domain the model must be compiled again). You can avoid this by precompiling the model. It is done at design time where you use some tool to generate code from the model and you include that code into your project (it must be done again after each change in the model). For EDMX based models you can use EdmGen.exe to generate views and for code first based models you can use EF Power Tools CTP1.
EDMX (the designer) was improved in VS 2010 SP1 to be able to work with large models but I still think the large in this case is around 100 entities / tables. In the same time you rarely need 715 tables in the same model. I believe that these 715 tables indeed model several domains so you can divide them into multiple models.
The same is true when you are using DbContext and code first. If you model a class do you think that it is correct design when the class exposes 715 properties? I don't think so but that is exactly what your derived DbContext looks like - it has a public property for each exposed entity set (in the simplest mapping it means one property per table).
Same entity can be used in multiple models but you should try to avoid it as much as possible because it can introduce some complexities when loading entity in one context type and using it in other context type.
Code only = code first = Entity framework when you define mapping in the code without using EDMX.
take a look this post.
http://blogs.msdn.com/b/adonet/archive/2008/11/24/working-with-large-models-in-entity-framework-part-1.aspx
Related
Is there a way to combine code-first and database-first in the same context? We are running into massive development-time performance problems when editing the EDMX file (it takes 1.5 minutes to save). I've moved our non-insert/update/delete UDFs/stored procs to some custom T4 templates that automatically generate model-first code, but I can't seem to get OnModelCreating to be called when EDMX is involved.
Other things we've considered, but won't work for one reason or another:
We can't (reasonably) separate our code to multiple contexts as there is a lot of overlap in our entity relationships. It also seems like quite a people who have gone this route regret it.
We tried having 2 different contexts, but there are quite a few joins between Entities & UDFs. This may be our last hope, but I'd REALLY like to avoid it.
We can't switch to Dapper since we have unfortunately made heavy use of IQueryable.
We tried to go completely to Code-First, but there are features that we are using in EDMX that aren't supported (mostly related to insert/update/delete stored procedure mapping).
Take a look at the following link. I answered another question in a similar fashion:
How to use Repository pattern using Database first approach in entity framework
As I mentioned in that post, I would personally try to switch to a Code First approach and get rid of the EDMX files as it is already deprecated and most importantly, the maintenance effort is considerable and much more complex compared with the Code First approach.
It is not that hard switching to Code First from a Model First approach. Some steps and images down below:
Display all files at the project level and expand the EDMX file. You will notice that the EDMX file has a .TT file which will have several files nested, the Model Context and POCO clases between them as .cs or .vb classes (depending on the language you are using). See image down below:
Unload the project, right click and then edit.
See the image below, notice the dependencies between the context and the TT file
Remove the dependencies, the xml element should look like the image below:
Repeat the procedure for the Model classes (The ones with the model definition)
Reload your project, remove the EDMX file(s)
You will probably need to do some tweeks and update names/references.
I did this a few times in the past and it worked flawlessly on production. You can also look for tools that do this conversion for you.
This might be a good opportunity for you to rethink the architecture as well.
BTW: Bullet point 4 shouldn't be a show stopper for you. You can map/use Stored Procedures via EF. Look at the following link:
How to call Stored Procedure in Entity Framework 6 (Code-First)?
It also seems like quite a people who have gone this route [multiple contexts] regret it.
I'm not one of them.
Your core problem is a context that gets too large. So break it up. I know that inevitably there will be entities that should be shared among several contexts, which may give rise to duplicate class names. An easy way to solve this is to rename the classes into their context-specific names.
For example, I have an ApplicationUser table (who hasn't) that maps to a class with the same name in the main context, but to a class AuthorizationUser in my AuthorizationContext, or ReportingUser in a ReportingContext. This isn't a problem at all. Most use cases revolve around one context type anyway, so it's impossible to get confused.
I even have specialized contexts that work on the same data as other contexts, but in a more economical way. For example, a context that doesn't map to calculated columns in the database, so there are no reads after inserts and updates (apart from identity values).
So I'd recommend to go for it, because ...
Is there a way to combine code-first and database-first in the same context?
No, there isn't. Both approaches have different ways of building the DbModel (containing the store model, the class model, and the mappings between both). In a generated DbContext you even see that an UnintentionalCodeFirstException is thrown, to drive home that you're not supposed to use that method.
mostly related to insert/update/delete stored procedure mapping
As said in another answer, mapping CUD actions to stored procedures is supported in EF6 code-first.
I got here from a link in your comment on a different question, where you asked:
you mentioned that code-first & database-first is "technically possible" could you explain how to accomplish that?
First, the context of the other question was completely different. The OP there was asking if it was possible to use both database-first and code-first methodologies in the same project, but importantly, not necessarily the same context. My saying that it was "technically possible" applies to the former, not the latter. There is absolutely no way to utilize both code-first and database-first in the same context. Actually, to be a bit more specific, let's say there's no way to utilize an existing database and also migrate that same database with new entities.
The terminology gets a bit confused here due to some unfortunate naming by Microsoft when EF was being developed. Originally, you had just Model-first and Database-first. Both utilized EDMX. The only difference was that Model-first would let you design your entities and create a database from that, while Database-first took an existing database and created entities from that.
With EF 4.1, Code-first was introduced, which discarded EDMX entirely and let you work with POCOs (plain old class objects). However, despite the name, Code-first can and always has been able to work with an existing database or create a new one. Code-first, then is really Model-first and Database-first, combined, minus the horrid EDMX. Recently, the EF team has finally taken it a step further and deprecated EDMX entirely, including both the Model-first and Database-first methodologies. It is not recommended to continue to use either one at this point, and you can expect EDMX support to be dropped entirely in future versions of Visual Studio.
With all that said, let's go with the facts. You cannot both have an existing database and a EF-managed database in a single context. You would at least need two: one for your existing tables and one for those managed by EF. More to the point, these two contexts must reference different databases. If there are any existing tables in an EF-managed database, EF will attempt to remove them. Long and short, you have to segregate your EF-managed stuff from your externally managed stuff, which means you can't create foreign keys between entities in one context and another.
Your only real option here is to just do everything "database-first". In other words, you'll have to just treat your database as existing and manually create new tables, alter columns, etc. without relying on EF migrations at all. In this regard, you should also go ahead and dump the EDMX. Generate all your entities as POCOs and simply disable the database initializer in your context. In other words, Code-first with an existing database. I have additional information, if you need it.
Thank you to everyone for the well thought out and thorough answers.
Many of these other answers assume that the stored procedure mappings in EF Code-First work the same, but they do not. I'm a bit fuzzy on this as it's been about 6 months since I looked at it, but I believe as of EF 6.3 code first stored procedures require that you pass every column from your entity to your insert/update stored procedure and that you only pass the key column(s) to your delete procedure. There isn't an option to pick and choose which columns you can pass. We have a requirement to maintain who deleted a record so we have to pass some additional information besides just a simple key.
That being said, what I ended up doing was using a T4 template to automatically generate my EDMX/Context/Model files from the database (with some additional meta-data). This took our developer time experience down from 1.5 minutes to about 5 seconds.
My hope is EF stored procedure mappings will be improved to achieve parody with EDMX and I can then just code-generate the Code-First mappings and remove the EDMX generation completely.
In one of our environments, we have built a code generator where the power user can create models at runtime and deploy them. No need to mention that we are using EF 5.0 Code First.
Our application itself also has many models that it is using internally. So the dbContext may end up having at least 80 properties and more (80+ tables).
What we are observing is that the initial asp.net application load time is rather high and the memory consumption in the IIS process is high as well (600MB+).
What are the ways that we can configure EF to be more careful with performance and memory usage?
UPDATE:
Seems like the only way to do pre-compiled view generation for fixed entities. This way we can boost the performance since ratio of fixed models to dynamic ones is 60% to 40% in our projects
How do we generate per-compiled views for Code-First?
UPDATE:
Using EF Power Tools I was able to generate the views. Now i am wondering if I can have two separate Code-First Contexts in the same project. For one I intend to generate views since the models are fixed. The second one models can be changed by administrators, so for now I have no easy way of generating views for them.
Do you think this would be possible?
You have a very specific situation - given the dynamic nature of your code-gen vs code-first models -and as #Ladislav mentioned. I'm not sure there is an 'easy answer' if at all.
Normally you can make views at (pre)compile-time - e.g. EF Power Tools. That's not going to work for you.
'EdmGen.exe' is the utility that does most of that, so automating that somehow may be one option - but I haven't tried, and is probably a challenge given what you have.
T4 templates - is the other option.
This is the best source of info for what you require -
How to use a T4 template for View Generation
However I'm not sure if that's going to help you either. Problem is it needs the EDMX model generated in the first place - and that's usually done by hand.
Now, you could invoke a tool for that - or try to sort of 'reverse engineer' the EDMX model somehow - and see how it's constructed by EF in the first place. I'm not sure, it's a bit 'speculative'.
Here is the post I made about what I know on the EDMX reader/writer, and how you can use that. Not sure, but maybe expands your horizons a bit:)
EF 4.3 Code First Migrations - Uncompress __MigrationHistory Model
Also I can link in some more info about how to access metadata from code, via DbContext, but again I'm not sure if that is of any use to you, you probably need all that info 'before hand'.
And you'll probably need to engage into EF source code, to be able to follow and understand how it works, and how to do what you need, if you haven't already.
http://blogs.msdn.com/b/adonet/archive/2008/06/20/how-to-use-a-t4-template-for-view-generation.aspx
EF 4.3 Code First Migrations - Uncompress __MigrationHistory Model
http://edmxdom.codeplex.com/ (EF 6 but it overlaps, you can also find the EF5)
I've system that has a primary data model to perform most of the work.
The model has quite a few tables and with performance in mind when I came to add an administrative feature to the application I decided to use a second separate data model.
All works well until my second data model needs to access a table that is also in the primary data model. Now, from digging around I can see this can cause problems.
The two possible workaround I've come up with are to either:
Put the data models in separate projects.
Use views / stored procedures for accessing the table in question when required.
Method 1 seems the simpliest but I'm concerned about whether there would be any performance loss. Method 2 seems a bit messy and takes the point out of using EF.
Before I plump for using method 1, is there an easier work around that I could use?
In the end I decided to put the two data models into separate projects and I've there hasn't been any slowdown that I've been able to notice (I've not done any benchmarking but it's passed the perception test).
In one of her online tutorials EF guru Julie Lerman says that you should put your data model in a separate project anyway, so I don't think this has been a bad workaround.
I am working with 2 models in the same project, because I connect to 2 different databases. I have put different namespaces using "Custom Tool Namespace" on *.tt files but it is not necessary. It generally works, but it cannot handle situation when the entity (table) with the same name is in both models. When you save one model the entity with the same name is deleted from the second model.
I'm currently assign to a project where their legacy system is designed in a horrible way and it's been too much focus on database design. I trying to put together a new design where the customer can migrate the legacy system bit by bit.
They are currently using EF 4.1 BUT not code first approach with entity descriptive/mapping is located in an edmx file. They do Reverse engineering everytime to want to extend the model (First make changes in database, then reflect them upwards to Model layer through a custom tool).
What I would like to know, if anyone has used BOTH edmx and code first approach with mapping classes. And is there drawbacks to know about?
You can use EDMX and code mapping together only if you have separate context type for each approach (you cannot mix approaches in single context type). That is probably the biggest disadvantage because it leads to more complex code and maintenance.
For example if you need to have some entity in both contexts types to use it with both new and legacy code you must maintain its mapping twice. You must also be very careful about not duplicating entity class itself = your code first must use class generated by custom tool for EDMX but this will not be possible if they are not using POCOs in current solution.
Another problem will be database integrity. If you will need to save changes to both context types in single transaction you will have to use TransactionScope and distributed transaction = MSDTC (each context instance will handle its own database connection).
If you are sure that whole system will be migrated you can probably think about using code first instead of EDMX (but be aware that code first mapping and DbContext generally offers more limited feature set). If you are not sure that you will be able to complete whole migration don't even think about using code first because leaving system in the state where half uses code first and half EDMX will make everything only worse and much more horrible.
Being sure is little bit theoretical because in SW development the only think you can be sure about is that requirements / situation will change. It means that migration should be very carefully considered.
I also was struck with this problem. What I found was that you can model the database and "generate the database from the model" in a "Ado.NET Entity model Project".
But you can not create stored procedures in that project, What only you can do is you can import the stored procedures from the server.
But if you do not want to create stored procedures on the server, you can create another project on VS, "SQl CLR Database Project" and you can code your stored procedures and tigers in that project and deploy them to the server.
then you can again import these stored procedures from the "Ado.NET Entity model Project" by "Update Model From Database".
Like wise you can develop your server project using both approaches(Code first and Model first)
Hope this will add something more :)
The method - Entity Framework Code-First - looks good. But its very difficult to create all the classes for a large database.
Is there any easy way to generate the Entity Framework Code-First classes?
You can use the recently released Entity Framework Power Tools CTP1. The tool gives you the ability to reverse engineer code first, meaning the Database will be mapped to Code.
Note that all tables in your large database will be mapped. There currently is no way to choose which tables will be mapped to code. Reading through the comments, this feature will most likely be implemented in a future release.
The point of EF Code-First is that you define your domain model in code, then your user-interface and database can be easily generated from that domain model. This has a number of advantages including reducing the amount of tedious code which needs to be written, and helping to ensure your database, your domain model, and your UI match each other.
However, at some point you are going to have to write your domain model - there's no way that can be "generated" (by which I assume you mean computer-generated) as it is personal to your application.
If I've misunderstood your question, please leave a comment and I'll update my answer.
If you want to use the code-first model, but already have an existing database, you can use the Entity Framework Power Tools to generate classes.
If you're reading this after May/2012, the above tool may be out of beta!
No there is no way to generate classes for you if you are using code-first. Code first means that there is no model and no database so you can't generate classes unless you have some upfront design in any case system (UML) which will autogenerate code for you. Simply generating classes without any input about how they should look like sounds like AI from Sci-fi, doesn't it?
If you already have databse you are not using code first but database first. In such case you can have your classes generated.
Check out the link below. It's a program that will generate POCO classes from your databases. I think that's what you're looking for.
http://msormcodegen.codeplex.com/
Generate the code from the database first using database first generation and then modify the resulting code to start your code first version