We are using DTOs server side, and have configured a dbcontext using fluent api in order to give breeze the metadata it needs. We map 1:1 to our real database entities, and each DTO contains a simple subset of the real database entity properties.
This works great, but now I need to figure out a way to make queries efficient - i.e. if the Breeze client queries for a single item I don't want to have to create a whole set of DTO objects before I can filter. i.e. I want to figure out a way to execute the filter/sort on the actual entities, but still return DTO objects.
I guess I need to figure out a way to intercept the query execution in order to query my real database entities and return a DTO instead of the real database entity.
Any ideas for how to best approach this?
Turns out that if you use projection in a link statement, e.g.
From PossibleCustomer As Customer
In Customers
Select New CustomerDto With {.Id = PossibleCustomer.Id,
.Name = PossibleCustomer.Name,
.Email = PossibleCustomer.Email}
.. then linq is smart enough to still optimize any queries to the database - i.e. if I query on the linq statement above to filter for a single item by Id, the database is hit with that query for just a single item and a single DTO is created. Pretty clever stuff. This only works if you do a direct projection in the linq statement - if you call off to a function to create your DTO then this won't work.
Just in case others are facing the same scenario, you might want to look at AutoMapper - it can create these projections for you using a model you create just once - avoids all those huge linq statements that are hard to read and validate. The automapper projections (assuming you stick to the simple stuff) still allow the linq to entities magic that ensures you don't have to do table scans when you create your DTOs.
Related
I have this situation:
Spring Data JPA: Work with Pageable but with a specific set of fields of the entity
It about to work with Spring Data and working with a specific set of fields of an #Entity
The two suggestions are totally valid for me:
DTO projections
Projection interfaces
Even more, in spring-data-examples appears both together (I know for sample purposes):
CustomerRepository.java
Thus:
When is mandatory use one over the other and why?
Exists a cost of performance one over the other?
Note in the Class-based Projections (DTOs) section says the following:
Another way of defining projections is by using value type DTOs (Data
Transfer Objects) that hold properties for the fields that are
supposed to be retrieved. These DTO types can be used in exactly the
same way projection interfaces are used, except that no proxying
happens and no nested projections can be applied.
Seems the advantages are: except that no proxying happens and no nested projections can be applied
DTO Approach
Pro
Simple and straigt forward
Con
It will result in more code as you have to create DTO class with constructor and getters/setters (unless you utilize Project Lombok to avoid boilerplate
code for DTOs).
No nested projections can be applied.
Projections
Pro
Less code as it uses only interfaces.
Nested projections can be applied
Dynamic projection allows you write one generic repository method to return
different subset of the attributes in entity object based on client's needs.
Con
Spring generates proxy at runtime
Query could return the entire entity object from database to Spring layer though a trimmed version (via Projection) is returned from Spring layer to client. I wasn't sure about this specific disadvantage, hoping someone to edit this answer if necessary.
If you need nested or dynamic projection, you probably want Projection approach rather than DTO approach.
Refer to official Spring doc for details.
I think that DTO was the first possible solution to work with a small set of data from the Entities. Today, many operations can also be made with projections, but you need to be careful with performance. If you see this Janssen's post Entities or DTOs – When should you use which projection? you will note that DTOs have better performance than projections for reading operations.
If you don't have the problem with performance, projections will be more graceful.
I am developing a SPA using Angular-Breeze-WebAPI-EntityFramework.
Now Breeze uses the Entity Framework metadata information to create it's own Breeze models. We use this in our application for Breeze validation.
So far, it's all been nice and easy. Now we are having to create a search page (say for querying customers). The search can be by Customer.Name or by Product.Id (which would return a list of customers who have bought that product). The result is a ng-repeater, which displays Customer.Name, Order.LastPlaced etc.
if you are getting confused by the tables and columns, forget that. What I am only trying to get to is that, both the search object and the result object are not 1:1 with Entity tables (or objects). So obviously I feel the need to create a custom object (one for the search and one for the results). My question primarily is where and how do I create that object?
If I create it at the data layer, Breeze would have no idea of the metadata for each of the properties (since it uses EF for that).
I obviously can't create just a JavaScript object, since I will have to query the database (using EF) to search and populate the object.
So where does one create such a custom object (traversing multiple tables) such that Breeze still can figure out the metadata and perform validation and such when the need arises?
Thank you all.
You can create metadata on the client for types that the server either doesn't know about or doesn't have the schema for. See http://www.breezejs.com/documentation/metadata-by-hand.
I will start a new project in couple of days which is based on ASP.NET MVC3 and I don't have enough experience in Web Development.
I just want to know about Entity Framework. What is Entity Framework? Why we use it?
and also want to know about Object Relational Mapping. How is it connect with entity framework?
I've googled but did not get exact idea about it.
I'm very eager to know what's basic concept behind all those things?
Entity Framework is an object-relational mapper. This means that it can return the data in your database as an object (e.g.: a Person object with the properties Id, Name, etc.) or a collection of objects.
Why is this useful? Well, it's really easy. Most of the time you don't have to write any SQL yourself, and iterating is very easy using your language built in functions. When you make any changes to the object, the ORM will usually detect this, and mark the object as 'modified'. When you save all the changes in your ORM to the database, the ORM will automatically generate insert/update/delete statements, based on what you did with the objects.
In code, you might want to work with objects in an object oriented fashion.
MyClass obj = new MyClass(); // etc.
However, it might be cumbersome to save data to databases from objects, since you might end up with mapping your object to an SQL query string
// Perhaps with parameter bindings instead, but the idea is the same
"INSERT INTO MYTBL name,phone VALUES(" + obj.Name + "," + obj.Phone + ")";
An ORM framework does this object to SQL mapping by generating SQL statements and the Entity manager will execute them when you need to save or load objects from the database. It comes at the cost of another abstraction layer, but it will make the code easier to write.
So it turns out that I am the last person to discover the fundamental floor that exists in Microsoft's Entity Framework when implementing TPT (Table Per Type) inheritance.
Having built a prototype with 3 sub classes, the base table/class consisting of 20+ columns and the child tables consisting of ~10 columns, everything worked beautifully and I continued to work on the rest of the application having proved the concept. Now the time has come to add the other 20 sub types and OMG, I've just started looking the SQL being generated on a simple select, even though I'm only interested in accessing the fields on the base class.
This page has a wonderful description of the problem.
Has anyone gone into production using TPT and EF, are there any workarounds that will mean that I won't have to:
a) Convert the schema to TPH (which goes against everything I try to achieve with my DB design - urrrgghh!)?
b) rewrite with another ORM?
The way I see it, I should be able to add a reference to a Stored Procedure from within EF (probably using EFExtensions) that has the the TSQL that selects only the fields I need, even using the code generated by EF for the monster UNION/JOIN inside the SP would prevent the SQL being generated every time a call is made - not something I would intend to do, but you get the idea.
The killer I've found, is that when I'm selecting a list of entities linked to the base table (but the entity I'm selecting is not a subclass table), and I want to filter by the pk of the Base table, and I do .Include("BaseClassTableName") to allow me to filter using x=>x.BaseClass.PK == 1 and access other properties, it performs the mother SQL generation here too.
I can't use EF4 as I'm limited to the .net 2.0 runtimes with 3.5 SP1 installed.
Has anyone got any experience of getting out of this mess?
This seems a bit confused. You're talking about TPH, but when you say:
The way I see it, I should be able to add a reference to a Stored Procedure from within EF (probably using EFExtensions) that has the the TSQL that selects only the fields I need, even using the code generated by EF for the monster UNION/JOIN inside the SP would prevent the SQL being generated every time a call is made - not something I would intend to do, but you get the idea.
Well, that's Table per Concrete Class mapping (using a proc rather than a table, but still, the mapping is TPC...). The EF supports TPC, but the designer doesn't. You can do it in code-first if you get the CTP.
Your preferred solution of using a proc will cause performance problems if you restrict queries, like this:
var q = from c in Context.SomeChild
where c.SomeAssociation.Foo == foo
select c;
The DB optimizer can't see through the proc implementation, so you get a full scan of the results.
So before you tell yourself that this will fix your results, double-check that assumption.
Note that you can always specify custom SQL for any mapping strategy with ObjectContext.ExecuteStoreQuery.
However, before you do any of this, consider that, as RPM1984 points out, your design seems to overuse inheritance. I like this quote from NHibernate in Action
[A]sk yourself whether it might be better to remodel inheritance as delegation in the object model. Complex inheritance is often best avoided for all sorts of reasons unrelated to persistence or ORM. [Your ORM] acts as a buffer between the object and relational models, but that doesn't mean you can completely ignore persistence concerns when designing your object model.
We've hit this same problem and are considering porting our DAL from EF4 to LLBLGen because of this.
In the meantime, we've used compiled queries to alleviate some of the pain:
Compiled Queries (LINQ to Entities)
This strategy doesn't prevent the mammoth queries, but the time it takes to generate the query (which can be huge) is only done once.
You'll can use compiled queries with Includes() as such:
static readonly Func<AdventureWorksEntities, int, Subcomponent> subcomponentWithDetailsCompiledQuery = CompiledQuery.Compile<AdventureWorksEntities, int, Subcomponent>(
(ctx, id) => ctx.Subcomponents
.Include("SubcomponentType")
.Include("A.B.C.D")
.FirstOrDefault(s => s.Id == id));
public Subcomponent GetSubcomponentWithDetails(int id)
{
return subcomponentWithDetailsCompiledQuery.Invoke(ObjectContext, id);
}
I'm using DataNucleus as a JPA implementation to store my classes in my web application. I use a set of converters which all have toDTO() and fromDTO().
My issue is, that I want to avoid the whole DB being sent over the wire:
If I lazy load, the converter will try to access ALL the fields, and load then (resulting in very eager loading).
If I don't lazy load, I'll get a huge part of the DB, since user contains groups, and groups contains users, and so on.
Is there a way to explicitly load some fields and leave the others as NULL in my loaded class?
I've tried the DataNucleus docs with no luck.
Your DTOs are probably too fine-grained. i.e. dont plan to have a DTO per JPA entity. If you have to use DTOs then make them more coarse grained and construct them manually.
Recently we have had the whole "to DTO or not to DTO, that is the question" discussion AGAIN. The requirement for them (especially in the context of a JPA app) is often no longer there, but one of the arguments FOR DTOs tends to be that the view has coarser data requirements.
To only load the data you really require, you would need to use a custom select clause containing only these elements that you are about to use for your DTOs. I know how painful this is, especially when it involves joins, which is why I created Blaze-Persistence Entity Views which will take care of making the query efficient.
You define your DTO as an interface with mappings to the entity, using the attribute name as default mapping, this looks very simple and a lot like a subset of an entity, though it doesn't have to. You can use any JPQL expression as mapping for your DTO attributes.