I am new to Breeze.js, but really enjoy it so far. I ran into an issue with updating a database with Breeze.js, when selecting only portion of columns of a model.
When I ran this statement:
$scope.emFac.entityQuery.from('Company');
the company entity matches my EF entity, retrieves all columns, creates entityAspect, and all is working fine when updating database:
However, when I retrieve only portion of corresponding Model's columns, Breeze.js returns anonymous object with specified properties (retrieving data works, but not updating does not), without the entityAspect, which is being used for tracking changes.
Here is the code with select statement:
$scope.emFac.entityQuery.from('Company').select('companyId, displayName');
Is there a way to retrieve only some columns of EF Model columns, and still track changes with Breeze.js, needed for database updates?
As you've discovered, Breeze treats the incoming data as plain objects instead of entities when you use select.
Your choices are:
On the server, Create a CustomerLite or similar object, and have a server endpoint that returns those without the need for select; OR
On the client, get the results from the query and create entities from each object, with status Unchanged
Example of #2:
var entities = [];
em.executeQuery(customerProjectionQuery).then(queryResult => {
queryResult.results.forEach(obj => {
// obj contains values to initialize entity
var entity = em.createEntity(Customer.prototype.entityType, obj, EntityState.Unchanged);
entities.push(entity);
});
})
Either way, you will need to ensure that your saveChanges endpoint on the server can handle saving the truncated Customer objects without wiping out the other fields.
Related
I have a query that joins about 10 tables some that are self referencing tables. I use an "IN" statement for the conditional on the ID column (indexed) of the top most table.
var aryOrderId = DetermineOrdersToGet(); //Logic to determine what orderids to get
var result = dbContext.Orders.Where(o=>aryOrderId.Contains(o.id)
.Include(o=>o.Customer)
.Include(o=>o.Items.Select(oi=>oi.ItemAttributes))
.Include(o=>o.Items.Select(oi=>oi.Dimensions))
.Include(o=>o.CustomOptions.Select(oc => oc.CustomOptions1))
.....A Bunch more.....
.ToList();
I would like to figure out a way to speed this up without redesigning my tables and flattening out the structure. Currently 50-200 records take 10-20 seconds.
This data can be read only. I don't need to update these records.
Can I convert this to a stored procedure?
How hard is this to do?
Will I be able to get noticeable performance gains?
One of the slower parts of the database query is the transport of the selected data from the DBMS to your local process. Hence it is wise to select only the properties you actually plan to use.
For example, it seems that an Order has zero or more ItemAttributes. Evey ItemAttribute belongs to exactly one Order, using a foreign key OrderId.
If you fetch all Orders with Id in ArryOrderId, each order with its thousand ItemAttributes, you know that every ItemAttribute will have a foreign key OrderId with the same value as the Id of the Order that it belongs to. It is a waste to send 1000 times the same value.
When querying data using entity framework, always use Select. Select only the properties yo actually plan to use. Only use Include if you intend to change the fetched objects.
var result = dbContext.Orders
.Where(order=>aryOrderId.Contains(order.id)
.Select(order => new
{ // select only the properties you plan to use:
Id = order.Id,
...
Customer = order.Customer.Select(customer => new
{ // again: only the properties you plan to use
Id = order.Customer.Id,
Name = order.Customer.Name,
...
},
ItemAttributes = order.ItemAttributes.Select(itemAttribute => new
{
...
})
.ToList(),
Dimensions = order.Dimensions.Select(dimension => new
{
...
})
.ToList(),
....A Bunch more.....
})
.ToList();
If after selecting only the properties that you actually plan to use, the query still takes too long, think again: do I really need all these properties.
Another solution to limit the execution time is fetching the date 'per page', using Skip / Take. The danger is of course that when you are viewing page 10, the data of page 1 might be changed in a way that page 10 should be interpreted differently.
As jtate mentions, if you don't need everything from the joined tables, don't include them. Instead, utilize .Select() to retrieve just the data you want from the entity and it's associated relationships.
I.e.
var query = dbContext.Orders
.Where(x => aryOrderId.Contains(x => x.OrderId))
.Select(x => new
{
x.OrderId,
x.OrderNumber,
OrderItems = x.Items.Select(i => new
{
i.ItemId,
Attributes = i.Attributes.Select(a => a.AttributeName).ToList(),
Dimensions = i.Dimensions.Select(d => new {d.DimensionId, d.Name}).ToList(),
}).ToList(),
// ...
}).ToList();
You can structure the query, or queries however you like to find an optimal result.
Alternatively you can consider utilizing a view on the database and binding an entity to the view. This option works well for read-only views of data. Provided you include the relevant IDs you can always retrieve the applicable "real" entities at any time to load a details page or perform an action/update against the entity.
Answering your 3 questions. Yes, you can use a stored procedure and that's what I would do in this situation. It is not hard at all; EF makes it quite simple. You can either have it return a new complex type or you can map it to an entity. Since you're saying the data can be readonly, you probably are okay with a basic function import returning a complex type (EF's default behavior). Either way, you will have noticeable performance gains.
For db-first, see http://www.entityframeworktutorial.net/stored-procedure-in-entity-framework.aspx
Basically, you'll follow these steps.
Create the stored procedure on your database
Update the model from the database. When it asks which objects to include, you should be able to select your stored procedure.
Click Finish. EF will generate a complex type that has all the properties returned by your stored procedure, and it will add a signature to your context for executing the stored procedure, so it can be called like this: var results = myContext.myProcedure(param1, param2); There are screenshots of this at the link above.
You can also go in and modify the model to customize the details, such as the name of the complex type and the name of the function (by default the function will match the name of the SP and will return an ObjectResult<T> where T is your complex type, which will be the name of the procedure with "_Result" as a suffix).
I have a legacy database with a particular table -- I will call it ItemTable -- that can have billions of rows of data. To overcome database restrictions, we have decided to split the table into "silos" whenever the number of rows reaches 100,000,000. So, ItemTable will exist, then a procedure will run in the middle of the night to check the number of rows. If numberOfRows is > 100,000,000 then silo1_ItemTable will be created. Any Items added to the database from now on will be added to silo1_ItemTable (until it grows to big, then silo2_ItemTable will exist...)
ItemTable and silo1_ItemTable can be mapped to the same Item entity because the table structures are identical, but I am not sure how to set this mapping up at runtime, or how to specify the table name for my queries. All inserts should be added to the latest siloX_ItemTable, and all Reads should be from a specified siloX_ItemTable.
I have a separate siloTracker table that will give me the table name to insert/read the data from, but I am not sure how I can use this with entity framework...
Thoughts?
You could try to use the Entity Inheritance to get this. So you have a base class which has all the fields mapped to ItemTable and then you have descendant classes that inherit from ItemTable entity and is mapped to the silo tables in the db. Every time you create a new silo you create a new entity mapped to that silo table.
[Table("ItemTable")]
public class Item
{
//All the fields in the table goes here
}
[Table("silo1_ItemTable")]
public class Silo1Item : Item
{
}
[Table("silo2_ItemTable")]
public class Silo2Item : Item
{
}
You can find more information on this here
Other option is to create a view that creates a union of all those table and map your entity to that view.
As mentioned in my comment, to solve this problem I am using the SQLQuery method that is exposed by DBSet. Since all my item tables have the exact same schema, I can use the SQLQuery to define my own query and I can pass in the name of the table to the query. Tested on my system and it is working well.
See this link for an explanation of running raw queries with entity framework:
EF raw query documentation
If anyone has a better way to solve my question, please leave a comment.
[UPDATE]
I agree that stored procedures are also a great option, but for some reason my management is very resistant to make any changes to our database. It is easier for me (and our customers) to put the sql in code and acknowledge the fact that there is raw sql. At least I can hide it from the other layers rather easily.
[/UPDATE]
Possible solution for this problem may be using context initialization with DbCompiledModel param:
var builder = new DbModelBuilder(DbModelBuilderVersion.V6_0);
builder.Configurations.Add(new EntityTypeConfiguration<EntityName>());
builder.Entity<EntityName>().ToTable("TableNameDefinedInRuntime");
var dynamicContext = new MyDbContext(builder.Build(context.Database.Connection).Compile());
For some reason in EF6 it fails on second table request, but mapping inside context looks correct on the moment of execution.
None of the many questions on this topic seem to match my situation. I have a large data model. In certain cases, only a few of the fields need be displayed on the UI, so for those I replaced the LINQ to Entity query that pulls in everything with an Entity SQL query retrieving only the columns needed, using a Type constructor so that I got an entity returned and not a DbDataRecord, like this:
SELECT VALUE MyModelNameSpace.INCIDENT(incident.FieldA, incident.FieldB, ...) FROM ... AS ...
This works and displays the fields in the UI. And if I make a change, the change makes it back to the entity model when I tab out of the UI element. But when I do a SaveChanges, the changes do not get persisted to the database. No errors show up in the Log. Now if I very carefully replace the above query with an Entity Sql query that retrieves the entire entity, like this:
SELECT VALUE incident FROM MyDB.INCIDENTs AS incident...
Changes do get persisted in the database! So as a test, I created another query like the first that named every column in the entity, which should be the exact equivalent of the second Entity SQL query. Yet it did not persist changes to the database either.
I've tried setting the MergeOption on the returned result to PreserveChanges, to start tracking, like this:
incidents.MergeOption = MergeOption.PreserveChanges;
But that has no effect. But really, if retrieving the entire entity with Entity Sql persists changes, what logical purpose would there be for behaving differently when a subset of the fields are retrieved? I'm wondering if this is a bug?
Gert was correct, the problem was that the entity was not attached. Dank U wel, Gert! Ik was ervan verbluft!
I just wanted to add a little detail to show the full solution. Basically, the ObjectContext has an Attach method, so you'd think that would be it. However, when your Entity SQL select statement names columns, and you create the object using a Type as I did, the EntityKey is not created, and ObjectContext.Attach fails. After trying and failing to insert the EntityKey I created myself, I stumbled across ObjectSet.Attach, added in Entity Framework 4. Instead of failing, it creates the EntityKey if it is missing. Nice touch.
The code was (this can probably be done in fewer steps, but I know this works):
var QueryString = "SELECT VALUE RunTimeUIDesigner.INCIDENT (incident.INCIDENT_NBR,incident.LOCATION,etc"
ObjectQuery<INCIDENT> incidents = orbcadDB.CreateQuery<INCIDENT>(QueryString);
incidents.MergeOption = MergeOption.PreserveChanges;
List<INCIDENT> retrievedIncidents = incidents.ToList<INCIDENT>();
orbcadDB.INCIDENTs.Attach(retrievedIncidents[0]);
iNCIDENTsViewSource.Source = retrievedIncidents;
We have a customer very large table with over 500 columns (i know someone does that!)
Many of these columns are in fact foreign keys to other tables.
We also have the requirement to eager load some of the related tables.
Is there any way in Linq to SQL or Dynamic Linq to specify what columns to be retrieved from the database?
I am looking for a linq statement that actually HAS this effect on the generated SQL Statement:
SELECT Id, Name FROM Book
When we run the reguar query generated by EF, SQL Server throws an error that you have reached the maximum number of columns that can be selected within a query!!!
Any help is much appreciated!
Yes exactly this is the case, the table has 500 columns and is self referencing our tool automatically eager loads the first level relations and this hits the SQL limit on number of columns that can be queried.
I was hoping that I can set to only load limited columns of the related Entities such as Id and Name (which is used in the UI to view the record to user)
I guess the other option is to control what FK columns should be eager loaded. However this still remains problem for tables that has a binary or ntext column which you may not want to load all the times.
Is there a way to hook multiple models (Entities) to the same table in Code First? We tried doing this I think the effort failed miserably.
Yes you can return only subset of columns by using projection:
var result = from x in context.LargeTable
select new { x.Id, x.Name };
The problem: projection and eager loading doesn't work together. Once you start using projections or custom joins you are changing shape of the query and you cannot use Include (EF will ignore it). The only way in such scenario is to manually include relations in the projected result set:
var result = from x in context.LargeTable
select new {
Id = x.Id,
Name = x.Name,
// You can filter or project relations as well
RelatedEnitites = x.SomeRelation.Where(...)
};
You can also project to specific type BUT that specific type must not be mapped (so you cannot for example project to LargeTable entity from my sample). Projection to the mapped entity can be done only on materialized data in Linq-to-objects.
Edit:
There is probably some misunderstanding how EF works. EF works on top of entities - entity is what you have mapped. If you map 500 columns to the entity, EF simply use that entity as you defined it. It means that querying loads entity and persisting saves entity.
Why it works this way? Entity is considered as atomic data structure and its data can be loaded and tracked only once - that is a key feature for ability to correctly persist changes back to the database. It doesn't mean that you should not load only subset of columns if you need it but you should understand that loading subset of columns doesn't define your original entity - it is considered as arbitrary view on data in your entity. This view is not tracked and cannot be persisted back to database without some additional effort (simply because EF doesn't hold any information about the origin of the projection).
EF also place some additional constraints on the ability to map the entity
Each table can be normally mapped only once. Why? Again because mapping table multiple times to different entities can break ability to correctly persist those entities - for example if any non-key column is mapped twice and you load instance of both entities mapped to the same record, which of mapped values will you use during saving changes?
There are two exceptions which allow you mapping table multiple times
Table per hierarchy inheritance - this is a mapping where table can contains records from multiple entity types defined in inheritance hierarchy. Columns mapped to the base entity in the hierarchy must be shared by all entities. Every derived entity type can have its own columns mapped to its specific properties (other entity types have these columns always empty). It is not possible to share column for derived properties among multiple entities. There must also be one additional column called discriminator telling EF which entity type is stored in the record - this columns cannot be mapped as property because it is already mapped as type discriminator.
Table splitting - this is direct solution for the single table mapping limitation. It allows you to split table into multiple entities with some constraints:
There must be one-to-one relation between entities. You have one central entity used to load the core data and all other entities are accessible through navigation properties from this entity. Eager loading, lazy loading and explicit loading works normally.
The relation is real 1-1 so both parts or relation must always exists.
Entities must not share any property except the key - this constraint will solve the initial problem because each modifiable property is mapped only once
Every entity from the split table must have a mapped key property
Insertion requires whole object graph to be populated because other entities can contain mapped required columns
Linq-to-Sql also contains ability to mark a column as lazy loaded but this feature is currently not available in EF - you can vote for that feature.
It leads to your options for optimization
Use projections to get read-only "view" for entity
You can do that in Linq query as I showed in the previous part of this answer
You can create database view and map it as a new "entity"
In EDMX you can also use Defining query or Query view to encapsulate either SQL or ESQL projection in your mapping
Use table splitting
EDMX allows you splitting table to many entities without any problem
Code first allows you splitting table as well but there are some problems when you split table to more than two entities (I think it requires each entity type to have navigation property to all other entity types from split table - that makes it really hard to use).
Create stored procedures that query the number of columns needed and then call the stored procs from code.
I've created a function import that returns the results of a stored proceedure as one of my entities. however I can't seem to traverse my through navigation properties to access the data in other entities. I know that you can use include() for objectQueries but can't find anything that will force the EF to load my relations for entity results of function imports.
Any ideas??
Thanks in advance.
This is not possible in EF 1.0
The reason is that EF will consider stored procedure values to be just values and not navigation properites.
For example, Employee entity has multiple Order entities. In Order you have a property called EmployeeID. When the database fills your query using include statements, it creates 1 projection query in SQL to populate all of the Order data that a particular Employee could have.
So if I said
var employee = context.Employees.Include("Orders").Where(e => e.ID == 1).First();
var orders = employee.Orders;
The SQL for the first query will create a projection query which will contain orders where the EmployeeID = 1.
Now when your stored procedure runs, this can do any code behind the scenes (in otherwords it can return any set of data). So when SQL runs the stored procedure, it just runs the code in that stored procedure and does not have any knowledge that EmployeeID on Order is an FK to that property. Additionally, if your stored procedure returns an Employee entity, then you are looking at another scenario where you will not even have an OrderID to pursue.
To work around this though, you can setup your query in EF using Include statements that can mirror any stored procedure. If you use the proper mix of .Select and .Include statements you should be able to do the same thing.