How should I transform this query into a valid Entity Framework query?
dbSet.Include(x=>x.Offer.Images.Where(y=>y.IsDeleted == false))
The Where clause throws an exception:
The Include path expression must refer to a navigation property defined on the type. Use dotted paths for reference navigation properties and the Select operator for collection navigation properties.
I already tried third party library IncludeFilter, which works. However it doesn't allow me to use another include:
dbSet.Include(x=>x.Offer.Images.Where(y=>y.IsDeleted == false)).Include(x=>x.Demand)
Many thanks :-)
Include is just a marker to tell EF to eager-load related entities. It doesn't support eager-loading a selected sub-set of relatives, such as active items.
When working with a soft-delete model (IsDeleted/IsActive) will populate the view models with the active children. The benefit of returning view models for read operations is that you avoid potential lazy-load scenarios and just retrieve the data you need to consume. For edit operations I will eager-load related data and typically want both active and inactive records.
var viewModels = dbset.Select(x => new MyVM
{
Objectid = x.ObjectId,
// ...
Offer = new OfferViewModel
{
OfferId = x.Offer.OfferId,
// ...
Images = x.Offer.Images
.Where(i => !i.IsDeleted) // Load only active images
.Select(i => new ImageViewModel
{
ImageId = i.ImageId,
//...
}).ToList()
}
}
Note that this doesn't need explicit .Include() statements, just load the data that you need.
When I go to save changes to the entity graph then I would load the root entity with .Include(x => x.Order).Include(x => x.Order.Images) to ensure that I have all relative data included. EF should be aware of the full collection before applying updates to avoid surprises such as duplicate record insertion attempts.
I already tried third party library IncludeFilter, which works. However it doesn't allow me to use another include:
When starting to use IncludeFilter, you need to continue using it all the way even if you don't filter.
dbSet.IncludeFilter(x=>x.Offer.Images.Where(y=>y.IsDeleted == false))
.IncludeFilter(x=>x.Demand)
Related
I have two entities, user and messages. I like only to load messages and not the user a message belongs.
This usually works as I do not include the related entity in the linq query. But it doesn't work if I load the user separately with another query to get only a string from that user (the name).
I am using EF 6.1.0, POCO without proxy generation and WCF services to pass the data. The unwanted
navigation property value is then transferred accidentally.
var message = context.messages.FirstOrDefault(m => m.id == 123);
var user = contect.users.FirstOrDefault(u => u.id = 456);
var message_username_composition = new M_U_C{ m = message, username = user.name };
return message_username_composition;
In this situation I have the navigation property (user) of the message loaded even I do not like so.
Disabling LazyLoading doesn't help, only using a separate context solves the problem.
Is there a way other than using two contexts?
Is this by design?
I also do not like to manually set the navigation properties to null to fix this, because it may also
get forgotten, because one did not realize this behavior.
You'll have to disable Lazy-Load and use AsNoTracking() as follows:
ctx.Configuration.LazyLoadingEnabled = false;
var message = context.messages.AsNoTracking().FirstOrDefault(m => m.id == 123);
My question is this: How can you implement a default server-side "filter" for a navigation property?
In our application we seldom actually delete anything from the database. Instead, we implement "soft deletes" where each table has a Deleted bit column. If this column is true the record has been "deleted". If it is false, it has not.
This allows us to easily "undelete" records accidentally deleted by the client.
Our current ASP.NET Web API returns only "undeleted" records by default, unless a deleted argument is sent as true from the client. The idea is that the consumer of the service doesn't have to worry about specifying that they only want undeleted items.
Implementing this same functionality in Breeze is quite simple, at least for base entities. For example, here would be the implementation of the classic Todo's example, adding a "Deleted" bit field:
// Note: Will show only undeleted items by default unless you explicitly pass deleted = true.
[HttpGet]
public IQueryable<BreezeSampleTodoItem> Todos(bool deleted = false) {
return _contextProvider.Context.Todos.Where(td => td.Deleted == deleted);
}
On the client, all we need to do is...
var query = breeze.EntityQuery.from("Todos");
...to get all undeleted Todos, or...
var query = breeze.EntityQuery.from("Todos").withParameters({deleted: true})
...to get all deleted Todos.
But let's say that a BreezeSampleTodoItem has a child collection for the tools that are needed to complete that Todo. We'll call this "Tools". Tools also implements soft deletes. When we perform a query that uses expand to get a Todo with its Tools, it will return all Tools - "deleted" or not.
But how can I filter out these records by default when Todo.Tools is expanded?
It has occurred to me to have separate Web API methods for each item that may need expanded, for example:
[HttpGet]
public IQueryable<Todo> TodoAndTools(bool deletedTodos = false, bool deletedTools = false)
{
return // ...Code to get filtered Todos with filtered Tools
}
I found some example code of how to do this in another SO post, but it requires hand-coding each property of Todo. The code from the above-mentioned post also returns a List, not an IQueryable. Furthermore this requires methods to be added for every possible expansion which isn't cool.
Essentially what I'm looking for is some way to define a piece of code that gets called whenever Todos is queried, and another for whenever Tools is queried - preferably being able to pass an argument that defines if it should return Deleted items. This could be anywhere on the server-side stack - be it in the Web API method, itself, or maybe part of Entity Framework (note that filtering Include extensions is not supported in EF.)
Breeze cannot do exactly what you are asking for right now, although we have discussed the idea of allowing the filtering of "expands", but we really need more feedback as to whether the community would find this useful. Please add this to the breeze User Voice and vote for it. We take these suggestions very seriously.
Moreover, as you point out, EF does not support this.
But... what you can do is use a projection instead of an expand to do something very similar:
public IQueryable<Object> TodoAndTools(bool deleted = false
,bool deletedTools = false) {
var baseQuery = _contextProvider.Context.Todos.Where(td => td.Deleted == deleted);
return baseQuery.Select(t => new {
Todo: t,
Tools: t.Tools.Where( tool => tool.Deleted = deletedTools);
});
}
Several things to note here:
1) We are returning an IQueryable of Object instead of IQueryable of ToDo
2) Breeze will inspect the returned payload and automatically create breeze entities for any 'entityTypes' returned (even within a projection). So the result of this query will be an array of javascript objects each with two properties; 'ToDo' and 'Tools' where Tools is an array of 'Tool' entities. The nice thing is that both ToDo and Tool entities returned within the projection will be 'full' breeze entities.
3) You can still pass client side filters based on the projected property names. i.e.
var query = EntityQuery.from("TodoAndTools")
.where("Todo.Description", "startsWith", "A")
.using(em);
4) EF does support this.
I have read so much (dozens of posts) about one thing:
How to unit test business logic code that has Entity Framework code in it.
I have a WCF service with 3 layers :
Service Layer
Business Logic Layer
Data Access Layer
My business logic uses the DbContext for all the database operations.
All my entities are now POCOs (used to be ObjectContext, but I changed that).
I have read Ladislav Mrnka's answer here and here on the reasons why we should not mock \ fake the DbContext.
He said:
"That is the reason why I believe that code dealing with context / Linq-to-entities should be covered with integration tests and work against the real database."
and:
"Sure, your approach works in some cases but unit testing strategy must work in all cases - to make it work you must move EF and IQueryable completely from your tested method."
My question is - how do you achieve this ???
public class TaskManager
{
public void UpdateTaskStatus(
Guid loggedInUserId,
Guid clientId,
Guid taskId,
Guid chosenOptionId,
Boolean isTaskCompleted,
String notes,
Byte[] rowVersion
)
{
using (TransactionScope ts = new TransactionScope())
{
using (CloseDBEntities entities = new CloseDBEntities())
{
User currentUser = entities.Users.SingleOrDefault(us => us.Id == loggedInUserId);
if (currentUser == null)
throw new Exception("Logged user does not exist in the system.");
// Locate the task that is attached to this client
ClientTaskStatus taskStatus = entities.ClientTaskStatuses.SingleOrDefault(p => p.TaskId == taskId && p.Visit.ClientId == clientId);
if (taskStatus == null)
throw new Exception("Could not find this task for the client in the database.");
if (taskStatus.Visit.CustomerRepId.HasValue == false)
throw new Exception("No customer rep is assigned to the client yet.");
TaskOption option = entities.TaskOptions.SingleOrDefault(op => op.Id == optionId);
if (option == null)
throw new Exception("The chosen option was not found in the database.");
if (taskStatus.RowVersion != rowVersion)
throw new Exception("The task was updated by someone else. Please refresh the information and try again.");
taskStatus.ChosenOptionId = optionId;
taskStatus.IsCompleted = isTaskCompleted;
taskStatus.Notes = notes;
// Save changes to database
entities.SaveChanges();
}
// Complete the transaction scope
ts.Complete();
}
}
}
In the code attached there is a demonstration of a function from my business logic.
The function has several 'trips' to the database.
I don't understand how exactly I can strip the EF code from this function out to a separate assembly, so that I am able to unit test this function (by injecting some fake data instead of the EF data), and integrate test the assembly that contains the 'EF functions'.
Can Ladislav or anyone else help out?
[Edit]
Here is another example of code from my business logic, I don't understand how I can 'move the EF and IQueryable code' out from my tested method :
public List<UserDto> GetUsersByFilters(
String ssn,
List<Guid> orderIds,
List<MaritalStatusEnum> maritalStatuses,
String name,
int age
)
{
using (MyProjEntities entities = new MyProjEntities())
{
IQueryable<User> users = entities.Users;
// Filter By SSN (check if the user's ssn matches)
if (String.IsNullOrEmusy(ssn) == false)
users = users.Where(us => us.SSN == ssn);
// Filter By Orders (check fi the user has all the orders in the list)
if (orderIds != null)
users = users.Where(us => UserContainsAllOrders(us, orderIds));
// Filter By Marital Status (check if the user has a marital status that is in the filter list)
if (maritalStatuses != null)
users = users.Where(pt => maritalStatuses.Contains((MaritalStatusEnum)us.MaritalStatus));
// Filter By Name (check if the user's name matches)
if (String.IsNullOrEmusy(name) == false)
users = users.Where(us => us.name == name);
// Filter By Age (check if the user's age matches)
if (age > 0)
users = users.Where(us => us.Age == age);
return users.ToList();
}
}
private Boolean UserContainsAllOrders(User user, List<Guid> orderIds)
{
return orderIds.All(orderId => user.Orders.Any(order => order.Id == orderId));
}
If you want to unit test your TaskManager class, you should employ the Repository dessign pattern and inject repositories such as UserRepository or ClientTaskStatusRepository into this class. Then instead of constructing CloseDBEntities object you will use these repositories and call their methods, for example:
User currentUser = userRepository.GetUser(loggedInUserId);
ClientTaskStatus taskStatus =
clientTaskStatusRepository.GetTaskStatus(taskId, clientId);
If yout wanto to integration test your TaskManager class, the solution is much more simple. You just need to initialize CloseDBEntities object with a connection string pointing to the test database and that's it. One way how to achieve this is injecting the CloseDBEntities object into the TaskManager class.
You will also need to re-create the test database before each integration test run and populate it with some test data. This can be achieved using Database Initializer.
There are several misunderstandings here.
First: The Repository Pattern. It's not just a facade over DbSet for unit testing! The repository is a pattenr strongly related to Aggregate and Aggreate Root concepts of Domain Driven Design. An aggregate is a set of related entities that should stay consistent to each other. I mean a business consistency, not just only a foreign keys validity. For example: a customer who have made 2 orders should get a 5% discount. So we should somehow manage the consistency between the number of order entities related to a customer entity and a discount property of the customer entity. A node responsible for this is an aggregate root. It is also the only node that should be accessible directly from outside of the aggregate. And the repository is an utility to obtain an aggregate root from some (maybe persistent) storage.
A typical use case is to create a UoW/Transaction/DbContext/WhateverYouNameIt, obtain one aggregate root entity from the repository, call some methods on it or access some other entities by traversing from the root, Commit/SaveChanges/Whatever. Look, how far it differs from yur samples.
Second: The Business Logic. I've already showed you one example: a customer who have made 2 orders should get a 5% discount. In contrary: your second code sample is not a business logic. It's just a query. The responsibility of this code is to obtain some data from the storage. In such a case, the storage technology behind it does matter. So I would recomend integration tests here rather than pretending the storage doesn't matter when interacting with the storage is the sole purpose of this function.
I would also encapsulate that in a Query Object that was already suggested. Then - such a query object could be mocked. Not just DbContext behind it. The whole QO.
The first code sample is a bit better because it probably ivolves some business logic, but that's dificult to identify. Wich leads us to the third problem.
Third: Anemic Domain Model. Your domain doesnt' look very object oriented. You have some dumb entities and transaction scripts over them. With 7 parameters! Thats pure procedural programming.
Moreover, in your UpdateTaskStatus use case - what is the aggregate root? Befere you answer that, the most important question first: what exactly do you want to do? Is that... hmm... marking a current task of a user done when he was visited? Than, maybe there should be a method Visit() inside a Customer Entity? And this method should have something like this.CurrentTaskStatus.IsCompleted = true?
That was just a random guess. If I missed, that would clearly show another issue. The domain model should use the ubiquitous language - something common for the programmer and a business. Your code doesn't have that expressive power that a common language gives. I just don't know what is going on there in UpdateTaskStatus with 7 parameters.
If you place proper expressive methods for performing business operations in your entities that will also enforce you to not use DbContext there at all, as you need your entities to stay persistence ignorant. Then the problem with mocking disappears. You can test the pure business logic without persistence concerns.
So the final word: Reconsider your model first. Make your API expressive by using ubiquitous language first.
PS: Please don't treat me as an authority. I may be completely wrong as I'm just starting to learn DDD.
This is my first post here, so I hope everything is fine.
Here is my problem:
I have a table in my database called UserTypes. It has:
ID;
IsPrivate;
Parent_ID;
The relevant ones are the first and the third one.
I have another table called UserTypes_T which has information for the different types, that is language specific. The fields are:
Language_ID;
UserType_ID;
Name;
What I'm trying to achieve is load the entire hierarchy from the UserTypes table and show it in a TreeView (this is not relevant for now). Then, by selecting some of the user types I can edit them in separate edit box (the name) and a combo box (the parent).
Everything works fine until I try to persist the changes in the database. EF has generated for me two entity classes for those tables:
The class for the user types has:
ID;
IsPrivate;
Parent_ID;
A navigational property for the self-reference (0..1);
A navigational property for the child elements;
Another navigational property for the UserTypes_T table (1..*);
The class for the translated information has:
UserType_ID;
Language_ID;
Name;
A navigational property to the UserTypes table (*..1);
A navigational property to the Languages table (*..1);
I get the data I need using:
return context.UserTypes.Include("UserTypes_T").Where(ut => ut.IsPrivate==false).ToList();
in my WCF Web service. I can add new user types with no problems, but when I try to update the old ones, some strange things happen.
If I update a root element (Parent_ID==null) everything works!
If I update an element where Parent_ID!=null I get the following error:
AcceptChanges cannot continue because the object’s key values conflict with another object in the ObjectStateManager.
I searched all over the internet and read the blog post from Diego B Vega (and many more) but my problem is different. When I change a parent user type, I actually change the Parent_ID property, not the navigational property. I always try to work with the IDs, not the generated navigational properties in order to avoid problems.
I did a little research, tried to see what is the object graph that I get and saw that there were lots of duplicate entities:
The root element had a list of its child elements. Each child element had a back reference to the root or to its parent and so on. You can imagine. As I wasn't using those navigational properties, because I used the IDs to get/set the data I needed, I deleted them from the model. To be specific I deleted points 4 and 5 from the UserTypes entity class. Then I had an object graph with each element only once. I tried a new update but I had the same problem:
The root element was updated fine, but the elements, that had some parents, threw the same exception.
I saw that I had a navigational property in the UserTypes_T entity class, pointing to a user type, so I deleted it too. Then this error disappeared. All the items in the object graph were unique. But the problem remained - I could update my root element with no problems, but when trying to update the children (with no exclusions) I got a null reference exception in the generated Model.Context.Extensions class:
if (!context.ObjectStateManager.TryGetObjectStateEntry(entityInSet.Item2, out entry))
{
context.AddObject(entityInSet.Item1, entityInSet.Item2);//here!
}
I tried to update only the name (which is in UserTypes_T) but the error is the same.
I'm out of ideas and I've been trying to solve this problem for 8 hours now, so I'll appreciate if someone gives me ideas or share their experience.
PS:
The only way I succeeded updating a child object was using the following code to retrieve the data:
var userTypes = argoContext.UserTypes.Include("UserTypes_T").Where(ut => ut.IsPrivate==false).ToList();
foreach (UserType ut in userTypes)
{
ut.UserType1 = null;
ut.UserTypes1 = null;
}
return userTypes;
where UserType1 is the navigational property, pointing to the parent user type and UserTypes1 is the navigational property, holding a list of the child element. The problem here was that EF "fixups" the objects and changes the Parent_ID to null. If I set it back again, EF sets the UserTypes1, too... Maybe there is a way to stop this behavior?
OK everybody, I just found what the problem was and I'm posting the answer if anybody else encounters the same issue.
The problem was that I was making some validation on the server in order to see if there isn't a circular reference between the user types. So, my method on the server looked something like:
using (MyEntities context = new MyEntities())
{
string errMsg = MyValidator.ValidateSomething(context.UserTypes,...);
if (!string.IsNullOrEmpty(errMsg)) throw new FaultException(errMsg);
//some other code here...
context.UserTypes.ApplyChanges(_userType);//_userType is the one that is updated
context.UserTypes.SaveChanges();
}
The problem is that when making the validation, the context is filled and when trying to save the changes, there are objects with the same key values.
The solution is simple - to use different context for validating things on the server:
using (MyEntities validationContext = new MyEntities())
{
//validation goes here...
}
using (MyEntities context = new MyEntities())
{
//saving changes and other processing...
}
Another one can be:
using (MyEntities context = new MyEntities())
{
using (MyEntities validationContext = new MyEntities())
{
//validation
}
//saving changes and other processing...
}
That's it! I hope it can be useful to somebody!
I've two entities with 1 to N relation in between. Let's say Books and Pages.
Book has a navigation property as Pages. Book has BookId as an identifier and Page has an auto generated id and a scalar property named PageNo. LazyLoading is set to true.
I've generated this using VS2010 & .net 4.0 and created a database from that.
In the partial class of Book, I need a GetPage function like below
public Page GetPage(int PageNumber)
{
//checking whether it exist etc are not included for simplicity
return Pages.Where(p=>p.PageNo==PageNumber).First();
}
This works. However, since Pages property in the Book is an EntityCollection it has to load all Pages of a book in memory in order to get the one page (this slows down the app when this function is hit for the first time for a given book). i.e. Framework does not merge the queries and run them at once. It loads the Pages in memory and then uses LINQ to objects to do the second part
To overcome this I've changed the code as follows
public Page GetPage(int PageNumber)
{
MyContainer container = new MyContainer();
return container.Pages.Where(p=>p.PageNo==PageNumber && p.Book.BookId==BookId).First();
}
This works considerably faster however it doesn't take into account the pages that have not been serialized to the db.
So, both options has its cons. Is there any trick in the framework to overcome this situation. This must be a common scenario where you don't want all of the objects of a Navigation property loaded in memory when you don't need them.
Trick? Besides "Try both?"
public Page GetPage(int pageNumber)
{
// check local values, possibly not persisted yet.
// works fine if nothing loaded.
var result = Pages.Where(p => p.PageNo == pageNumber).FirstOrDefault();
if (result != null)
{
return result;
}
// check DB if nothing found
MyContainer container = new MyContainer();
return container.Pages.Where(p => p.PageNo == pageNumber && p.Book.BookId==BookId).First();
}
There's nothing to do this automatically except for the specific case of loading by the PK value, for which you can use ObjectContext.[Try]GetObjectByKey.