EF Core load references of unknown entity - entity-framework

DISCLAIMER: Since we are all familiar with it, i will be using contoso university design to explain my question. Also, i am using EF core and .net core 2.0 on a mvc code first design.
I am developing a very generic RESTful API that works on any model. It has one method for each of create, read, update and delete operation in only one controller, the route of this is
[Route("/api/{resource}")]
Resource is the entity that the client wants to work with, for example if someone wants to get all Courses using the api he has to do a GET request on http://www.example.com/api/course/ or http://www.example.com/api/course/2 to get one by id and the following code will do the job.
[HttpGet("{id:int:min(1)?}")]
public IActionResult Read([FromRoute] string resource, [FromRoute] int? id)
{
//find resourse in models
IEntityType entityType = _context.Model
.GetEntityTypes()
.FirstOrDefault(x => x.Name.EndsWith($".{resource}", StringComparison.OrdinalIgnoreCase));
if (entityType == null) return NotFound(resource);
Type type = entityType.ClrType;
if (id == null)//select all from table
{
var entityRows = context.GetType().GetMethod("Set").MakeGenericMethod(type).Invoke(context, null);
if (entityRows == null)
return NoContent();
//TODO: load references (1)
return Ok(entityRows);
}
else //select by id
{
var entityRow = _context.Find(type, id);
if (entityRow == null)
return NoContent();
//TODO: load references (2)
return Ok(entityRows);
}
}
This small piece of code will do the magic with one small exception, intermediate collections will not be loaded. Given our example, the fetched course or courses will have no info for CourseInstructor (the intermediate collection in between Course and Person). I am trying to find a way to Eager load the navigation properties only if it is a collection; or by any other condition that will ensure that only many-to-many relationships are loaded.
For //TODO: load reference (2) i could use
_context.Entry(entityRow).Collection("CourseInsructor").Load();
On runtime if i could find all the navigation properties (filtered by spoken condition) and foreach of them i did Load(), i should get the desired result. My problem is when i get all (when id is null) the entityRows is type 'InternalDbSet' which is an unknown model.
So for the two TODOs i need some help on doing the following steps
1: find navigation properties of many-to-many relationships only
2: load them
Any suggestions?

In general, this seems like a very bad idea to me. While the CRUD stuff is going to be identical for most resources, there will be variances (as you've now run into). There's also something to be said for having a self-documenting API: with individual controllers, you know which resources can be accessed by nature of having a controller associated with that resource. With they way you're doing it, it's a complete black box. This also will of course effect any sort of actual generated API documentation. For example, if you were to include Swagger in your project, it would not be able to determine what you're doing here. Finally, you're now having to use reflection for everything, which will effect your performance.
What I would suggest instead is creating a base abstract controller and then creating a controller for each unique resource that inherits from that, for example:
public abstract class BaseController<TEntity> : Controller
where TEntity : class, new()
{
protected readonly MyContext _context;
public BaseController(MyContext context)
{
_context = context ?? throw new ArgumentNullException(nameof(context));
}
...
[HttpGet("create")]
public IActionResult Create()
{
var model = new TEntity();
return View(model);
}
[HttpPost("create")]
public async Task<IActionResult> Create(TEntity model)
{
if (ModelState.IsValid)
{
_context.Add(model);
await _context.SaveChangesAsync();
return RedirectToAction("Index");
}
return View(model);
}
...
}
I just wanted to give a quick example, but you'd build out all the rest of the CRUD methods in the same fashion, generically using TEntity. Then, for each actual resource, you simply do:
public class WidgetController : BaseController<Widget>
{
public WidgetController(MyContext context)
: base(context)
{
}
}
No duplication of code, but you've now got an actual real controller backing the resource, aiding both the innate and possibly explicit documentation of your API. And, no reflection anywhere.
Then, to solve problems like what you have here, you can add hooks to your base controller: essentially just virtual methods that are utilized in your base controller's CRUD actions and do nothing or just default things. However, you can then override these in your derived controllers to stub in additional functionality. For example, you can add something like:
public virtual IQueryable<TEntity> GetQueryable()
=> _context.Set<TEntity>();
Then, in your derived controller, you can do something like:
public class CourseController : BaseController<Course>
{
...
public override IQueryable<Course> GetQueryable()
=> base.GetQueryable().Include(x => x.CourseInstructors).ThenInclude(x => x.Instructor);
So, for example, you'd make your BaseController.Index action, perhaps, utilize GetQueryable() to get the list of entities to display. Simply by overriding this on the derived class, you can alter what happens based on the context of a particular type of resource.

Related

Entity Framework Multiple Include at runtime

I have a service that passes in parameters for how much I want to include for navigation properties. Based upon the boolean args it concatenates an entity list to include each required foreign entity.
At runtime I want to include either no navigation entities or many.
What I can't do is daisy chain with .Include().Include as I don't know which and how many to include based around passed in args.
I want to achieve this, but I don't seem to be able to pass in a comma separated entity list. Any ideas?
var res = db.Entity.Include(entityListCommaSeparated).Where(_=>_.ID == ID).FirstOrDefault();
This looks like a repository pattern, and generally gets messy if you want to try and "hide" EF / the DbContext from calling code.
A couple options you can consider:
Down the complexity rabit hole: use a params Expression<Func<TEntity, object>>[] includes in your applicable repository methods, and then be prepared to also pass OrderBy expressions, as well as pagination values when you want to return multiple entities.
THrough the simplicity mirror: Embrace IQueryable as a return type and let the consumers handle Includes, OrderBy's, Counts/Any/Skip/Take/First/ToList, and .Select() as they need.
Option 1:
public Order GetById(int id, params Expression<Func<Order, object>>[] includes)
{
var query = db.Orders.Where(x => x.ID == id);
// This part can be moved into an extension method or a base repository method.
if(includes.Any)
includes.Aggregate(query, (current, include) =>
{
current.Include(include);
}
// Don't use .FirstOrDefault() If you intend for 1 record to be returned, use .Single(). If it really is optional to find, .SingleOrDefault()
return query.Single();
}
//ToDo
public IEnumerable<Order> GetOrders(/* criteria?, includes?, order by?, (ascending/descending) pagination? */)
{ }
// or
public IEnumerable<Order> GetOrdersByCustomer(/* includes?, order by?, (ascending/descending) pagination? */)
{ }
// plus..
public IEnumerable<Order> GetOrdersByDate(/* includes?, order by?, (ascending/descending) pagination? */)
{ }
public bool CustomerHasOrders(int customerId)
{ }
public bool OrderExists(int id)
{ }
public int OrdersOnDate(DateTime date)
{ }
// etc. etc. etc.
Keep in mind this doesn't handle custom order by clauses, and the same will be needed for methods that are returning lists of entities. Your repository is also going to need to expose methods for .Any() (DoesExist) because everyone loves checking for #null on every return. :) Also .Count().
Option 2:
public IQueryable<Order> GetById(int id)
{
return db.Orders.Where(x => x.ID == id);
}
public IQueryable<Order> GetOrders()
{
return db.Orders.AsQueryable();
}
Callers can grok Linq and .Include() what they want before calling .Single(), or do a .Any().. They may not need the entire entity graph so they can .Select() from the entity and related entities without .Include() to compose and execute a more efficient query to populate a ViewModel / DTO. GetById might be used in a number of places so we can reduce duplication and support it in the repository. We don't need all of the filter scenarios etc, callers can call GetOrders and then filter as they see fit.
Why bother with a repository if it just returns DBSets?
Centralize low-level data filtering. For instance if you use Soft Deletes (IsActive) or are running multi-tenant, or explicit authorization. These common rules can be centralized at the repository level rather than having to remembered everywhere a DbSet is touched.
Testing is simpler. While you can mock a DbContext, or point it at an in-memory database, mocking a repository returning IQueryable is simpler. (Just populate a List<TEntity> and return .AsQueryable().
Repositories handle Create and Delete. Create to serve as a factory to ensure that all required data and relationships are established for a viable entity. Delete to handle soft-delete scenarios, cascades/audits etc. beyond what the DB handles behind the scenes.

EF Filtering a Child Table with Lazy Load

I'm using entity framework with POCOs and the repository pattern and am wondering if there is any way to filter a child list lazy load. Example:
class Person
{
public virtual Organisation organisation {set; get;}
}
class Organisation
{
public virtual ICollection<Product> products {set; get;}
}
class Product
{
public bool active {set; get;}
}
Currently I only have a person repository because I'm always starting from that point, so ideally I would like to do the following:
Person person = personRepo.GetById(Id);
var products = person.organisation.products;
And have it only load products where active = true from the database.
Is this possible and if so how?
EDIT My best guess would be either a filter can be added to the configuration of the entity. Or there might be a way to intercept/override the lazy load call and modify it. Obviously if I created an Organisation Repository I could manually load it as I please but I am trying to avoid that.
There's not a direct way to do this via lazy loading, but if you were willing to explicitly load the collection, you could follow whats in this blog, see the Applying filters when explicitly loading related entities section.
context.Entry(person)
.Collection(p => p.organisation.products)
.Query()
.Where(u => u.IsActive)
.Load();
You can do what Mark Oreta and luksan suggest while keeping all the query logic within the repository.
All you have to do is pass a Lazy<ICollection<Product>> into the organization constructor, and use the logic they provided. It will not evaluate until you access the value property of the lazy instance.
UPDATE
/*
First, here are your changes to the Organisation class:
Add a constructor dependency on the delegate to load the products to your
organization class. You will create this object in the repository method
and assign it to the Person.Organization property
*/
public class Organisation
{
private readonly Lazy<ICollection<Product>> lazyProducts;
public Organisation(Func<ICollection<Product>> loadProducts){
this.lazyProducts = new Lazy<ICollection<Product>>(loadProducts);
}
// The underlying lazy field will not invoke the load delegate until this property is accessed
public virtual ICollection<Product> Products { get { return this.lazyProducts.Value; } }
}
Now, in your repository method, when you construct the Person object you will assign the Organisation property with an Organisation object containing the lazy loading field.
So, without seeing your whole model, it will looks something like
public Person GetById(int id){
var person = context.People.Single(p => p.Id == id);
/* Now, I'm not sure about the cardinality of the person-organization or organisation
product relationships, but let's assume you have some way to access the PK of the
organization record from the Person and that the Product has a reference to
its Organisation. I may be misinterpreting your model, but hopefully you
will get the idea
*/
var organisationId = /* insert the aforementioned magic here */
Func<ICollection<Product>> loadProducts = () => context.Products.Where(product => product.IsActive && product.OrganisationId == organisationId).ToList();
person.Organisation = new Organisation( loadProducts );
return person;
}
By using this approach, the query for the products will not be loaded until you access the Products property on the Organisationinstance, and you can keep all your logic in the repository. There's a good chance that I made incorrect assumptions about your model (as the sample code is quite incomplete), but I think there is enough here for you to see how to use the pattern. Let me know if any of this is unclear.
This might be related:
Using CreateSourceQuery in CTP4 Code First
If you were to redefine your properties as ICollection<T> rather than IList<T> and enable change-tracking proxies, then you might be able to cast them to EntityCollection<T> and then call CreateSourceQuery() which would allow you to execute LINQ to Entities queries against them.
Example:
var productsCollection = (EntityCollection<Product>)person.organisation.products;
var productsQuery = productsCollection.CreateSourceQuery();
var activeProducts = products.Where(p => p.Active);
Is your repository using something like:
IQueryable<T> Find(System.Linq.Expressions.Expression<Func<T, bool>> expression)
If so you can do something like this:
var person = personRepo.Find(p => p.organisation.products.Any(e => e.active)).FirstOrDefault();
You could possibly use Query() method to achieve this. Something like:
context.Entry(person)
.Collection(p => p.organisation.products)
.Query()
.Where(pro=> pro.Active==true)
.Load();
Have a look at this page click here

Decouple EF queries from BL - Extension Methods VS Class-Per-Query

I have read dozens of posts about PROs and CONs of trying to mock \ fake EF in the business logic.
I have not yet decided what to do - but one thing I know is - I have to separate the queries from the business logic.
In this post I saw that Ladislav has answered that there are 2 good ways:
Let them be where they are and use custom extension methods, query views, mapped database views or custom defining queries to define reusable parts.
Expose every single query as method on some separate class. The method
mustn't expose IQueryable and mustn't accept Expression as parameter =
whole query logic must be wrapped in the method. But this will make
your class covering related methods much like repository (the only one
which can be mocked or faked). This implementation is close to
implementation used with stored procedures.
Which method do you think is better any why ?
Are there ANY downsides to put the queries in their own place ? (maybe losing some functionality from EF or something like that)
Do I have to encapsulate even the simplest queries like:
using (MyDbContext entities = new MyDbContext)
{
User user = entities.Users.Find(userId); // ENCAPSULATE THIS ?
// Some BL Code here
}
So I guess your main point is testability of your code, isn't it? In such case you should start by counting responsibilities of the method you want to test and than refactor your code using single responsibility pattern.
Your example code has at least three responsibilities:
Creating an object is a responsibility - context is an object. Moreover it is and object you don't want to use in your unit test so you must move its creation elsewhere.
Executing query is a responsibility. Moreover it is a responsibility you would like to avoid in your unit test.
Doing some business logic is a responsibility
To simplify testing you should refactor your code and divide those responsibilities to separate methods.
public class MyBLClass()
{
public void MyBLMethod(int userId)
{
using (IMyContext entities = GetContext())
{
User user = GetUserFromDb(entities, userId);
// Some BL Code here
}
}
protected virtual IMyContext GetContext()
{
return new MyDbContext();
}
protected virtual User GetUserFromDb(IMyDbContext entities, int userId)
{
return entities.Users.Find(userId);
}
}
Now unit testing business logic should be piece of cake because your unit test can inherit your class and fake context factory method and query execution method and become fully independent on EF.
// NUnit unit test
[TestFixture]
public class MyBLClassTest : MyBLClass
{
private class FakeContext : IMyContext
{
// Create just empty implementation of context interface
}
private User _testUser;
[Test]
public void MyBLMethod_DoSomething()
{
// Test setup
int id = 10;
_testUser = new User
{
Id = id,
// rest is your expected test data - that is what faking is about
// faked method returns simply data your test method expects
};
// Execution of method under test
MyBLMethod(id);
// Test validation
// Assert something you expect to happen on _testUser instance
// inside MyBLMethod
}
protected override IMyContext GetContext()
{
return new FakeContext();
}
protected override User GetUserFromDb(IMyContext context, int userId)
{
return _testUser.Id == userId ? _testUser : null;
}
}
As you add more methods and your application grows you will refactor those query execution methods and context factory method to separate classes to follow single responsibility on classes as well - you will get context factory and either some query provider or in some cases repository (but that repository will never return IQueryable or get Expression as parameter in any of its methods). This will also allow you following DRY principle where your context creation and most commonly used queries will be defined only once on one central place.
So at the end you can have something like this:
public class MyBLClass()
{
private IContextFactory _contextFactory;
private IUserQueryProvider _userProvider;
public MyBLClass(IContextFactory contextFactory, IUserQueryProvider userProvider)
{
_contextFactory = contextFactory;
_userProvider = userProvider;
}
public void MyBLMethod(int userId)
{
using (IMyContext entities = _contextFactory.GetContext())
{
User user = _userProvider.GetSingle(entities, userId);
// Some BL Code here
}
}
}
Where those interfaces will look like:
public interface IContextFactory
{
IMyContext GetContext();
}
public class MyContextFactory : IContextFactory
{
public IMyContext GetContext()
{
// Here belongs any logic necessary to create context
// If you for example want to cache context per HTTP request
// you can implement logic here.
return new MyDbContext();
}
}
and
public interface IUserQueryProvider
{
User GetUser(int userId);
// Any other reusable queries for user entities
// Non of queries returns IQueryable or accepts Expression as parameter
// For example: IEnumerable<User> GetActiveUsers();
}
public class MyUserQueryProvider : IUserQueryProvider
{
public User GetUser(IMyContext context, int userId)
{
return context.Users.Find(userId);
}
// Implementation of other queries
// Only inside query implementations you can use extension methods on IQueryable
}
Your test will now only use fakes for context factory and query provider.
// NUnit + Moq unit test
[TestFixture]
public class MyBLClassTest
{
private class FakeContext : IMyContext
{
// Create just empty implementation of context interface
}
[Test]
public void MyBLMethod_DoSomething()
{
// Test setup
int id = 10;
var user = new User
{
Id = id,
// rest is your expected test data - that is what faking is about
// faked method returns simply data your test method expects
};
var contextFactory = new Mock<IContextFactory>();
contextFactory.Setup(f => f.GetContext()).Returns(new FakeContext());
var queryProvider = new Mock<IUserQueryProvider>();
queryProvider.Setup(f => f.GetUser(It.IsAny<IContextFactory>(), id)).Returns(user);
// Execution of method under test
var myBLClass = new MyBLClass(contextFactory.Object, queryProvider.Object);
myBLClass.MyBLMethod(id);
// Test validation
// Assert something you expect to happen on user instance
// inside MyBLMethod
}
}
It would be little bit different in case of repository which should have reference to context passed to its constructor prior to injecting it to your business class.
Your business class can still define some queries which are never use in any other classes - those queries are most probably part of its logic. You can also use extension methods to define some reusable part of queries but you must always use those extension methods outside of your core business logic which you want to unit test (either in query execution methods or in query provider / repository). That will allow you easy faking query provider or query execution methods.
I saw your previous question and thought about writing a blog post about that topic but the core of my opinion about testing with EF is in this answer.
Edit:
Repository is different topic which doesn't relate to your original question. Specific repository is still valid pattern. We are not against repositories, we are against generic repositories because they don't provide any additional features and don't solve any problem.
The problem is that repository alone doesn't solve anything. There are three patterns which have to be used together to form proper abstraction: Repository, Unit of Work and Specifications. All three are already available in EF: DbSet / ObjectSet as repositories, DbContext / ObjectContext as Unit of works and Linq to Entities as specifications. The main problem with custom implementation of generic repositories mentioned everywhere is that they replace only repository and unit of work with custom implementation but still depend on original specifications => abstraction is incomplete and it is leaking in tests where faked repository behaves in the same way as faked set / context.
The main disadvantage of my query provider is explicit method for any query you will need to execute. In case of repository you will not have such methods you will have just few methods accepting specification (but again those specifications should be defined in DRY principle) which will build query filtering conditions, ordering etc.
public interface IUserRepository
{
User Find(int userId);
IEnumerable<User> FindAll(ISpecification spec);
}
The discussion of this topic is far beyond the scope of this question and it requires you to do some self study.
Btw. mocking and faking has different purpose - you fake a call if you need to get testing data from method in the dependency and you mock the call if you need to assert that method on dependency was called with expected arguments.

Dependency Injection & Model Binding (ASP MVC, Autofac), When to use what?

This is more like a conceptual question. When to use Model Binding (in ASP.NET MVC Framework) and when to inject objects using IoC (lets say Autofac here) ?
One specific scenario is like lets say, I have the following action method
public ActionResult EditProfile(string UserId)
{
// get user object from repository using the the UserId
// edit profile
// save changes
// return feedback
}
In the above scenario, is it possible to inject a user object to action method such that it automatically gets the user object using the UserId ? The resulting signature being:
public ActionResult EditProfile(UserProfile userObj) //userObj injected *somehow* to automatically retreive the object from repo using UserId ?
Sorry if it all doesn't makes sense. It`s my first time using IoC.
EDIT:
This is the way to do it > http://buildstarted.com/2010/09/12/custom-model-binders-in-mvc-3-with-imodelbinder/
You can do what you need using a custom action filter. By overriding OnActionExecuting, we have access to the route data, and the action parameters of the action that will be executed. Given:
public class BindUserProfileAttribute : ActionFilterAttribute
{
public override OnActionExecuting(FilterContext filterContext)
{
string id = (string)filterContext.RouteData.Values["UserId"];
var model = new UserProfile { Id = id };
filtextContext.ActionParameters["userObj"] = model;
}
}
This attribute allows us to create the parameters that will be passed into the action, so we can load the user object at this point.
[BindUserProfile]
public ActionResult EditProfile(UserProfile userObj)
{
}
You'll probably need to get specific with your routes:
routes.MapRoute(
"EditProfile",
"Account/EditProfile/{UserId}",
new { controller = "Account", action = "EditProfile" });
In MVC3 we get access to the new IDepedencyResolver interface, which allows us to perform IoC/SL using whatever IoC container or service locator we want, so we can push a service like a IUserProfileFactory into your filter, to then be able to create your UserProfile instance.
Hope that helps?
Model binding is used for your data. Dependency injection is used for your business logic.

Entity Framework and Entity Tracker Problems

If I run the following code it throws the following error:
An entity object cannot be referenced by multiple instances of IEntityChangeTracker
public void Save(Category category)
{
using(var db = new NorthwindContext())
{
if(category.CategoryID == 0)
{
db.AddToCategorySet(category);
}
else
{
//category.RemoveTracker();
db.Attach(category);
}
db.SaveChanges();
}
}
The reason is of course that the category is sent from interface which we got from GetById method which already attached the EntityChangeTracker to the category object. I also tried to set the entity tracker to null but it did not update the category object.
protected void Btn_Update_Category_Click(object sender, EventArgs e)
{
_categoryRepository = new CategoryRepository();
int categoryId = Int32.Parse(txtCategoryId.Text);
var category = _categoryRepository.GetById(categoryId);
category.CategoryName = txtUpdateCategoryName.Text;
_categoryRepository.Save(category);
}
I'm still learning Entity Framework myself, but maybe I can help a little. When working with the Entity Framework, you need to be aware of how you're handling different contexts. It looks like you're trying to localize your context as much as possible by saying:
public void Save(Category category)
{
using (var db = new NorthwindContext())
{
...
}
}
... within your data access method. Did you do the same thing in your GetById method? If so, did you remember to detach the object you got back so that it could be attached later in a different context?
public Category GetById(int categoryId)
{
using (var db = new NorthwindContext())
{
Category category = (from c in db.Category where Category.ID == categoryId select c).First();
db.Detach(category);
}
}
That way when you call Attach it isn't trying to step on an already-attached context. Does that help?
As you pointed out in your comment, this poses a problem when you're trying to modify an item and then tell your database layer to save it, because once an item is detached from its context, it no longer keeps track of the changes that were made to it. There are a few ways I can think of to get around this problem, none of them perfect.
If your architecture supports it, you could expand the scope of your context enough that your Save method could use the same context that your GetById method uses. This helps to avoid the whole attach/detach problem entirely, but it might push your data layer a little closer to your business logic than you would like.
You can load a new instance of the item out of the new context based on its ID, set all of its properties based on the category that is passed in, and then save it. This costs two database round-trips for what should really only need one, and it isn't very maintainable.
You can dig into the context itself to mark the Category's properties as changed.
For example:
public void Save(Category category)
{
using (var db = new NorthwindContext())
{
db.Attach(category);
var stateEntry = db.ObjectStateManager.GetObjectStateEntry(category);
foreach (var propertyName in stateEntry.CurrentValues.DataRecordInfo.FieldMetadata.Select(fm => fm.FieldType.Name)) {
stateEntry.SetModifiedProperty(propertyName);
}
db.SaveChanges();
}
}
This looks a little uglier, but should be more performant and maintainable overall. Plus, if you want, you could make it generic enough to throw into an extension method somewhere so you don't have to see or repeat the ugly code, but you still get the functionality out of it.