Decouple EF queries from BL - Extension Methods VS Class-Per-Query - entity-framework

I have read dozens of posts about PROs and CONs of trying to mock \ fake EF in the business logic.
I have not yet decided what to do - but one thing I know is - I have to separate the queries from the business logic.
In this post I saw that Ladislav has answered that there are 2 good ways:
Let them be where they are and use custom extension methods, query views, mapped database views or custom defining queries to define reusable parts.
Expose every single query as method on some separate class. The method
mustn't expose IQueryable and mustn't accept Expression as parameter =
whole query logic must be wrapped in the method. But this will make
your class covering related methods much like repository (the only one
which can be mocked or faked). This implementation is close to
implementation used with stored procedures.
Which method do you think is better any why ?
Are there ANY downsides to put the queries in their own place ? (maybe losing some functionality from EF or something like that)
Do I have to encapsulate even the simplest queries like:
using (MyDbContext entities = new MyDbContext)
{
User user = entities.Users.Find(userId); // ENCAPSULATE THIS ?
// Some BL Code here
}

So I guess your main point is testability of your code, isn't it? In such case you should start by counting responsibilities of the method you want to test and than refactor your code using single responsibility pattern.
Your example code has at least three responsibilities:
Creating an object is a responsibility - context is an object. Moreover it is and object you don't want to use in your unit test so you must move its creation elsewhere.
Executing query is a responsibility. Moreover it is a responsibility you would like to avoid in your unit test.
Doing some business logic is a responsibility
To simplify testing you should refactor your code and divide those responsibilities to separate methods.
public class MyBLClass()
{
public void MyBLMethod(int userId)
{
using (IMyContext entities = GetContext())
{
User user = GetUserFromDb(entities, userId);
// Some BL Code here
}
}
protected virtual IMyContext GetContext()
{
return new MyDbContext();
}
protected virtual User GetUserFromDb(IMyDbContext entities, int userId)
{
return entities.Users.Find(userId);
}
}
Now unit testing business logic should be piece of cake because your unit test can inherit your class and fake context factory method and query execution method and become fully independent on EF.
// NUnit unit test
[TestFixture]
public class MyBLClassTest : MyBLClass
{
private class FakeContext : IMyContext
{
// Create just empty implementation of context interface
}
private User _testUser;
[Test]
public void MyBLMethod_DoSomething()
{
// Test setup
int id = 10;
_testUser = new User
{
Id = id,
// rest is your expected test data - that is what faking is about
// faked method returns simply data your test method expects
};
// Execution of method under test
MyBLMethod(id);
// Test validation
// Assert something you expect to happen on _testUser instance
// inside MyBLMethod
}
protected override IMyContext GetContext()
{
return new FakeContext();
}
protected override User GetUserFromDb(IMyContext context, int userId)
{
return _testUser.Id == userId ? _testUser : null;
}
}
As you add more methods and your application grows you will refactor those query execution methods and context factory method to separate classes to follow single responsibility on classes as well - you will get context factory and either some query provider or in some cases repository (but that repository will never return IQueryable or get Expression as parameter in any of its methods). This will also allow you following DRY principle where your context creation and most commonly used queries will be defined only once on one central place.
So at the end you can have something like this:
public class MyBLClass()
{
private IContextFactory _contextFactory;
private IUserQueryProvider _userProvider;
public MyBLClass(IContextFactory contextFactory, IUserQueryProvider userProvider)
{
_contextFactory = contextFactory;
_userProvider = userProvider;
}
public void MyBLMethod(int userId)
{
using (IMyContext entities = _contextFactory.GetContext())
{
User user = _userProvider.GetSingle(entities, userId);
// Some BL Code here
}
}
}
Where those interfaces will look like:
public interface IContextFactory
{
IMyContext GetContext();
}
public class MyContextFactory : IContextFactory
{
public IMyContext GetContext()
{
// Here belongs any logic necessary to create context
// If you for example want to cache context per HTTP request
// you can implement logic here.
return new MyDbContext();
}
}
and
public interface IUserQueryProvider
{
User GetUser(int userId);
// Any other reusable queries for user entities
// Non of queries returns IQueryable or accepts Expression as parameter
// For example: IEnumerable<User> GetActiveUsers();
}
public class MyUserQueryProvider : IUserQueryProvider
{
public User GetUser(IMyContext context, int userId)
{
return context.Users.Find(userId);
}
// Implementation of other queries
// Only inside query implementations you can use extension methods on IQueryable
}
Your test will now only use fakes for context factory and query provider.
// NUnit + Moq unit test
[TestFixture]
public class MyBLClassTest
{
private class FakeContext : IMyContext
{
// Create just empty implementation of context interface
}
[Test]
public void MyBLMethod_DoSomething()
{
// Test setup
int id = 10;
var user = new User
{
Id = id,
// rest is your expected test data - that is what faking is about
// faked method returns simply data your test method expects
};
var contextFactory = new Mock<IContextFactory>();
contextFactory.Setup(f => f.GetContext()).Returns(new FakeContext());
var queryProvider = new Mock<IUserQueryProvider>();
queryProvider.Setup(f => f.GetUser(It.IsAny<IContextFactory>(), id)).Returns(user);
// Execution of method under test
var myBLClass = new MyBLClass(contextFactory.Object, queryProvider.Object);
myBLClass.MyBLMethod(id);
// Test validation
// Assert something you expect to happen on user instance
// inside MyBLMethod
}
}
It would be little bit different in case of repository which should have reference to context passed to its constructor prior to injecting it to your business class.
Your business class can still define some queries which are never use in any other classes - those queries are most probably part of its logic. You can also use extension methods to define some reusable part of queries but you must always use those extension methods outside of your core business logic which you want to unit test (either in query execution methods or in query provider / repository). That will allow you easy faking query provider or query execution methods.
I saw your previous question and thought about writing a blog post about that topic but the core of my opinion about testing with EF is in this answer.
Edit:
Repository is different topic which doesn't relate to your original question. Specific repository is still valid pattern. We are not against repositories, we are against generic repositories because they don't provide any additional features and don't solve any problem.
The problem is that repository alone doesn't solve anything. There are three patterns which have to be used together to form proper abstraction: Repository, Unit of Work and Specifications. All three are already available in EF: DbSet / ObjectSet as repositories, DbContext / ObjectContext as Unit of works and Linq to Entities as specifications. The main problem with custom implementation of generic repositories mentioned everywhere is that they replace only repository and unit of work with custom implementation but still depend on original specifications => abstraction is incomplete and it is leaking in tests where faked repository behaves in the same way as faked set / context.
The main disadvantage of my query provider is explicit method for any query you will need to execute. In case of repository you will not have such methods you will have just few methods accepting specification (but again those specifications should be defined in DRY principle) which will build query filtering conditions, ordering etc.
public interface IUserRepository
{
User Find(int userId);
IEnumerable<User> FindAll(ISpecification spec);
}
The discussion of this topic is far beyond the scope of this question and it requires you to do some self study.
Btw. mocking and faking has different purpose - you fake a call if you need to get testing data from method in the dependency and you mock the call if you need to assert that method on dependency was called with expected arguments.

Related

Entity Framework Multiple Include at runtime

I have a service that passes in parameters for how much I want to include for navigation properties. Based upon the boolean args it concatenates an entity list to include each required foreign entity.
At runtime I want to include either no navigation entities or many.
What I can't do is daisy chain with .Include().Include as I don't know which and how many to include based around passed in args.
I want to achieve this, but I don't seem to be able to pass in a comma separated entity list. Any ideas?
var res = db.Entity.Include(entityListCommaSeparated).Where(_=>_.ID == ID).FirstOrDefault();
This looks like a repository pattern, and generally gets messy if you want to try and "hide" EF / the DbContext from calling code.
A couple options you can consider:
Down the complexity rabit hole: use a params Expression<Func<TEntity, object>>[] includes in your applicable repository methods, and then be prepared to also pass OrderBy expressions, as well as pagination values when you want to return multiple entities.
THrough the simplicity mirror: Embrace IQueryable as a return type and let the consumers handle Includes, OrderBy's, Counts/Any/Skip/Take/First/ToList, and .Select() as they need.
Option 1:
public Order GetById(int id, params Expression<Func<Order, object>>[] includes)
{
var query = db.Orders.Where(x => x.ID == id);
// This part can be moved into an extension method or a base repository method.
if(includes.Any)
includes.Aggregate(query, (current, include) =>
{
current.Include(include);
}
// Don't use .FirstOrDefault() If you intend for 1 record to be returned, use .Single(). If it really is optional to find, .SingleOrDefault()
return query.Single();
}
//ToDo
public IEnumerable<Order> GetOrders(/* criteria?, includes?, order by?, (ascending/descending) pagination? */)
{ }
// or
public IEnumerable<Order> GetOrdersByCustomer(/* includes?, order by?, (ascending/descending) pagination? */)
{ }
// plus..
public IEnumerable<Order> GetOrdersByDate(/* includes?, order by?, (ascending/descending) pagination? */)
{ }
public bool CustomerHasOrders(int customerId)
{ }
public bool OrderExists(int id)
{ }
public int OrdersOnDate(DateTime date)
{ }
// etc. etc. etc.
Keep in mind this doesn't handle custom order by clauses, and the same will be needed for methods that are returning lists of entities. Your repository is also going to need to expose methods for .Any() (DoesExist) because everyone loves checking for #null on every return. :) Also .Count().
Option 2:
public IQueryable<Order> GetById(int id)
{
return db.Orders.Where(x => x.ID == id);
}
public IQueryable<Order> GetOrders()
{
return db.Orders.AsQueryable();
}
Callers can grok Linq and .Include() what they want before calling .Single(), or do a .Any().. They may not need the entire entity graph so they can .Select() from the entity and related entities without .Include() to compose and execute a more efficient query to populate a ViewModel / DTO. GetById might be used in a number of places so we can reduce duplication and support it in the repository. We don't need all of the filter scenarios etc, callers can call GetOrders and then filter as they see fit.
Why bother with a repository if it just returns DBSets?
Centralize low-level data filtering. For instance if you use Soft Deletes (IsActive) or are running multi-tenant, or explicit authorization. These common rules can be centralized at the repository level rather than having to remembered everywhere a DbSet is touched.
Testing is simpler. While you can mock a DbContext, or point it at an in-memory database, mocking a repository returning IQueryable is simpler. (Just populate a List<TEntity> and return .AsQueryable().
Repositories handle Create and Delete. Create to serve as a factory to ensure that all required data and relationships are established for a viable entity. Delete to handle soft-delete scenarios, cascades/audits etc. beyond what the DB handles behind the scenes.

Can I keep Entity Framework context as class variable?

I'm used to working the database connections where you connect/open/close as fast as possible in each method. I'm now working with the Entity Framework and so my methods all do this type of thing:
using (var context = new FooEntities()) {
// linq to sql query here
}
I've been told that with Entity Framework I can actually have that context variable be a class level variable and not have to instantiate it in each method. Is that really the case, or should I continue this pattern in each method?
I'm using version 5.0.0 of the framework if that makes a difference.
It depends on how you are expecting it to act. The only reason you'd want it to stick around is if you wanted to use the caching feature of DbContext across multiple method calls. But since its pulling connections from the Pool anyway, disposing of a DbContext shouldn't really impact performance when creating a new one.
For me personally, I create the context as close as possible and kill it as soon as possible. Thus, the Get calls should use AsNoTracking() to speed up the calls a lot if you don't care about trying to update them later. You could also create a DbContextFactory so each class could control that interaction as it sees fit. (i.e. Method A always creates a new one, but Methods B and C could share if either one called first). Though, that could cause its own issues down the road, but then you can opt into those conditions.
You can have Context as a property of a class, but you have to consider how to control the disposing of the Context. For example:
public class UnitOfWork:IDisposable
{
public DbContext Context { get; set; }
public UnitOfWork()
{
Context = null; //initialize context here
}
public void DoWorkWithContext1()
{
//anything you need
}
public void DoWorkWithContext2()
{
//anything you need
}
public void Dispose()
{
if (Context != null)
Context.Dispose();
}
}
Then you'll use the class in this way:
using (var unit= new UnitOfWork())
{
unit.DoWorkWithContext1();
unit.DoWorkWithContext2();
}

EF Core load references of unknown entity

DISCLAIMER: Since we are all familiar with it, i will be using contoso university design to explain my question. Also, i am using EF core and .net core 2.0 on a mvc code first design.
I am developing a very generic RESTful API that works on any model. It has one method for each of create, read, update and delete operation in only one controller, the route of this is
[Route("/api/{resource}")]
Resource is the entity that the client wants to work with, for example if someone wants to get all Courses using the api he has to do a GET request on http://www.example.com/api/course/ or http://www.example.com/api/course/2 to get one by id and the following code will do the job.
[HttpGet("{id:int:min(1)?}")]
public IActionResult Read([FromRoute] string resource, [FromRoute] int? id)
{
//find resourse in models
IEntityType entityType = _context.Model
.GetEntityTypes()
.FirstOrDefault(x => x.Name.EndsWith($".{resource}", StringComparison.OrdinalIgnoreCase));
if (entityType == null) return NotFound(resource);
Type type = entityType.ClrType;
if (id == null)//select all from table
{
var entityRows = context.GetType().GetMethod("Set").MakeGenericMethod(type).Invoke(context, null);
if (entityRows == null)
return NoContent();
//TODO: load references (1)
return Ok(entityRows);
}
else //select by id
{
var entityRow = _context.Find(type, id);
if (entityRow == null)
return NoContent();
//TODO: load references (2)
return Ok(entityRows);
}
}
This small piece of code will do the magic with one small exception, intermediate collections will not be loaded. Given our example, the fetched course or courses will have no info for CourseInstructor (the intermediate collection in between Course and Person). I am trying to find a way to Eager load the navigation properties only if it is a collection; or by any other condition that will ensure that only many-to-many relationships are loaded.
For //TODO: load reference (2) i could use
_context.Entry(entityRow).Collection("CourseInsructor").Load();
On runtime if i could find all the navigation properties (filtered by spoken condition) and foreach of them i did Load(), i should get the desired result. My problem is when i get all (when id is null) the entityRows is type 'InternalDbSet' which is an unknown model.
So for the two TODOs i need some help on doing the following steps
1: find navigation properties of many-to-many relationships only
2: load them
Any suggestions?
In general, this seems like a very bad idea to me. While the CRUD stuff is going to be identical for most resources, there will be variances (as you've now run into). There's also something to be said for having a self-documenting API: with individual controllers, you know which resources can be accessed by nature of having a controller associated with that resource. With they way you're doing it, it's a complete black box. This also will of course effect any sort of actual generated API documentation. For example, if you were to include Swagger in your project, it would not be able to determine what you're doing here. Finally, you're now having to use reflection for everything, which will effect your performance.
What I would suggest instead is creating a base abstract controller and then creating a controller for each unique resource that inherits from that, for example:
public abstract class BaseController<TEntity> : Controller
where TEntity : class, new()
{
protected readonly MyContext _context;
public BaseController(MyContext context)
{
_context = context ?? throw new ArgumentNullException(nameof(context));
}
...
[HttpGet("create")]
public IActionResult Create()
{
var model = new TEntity();
return View(model);
}
[HttpPost("create")]
public async Task<IActionResult> Create(TEntity model)
{
if (ModelState.IsValid)
{
_context.Add(model);
await _context.SaveChangesAsync();
return RedirectToAction("Index");
}
return View(model);
}
...
}
I just wanted to give a quick example, but you'd build out all the rest of the CRUD methods in the same fashion, generically using TEntity. Then, for each actual resource, you simply do:
public class WidgetController : BaseController<Widget>
{
public WidgetController(MyContext context)
: base(context)
{
}
}
No duplication of code, but you've now got an actual real controller backing the resource, aiding both the innate and possibly explicit documentation of your API. And, no reflection anywhere.
Then, to solve problems like what you have here, you can add hooks to your base controller: essentially just virtual methods that are utilized in your base controller's CRUD actions and do nothing or just default things. However, you can then override these in your derived controllers to stub in additional functionality. For example, you can add something like:
public virtual IQueryable<TEntity> GetQueryable()
=> _context.Set<TEntity>();
Then, in your derived controller, you can do something like:
public class CourseController : BaseController<Course>
{
...
public override IQueryable<Course> GetQueryable()
=> base.GetQueryable().Include(x => x.CourseInstructors).ThenInclude(x => x.Instructor);
So, for example, you'd make your BaseController.Index action, perhaps, utilize GetQueryable() to get the list of entities to display. Simply by overriding this on the derived class, you can alter what happens based on the context of a particular type of resource.

Unit testing with EF Code First DataContext

This is more a solution / work around than an actual question. I'm posting it here since I couldn't find this solution on stack overflow or indeed after a lot of Googling.
The Problem:
I have an MVC 3 webapp using EF 4 code first that I want to write unit tests for. I'm also using NCrunch to run the unit tests on the fly as I code, so I'd like to avoid backing onto an actual database here.
Other Solutions:
IDataContext
I've found this the most accepted way to create an in memory datacontext. It effectively involves writing an interface IMyDataContext for your MyDataContext and then using the interface in all your controllers. An example of doing this is here.
This is the route I went with initially and I even went as far as writing a T4 template to extract IMyDataContext from MyDataContext since I don't like having to maintain duplicate dependent code.
However I quickly discovered that some Linq statements fail in production when using IMyDataContext instead of MyDataContext. Specifically queries like this throw a NotSupportedException
var siteList = from iSite in MyDataContext.Sites
let iMaxPageImpression = (from iPage in MyDataContext.Pages where iSite.SiteId == iPage.SiteId select iPage.AvgMonthlyImpressions).Max()
select new { Site = iSite, MaxImpressions = iMaxPageImpression };
My Solution
This was actually quite simple. I simply created a MyInMemoryDataContext subclass to MyDataContext and overrode all the IDbSet<..> properties as below:
public class InMemoryDataContext : MyDataContext, IObjectContextAdapter
{
/// <summary>Whether SaveChanges() was called on the DataContext</summary>
public bool SaveChangesWasCalled { get; private set; }
public InMemoryDataContext()
{
InitializeDataContextProperties();
SaveChangesWasCalled = false;
}
/// <summary>
/// Initialize all MyDataContext properties with appropriate container types
/// </summary>
private void InitializeDataContextProperties()
{
Type myType = GetType().BaseType; // We have to do this since private Property.Set methods are not accessible through GetType()
// ** Initialize all IDbSet<T> properties with CollectionDbSet<T> instances
var DbSets = myType.GetProperties().Where(x => x.PropertyType.IsGenericType && x.PropertyType.GetGenericTypeDefinition() == typeof(IDbSet<>)).ToList();
foreach (var iDbSetProperty in DbSets)
{
var concreteCollectionType = typeof(CollectionDbSet<>).MakeGenericType(iDbSetProperty.PropertyType.GetGenericArguments());
var collectionInstance = Activator.CreateInstance(concreteCollectionType);
iDbSetProperty.SetValue(this, collectionInstance,null);
}
}
ObjectContext IObjectContextAdapter.ObjectContext
{
get { return null; }
}
public override int SaveChanges()
{
SaveChangesWasCalled = true;
return -1;
}
}
In this case my CollectionDbSet<> is a slightly modified version of FakeDbSet<> here (which simply implements IDbSet with an underlying ObservableCollection and ObservableCollection.AsQueryable()).
This solution works nicely with all my unit tests and specifically with NCrunch running these tests on the fly.
Full Integration Tests
These Unit tests test all the business logic but one major downside is that none of your LINQ statements are guaranteed to work with your actual MyDataContext. This is because testing against an in memory data context means you're replacing the Linq-To-Entity provider but a Linq-To-Objects provider (as pointed out very well in the answer to this SO question).
To fix this I use Ninject within my unit tests and setup InMemoryDataContext to bind instead of MyDataContext within my unit tests. You can then use Ninject to bind to an actual MyDataContext when running the integration tests (via a setting in the app.config).
if(Global.RunIntegrationTest)
DependencyInjector.Bind<MyDataContext>().To<MyDataContext>().InSingletonScope();
else
DependencyInjector.Bind<MyDataContext>().To<InMemoryDataContext>().InSingletonScope();
Let me know if you have any feedback on this however, there are always improvements to be made.
As per my comment in the question, this was more to help others searching for this problem on SO. But as pointed out in the comments underneath the question there are quite a few other design approaches that would fix this problem.

What is the difference between factory and pipeline design patterns?

What is the difference between factory and pipeline design patterns?
I am asking because I need making classes, each of which has a method that will transform textual data in a certain way.
I have other classes whose data needs to be transformed. However, the order and selection of the transformations depends on (and only on) which base class from which these classes inherit.
Is this somehow related pipeline and/or a factory pattern?
Factory creates objects without exposing the instantiation logic to the client and refers to the newly created object through a common interface. So, goal is to make client completely unaware of what concrete type of product it uses and how that instance created.
public interface IFactory // used by clients
{
IProduct CreateProduct();
}
public class FooFactory : IFactory
{
public IProduct CreateProduct()
{
// create new instance of FooProduct
// setup something
// setup something else
// return it
}
}
All creation details are encapsulated. You can create instance via new() call. Or you can clone some existing sample FooProduct. You can skip setup. Or you can read some data from database before. Anything.
Here we go to Pipeline. Pipeline purpose is to divide a larger processing task into a sequence of smaller, independent processing steps (Filters). If creation of your objects is a large task AND setup steps are independent, you can use pipeline for setup inside factory. But instantiation step definitely not independent in this case. It mast occur prior to other steps.
So, you can provide Filters (i.e. Pipeline) to setup your product:
public class BarFilter : IFilter
{
private IFilter _next;
public IProduct Setup(IProduct product)
{
// do Bar setup
if (_next == null)
return product;
return _next.Setup(product);
}
}
public abstract class ProductFactory : IProductFactory
{
protected IFilter _filter;
public IProduct CreateProduct()
{
IProduct product = InstantiateProduct();
if (_filter == null)
return product;
return _filter.Setup(product);
}
protected abstract IProduct InstantiateProduct();
}
And in concrete factories you can setup custom set of filters for your setup pipeline.
Factory is responsible for creating objects:
ICar volvo = CarFactory.BuildVolvo();
ICar bmw = CarFactory.BuildBMW();
IBook pdfBook = BookFactory.CreatePDFBook();
IBook htmlBook = BookFactory.CreateHTMLBook();
Pipeline will help you to separate processing into smaller tasks:
var searchQuery = new SearchQuery();
searchQuery.FilterByCategories(categoryCriteria);
searchQuery.FilterByDate(dateCriteria);
searchQuery.FilterByAuthor(authorCriteria);
There is also a linear pipeline and non-linear pipeline. Linear pipeline would require us to filter by category, then by date and then by author. Non-linear pipeline would allow us to run these simultaneously or in any order.
This article explains it quite well:
http://www.cise.ufl.edu/research/ParallelPatterns/PatternLanguage/AlgorithmStructure/Pipeline.htm