IQueryable and Mixing Lazy Loading and Eager Loading - entity-framework

I have a situation where I will be using a repository pattern and pulling objects from the database with a lazy loaded GetAll method that returns IQueryable. However I also need to build dynamic objects that will be included with the lazy loaded objects(query).
Is it possible to add built objects to a lazy loaded IQueryable and still keep the lazy loaded benefits? For instance
public override IQueryable<Foo> GetAll()
{
return _entities; // lazy loaded
}
public override IQueryable<Foo> GetAllPlusDynamic()
{
var entities = GetAll();
foreach(var d in GetAllDynamic())
{
entities.Add(d); // eagerly loaded
}
return entities;
}

I am unsure if I understand you correctly but refering to your comment...
Yes, basically query the database for a set of objects and then query
another data source (in this case a service) and build a set of
objects.
... I would say that it's not possible.
An object of type IQueryable<T> used with Entity Framework (LINQ to Entities) is basically a description of a query which the underlying data store can execute, usually an abstract description (expression tree) which gets translated into SQL.
Every part of such a query description - where expressions, select expression, Any(...) expressions, etc. - must be translatable into the native language (SQL) of the data store. It's especially not possible to include some method calls - like a service call - in an expression that the database cannot understand and perform.
IQueryable<T> knows an underlying "provider". This provider is responsible to translate the expression tree hold by the IQueryable<T> object into "something", for example T-SQL used by SQL Server, or the SQL dialects used by MySQL or Oracle. I believe it's possible to write your own provider which might then be able to perform somehow the service calls and the database queries. But I also believe that this is not an easy task.
Using the standard SQL providers for Entity Framework you have to perform database query and calling the service separately one after each other: Run query and materialize entities in memory - then run the service call on the result collection for each entity.

Related

Is a good practice to use Queryable.ProjectTo<DtoModel> inside a repository method, having more optimal sql queries instead?

As Repository pattern recommends, its methods should return Data Access Objects.
Using Queryable Extension method ProjectTo from AutoMapper(http://docs.automapper.org/en/stable/Queryable-Extensions.html) on our DbContext objects, makes the sql query simpler and shorter, but it makes our methods to return Data Transfer Objects.
DbContext.Books.ProjectTo().ToListAsync() - will query only fields contained in BookDTO(Only Id and Name for instance), but not the entire object.
So, its a good idea to make our repository to return DTO objects and having instead more optimal sql queries ?
The DbContext class is already a repository - so we just use the queryables directly on it. If you're wrapping the DbContext, then you can just expose the DbSet directly.

DbContext with dynamic DbSet

Is it possible to have a DbContext that has only one property of the generic type IDbSet and not a collection of concrete IDbSet e.g. DbSet.
More specifically, i want to create only one generic DbSet where the actual type will be determined dynamically e.g.
public new IDbSet<T> Set<T>() where T : class
{
return context.Set<T>();
}
I don't want to create multiple DbSets e.g.
DbSet<product> Products { get; set; }
...
Actually i tried to use that generic DbSet but there seems to be one problem. The DbContext doesn't create the corresponding tables in the database. So although i can work with the in-memory entity graph, when the time comes to store the entites into the DB an exception is thrown (Invalid object name 'dbo.Product'.)
Is there any way to force the EF to create tables that correspond to dynamicaly creates DbSets?
Yes you can do this.
modelBuilder.Configurations.Add
The DBSet entries will be derived.
If you plan to use POCOs and just build the model this way ok.
So you save Manual DBSet<> declaration...
But if you plan on being more Dynamic without POCOs...
Before you go down the this route, there are a number of things to consider.
Have you selected the right ORM ?
Do you plan on having a POCOs ?
Why is DbSet Products { get; set; } so bad ?
You get a lot of action for that 1 line of code.
What Data access approach you plan to use without types DBSets
Do you plan to use Linq to Entity statements?
do you plan on creating Expression trees for the Dynamic Data access necessary. Since the types arent known at compile time.
Do you plan to use the DB Model cache,?
How will the cache be managed, especially in Web. ASP environments.
There are most likely other issues i did think of off the top of my head.
Constructing the model yourself is a big task. The Linq access is compromised when compile time types/POCOs are NOT used and the model cache and performance become critical management tasks.
The practical side of this task is not to under estimate
Start here bContext.OnModelCreating
Typically, this method is called only once when the first instance of
a derived context is created. The model for that context is then
cached and is for all further instances of the context in the app
domain. This caching can be disabled by setting the ModelCaching
property on the given ModelBuidler, but this can seriously degrade
performance. More control over caching is provided through use of the
DbModelBuilder and DbContext classes directly.
The modelbuilder class
Good Luck

Spring Data JPA Fetching

Is there a way to define a Spring Data Specification (returning a JPA predicate) whose sole purpose is to perform eager fetching?
I have an entity which defines various relationships using lazy loading, but there are several queries where I want to return the entire entity representation including all related collections, but the criteria of these queries may vary. I've seen a few posts (e.g. on the spring forum) that discuss the potential introduction of fetch groups, which would likely be the ideal solution; however, since this is not yet part of the JPA spec, Spring Data does not provide support for it.
I'm looking for a reusable way to perform eager fetching on a variety of dynamic queries.
For example, I've considered developing a reusable Specification whose sole purpose is to perform the eager loading, and which could be combined with other specifications, e.g:
private static Specification<MyEntity> eager() {
return new Specification<MyEntity>() {
#Override
public Predicate toPredicate(Root<MyEntity> root, CriteriaQuery<?> query, CriteriaBuilder cb) {
for (PluralAttribute<? super MyEntity, ?, ?> fetch : root.getModel().getPluralAttributes()) {
root.fetch(fetch, JoinType.LEFT);
}
query.distinct(true);
return null;
}
};
}
The goal of this specification is to reuse it in various queries, e.g:
repository.findAll(where(eager()).and(otherCriteria).or(otherCriteria));
However, the eager fetching specification is not a true predicate, so it returns null, and would cause obvious problems (i.e. NullPointerExceptions) when chained with other predicates.
(Note that this predicate alone does work as expected; i.e. the following query will properly fetch: repository.findAll(eager());).
Is there an appropriate non-null Predicate that can be returned from the "eager" specification, or are there any other reusable approaches for triggering eager fetches using Spring Data JPA specifications (without having to tack the eager load onto another specification)?
We have improved the handling of null Specifications and Predicates in the course of fixing DATAJPA-300. You might wanna give the 1.4 snapshots a try and see how this affects your scenario.

Query against Domain Classes should resolve against Entities

I am trying to build a generic repository that allows querying against domain classes.
My Repository interface looks like the following:
public interface IRepository<T>
{
T Get(int id);
IQueryable<T> Query();
void Add(T model);
void Remove(T model);
}
Given I have an UserEntity entity framework class and a User domain class, I want to query against the User. The UserEntity should not be exposed to other services, because it should be internal to the Entity Framework layer.
A query like userRepository.Query().Single(user => user.UserName == "Toni") should return a User domain class. However internally it should query against an IDbSet<UserEntity> returned from my entity framework. The Expression Tree (which contains the Single query operation) should be attached to a query against IDbSet<UserEntity>. After querying against IDbSet<UserEntity> I want to convert the UserEntity to a User domain class. Is this possible?
I have in mind to cerate an IQueryable implementation for my User class that internally queries against UserEntity.
public class MappedEntityQuery<TModel, TEntity> : IQueryable<TModel>
{
}
Code First requires the convention to have all IDbSet properties to access the tables to be in the DbContext
That is not true. You don't need to have any set declared in the context if you provide mapping to the entities in the model builder. In your case you should declare mapping through EntityTypeConfiguration<T> and ComplexTypeConfiguration<T> derived classes. You can create any DbSet instance of mapped entity type by calling Set<T>() on the context.
However for this project I am using a Database First approach, which also does not allow to load compose a DbContext using entities from different projects, because you have to specify the Database metadata in one single metadata file (which will be embedded).
That is true only partially. EDMX metadata must be in the single file embedded in the main project but entity classes do not have to if you use your own instead of auto-generated. So your proposed approach should work.
But if you really want to achieve modularity you should not use EDMX. If you decide to add or change any module in the future it would require changing central project as well but that can affect all other modules - it breaks the idea of modularity, doesn't it?

Is there an in-memory provider for Entity Framework?

I am unit testing code written against the ADO .NET Entity Framework. I would like to populate an in-memory database with rows, and make sure that my code retrieves them properly.
I can mock the Entity Framework using Rhino Mocks, but that would not be sufficient. I would be telling the query what entities to return to me. This would neither test the where clause nor the .Include() statements. I want to be sure that my where clause matches only the rows I intend, and no others. I want to be sure that I have asked for the entities that I need, and none that I don't.
For example:
class CustomerService
{
ObjectQuery<Customer> _customerSource;
public CustomerService(ObjectQuery<Customer> customerSource)
{
_customerSource = customerSource;
}
public Customer GetCustomerById(int customerId)
{
var customers = from c in _customerSource.Include("Order")
where c.CustomerID == customerId
select c;
return customers.FirstOrDefault();
}
}
If I mock the ObjectQuery to return a known customer populated with orders, how do I know that CustomerService has the right where clause and Include? I would rather insert some customer rows and some order rows, then assert that the right customer was selected and the orders are populated.
An InMemory provider is included in EF7 (pre-release).
You can use either the NuGet package, or read about it in the EF repo on GitHub (view source).
The article http://www.codeproject.com/Articles/460175/Two-strategies-for-testing-Entity-Framework-Effort  describes Effort  -Entity Framework provider that runs in memory.
You can still use your DbContext or ObjectContext classes within unit tests, without having to have an actual database.
A better approach here might be to use the Repository pattern to encapsulate your EF code. When testing your services you can use mocks or fakes. When testing your repositories you will want to hit the real DB to ensure that you are getting the results you expect.
There is not currently a in memory provider for EF, but if you take a look at Highway.Data it has a base abstraction interface and an InMemoryDataContext.
Testing Data Access and EF with Highway.Data
Yes, there is at least one such provider - SQLite. I have used it a bit and it works. Also you can try SQL Server Compact. It's an embeded database and has EF providers too.
Edit:
SQLite has support for in-memory databases (link1). All you need is to specify a connection string like: "Data Source=:memory:;Version=3;New=True;". If you need in an example you may look at SharpArchitecture.
I am not familiar with Entity Framework and the ObjectQuery class but if the Include method is virtual you can mock it like this:
// Arrange
var customerSourceStub = MockRepository.GenerateStub<ObjectQuery<Customer>>();
var customers = new Customer[]
{
// Populate your customers as if they were coming from DB
};
customerSourceStub
.Stub(x => x.Include("Order"))
.Return(customers);
var sut = new CustomerService(customerSourceStub);
// Act
var actual = sut.GetCustomerById(5);
// Assert
Assert.IsNotNull(actual);
Assert.AreEqual(5, actual.Id);
You could try SQL Server Compact but it has some quite wild limitations:
SQL Server Compact does not support SKIP expressions in paging queries when it is used with the Entity Framework
SQL Server Compact does not support entities with server-generated keys or values when it is used with the Entity Framework
No outer joins, collate, modulo on floats, aggregates
In EF Core there are two main options for doing this:
SQLite in-memory mode allows you to write efficient tests against a provider that behaves like a relational database.
The InMemory provider is a lightweight provider that has minimal dependencies, but does not always behave like a relational database
I am using SQLite and it supports all queries, that I need to do with Azure SQL production database.