How to convert a method of a ValueObject to SQL with Entity Framework Core 3.1 - ef-core-3.1

I have this value object
public class ProductReference : ValueObject
{
protected ProductReference(){}
public ProductReference(string value){}
public string Value{get; protected set;}
}
I use it in my entity as :
public class Product : Entity<long>
{
protected Product(){}
public ProductReference Reference{get; protected set;}
}
In the OnModelCreating of my DbContext I defined :
modelBuilder.Entity<Product>(entity => {
entity.Property(a => a.Reference)
.HasColumnName("Reference")
.HasConversion(
a => a.Value,
s => new ProductReference (s);
});
When I do :
await dbcontext.Products.Where(p=>p.Reference.Value.Contains("some text")).toArrayAsync();
I get an exception
Expression cannot be converted to a valid SQL statement
I know for sure there is a way to create a custom expression converter, but I cannot find a good, simple and EF Core 3.1 compatible example to deal with my issue and that explain me clearly the concepts I miss.
I found this very interesting project
https://github.com/StevenRasmussen/EFCore.SqlServer.NodaTime
but it is too advanced for me to reproduce it for only my use case.
[EDIT] the ValueObject ans Entity are from
CSharpFunctionalExtensions nuget package, I dont think they are really relevant in my question.

I am not completely sure if i understand correctly what you want to accomplish, but you could try to configure your ProductReference as an Owned Entity Type.
Here you would transform the following code from:
modelBuilder.Entity<Product>(entity => {
entity.Property(a => a.Reference)
.HasColumnName("Reference")
.HasConversion(
a => a.Value,
s => new ProductReference (s);
});
to
modelBuilder.Entity<Product>(entity => {
entity.OwnsOne(a => a.Reference, referenceBuilder => {
referenceBuilder.Property(p => p.Value).HasColumnName("Reference");
});
});
With that your select statement should work.
It could be that you have to play around with the properties of your class ProductReference, or use some modifiers of the fluent API.

So first for some context on what is happening here behind the scenes and why its not gonna work even for build in simple converters like BoolToZeroOneConverter.
The problem here is that you are calling when converting the new ProductReference(s). This is method where you can do whatever you want in it. For example if use it in a Select statement it will again fail. For example:
await dbcontext.Products
.Select(x=>new ProductReference(x.Value))
.toArrayAsync();
The reason is obvious, it won't be able to translate. But why it cant transform it to a query?
Because you are passing a constructor. Inside this constructor you could be doing API calls or using Reflections to set the variables to your object, pretty much anything. That of course is not able to be translated in an SQL query.
Converters are generally used for in memory but they can be used for databse operations as well. This would mean that you will need something like this:
await dbcontext.Products
.Select(x=>new ProductReference() // empty constructor that does nothing
{
Property1 = x.Property1 // I don't know how the constructor maps them
})
.toArrayAsync();
Using this type of expression allow you to actually transalte the expression to an SQL statement and not making the conversion on the SQL DB and not in memory.
Now in your specific case using:
.HasConversion(
a => a.Value,
s => new ProductReference (){};
});
Should fix your issues but I fail to understand why would you want to initialize or convert a ProductReference to a ProductReference.

Related

EF Core and PostgreSQL enum two-way name translation

I am trying to figure out how to enable two-way name translation between postgres DB and EF Core. I have found a link where it is told how values are translated from EF Core to DB, but nothing about from DB to EF Core. The problem is that when I read from DB I get values in snake case, but I need them in pascal case.
How are you reading your data from the DB?
Providing you are writing from and reading into the C# enum type that corresponds to the postgres enum type, Npgsql takes care of translating the values both ways.
For example:
Define your types:
public enum SomeDbEnum
{
FirstValue,
SecondValue
}
public class SomeDbObject
{
public SomeDbEnum DbEnum { get; set; }
}
Map your enum types
protected override void OnModelCreating(ModelBuilder builder)
=> builder.HasPostgresEnum<SomeDbEnum>();
static MyDbContext()
=> NpgsqlConnection.GlobalTypeMapper.MapEnum<SomeDbEnum>();
Try it out:
var toWriteToDb = new SomeDbObject { DbEnum = SomeDbEnum.SecondValue };
context.SomeDbObject.Add(toWriteToDb);
// value inserted as "second_value"
var readFromDb = context.SomeDbObject.FirstOrDefault(x => x.DbEnum == SomeDbEnum.SecondValue);
Console.WriteLine(readFromDb.DbEnum);
// value output as SecondValue
The same thing applies even if you use something other than the default naming translations, i.e. registered some other INpgsqlNameTranslator (other than the default NpgsqlSnakeCaseNameTranslator) or decorate your enum members with the PgNameAttribute.
Had to research myself, just use the NpgsqlNullNameTranslator. For some reason NpgSql uses NpgsqlSnakeCaseNameTranslator as default, at least for EF Core 7.0
modelBuilder.HasPostgresEnum<MyEnum>(nameTranslator: new NpgsqlNullNameTranslator());
EDIT: in order to save them as string in DB:
modelBuilder.Entity<MyEntity>()
.Property(u => u.EnumProperty)
.HasConversion(new EnumToStringConverter<MyEnum>());

Entity Framework Core - Has Conversion - Support Null Values

I have an EF model with a notification emails property. The notification emails are saved in the database as string separated by ';'. I added a conversion to retrieve the data as a ICollection in the model. This is working well except one thing: when the string is null the collection is also null, and I want to convert it to an empty collection instead. is it possible?
//This is my code
entity.Property(e => e.NotificationEmails)
.HasConversion(
v => string.Join(",", v.Select(s => s.Trim())),
v => v.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries));
I tried to add String.IsNullOrEmpty(v) but EF ignores it.
Currently, it isn't possible :
https://learn.microsoft.com/en-us/ef/core/modeling/value-conversions#configuring-a-value-converter
A null value will never be passed to a value converter. This makes the implementation of conversions easier and allows them to be shared amongst nullable and non-nullable properties.
It isn't elegant, but you can use a backing field :
public class Notification
{
private List<string> _emails = new List<string>();
public List<string> Emails
{
get => _emails;
set => _emails = value ?? new List<string>();
}
}
public class NotificationContext : DbContext
{
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Entity<Notification>().Property(d => d.Emails).HasConversion(
v => string.Join(",", v.Select(s => s.Trim())),
v => v.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries).ToList()
);
modelBuilder.Entity<Notification>()
.Property(b => b.Emails)
.HasField("_emails")
.UsePropertyAccessMode(PropertyAccessMode.Property);
}
}
Note : in where, a empty list will not be translated by null, but by a empty string.
Edit : This feature is available from EF Core 6, but bugged.
See this comment :
For anyone watching this issue: there are significant problems when executing queries that either convert nulls in the database to non-nulls in code or vice-versa. Therefore, we have marked this feature as internal for EF Core 6.0. You can still use it, but you will get a compiler warning. The warning can be disabled with a #pragma.

How to Mock an Entity Framework 6 Async Projecting Query

By leveraging the Testing with async queries section of the Testing with a Mocking Framework article on MSDN, I've been able to create many successfully passing tests.
Here's my test code, which uses NSubstitute for mocks:
var dummyQueryable = locations.AsQueryable();
var mock = Substitute.For<DbSet<Location>, IDbAsyncEnumerable<Location>, IQueryable<Location>>();
((IDbAsyncEnumerable<Location>)mock).GetAsyncEnumerator().Returns(new TestDbAsyncEnumerator<Location>(dummyQueryable.GetEnumerator()));
((IQueryable<Location>)mock).Provider.Returns(new TestDbAsyncQueryProvider<Location>(dummyQueryable.Provider));
((IQueryable<Location>)mock).Expression.Returns(dummyQueryable.Expression);
((IQueryable<Location>)mock).ElementType.Returns(dummyQueryable.ElementType);
((IQueryable<Location>)mock).GetEnumerator().Returns(dummyQueryable.GetEnumerator());
sut.DataContext.Locations = mock;
var result = await sut.Index();
result.Should().BeView();
sut.Index() doesn't do much, but it makes the following query:
await DataContext.Locations
.GroupBy(l => l.Area)
.ToListAsync());
This works fine until I add a projection into the query:
await DataContext.Locations
.GroupBy(l => l.Area)
.Select(l => new LocationsIndexVM{ Area = l.Key }) // added projection
.ToListAsync());
which results in this exception:
System.InvalidOperationException
The source IQueryable doesn't implement IDbAsyncEnumerable<LocationsIndexVM>. Only sources that implement IDbAsyncEnumerable can be used for Entity Framework asynchronous operations. For more details see http://go.microsoft.com/fwlink/?LinkId=287068.
at System.Data.Entity.QueryableExtensions.AsDbAsyncEnumerable(IQueryable`1 source)
at System.Data.Entity.QueryableExtensions.ToListAsync(IQueryable`1 source)
at Example.Web.Controllers.HomeController.<Index>d__0.MoveNext() in HomeController.cs: line 25
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at Example.Test.Web.Controllers.HomeControllerShould.<TempTest>d__4.MoveNext() in HomeControllerShould.cs: line 71
UPDATE: I've uploaded a small, simple solution that reproduces this problem.
Can anyone provide an example of what is required to unit test a query that is both async and contains a .Select() projection?
So I did a bit of digging, and the issue is to do with the way the TestDbAsyncEnumerable<T> exposes the IQueryProvider. My best guess as to the reasoning is below, and the solution below that.
TestDbAsyncEnumerable<T> inherits from EnumerableQuery<T>, which in turn inherits from IQueryable<T>, and explicitly implements the Provider property of this interface:
IQueryProvider IQueryable.Provider { get ... }
Given that it's implemented explicitly, I am assuming that the LINQ internals explicitly cast a type before trying to get the Provider:
((IQueryable<T>)source).Provider.CreateQuery(...);
I don't have a source on hand (and can't be bothered looking for one), but I believe the type binding rules are different for explicit implementations; essentially, the Provider property on your TestDbAsyncEnumerable<T> is not considered to be an implementation of IQueryable<T>.Provider as an explicit one exists further up the chain, so your TestDbAsyncQueryProvider<T> is never returned.
The fix for this is to make TestDbAsyncEnumerable<T> also inherit IQueryable<T> and explicitly implement the Provider property, as below (adjusted from the MSDN article you linked):
public class TestDbAsyncEnumerable<T> : EnumerableQuery<T>, IDbAsyncEnumerable<T>, IQueryable<T>
{
public TestDbAsyncEnumerable(IEnumerable<T> enumerable) : base(enumerable)
{ }
public TestDbAsyncEnumerable(Expression expression) : base(expression)
{ }
public IDbAsyncEnumerator<T> GetAsyncEnumerator()
{
return new TestDbAsyncEnumerator<T>(this.AsEnumerable().GetEnumerator());
}
IDbAsyncEnumerator IDbAsyncEnumerable.GetAsyncEnumerator()
{
return GetAsyncEnumerator();
}
IQueryProvider IQueryable.Provider
{
get { return new TestDbAsyncQueryProvider<T>(this); }
}
}

How to mock the limitations of EntityFramework's implementation of IQueryable

I am currently writing unit tests for my repository implementation in an MVC4 application. In order to mock the data context, I started by adopting some ideas from this post, but I have now discovered some limitations that make me question whether it is even possible to properly mock IQueryable.
In particular, I have seen some situations where the tests pass but the code fails in production and I have not been able to find any way to mock the behavior that causes this failure.
For example, the following snippet is used to select Post entities that fall within a predefined list of categories:
var posts = repository.GetEntities<Post>(); // Returns IQueryable<Post>
var categories = GetCategoriesInGroup("Post"); // Returns a fixed list of type Category
var filtered = posts.Where(p => categories.Any(c => c.Name == p.Category)).ToList();
In my test environment, I have tried mocking posts using the fake DbSet implementation mentioned above, and also by creating a List of Post instances and converting it to IQueryable using the AsQueryable() extension method. Both of these approaches work under test conditions, but the code actually fails in production, with the following exception:
System.NotSupportedException : Unable to create a constant value of type 'Category'. Only primitive types or enumeration types are supported in this context.
Although LINQ issues like this are easy enough to fix, the real challenge is finding them, given that they do not reveal themselves in the test environment.
Am I being unrealistic in expecting that I can mock the behavior of Entity Framework's implementation of IQueryable?
Thanks for your ideas,
Tim.
I think it is very very hard, if impossible, to mock Entity Framework behaviour. First and foremost because it would require profound knowledge of all peculiarities and edge cases where linq-to-entites differs from linq-to-objects. As you say: the real challenge is finding them. Let me point out three main areas without claiming to be even nearly exhaustive:
Cases where Linq-to-Objects succeeds and Linq-to-Entities fails:
.Select(x => x.Property1.ToString(). LINQ to Entities does not recognize the method 'System.String ToString()' method... This applies to nearly all methods in native .Net classes and of course to own methods. Only a few .Net methods will be translated into SQL. See CLR Method to Canonical Function Mapping. As of EF 6.1, ToString is supported by the way. But only the parameterless overload.
Skip() without preceding OrderBy.
Except and Intersect: can produce monstrous queries that throw Some part of your SQL statement is nested too deeply. Rewrite the query or break it up into smaller queries.
Select(x => x.Date1 - x.Date2): DbArithmeticExpression arguments must have a numeric common type.
(your case) .Where(p => p.Category == category): Only primitive types or enumeration types are supported in this context.
Nodes.Where(n => n.ParentNodes.First().Id == 1): The method 'First' can only be used as a final query operation.
context.Nodes.Last(): LINQ to Entities does not recognize the method '...Last...'. This applies to many other IQueryable extension methods. See Supported and Unsupported LINQ Methods.
(See Slauma's comment below): .Select(x => new A { Property1 = (x.BoolProperty ? new B { BProp1 = x.Prop1, BProp2 = x.Prop2 } : new B { BProp1 = x.Prop1 }) }): The type 'B' appears in two structurally incompatible initializations within a single LINQ to Entities query... from here.
context.Entities.Cast<IEntity>(): Unable to cast the type 'Entity' to type 'IEntity'. LINQ to Entities only supports casting EDM primitive or enumeration types.
.Select(p => p.Category?.Name). Using null propagation in an expression throws CS8072 An expression tree lambda may not contain a null propagating operator. This may get fixed one day.
This question: Why does this combination of Select, Where and GroupBy cause an exception? made me aware of the fact that there are even entire query constructions that are not supported by EF, while L2O wouldn't have any trouble with them.
Cases where Linq-to-Objects fails and Linq-to-Entities succeeds:
.Select(p => p.Category.Name): when p.Category is null L2E returns null, but L2O throws Object reference not set to an instance of an object. This can't be fixed by using null propagation (see above).
Nodes.Max(n => n.ParentId.Value) with some null values for n.ParentId. L2E returns a max value, L2O throws Nullable object must have a value.
Using EntityFunctions (DbFunctions as of EF 6) or SqlFunctions.
Cases where both succeed/fail but behave differently:
Nodes.Include("ParentNodes"): L2O has no implementation of include. It will run and return nodes (if Nodes is IQueryable), but without parent nodes.
Nodes.Select(n => n.ParentNodes.Max(p => p.Id)) with some empty ParentNodes collections: both fail but with different exceptions.
Nodes.Where(n => n.Name.Contains("par")): L2O is case sensitive, L2E depends on the database collation (often not case sensitive).
node.ParentNode = parentNode: with a bidirectional relationship, in L2E this will also add the node to the nodes collection of the parent (relationship fixup). Not in L2O. (See Unit testing a two way EF relationship).
Work-around for failing null propagation: .Select(p => p.Category == null ? string.Empty : p.Category.Name): the result is the same, but the generated SQL query also contains the null check and may be harder to optimize.
Nodes.AsNoTracking().Select(n => n.ParentNode. This one is very tricky!. With AsNoTracking EF creates new ParentNode objects for each Node, so there can be duplicates. Without AsNoTracking EF reuses existing ParentNodes, because now the entity state manager and entity keys are involved. AsNoTracking() can be called in L2O, but it doesn't do anything, so there will never be a difference with or without it.
And what about mocking lazy/eager loading and the effect of context life cycle on lazy loading exceptions? Or the effect of some query constructs on performance (like constructs that trigger N+1 SQL queries). Or exceptions due to duplicate or missing entity keys? Or relationship fixup?
My opinion: nobody is going to fake that. The most alarming area is where L2O succeeds and L2E fails. Now what's the value of green unit tests? It has been said before that EF can only reliably be tested in integration tests (e.g. here) and I tend to agree.
However, that does not mean that we should forget about unit tests in projects with EF as data layer. There are ways to do it, but, I think, not without integration tests.
I have written a few Unit Tests with Entity Framework 6.1.3 using Moq and used it to override IQueryable. Note that all DbSet that should be tested needs to be marked as virtual. Example from Microsoft themselves:
Query:
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Moq;
using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
namespace TestingDemo
{
[TestClass]
public class QueryTests
{
[TestMethod]
public void GetAllBlogs_orders_by_name()
{
var data = new List<Blog>
{
new Blog { Name = "BBB" },
new Blog { Name = "ZZZ" },
new Blog { Name = "AAA" },
}.AsQueryable();
var mockSet = new Mock<DbSet<Blog>>();
mockSet.As<IQueryable<Blog>>().Setup(m => m.Provider).Returns(data.Provider);
mockSet.As<IQueryable<Blog>>().Setup(m => m.Expression).Returns(data.Expression);
mockSet.As<IQueryable<Blog>>().Setup(m => m.ElementType).Returns(data.ElementType);
mockSet.As<IQueryable<Blog>>().Setup(m => m.GetEnumerator()).Returns(0 => data.GetEnumerator());
var mockContext = new Mock<BloggingContext>();
mockContext.Setup(c => c.Blogs).Returns(mockSet.Object);
var service = new BlogService(mockContext.Object);
var blogs = service.GetAllBlogs();
Assert.AreEqual(3, blogs.Count);
Assert.AreEqual("AAA", blogs[0].Name);
Assert.AreEqual("BBB", blogs[1].Name);
Assert.AreEqual("ZZZ", blogs[2].Name);
}
}
}
Insert:
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Moq;
using System.Data.Entity;
namespace TestingDemo
{
[TestClass]
public class NonQueryTests
{
[TestMethod]
public void CreateBlog_saves_a_blog_via_context()
{
var mockSet = new Mock<DbSet<Blog>>();
var mockContext = new Mock<BloggingContext>();
mockContext.Setup(m => m.Blogs).Returns(mockSet.Object);
var service = new BlogService(mockContext.Object);
service.AddBlog("ADO.NET Blog", "http://blogs.msdn.com/adonet");
mockSet.Verify(m => m.Add(It.IsAny<Blog>()), Times.Once());
mockContext.Verify(m => m.SaveChanges(), Times.Once());
}
}
}
Example service:
using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
using System.Threading.Tasks;
namespace TestingDemo
{
public class BlogService
{
private BloggingContext _context;
public BlogService(BloggingContext context)
{
_context = context;
}
public Blog AddBlog(string name, string url)
{
var blog = _context.Blogs.Add(new Blog { Name = name, Url = url });
_context.SaveChanges();
return blog;
}
public List<Blog> GetAllBlogs()
{
var query = from b in _context.Blogs
orderby b.Name
select b;
return query.ToList();
}
public async Task<List<Blog>> GetAllBlogsAsync()
{
var query = from b in _context.Blogs
orderby b.Name
select b;
return await query.ToListAsync();
}
}
}
Source: https://learn.microsoft.com/en-us/ef/ef6/fundamentals/testing/mocking

autofac registration issue in release v2.4.5.724

I have the following registration
builder.Register<Func<Type, IRequestHandler>>(
c => request => (IRequestHandler)c.Resolve(request));
Basically I am trying to register a factory method that resolves an instance of IRequestHandler from a given type.
This works fine until the version 2.4.3.700. But now I am getting a the following error..
Cannot access a disposed object.
Object name: 'This resolve operation has already ended. When
registering components using lambdas,
the IComponentContext 'c' parameter to
the lambda cannot be stored. Instead,
either resolve IComponentContext again
from 'c', or resolve a Func<> based
factory to create subsequent
components from.'.
UPDATE
I was trying to limit autofac's exposure to the rest of the projects in the solution. Nick, thanks for the hint, now my registration looks like this...
builder.Register<Func<Type,IRequestHandler>>(c =>
{
var handlers = c.Resolve<IIndex<Type,RequestHandler>>();
return request => handlers[request];
});
The c in this expression is a temporary, so this code while previously functional, is broken. Autofac 2.4.5 detects this problem while earlier versions silently ignored it.
To fix the issue, explicitly resolve IComponentContext:
builder.Register<Func<Type, IRequestHandler>>(c => {
var ctx = c.Resolve<IComponentContext>();
return request => (IRequestHandler)ctx.Resolve(request));
});
The functionality you're emulating here might be better represented using keys and indexes, e.g. see Interrupted chain of IoC or http://code.google.com/p/autofac/wiki/TypedNamedAndKeyedServices.
I had a similar problem as the user6130. I wanted to avoid using IIndex in my class implementation and pass in a service resolver into my constructor instead.
So now I have my service implementation with the following constructor:
public MvcMailer(Converter<string, MailerBase> mailerResolver)
{
_resolver = mailerResolver;
}
I wanted to used keyed services without directly relying on the Autofac namespace. I was getting the same error until I restructured the configuration as such.
1) Scan for all my mailer implementations and index via class name (could be improved)
builder.RegisterAssemblyTypes(System.Reflection.Assembly.GetExecutingAssembly())
.Where(t => t.Name.EndsWith("Mailer")).Keyed<Mvc.Mailer.MailerBase>(t => t.Name.Replace("Mailer", "").ToLower());
2) Register the converter in Autofac config
builder.Register<System.Converter<string,Mvc.Mailer.MailerBase>>(c => {
var all = c.Resolve<Autofac.Features.Indexed.IIndex<string,Mvc.Mailer.MailerBase>>();
return delegate(string key)
{
return all[key];
};
});
3) Register like other types of components and let Autofac handle the Converter injection
builder.RegisterType<Mailers.MvcMailer>().As<Mailers.IMailer>();