EF custom select with entity property as parameters - entity-framework

For every table (and I have a lot of them) I provide Lookup REST API method in my ASP.Net Core application.
This is part of my query for every table:
context.Users.Select(t => new ViewLookupModel()
{
id = t.Id,
title = t.DisplayName
})
//......
context.Groups.Select(t => new ViewLookupModel()
{
id = t.Id,
title = t.Name
})
But I want to write extension to shrink code I need to write for every table. Something like this:
public static IQueryable<ViewLookupModel> SelectLookup<T>(this IQueryable<T> query, Func<int> idField, Func<string> titleField)
{
return query.Select(t => new ViewLookupModel()
{
id = idField(),
title = titleField()
});
}
And use case:
context.Users.SelectLookup(t => t.Id, t => t.DisplayName)
//......
context.Groups.SelectLookup(t => t.Id, t => t.Title)
But I get error:
Delegate 'Func' does not take 1 arguments.
This and this question seems similar, but I can not get it to work.
I am also interesting in any performance issues when querying database with custom SELECT extension method.

Change your extension method to this and try. Extension method takes T as input and returns the corresponding int or string etc.
public static IQueryable<ViewLookupModel> SelectLookup<T>(this IQueryable<T> query, Func<T,int> idField, Func<T,string> titleField)
{
return query.Select(t => new ViewLookupModel()
{
id = idField(t),
title = titleField(t)
});
}

Related

How to use data from entity Raw SQL query in entity framework

I will state up front I am still very new to asp.net core and entity framework but I am familiar with ADO and recordset which it seems entity framework is built from. I am struggling to use entity framework because I am able to run the query but I am not sure how to use the results and most help docs I see online only discuss the procedures and methods and not how to use the results. I am building login functionality on my site and have developed the following code to query DB in my UserAccount table. FOr this login I really only want the Username, Password, and the ID however I have multiple fields in this table that I dont need for this aspect. I come from using ADO and recordsets so really I just prefer to use raw sql and determine what i want to do with those results and not bind them to some objects which it seems entity framework does for the most part. I do liek the ease of use of the entity framework and using parameters though so I prefer to just go this route (not sure if this is bad practice or not). Everything works great except when I add ToList it starts to say error "InvalidOperationException: The required column 'AccessLevel' was not present in the results of a 'FromSql' operation." I am not even tryign to use that field in the query so not sure why that is even coming up and I am using a rawsql query method so why is it trying to see what I have in the model for that table? Lastly, this shoudl return a single record in which case I want to pull the password value from that record field similar to how i use to do in ADO as you will see in my passwordhash verification but I cannot figure out how to even pull that from the query result. Thanks for any help on this as I am getting a headache trying to learn this stuff!!
var UserResults = _context.UserAccounts.FromSqlInterpolated($"SELECT USERACCOUNT.USERNAME, USERACCOUNT.PASSWORD, USERACCOUNT.ID,USERACCOUNT.ACCESS_LEVEL FROM DBO.USERACCOUNT WHERE USERACCOUNT.USERNAME={UserName}");
if (UserResults.Count() == 1) //if we have more than 1 result we have security issue so do not allow login
{
var passwordHasher = new PasswordHasher<string>();
var hashedPassword = passwordHasher.HashPassword(null, Password);
var testResults = UserResults.ToList();
if (passwordHasher.VerifyHashedPassword(null, hashedPassword, THIS SHOULD BE VALUE FROM DB RIGHT HERE!!!) == PasswordVerificationResult.Success)
{
var claims = new List<Claim>
{
new Claim(ClaimTypes.Name, UserName)
};
My DBContext is as follows:
public partial class LoginDBContext : DbContext
{
public DbSet<UserAccount> UserAccounts { get; set; }
public LoginDBContext(DbContextOptions<LoginDBContext> options) : base(options)
{
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<UserAccount>(entity =>
{
//entity.ToTable("UserAccount");
entity.HasNoKey();
//entity.Property(e => e.Id)
// .HasMaxLength(50)
// .IsUnicode(false);
//entity.Property(e => e.Username)
// .HasMaxLength(50)
// .IsUnicode(false);
//entity.Property(e => e.Password)
// .HasMaxLength(50)
// .IsUnicode(false);
});
}
}
If you don't want to load all columns of the user table you can return anonymous object with properties you need or create a class for the columns you need and return it.
var users = _context.UserAccounts
.Where(a => a.UserName == UserName)
.Select(a => new { a.Id, a.UserName, a.Password })
.ToArray();
if (users.Length == 1)
{
var user = users.First();
if (passwordHasher.VerifyHashedPassword(null, hashedPassword, user.Password) == PasswordVerificationResult.Success)
{
// ... your magic
}
}

What is the difference between creating a new object inside select LINQ clause and inside a method

I have an entity class which is mapped to an SQL table:
public class EntityItem {
public virtual ICollection<EntityItem2> SomeItems { get; set; }
}
And I have the following two snippets:
var items = _repository.Table.Where(x => x.Id == id)
.Select(x => new ItemModel {
Items = x.SomeItems.Select(y => new SomeItem { //mapping is here...}).ToList()
});
And
var items = _repository.Table.Where(x => x.Id == id).Select(x => someModelMapper.BuildModel(x));
//inside a mapper
public ItemModel BuildModel(EntityType entity){
var model = new ItemModel();
model.Items = entity.SomeItems.Select(x => anotherMapper.BuildModel(x));
return model;
}
As a result, I am getting different SQL queries in both cases. Moreover, the second snippet is working much slower than the first one. As I can see in SQL profiler the second snipper is generating many SQL queries.
So my questions:
Why is that happening?
How to create new objects like in the second snippet but to avoid
lots of SQL queries?
The likely reason you are seeing a difference in performance is due to EF Core materializing the query prematurely. When a Linq statement is compiled, EF attempts to translate it into SQL. If you call a function within the expression, EF6 would have raised an exception to the effect that the method cannot be converted to SQL. EF Core tries to be clever, and when it encounters a method it cannot convert, it executes the query up to the point it could get to, and then continues to execute the rest as Linq2Object where you method can run. IMO this is a pretty stupid feature and represents a huge performance landmine, and while it's fine to offer it as a possible option, it should be disabled by default.
You're probably seeing extra queries due to lazy loading after the main query runs, to populate the view models in the mapping method.
For instance if I execute:
var results = context.Parents.Select(x => new ParentViewModel
{
ParentId = x.ParentId,
Name = x.Name,
OldestChildName = x.Children.OrderByDescending(c => c.BirthDate).Select(c => c.Name).FirstOrDefault() ?? "No Child"
}).Single(x => x.ParentId == parentId);
That would execute as one statement. Calling a method to populate the view model:
var results = context.Parents
.Select(x => buildParentViewModel(x))
.Single(x => x.ParentId == parentId);
would execute something like:
var results = context.Parents
.ToList()
.Select(x => new ParentViewModel
{
ParentId = x.ParentId,
Name = x.Name,
OldestChildName = x.Children.OrderByDescending(c => c.BirthDate).Select(c =>
c.Name).FirstOrDefault() ?? "No Child"
}).Single(x => x.ParentId == parentId);
at worst or:
var results = context.Parents
.Where(x => x.ParentId == parentId)
.ToList()
.Select(x => new ParentViewModel
{
ParentId = x.ParentId,
Name = x.Name,
OldestChildName = x.Children.OrderByDescending(c => c.BirthDate).Select(c =>
c.Name).FirstOrDefault() ?? "No Child"
}).Single();
... at best. These are due to the Extra .ToList() call prior to the Select which is roughly what the premature execution will do automatically. The issue with these queries compared to the first one is that when it comes to loading the child's name. In the first query, the SQL generated pulls the parent and related child's details in one query. In the alternative cases the query will execute to pull the parent's details, but getting the child details will constitute a lazy load call to get more details since that will be executed as Linq2Object.
The solution would be to use Automapper and it's built in ProjectTo method to populate your view model. This will place the mapping code in automatically so that it works like the first scenario without you needing to write out all of the mapping code.

Custom extension function in select statment of an EF query

In our database we have a table that looks like this which we have mapped to an entity in our Database-First EF model:
CREATE TABLE Texts(
Id integer,
Eng nvarchar(max),
Nob nvarchar(max),
...
)
A row in this table may be quite large, so we only want to get the value of the column that is currently need by a language selection the user has done.
My idea was to have an extension function to do it for me, but I dont have any idea nor can't find any way to write it (if it is even possible). I have tried a few variants, but (obviously) it failed with an exception that states that it cannot be translated into a store expression. So I am a bit stuck.
The idea of usage for this function is:
context.Codes.Where(row => row.Id == 234).Select(row => new {
row.Id,
Text = Text.GetLocalizedText("Eng") // This should generate an SQL that only retrieves the Eng
// column of the Text navigation property (which is
// connected to the Texts table.
});
That should generate a select similar to this (which are similar to the example above except using Text.Eng directly):
SELECT
[Extent1].[Id] AS [Id],
[Extent2].[Eng] AS [Eng]
FROM [dbo].[Codes] AS [Extent1]
INNER JOIN [dbo].[Texts] AS [Extent2] ON [Extent1].[TextId] = [Extent2].[Id]
WHERE 234 = [Extent1].[Id]
Does anyone know if this is possible, and if it is; how to write it? If it isn't possible, does anyone have any other idea on how to solve this without retrieving the whole Text entity with all of it's columns?
An extension method of IQueryable<Code> would work but it is not as flexible as you probably want to have it because you would need to have an extension per type of projection you want to perform and you cannot work with an anonymous result object.
The idea is basically like so:
You need a named class (instead of anonymous) which you can project into:
public class CodeData
{
public int Id { get; set; }
public string LocalizedText { get; set; }
}
And then an extension method with the language parameter:
public static class CustomExtensions
{
public static IQueryable<CodeData> SelectCodeData(
this IQueryable<Code> query, string language)
{
switch (language)
{
case "Eng":
return query.Select(code => new CodeData
{
Id = code.Id,
LocalizedText = code.Text.Eng
});
case "Nob":
return query.Select(code => new CodeData
{
Id = code.Id,
LocalizedText = code.Text.Nob
});
//... more languages
}
throw new ArgumentException("Invalid language code.", "language");
}
}
Then it can be called like this:
using CustomExtensions;
// ...
IQueryable<CodeData> codeDataQuery = context.Codes
.Where(row => row.Id == 234)
.SelectCodeData("Eng");

DbSet<T>.Include() causes SELECT N+1 when used in extension method

I have an extension on IQueryable that allows passing in delimited string of property names which, when used causes query not to construct JOINs and effectively causes SELECT N+1 issue.
What I noticed is that if I call native EF extension .Include("property") directly off of DbSet everything works fine. But if I use my extension (I even simplified it to just call .Include("property") SELECT N+1 occurs...
My questions is why? What am I doing wrong?
Here is calling method (from service)
public MyModel[] GetAll(int page, out int total, int pageSize, string sort, string filter)
{
return _myModelRepository
.Get(page, out total, pageSize, sort, filter, "PropertyOnMyModelToInclude")
.ToArray();
}
Here is the repository method that uses extension
public virtual IQueryable<T> Get(int page, out int total, int pageSize, string sort, string filter = null, string includes = null)
{
IQueryable<T> query = DatabaseSet;
if (!String.IsNullOrWhiteSpace(includes))
{
//query.IncludeMany(includes); // BAD: SELECT N+1
//query.Include(includes); // BAD: SELECT N+1
}
if (!String.IsNullOrWhiteSpace(filter))
{
query.Where(filter);
}
total = query.Count(); // needed for pagination
var order = String.IsNullOrWhiteSpace(sort) ? DefaultOrderBy : sort;
var perPage = pageSize < 1 ? DefaultPageSize : pageSize;
//return query.OrderBy(order).Paginate(page, total, perPage); // BAD: SELECT N+1 (in both variations above)
//return query.IncludeMany(includes).OrderBy(order).Paginate(page, total, perPage); // BAD: SELECT N+1
return query.Include(includes).OrderBy(order).Paginate(page, total, perPage); // WORKS!
}
Here is the extension (reduced just to call Include() to illustrate the issue)
public static IQueryable<T> IncludeMany<T>(this IQueryable<T> query, string includes, char delimiter = ',') where T : class
{
// OPTION 1
//var propertiesToInclude = String.IsNullOrWhiteSpace(includes)
// ? new string[0]
// : includes.Split(new[] {delimiter}, StringSplitOptions.RemoveEmptyEntries).Select(p => p.Trim()).ToArray();
//foreach (var includeProperty in propertiesToInclude)
//{
// query.Include(includeProperty);
//}
// OPTION 2
//if (!String.IsNullOrWhiteSpace(includes))
//{
// var propertiesToInclude = includes.Split(new[] { delimiter }, StringSplitOptions.RemoveEmptyEntries).AsEnumerable(); //.Select(p => p.Trim());
// propertiesToInclude.Aggregate(query, (current, include) => current.Include(include));
//}
// OPTION 3 - for testing
query.Include(includes);
return query;
}
I think the fundamental problem here is in the way you are using the Include method, and also incidentally, the Where method. These methods, as is typical with LINQ extension methods, do not modify the object that they are called on. Instead they return a new object which represents the query after the operator has been applied. So, for example, in this code:
var query = SomeQuery();
query.Include(q => q.Bing);
return query;
the Include method basically does nothing because the new query returned by Include is thrown away. On the other hand, this:
var query = SomeQuery();
query = query.Include(q => q.Bing);
return query;
applies the Include to the query and then updates the query variable with the new query object returned from Include.
It's not in the code you have posted, but I think you are still seeing N+1 with your code because the Include is being ignored and the related collections are therefore still being loaded using lazy loading.

Usage of "Expression<Func<Object, bool>>"

please take a look at the following snippet:
public IEnumerable<T> Query(Expression<Func<T, bool>> filter)
{
return filter != null ? this.ObjectSet.Where(filter) : null;
}
As for example:
public IQueryable<Department> Query(Expression<Func<Department, bool>> filter)
{
return _context.Departments.Where(filter);
}
(original source: http://blogs.microsoft.co.il/blogs/gilf/archive/2010/06/21/revisiting-the-repository-and-unit-of-work-patterns-with-entity-framework.aspx)
Can anyone provide some concrete examples of how to go about actually calling the second method with an actual query?
Thanks!
To say get all departments with Sales over $100
var deparments = deparmentsRepository.Query(d => d.TotalSales > 100.0m);
Func and Expression> can both be declared the same way using lambda syntax.
In fact there's an example in the article (in the last code snippet) you linked to already for this.
foreach(var department in uow.Departments.Query(d => d.Budget > 150000)) { ... }
Given that Query returns IQueryable there's no reason you can't build into more complex queries.
var deparments = deparmentsRepository.Query(d => d.TotalSales > 100.0m)
.Select(d => new { ... });
Assuming Manager and Department have a foreign key you don't need that join.
var deparments = deparmentsRepository.Query(d => d.Manager.Name.StartsWith("A"));