EF core paging. Select total count in same query - entity-framework-core

I have paging and i want to select count in the same query using simple sql (EF7):
var selectSql = " SELECT TotalCount = COUNT(*) OVER(), E.* FROM [table] E ...";
var rows = context.Set<EventTable>().FromSql<EventTable>(selectSql, parameters.Select(p => p.Value).ToArray()).ToArray();
This select works, but i don't have TotalCount property in my EventTable class, because i don't want that property in database.
I try get TotalCount property from entity tracker:
var row = rows.First();
var entity = context.Entry(row);
var totalCount = entity.Property<int>("TotalCount").CurrentValue;
But then i get error:
The property 'TotalCount' on entity type 'EventTable' could not be found. Ensure that the property exists and has been included in the model.
Then i try to add property in model like this:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<EventTable>(b => b.Property<int>("TotalCount"));
}
It works fine when i want to select, but it throws an exception on insert, because column in database not exist.
And EF will add that column on migration. But i notice, that if before migration generation i add line b.Property("TotalCount"); into ModelSnapshot class it will avoid to add property on migration. But problem on insert still exist.
I try to create another class:
[NotMapped]
public class EventSearchTable : EventTable
{
[Column("total_count")]
[Required]
public int TotalCount { get; set; }
}
and then do this:
var rows = context.Set<EventSearchTable>().FromSql<EventSearchTable>(..);
It works on EF6, but not on EF7, i got error: Value cannot be null.
Parameter name: entityType Because no entity in my DbContext. If i will add EventSearchTable class on my DbContext then it will expect columns like discriminator and etc and will create table in migrations.
Any ideas how to get property TotalCount ?

The following query will get the count and page results in one trip to the database
var query = context.Set<EventTable>();
var page = query.OrderBy(e => e.Id)
.Select(e => e)
.Skip(100).Take(100)
.GroupBy(e => new { Total = query.Count() })
.FirstOrDefault();
if (page != null)
{
int total = page.Key.Total;
List<EventTable> events = page.Select(e => e).ToList();
}

Related

EF 3.1/6.0 throw exception `Argument types do not match`, when select field as AsQueryable

I try upgrade project from netcore 2.2 to netcore 3.1, after upgrade to 6.0.7. On netcore 2.2 code work.
If i select only data (without AsQueryable(), only id ) code work on 3.1. If try get any of fields, which defined as IQueryable<...> - always throw exception.
EF 6 return
Message=The query contains a projection 's => s.Emails
.Select(e => new Email{ Value = e.Value }
)
.AsQueryable()' of type 'IQueryable'. Collections in the final projection must be an 'IEnumerable' type such as
'List'. Consider using 'ToList' or some other mechanism to convert
the 'IQueryable' or 'IOrderedEnumerable' into an
'IEnumerable'.
How it work in netcore 2.2:
function return general Queryable object
for table request called this function, and apply same filters, extended query - such by email - add where <condition> for email
if add where for emails - ef generate sql with same condition for filter by email
if not added where - nothing happened
Simplified code sample:
IQueryable<UserView> users = query
.AsNoTracking()
.Select(s => new UserView()
{
UserId = s.Id,
AccountId = null,
Emails = s.Emails.Select(e => new Email()
{
Value = e.Value
}).AsQueryable()
};
var tmp = users.Take(2).Skip(20).ToList();
class definition
public class UserView
{
public int UserId { get; set; }
public int? AccountId { get; set; }
public IQueryable<Email> Emails { get; internal set; }
}

Where is client evaluation needed?

I have a query which works fine in Linq to Objects in this fiddle:
var list = from order in orders
join detail in details
on order.id equals detail.order into od
select new { order = order, details = od };
I tried applying the same query when the data is in a database (note I am mapping Linq to Sql manually):
public class dbContext: DbContext {
public DbSet<Order> Orders { get; set; }
public DbSet<Detail> Details { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder oB) {
oB.UseSqlServer("...connection string...");
}
}
using (var db = new dbContext() {
var list = from order in db.Orders
join detail in db.Details
on order.id equals detail.order into orderDetails
select new { order = order, details = orderDetails };
}
The above gives:
could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
I tried details = orderDetails.ToList() in the last line but the same error is there. Where should I add the manual client evaluation?
Some background information: the following database query (without the into) works fine:
var list = from order in db.Orders
join detail in db.Details
on order.id equals detail.order
select new { order = order, detail = detail };
Instead of a join you should declare Navigation Properties and use something like:
var query = from order in db.Orders
select new { order = order, details = order.OrderDetails };
var list = query.ToList();
or simply
var list = db.Orders.Include(o => o.OrderDetails).ToList();

Issue : EF Core 3.1 gives incorrect value for Tables column which is updated using Ado.Net

I have a situation in my code where first when i update a table column using Ado.net and then when i use EFCore 3.1 to fetch the particular table's data using DbContext.Entities, i dont get the correct value.
Below is the code which i am using:
var sqlCommandText = "Update testTable set Value = 10";
var sqlConnection = (SqlConnection)_dbContext.Database.GetDbConnection();
using var sqlCommand = new SqlCommand(sqlCommandText);
sqlConnection.Open();
sqlCommand.ExecuteNonQuery();
sqlConnection.Close();
Below is the Model and DbContext to be used by EF Core-
public class TestTable
{
public int Id {get;set;}
public int Value {get;set;}
}
public class TestDbContext : DbContext
{
public TestDbContext() : base("connectionString")
{
}
public DbSet<TestTable> TestTable {get;set;}
}
Now when i use the below code to get the TestTable data from Context, i dont see the newly updated value for 'Value' column.
var columnsValue = _dbContext.TestTable.ToList()[0].Value;
This value is coming as 0 whereas it should be 10 as per Ado.Net code.
Pls Let me know what is the issue here.
I am not sure that your code sets Value=10.
Try this
var sqlCommandText = "Update testTable set Value = 10;";
var result= _dbContext.DataBase.ExecuteSqlCommand(sqlCommandText);
if(result==0) ...error
The dbContext is not getting refreshed automatically after executeSql. You have to reload the entry that was changed. And try to use FirstOrDefault instead of list:
var item= _dbContext.TestTable.FirstOrDefault();
if(item!=null)
{
_dbContext.Entry(item).Reload();
var columnsValue=item.Value;
}

How to query in LINQ & Entity Framework the unmapped property

So I partially followed from an SO answer on how to store a property with array datatype in Entity Framework. What I didn't follow on that answer is setting the string InternalData to be private instead of public as I find it a code smell if it is set to public (not enough reputation to comment there yet).
I also managed to map the private property in entity framework from this blog.
When I perform CR (create, read) from that entity, all goes well. However, when my LINQ query has a where clause using that property with array datatype, it says that "System.NotSupportedException: 'The specified type member is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.'".
How to work around on this? Here are the relevant code blocks:
public class ReminderSettings
{
[Key]
public string UserID { get; set; }
[Column("RemindForPaymentStatus")]
private string _remindForPaymentStatusCSV { get; set; }
private Status[] _remindForPaymentStatus;
[NotMapped]
public Status[] RemindForPaymentStatus
{
get
{
return Array.ConvertAll(_remindForPaymentStatusCSV.Split(','), e => (Status)Enum.Parse(typeof(Status), e));
}
set
{
_remindForPaymentStatus = value;
_remindForPaymentStatusCSV = String.Join(",", _remindForPaymentStatus.Select(x => x.ToString()).ToArray());
}
}
public static readonly Expression<Func<ReminderSettings, string>> RemindForPaymentStatusExpression = p => p._remindForPaymentStatusCSV;
}
public enum Status
{
NotPaid = 0,
PartiallyPaid = 1,
FullyPaid = 2,
Overpaid = 3
}
protected override void OnModelCreating(DbModelBuuilder modelBuilder)
{
modelBuilder.Entity<ReminderSettings>().Property(ReminderSettings.RemindForPaymentStatusExpression);
}
//This query will cause the error
public IEnumerable<ReminderSettings> GetReminderSettingsByPaymentStatus(Status[] statusArray)
{
var query = ApplicationDbContext.ReminderSettings.Where(x => x.RemindForPaymentStatus.Intersect(statusArray).Any());
return query.ToList(); //System.NotSupportedException: 'The specified type member 'RemindForPaymentStatus' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.'
}
Entity Framework can not translate LINQ expressions to SQL if they access a property that is annotated as [NotMapped]. (It also can not translate if the property contains custom C# code in its getter/setter).
As a quick (but potentially low performance) workaround, you can execute the part of the query that does not cause problems, then apply additional filtering in-memory.
// execute query on DB server and fetch items into memory
var reminders = dbContext.ReminderSettings.ToList();
// now that we work in-memory, LINQ does not need to translate our custom code to SQL anymore
var filtered = reminders.Where(r => r.RemindForPaymentStatus.Contains(Status.NotPaid));
If this causes performance problems, you have to make the backing field of your NotMapped property public and work directly with it.
var filtered = dbContext.ReminderSettings
.Where(r => r._remindForPaymentStatusCSV.Contains(Status.NotPaid.ToString("D"));
Edit
To handle multiple Status as query parameters, you can attach Where clauses in a loop (which behaves like an AND). This works as long as your Status enum values are distinguishable (i.e. there is no Status "11" if there is also a Status "1").
var query = dbContext.ReminderSettings.Select(r => r);
foreach(var statusParam in queryParams.Status) {
var statusString = statusParam.ToString("D");
query = query.Where(r => r._remindForPaymentStatusCSV.Contains(statusString));
}
var result = query.ToArray();

Entity splitting with optional relation - (EF code first)

I have 2 tables: an Orders table and an OrderActivity table. If no activity has been taken on an order, there will be no record in the OrderActivity table. I currently have the OrderActivity table mapped as an optional nav property on the Order entity and I handle updates to OrderActivity like this:
if (order.OrderActivity == null)
{
order.OrderActivity = new OrderActivity();
}
order.OrderActivity.LastAccessDateTime = DateTime.Now;
Is it possible to consolidate this such that the columns of the OrderActivity table are mapped to properties on the Orders entity, and will default if there is no OrderActivity record? Configuration for entity splitting only appears to work if records exist in both tables. If it is not possible, what is the best practice to obscure the child entity from my domain model? My goal is to keep the model as clean as possible while interacting with a DB schema that I have no control over.
You can create the mapping and specify the type of LastAccessDate as Nullable<DateTime>. The mapping will create one-to-one with LastAccessDate being optional.
public class Order {
[Key]
public int Id { get; set; }
public string Name { get; set; }
public DateTime? LastAccessDate { get; set; }
}
modelBuilder.Entity<Order>().Map(m => {
m.Properties(a => new { a.Id, a.Name });
m.ToTable("Order");
}).Map(m => {
m.Properties(b => new { b.Id, b.LastAccessDate });
m.ToTable("OrderActivity");
});
In this case, specifying LastAccessDate property is optional when inserting new orders.
var order = new Order();
order.Name = "OrderWithActivity";
order.LastAccessDate = DateTime.Now;
db.Orders.Add(order);
db.SaveChanges();
order = new Order();
order.Name = "OrderWithoutActivity";
db.Orders.Add(order);
db.SaveChanges();
Note this will always create one entry in each table. This is necessary because EF creates INNER JOIN when you retrieve Orders and you want to get all orders in this case. LastAccessDate will either have a value or be null.
// Gets both orders
var order = db.Orders.ToList();
// Gets only the one with activity
var orders = db.Orders.Where(o => o.LastAccessDate != null);