I'm trying to get just the ids for dependents if a principal is queried, every time the principal is queried.
My initial thought is to add it somehow in the OnModelCreating definitions, however that appears to be limited to filtering down larger sets of data, unless I'm missing something.
Something like this:
builder.Entity<ListingModel>()
.AlsoDoThis(
x => x.MenuIds.AddRange(
Menus.Where(y => y.ListingId == x.Id).Select(y => y.Id).ToList()
)
);
There is a need to not do this in code for each individual place I have a Select, since that functionality is normalized in some base classes. The base classes have a <TModel> passed in and don't inherently know what properties need to be handled this way.
I do have a workaround where I'm grabbing everything with an AutoInclude(), then filtering it out in the model definition with customer getter/setter to return a list of ids. But rather than being more performant (grabbing related FK ids at the DB level) it's transferring all of that data to the server and then programmatically selecting a list of ids, as far as I understand it.
private List<int> _topicsIds = new();
[NotMapped]
public List<int> TopicsIds
{
get { return Topics.Count > 0 ? Topics.Select(x => x.Id).ToList() : _topicsIds; }
set { _topicsIds = value; }
}
public List<TopicModel> Topics { get; set; } = new();
"Extra SQL that gets called with every select in a context" is (to my limited knowledge) almost what HasQueryFilter does, with a just slightly broader operation. I think this is the approach I'm looking for, just selecting more stuff instead of filtering stuff out.
You can project everything via Select
var result = ctx.ListingModels
.Select(lm => new // or to DTO
{
Id = lm.Id,
OtherProperty = lm.OtherProperty,
Ids = x.MenuIds.Select(m => m.Id).ToList()
})
.ToList();
To make more general solution we can use annotations and define how to project such entities.
During Model defining:
builder.Entity<TopicModel>()
.WithProjection(
x => x.MenuIds,
x => x.Menus.Where(y => y.ListingId == x.Id).Select(y => y.Id).ToList()
);
Then usage in common code:
public virtual List<TModel> GetList(List<int> ids)
{
var list = _context.Set<TModel>().Where(x => ids.Any(id => id == x.Id))
.ApplyCustomProjection(_context)
.ToList();
return list;
}
ApplyCustomProjection(_context) will find previously defined annotation and will apply custom projection.
And extensions implementation:
public static class ProjectionExtensions
{
public const string CustomProjectionAnnotation = "custom:member_projection";
public class ProjectionInfo
{
public ProjectionInfo(MemberInfo member, LambdaExpression expression)
{
Member = member;
Expression = expression;
}
public MemberInfo Member { get; }
public LambdaExpression Expression { get; }
}
public static bool IsUnderDotnetTool { get; }
= Process.GetCurrentProcess().ProcessName == "dotnet";
public static EntityTypeBuilder<TEntity> WithProjection<TEntity, TValue>(
this EntityTypeBuilder<TEntity> entity,
Expression<Func<TEntity, TValue>> propExpression,
Expression<Func<TEntity, TValue>> assignmentExpression)
where TEntity : class
{
// avoid registering non serializable annotations during migrations update
if (IsUnderDotnetTool)
return entity;
var annotation = entity.Metadata.FindAnnotation(CustomProjectionAnnotation);
var projections = annotation?.Value as List<ProjectionInfo> ?? new List<ProjectionInfo>();
if (propExpression.Body is not MemberExpression memberExpression)
throw new InvalidOperationException($"'{propExpression.Body}' is not member expression");
if (memberExpression.Expression is not ParameterExpression)
throw new InvalidOperationException($"'{memberExpression.Expression}' is not parameter expression. Only single nesting is allowed");
// removing duplicate
projections.RemoveAll(p => p.Member == memberExpression.Member);
projections.Add(new ProjectionInfo(memberExpression.Member, assignmentExpression));
return entity.HasAnnotation(CustomProjectionAnnotation, projections);
}
public static IQueryable<TEntity> ApplyCustomProjection<TEntity>(this IQueryable<TEntity> query, DbContext context)
where TEntity : class
{
var et = context.Model.FindEntityType(typeof(TEntity));
var projections = et?.FindAnnotation(CustomProjectionAnnotation)?.Value as List<ProjectionInfo>;
// nothing to do
if (projections == null || et == null)
return query;
var propertiesForProjection = et.GetProperties().Where(p =>
p.PropertyInfo != null && projections.All(pr => pr.Member != p.PropertyInfo))
.ToList();
var entityParam = Expression.Parameter(typeof(TEntity), "e");
var memberBinding = new MemberBinding[propertiesForProjection.Count + projections.Count];
for (int i = 0; i < propertiesForProjection.Count; i++)
{
var propertyInfo = propertiesForProjection[i].PropertyInfo!;
memberBinding[i] = Expression.Bind(propertyInfo, Expression.MakeMemberAccess(entityParam, propertyInfo));
}
for (int i = 0; i < projections.Count; i++)
{
var projection = projections[i];
var expression = projection.Expression.Body;
var assignExpression = ReplacingExpressionVisitor.Replace(projection.Expression.Parameters[0], entityParam, expression);
memberBinding[propertiesForProjection.Count + i] = Expression.Bind(projection.Member, assignExpression);
}
var memberInit = Expression.MemberInit(Expression.New(typeof(TEntity)), memberBinding);
var selectLambda = Expression.Lambda<Func<TEntity, TEntity>>(memberInit, entityParam);
var newQuery = query.Select(selectLambda);
return newQuery;
}
}
I am trying to find records in a Postgresql table with a column of type Date using a simple equality. I am using DateTime properties (not noda)
I am sure that I am missing something very basic because I am not succeeding after trying many different ways.
The following code does not return any result even though the date is equal to Dates in the database:
var m = new Model {
dateprop = default(DateTime)
}
var result = await context.Model
.Where(a => a.dateprop == m.dateprop)
.ToListAsync()
In the database, there are plenty of records with such date, 0001-01-01, in the column.
Besides, I have tried:
.Where(a => a.dateprop == m.dateprop.Date)
.Where(a => a.dateprop.Date == m.dateprop.Date)
.Where(a => a.dateprop.Date == m.dateprop)
.Where(a => a.dateprop.Equals( m.dateprop)) and all the varieties above
Then, if I get the query that EF produces for .Where(a => a.dateprop.Date == m.dateprop.Date) and run it from a SQL client, it works and returns all the records.
What is the correct way to define the Date equality condition in the where clause?
Edit
I have edited the question since it was misleading, my apologies. Originally, the code showed as:
var m = new Model {
dateprop = DateTime.Parse("1970-01-01")
}
var result = await context.Model
.Where(a => a.dateprop == m.dateprop)
.ToListAsync()
The following runnable code sample works. I'd double-check my database to see exactly what's in there and whether it corresponds to default(DateTime), and if you're still convinced there's an issue, try tweaking the code below to make it appear.
class Program
{
static async Task Main(string[] args)
{
await using var ctx = new BlogContext();
await ctx.Database.EnsureDeletedAsync();
await ctx.Database.EnsureCreatedAsync();
var m = new Model
{
DateProperty = default
};
var result = await ctx.Model
.Where(a => a.DateProperty == m.DateProperty)
.ToListAsync();
Console.WriteLine($"Count: {result.Count}");
}
}
public class BlogContext : DbContext
{
public DbSet<Model> Model { get; set; }
static ILoggerFactory ContextLoggerFactory
=> LoggerFactory.Create(b => b.AddConsole().AddFilter("", LogLevel.Information));
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
=> optionsBuilder
.UseNpgsql(#"Host=localhost;Username=test;Password=test")
.EnableSensitiveDataLogging()
.UseLoggerFactory(ContextLoggerFactory);
protected override void OnModelCreating(ModelBuilder modelBuilder)
=> modelBuilder.Entity<Model>().HasData(
new Model { Id = -1, DateProperty = default });
}
public class Model
{
public int Id { get; set; }
public DateTime DateProperty { get; set; }
}
I have a DTO class like this
public string Make { get; set; };
public List<string> Models { get; set; }
Then there is a table which contains a list of vehicles, with make and model columns.
My API endpoint accepts a list of strings (the Makes)
I need to return a list of the DTO class with each make and the list of models.
public async Task<ActionResult<List<MakeModelDTO>>> GetModelsByMakes([FromQuery] List<string> make_list)
{
return await _context.Vehicles.Where(x => x.Make.????).Select(x => x.Model).Distinct().ToListAsync();
}
I don't even want to show the code I've tried, because all versions turned out to be a mess.
I know this is suppose to be a very simple task, I just can't figure it out.
Firstly, In condition you can do this
Where(x => make_list.Contains(x.Make)
or
Where(x => make_list.Any(m => m == x.Make)
Secondly, the method is returning List<MakeModelDTO>, So you should adjust select result like below
.GroupBy(p => p.Make).Select(g =>
new MakeModelDTO { Make = g.Key, Models = g.Select(p => p.Model).ToList() }).ToListAsync();
FullCode
public async Task<ActionResult<List<MakeModelDTO>>> GetModelsByMakes([FromQuery] List<string> make_list)
{
return await _context.Vehicles.Where(x => make_list.Contains(x.Make)).GroupBy(p => p.Make).Select(g =>
new MakeModelDTO { Make = g.Key, Models = g.Select(p => p.Model).ToList() }).ToListAsync();
}
As #Fabio suggested you can filter the make like that. Adding to it convert to your DTO you need to group by that also.
You could try something like this
public async Task<ActionResult<List<MakeModelDTO>>> GetModelsByMakes([FromQuery] List<string> make_list)
{
var resultAsDto = await _context.Vehicles
.Where(vehicle => makeList.Contains(vehicle.Make))
.GroupBy(v=>v.Make)
.Select(g=> new YourDto
{
Make= g.Key -- As it grouped by make
Models = g.Select(v=>v.Model)
}).ToListAsync()
return resultAsDto
}
I use entity framework migration (in Automatic migration mode). Everything is okay, but I have one question:
How should I seed data when I have many-to-many relationships?
For example, I have two model classes:
public class Parcel
{
public int Id { get; set; }
public string Description { get; set; }
public double Weight { get; set; }
public virtual ICollection<BuyingItem> Items { get; set; }
}
public class BuyingItem
{
public int Id { get; set; }
public decimal Price { get; set; }
public virtual ICollection<Parcel> Parcels { get; set; }
}
I understand how to seed simple data (for PaymentSystem class) and one-to-many relationships, but what code should I write in the Seed method to generate some instances of Parcel and BuyingItem? I mean using DbContext.AddOrUpdate(), because I don't want to duplicate data every time I run Update-Database.
protected override void Seed(ParcelDbContext context)
{
context.AddOrUpdate(ps => ps.Id,
new PaymentSystem { Id = 1, Name = "Visa" },
new PaymentSystem { Id = 2, Name = "PayPal" },
new PaymentSystem { Id = 3, Name = "Cash" });
}
protected override void Seed(Context context)
{
base.Seed(context);
// This will create Parcel, BuyingItems and relations only once
context.AddOrUpdate(new Parcel()
{
Id = 1,
Description = "Test",
Items = new List<BuyingItem>
{
new BuyingItem() { Id = 1, Price = 10M },
new BuyingItem() { Id = 2, Price = 20M }
}
});
context.SaveChanges();
}
This code creates Parcel, BuyingItems and their relationship, but if I need the same BuyingItem in another Parcel (they have a many-to-many relationship) and I repeat this code for the second parcel - it will duplicate BuyingItems in the database (though I set the same Ids).
Example:
protected override void Seed(Context context)
{
base.Seed(context);
context.AddOrUpdate(new Parcel()
{
Id = 1,
Description = "Test",
Items = new List<BuyingItem>
{
new BuyingItem() { Id = 1, Price = 10M },
new BuyingItem() { Id = 2, Price = 20M }
}
});
context.AddOrUpdate(new Parcel()
{
Id = 2,
Description = "Test2",
Items = new List<BuyingItem>
{
new BuyingItem() { Id = 1, Price = 10M },
new BuyingItem() { Id = 2, Price = 20M }
}
});
context.SaveChanges();
}
How can I add the same BuyingItem in different Parcels?
Updated Answer
Make sure you read "Using AddOrUpdate Properly" section below for a complete answer.
First of all, let's create a composite primary key (consisting of parcel id and item id) to eliminate duplicates. Add the following method in the DbContext class:
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Entity<Parcel>()
.HasMany(p => p.Items)
.WithMany(r => r.Parcels)
.Map(m =>
{
m.ToTable("ParcelItems");
m.MapLeftKey("ParcelId");
m.MapRightKey("BuyingItemId");
});
}
Then implement the Seed method like so:
protected override void Seed(Context context)
{
context.Parcels.AddOrUpdate(p => p.Id,
new Parcel { Id = 1, Description = "Parcel 1", Weight = 1.0 },
new Parcel { Id = 2, Description = "Parcel 2", Weight = 2.0 },
new Parcel { Id = 3, Description = "Parcel 3", Weight = 3.0 });
context.BuyingItems.AddOrUpdate(b => b.Id,
new BuyingItem { Id = 1, Price = 10m },
new BuyingItem { Id = 2, Price = 20m });
// Make sure that the above entities are created in the database
context.SaveChanges();
var p1 = context.Parcels.Find(1);
// Uncomment the following line if you are not using lazy loading.
//context.Entry(p1).Collection(p => p.Items).Load();
var p2 = context.Parcels.Find(2);
// Uncomment the following line if you are not using lazy loading.
//context.Entry(p2).Collection(p => p.Items).Load();
var i1 = context.BuyingItems.Find(1);
var i2 = context.BuyingItems.Find(2);
p1.Items.Add(i1);
p1.Items.Add(i2);
// Uncomment to test whether this fails or not, it will work, and guess what, no duplicates!!!
//p1.Items.Add(i1);
//p1.Items.Add(i1);
//p1.Items.Add(i1);
//p1.Items.Add(i1);
//p1.Items.Add(i1);
p2.Items.Add(i1);
p2.Items.Add(i2);
// The following WON'T work, since we're assigning a new collection, it'll try to insert duplicate values only to fail.
//p1.Items = new[] { i1, i2 };
//p2.Items = new[] { i2 };
}
Here we make sure that the entities are created/updated in the database by calling context.SaveChanges() within the Seed method. After that, we retrieve the required parcel and buying item objects using context. Thereafter we use the Items property (which is a collection) on the Parcel objects to add BuyingItem as we please.
Please note, no matter how many times we call the Add method using the same item object, we don't end up with primary key violation. That is because EF internally uses HashSet<T> to manage the Parcel.Items collection. A HashSet<Item>, by its nature, won't let you add duplicate items.
Moreover, if you somehow manage to circumvent this EF behavior as I have demonstrated in the example, our primary key won't let the duplicates in.
Using AddOrUpdate Properly
When you use a typical Id field (int, identity) as an identifier expression with AddOrUpdate method, you should exercise caution.
In this instance, if you manually delete one of the rows from the Parcel table, you'll end up creating duplicates every time you run the Seed method (even with the updated Seed method I have provided above).
Consider the following code,
context.Parcels.AddOrUpdate(p => p.Id,
new Parcel { Id = 1, Description = "Parcel 1", Weight = 1.0 },
new Parcel { Id = 2, Description = "Parcel 1", Weight = 1.0 },
new Parcel { Id = 3, Description = "Parcel 1", Weight = 1.0 }
);
Technically (considering the surrogate Id here), the rows are unique, but from the end-user point of view, they are duplicates.
The true solution here is to use the Description field as an identifier expression. Add this attribute to the Description property of the Parcel class to make it unique: [MaxLength(255), Index(IsUnique=true)]. Update the following snippets in the Seed method:
context.Parcels.AddOrUpdate(p => p.Description,
new Parcel { Description = "Parcel 1", Weight = 1.0 },
new Parcel { Description = "Parcel 2", Weight = 2.0 },
new Parcel { Description = "Parcel 3", Weight = 3.0 });
// Make sure that the above entities are created in the database
context.SaveChanges();
var p1 = context.Parcels.Single(p => p.Description == "Parcel 1");
Note, I'm not using the Id field as EF is going to ignore it while inserting rows. And we are using Description to retrieve the correct parcel object, no matter what Id value is.
Old Answer
I would like to add a few observations here:
Using Id is probably not going to do any good if the Id column is a database generated field. EF is going to ignore it.
This method seems to be working fine when the Seed method is run once. It won't create any duplicates, however, if you run it for a second time (and most of us have to do that often), it may inject duplicates. In my case it did.
This tutorial by Tom Dykstra showed me the right way of doing it. It works because we don't take anything for granted. We don't specify IDs. Instead, we query the context by known unique keys and add related entities (which again are acquired by querying context) to them. It worked like a charm in my case.
You must fill many-to-many relation in the same way as you build many-to-many relation in any EF code:
protected override void Seed(Context context)
{
base.Seed(context);
// This will create Parcel, BuyingItems and relations only once
context.AddOrUpdate(new Parcel()
{
Id = 1,
Description = "Test",
Items = new List<BuyingItem>
{
new BuyingItem() { Id = 1, Price = 10M },
new BuyingItem() { Id = 2, Price = 20M }
}
});
context.SaveChanges();
}
Specifying Id which will be used in database is crucial otherwise each Update-Database will create new records.
AddOrUpdate doesn't support changing relations in any way so you cannot use it to add or remove relations in next migration. If you need it you must manually remove relation by loading Parcel with BuyingItems and calling Remove or Add on navigation collection to break or add new relation.
Ok. I understand how I should be in that situation:
protected override void Seed(Context context)
{
base.Seed(context);
var buyingItems = new[]
{
new BuyingItem
{
Id = 1,
Price = 10m
},
new BuyingItem
{
Id = 2,
Price = 20m,
}
}
context.AddOrUpdate(new Parcel()
{
Id = 1,
Description = "Test",
Items = new List<BuyingItem>
{
buyingItems[0],
buyingItems[1]
}
},
new Parcel()
{
Id = 2,
Description = "Test2",
Items = new List<BuyingItem>
{
buyingItems[0],
buyingItems[1]
}
});
context.SaveChanges();
}
There are no duplicates in database.
Thank you, Ladislav, you gave me a right vector to find a solution for my task.
I have a Team table and a Player table in many to many relationship. There is a linking table called TeamOnPlayer. EF with POCO generates navigation propertie called Person for the Team entity and also generates a nav. prop. called Team for the People entity.
I'm trying to insert a new record into the TeamOnPlayer table, but EF and POCO hides it. I tried to do this:
public static void AddPersonToTeam(int TeamId, int PersonId)
{
using (var ef = new korfballReportEntities())
{
var team = GetTeam(TeamId);
var person = GetPerson(PersonId);
team.Person.Add(person);
person.Team.Add(team);
ef.SaveChanges();
}
}
The GetTeam(TeamId) and GetPerson(PersonId) gets the right team and person:
public static Team GetTeam(int id)
{
using (var ef = new korfballReportEntities())
{
var q = from l in ef.Team
where l.Id == id
select l;
return q.Single();
}
}
public static Person GetPerson(int id)
{
using (var ef = new korfballReportEntities())
{
var query = from p in ef.Person
where p.Id == id
select p;
return query.Single();
}
}
When it tries to call the team.Person.Add(person) it throws an exception:
"The ObjectContext instance has been disposed and can no longer be used for operations that require a connection." System.Exception {System.ObjectDisposedException}
Can anyone please show me the correct way?
Edit
Now I understand what the problem was, thanks to you. I was a bit confused about the using blocks you included. For example this:
using (var ef = new korfballReportEntities())
{
//switch lazy loading off, only in this single context
ef.Configuration.LazyLoadingEnabled = false;
var repository = new MyRepository(ef);
repository.AddPersonToTeam(int TeamId, int PersonId);
}
Where should I put it?
I've done something else. I simply did this, and it worked fine.
public static void AddPersonToTeam(int TeamId, int PersonId)
{
using (var ef = new korfballReportEntities())
{
var q = from t in ef.Team
where t.Id == TeamId
select t;
var team = q.Single();
var q2 = from p in ef.Person
where p.Id == PersonId
select p;
var person = q2.Single();
try
{
team.Person.Add(person);
person.Team.Add(team);
}
catch (Exception e)
{
}
ef.SaveChanges();
}
}
The only problem is, that i coludn't reuse my GetPerson(int id) and GetTeam(int id) method.
What do you think? Is it okay? Is this an ugly way?
My guess is that you are working with lazy loading - your navigation properties Team.Person and Person.Team are marked as virtual in your entity classes. The result is that your methods GetTeam and GetPerson do not exactly return Team and Person objects but instances of dynamically created proxy classes which are derived from those entities. This dynamic proxy supports lazy loading which means that EF tries to load the navigation collections Team.Person and Person.Team when you access them for the first time. This happens in your AddPersonToTeam method when you call Add on these collections.
Now the problem is that the proxies are created within an context which you immediately dispose in your GetTeam and GetPerson methods (at the end of the using block). The proxies have stored a reference to this context internally and will use this context to load the navigation collections from the database.
Because these contexts are already disposed you get the exception.
You should redesign your code a bit: Don't create a new context in your repository methods GetTeam and GetPerson. You should instead use the same context for all operations: Retrieving the Team, retrieving the Person and adding the relationship. For example:
public static void AddPersonToTeam(int TeamId, int PersonId)
{
using (var ef = new korfballReportEntities())
{
var team = GetTeam(ef, TeamId);
var person = GetPerson(ef, PersonId);
team.Person.Add(person);
//person.Team.Add(team); <- not necessary, EF will handle this
ef.SaveChanges();
}
}
public static Team GetTeam(korfballReportEntities ef, int id)
{
var q = from l in ef.Team
where l.Id == id
select l;
return q.Single();
}
public static Person GetPerson(korfballReportEntities ef, int id)
{
var query = from p in ef.Person
where p.Id == id
select p;
return query.Single();
}
Another approach is to make your "Repository"/"Service" not static, inject the context into the constructor and then use this context throughout the repository. Then you don't need to pass in the context into every method. A rough sketch:
using (var ef = new korfballReportEntities())
{
var repository = new MyRepository(ef);
repository.AddPersonToTeam(int TeamId, int PersonId);
}
public class MyRepository
{
private readonly korfballReportEntities _ef;
public MyRepository(korfballReportEntities ef)
{
_ef = ef;
}
public void AddPersonToTeam(int TeamId, int PersonId)
{
var team = GetTeam(TeamId);
var person = GetPerson(PersonId);
team.Person.Add(person);
_ef.SaveChanges();
}
public Team GetTeam(int id)
{
var q = from l in _ef.Team
where l.Id == id
select l;
return q.Single();
}
public Person GetPerson(int id)
{
var query = from p in _ef.Person
where p.Id == id
select p;
return query.Single();
}
}
Edit
One little thing about performance tuning: In this specific case lazy loading is not necessary and more disturbing. It causes to load a (potentially long) collection team.Person when you want to add only one additional Person to the collection. You can switch off lazy loading for this particular operation (I refer to my second example):
using (var ef = new korfballReportEntities())
{
//switch lazy loading off, only in this single context
ef.Configuration.LazyLoadingEnabled = false;
var repository = new MyRepository(ef);
repository.AddPersonToTeam(int TeamId, int PersonId);
}
public void AddPersonToTeam(int TeamId, int PersonId)
{
var team = GetTeam(TeamId);
var person = GetPerson(PersonId);
// if lazy loading is off, the collecton is null, so we must instantiate one
if (team.Person == null)
team.Person = new List<Person>();
team.Person.Add(person);
_ef.SaveChanges();
}