Cannot insert explicit value for identity column in table when IDENTITY_INSERT is set to OFF - EF Core many-to-many (child) data - entity-framework

Following through Julie Lerman's Pluralsight course EF Core 6 Fundamentals I've created two classes in my own project (my own design, but identical to the course in terms of class structure/data hierarchy):
Class 1: Events - To hold information about an event being held (e.g. a training course), with a title and description (some fields removed for brevity):
public class EventItem
{
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int EventItemId { get; set; }
[Required(AllowEmptyStrings = false)]
public string EventTitle { get; set; }
public string? EventDescription { get; set; }
[Required]
public List<EventCategory> EventCategories { get; set; } = new();
}
Class 2: Event categories - Each event can be linked to one or more pre-existing (seeded) categories (e.g. kids, adult).
public class EventCategory
{
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int EventCategoryId { get; set; }
[Required]
public string EventCategoryName { get; set; }
public List<EventItem>? EventItems { get; set; }
}
In my Razor form to create the event, the user can select from multiple categories. Using EF Core I take the posted data (via a VM/DTO object) and construct the relevant parent/child entities. However upon saving to the database I get an exception as EF Core tries to re-create the categories when they already exist:
Cannot insert explicit value for identity column in table
'EventCategories' when IDENTITY_INSERT is set to OFF.
My code explicitly looks up the existing categories selected by the user, but the context tracker appears to still believe they need inserting, in addition to creating the many-to-many relationship.
I'd appreciate any input as to why this is happening please:
using (var dbcontext = DbFactory.CreateDbContext())
{
// Get selected categories from user's check box list
var selectedCategoryIds = _eventCagetories.Where(c => c.isSelected).Select(c => c.EventCategoryId).ToList();
// Create new Event
var newEventItem = new EventFinderDomain.Models.EventItem() {
EventTitle = _eventItemDTO.EventTitle,
EventDescription = _eventItemDTO.EventDescription,
EventUrl = _eventItemDTO.EventUrl,
TicketUrl = _eventItemDTO.TicketUrl
};
// Find categories from the database based on their ID value
var selectedEventCategories = dbcontext.EventCategories.Where(c => selectedCategoryIds.Contains(c.EventCategoryId)).ToList();
// Add the categories to the event
newEventItem.EventCategories!.AddRange(selectedEventCategories);
// Add the event to the change tracker
await dbcontext.EventItems.AddAsync(newEventItem); // <-- Created correctly with child list objects added
// Detect changes for debugging
dbcontext.ChangeTracker.DetectChanges();
var debugView = dbcontext.ChangeTracker.DebugView; // <-- Incorrectly shows newEventItem.Categories being added
// Save to database
await dbcontext.SaveChangesAsync(); // <-- Cannot insert explicit value for identity column
}
The Event entity appears to be correctly created in the debugger with its related child categories included:
The change tracker however incorrectly shows the selected categories being added again when they already exist:

After commenting out every line of code in the app and adding back in until it broke, it emerges the problem was elsewhere within Program.cs:
builder.Services.AddDbContextFactory<EventFinderContext>(
opt => opt.UseSqlServer(new SqlConnectionStringBuilder() {/*...*/}.ConnectionString)
.EnableSensitiveDataLogging()
.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking) // <-- THE CULPRIT
);
In the training video this method was described as a way of reducing overhead for disconnected apps. I had assumed that because of the disconnected nature of HTTP, this would be beneficial and that context would be re-established when creating the model's child data. This was incorrect on my part.
I should have used .AsNoTracking() only when retriving read-only data from my database. For example, loading in the child-data for a new model that wouldn't be modified directly, but used to create the many-to-many data (explicitly, for the category data option items only and not for the event data).

Related

How do you use EF Core to simultaneously insert into multiple tables?

I am new to EF Core 6.0.1, using it with Blazor (WebAssembly), .NET 6.0, and Visual Studio 2022. I am creating a database of internal software projects, including their author(s) and maintainer(s).
I am having trouble getting EF Core to take in a List of Authors / List of Maintainers as part of creating a new SoftwareItem from a webform submission.
SoftwareItem in defined (in part) as follows:
public class SoftwareItem
{
[Key]
public int SoftwareId { get; set; }
public string Name { get; set; }
public string CurrentVersion { get; set; }
public string Status { get; set; }
public List<Author> Authors { get; set; }
public List<Maintainer> Maintainers { get; set;}
[other properties omitted]
}
An Author is defined as follows:
public class Author
{
[Key]
public int AuthorId { get; set; }
public int SoftwareItemId { get; set; }
public int ProgrammerId { get; set; }
public Programmer Programmer { get; set; }
}
Maintainer is identical, except for having a MaintainerId instead of an AuthorId.
Programmer is defined as:
public class Programmer
{
[Key]
public int ProgrammerId { get; set; }
public string ProgrammerName { get; set; }
}
EF Core created the tables for me based on a migration, and I have manually populated the Programmer table with the nine people who might be an Author and/or a Maintainer.
I have a webform where the user can create a new SoftwareItem, with pre-populated drop-downs for Authors and Maintainers that, after querying the database, contain the potential ProgrammerNames. The user can assign up to three Authors and up to three Maintainers before submitting the webform (via an Author1 dropdown, an Author2 dropdown etc.) Submitting the webform calls the InsertSoftware method, included below.
Note that I'm not a fan of the repetition between the Author logic and Maintainer logic, and the List should probably be a HashSet (in case the same author is set in Author1 and Author2) but those are issues for another day. The Author1 and similar variables are the int IDs set by the webform. I've previously verified they are being set to the appropriate values via a JavaScript alert. An ID of 0 means the value was never set (e.g. there is no second author).
The SoftwareItem here is instantiated as a new object on OnIntializedAsync and bound as the webform's model.
public async Task InsertSoftware()
{
List<int> authorIdsToAdd = new List<int>();
authorIdsToAdd.Add(Author1);
authorIdsToAdd.Add(Author2);
authorIdsToAdd.Add(Author3);
SoftwareItem.Authors = new List<Author>();
foreach (int author in authorIdsToAdd)
{
if (author != 0)
{
foreach (Programmer programmer in ProgrammerList)
{
if (programmer.ProgrammerId == author)
{
Author addedAuthor = new Author();
addedAuthor.Programmer = new Programmer();
addedAuthor.Programmer.ProgrammerId = author;
SoftwareItem.Authors.Add(addedAuthor);
}
}
}
}
[repeat code for the Maintainers]
await Http.PostAsJsonAsync("api/softwareitem", SoftwareItem);
Navigation.NavigateTo("software/fetchsoftware");
}
The SoftwareItem API is (in part) as follows:
[HttpPost]
public async Task<IActionResult> Create([FromBody] SoftwareItem softwareItem)
{
_context.Software.Add(softwareItem);
await _context.SaveChangesAsync();
return Ok(softwareItem);
}
My understanding from this Stack Overflow question is that if objects have been instantiated for a navigation property when the parent entity is added and saved to the database context, then EF Core will also add the new navigation property values to their appropriate tables. However, that isn't happening, and all I'm getting is a 500 error in the console.
What I'm expecting is that...
A new entry will be inserted into the SoftwareItem table
New entries will be inserted into the Author table, containing an auto-incremented AuthorId, the SoftwareItem's SoftwareItemId, and the ProgrammerId from the webform
New entries will be inserted into the Maintainer table, containing an auto-incremented MaintainerId, the SoftwareItem's SoftwareItemId, and the ProgrammerId from the webform.
Ok, it's a bit difficult to make out what your code is precisely doing but there are a few issues I see.
First, with entities you should always avoid ever reinitializing navigation property lists. During inserts it's "ok", but anywhere else it would lead to bugs/errors so it's better to simply not see it in the code. Pre-initialize your properties in the entity itself:
public class SoftwareItem
{
// ...
public virtual ICollection<Author> Authors { get; set; } = new List<Author>();
public virtual ICollection<Maintainer> Maintainers { get; set;} = new List<Maintainer>();
}
This ensures the collections are ready to go when you need them for a new entity.
Next, it can be helpful to structure your code to avoid things like module level variables. Your InsertSoftware() method references an instance of SoftwareItem and it isn't clear where, or what this reference would be pointing at. If you have a method chain that loaded a particular software item instance to be updated, pass the reference through the chain of methods as a parameter. This helps encapsulate the logic. You should also look to define a scope for whenever you are referencing a DbContext. With Blazor this needs to be done a bit more explicitly to avoid DbContext instances from being too long-lived. Long-lived DbContext instances are a problem because they lead to performance degradation as they track increasing numbers of entities, and can easly become "poisoned" with invalid entities that prevent things like SaveChanges() calls from succeeding. Keep instances alive only as long as absolutely necessary. I would strongly recommend looking at unit of work patterns to help encapsulate the lifetime scope of a DbContext. Ideally entities loaded by a DbContext should not be passed outside of that scope to avoid issues and complexity with detached or orphaned entities.
Next, it is important to know when you are looking to create new entities vs. reference existing data. Code like this is a big red flag:
Author addedAuthor = new Author();
addedAuthor.Programmer = new Programmer();
addedAuthor.Programmer.ProgrammerId = author;
From what I can make out, the Author (and Maintainer) are linking entities so we will want to create one for each "link" between a software item and a programmer. However, Programmer is a reference to what should be an existing row in the database.
If you do something like:
var programmer = new Programmer { ProgrammerId == author };
then associate that programmer as a reference to another entity, you might guess this would tell EF to find and associate an existing programmer.. Except it doesn't. You are telling EF to associate a new programmer with a particular ID. Depending on how EF has been configured for that entity (whether to use an identity column for the PK or not) this will result in one of three things happening if that programmer ID already exists:
A new programmer is created with an entirely new ID (identity gives it a new id and ProgrammerId is ignored)
EF throws an exception when it tries to insert a new programmer with the same ID. (Duplicate PK)
EF throws an exception if you tell it add a new programmer and it happens to already be tracking an instance with the same ID.
So, to fix this, load your references:
List<int> authorIdsToAdd = new List<int>();
// likely need logic to only add authors if they are selected, and unique.
authorIdsToAdd.Add(Author1);
authorIdsToAdd.Add(Author2);
authorIdsToAdd.Add(Author3);
// Define your own suitable scope mechanism for this method or method chain
using (var context = new AppDbContext())
{
var softwareItem = new SoftwareItem { /* populate values from DTO or Map from DTO */ }
// Retrieve references
var authors = await context.Programmers.Where(x => authorIdsToAdd.Contains(x.ProgrammerId)).ToListAsync();
foreach(var author in authors)
{
softwareItem.Authors.Add(new Author { Programmer = author });
}
// Continue for Maintainers...
await context.SaveChangesAsync();
}

Can't create related entity in ASP.NET Core with EF Core

I have a problem creating a related entity in Entity Framework Core 2.0. I've just created the solution, consisting of an Asp.Net Core backend project, and a UWP project to act as client. Both solutions share model. The two models are:
public class UnitOfWork {
public int UnitOfWorkId { get; set; }
public string Name { get; set; }
public Human Human { get; set; }
}
public class Human {
public int HumanId { get; set; }
public string Name { get; set; }
public List<UnitOfWork> WorkDone { get; set; }
}
As you can see, model is very simple. One human has many units of work. By the way, the backend is connected to an Azure SQL database. I've seen the migration classes, and the database schema looks good to me.
The problem I have is when I want to create a unit of work referencing an existing human, using HTTP. The controller is fairly simple:
[HttpPost]
public UnitOfWork Post([FromBody] UnitOfWork unitOfWork) {
using (var db = new DatabaseContext()) {
db.UnitsOfWork.Add(unitOfWork);
var count = db.SaveChanges();
Console.WriteLine("{0} records saved to database", count);
}
return unitOfWork;
}
Again, nothing fancy here.
How can I create an unit of work, and assign it to an existing human? If I try it with an existing human, in this way
var humans = await Api.GetHumans();
var firstHuman = humans.First();
var unitOfWorkToCreate = new UnitOfWork() {
Name = TbInput.Text,
Human = firstHuman,
};
I get this error:
Cannot insert explicit value for identity column in table 'Humans' when IDENTITY_INSERT is set to OFF
I feel that setting IDENTITY_INSERT to ON will solve my problem, but this is not what I want to do. In the client, I'll select an existing human, write down a name for the unit of work, and create the latter. Is this the correct way to proceed?
EDIT: Following #Ivan Stoev answer, I've updated the UnitOfWork controller to attach unitofwork.Human. This led to
Newtonsoft.Json.JsonSerializationException: 'Unexpected end when deserializing array. Path 'human.workDone', line 1, position 86.'
Investigating - seen here - EFCore expects to create collections (like human.WorkDone) in the constructor, so I did it, and no more nulls deserializing. However, now I have a self-referencing loop:
Newtonsoft.Json.JsonSerializationException: Self referencing loop detected with type 'PlainWorkTracker.Models.UnitOfWork'. Path 'human.workDone'.
Any ideas? Thanks!
The operation in question is falling into Saving Disconnected Entities category.
Add methods marks all entities in the graph which are not currently tracked as new (Added) and then SaveChanges will try to insert them in the database.
You need a way to tell EF that unitOfWork.Human is an existing entity. The simplest way to achieve that is to Attach it (which will mark it as Unchanged, i.e. existing) to the context before calling Add:
db.Attach(unitOfWork.Human);
db.Add(unitOfWork);
// ...

Entity Framework Navigation Property preload/reuse

Why is Entity Framework executing queries when I expect objects can be grabbed from EF cache?
With these simple model classes:
public class Blog
{
public int Id { get; set; }
public string Name { get; set; }
public virtual ICollection<Post> Posts { get; set; }
}
public class Post
{
public int Id { get; set; }
public string Content { get; set; }
public virtual Blog Blog { get; set; }
}
public class BlogDbContext : DbContext
{
public BlogDbContext() : base("BlogDbContext") {}
public DbSet<Blog> Blogs { get; set; }
public DbSet<Post> Posts { get; set; }
}
I profile the queries of following action
public class HomeController : Controller
{
public ActionResult Index()
{
var ctx = new BlogDbContext();
// expecting posts are retrieved and cached by EF
var posts = ctx.Posts.ToList();
var blogs = ctx.Blogs.ToList();
var wholeContent = "";
foreach (var blog in blogs)
foreach (var post in blog.Posts) // <- query is executed
wholeContent += post.Content;
return Content(wholeContent);
}
}
Why doesn't EF re-use the Post entities which I had already grabbed with the var posts = ctx.Posts.ToList(); statement?
Further explanation:
An existing application has an Excel export report. The data is grabbed via a main Linq2Sql query with a tree of includes (~20). Then it is mapped via automapper and additional data from manual caches (which previously slowed down the execution if added to the main query) is added.
Now the data is grown and SQL Server crashes when trying to execute the query with an error:
The query processor ran out of internal resources and could not produce a query plan.
Lazy loading would result in >100.000 queries. So I thought I could preload all the required data with a few simple queries and let EF use the objects automatically from cache during lazy loading.
There I initial had additional problems with limits of the TSQL IN() clause which I solved with MoreLinq´s Batch extension.
When you have Lazy Loading enabled, EF will still reload the Collection Navigation Properties. Probably because EF doesn't know whether you have really loaded all the Posts. EG code like
var post = db.Posts.First();
var relatedPosts = post.Blog.Posts.ToList();
Would be tricky, as the Blog would have one Post already loaded, but obviously the others need to be fetched.
In any case when relying on the Change Tracker to fix-up your Navigation Properties, you should disable Lazy Loading anyway. EG
using (var db = new BlogDbContext())
{
db.Configuration.LazyLoadingEnabled = false;
. . .
Given you have the navigation properties, look at leveraging them in your query to feed Automapper a dynamic object to map to your ViewModel/DTO rather than a top-level entity which you'd be relying on eager loading or waiting on lazy loading.
This is done by issuing a .Select() on your query. To use a simple example of extracting order details including the customer name, list of product names and quantities from order lines, and the delivery address where an Order has a reference to customer, and that customer has a delivery address, a collection of order lines, each with a product...
var orderDetails = dbContext.Orders
.Where(o => /* Insert criteria */)
.Select(o => new
{
o.OrderId,
o.OrderNumber,
o.Customer.CustomerId,
CustomerName = x.Customer.FullName,
o.Customer.DeliveryAddress, // Address entity if no further dependencies, or extract fields/relations from the Address.
o.OrderLines.Select( ol = > new
{
ol.OrderLineId,
ProductName = ol.Product.Name,
ol.Quantity
}
}).ToList(); // Ready to feed into Automapper.
With ~20 includes your Select will undoubtedly be a bit more involved, but the idea is to feed SQL Server a query to retrieve just the data you want that you can then feed into Automapper to navigate through where any child relationships can either be flattened by EF or simplified and returned for your mapper to flesh out into the resulting models.
With growing systems you will also want to consider leveraging paging /w Skip and Take rather than ToList, or at least leveraging Take to ensure that there is a cap to the amount of data your return. ToList is a primary performance troll that I look for in EF code because its misuse can kill applications.

EF6 best practice for loading not mapped fields of an entity tracked by the context

Considering the blogging data model:
Blog:
int Id
ICollection<Post> Posts
Post:
int Id
int BlogId
DateTime Date
Then loading Blogs with the date of their latest post (LatestPostDate) and bind to the UI, while they are tracked by the context.
There are some solutions, such as using DTO, but the result entities are not tracked by the context.
Also I can set the LatestPostDate as NotMapped, define a Table-valued function, and apply SqlQuery on DbSet. Although, the NotMapped fields are not loaded in this way.
What are the best practices?
I try not to add column to the table, also avoid calculating the values after loading.
Best practice would be to handle display concerns in a ViewModel.
But as you do not want to map the Entity to another class, let's first take a look at the [NotMapped] variant, using LINQ to calculate the latest post date instead of plain SQL.
using System.Linq;
public class Blog {
public int Id { get; set; }
public virtual ICollection<Post> Posts { get; set; }
[NotMapped]
public DateTime? LatestPostDate {
get {
return Posts.OrderBy(p => p.Date).LastOrDefault()?.Date;
}
}
}
This way, the value is calculated only when you access the property LatestPostDate (probably during UI rendering). You can reduce the number of DB accesses by eager loading the Posts, although this will increase the size of the data set you are working with.
var blogs = _dbContext.Blogs.Include(b => b.Posts).ToArray();
But if you use a ViewModel, you can fill the LatestPostDate in one go:
public class BlogViewModel {
public int Id { get; set; }
public DateTime? LatestPostDate { get; set; }
}
var viewModels = _dbContext.Blogs.Select(b => new BlogViewModel {
Id = b.Id,
LatestPostDate = b.Posts.OrderBy(p => p.Date).LastOrDefault()?.Date;
}).ToArray();
Regarding your concerns that the ViewModel is not tracked by the context: in the edit usecase, load the Entity again using the Id provided by the ViewModel and map the updated properties. This gives you full control over the properties that should be editable. As a bonus, the ViewModel is a good place to do input validation, formatting etc.

Entity Framework table splitting - how to initialize lazy-loaded properties?

Using Entity Framework 6.0, I am attempting to implement table splitting to improve query performance on tables with columns that contain BLOB data. I have followed the recommendations in this tutorial and it does indeed work as described.
Here's a very simplified example of the entity classes that map to one of my split tables ...
public class MyEntity
{
public string Id { get; set; }
public virtual MyEntityContent Content { get; set; }
public string Name { get; set; }
}
public class MyEntityContent
{
public string Id { get; set; }
public virtual MyEntity Entity { get; set; }
public byte[] Blob { get; set; }
}
... and the corresponding configuration code in the associated DbContext implementation ...
modelBuilder.Entity<MyEntity>().HasKey(e => e.Id).ToTable("MyEntities");
modelBuilder.Entity<MyEntityContent>().HasKey(c => c.Id).ToTable("MyEntities");
modelBuilder.Entity<MyEntity>().HasRequired(e => e.Content).WithRequiredPrincipal(d => d.Entity);
Given that the lazy-loaded Content property is Required by Entity Framework, it seems sensible to initialize it to a default value in the constructor of the containing MyEntity class ...
public MyEntity()
{
Content = new MyEntityContent();
}
... which enables a new instance of the class to be created and partially populated, without the risk of an exception being thrown by forgetting to initialize the required property value:
var entity = new MyEntity {Id = "XXX", Name = "something"};
I typically use a similar technique to initialize collection properties on EF entities and it works fine. However, in the above scenario, this initialization in the constructor has an unexpected effect: when retrieving existing entity instances from the database, the database value in the lazy-loaded property is ignored in favor of the empty default value.
This seems illogical to me. Doesn't Entity Framework create an entity object by first calling its default constructor and then applying its own property values to the created instance? If so, this should overwrite my default Content property value with a new instance of MyEntityContent, based on database data. This is how it seems to work with lazy-loaded collection properties.
If it's not possible to do this in the way I am expecting, is there an alternative technique for initializing lazy-loaded properties?
Don't initialize virtual members and perhaps, if you have to, handle any exceptions from uninitialized members.
I just had this issue with an entity with two virtual fields. Originally I had it initialize those two, but after removing them (and initializing the other fields to some default value), it started working for me. Try it out and let me know!
[Edit] I just realized I replied this to a slightly old post, didn't see the date. I guess I'll leave this answer here in case.