We are using Entity Framework 6.4.4 in our project and have ran into a serious problem recently. After running for 10-20 hours (maybe around a million requests) we suddenly get a massive amount of EF materialization errors. Looks like all EF mappings get messed up for some reason and wrong properties are being used even for basic methods like Find.
Couple examples of the errors we receive:
The '...' property on '...' could not be set to a 'System.Byte' value. You must set this property to a non-null value of type 'System.Guid'. at
System.Data.Entity.Core.Common.Internal.Materialization.Shaper.HandleEntityAppendOnly[TEntity](Func`2 constructEntityDelegate, EntityKey entityKey, EntitySet entitySet) at lambda_method(Closure , Shaper )
Shaper....SingleOrDefault[TSource]_InternalSet`1.FindInStore_InternalSet`1.Find
The '...' property on '...' could not be set to a 'System.Byte' value. You must set this property to a non-null value of type 'System.DateTime'
All of this code works until some specific moment so it does not look like an issue with our existing mappings. We were not able to find anyone similar issues so far. Perhaps upgrading to EF Core would resolve them but that would require a huge time investment so right now we are looking for a temporary solution.
Does anyone have information as to what could be causing this? We hope to at least figure out if this is mostly caused by heavy load so we can optimize and scale our code or some sort of error on our part.
Related
I maintain a suite of applications for a SqlServer database that has no simple creation process and various instances in production have slight differences (with regards nullability of columns and varchar sizes). I am moving the data layer of these applications from EF 6 to EF Core 2.1 to increase platform support and to finally have a straightforward way to create new databases with a consistent layout.
I might take this opportunity to clean up my POCOs somewhat. One pattern I wish to do away with is that original SqlServer database often uses tinyint null instead of bit columns with a default constraint on them. These are mapped to byte? rather than bool in my C# code which I think needlessly complicates their usage. Going forward, I'd like new databases to use bit fields instead, and in many cases it is appropriate for them to be not null and defaulted to 0. I have this working, but I figured that with all the flexibility of EF Core, I should be able to subclass my DbContext and provide different mappings in order to enable the same code to run against the original "legacy" databases there these are still nullable tinyints instead.
Attempt 1
I was hoping to use a ValueConverter<bool, byte?> in my LegacyDbContext subclass to accomplish this (passing it into PropertyBuilder<bool>.HasConversion) until I learnt that these cannot be used to convert nulls under the limitations section of the EF Core docs, so I get a System.InvalidOperationException stating:
An exception occurred while reading a database value for property '<tableName>.<columnName>'. The expected type was 'System.Boolean' but the actual value was null.
Attempt 2
Registering a custom Microsoft.EntityFrameworkCore.Metadata.Internal.IEntityMaterializerSource implementation to take care of this conversion for me but I couldn't get that working either, I think it is invoked too late for me to do the type conversion...
Surely it is possible to replace nulls with 0 prior to the type conversion from byte to bool, or some other mechanism to accomplish my new POCOs using bools to be mapped back to the nullable tinyints of older databases?
I am currently in the process of attempting to integrate an Entity Framework application with a legacy database that is about ten years old or so. One of the many problems that this database has (alongside having no relations or constraints whatsoever) is that almost every column is set to null, even though in almost all cases, this wouldn't make sense.
Invariably, I will encounter an exception along these lines:
The 'SortOrder' property on 'MyRecord' could not be set to a 'null' value. You must set this property to a non-null value of type 'Int32'.
I have seen many questions that refer to the exception above, but these all seem to be genuine mistakes where the developer did not write classes that properly represent the data in the database. I would like to deliberately write a class that does not properly represent the data in the database. I am fully aware that this is against the rules of Entity Framework, and that is most likely why I am having so much difficulty doing it.
It is not possible to change the schema at this point as it will break existing applications. It is also not possible to fix the data, because new data will be inserted by old applications. I would like to map the database with Entity Framework as it should be, slowly move all the applications over the next couple of years or so to rely on it for data access before finally being able to move on to the database redesign phase.
One method I have used to get around this is to transparently proxy the variable:
internal int? SortOrderInternal { get; set; }
public int SortOrder
{
get { return this.SortOrderInternal ?? 0; }
set { this.SortOrderInternal = value; }
}
I can then map the field in CodeFirst:
entity.Ignore(model => model.SortOrder);
entity.Property(model => model.SortOrderInternal).HasColumnName("SortOrder");
Using the internal keyword in this method does allow me to nicely encapsulate this nastiness so I can at the very least keep it from leaking outside my data access assembly.
But unfortunately I am now unable to use the proxy field in a query as a NotSupportedException will be thrown:
The specified type member 'SortOrder' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.
Perhaps it might be possible to transparently rewrite the expression once it is received by the DbSet? I would be interested to hear if this would even work; I'm not skilled enough with expression trees to say. I have so far been unsuccessful in finding a method in DbSet that I could override to manipulate the expression, but I'm not above making a new class that implements IDbSet and passes through to DbSet, horrible though that would be.
Whilst investigating the stack trace, I found a reference to an internal Entity Framework concept called a Shaper, which appears to be the thing that takes the data and inputs it to A quick bit of Googling on this concept doesn't yield anything, but investigating System.Data.Entity.dll with dotPeek indicates that this would certainly be something that would help me... assuming Shaper<T> wasn't internal and sealed. I'm almost certainly barking up the wrong tree here, but I'd be interested to hear if anyone has encountered this before.
That's a fairly tough nut to crack, but you might be able to do it via Microsoft.Linq.Translations.
Can someone explain why data binding against a nullable property in an ADO Data Entity Model does not seem to work. I have a decimal field named "Weight" that when it is set to allow nulls in the database, I can not seem to change the value on a windows form. However if I turn off the allow nulls in the database and update the model, the databinding works as i expect. Looking at the Designer.cs file the only difference I can tell is the underlying value seems to be a decimal versus a decimal?.
It may be related to this nasty bug (there was a microsoft connect bug report, but the EF team apparently deleted all the bug reports...)
I'm getting
System.NotSupportedException: All
objects in the EntitySet
'Entities.Message' must have unique
primary keys. However, an instance of
type 'Model.Message' and an instance
of type 'Model.Comment' both have the
same primary key value
but I have no idea what this means.
Using EF4, I have a bunch of entities of type Message. Some of these messages are actually a subtype, Comment, inheritance by table-per-type. Just
DB.Message.First();
will produce the exception. I have other instances of subtyping where I don't experience problems but I can't see any discrepencies. Sometimes, though, the problem goes away if I restart the development server, but not always.
Edit:
I've worked out (should have before) that the problem is a fault of the stored procedure fetching my Messages. The way this is currently set up as that all the fields pertaining to Message is fetched, the Comment table is ignored by the sproc. The context then proceeds to muck this up, probably by fetching those Messages that are also Comments again, as you suggested. How to do this properly is the central issue at hand. I've found some indications to a solution at http://social.msdn.microsoft.com/Forums/en-US/adodotnetentityframework/thread/bb0bb421-ba8e-4b35-b7a7-950901adb602.
As you infer, it looks like the Context is fetching a Comment as a Message (not knowing that it is a comment). Later, you ask for the actual Comment, so the context fetches the Comment. Now you have two object instances in the Context with the same ID - one is a Message and one is a Comment.
It seems that the exception is not being thrown until after both objects have been loaded (ie when you try to access the Message the second time). If you can find a way to remove the Message from Context when the Comment is loaded, this may solve your problem.
Another option might be to use the Table-per-hierarchy model. This results in a bad database design but at the end of the day you have to use what works.
You might be able to avoid the problem by ensuring that the objects are loaded as Comments first. This way, when you ask for the Message, the Context already knows about it.
Also consider using Composition over Inheritance, such that a Message has 0..1 CommentDetails.
The final suggestion is to remove the dependency on the Entity Framework from your Control code, and create a Data Access Layer which references the EF and retrieves your objects. The DAL can turn Entity Framework objects into a different set of Entity objects which are easier to use in code. This approach will produce a lot of code overhead, but may be suitable if you cannot use the Entity Framework to produce an Entity model which represents your Entities in the way you want to work with them.
To summarize, unless MS fix this issue, there is no solution to your problem which does not involve a rethink of your approach. Unfortunately the Entity Framework is not ideal, especially for complex Entity models - you might be better off creating your own DAL and bypassing the EF altogether.
It sounds like you are pulling two records into memory one into message and one into comment.
Possible prblems:
There are two physical messages with the same id
The same message is being pulled up as a message and a comment
The same message is being pulled up twice into the same context
That the problem sometimes goes away when you restart, points to a problem with cleaning up of context. Are you using "using" statements.
Do you have functionality for changing from a message to a comment?
I am not an EF kind of guy (busy working with NHibernate, haven't had time to get up to date with EF yet) so I may be totally wrong here, but could the problem be that the two tables (since you are using inheritance by table-per-type) have primary keys that collide?
If you check the data in both tables, do primary key values collide?
I have an issue with EF 4.0 that I hope someone can help with. I currently have an entity that I want to update in a last in wins fashion (i.e. ignore concurrency checks and just overwrite whats in the db with what is submitted). It seems Entity Framework not only includes the primary key of the entity in the where clause of the generated sql, but also any foreign key fields. This is annoying as it means that I don't get true last in wins semantics and need to know what value the fk field had before the update or I get a concurrency exception.
I am aware that this can be short circuited by including a foreign key field as well as the navigation property on the entity. I would like to avoid this if possible as it's not a very clean solution.
I was just wondering if there was any other way to override this behaviour? It seems like more of a bug than a feature. I have no problem with ef doing concurrency checks if I instruct it to do so but not being able to bypass concurrency completely is a bit of a hindrance as there are many valid scenarios where this is not needed
I am not 100% sure this will work, but I was curious myself and think I might have found a solution.
What if you set the [ConcurrencyCheck] attribute on a field that is immutable (in my case I used the ID (Primary Key). Since that will never be changed it will never trigger a concurrency exception and therefore clobber your entry.
Give it a whirl and see if that solves your issue.