I'm trying to figure out how to cache EF query results in an Azure AppFabric cache. Currently I'm using the LoreSoft EntityFramework extensions to deal with the caching (http://bit.ly/LWSywm). It works perfectly with an in memory cache, but not so much with AppFabric across multiple VM's. The issue is that I've got virtual properties in my EF objects, and they're being serialized as Dynamic Proxy objects, which of course can't be deserialized on a different VM, or even after a single VM restarts the application.
I only have a few queries I need to cache, so I'd rather not load every single related object manually across the whole project. Is there any way I can serialize an EF object with virtual properties? I don't need the virtual properties to magically start lazy loading again after I deserialize them. I've tried turning off lazy loading the DbContext before serializing the results, but that doesn't work. The only way I've found to get a serializable EF object is the remove all virtual properties.
By the way I've looked at the Julie Lerman article here: http://bit.ly/LWToZT
Seems like a cool project, but I'm not entirely sure it's going to solve my problem of not being able to serialize EF objects. Don't want to go down that road if I'm just going to end up where I started.
Any ideas most appreciated!
You can configure your dbcontext (or object context) to not use proxy objects. Obviously, this means no change tracking and no lazy loading. If you don't eager load an object's navigation properties, they will simply be null instead of references to proxies. It's worth noting that you can toggle this on and off throughout the context's lifetime, so it's not an all or nothing decision.
If you are using DbContext, the syntax is:
context.Configuration.ProxyCreationEnabled = false;
We have had a similar problem. We had a service that returned data that was read from a database using EF. This data needed to be cached in Appfabric.
In our case the services returned DTO objects, so we mapped the EF objects to DTO objects, and then stored the DTO objects in Appfabric cache.
Related
I am facing an issue: we have an ASP.NET Core 2.2 Web API project with EF Core 2.2. We are using default IOC framework to create the DbContext with scope lifetime. And we have a socket pipeline connected to our ASP.NET Web API service.
I find that when we change the data in the web frontend, the socket pipeline will always get the old result (we are using .FirstOrDefault() to fetch the data, it should not be the problem with first-level cache).
So I infer that it might be because of that the scope lifetime for DbContext, so I changed it to transient lifetime. And it works! We get the modified record.
I have two questions:
Is that behavior of DbContext by design？ Or maybe I have some tricky issue in my code.
How much performance will the transient lifetime DbContext cost？ Since maybe I will make every DbContext transient
1) Is that behavior of DbContext by design？
Yes
For each item in the result set If this is a tracking query, EF checks
if the data represents an entity already in the change tracker for the
context instance If so, the existing entity is returned If not, a new
entity is created, change tracking is setup, and the new entity is
returned
How Queries Work
2) How much performance will the transient lifetime DbContext cost？
Very little. Especially in ASP.NET Core, which has DbContext Pooling
Since maybe I will make every DbContext transient
But you shouldn't do that. Using a request-scoped DbContext is very useful. For instance you can use the DbContext in various layers of your application without having to pass one around, and you can manage transactions more easily.
I'm using EF (EF Core, actually, with ASP.NET Core on OSX, but I believe this is more of a general "newbie-style" EF question, so please read on...)
I built a little logging routine that uses EF to publish log entries to my database. Sort of like this, called from a repository class:
WebLog log = new WebLog(source, path, message);
Context.WebLogs.Add(log);
Context.SaveChanges();
Where WebLog is a simple model class, Context.WebLogs is a DbSet<WebLog> collection, and Context is obviously the DbContext. I believe this is quite straightforward.
But my question is this: if I continue to add new log entries to the Context.WebLogs collection and I never do anything like reboot my server, isn't the collection just going to grow without bounds? Is there some kind of "purge" or "flush" action I can take periodically to manage memory usage (without affecting the committed rows in the database, of course--I want those to persist). Or is DbSet some sort of a special collection that won't do this?
As mentioned by DevilSuichiro above, the recommended approach is to limit the lifetime of the instances of DbContext. E.g. in a Web application you typically use a DbContext instance per request, so an unbounded number of entities added doesn't become a problem.
The closest thing to a "flush" operation is SaveChanges() that method will not try to remove references to tracked entities, as DbContext is designed to be reused after SaveChanges().
In previous versions of EF we had a Detach() API that you could use to get rid of an individual tracked reference but we don't have that API in DbContext or anywhere in EF Core.
BTW, having an instance of DbContext that is shared between multiple requests is extremely problematic because DbContext is not thread safe.
Is there a way to configure a DbContext to eagerly load all of its entities? we have attempted:
Configuration.LazyLoadingEnabled = false;
Configuration.ProxyCreationEnabled = false;
We tried both with and without ProxyCreationEnabled being turned off.
I just happen to know that the DB we are loading in is not very large, doesn't change very often, and is referenced a lot. So keeping it all in memory will be very advantageous.
No, LazyLoadingEnabled disabled does not mean "EagerLoadingEnabled". It is: "not even lazy loading enabled, only loading explicitly requested by you". An EF context will never automatically do eager loading. Just think what could happen with many associations between entities and large databases.
You won't benefit much from pre-loading everything into your context unless you use methods to access the local entities only. You can access local entities by DbSet.Local or DbSet.Find.
Just accessing a DbSet (like context.Customers) will execute a database query anyway. If lazy loading is disabled it may use local entities for navigation properties (called relationship fixup) but otherwise accessing navigation properties will also cause database queries.
It sounds like you should populate an object graph from your database, using a context, and cache it outside the context. Note that if you fetch the data read-only you may want to use the AsNoTracking extension method. (Like context.Customers.AsNoTracking()).
Nope... but if you're looking for the most performant way to load bulk data into EF DbContext, I recommend writing a stored procedure that returns multiple result sets (one for each entity type). EF will automatically hook up the nav properties when the entities are loaded into the context.
Here's instruction on how to do it from both the DB first and code first perspectives:
http://msdn.microsoft.com/en-us/data/jj691402.aspx
we use Azure Caching directly (and not through one of the available Entity Framework wrappers). Apparently, for distributed caching, we need to serialize the objects. Unfortunately, this causes issues with lazy-loaded DbContext-based proxies used for navigation properties.
I see we can use a custom serializer in order to map proxies to empty collections (if not loaded) or to normal objects (if loaded), but I am not sure about the implementation. One possible implementation can be based on the one used by WCF, but I am not sure Azure works the same way.
The ideal solution (and that's why I point to ProxyDataContractResolver) would be one where, when serialization happens:
IF the navigation property has been already loaded the data would be serialized as if it were a normal Collection,
and if they are not loaded, they won't be serialized (I would like lazy loading to work back after deserialization for the latter case, but it's acceptable if it doesn't).
Has anyone manually fixed that problem in an elegant way?
Thanks in advance!
I will presume that if you are wanting to cache EF objects, you don't require lazy loading or change tracking on those entities.
I believe that both of those are enabled through object proxies that will cause serialization issues (since you don't want to serialize the proxy).
If you disable the property DbContext.Configuration.ProxyCreationEnabled then serialization of the actual object, not the proxy, should work fine. This is typically required when returning POCO objects over WCF but is likley the same for other serializations scenarios such as this.
If you detach the EF entity from the DbContext before serializing it, that disables lazy loading, so your custom serializer won't try to serialize anything that isn't already part of the entity's graph.
Then when you get it back from the cache, if you attach it to a new (identical) DbContext, that should reenable lazy loading.
(Caveat: once you detach the entity from the context, any new queries that include that same object will create a new, attached, copy, so you will need to code with some care to avoid running into trouble with multiple potentially-different versions of the same object running around. But that said, this should let you do what you want.)
I've just been talking with a colleague about Entity Framework change tracking. We eventually figured out that my context interface should have
IDBSet<MyPoco> MyThings { get; }
rather than
IQueryable<MyPoco> MyThings { get; }
and that my POCO should also have all it's properties as virtual.
Using the debugger we could then see the tracking objects and also that the results contained proxies to my actual POCOs.
If I don't have my POCO properties as virtual and have my context interface using IQueryable<> instead of IDbSet<> I don't get any of that.
In this instance I am only querying the database, but in the future will want to update the database via Entity Framework.
So, to make my life easier in the future when I come to look at this code as a reference, is there any performance penalty in having the tracking info/proxies there when I will never make use of them?
There is a performance penalty of tacking entities in EF. When you query using entity framework EF will keep a copy of values loaded from database. Also single Context instance keeps track of only single instance of an entity. So EF has to check whether it already has a copy of the entity before it creates an instance(ie. There will be lot of comparisons going behind the scenes).
So avoid it if you don't need it. You can do so as follows.
IQueryable<MyPoco> MyThings { get { return db.MyThings.AsNoTracking(); } }
MSDN page on Stages of Query Execution details the cost associated with each step of query execution.
Edit:
You should not expose IDBSet<MyPoco> MyThings because that tells the consumer of your API that your entities can be added, updated and deleted when in fact you intend to query the data.
Navigation properties in the model classes as declared as virtual so as to imply lazy load feature which means the navigation property will only be needed if required. As far as the Entity objects are concerned, there main aim is to load the specific table records from the database into the DbSet which comes from DbContext. You can't use IQueryable in this case. Also, it doesn't make any sense with the DataContext. IQueryable is an altogether different interface