I'm trying to migration EF to Dapper and I'm looking for a better and efficient way on how to migrate existing linq expressions "IQueryable" to use Dapper.
Ex:
public class MyDbContext : DbContext
{
public DbSet<MyEntity> MyEntity { get; set; }
+20 more entities..
}
// Inside repository
using (var context = new MyDbContext())
{
context.MyEntity.Where( x => x.Id == 1) // How can I easily migrate this linq to Dapper?
}
The above code is a simple example only of what I'm trying to migrate. Some of those queries are mix of simple and complex. Currently, I have 20+ repositories and 20+ DbSet in MyDbContext that used that kind of approach inside repositories.
I searched in the internet and I haven't found a better approach. So far the only way to do this is to convert linq into query string one by one and use it in Dapper. It is doable but tedious and huge effort. Performance is my biggest concern why I'am migrating to Dapper.
Does anyone has a better way to do this than what I'm currently thinking?
I found a medicine to my own problem. Instead of intercepting the query, I allow to pass the predicate and create Linq like function same with existing.
public class QueryLinq
{
private readonly IDatabase _database;
public QueryLinq(IDatabase database)
{
_database = database; // Dapper implementation
}
public IEnumerable<T> Where<T>(Func<T,bool> predicate)
{
var enumerable = _database.Query<T>();
return enumerable(predicate);
}
}
// Existing Repository
public class MyRepository
{
private readonly QueryLinq _queryLinq;
public MyRepository(QueryLinq queryLinq)
{
_queryLinq = queryLinq;
}
public IEnumerable<MyEntity> SelectMyEntity()
{
// Before
// using (var context = new MyDbContext())
// {
// context.MyEntity.Where( x => x.Id == 1);
// }
// Now
return _queryLinq.Where<MyEntity>( x => x.Id == 1);
}
}
In this approach, I don't need to worry about existing queries.
Update 8/16/2018: for the complete details of this approach kindly visit here.
You can use Entity Framework Logging to output the generated SQL to the Visual Studio console. It's as simple as adding:
_context.Database.Log = x => Trace.WriteLine(x);
to your service or respository class(es).
I am doing exactly what you're doing, and for the same reason. You'll see that the SQL that LINQ generates can be sub-optimal so directly copying the same sub-optimal SQL for Dapper to use seemed to me like a pointless exercise.
What I've done is identify the LINQ queries that perform the worst, and rewrite them in SQL - from scratch - for use with Dapper. What I've ended up with is a combination of LINQ and Dapper throughout the system, benefitting from the strengths of both approaches ie LINQ's rapid development, and the performance gains of Dapper and optimised SQL queries.
Related
I'm reviewing some code that was written in the EF 4 days because it stands out during performance benchmarking.
The purpose of the code is to materialize an ICollection<MyBaseClass> using Entity Framework (we're now on EF 6.1).
The code exists because references present in specific subclasses aren't materialized when retrieving
public Parent
{
public virtual ICollection<MyBaseClass>() Base { get; set; }
}
from the database, when the actual types stored are subclasses of MyBaseClass.
Example subclass:
public SubA : MyBaseClass
{
public virtual ICollection<Options> Ref1 { get; set; }
}
Currently, the code does something like this:
var parent = ctx.Parents.Include(p => p.Base).Where(...).Single();
LoadSubclasses(parent.Base);
...
private void LoadSubclasses(IEnumerable<MyBaseClass> myBase)
{
foreach (var my in myBase)
{
if (my is SubA)
{
this.Entry(my).Reference("Ref1").Load();
this.Entry((SubA)my).Ref1).Collection("Options").Load();
}
else... // Similar for other subclasses
}
}
Note that ICollection<MyBaseClass>() Base contains a mix of several concrete subclasses. There are generally a few hundred objects in the ICollection.
Is there a more efficient way to materialize Base?
It cannot be said in advance if the performance will be better (sometimes executing a single complex query, especially with sub collection includes may have actually negative impact), but you can minimize the number of database queries to K, where K is the number of subclass types that need additional includes.
You need to base the LoadSubclasses method on IQueryable<TBase> representing all base entities, and execute one query per each subclass type using OfType filter:
private void LoadSubclasses(IQueryable<MyBaseClass> baseQuery)
{
// SubA
baseQuery.OfType<SubA>()
.Include(x => x.Ref1.Options)
.Load();
// Similar for other subclasses
}
The usage with your sample would be:
var parent = ctx.Parents.Include(p => p.Base).Where(...).Single();
LoadSubclasses(ctx.Entry(parent).Collection(p => p.Base).Query());
or more generally:
var parentQuery = ctx.Parents.Where(...);
var parents = parentQuery.Include(p => p.Base).ToList();
LoadSubclasses(parentQuery.SelectMany(p => p.Base));
For those on EF Core 2.1 or later, this feature is now supported out-of-the-box.
Request from 2010:
When in an data model for entity framework has a navigation property
it is not posseble to eager load that navigation property besides when
using OfType<> or when eager loading the derived type itself by a
navigation property.
Response from 2018:
The feature is part of EF Core 2.1, which is currently in preview.
Please create issues in our issue tracker if you find any problems.
I'm a little confused as to the purpose of a data model in Entity Framework code-first. Because EF will auto-generate a database from scratch for you if it doesn't already exist using nothing more than the data model (including data annotations and Fluent API stuff in DbContext.OnModelCreating), I was assuming that the data model should fully describe your database's structure, and you wouldn't need to modify anything fundamental after that.
However, I came across this Codeplex issue in which one of the EF Triage Team members suggests that custom indexes be added in data migrations, but not as annotations to your data model fields, or Fluent API code.
But wouldn't that mean that anyone auto-generating the database from scratch would not get those custom indexes added to their DB? The assumption seems to be that once you start using data migrations, you're never going to create the database from scratch again. What if you're working in a team and a new team member comes along with a new SQL Server install? Are you expected to copy over a database from another team member? What if you want to start using a new DBMS, like Postgres? I thought one of the cool things about EF was that it was DBMS-independent, but if you're no longer able to create the database from scratch, you can no longer do things in a DBMS-independent way.
For the reasons I outlined above, wouldn't adding custom indexes in a data migration but not in the data model be a bad idea? For that matter, wouldn't adding any DB structure changes in a migration but not in the data model be a bad idea?
Are EF code-first models intended to fully describe a database's structure?
No, they don't fully describe the database structure or schema.Still there are methods to make the database fully described using EF. They are as below:
You can use the new CTP5’s ExecuteSqlCommand method on Database class which allows raw SQL commands to be executed against the database.
The best place to invoke SqlCommand method for this purpose is inside a Seed method that has been overridden in a custom Initializer class. For example:
protected override void Seed(EntityMappingContext context)
{
context.Database.ExecuteSqlCommand("CREATE INDEX IX_NAME ON ...");
}
You can even add Unique Constraints this way.
It is not a workaround, but will be enforced as the database will be generated.
OR
If you are badly in need of the attribute, then here it goes
[AttributeUsage(AttributeTargets.Property, Inherited = false, AllowMultiple = true)]
public class IndexAttribute : Attribute
{
public IndexAttribute(string name, bool unique = false)
{
this.Name = name;
this.IsUnique = unique;
}
public string Name { get; private set; }
public bool IsUnique { get; private set; }
}
After this , you will have an initializer which you will call in your OnModelCreating method as below:
public class IndexInitializer<T> : IDatabaseInitializer<T> where T : DbContext
{
private const string CreateIndexQueryTemplate = "CREATE {unique} INDEX {indexName} ON {tableName} ({columnName});";
public void InitializeDatabase(T context)
{
const BindingFlags PublicInstance = BindingFlags.Public | BindingFlags.Instance;
Dictionary<IndexAttribute, List<string>> indexes = new Dictionary<IndexAttribute, List<string>>();
string query = string.Empty;
foreach (var dataSetProperty in typeof(T).GetProperties(PublicInstance).Where(p => p.PropertyType.Name == typeof(DbSet<>).Name))
{
var entityType = dataSetProperty.PropertyType.GetGenericArguments().Single();
TableAttribute[] tableAttributes = (TableAttribute[])entityType.GetCustomAttributes(typeof(TableAttribute), false);
indexes.Clear();
string tableName = tableAttributes.Length != 0 ? tableAttributes[0].Name : dataSetProperty.Name;
foreach (PropertyInfo property in entityType.GetProperties(PublicInstance))
{
IndexAttribute[] indexAttributes = (IndexAttribute[])property.GetCustomAttributes(typeof(IndexAttribute), false);
NotMappedAttribute[] notMappedAttributes = (NotMappedAttribute[])property.GetCustomAttributes(typeof(NotMappedAttribute), false);
if (indexAttributes.Length > 0 && notMappedAttributes.Length == 0)
{
ColumnAttribute[] columnAttributes = (ColumnAttribute[])property.GetCustomAttributes(typeof(ColumnAttribute), false);
foreach (IndexAttribute indexAttribute in indexAttributes)
{
if (!indexes.ContainsKey(indexAttribute))
{
indexes.Add(indexAttribute, new List<string>());
}
if (property.PropertyType.IsValueType || property.PropertyType == typeof(string))
{
string columnName = columnAttributes.Length != 0 ? columnAttributes[0].Name : property.Name;
indexes[indexAttribute].Add(columnName);
}
else
{
indexes[indexAttribute].Add(property.PropertyType.Name + "_" + GetKeyName(property.PropertyType));
}
}
}
}
foreach (IndexAttribute indexAttribute in indexes.Keys)
{
query += CreateIndexQueryTemplate.Replace("{indexName}", indexAttribute.Name)
.Replace("{tableName}", tableName)
.Replace("{columnName}", string.Join(", ", indexes[indexAttribute].ToArray()))
.Replace("{unique}", indexAttribute.IsUnique ? "UNIQUE" : string.Empty);
}
}
if (context.Database.CreateIfNotExists())
{
context.Database.ExecuteSqlCommand(query);
}
}
private string GetKeyName(Type type)
{
PropertyInfo[] propertyInfos = type.GetProperties(BindingFlags.FlattenHierarchy | BindingFlags.Instance | BindingFlags.Public);
foreach (PropertyInfo propertyInfo in propertyInfos)
{
if (propertyInfo.GetCustomAttribute(typeof(KeyAttribute), true) != null)
return propertyInfo.Name;
}
throw new Exception("No property was found with the attribute Key");
}
}
Then overload OnModelCreating in your DbContext
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
Database.SetInitializer(new IndexInitializer<MyContext>());
base.OnModelCreating(modelBuilder);
}
Apply the index attribute to your Entity type, with this solution you can have multiple fields in the same index just use the same name and unique.
OR
You can do the migrations later on.
Note:
I have taken a lot of this code from here.
The question seems to be if there is value in having migrations added mid-stream, or if those will cause problems for future database initializations on different machines.
The initial migration that is created also contains the entire data model as it exists, so by adding migrations (enable-migrations in the Package Manager Console) you are, in effect, creating the built-in mechanism for your database to be properly created down the road for other developers.
If you're doing this, I do recommend modifying the database initialization strategy to run all your existing migrations, lest EF should start up and get the next dev's database out of sync.
Something like this would work:
Database.SetInitializer(new MigrateDatabaseToLatestVersion<YourNamespace.YourDataContext, Migrations.Configuration>());
So, no, this won't inherently introduce problems for future work/developers. Remember that migrations are just turned into valid SQL that executes against the database...you can even use script mode to output the TSQL required to make the DB modifications based on anything in the migrations you have created.
Cheers.
I'm using the newest Entity Framework and ran into a problem with Many To Many Relationship when I want to create an extra column.
The issue is the same raised in this older post:
EF Code First Additional column in join table for ordering purposes
Is it still the problem today that one can not add an extra column without loosing the many to many relation ship (link from object A to B as A.B because the mapping becomes and entity it self) ?
What are the work a rounds ?
Look up the a of class A I need and then query for mapping table where(e=>e.A == a) to get my Bs? And when I need the extra colums i would do MappingTable.find(a,b) ?
Are there other modeling options, linq to sql that would make it easier ?
As far as I know things haven't changed with EF 5. You would need to do it as the link says to. I like to stick with EF as its easy to use, but that's just my opinion...
I had the same problem. What I did to work-around it was create another derivative DbContext specifically to handle joins. I.E.:
public class JoinContext : DbContext
{
internal JoinContext() : base("name=SampleConnectionString")
{
PreventErrorIfDatabaseSchemaChanges();
// Get the ObjectContext related to this DbContext
var objectContext = (this as IObjectContextAdapter).ObjectContext;
}
public DbSet<StudentImage> StudentImages { get; set; }
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<StudentImage>()
.ToTable("StudentImages");
.HasKey(joinTable => new { joinTable.StudentId, joinTable.ImageId });
base.OnModelCreating(modelBuilder);
}
private static void PreventErrorIfDatabaseSchemaChanges()
{
Database.SetInitializer<JoinContext>(null);
}
}
I left the other application context with the Student/Image many-to-many join mapping as-is. Don't forget to specify a compounded key for the join table (refer to HasKey method above), else EF bombs on databse initialization.
After you have your special join context, use a repository to access this context and get or set the desired fields from mapped join table:
public class StudentRepository
{
public int GetImageSortOrder(int studentId, int imageId)
{
var joinContext = new JoinContext();
var joinTuple = joinContext.StudentImages.Find(studentId, imageId);
return joinTuple.SortOrder;
}
}
Hope this helps!
Does anyone have any idea how to limit result set of EntityFramework permanently? I'm speaking about something like this Conditional Mapping. This is exactly what I want to achieve with one exception: I want to do this programmatically. That's because condition value will be passed to EF only on context creation. Beside I don't want this column to disappear from mapping.
I know how to achieve this with EF2.0 and reflection. I was using CreateQuery() method to generate my own ObjectQuery. CreateQuery() allows to inject my own ESQL query with additional condition e.g. WHERE TABLE.ClientID == value.
Problem with EF40 is that there is no more ObjectQuery but only ObjectSet and CreateQuery() is not used. I have no idea how to inject my own ESQL query.
The reason why I want to limit result sets is that I want to separate clients data from each other. This separation should be done automatically inside context so that programmers will not have to add condition .Where(x => x.ClientID == 5) to each individual query.
Maybe my approach is completely bad — but I don't know any alternative.
You don't need reflection for this. You can simply use class inherited from ObjectContext or create custom implementation of UnitOfWork and Repositories which will wrap this functionality in better way (upper layer has access only to UnitOfWork and Repositories which do not expose EF context).
Simple example of object context:
public class CustomContext : ObjectContext
{
private ObjectSet<MyObject> _myObjectsSet;
private int _clientId;
public CustomContext(string connectionString, int clientId)
: base(connectionString)
{
_myObjectSet = CreateObjectSet<MyObject>();
_clientId = clientId;
}
public IQueryable<MyObject> MyObjectQuery
{
get
{
return _myObjectsSet.Where(o => o.ClientId == _clientId);
}
}
}
Could anyone advise me on how they've implemented the use of Function Imports when using the Repository pattern against EF 4.0?
We have a table mapped to a Candidate Entity and also a Function Import off an existing sproc that maps to Candidate. This works great in EF but we're abstracting by use of Repositories which take on their constructor the IObjectSet where T is the POCO entity. However this means I'm unable to get a reference to the function import. The only way I can see of doing this is passing a reference to the ObjectContext to the repositories that need it but this feels like a bit of a design smell to me.
Even though several of our Repositories are extended with custom interfaces we're still faced with the same issue.
public class CandidateRepository : Repository<Candidate>, ICandidateRepository
{
public CandidateRepository(IObjectSet<Candidate> entities)
: base(entities)
{
}
public Candidate GetByEmail(string email)
{
return Entities.SingleOrDefault(c => c.EmailAddress.Equals(email));
}
public bool CandidateExists(string candidateNumber)
{
return Entities.SingleOrDefault(c => c.Number.Equals(candidateNumber)) != null;
}
public Candidate GetByNumber(string number)
{
return Entities.SingleOrDefault(c => c.Number.Equals(number));
}
public Candidate GetMember(string number)
{
return new Candidate(); //This one needs to return from the Function Import
}
}
Any advice greatly appreciated.
To solve your problem directly you can cast entities to ObjectSet<T> and use entites.Context property to get ObjectContext.
public Candidate GetMember(string number)
{
var objectSet = Enities as ObjectSet<Candidate>;
if(objectSet == null) throw new Exception("Oh, it's not EF IObjectSet implementation");
return objectSet.Context.MyCustomFunction(string number);
}
As you can see this code relies on specific IObjectSet implementation which is not good at all.
The better idea is to create repositories for aggregate roots only rather then for each table. So it will be more natural to pass ObjectContext to repository ctor.
I have went down this route and i have experienced that it is less of a pain when you pass in an interface implementation of the ObjectContext to your repository. The interface implementation should have some way of calling the function. So when you pass in the concrete implementation of the ObjectContext everything should work fine.