I have a custom CLR Aggregate function on SQL Server to calculate percentiles. Is it possible to call my custom aggregate function through Entity Framework? How is the mapping configured to allow this?
I have tried using codefirstfunctions similar to what is described on Entity Framework 6 Code First Custom Functions, however the functions seem to only be allowed to take scaler parameters, where my function is an aggregate function so will need to take a list of items (similar to how Sum, Averagg and Count work).
The Aggregate functions has the following signature, taking in the value we want the median from and the percentile (50 is median, 25 lower quartile, 75 upper quartile)
CREATE AGGREGATE [dbo].[Percentile]
(#value [float], #tile [smallint])
RETURNS[float]
EXTERNAL NAME [SqlFuncs].[Percentile]
GO
I have tried adding a DbFunctionAttribute, but not entirely sure how to hook it up to entity framework store model using code first.
[DbFunction("SqlServer", "Percentile")]
public static double? Percentile(IEnumerable<int?> arg, int tile)
{
throw new NotSupportedException("Direct calls are not supported.");
}
What I am looking for is to be able to write something like
paymentsTable
.GroupBy(x=>x.CustomerId)
.Select(new{
Median = MyDbContext.Percentile(x.Select(g=>g.Amount), 50)
});
Which will map to SQL like
SELECT [dbo].[Percentile](Amount, 50) as Median
FROM Payments
GROUP BY CustomerId
As #srutzky alluded to in the comments, EF doesnt seem to like binding to aggregate functions with multiple parameters. So you have to change percentile function to a median function or whatever fixed percentile you are interested (you will need to update your SqlClr function so the parameters match as well)
public class MySqlFunctions
{
[DbFunction("dbo", "Median")]
public static float? Median(IEnumerable<float?> arg)
{
throw new NotSupportedException("Direct calls are not supported.");
}
}
The next step is letting EF know that a the database has a function called median We can do this in our DbContext. Create a new convention to access the the dbModel then we add the function in the dbModel. You must make sure the parameters and the parameter types match both the SQL and the C# function exactly.
public class EmContext : DbContext
{
...
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
//Register a convention so we can load our function
modelBuilder.Conventions.Add(new AddMedianFunction());
...
}
public class AddMedianFunction : IConvention, IStoreModelConvention<EntityContainer>
{
public void Apply(EntityContainer item, DbModel dbModel)
{
//these parameter types need to match both the database method and the C# method for EF to link
var edmFloatType = PrimitiveType.GetEdmPrimitiveType(PrimitiveTypeKind.Single);
//CollectionType constructor is internal making it impossible to get a collection type.
//We resort to reflection instantiation.
var edmFloatListType = CreateInstance<CollectionType>(edmFloatType);
var medianfunction = EdmFunction.Create("Median", "dbo", DataSpace.SSpace, new EdmFunctionPayload
{
ParameterTypeSemantics = ParameterTypeSemantics.AllowImplicitConversion,
IsComposable = true,
IsAggregate = true,
Schema = "dbo",
ReturnParameters = new[]
{
FunctionParameter.Create("ReturnType", edmFloatType, ParameterMode.ReturnValue)
},
Parameters = new[]
{
FunctionParameter.Create("input", edmFloatListType, ParameterMode.In),
}
}, null);
dbModel.StoreModel.AddItem(medianfunction);
dbModel.Compile();
}
public static T CreateInstance<T>(params object[] args)
{
var type = typeof(T);
var instance = type.Assembly.CreateInstance(
type.FullName, false,
BindingFlags.Instance | BindingFlags.NonPublic,
null, args, null, null);
return (T)instance;
}
}
}
With all that in place you should just be able to call your function as expected
paymentsTable
.GroupBy(x=>x.CustomerId)
.Select(new{
Median = MySqlFunctions.Median(x.Select(g=>g.Amount))
});
Note: I am already assume you have loaded your SqlClr function which I have not covered here
Related
I have the following child object that we use an expression to map our 'entity' to our 'domain' model. We use this when specifically calling our ChildRecordService method GetChild or GetChildren:
public static Expression<Func<global::Database.Models.ChildRecord, ChildRecord>> MapChildRecordToCommon = entity => new ChildRecord
{
DateTime = entity.DateTime,
Type = entity.Type,
};
public static async Task<List<ChildRecord>> ToCommonListAsync(this IQueryable<global::Database.Models.ChildRecord> childRecords)
{
var items = await
childRecords.Select(MapChildRecordToCommon).ToListAsync().EscapeContext();
return items;
}
public async Task<List<ChildRecord>> GetChildRecords()
{
using (var uow = this.UnitOfWorkFactory.CreateReadOnly())
{
var childRecords= await uow.GetRepository<IChildRecordRepository>().GetChildRecords().ToCommonListAsync().EscapeContext();
return childRecords;
}
}
So that all works just fine. However we have another object that is a parent to that child, that in SOME cases, we also wish to get the child during the materialisation and mapping process.
In other words the standard object looks as such:
private static Expression<Func<global::Database.Models.Plot, Plot>> MapPlotToCommonBasic = (entity) => new Plot
{
Id = entity.Id,
Direction = entity.Direction,
Utc = entity.Utc,
Velocity = entity.Velocity,
};
However what I also want to map is the Plot.ChildRecord property, using the expression MapChildRecordToCommon I have already created. I made a second expression just to test this:
private static Expression<Func<global::Database.Models.Plot, Plot>> MapPlotToCommonAdvanced = (entity) => new Plot
{
ChildRecord = MapChildRecordToCommon.Compile() (entity.ChildRecord)
};
This fails:
System.NotSupportedException
The LINQ expression node type 'Invoke' is not supported in LINQ to Entities.
Is there a way to reuse my existing expression for ChildRecord, to materialise the object of ChildRecord (ie. one to one/singular not multiple) on the Plot object? I think my trouble is caused by there being just one object and being unable to use the .Select(Map) method. I am not too great at expressions and have hit a wall with this.
For reference, there are actually up to 5 or 6 other child objects on the "Plot" object that I also want to make expressions for.
I resolved this by using the third party library LinqKit.
The library allowed the use of 2 methods, .AsExpandable() (which allows for the expressions to properly compile and be invoked as I understand), and .Invoke() as an extension method to an expression, rather than calling Expression.Invoke(yourexpression). I included a null check just in case.
My code now looks as follows:
public static async Task<List<Plot>> ToCommonListAsync(this IQueryable<global::Database.Models.Plot> plots)
{
var items = await
plots.AsExpandable().Select(MapPlotToCommon).ToListAsync().EscapeContext();
return items;
}
private static Expression<Func<global::Database.Models.Plot, Plot>> MapPlotToCommon = (entity) => new Plot
{
Id = entity.Id,
Direction = entity.Direction,
Utc = entity.Utc,
Velocity = entity.Velocity,
ChildRecord = entity.ChildRecord != null ? MapChildRecordToCommon.Invoke(entity.ChildRecord) : default
};
public static Expression<Func<global::Database.Models.ChildRecord, ChildRecord>> MapChildRecordToCommon = entity => new ChildRecord
{
DateTime = entity.DateTime,
Type = entity.Type,
};
I am currently using Mehdi El Gueddari's DbContextScope project, I think by the book, and it's awesome. But I came across a problem I'm unsure how to solve today. I have a query that I need to execute using a different database login/user because it requires additional permissions. I can create another connection string in my web.config, but I'm not sure how to specify that for this query, I want to use this new connection string. Here is my usage:
In my logic layer:
private static IDbContextScopeFactory _dbContextFactory = new DbContextScopeFactory();
public static Guid GetFacilityID(string altID)
{
...
using (_dbContextFactory.CreateReadOnly())
{
entity = entities.GetFacilityID(altID)
}
}
That calls into my data layer which would look something like this:
private AmbientDbContextLocator _dbcLocator = new AmbientDbContextLocator();
protected CRMEntities DBContext
{
get
{
var dbContext = _dbcLocator.Get<CRMEntities>();
if (dbContext == null)
throw new InvalidOperationException("No ambient DbContext....");
return dbContext;
}
}
public virtual Guid GetFaciltyID(string altID)
{
return DBContext.Set<Facility>().Where(f => f.altID = altID).Select(f => f.ID).FirstOrDefault();
}
Currently my connection string is set in the default way:
public partial class CRMEntities : DbContext
{
public CRMEntities()
: base("name=CRMEntities")
{}
}
Is it possible for this specific query to use a different connection string and how?
I ended up modifying the source code in a way that feels slightly hacky, but is getting the job done for now. I created a new IAmbientDbContextLocator with a Get<TDbContext> method override that accepts a connection string:
public TDbContext Get<TDbContext>(string nameOrConnectionString) where TDbContext : DbContext
{
var ambientDbContextScope = DbContextScope.GetAmbientScope();
return ambientDbContextScope == null ? null : ambientDbContextScope.DbContexts.Get<TDbContext>(nameOrConnectionString);
}
Then I updated the DbContextCollection to pass this parameter to the DbContext's existing constructor overload. Last, I updated the DbContextCollection maintain a Dictionary<KeyValuePair<Type, string>, DbContext> instead of a Dictionary<Type, DbContext> as its cached _initializedDbContexts where the added string is the nameOrConnectionString param. So in other words, I updated it to cache unique DbContext type/connection string pairs.
Then I can get at the DbContext with the connection I need like this:
var dbContext = new CustomAmbientDbContextLocator().Get<CRMEntities>("name=CRMEntitiesAdmin");
Of course you'd have to be careful your code doesn't end up going through two different contexts/connection strings when it should be going through the same one. In my case I have them separated into two different data access class implementations.
Is there any way how to Group By multiple columns dynamically?
Eg. group x by new { x.Column1, x.Column2 }
but the x.Column1 etc. I want to set dynamically (from UI)
The way to achieve this dynamically on db site is quite complicated as we cannot dynamically create anonymous types. To replace them I would suggest to create a class:
public class CustomTuple<T1, T2>
{
public T1 Item1 { get; set; }
public T2 Item2 { get; set; }
}
We cannot use Tuple here as it does not have default constructor. In CustomTuple class place as much parameters T and as much properties as you would need at max. If you will define in that class 5 properties but for the query you will use only 3 you just set only 3 properties to proper values and the remaining 2 properties you keep null - the query will still work. Alternatively you may dynamically at run time generate proper class with CodeDOM. Then comes query logic:
Type[] parameterTypes = new Type[] { typeof(int), typeof(object) };
Type tupleType = typeof(CustomTuple<,>).MakeGenericType(parameterTypes);
ParameterExpression x = Expression.Parameter(typeof(Entity));
NewExpression body = Expression.New(tupleType.GetConstructor(new Type[0]), new Expression[0]);
MemberBinding binding1 = Expression.Bind(
typeof(CustomTuple<,>).MakeGenericType(parameterTypes).GetProperty("Item1"),
Expression.Property(x, "Value"));
MemberInitExpression memberInitExpression =
Expression.MemberInit(
body,
binding1);
Expression<Func<Entity, object>> exp = Expression.Lambda<Func<Entity, object>>(memberInitExpression, x);
using (MyDbContext context = new MyDbContext())
{
var list = context.Entities.GroupBy(exp).ToList();
}
The above code groups Entities by Value property. parameterTypes may be dynamically build during program execution - this is list of types of properties anonymous type for key selection in group by would have. Basing on that we create proper CustomTuple type. Then we dynamically create at run time binding1 elements - one per each property to be set for grouping key. In the example above I create only one. With use of the NewExpression and MemberBinding expression we may build initialization expression with MemberInit method. Finally you build lambda expression from that and execute it against db.
I need to execute a dynamic set type, here's what I'm trying to do (pseudo):
var type = GetSetType(); //System.Type
var set = context.Set(type);
var results = set.ToArray();
I know this can't work for sure, Enumerable ex. methods are only for generic IEnumerables, but I tried set.AsQueryable().Cast<object>().ToArray(), and a NotSupportedException was thrown: "LINQ to Entities only supports casting EDM primitive or enumeration types.".
Any way to execute a non-generic DbSet?
I should have deleted my question since the answer is too simple, but for the record, let's just keep it up.
namespace System.Linq
{
public static class EntityFrameworkExtensions
{
public static IEnumerable<object> AsEnumerable(this DbSet set)
{
foreach (var entity in set)
yield return entity;
}
}
}
Usage:
var collection = context.Set(entityType).AsEnumerable();
Beware that any filtering performed on the retuned collection will not happen on the SQL server, but on the collection enumerator, the entire table will be returned. Use only when loading all rows anyway.
I am trying to use predicateBuilder with next expression definition but I always got the message
"LINQ to Entities does not recognize the method 'puedeConsultar' method, and this method cannot be translated into a store expression."
I think i understand more less this problem, but i don´t know how to solve it.
private static readonly IDictionary<int, List<string>> permisosAccesoSolicitudesEstado = new Dictionary<int, List<string>>(){{0, new List<string>(){"A"}}, {1, new List<string>(){"B"}}};
private static bool esPermisoConcedido(List<string> usuariosPermitidos, string erfilUsuario)
{
return usuariosPermitidos.Any(x => x.Equals(perfilUsuario) || perfilUsuario.StartsWith(x + "|") || perfilUsuario.EndsWith("|" + x));
}
public static bool puedeConsultar(int estadoActual, string perfilUsuario)
{
List<string> usuariosPermitidos = permisosAccesoSolicitudesEstado[estadoActual];
return esPermisoConcedido(usuariosPermitidos, perfilUsuario);
}
public static bool puedeConsultar(string estadoActual, string tipoUsuario)
{
return puedeConsultar(Convert.ToInt32(estadoActual), tipoUsuario);
}
public Expression<Func<Solicitud, Boolean>> predicadoEstadoCorrectoSolicitud(string perfil)
{
return x=> EstadosSolicitud.puedeConsultar(x.estado, perfil);
}
//Instantiated by reflection, this works fine
MethodInfo method = .....
Expression<Func<T, bool>> resultado = ConstructorPredicados.True<T>();
resultado = ConstructorPredicados.And(resultado, method);
objectSet.Where(resultado).ToList();
Note:
ConstructorPredicados is based in Monty´s Gush "A universal PredicateBuilder" on http://petemontgomery.wordpress.com/2011/02/10/a-universal-predicatebuilder/
Thanks in advance
You cannot do that. Your puedeConsultar is .NET function. You cannot execute .NET functions in Linq-to-entities query. When you use method in Linq-to-entities you can use only methods which has direct mapping to SQL. It means that method in the query is only placeholder which is translated to execution of some SQL function. There is set of predefined method mappings called cannonical functions and you can map your own SQL function when using EDMX but in your case you will most probably have to first load data to application by using ToList and after that execute predicadoEstadoCorrectoSolicitud on materialized result.