I'd like to make a query to my SQL Server 2008 R2 database using Linq-To-Entities.
I use Entity Framework 4.1 (Code-First)
the query has to compare the entities' hashed values with a paramater as follows:
myContext.Messages.Where(m => HashCode(m.MessageId) % 10000 == myValue)
But I can only use core extension methods and CLR methods defined for the EF provider.
So I'm trying to translate my HashCode() method to something that L2E understands just before execution.
FYI, HashCode's implementation is not complex. something like:
public int HashString(string text)
{
int hash = 23;
foreach (char c in text)
{
hash = hash * 31 + c;
}
return hash;
}
I know I can retrieve all the entities as enumerable and check them in my code (Linq To Objects), But that's be a disaster considering the amount of data I'm about to retrieve that way.
Is it doable at all (for GetHashCode method). If so, how can I implement it?
Thank you.
Could you write SQL to do this? I don't see how (without a SQL function, which requires modifying DB schema, which you say you can't do).
If you can't do it in SQL, you certainly can't do it in L2E, since all L2E is translated to SQL.
On the other hand, if you can do it in SQL, the fastest solution is probably ObjectContext.ExecuteStoreQuery.
Related
Suppose I want to enhance a model like
public class Person
{
public string Name { get; set; }
public string Address { get; set; }
}
so that I use a complex object for Address:
public class Person
{
public string Name { get; set; }
public Address Address { get; set; }
}
EF Core is quite nice in allowing this with the HasConversion feature:
modelBuilder.Entity<Person>
.Property(p => p.Address)
.HasConversion(addr => addr.ToString(), str => Address.FromString(str));
I even tested and this works in queries with == operator: The following will successfully convert to SQL
var whiteHouse = Address.Parse("1600 Pennsylvania Avenue NW");
var matches = from person in people
where person.Address == whiteHouse
select person;
However, suppose I want to string.Contains on the string version of Address, something like
var search = "1600";
var matches = from person in people
where person.Address.ToString().Contains(search)
select person;
This will fail to convert. Is there any feature of EF Core to map the ToString() method or otherwise map a complex object that converts to a string / VARCHAR so that I can write a query like this?
The problem with EF Core value converters and LINQ queries is that the LINQ query is against the CLR entity property, hence the CLR type rather than the provider type. This is partially mentioned under currently Limitations of the value conversion system section of the EF Core documentation:
Use of value conversions may impact the ability of EF Core to translate expressions to SQL. A warning will be logged for such cases. Removal of these limitations is being considered for a future release.
So having query expression against the CLR type combined with the inability to translate custom methods is causing your issue. Technically it's possible to add custom method/property translation, but it's quite complicated because requires a lot of non user friendly infrastructure plumbing code, which makes practically unusable in real life application development.
In this particular case though, you know that the provider type is string, and the database table values are generated by ToString method. So you just need to let the query use the provider type. And you can do that by using cast operator.
Normally C# compiler won't allow you to cast known object type to another known object type if there is no conversion between them. But you can trick it by using the "double cast" technique by first casting to object and then to the desired type. Fortunately EF Core translator supports such casts and properly (sort of) translates them to SQL. By sort of I mean it emits unnecessary (redundant) CAST inside the query, but at least it translates and executes server side.
With that being said, the solution for your example is
where ((string)(object)person.Address).Contains(search)
As a default behavior, EF Core use Server-side evaluation, EF Core try to translate your expression to standard DB provider T-SQL code (based on selected DB provider)
you expression can't translate to T-SQL code and the DB provider can't handle it (Because the logic you write in the overridden version of ToString() is in your C# code and is unknown to the database provider)
You should force EF Core to use client-side evaluation by fetching all data to memory and then query on the loaded entities, something like this:
var search = "1600";
var matches = from person in people.ToList()
where person.Address.ToString().Contains(search)
select person;
Note that fetching all data to memory in huge databases have performance impacts and use client-side evaluation carefully.
I am thinking of designing a business rule engine which basically generates an EF query from a set of string values stored in a database.
For e.g. I will store the connection string, table name, the where condition predicate, and select predicate as string fields in a db and would like to construct the EF query dynamically. For e.g.
var db = new DbContext(“connectionstring”);
var wherePredicate = Expression.FromString(“p => p.StartDate > new DateTime(2014,5,1))
var selectPredicate = Expression.FromString(“p => p”)
var results = db.Set(“Projects”).Where(wherepredicate).Select(selectPredicate)
For constructing the predicates I can use DynamicExpression or Dynamic LINQ library.
However how do I access db.Set(“Projects”) where Projects is the entity name and apply the where and select predicates? (or something like db[“Projects”].Where().Select).
I tried the non-generic version of the DbContext.Set(Type entityttype) method, however couldn’t figure out how to apply Where and Select predicates to the returned object.
I am trying to avoid generating SQL queries and instead rely on dynamically generated EF code.
This doesn't make much sense. You can create method that will work on string instead of generic type using reflection, but you'd have to return DbSet not DBSet<T>. And on that one you cannot execute LINQ's methods (basically), because there's no type (during compilation). Of course you can do it all the way using reflection, but then, why??? You're loosing 90% of what O/R mapper does for you.
I am trying to duplicate the following SQL statement as a LINQ to Entities query (where "PRODUCTS" is the table mapped to the entity) ... NOTE IQueryable ... most of what I have seen posted as solutions convert either the search parameters, or the dump the results into an IEnumerable and then proceed to convert from there. I am dealing with 100's of millions of records and cannot afford to load 200 million records into memory, only to have to filter through them again. I would like, if possible to do this in a single query to the databse.
select *
from PRODUCTS
where
MODEL_CODE = '65' and
CAST(SERIAL_NUMBER as int) > 927000 and
CAST(SERIAL_NUMBER as int) < 928000
I have tried the following ...
int startSN, endSN;
startSN = 9500
endSN = 9500
if (!int.TryParse(startSerialNumber, out startSN))
throw new InvalidCastException("The start serial number was not a valid value");
if (!int.TryParse(endSerialNumber, out endSN))
throw new InvalidCastException("The end serial number was not a valid value");
IQueryable<PRODUCT> resultList = base.Context.PRODUCTS.Where(b =>
(Convert.ToInt32(b.SERIAL_NUMBER) > startSN) &&
(Convert.ToInt32(b.SERIAL_NUMBER) < endSN)).AsQueryable();
I have tried a couple of other version of things similiar to this with no luck. I have looked at the following posts also with no luck.
Convert string to int in an Entity Framework linq query and handling the parsing exception - the solution converts query to a list before converting the entity properties.
Convert string to Int in LINQ to Entities ? -
This problem was just with converting the parameters which can be easily done outside the LINQ to Entities statement. I am already doing this for the parameters.
LINQ to Entities StringConvert(double)' cannot be translated to convert int to string - This problem is actually the reverse of mine, trying to convert an int to a string. 1) SqlFunctions does not provide a function for converting TO an int. 2) Ultimately the solution is to, again convert to an IEnumerable before converting/casting the values.
Anybody got any other ideas? I am little stumped on this one!
Thank you,
G
If you don't use code-first, but an EDMX based approach model defined functions are probably the best solution: Convert String to Int in EF 4.0
Alternatively you can use...
base.Context.PRODUCTS.SqlQuery(string sql, params object[] parameters)
...and then pass in the raw SQL statement from your question.
DbSet<T>.SqlQuery(...) returns a DbSqlQuery<T> as result. It is important to keep in mind that this type does not implement IQueryable<T>, but only IEnumerable<T>. Its signature is:
public class DbSqlQuery<TEntity> : IEnumerable<TEntity>, IEnumerable, IListSource
where TEntity : class
So you can extend this result with further LINQ methods, but it is only LINQ to Objects that will be executed in memory with the returned result set from the SQL query. You can not extend it with LINQ to Entities that would be executed in the database. Hence, adding .Where filters to DbSqlQuery<T> does not have any influence on the database query and the set of data that is loaded from the DB into memory.
That's actually not surprising as it would mean otherwise that a partial expression tree (from a Where method) had to be translated into SQL and then merged into a hand-written SQL statement so that a correct new composed SQL statement results and could be sent to the database. Sounds like a pretty hard task to me.
(as advised re-posting this question here... originally posted in msdn forum)
I am striving to write a "generic" routine for some simple CRUD operations using EF/Linq to Entities. I'm working in ASP.NET (C# or VB).
I have looked at:
Getting a reference to a dynamically selected table with "GetObjectByKey" (But I don't want anything from cache. I want data from database. Seems like not what this function is intended for).
CRM Dynamic Entities (here you can pass a tablename string to query) looked like the approach I am looking for but I don't get the idea that this CRM effort is necessarily staying current (?) and/or has much assurance for the future??
I looked at various ways of drilling thru Namespaces/Objects to get to where I could pass a TableName parameter into the oft used query syntax var query = (from c in context.C_Contacts select c); (for example) where somehow I could swap out the "C_Contacts" TEntity depending on which table I want to work with. But not finding a way to do this ??
Slightly over-simplyfing, I just want to be able to pass a tablename parameter and in some cases some associated fieldnames and values (perhaps in a generic object?) to my routine and then let that routine dynamically plug into LINQ to Entity data context/model and do some standard "select all" operations for parameter table or do a delete to parameter table based on a generic record id. I'm trying to avoid calling the various different automatically generated L2E methods based on tablename etc...instead just trying to drill into the data context and ultimately the L2E query syntax for dynamically passed table/field names.
Has anyone found any successful/efficient approaches for doing this? Any ideas, links, examples?
The DbContext object has a generic Set() method. This will give you
from c in context.Set<Contact>() select c
Here's method when starting from a string:
public void Test()
{
dynamic entity = null;
Type type = Type.GetType("Contract");
entity = Activator.CreateInstance(type);
ProcessType(entity);
}
public void ProcessType<TEntity>(TEntity instance)
where TEntity : class
{
var result =
from item in this.Set<TEntity>()
select item;
//do stuff with the result
//passing back to the caller can get more complicated
//but passing it on will be fine ...
}
This is a follow-up to an earlier question I posted on EF4 entity keys with SQL Compact. SQL Compact doesn't allow server-generated identity keys, so I am left with creating my own keys as objects are added to the ObjectContext. My first choice would be an integer key, and the previous answer linked to a blog post that shows an extension method that uses the Max operator with a selector expression to find the next available key:
public static TResult NextId<TSource, TResult>(this ObjectSet<TSource> table, Expression<Func<TSource, TResult>> selector)
where TSource : class
{
TResult lastId = table.Any() ? table.Max(selector) : default(TResult);
if (lastId is int)
{
lastId = (TResult)(object)(((int)(object)lastId) + 1);
}
return lastId;
}
Here's my take on the extension method: It will work fine if the ObjectContext that I am working with has an unfiltered entity set. In that case, the ObjectContext will contain all rows from the data table, and I will get an accurate result. But if the entity set is the result of a query filter, the method will return the last entity key in the filtered entity set, which will not necessarily be the last key in the data table. So I think the extension method won't really work.
At this point, the obvious solution seems to be to simply use a GUID as the entity key. That way, I only need to call Guid.NewGuid() method to set the ID property before I add a new entity to my ObjectContext.
Here is my question: Is there a simple way of getting the last primary key in the data store from EF4 (without having to create a second ObjectContext for that purpose)? Any other reason not to take the easy way out and simply use a GUID? Thanks for your help.
I ended up going with a GUID.
The size/performance issues aren't
critical (or even noticeable) with SQL Compact, since
it is a local, single-user system.
It's not like the app will be
managing an airline reservation
system.
And at least at this point, there
seems to be no way around the "no
server-generated keys" limitation of
the SQL Compact/EF4 stack. If someone has a clever hack, I'm still open to it.
That doesn't mean I would take the same approach in SQL Server or SQL Express. I still have a definite preference for integer keys, and SQL Compact's bigger siblings allow them in conjunction with EF4.
Use a Guid. AutoIncrement is not supported on Compact Framework with Entity Framework.
Also, if you ever want to create a application which uses multiple data sources, int PK's are going to fall apart on you very, very quickly.
With Guid's, you can juse call Guid.NewGuid() to get a new key.
With int's, you have to hit the database to get a valid key.
If you store data in multiple databases, int PK's will cause conflicts.
What I've done for SQL CE before, and I assume we have a single application accessing the database, is to calculate the MAX value on startup and put it in a static variable. You can now hand out sequential values easily and you can make the code to generate them thread safe very easily.
One reason to avoid Guids would be size = memory and storage space consumption.
You could also query SQL Compact metadata like so:
SELECT AUTOINC_NEXT FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'Categories' AND AUTOINC_NEXT IS NOT NULL