I have very simple Linq query that shows error on run time
the query is
// string[] Blocks = _TableView.permission.BlockList.Split(',');
// string[] Permissions = _TableView.permission.PermissionsList.Split(',');
string[] Blocks = ... some string array ...;
string[] Permissions = ... some string array ... ;
var test = await (from t in _context.DocumentIn
where
Permissions.Contains(t.Imp_Exp.ToString()) &&
Permissions.Contains(t.confLevel.ToString()) &&
Blocks.Contains(t.BlockName)
select t).ToListAsync();
the error is
could not be translated. Either rewrite the query in a form that can be translated,
or switch to client evaluation explicitly by inserting a call to either AsEnumerable(),
AsAsyncEnumerable(), ToList(), or ToListAsync().
some notes
when I trace I find Blocks and Permissions full of values.
when I remove from the query two conditions of Permissions it works like the following
var test = await (from t in _context.DocumentIn
where
/*Permissions.Contains(t.Imp_Exp.ToString()) &&
Permissions.Contains(t.confLevel.ToString()) &&*/
Blocks.Contains(t.BlockName)
select t).ToListAsync();
In case Permissions is an array of string based on integers, an easy solution is to cast the array to int:
int[] Permissions = _TableView.permission
.PermissionsList
.Split(',')
.Select(s => int.Parse(s)).ToArray();
And remove the ToString call which are that are causing the exception:
var test = await (from t in _context.DocumentIn
where Permissions.Contains(t.Imp_Exp) &&
Permissions.Contains(t.confLevel) &&
Blocks.Contains(t.BlockName)
select t).ToListAsync();
But the main idea is that, removing the ToString call, whatever is your scenario, you should try to remove them from your query
Related
I'm trying to execute a LINQ query within a plugin, using the OrganizationServiceContext, to retrieve some quotes. On these quotes, I'm using .Select() to only select the value for the field cgk_totalnetprice, as shown below:
quotes = OrganizationServiceContext.QuoteSet
.Where(_ =>
_.OpportunityId != null &&
_.OpportunityId.Id == opportunityId &&
_.QuoteId != currentQuote.Id &&
(_.StatusCode.Value == (int)Quote_StatusCode.Won || _.StatusCode.Value == (int)Quote_StatusCode.WonOrder) &&
_.cgk_quotetypecode != null &&
(_.cgk_quotetypecode.Value == (int)QuoteTypeCode.Regular || _.cgk_quotetypecode.Value == (int)QuoteTypeCode.ServiceUnderWarranty))
.Select(x => new Quote() { Id = x.Id, cgk_totalnetprice = x.cgk_totalnetprice})
.ToList();
However, when retrieving those quotes, the context does not return a value for all except one quote (and it is not the quote that triggered the update in the first place, but just a random one that was not updated at all)
Weird part: when I rewrite the query to a QueryExpression, everything works perfectly:
QueryExpression qe = new QueryExpression("quote");
//Exclude current quote
qe.Criteria.AddCondition("quoteid", ConditionOperator.NotEqual, currentQuote.Id);
//Opportunity
qe.Criteria.AddCondition("opportunityid", ConditionOperator.NotNull);
qe.Criteria.AddCondition("opportunityid", ConditionOperator.Equal, opportunityId);
//State-Status
FilterExpression statusFilter = new FilterExpression(LogicalOperator.Or);
statusFilter.AddCondition("statuscode", ConditionOperator.Equal, (int)Quote_StatusCode.Won);
statusFilter.AddCondition("statuscode", ConditionOperator.Equal, (int)Quote_StatusCode.WonOrder);
qe.Criteria.AddFilter(statusFilter);
//QuoteType
qe.Criteria.AddCondition("cgk_quotetypecode", ConditionOperator.NotNull);
FilterExpression typeFilter = new FilterExpression(LogicalOperator.Or);
typeFilter.AddCondition("cgk_quotetypecode", ConditionOperator.Equal, (int)QuoteTypeCode.Regular);
typeFilter.AddCondition("cgk_quotetypecode", ConditionOperator.Equal, (int)QuoteTypeCode.ServiceUnderWarranty);
qe.Criteria.AddFilter(typeFilter);
qe.ColumnSet = new ColumnSet("quoteid", "cgk_totalnetprice");
quotes = this.OrganizationService.RetrieveMultiple(qe).Entities.Cast<Quote>().ToList();
What could cause this difference between OrganizationServiceContext and OrganizationService + QueryExpression??
Queries on OrganizationServiceContext rely on LINQ for CRM, which in turn translates LINQ expressions into QueryExpression objects. LINQ for CRM comes with a few weaknesses:
it does not implement all capabilities of the underlying QueryExpression,
it only supports a limited set of LINQ constructs (see MS Docs),
in some cases it creates incorrect queries,
query processing is approx. 10% slower.
Your query looks pretty straightforward, yet it fails. Maybe you can leave the line _.cgk_quotetypecode != null && out. I guess it is not needed and combined with the subsequent filtering on the same attribute it may trick the LINQ parser into constructing the wrong filter and/or conditions.
Another option is to materialize the LINQ query first and then select the columns needed. Of course this will lead to a select *, but it's often worth trying while troubleshooting.
E.g. you could write:
.ToArray()
.Select(x => new Quote() { Id = x.Id, cgk_totalnetprice = x.cgk_totalnetprice});
Working with Dynamics CRM/365 CE I learned to avoid LINQ for CRM. Instead I use a bunch of extension methods allowing me to create QueryExpression queries in a much less verbose way.
Final suggestion: in some cases a filter's LogicalOperator.Or can be replaced by ConditionOperator.In or ConditionOperator.Between. Doing so the construct
//State-Status
FilterExpression statusFilter = new FilterExpression(LogicalOperator.Or);
statusFilter.AddCondition("statuscode", ConditionOperator.Equal, (int)Quote_StatusCode.Won);
statusFilter.AddCondition("statuscode", ConditionOperator.Equal, (int)Quote_StatusCode.WonOrder);
qe.Criteria.AddFilter(statusFilter);
can simply be replaced by this oneliner:
qe.Criteria.AddCondition("statuscode", ConditionOperator.In, (int)Quote_StatusCode.Won, (int)Quote_StatusCode.WonOrder);
I generally use the query syntax for Dynamics LINQ queries. I'd suggest standard query troubleshooting - start with no conditions and add them one-by-one.
Below is an idea of how my query would look. I'd keep adding / modifying the where clauses until the query returned what I expected.
We could use && operators instead of multiple where clauses, but I find that having the multiple where clauses often makes commenting and uncommenting easier.
using(var ctx = new OrganizationServiceContext(ctx))
{
var x = from q in ctx.CreateQuery<Quote>()
where q.QuoteId ! = currentQuote.Id
where q.OpportunityId != null
where q.cgk_quotetypecode != null
where q.cgk_quotetypecode == QuoteTypeCode.Regular || QuoteTypeCode.ServiceUnderWarranty
select new Quote
{
Id = q.Id,
cgk_totalnetprice = x.cgk_totalnetprice
};
var quotes = x.ToList();
}
I have a LINQ query
var age = new int[]{1,2,3};
dbContext.TA.WHERE(x=> age.Contains( x.age)).ToList()
In an online article #11 (https://medium.com/swlh/entity-framework-common-performance-mistakes-cdb8861cf0e7) mentioned it is not a good practice as it creates many execution plan at the SQL server.
In this case, how should LINQ be revised so that I can do the same thing but minimize the amount of execution plans generated?
(note that I have no intention to convert it into a stored procedure and pass & join with the UDT as again it requires too many effort to do so)
That article offers some good things to keep in mind when writing expressions for EF. As a general rule that example is something to keep in mind, not a hard "never do this" kind of rule. It is a warning over writing queries that allow for multi-select and to avoid this when possible as it will be on the more expensive side.
In your example with something like "Ages", having a hard-coded list of values does not cause a problem because every execution uses the same list. (until the app is re-compiled with a new list, or you have code that changes the list for some reason.) Examples where it can be perfectly valid to use this is with something like Statuses where you have a status Enum. If there are a small number of valid statuses that a record can have, then declaring a common array of valid statuses to use in an Contains clause is fine:
public void DeleteEnquiry(int enquiryId)
{
var allowedStatuses = new[] { Statuses.Pending, Statuses.InProgress, Statuses.UnderReview };
var enquiry = context.Enquiries
.Where(x => x.EnquiryId == enquiryId && allowedStatuses.Contains(x.Status))
.SingleOrDefault();
try
{
if(enquiry != null)
{
enquiry.IsActive = false;
context.SaveChanges();
}
else
{
// Enquiry not found or invalid status.
}
}
catch (Exception ex) { /* handle exception */ }
}
The statuses in the list aren't going to change so the execution plan is static for that context.
The problem is where you accept something like a parameter with criteria that include a list for a Contains clause.
it is highly unlikely that someone would want to load data where a user could select ages "2, 4, and 6", but rather they would want to select something like: ">=2", or "<=6, or "2>=6" So rather than creating a method that accepts a list of acceptable ages:
public IEnumerable<Children> GetByAges(int[] ages)
{
return _dbContext.Children.Where(x => ages.Contains( x.Age)).ToList();
}
You would probably be better served with ranging the parameters:
private IEnumerable<Children> GetByAgeRange(int? minAge = null, int? maxAge = null)
{
var query = _dbContext.Children.AsQueryable();
if (minAge.HasValue)
query = query.Where(x => x.Age >= minAge.Value);
if (maxAge.HasValue)
query = query.Where(x => x.Age <= maxAge.Value);
return query.ToList();
}
private IEnumerable<Children> GetByAge(int age)
{
return _dbContext.Children.Where(x => x.Age == age).ToList();
}
[Authorize(Roles="Admin,Manager")]
public class PermissionController : ApiController
{
[HttpGet]
public IEnumerable<permission> Get()
{
using (var mydb = new ModelContainer())
{
if(User.IsInRole("Admin")){
return mydb.permissionSet.ToList();
}
else
{
//////Work
return mydb.permissionSet.Where(x=>x.name!="Admin").ToList().Where(x=>x.name!="Coder").ToList().Where(x=>x.name!="Tester").ToList();
//////Not work
return from a in mydb.permissionSet where a.name!="Admin" && a.name!="Manager" && a.name!="Coder" && a.name != "Tester" select a;
}
}
}
}
I think there are two ways of return the correct result, the first way above will work, but seems weird and I think it will query many times. The second way seems better way, but not work.
With LINQ you are building a query. Calling ToList() on such a query will force the query to execute and return the results. Your first option will build the query and return the results. Your second option returns the query. That is the difference.
mydb.permissionSet.Where(x=>x.name!="Admin").ToList().Where(x=>x.name!="Coder").ToList().Where(x=>x.name!="Tester").ToList();
Can be broken up as this:
var qryNoAdmin = mydb.permissionSet.Where(x=>x.name!="Admin");
var permissionSetNoAdmins = qryNoAdmin.ToList();
var qryNoAdminNoCoder = permissionSetNoAdmins.Where(x=>x.name!="Coder");
var permissionSetNoAdminsNoCoders = qryNoAdminNoCoder.ToList();
var qryNoAdminNoCoderNoTester = permissionSetNoAdminsNoCoders .Where(x=>x.name!="Tester");
var permissionSetNoAdminsNoCodersNoTesters = qryNoAdminNoCoderNoTester.ToList();
return permissionSetNoAdminsNoCodersNoTesters;
If that looks inefficient to you... Yes it is. Look at #user1778606 suggestion to make it more efficient.
Now your second option only returns the query. If you call ToList() on that, the query will be executed and you get the results.
var qryNoAdminNoCoderNoTester = from a in mydb.permissionSet where a.name!="Admin" && a.name!="Manager" && a.name!="Coder" && a.name != "Tester" select a;
var permissionSetNoAdminsNoCodersNoTesters = qryNoAdminNoCoderNoTester.ToList();
return permissionSetNoAdminsNoCodersNoTesters;
Please note I didn't test this code. There might be typos.
Suggested reading: Introduction to LINQ Queries (C#)
try to put them all in the same where clause line, ie.
return mydb.permissionSet.Where(x=>x.name!="Admin" & x.name!="Coder" & x.name!="Tester").ToList();
btw, I dont think it re-queries in the way you did it first, it just filters the resulting list, but there may be a bit of overhead in recreating the list each time after the filter, ie. I think
return db.permissionSet.Where(x=>x.name!="Admin").Where(x=>x.name!="Coder").Where(x=>x.name!="Tester").ToList();
should be equally efficient to the consolidated where clause above
I have a call made to Simple.Data where I want to limit the columns that are being brought back. However I am hitting problems..
This works fine:
var db = Database.Open();
var questionIdRow = db.Question.FindByFriendlyId(friendlyId);
if (questionIdRow == null) return Guid.Empty;
return questionIdRow.QuestionId;
However, the following doesn't work (I get a Simple.Data.UnresolvableObjectException 'Column not found')
var db = Database.Open();
var questionIdRow = db.Question.FindByFriendlyId(friendlyId)
.Select(db.Question.QuestionId);
if (questionIdRow == null) return Guid.Empty;
return questionIdRow.QuestionId;
I was under the impression from the Simple.Data documentation that this was all that I needed to do to limit the selected columns. Note that the selection is simply selecting the same column that is referenced later on.
The actual exception is thrown on the var questionIdRow = line.
Can anybody give me some guidance?
this is a common problem, and has actually led to FindBy being deprecated before we even get to 1.0. The problem is that FindBy returns a record straight away, so you can't continue to call query methods on it.
The correct approach is to call FindAllBy and end with a First or FirstOrDefault:
var db = Database.Open();
var questionIdRow = db.Question.FindAllByFriendlyId(friendlyId)
.Select(db.Question.QuestionId)
.FirstOrDefault();
if (questionIdRow == null) return Guid.Empty;
return questionIdRow.QuestionId;
*Mongo Newbie here
I have a document containing several hundred numeric fields which I need to query in combination.
var collection = _myDB.GetCollection<MyDocument>("collection");
IMongoQuery mongoQuery; // = Query.GT("field", value1).LT(value2);
foreach (MyObject queryObj in Queries)
{
// I have several hundred fields such as Height, that are in queryObj
// how do I build a "boolean" query in C#
mongoQuery = Query.GTE("Height", Convert.ToInt16(queryObj.Height * lowerbound));
}
I have several hundred fields such as Height (e.g. Width, Area, Perimeter etc.), that are in queryObj how do I build a "boolean" query in C# that combines range queries for each field in conjunction.
I have tried to use the example Query.GT("field", value1).LT(value2);, however the compiler does not accept the LT(Value) construct. In any event I need to be able to build a complex boolean query by looping through each of the numeric field values.
Thanks for helping a newbie out.
EDIT 3:
Ok, it looks like you already have code in place to build the complicated query. In that case, you just needed to fix the compiler issue. Am assuming you want to do the following (x > 20 && x < 40) && (y > 30 && y < 50) ...
var collection = _myDB.GetCollection<MyDocument>("collection");
var queries = new List<IMongoQuery>();
foreach (MyObject queryObj in Queries)
{
//I have several hundred fields such as Height, that are in queryObj
//how do I build a "boolean" query in C#
var lowerBoundQuery = Query.GTE("Height", Convert.ToInt16(queryObj.Height * lowerbound));
var upperBoundQuery = Query.LTE("Height", Convert.ToInt16(queryObj.Height * upperbound));
var query = Query.And(lowerBoundQuery, upperBoundQuery);
queries.Add(query);
}
var finalQuery = Query.And(queries);
/*
if you want to instead do an OR,
var finalQuery = Query.Or(queries);
*/
Original Answer.
var list = _myDb.GetCollection<MyDoc>("CollectionName")
.AsQueryable<MyDoc>()
.Where(x =>
x.Height > 20 &&
x.Height < 40)
.ToList();
I have tried to use the example Query.GT("field", value1).LT(value2);,
however the compiler does not accept the LT(Value) construct.
You can query MongoDB using linq, if you are using the official C# driver. That ought to solve the compiler issue I think.
The more interesting question I have in mind is, how are you going to construct that complicated boolean query?
One option is to dynamically build an Expression and then pass that to the Where
My colleague is using the following code for something similar...
public static IQueryable<T> Where<T>(this IQueryable<T> query,
string column, object value, WhereOperation operation)
{
if (string.IsNullOrEmpty(column))
return query;
ParameterExpression parameter = Expression.Parameter(query.ElementType, "p");
MemberExpression memberAccess = null;
foreach (var property in column.Split('.'))
memberAccess = MemberExpression.Property
(memberAccess ?? (parameter as Expression), property);
//change param value type
//necessary to getting bool from string
ConstantExpression filter = Expression.Constant
(
Convert.ChangeType(value, memberAccess.Type)
);
//switch operation
Expression condition = null;
LambdaExpression lambda = null;
switch (operation)
{
//equal ==
case WhereOperation.Equal:
condition = Expression.Equal(memberAccess, filter);
lambda = Expression.Lambda(condition, parameter);
break;
//not equal !=
case WhereOperation.NotEqual:
condition = Expression.NotEqual(memberAccess, filter);
lambda = Expression.Lambda(condition, parameter);
break;
//string.Contains()
case WhereOperation.Contains:
condition = Expression.Call(memberAccess,
typeof(string).GetMethod("Contains"),
Expression.Constant(value));
lambda = Expression.Lambda(condition, parameter);
break;
}
MethodCallExpression result = Expression.Call(
typeof(Queryable), "Where",
new[] { query.ElementType },
query.Expression,
lambda);
return query.Provider.CreateQuery<T>(result);
}
public enum WhereOperation
{
Equal,
NotEqual,
Contains
}
Currently it only supports == && !=, but it shouldn't be that difficult to implement >= or <= ...
You could get some hints from the Expression class: http://msdn.microsoft.com/en-us/library/system.linq.expressions.expression.aspx
EDIT:
var props = ["Height", "Weight", "Age"];
var query = _myDb.GetCollection<MyDoc>("CName").AsQueryable<MyDoc>();
foreach (var prop in props)
{
query = query.Where(prop, GetLowerLimit(queryObj, prop), WhereOperation.Between, GetUpperLimit(queryObj, prop));
}
// the above query when iterated over, will result in a where clause that joins each individual `prop\condition` with an `AND`.
// The code above will not compile. The `Where` function I wrote doesnt accept 4 parameters. You will need to implement the logic for that yourself. Though it ought to be straight forward I think...
EDIT 2:
If you don't want to use linq, you can still use Mongo Query. You will just need to craft your queries using the Query.And() and Query.Or().
// I think this might be deprecated. Please refer the release notes for the C# driver version 1.5.0
Query.And(Query.GTE("Salary", new BsonDouble(20)), Query.LTE("Salary", new BsonDouble(40)), Query.GTE("Height", new BsonDouble(20)), Query.LTE("Height", new BsonDouble(40)))
// strongly typed version
new QueryBuilder<Employee>().And(Query<Employee>.GTE(x => x.Salary, 40), Query<Employee>.LTE(x => x.Salary, 60), Query<Employee>.GTE(x => x.HourlyRateToClients, 40), Query<Employee>.LTE(x => x.HourlyRateToClients, 60))