I'm creating a query to search the db using EF. TdsDb being the EF context.
string searchValue = "Widget";
TdsDb tdsDb = new TdsDb();
IQueryable<Counterparty> counterparties;
I can do exact match:
counterparties = tdsDb.Counterparties.Where(x => x.CounterpartyName == searchValue);
or wildcard match:
counterparties = tdsDb.Counterparties.Where(x => x.CounterpartyName.Contains(searchValue));
But I want to be able to do both i.e. (psudo code)
counterparties = tdsDb.Counterparties.Where(x =>
if (searchValue.EndsWith("%"))
{
if (searchValue.StartsWith("%"))
{x.CounterpartyName.Contains(searchValue)}
else
{x.CounterpartyName.StartsWith(searchValue)}
}
else
{x => x.CounterpartyName == searchValue}
);
Now clearly I can't put an if statement in the where clause like that. But I also can't duplicate the queries: shown here they are hugely dumbed down. The production query is far longer, so having multiple versions of the same long query that vary on only one clause seems very unhealthy and unmaintainable.
Any ideas?
You should be able to use the ternary operator:
bool startsWithWildCard = searchValue.StartsWith("%");
bool endsWithWildCard = searchValue.EndsWith("%");
counterparties = tdsDb.Counterparties.Where(x =>
endsWithWildCard
? (startsWithWildCard
? x.CounterpartyName.Contains(searchValue)
: (x.CounterpartyName.StartsWith(searchValue)))
: (x.CounterpartyName == searchValue));
Did you test btw if querying by a searchValue that has an % at the beginning or end works as you expect? It might be possible that % will be escaped as a character to query for because StartsWith and Contains will prepend/append % wildcards to the generated SQL search term anyway. In that case you need to cut off the % from the searchValue before you pass it into StartsWith or Contains.
Related
I'm trying to use LinQ Intersect (or equivalent) into an IQueryable method but it seems like I'm doing it wrong.
I have some PRODUCTS that match some SPECIFITY (like colors, materials, height...), those specifications have different values, for example:
color : blue, red, yellow
height : 128cm, 152cm...
I need to get the products that match ALL the list of couple specifityId / specifityValue I provide.
Here what I'm trying to do:
// The list of couple SpecifityID (color, material..) / SpecifityValue (red, yellow, wood...)
List<string> SpecId_SpecValue = new List<string>();
SpecId_SpecValue.Add("3535a444-1139-4a1e-989f-795eb9be43be_BEA");
SpecId_SpecValue.Add("35ad6162-a885-4a6a-8044-78b68f6b2c4b_Purple");
int filterCOunt = SpecId_SpecValue.Count;
var query =
Products
.Include(pd => pd.ProductsSpecifity)
.Where(z => SpecId_SpecValue
.Intersect(z.ProductsSpecifity.Select(x => (x.SpecifityID.ToString() + "_" + x.SpecifityValue)).ToList()).Count() == filterCOunt);
I got the error : InvalidOperationException: The LINQ expression 'DbSet() could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information. which mean it can't be translated to SQL and I need to ToList before my filter.
The problem is, I don't want to call ToList() because I got huge number of products in my Database and I don't want to load them in memory before filtering them.
Is there an other way to achieve what I need to do?
I ended up using a solution found in the link #Gert Arnold provide here.
I used BlazarTech.QueryableValues.SqlServer #yv989c's answers
Here's what is now working like a charm :
// The list of couple SpecifityID (color, material..) / SpecifityValue (red, yellow, wood...)
Dictionary<Guid, string> SpecId_SpecValue = new Dictionary<Guid, string>();
SpecId_SpecValue.Add(new Guid("3535a444-1139-4a1e-989f-795eb9be43be"), "BEA");
SpecId_SpecValue.Add(new Guid("35ad6162-a885-4a6a-8044-78b68f6b2c4b"), "Purple");
// BlazarTech.QueryableValues.SqlServer
var queryableValues = DbContext.AsQueryableValues(SpecId_SpecValue);
var query = Products.Include(pd => pd.ProductsSpecifity)
.Where(x => x.ProductsSpecifity
.Where(e => queryableValues
.Where(v =>
v.Key == e.SpecifityID &&
v.Value == e.SpecifityValue
)
.Any()
).Count() == dynamicFilter.Count);
The query expresses "products of which all x.SpecifityID.ToString() + "_" + x.SpecifityValue combinations exactly match some given combinations".
Set combination operators like Except often don't play nice with EF for various reasons I'm not going into here. Fortunately, in many of these cases a work-around can be found by using Contains, which EF does support well. In your case:
var query = Products.Include(pd => pd.ProductsSpecifity)
.Where(z => z.ProductsSpecifity
.Select(x => x.SpecifityID.ToString() + "_" + x.SpecifityValue)
.Count(s => SpecId_SpecValue.Contains(s)) == filterCount);
Please note that the comparison is not efficient. Transforming database values before comparison disables any use of indexes (is not sargable). But doing this more efficiently isn't trivial in EF, see this.
I am trying to understand what is the best way to create a dynamic query.
I have a requirement where I will be writing an API to retrieve data from DB. the API has lot of filter paramters. eg. I need to retrieve movies that can be filtered on following properties.
MovieName, Genere, Rating, Language, Category
I can give these parameters in any combination.. so in my Data layer I started framing my dynamic query like this.
IQueryable<Movie> qryContext = null;
if(!string.isnullorEmpty(request.MovieName))
qryContext = context.Movies.Where(x => x.MovieName == request.MovieName)
if(!string.isnullorEmpty(request.Genere))
qryContext = context.Movies.Where(x => x.Genere == request.Genere)
if(!string.isnullorEmpty(request.Language))
qryContext = context.Movies.Where(x => x.Language == request.Language)
if(!string.isnullorEmpty(request.Category))
qryContext = context.Movies.Where(x => x.Category == request.Category)
if(qryContext!= null)
return qryContext.ToList();
else
return null;
Based on the given parameters, the sql query is framed..
But When I search in google reg dynamic queries in EF, most of the links refer to using Expression. Do I need to make use of Expression or can I proeeed with the above method.
Also let me know what advantage I get on using expressions.
Basically what you see inside the Where(...) is expression.
In your example this would be x => x.Language == request.Language
if(!string.isnullorEmpty(request.Language)) qryContext = context.Movies.Where(x => x.Language == request.Language)
Also, I would recommend you to take a look into the Dynamic expression library from EF Plus team. Here https://dynamic-linq.net/basic-simple-query
This allows you to pass expression as a string. And you can construct your filters in the Frontend, and pass them as a string, which helps you to write a much cleaner implementation of filters.
My Linq Query keeps returning the null error on FirstOrDefault
The cast to value type 'System.Int32' failed because the materialized value is null
because it can't find any records to match on the ClinicalAssetID form the ClinicalReading Table, fair enough!
But I want the fields in my details page just to appear blank if the table does not have matching entry.
But how can I handle the null issue when using the order by function ?
Current Code:
var ClinicalASSPATINCVM = (from s in db.ClinicalAssets
join cp in db.ClinicalPATs on s.ClinicalAssetID equals cp.ClinicalAssetID into AP
from subASSPAT in AP.DefaultIfEmpty()
join ci in db.ClinicalINSs on s.ClinicalAssetID equals ci.ClinicalAssetID into AI
from subASSINC in AI.DefaultIfEmpty()
join co in db.ClinicalReadings on s.ClinicalAssetID equals co.ClinicalAssetID into AR
let subASSRED = AR.OrderByDescending(subASSRED => subASSRED.MeterReadingDone).FirstOrDefault()
select new ClinicalASSPATINCVM
{
ClinicalAssetID = s.ClinicalAssetID,
AssetTypeName = s.AssetTypeName,
ProductName = s.ProductName,
ModelName = s.ModelName,
SupplierName = s.SupplierName,
ManufacturerName = s.ManufacturerName,
SerialNo = s.SerialNo,
PurchaseDate = s.PurchaseDate,
PoNo = s.PoNo,
Costing = s.Costing,
TeamName = s.TeamName,
StaffName = s.StaffName,
WarrantyEndDate = subASSPAT.WarrantyEndDate,
InspectionDate = subASSPAT.InspectionDate,
InspectionOutcomeResult = subASSPAT.InspectionOutcomeResult,
InspectionDocumnets = subASSPAT.InspectionDocumnets,
LastTypeofInspection = subASSINC.LastTypeofInspection,
NextInspectionDate = subASSINC.NextInspectionDate,
NextInspectionType = subASSINC.NextInspectionType,
MeterReadingDone = subASSRED.MeterReadingDone,
MeterReadingDue = subASSRED.MeterReadingDue,
MeterReading = subASSRED.MeterReading,
MeterUnitsUsed = subASSRED.MeterUnitsUsed,
FilterReplaced = subASSRED.FilterReplaced
}).FirstOrDefault(x => x.ClinicalAssetID == id);
Tried this but doesn't work
.DefaultIfEmpty(new ClinicalASSPATINCVM())
.FirstOrDefault()
Error was:
CS1929 'IOrderedEnumerable<ClinicalReading>' does not contain a definition for 'DefaultIfEmpty' and the best extension method overload 'Queryable.DefaultIfEmpty<ClinicalASSPATINCVM>(IQueryable<ClinicalASSPATINCVM>, ClinicalASSPATINCVM)' requires a receiver of type 'IQueryable<ClinicalASSPATINCVM>'
Feel a little closer with this but still errors
let subASSRED = AR.OrderByDescending(subASSRED => (subASSRED.MeterReadingDone != null) ? subASSRED.MeterReadingDone : String.Empty).FirstOrDefault()
Error:
CS0173 Type of conditional expression cannot be determined because there is no implicit conversion between 'System.DateTime?' and 'string'
The original error means that some of the following properties of the ClinicalASSPATINCVM class - MeterReadingDone, MeterReadingDue, MeterReading, MeterUnitsUsed, or FilterReplaced is of type int.
Remember that subASSRED here
let subASSRED = AR.OrderByDescending(subASSRED => subASSRED.MeterReadingDone).FirstOrDefault()
might be null (no corresponding record).
Now look at this part of the projection:
MeterReadingDone = subASSRED.MeterReadingDone,
MeterReadingDue = subASSRED.MeterReadingDue,
MeterReading = subASSRED.MeterReading,
MeterUnitsUsed = subASSRED.MeterUnitsUsed,
FilterReplaced = subASSRED.FilterReplaced
If that was LINQ to Objects, all these would generate NRE (Null Reference Exception) at runtime. In LINQ to Entities this is converted and executed as SQL. SQL has no issues with expression like subASSRED.SomeProperty because SQL supports NULL naturally even if SomeProperty normally does not allow NULL. So the SQL query executes normally, but now EF must materialize the result into objects, and the C# object property is not nullable, hence the error in question.
To solve it, find the int property(es) and use the following pattern inside query:
SomeIntProperty = (int?)subASSRED.SomeIntProperty ?? 0 // or other meaningful default
or change receiving object property type to int? and leave the original query as is.
Do the same for any non nullable type property, e.g. DateTime, double, decimal, Guid etc.
You're problem is because your DefaultIfEmpty is executed AsQueryable. Perform it AsEnumerable and it will work:
// create the default element only once!
static readonly ClinicalAssPatInVcm defaultElement = new ClinicalAssPatInVcm ();
var result = <my big linq query>
.Where(x => x.ClinicalAssetID == id)
.AsEnumerable()
.DefaultIfEmpty(defaultElement)
.FirstOrDefault();
This won't lead to a performance penalty!
Database management systems are extremely optimized for selecting data. One of the slower parts of a database query is the transport of the selected data to your local process. Hence it is wise to let the DBMS do most of the selecting, and only after you know that you only have the data that you really plan to use, move the data to your local process.
In your case, you need at utmost one element from your DBMS, and if there is nothing, you want to use a default object instead.
AsQueryable will move the selected data to your local process in a smart way, probably per "page" of selected data.
The page size is a good compromise: not too small, so you don't have to ask for the next page too often; not too large, so that you don't transfer much more items than you actually use.
Besides, because of the Where statement you expect at utmost one element anyway. So that a full "page" is fetched is no problem, the page will contain only one element.
After the page is fetched, DefaultIfEmpty checks if the page is empty, and if so, returns a sequence containing the defaultElement. If not, it returns the complete page.
After the DefaultIfEmpty you only take the first element, which is what you want.
I would like to filter the files based on some patterns like :
- Team_*.txt (e.g.: Team_Orlando.txt);
- Name.*.City.txt (e.g.: Name.Robert.California.txt);
Or any name (the pattern * . * - it has spaces because was broken my text).
All the filters come from a database table and they are dynamic.
I'm trying to avoid use commands from SO like cp or mv. Is possible to filter files using patterns like the above ?
Here is what i've tried but got a regex error:
def getFiles(dir:File, filter:String) = {
(dir.isDirectory, dir.exists) match {
case (true, true) =>
dir.listFiles.filter(f => f.getName.matches(filter))
case _ =>
Array[File]()
}
}
You can use java.nio Files.newDirectoryStream() for that, it will accept pattern in desired format:
val stream = Files.newDirectoryStream(dir, pattern)
Check http://docs.oracle.com/javase/tutorial/essential/io/dirs.html#glob for detailed description.
I have a publication, essentially what's below:
Meteor.publish('entity-filings', function publishFunction(cik, queryArray, limit) {
if (!cik || !filingsArray)
console.error('PUBLICATION PROBLEM');
var limit = 40;
var entityFilingsSelector = {};
if (filingsArray.indexOf('all-entity-filings') > -1)
entityFilingsSelector = {ct: 'filing',cik: cik};
else
entityFilingsSelector = {ct:'filing', cik: cik, formNumber: { $in: filingsArray} };
return SB.Content.find(entityFilingsSelector, {
limit: limit
});
});
I'm having trouble with the filingsArray part. filingsArray is an array of regexes for the Mongo $in query. I can hardcode filingsArray in the publication as [/8-K/], and that returns the correct results. But I can't get the query to work properly when I pass the array from the router. See the debugged contents of the array in the image below. The second and third images are the client/server debug contents indicating same content on both client and server, and also identical to when I hardcode the array in the query.
My question is: what am I missing? Why won't my query work, or what are some likely reasons it isn't working?
In that first screenshot, that's a string that looks like a regex literal, not an actual RegExp object. So {$in: ["/8-K/"]} will only match literally "/8-K/", which is not the same as {$in: [/8-K/]}.
Regexes are not EJSON-able objects, so you won't be able to send them over the wire as publish function arguments or method arguments or method return values. I'd recommend sending a string, then inside the publish function, use new RegExp(...) to construct a regex object.
If you're comfortable adding new methods on the RegExp prototype, you could try making RegExp an EJSON-able type, by putting this in your server and client code:
RegExp.prototype.toJSONValue = function () {
return this.source;
};
RegExp.prototype.typeName = function () {
return "regex";
}
EJSON.addType("regex", function (str) {
return new RegExp(str);
});
After doing this, you should be able to use regexes as publish function arguments, method arguments and method return values. See this meteorpad.
/8-K/.. that's a weird regex. Try /8\-K/.
A minus (-) sign is a range indicator and usually used inside square brackets. The reason why it's weird because how could you even calculate a range between 8 and K? If you do not escape that, it probably wouldn't be used to match anything (thus your query would not work). Sometimes, it does work though. Better safe than never.
/8\-K/ matches the string "8-K" anywhere once.. which I assume you are trying to do.
Also it would help if you would ensure your publication would always return something.. here's a good area where you could fail:
if (!cik || !filingsArray)
console.error('PUBLICATION PROBLEM');
If those parameters aren't filled, console.log is probably not the best way to handle it. A better way:
if (!cik || !filingsArray) {
throw "entity-filings: Publication problem.";
return false;
} else {
// .. the rest of your publication
}
This makes sure that the client does not wait unnecessarily long for publications statuses as you have successfully ensured that in any (input) case you returned either false or a Cursor and nothing in between (like surprise undefineds, unfilled Cursors, other garbage data.