Using LINQ to search comma separated string - entity-framework

There are a number of records in the table, and there is a column called AssignedTo, and the value for AssignedTo is comma separated string, the possible values for it could be something like:
"1"
"2"
"3"
"11"
"12"
"1,2"
"1,3"
"2,3"
"1,2,3"
"1,3,11"
"1,3,12"
If I use the following LINQ query to search, in case value = 1
records = records.Where(x => x.AssignedTo.Contains(value) || search == null);
It returns the records with AssignedTo value
"1", "11", "12", "1,2", "1,3", "1,2,3", "1,3,11", "1,3,12"
I really want to only return the records with AssignedTo containing "1",
which are "1", "1,2", "1,3", "1,2,3", "1,3,11", "1,3,12", do not want "11" and "12"
If I use the following LINQ query to search the qualified records, still value = 1
records = records.Where(x => x.AssignedTo.Contains("," + value + ",") ||
x.AssignedTo.StartsWith(value + ",") ||
x.AssignedTo.EndsWith("," + value) ||
value == null);
It returns the records with AssignedTo value "1,2", "1,3", "1,2,3", "1,3,11", "1,3,12", but missing the record with AssignedTo value "1".

Since something like this is likely a search filter, doing the operation in-memory likely isn't a very good option unless the row count is guaranteed to be manageable. Ideally something like this should be re-factored to use a proper relational structure rather than a comma-delimited string.
However, the example you had was mostly there, just missing an Equals option to catch the value by itself. I'd also take the 'value == null' check out of the Linq expression into a conditional as to whether to add the WHERE clause. The difference is with the condition in the Linq, this will generate that into the SQL, where-as by pre-checking you can avoid the SQL conditions all-together if there is no value specified.
if (!string.IsNullOrEmpty(value))
records = records.Where(x => x.AssignedTo.Contains("," + value + ",") ||
x.AssignedTo.StartsWith(value + ",") ||
x.AssignedTo.EndsWith("," + value) ||
x.AssignedTo == value);
This would catch "n,...", "...,n,...", "...,n", and "n".

A better method would be to split the string and search the results:
records = records.Where(x => x.AssignedTo.Split(',').Contains(value) || search == null);
Note that you can't use this method directly in an EF query since there's no way to translate it to standard SQL. So you may want to filter using your Contains as a starting spot (to reduce the number of false positives) and then filter in-memory:
records = records.Where(x => x.AssignedTo.Contains(value) || search == null)
.AsEnumerable() // do subsequent filtering in-memory
.Where(x => x.AssignedTo.Split(',').Contains(value) || search == null)
Or redesign
your database to use related tables rather than storing a comma-delimited list of strings...

If you are building a linq expression against database then Split function will throw an error. You can use expression below there.
if (!string.IsNullOrEmpty(value))
{
records = records.Where(x => (',' + x.AssignedTo + ',').Contains(',' + value + ',')
}

Related

Filter Postgres Json array is subset of array using JsonPath

I fail to find any information on how to do in / subsetof / contains queries using JsonPath in Postgres.
e.g.
Assuming the following data in a jsonb column named data
{
"name": "foo",
"somearray" : [1,2,3,4,5]
}
Then I want to query this using something like
SELECT *
FROM mytable
where jsonb_path_exists(data, '($.somearray ??????? [2,4,6,8] ');
This does work:
SELECT *
FROM mytable
where jsonb_path_exists(data, '($ ? (#.somearray[*] == 2 || #.somearray[*] == 4 /*etc*/) ');
But I am hoping that there is some shorter syntax to do a proper subset test
Unfortunately no jsonpath array operators, but you can still use the array operators :
SELECT *
FROM mytable
where (data->>'somearray') :: integer[] #> [2,4,6,8] ;

Searching a column in Entity Framework with multiple values

I am trying to run a search on one particular field of a table with a list of values. Not able to find a solution so far. Any help is really appreciated.
Here is the scenario
var records = new PagedList<Required>();
var result = db.Required.Where(x => filter == null || (x.Title.Contains(filter)) || x.CID.Contains(filter));
foreach (string str in SelectedNetwork)
{
string tempStr = str;
result = result.Where(x => x.Network == tempStr);
records.TotalRecords = result.Count();
}
records.Content = result
.Where(x => filter == null ||
(x.Title.Contains(filter))
|| x.CID.Contains(filter)
)
.OrderBy(sort + " " + sortdir)
.Skip((page - 1) * Convert.ToInt32(records.PageSize))
.Take(Convert.ToInt32(records.PageSize))
.ToList();
highlighted code in the foreach loop fails to run as per expectation. Is there any way, I can fix it?
Thanks
Tutumon
You must take into account that LINQ expressions are queries, until you materialize them. To materialize them you need to either enumerate them, or convert them to a list, array, or whatever, i.e. enumerate their members in a foreach, or call a method like ToList(), or ToArray().
In your code the original query stored in result is not materialized, so everytime a foreach loop is executed, a new Where contidion is added to the original query. To vaoid this behavior you need to recreate the whole results query in each iteration, so that you get a fresh copy of the unfilterd expression.
There would be another solution which would be to materialize the result query and then run the foreach loop as is. The problem of this solution would be that you would get all the data from the database, keep it in memory and run the Where and the Count on the in-memory copy. Unless there is a very small number of rows in Required that would be a very bad idea.

How to update rows based on condition in spark-sql

I am working on spark-sql for data preparation.
The problem I am facing is after getting the result of sql query. How should I update rows based on the If-then-else condition.
What I am doing
val table_join = sqlContext.sql(""" SELECT a.*,b.col as someCol
from table1 a LEFT JOIN table2 b
on a.ID=b.ID """)
table_join.registerTempTable("Table_join")
Now when I have final joined table which is in df format. How should I update rows?
//Final filtering operation
val final_filtered_table = table_join.map{ case record=>
if(record.getAs[String]("col1") == "Y" && record.getAs[String]("col2") == "") record.getAs[String]("col2")="UNKNOWN"
else if (record.getAs[String]("col1") == "N") record("col1")=""
else record
}
In the above map the if syntax works properly but the moment I apply the update condition to modify It gives me error.
But why the below query is working
if(record.getAs[String]("col1") == "Y" && record.getAs[String]("col2") == "") "UNKNOWN"
But the moment I change "UNKNOWN" to record.getAs[String]("col2")="UNKNOWN" It gives me error at at .getAs
Another approach I tried is this:
val final_filtered_sql = table_join.map{row =>
if(row.getString(6) == "Y" && row.getString(33) == "") row.getString(6) == "UNKNOWN"
else if(row.getString(6) == "N") row.getString(6) == ""
else row
}
This is working but is this the right approach as I should not call the columns by their no's but instead their names. What approach should I follow to get names of the column and then update ??
Please help me regarding this. What syntax should I do to update rows based on the condition in dataframe in spark-sql
record.getAs[String]("col2")="UNKNOWN" won't work because record.getAs[String](NAME) will return a String which doesn't have a = method and assigning a new value to a string doesn't make sense.
DataFrame records don't have any setter methods because DataFrames are based on RDD which are immutable collections, meaning you cannot change their state and that's how you're trying to do here.
One way would be to create a new DataFrame using selectExpr on table_join and put that if/else logic there using SQL.

How to search in SphinxQL by sql_attr_multi field

This is my query to sphinxQL:
SELECT option_id FROM items WHERE cat IN (10,11) GROUP BY option_id LIMIT 100000 OPTION max_matches=100000
cat is sql_attr_multi field, this query not return to me correct result. Anybody knows how to search by fields by this sphinx attribute?
That query looks for items where cat attribute contains either 10 OR 11, is that what you trying to do?
If its not, would help to know what you are trying to query!
I had similar problem.
When I pass array to IN condition for MVA attribute I have no result, however there is several ones in index.
When I debug condition (attribute array(10, 11) in you case) I see that array values is string integer instead.
array(
0 => "10",
1 => "11"
)
For every single value in condition uses quoteArr() function
https://github.com/FoolCode/SphinxQL-Query-Builder/blob/master/src/SphinxQL.php#L518
wich escape value according with https://github.com/FoolCode/SphinxQL-Query-Builder/blob/master/src/Drivers/ConnectionBase.php#L95
The quote function use is_int() PHP internal function:
$a = "1";
var_dump(is_int($a)); // return bool(false)
It mean, thet instead
cat IN (10, 11)
you have
cat IN ("10", "11")
But sphinx can't filter MVA attribute by not integer (string) values no metter IN or OR WHERE notation you use.
[1064] index document : unsupported filter type 'string' on MVA column [ SELECT * FROM `document` WHERE MATCH('(some query)') AND `_category` = '5' LIMIT 0, 10]
You should use strict value type:
foreach ($category as &$item) {
$item = (int)$item;
} unset($item);
I am not sure that it is your incindent. Unfortunately, there isn't enough data to say it for sure in this case.

Count Group Ordinal in LINQ to Dataset

I have an old FoxPro program that does a SQL query which includes the following:
SELECT Region,
Year AS yr_qtr,
SUM(Stock) AS inventory
**...
COUNT(Rent) AS rent_ct
FROM
**...
GROUP BY Region, Year
ORDER BY Region, Year
INTO CURSOR tmpCrsr
The query is against a .DBF table file, and includes data from an Excel file. I've used both to populate an enumeration of user-defined objects in my C# program. (Not sure .AsEnumerable is needed or not.) I then attempt to use LINQ to Dataset to query the list of user objects and create the same result set:
var rslt1 = from rec in recs_list //.AsEnumerable()
group rec by new {rec.Region, rec.Year} into grp
select new
{
RegName = grp.Key.Region,
yr_qtr = grp.Key.Year,
inventory = grp.Sum(s => s.Stock),
// ...
rent_count = grp.Count(r => r.Rent != null)
};
This gives me the warning that "The result of the expression is always 'true' since a value of type 'decimal' is never equal to 'null' of type 'decimal'" for the Count() of the Rent column.
This makes sense, but then how do I do a count exclusive of the rows that have a value of .NULL. for that column in the FoxPro table (or NULL in any SQL database table, for that matter)? I can't do a null test of a decimal value.
If rent is based off of a column which is not a nullable value, then checking for null makes no sense which I believe the compiler accurately shows. Change the line to
rent_count = grp.Count(r => r.Rent != 0)
instead.
For if the code is actuall nullable such as:
Decimal? rent;
That would make checking rent against null valid. If that is the case then the line would be:
rent_count = grp.Count(r => (r.Rent ?? 0) != 0)
where null coalesding operator ?? can be used. Which states if r.rent is null, use the value 0 (or any value you want technically) for r.Rent. in the next process.