Split One Row into many based on splitting string in multiple cells - entity-framework

Trying to split one row into many based on string in two cells. it is similar to the question
LINQ to separate column value of a row to different rows in .net
but i need to split based on Product & Cost Columns rather than product column only
SNo.
Product
Cost
1
colgate,closeup,pepsodent
50,100,150
2
rin,surf
100
into
SNo.
Product
Cost
1
colgate
50
1
closeup
100
1
pepsodent
150
2
rin
100
2
surf
100
I'm using Linq to Object with Entity Framework

Try the following. Since you have not presented any model it can be inaccurate in names.
var loaded = ctx.Products.ToList();
var query =
from p in loaded
from sp in p.Product.Split(',').Zip(p.Cost.Split(','), (p, c) => (p, c))
select new
{
Sno = p.Sno,
Product = sp.p,
Cost = sp.c
};
var splitted = query.ToList();

Using #SvyatoslavDanyliv naming, here is an answer:
var loaded = ctx.Products.ToList();
var query =
from p in loaded
from sp in p.Product.Split(',').Zip(p.Cost.Split(','), (p, c) => (p, c))
select new
{
Sno = p.Sno,
Product = sp.p,
Cost = sp.c
};
var splitted = query.ToList();
It feels a bit complicated to me. I would prefer using an extension method to create a variant of Zip that repeats the last element of a shorter sequence to match the longer sequence:
public static class EnumerableExt {
public static IEnumerable<(T1 First,T2 Second)> ZipExtend<T1,T2>(this IEnumerable<T1> s1, IEnumerable<T2> s2) {
var s1e = s1.GetEnumerator();
var s2e = s2.GetEnumerator();
T1 s1eLast = default;
T2 s2eLast = default;
bool has_s2 = false;
if (s1e.MoveNext()) {
do {
s1eLast = s1e.Current;
if (s2e.MoveNext()) {
s2eLast = s2e.Current;
has_s2 = true;
}
else if (!has_s2)
yield break;
yield return (s1eLast, s2eLast);
} while (s1e.MoveNext());
if (has_s2)
while (s2e.MoveNext())
yield return (s1eLast, s2e.Current);
}
yield break;
}
}
Then the answer is:
var query =
from p in loaded
from pr in p.Product.Split(',').ZipExtend(p.Cost.Split(','))
select new
{
Sno = p.Sno,
Product = pr.First,
Cost = pr.Second
};
var splitted = query.ToList();

Related

How to only emit consistent calculations?

I'm using reactive programming to do a bunch of calculations. Here is a simple example that tracks two numbers and their sum:
static void Main(string[] args) {
BehaviorSubject<int> x = new BehaviorSubject<int>(1);
BehaviorSubject<int> y = new BehaviorSubject<int>(2);
var sum = Observable.CombineLatest(x, y, (num1, num2) => num1 + num2);
Observable
.CombineLatest(x, y, sum, (xx, yy, sumsum) => new { X = xx, Y = yy, Sum = sumsum })
.Subscribe(i => Console.WriteLine($"X:{i.X} Y:{i.Y} Sum:{i.Sum}"));
x.OnNext(3);
Console.ReadLine();
}
This generates the following output:
X:1 Y:2 Sum:3
X:3 Y:2 Sum:3
X:3 Y:2 Sum:5
Notice how second output result is "incorrect" because it is showing that 3+2=3. I understand why this is happening (x is updated before the sum is updated) but I want my output calculations to be atomic/consistent - no value should be emitted until all dependent calculations are complete. My first approach was this...
Observable.When(sum.And(Observable.CombineLatest(x, y)).Then((s, xy) => new { Sum = s, X = xy[0], Y = xy[1] } ));
This seems to work for my simple example. But my actual code has LOTS of calculated values and I couldn't figure out how to scale it. For example, if there was a sum and squaredSum, I don't know how to wait for each of these to emit something before taking action.
One method that should work (in-theory) is to timestamp all the values I care about, as shown below.
Observable
.CombineLatest(x.Timestamp(), y.Timestamp(), sum.Timestamp(), (xx, yy, sumsum) => new { X = xx, Y = yy, Sum = sumsum })
.Where(i=>i.Sum.Timestamp>i.X.Timestamp && i.Sum.Timestamp>i.Y.Timestamp)
// do the calculation and subscribe
This method could work for very complicated models. All I have to do is ensure that no calculated value is emitted that is older than any core data value. I find this to be a bit of a kludge. It didn't actually work in my console app. When I replaced Timestamp with a custom extension that assigned a sequential int64 it did work.
What is a simple, clean way to handle this kind of thing in general?
=======
I'm making some progress here. This waits for a sum and sumSquared to emit a value before grabbing the data values that triggered the calculation.
var all = Observable.When(sum.And(sumSquared).And(Observable.CombineLatest(x, y)).Then((s, q, data)
=> new { Sum = s, SumSquared = q, X = data[0], Y = data[1] }));
This should do what you want:
Observable.CombineLatest(x, y, sum)
.DistinctUntilChanged(list => list[2])
.Subscribe(list => Console.WriteLine("{0}+{1}={2}", list[0], list[1], list[2]));
It waits until the sum has been updated, which means that all its sources must have been updated too.
You problem isn't because x is updated before the sum is updated per se. It's really about the way that you've constructed your query.
You've effectively created two queries: Observable.CombineLatest(x, y, (num1, num2) => num1 + num2) & Observable.CombineLatest(x, y, sum, (xx, yy, sumsum) => new { X = xx, Y = yy, Sum = sumsum }). Since in each you're subscribing to x then you've create two subscriptions. Meaning that when x updates then two lots of updates occur.
You need to avoid creating two subscriptions.
If you write your code like this:
BehaviorSubject<int> x = new BehaviorSubject<int>(1);
BehaviorSubject<int> y = new BehaviorSubject<int>(2);
Observable
.CombineLatest(x, y, (num1, num2) => new
{
X = num1,
Y = num2,
Sum = num1 + num2
})
.Subscribe(i => Console.WriteLine($"X:{i.X} Y:{i.Y} Sum:{i.Sum}"));
x.OnNext(3);
...then you correctly get this output:
X:1 Y:2 Sum:3
X:3 Y:2 Sum:5
I've started to get my head around this some more. Here is a more detailed example of what I'm trying to accomplish. This is some code that validates a first and last name, and should only generate a whole name when both parts are valid. As you can see I'm trying to use a bunch of small independently defined functions, like "firstIsValid", and then compose them together to calculate something more complex.
It seems like the challenge I'm facing here is trying to correlate inputs and outputs in my functions. For example, "firstIsValid" generates an output that says some first name was valid, but doesn't tell you which one. In option 2 below, I'm able to correlate them using Zip.
This strategy won't work if a validation function does not generate one output for each input. For example, if the user is typing web addresses and we're trying to validate them on the web, maybe we'd do a Throttle and/or Switch. There might be 10 web addresses for a single "webAddressIsValid". In that situation, I think I have to include the output with the input. Maybe have an IObservable> where the string is the web address and the bool is whether it is valid or not.
static void Main(string[] args) {
var first = new BehaviorSubject<string>(null);
var last = new BehaviorSubject<string>(null);
var firstIsValid = first.Select(i => string.IsNullOrEmpty(i) || i.Length < 3 ? false : true);
var lastIsValid = last.Select(i => string.IsNullOrEmpty(i) || i.Length < 3 ? false : true);
// OPTION 1 : Does not work
// Output: bob smith, bob, bob roberts, roberts
// firstIsValid and lastIsValid are not in sync with first and last
//var whole = Observable
// .CombineLatest(first, firstIsValid, last, lastIsValid, (f, fv, l, lv) => new {
// First = f,
// Last = l,
// FirstIsValid = fv,
// LastIsValid = lv
// })
// .Where(i => i.FirstIsValid && i.LastIsValid)
// .Select(i => $"{i.First} {i.Last}");
// OPTION 2 : Works as long as every change in a core data value generates one calculated value
// Output: bob smith, bob robert
var firstValidity = Observable.Zip(first, firstIsValid, (f, fv) => new { Name = f, IsValid = fv });
var lastValidity = Observable.Zip(last, lastIsValid, (l, lv) => new { Name = l, IsValid = lv });
var whole =
Observable.CombineLatest(firstValidity, lastValidity, (f, l) => new { First = f, Last = l })
.Where(i => i.First.IsValid && i.Last.IsValid)
.Select(i => $"{i.First.Name} {i.Last.Name}");
whole.Subscribe(i => Console.WriteLine(i));
first.OnNext("bob");
last.OnNext("smith");
last.OnNext(null);
last.OnNext("roberts");
first.OnNext(null);
Console.ReadLine();
}
Another approach here. Each value gets a version number (like a timestamp). Any time a calculated value is older than the data (or other calculated values it relies upon) we can ignore it.
public class VersionedValue {
static long _version;
public VersionedValue() { Version = Interlocked.Increment(ref _version); }
public long Version { get; }
}
public class VersionedValue<T> : VersionedValue {
public VersionedValue(T value) { Value = value; }
public T Value { get; }
public override string ToString() => $"{Value} {Version}";
}
public static class ExtensionMethods {
public static IObservable<VersionedValue<T>> Versioned<T>(this IObservable<T> values) => values.Select(i => new VersionedValue<T>(i));
public static VersionedValue<T> AsVersionedValue<T>(this T obj) => new VersionedValue<T>(obj);
}
static void Main(string[] args) {
// same as before
//
var whole = Observable
.CombineLatest(first.Versioned(), firstIsValid.Versioned(), last.Versioned(), lastIsValid.Versioned(), (f, fv, l, lv) => new {
First = f,
Last = l,
FirstIsValid = fv,
LastIsValid = lv
})
.Where(i => i.FirstIsValid.Version > i.First.Version && i.LastIsValid.Version > i.Last.Version)
.Where(i => i.FirstIsValid.Value && i.LastIsValid.Value)
.Select(i => $"{i.First.Value} {i.Last.Value}");

How to improve query with two contexts - MVC5, LINQ n EF

I have two contexts. In one of them i have two views of which i get cods related to an entity from the another context. This query is taking too long time. How to improve it?
var negociacoes = _db.Negociacoes.Include(o=> o.User).ToArray();
var produtos = _oriDb.Vw_Produtos.ToArray();
var clientesVendedor = _oriDb.Vw_ClientesVendedores.ToArray();
var query = from n in negociacoes
join p in produtos on n.ProdutoId equals p.ProdutoId
join c in clientesVendedor on n.ClienteId equals c.codigo_entidade
select new NegociacaoView
{
NegociacaoId = n.NegociacaoId,
ProdutoId = n.ProdutoId,
Produto = p.descricao,
ClienteId = n.ClienteId,
Cliente = c.razao_social,
Rca = n.Rca,
Quantidade = n.Quantidade,
Preco = n.Preco,
Situacao = n.Situacao,
UserId = n.User.UserName,
Atendente = n.Atendente,
CondicaoId = n.CondicaoId,
DataCriacao = n.DataCriacao,
DataLiberacao = n.DataLiberacao,
Observacao = n.Observacao,
User = n.User
};
return query.ToList();
There are a couple of ways to speed this up:
It helps to run the smallest most efficient query first, then use those results to constrain the following queries.
Defining a select list so the database doesn't have to materialize every column will speed things up and use less memory.
Unfortunately, no matter how you do it in LINQ, you will end up with sql that uses large IN statements. A sproc would give you access to temp tables and joins that would be even better.
var negociacoes = _db.Negociacoes.Include(o=> o.User).ToArray();
//Use results of first query to constrain the second two. You could maybe combine the second two into one query.
var clientIds = negociacoes.Select(x => x.ClienteId);
var productIds = negociacoes.Select(x => x.ProdutoId);
var produtos = _oriDb.Vw_Produtos
.Where(x => productIds.Contains(x.ProdutoId))
//add a select. You're only using two columns from this table.
//.Select(x => new { })
.ToArray();
var clientesVendedor = _oriDb.Vw_ClientesVendedores
.Where(x => clientIds.Contains(x.codigo_entidade))
//add a select. You're only using two columns from this table.
//.Select(x => new { })
.ToArray();
var query = from n in negociacoes
join p in produtos on n.ProdutoId equals p.ProdutoId
join c in clientesVendedor on n.ClienteId equals c.codigo_entidade
select new NegociacaoView
{
NegociacaoId = n.NegociacaoId,
ProdutoId = n.ProdutoId,
Produto = p.descricao,
ClienteId = n.ClienteId,
Cliente = c.razao_social,
Rca = n.Rca,
Quantidade = n.Quantidade,
Preco = n.Preco,
Situacao = n.Situacao,
UserId = n.User.UserName,
Atendente = n.Atendente,
CondicaoId = n.CondicaoId,
DataCriacao = n.DataCriacao,
DataLiberacao = n.DataLiberacao,
Observacao = n.Observacao,
User = n.User
};
return query.ToList();

EF Append to an IQueryable a Where that generates an OR

I'm trying to achieve dynamic filtering on a table. My UI has filters that can be enabled or disabled on demand, and as you can imagine, my query should be able to know when to add filters to the query.
What I have so far is that I check if the filter object has a value, and if it does it adds a where clause to it. Example:
var q1 = DBContext.Table1
if (!string.IsNullOrEmpty(filterModel.SubjectContains))
q1 = q1.Where(i => i.Subject.Contains(filterModel.SubjectContains));
if (filterModel.EnvironmentId != null)
q1 = q1.Where(i => i.EnvironmentId == filterModel.EnvironmentId);
if (filterModel.CreatedBy != null)
q1 = q1.Where(i => i.CreatedByUserId == filterModel.CreatedBy);
var final = q1.Select(i => new
{
IssuesId = i.IssuesId,
Subject = i.Subject,
EnvironmentId = i.EnvironmentId,
CreatedBy = i.CreatedByUser.FullName,
});
return final.ToList();
The code above generates T-SQL that contains a WHERE clause for each field that uses AND to combine the conditions. This is fine, and will work for most cases.
Something like:
Select
IssueId, Subject, EnvironmentId, CreatedById
From
Table1
Where
(Subject like '%stackoverflow%')
and (EnvironmentId = 1)
and (CreatedById = 123)
But then I have a filter that explicitly needs an IssueId. I'm trying to figure out how the EF Where clause can generate an OR for me. I'm looking something that should generate a Tsql that looks like this:
Select
IssueId, Subject, EnvironmentId, CreatedById
From
Table1
Where
(Subject like '%stackoverflow%')
and (EnvironmentId = 1)
and (CreatedById = 123)
or (IssueId = 10001)
Found a solution for this that doesn't have to do multiple database call and works for me.
//filterModel.StaticIssueIds is of type List<Int32>
if (filterModel.StaticIssueIds != null)
{
//Get all ids declared in filterModel.StaticIssueIds
var qStaticIssues = DBContext.Table1.Where(i => filterModel.StaticIssueIds.Contains(i.IssuesId));
//Let's get all Issues that isn't declared in filterModel.StaticIssueIds from the original IQueryable
//we have to do this to ensure that there isn't any duplicate records.
q1 = q1.Where(i => !filterModel.StaticIssueIds.Contains(i.IssuesId));
//We then concatenate q1 and the qStaticIssues.
q1 = q1.Concat(qStaticIssues);
}
var final = q1.Select(i => new
{
IssuesId = i.IssuesId,
Subject = i.Subject,
EnvironmentId = i.EnvironmentId,
CreatedBy = i.CreatedByUser.FullName,
});
return final.ToList();

How to avoid writing duplicate queries when needing counts and then items

I have constructed a LINQ query that joins about a half dozen tables. The problem is, for paging purposes, I want to get a count first of how many items will be returned. So the issue I'm running into is having to write the exact same query twice: one to get the item count then another to build my collection of items.
Example:
using (var context = new DbContext())
{
var items = from i in context.Table1
join a in context.TableA on i.SomeProperty equals a.SomeProperty
join b in context.TableB on i.SomeOtherProperty equals b.SomeProperty
join c in context.TableC on i.AnotherProperty equals c.SomeProperty
etc.
etc.
select i;
count = items.Count();
}
return count;
.
.
.
using (var context = new DbContext())
{
var items = from i in context.Table1
join a in context.TableA on i.SomeProperty equals a.SomeProperty
join b in context.TableB on i.SomeOtherProperty equals b.SomeProperty
join c in context.TableC on i.AnotherProperty equals c.SomeProperty
etc.
etc.
select new
{
DynamicProp1 = i.SomeProperty,
DyanmicProp2 = a.SomeProperty,
DyanmicProp3 = b.SomePropery,
etc.
etc.
}
... do some stuff with 'items'...
}
I cannot think of any way to avoid this duplicate query. I need access to all the joined tables in order to build my collection. I would appreciate any tips or suggestions.
You can create method which get context and return IQueryable of items with all needed entities:
class Holder
{
TableAItem A{get;set;}
TableBItem B{get;set;}
...
}
IQueryable<Holder> GetQuery(DbContext context)
{
return from i in context.Table1
join a in context.TableA on i.SomeProperty equals a.SomeProperty
join b in context.TableB on i.SomeOtherProperty equals b.SomeProperty
join c in context.TableC on i.AnotherProperty equals c.SomeProperty
...
select new Holder
{
A = i,
B = b
....
};
}
using (var context = new DbContext())
{
var items = GetQuery(context);
count = items.Count();
}
return count;
using (var context = new DbContext())
{
var items = from r in GetQuery(context)
select new
{
DynamicProp1 = r.a.SomeProperty,
DyanmicProp2 = r.a.SomeProperty,
DyanmicProp3 = r.b.SomePropery,
etc.
etc.
}
... do some stuff with 'items'...
}
Remember that making a query doesn't execute it, this is called deferred execution. So why not make the query and then pass it around as an IQueryable<> object. For example, consider this code:
Just a simple method to return the last char from a string, but it also writes out what it's doing:
public char GetLastChar(string input)
{
Console.WriteLine("GetLastChar from {0}", input);
return input.Last();
}
Now this code using the method:
var listOfStuff = new List<string> { "string1", "string2", "string3" };
Console.WriteLine("Making the query");
var results = from s in listOfStuff
select GetLastChar(s);
Console.WriteLine("Before getting count");
var count = results.Count();
Console.WriteLine("Now enumerating the query");
foreach(var s in results)
{
Console.WriteLine(s);
}
You will see the output as follows:
Making the query
Before getting count
GetLastChar from string1
GetLastChar from string2
GetLastChar from string3
3
Now enumerating the query
GetLastChar from string1
1
GetLastChar from string2
2
GetLastChar from string3
3

Many to many - get appropriate records from table - EF LINQ

I have two tables with relationship many-to-many. Let's say A and B tables.
I also have List<List<int>> TagIdList with ids of B table's elements.
How can I find every elements from table A, who have all TagIdList[i] elements? I need just ids from table A, so it doesn't have to be all TASKS rows from table.
Example:
A: TASKS:
id: 1,2,3,4,5,6
B: TAGS:
id: 1,2,3,4
A-B links:
1-2; 1-3; 2-1; 2-2; 5-3; 5-4; 6-1; 6-6;
List<List<int>> TagIdList //(ids from TAGS)
TagIdList[0]= {2,3}
TagIdList[1]= {1}
TagIdList[2]= {2,6}
Result: (ids from TASKS)
i=0; -> 1
i=1; -> 2,6
i=2; -> null
I've tried:
List<int> tags = model.TagIdList[i].IdList; //I've got it from my View
List<TASKS> tasks = myEntity.TASKS.Where(t => t.TAGS == tags).ToList();
And I can't get tasks, because there was an error: Unable to create a constant value of type. Only primitive types are supported in this context.
Any ideas?
Your problem is here: myEntity.TASKS.Where(t => t.TAGS == tags)
If i understand the question correct you need something like this:
A compare method for two lists of int (Taken from here)
public static bool ScrambledEquals<T>(IEnumerable<T> list1, IEnumerable<T> list2) {
var cnt = new Dictionary<T, int>();
foreach (T s in list1) {
if (cnt.ContainsKey(s)) {
cnt[s]++;
} else {
cnt.Add(s, 1);
}
}
foreach (T s in list2) {
if (cnt.ContainsKey(s)) {
cnt[s]--;
} else {
return false;
}
}
return cnt.Values.All(c => c == 0);
}
Than use this method inside linq expression:
myEntity.TASKS.AsEnumerable().Where(t => ScrambledEquals<int>(t.TAGS.Select(tag=>tag.id).ToList(),tags))
I found solution. It's maybe not ideal, but works.
List<int> tags = model.TagIdList[i].IdList;
List<List<int>> whole_list = new List<List<int>>();
foreach (var t in tags) //I'm looking for every tasks' ids in every given tag
{
var temp = myEntity.TAGS.Find(t).TASKS.Select(task => task.Id).ToList();
whole_list.Add(temp);
}
//now I want first list from whole_list to compare with the other lists
List<int> collection = whole_list[0]; //collection it's tasks
whole_list.Remove(collection);
//I'm taking a part which is common to all lists from whole_list
foreach (List<int> k in whole_list)
{
var temp = collection.Intersect(k);
collection = temp.ToList();
}
//the collection is now what I wanted for TagIdList[i] - every task which has every tags from TagIdList[i].IdList