How do check for duplicate before AddRange? - entity-framework-core

Question

This is my sample model,
Name
Age
I have a list of records for which I will use Entity Framework Core AddRange. This is my sample data, say list 1
John, 21
Mike, 18
Rick, 19
Alex, 20
I have another query which is a list of the existing records, say list 2.
John, 21
Alex, 20
I want to compare these 2 lists to get what exists in list 1 but not in list 2.
Mike, 18
Rick, 19
I tried using LINQ except but it can't be used for complex types. I also tried to use .Any like below but it returns nothing.
var result = list1.Where(x=> !list2.Any(y=>x.Name == y.Name));
I do not want to check and insert one record at one time. I want to insert by list of objects using AddRange.
Is there any Nuget package that can easily get this?

Answer 1

For not so big amount of records, you can use the following extension:
var nonExistent = await ctx.Some.GetNonExistentAsync(list2, x => x.Name);
Or with complex key:
var nonExistent = await ctx.Some.GetNonExistentAsync(list2,
x => new { x.Name, x.Other });
It will generate effective SQL to check which records do not exist.
And implementation:
public static class QueryableExtensions
{
public static async Task<List<T>> GetNonExistentAsync<T, TKey>(this IQueryable<T> query, IEnumerable<T> records,
Expression<Func<T, TKey>> keySelector, CancellationToken cancellationToken = default)
{
var recordsList = records.ToList();
var keySelectorCompiled = keySelector.Compile();
var predicate = BuildPredicate(recordsList, keySelector, keySelectorCompiled);
var existent = (await query
.Where(predicate)
.Select(keySelector)
.ToListAsync(cancellationToken))
.ToHashSet();
var result = recordsList.Where(r => !existent.Contains(keySelectorCompiled(r))).ToList();
return result;
}
private static Expression<Func<T, bool>> BuildPredicate<T, TKey>(IEnumerable<T> records, Expression<Func<T, TKey>> keySelector, Func<T, TKey> keySelectorCompiled)
{
var members = CollectMembers(keySelector.Body).ToList();
if (members.Count == 0)
throw new InvalidOperationException("No key found");
Expression? predicate = null;
if (members.Count == 1 && keySelector.Body.NodeType == ExpressionType.MemberAccess)
{
// we can use Contains
var recordsSequence = Expression.Constant(records.Select(keySelectorCompiled));
predicate = Expression.Call(typeof(Enumerable), nameof(Enumerable.Contains), new[] { typeof(TKey) },
recordsSequence, keySelector.Body);
}
else
{
foreach (var record in records)
{
Expression? subPredicate = null;
var recordExpression = Expression.Constant(record);
foreach (var m in members)
{
var memberExpression =
ReplacingExpressionVisitor.Replace(keySelector.Parameters[0], recordExpression, m);
var equality = Expression.Equal(m, memberExpression);
subPredicate = subPredicate == null ? equality : Expression.AndAlso(subPredicate, equality);
}
if (subPredicate == null)
throw new InvalidOperationException(); // should never happen
predicate = predicate == null ? subPredicate : Expression.OrElse(predicate, subPredicate);
}
}
predicate ??= Expression.Constant(false);
return Expression.Lambda<Func<T, bool>>(predicate, keySelector.Parameters);
}
private static IEnumerable<Expression> CollectMembers(Expression expr)
{
switch (expr.NodeType)
{
case ExpressionType.New:
{
var ne = (NewExpression)expr;
foreach (var a in ne.Arguments)
foreach (var m in CollectMembers(a))
{
yield return m;
}
break;
}
case ExpressionType.MemberAccess:
yield return expr;
break;
default:
throw new InvalidOperationException();
}
}
}