Linq to Entities Complex Dynamic Search - entity-framework

We're using the Entity Framework (MySQL connector) and are creating a central Search facility on our web application.
This link is almost exactly what I need, aside from the fact that he's using pre-defined entities and properties. In our search scenario, we'll have a dynamic number of search terms and fields (ie: user chooses to search on surname, value and city, or provider and advisor).
Is it possible to achieve this kind of functionality with LINQ, so that we can leverage the deferred loading mechanism? I really wanted to avoid generating SQL strings, if possible. I looked at Dynamic LINQ with Expression Trees but couldn't get this to work (or this).

I know you indicated that you wanted to avoid generating SQL strings, but that is often the easiest way. (Much easier than custom Expression Trees). If you are doing this in EF, I recommend you check out Entity Sql which works against your conceptual model but allows for more dynamic querying options than LINQ. LINQ is really suited to compile time query rather than run time queries. You can read up on Entity SQL at http://msdn.microsoft.com/en-us/library/bb387145.aspx.

since last week, we have a similar problem to face, here is an idea i just had for it. thought i share it with you.
interface IPerson
{
DateTime BirthDay { get; set; }
string City { get; set; }
string FirstName { get; set; }
string LastName { get; set; }
}
interface IFilter { }
interface IPersonFilter : IFilter { }
class PersonFilter : IPersonFilter
{
public DateTime? BirthDay { get; set; }
public string City { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
}
static IQueryable<TSource> ApplyFilter<TSource, TFilter>(IQueryable<TSource> source, TFilter filter) where TFilter : IFilter
{
const BindingFlags bindingFlags = BindingFlags.Public|BindingFlags.Instance|BindingFlags.GetProperty;
var retval = source;
foreach (var filterProperty in filter.GetType().GetProperties(bindingFlags))
{
var elementParameter = Expression.Parameter(source.ElementType, "type");
var elementProperty = Expression.Property(elementParameter, filterProperty.Name);
var value = filterProperty.GetGetMethod().Invoke(filter, null);
if (value != null)
{
var constantValue = Expression.Constant(value, elementProperty.Type);
var expression = Expression.Equal(elementProperty, constantValue);
retval = retval.Where(Expression.Lambda<Func<TSource, bool>>(expression, elementParameter));
}
}
return retval;
}
so the idea is, that you have a filter where the names of the properties of filter match the property names of the object you want to run the filter against. and if the value of the property is not null, i build a expression for it. For the simplicity i do build Expression.Equal expressions only, but i am thinking about extending it.

Related

Save document with the member number instead the name with protobuf-net and MongoDB

I saw somewhere that with the Go MongoDB driver it is possible to save a document with the order number instead of the field name.
They end up with this in the database:
{
"3": "foo",
"10": 1,
"33": 123456
"107": {
"2": "bar",
"1": "foo"
}
}
I like the idea!
So, I tried to find a way to do the same with the MongoDB C# driver.
I have the code below but I am not sure what I should bring from the protobut-net to get the member order number.
var pack = new ConventionPack();
pack.AddMemberMapConvention("numbered", m => m.SetElementName( WHAT TO PUT HERE ));
ConventionRegistry.Register("numbered", pack, type => true);
The SetElementName takes a string parameter.
How can I grab the order number of a member from protobuf-net?
Something like ...Member.Order.ToString()
I don't know if this whole thing is a great idea but I want to test it.
Thanks
-- UPDATE --
Just to add more information. I am using inheritance for my models to use generics.
[BsonDiscriminator("Base", RootClass = true)]
[DataContract]
public abstract class Base
{
[BsonId]
[BsonRepresentation(BsonType.ObjectId)]
[ProtoMember(1)]
public string Id { get; set; }
[BsonDateTimeOptions]
[ProtoMember(2)]
public DateTime CreatedDate { get; private set; } = DateTime.UtcNow;
[BsonDateTimeOptions]
[ProtoMember(3)]
public DateTime UpdatedDate { get; set; } = DateTime.UtcNow;
}
[ProtoContract]
public class Todo : Base
{
[ProtoMember(10)]
public string Title { get; set; }
[ProtoMember(20)]
public string Content { get; set; }
[ProtoMember(30)]
public string Category { get; set; }
}
And I added this line as shown in the protobuf-net documentation:
RuntimeTypeModel.Default[typeof(Base)].AddSubType(42, typeof(Todo));
So with that and what Marc showed to get the member's number, I end up having a custom Convention Class in MongoDB with <T> so I can use it for other objects:
public class NumberedElementNameConvention<T> : ConventionBase, IMemberMapConvention where T : Base
{
public void Apply(BsonMemberMap memberMap)
{
var members = RuntimeTypeModel.Default[typeof(T)].GetFields();
foreach (var member in members)
{
memberMap.SetElementName(member.FieldNumber.ToString());
}
}
}
And the registration of this Convention is done like so:
var pack = new ConventionPack { new NumberedElementNameConvention<Todo>() };
ConventionRegistry.Register("NumberedName", pack, type => true);
After running this I get this error:
Grpc.AspNetCore.Server.ServerCallHandler[6]
Error when executing service method 'CreateOne'.
MongoDB.Bson.BsonSerializationException: The property 'UpdatedDate' of type 'Nnet.Models.Base' cannot use element name '30' because it is already being used by property 'CreatedDate'...
Also, when I run the code below I am expecting to get all members of the Todo object.
var members = RuntimeTypeModel.Default[typeof(Todo)].GetFields();
foreach (var member in members)
{
Console.WriteLine($"{member.FieldNumber}: {member.Member.Name}");
}
However, I am not getting those inherited from the Base object:
❯ dotnet run
10: Title
20: Content
30: Category
The field metadata for protobuf-net is available from the RuntimeTypeModel API, for example:
var members = RuntimeTypeModel.Default[yourType].GetFields();
foreach (var member in members)
{
Console.WriteLine($"{member.FieldNumber}: {member.Member.Name}");
}
The .FieldNumber gives the protobuf field-number, and .Member gives the MemberInfo of the corresponding field or property. You may want to do some level of caching if the m => m.SetElementName( WHAT TO PUT HERE ) is evaluated lots of times for the same m, so you don't perform unnecessary work - but: before you do, just add some logging to the lambda first, and see how often it gets called: if it isn't too often, maybe don't worry about it.
Note that there is also a lookup on MetaType that allows query by MemberInfo:
var member = RuntimeTypeModel.Default[yourType][memberInfo];
Re the edit; in this region:
var members = RuntimeTypeModel.Default[typeof(T)].GetFields();
foreach (var member in members)
{
memberMap.SetElementName(member.FieldNumber.ToString());
}
I believe you're meant to identify the relevant field from memberMap - i.e. in this context you're only talking about one field at the time; I suspect what is happening is that for each member in turn you're changing the element name multiple times, leaving it at the last protobuf field defined.
Separately, there's a complication of inheritance; protobuf-net doesn't implement inheritance in a flat way - instead, the base type is also expected to be a [ProtoContract] and is meant to define a [ProtoInclude(...)] for each derived type; the field numbers are type-specific, meaning: both the base type and the derived type can legally have a field 1. If you need to describe inheritance, and you are determined to use protobuf-net's model, then you would need to handle this; for example, you could use the [ProtoInclude(...)] number as a prefix on each, so Base.Id is "1", and if we imagine that Todo has field 5 in the [ProtoInclude(...)], then Todo.Title could be "5.10".
Alternatively: if you're not actively using protobuf-net: maybe just use your own attribute for the numbers? or there's usually an inbuilt attribute that the serializer you've chosen would use directly.
Okay now! So after a some investigation I end up with this simple way to do it with Marc's help. In MongoDB instead of using attributes to decorate models and its properties, it is possible to use code within BsonClassMap. Within that class I add the foreach loop that Marc provided and the right parameters, we can now have numbers instead names.
On the Client side and Server side it is this same code:
//Base Model ClassMap
BsonClassMap.RegisterClassMap<Base>(cm =>
{
cm.AutoMap();
foreach (var member in RuntimeTypeModel.Default[typeof(Base)].GetFields())
{
cm.MapMember(typeof(Base).GetMember(member.Member.Name)[0])
.SetElementName(member.FieldNumber.ToString())
.SetOrder(member.FieldNumber);
}
});
//Todo Model ClassMap
BsonClassMap.RegisterClassMap<Todo>(cm =>
{
cm.AutoMap();
foreach (var member in RuntimeTypeModel.Default[typeof(Todo)].GetFields())
{
cm.MapMember(typeof(Todo).GetMember(member.Member.Name)[0])
.SetElementName(member.FieldNumber.ToString())
.SetOrder(member.FieldNumber);
}
});
it's a little ugly but you can rework it.
One thing to note is that MongoDB has the control over the Id. In the database anything that represent the object id become _id. Same thing when you insert a new document in the database a _t field is added if you use Discriminator (I am not sure if it's full related). Basically, every member beginning with a underscore is reserved. See the image below after running de code:
You can refer to the question above in the update section to see if this result represent the models with the given orders (it does).
Here is the code I use for insertion and queries:
// INSERT
var client = channel.CreateGrpcService<IBaseService<Todo>>();
var reply = await client.CreateOneAsync(
new Todo
{
Title = "Some Title"
}
);
// FIND BY ID
var todoId = new UniqueIdentification { Id = "613c110a073055f0d87a0e27"};
var res = await client.GetById(todoId);
// FIND ONE BY QUERY FILTER REQUEST
...
var filter = Builders<Todo>.Filter.Eq("10", "Some Title");
var filterString = filter.Render(documentSerializer, serializerRegistry);
...
The last one above it's a query with the number ("10") of the property Title. But it's possible in the same way to query with the property name, like so:
// FIND ONE BY QUERY FILTER REQUEST
...
var filter = Builders<Todo>.Filter.Eq(e => e.Title, "Some Title");
var filterString = filter.Render(documentSerializer, serializerRegistry);
...
What is great with this approach is that these BsonClassMap are called once on the Client or/and Server when they are initiated.
I just realize that this might not be a good idea because it is going to be painful to prevent collision between numbers. The order numbers in the code below is possible:
[BsonDiscriminator("Base", RootClass = true)]
[DataContract]
public abstract class Base
{
[BsonId]
[BsonRepresentation(BsonType.ObjectId)]
[ProtoMember(1)]
public string Id { get; set; }
[BsonDateTimeOptions]
[ProtoMember(2)]
public DateTime CreatedDate { get; private set; } = DateTime.UtcNow;
[BsonDateTimeOptions]
[ProtoMember(3)]
public DateTime UpdatedDate { get; set; } = DateTime.UtcNow;
}
[ProtoContract]
public class Todo : Base
{
[ProtoMember(1)]
public string Title { get; set; }
[ProtoMember(2)]
public string Content { get; set; }
[ProtoMember(3)]
public string Category { get; set; }
}
but there is going to be three collisions if the foreach loop runs.
Yeah... :/
This is where Marc's second solution comes in, where you put a prefix... I am going to keep the name convention by default.
Cheers!

How do you map strings in the database to enums in your model without introducing a second property?

My database has a table like this:
Cats
- CatId INT PK
- Name VARCHAR(100)
- FavoriteToy VARCHAR(100)
And my code looks like this:
Cat.cs
public int CatId { get; set; }
public string Name { get; set; }
public Toy FavoriteToy {get; set; }
StaticVariables.cs
public enum Toy { Box, Ball, StuffedAnimal }
In a normalized database I would use a lookup table in the database to store all the toys and then the Cats table would just store a ToyId. But for this situation it's a lot easier to just store the FavoriteToy as a string even though it will be redundant.
The problem is I don't know how to convert a string in the database to an enum in code without creating a second FavoriteToyString property and having FavoriteToy just be a computed that returns the enum derived from FavoriteToyString.
I've heard this might be possible in the current version of entity framework. Is that true? Can you please show me how to do this?
You may use DTO class and automapper to solve your issue :)
Generally, yes a lookup table reference is a better option since your data can comply with referential integrity. That is, No cat records with toys that your Enum hopefully doesn't contain. (Though your Enum would need to be kept in sync with the Toys table.) You can configure EF to store enumerations as a string using a bit of a trick with the mapping:
public class Cat
{
[Key]
public int CatId { get; set; }
public string Name { get; set; }
[Column("FavoriteToy")]
public string FavoriteToyMapped { get; set; }
[NotMapped]
public Toy FavoriteToy
{
get { return (Toy)Enum.Parse(typeof(Toy), FavoriteToyMapped); }
set { FavoriteToyMapped = value.ToString(); }
}
}
The caveat of this approach is that where you might use Linq to Entity to filter on your cat's favorite toy, you need to reference the FavoriteToyMapped value in the query expression because EF/DB won't know what FavoriteToy is.
I.e.
Cats with a favorite toy of "Yarn"
var catsThatLoveYarn = context.Cats.Where(c => c.FavoriteToyMapped == Toys.Yarn.ToString()).ToList();
// not
var catsThatLoveYarn = context.Cats.Where(c => c.FavoriteToy == Toys.Yarn).ToList();
// Will error because EF doesn't map that property.
Once you are working with instances of entities, that the set of entities has been pulled back from the database, you can further access/refine queries with FavoriteToy. Just be cautious and prepared for the unknown field if you use it too early and EF goes and tries to compose a query.
var threeYearOldCats = context.Cats.Where(c => c.Age == 3).ToList();
var threeYearOldCatsThatLoveYarn = threeYearOldCats.Where(c => c.FavoriteToy == Toys.Yarn).ToList();
This is Ok because the .ToList() in the first query executed the EF-to-SQL, so threeYearOldCats is now a local List<Cat> of cat entities, not an IQueryable<Cat>.

DDD Entity Framework Value Type

I'm struggling with using EF6 with DDD principles, namely value objects attached to aggregates. I can't seem to get migrations to generate that reflect the model and I feel like I'm fighting the tooling instead of actually being productive. Given that a NoSQL implementation is probably more appropriate, this is what I'm stuck with.
The first thing that I ran into was the lack of support for interface properties on an EF entity. The work around for that was to add concrete properties to the entity for each of the implementations, but not to the interface. When I implemented the interface, I added logic to return the right one. I had to do this in order to get any migrations to create the properties for the Policies. See Fund.LargestBalanceFirstAllocationPolicy and Fund.PercentageBasedAllocationPolicy This was annoyance one.
The current annoyance and the genesis of the question is the PercentageBasedAllocationPolicy.AllocationValues property. No matter what I do, when running add-migration, I don't get any tables or fields to represent the AllocationValues. This is basically a collection of DDD value objects hanging off of another value object, which hangs off of an aggregate.
I'm convinced that the model and code are correct to do what I want, but EF keeps getting in the way. In MongoDB, when dealing with an interface property, it actually stores the object type in a string so that it knows how to rehydrate the object. I'm considering serializing the problem areas here to a blob and storing it on the object now, which is just as evil...
public interface IFund
{
Guid Id {get;}
string ProperName {get;}
IAllocationPolicy AllocationPolicy{get;}
void ChangeAllocationPolicy(IAllocationPolicy newAllocationPolicy)
}
public class Fund : IFund
{
public Fund()
{
}
public Fund(Guid id, string nickName, string properName)
{
Id = id;
Nickname = nickName;
ProperName = properName;
// This is stupid too, but you have to instantiate these objects inorder to save or you get some EF errors. Make sure the properties on these objects are all defaulted to null.
LargestBalanceFirstAllocationPolicy = new LargestBalanceFirstAllocationPolicy();
PercentageBasedAllocationPolicy = new PercentageBasedAllocationPolicy();
}
public Guid Id { get; private set; }
public string ProperName { get; private set; }
// Do not add this to the interface. It's here for EF reasons only. Do not use internally either. Use the interface implemention of AllocationPolicy instead
public LargestBalanceFirstAllocationPolicy LargestBalanceFirstAllocationPolicy
{
get; private set;
}
// Do not add this to the interface. It's here for EF reasons only. Do not use internally either. Use the interface implemention of AllocationPolicy instead
public PercentageBasedAllocationPolicy PercentageBasedAllocationPolicy
{
get; private set;
}
public void ChangeAllocationPolicy(IAllocationPolicy newAllocationPolicy)
{
if (newAllocationPolicy == null) throw new DomainException("Allocation policy is required");
var allocationPolicy = newAllocationPolicy as PercentageBasedAllocationPolicy;
if (allocationPolicy != null) PercentageBasedAllocationPolicy = allocationPolicy;
var policy = newAllocationPolicy as LargestBalanceFirstAllocationPolicy;
if (policy != null ) LargestBalanceFirstAllocationPolicy = policy;
}
public IAllocationPolicy AllocationPolicy
{
get {
if (LargestBalanceFirstAllocationPolicy != null)
return LargestBalanceFirstAllocationPolicy;
if (PercentageBasedAllocationPolicy != null)
return PercentageBasedAllocationPolicy;
return null;
}
}
}
public interface IAllocationPolicy
{
T Accept<T>(IAllocationPolicyVisitor<T> allocationPolicyVisitor);
}
public class LargestBalanceFirstAllocationPolicy : IAllocationPolicy
{
public T Accept<T>(IAllocationPolicyVisitor<T> allocationPolicyVisitor)
{
return allocationPolicyVisitor.Visit(this);
}
}
[ComplexType]
public class PercentageBasedAllocationPolicy : IAllocationPolicy
{
public PercentageBasedAllocationPolicy()
{
AllocationValues = new List<PercentageAllocationPolicyInfo>();
}
public List<PercentageAllocationPolicyInfo> AllocationValues { get; private set; }
public T Accept<T>(IAllocationPolicyVisitor<T> allocationPolicyVisitor)
{
return allocationPolicyVisitor.Visit(this);
}
}
[ComplexType]
public class PercentageAllocationPolicyInfo
{
public Guid AssetId { get; private set; }
public decimal Percentage { get; private set; }
}
A value type (in EF marked as ComplexType) will never have any tables. The reason being is that a value types are (by definition) really just values. They don't have any Id( otherwise they would be enities) thus you can't create a table for them.
also if i review the requirements for complex type in entity framework https://msdn.microsoft.com/en-us/library/bb738472(v=vs.100).aspx i notice that you can't use inheritance on complex types. Thus if you want to use complex type in your entity framework as you've shown here then you need to make your property a PercentageBasedAllocationPolicy instead of an IAllocationPolicy.
Alternatively you could turn it into an entity with automatic generated keys.

Searching the Entity Framework domain model utilising Code First

Got a very difficult EntityFramework Code First question. I'll keep this as simple as possible.
Imagine we have n number of classes, lets start with 2 for now
public class Person
{
public string Name { get; set; }
}
public class Address
{
public string AddressLine1 { get; set; }
public string AddressLine2 { get; set; }
}
Now then, what I want to do is be able to search the domain model with a single string, i.e. something like DbContext.Search( "Foo" ). The call would search both the person and address tables for a string match and would return a list populated with both Person and Address entities.
Have to say I am not entirely clear how to go about it but I am considering using DataAnnotations to do something like this
public class Person
{
**[Searchable]**
public string Name { get; set; }
}
public class Address
{
**[Searchable]**
public string AddressLine1 { get; set; }
**[Searchable]**
public string AddressLine2 { get; set; }
}
Am I on the right track?
Should I use the Fluent API instead?
Reflection?
Any and all thoughts massively appreciated.
the Find method searches only in the Primary Key column. If we don't make any column explicitly primary key column then find method will throw error. Generally EF convention takes propertyName+id as the primary key in the class. But if you want to search with Name then Make add [Key] to the property. it will become primary key and u will be able to find properties.
dbContext.Addresses.find("Foo");
Create a new object type onto which you'll project 2 types of search results:
public class Result
{
public string MainField { get; set; }
// you may have other properties in here.
}
Then find entities of each type that match your criteria, projecting them onto this type:
var personResults = DbContext.Persons
.Where(p => p.Name == "Foo")
.Select(p => new Result{MainField = p.Name});
// don't forget to map to any other properties you have in Result as well
var addressResults = DbContext.Adresses
.Where(a =>
a.AddressLine1 == "Foo" ||
a.AddressLine2 == "Foo"
).
.Select(a => new Result{MainField = a.AddressLine1 + ", " + a.AddressLine2 });
// again, don't forget to map to any other properties in Result
Then merge the lists:
var allResults = personResults.Union(addressResults).ToList();
...at which point you can sort the list however you like.
"Result" and "MainField", are rather generic; just using them because I am not thoroughly aware of your domain model.

Entity Framework Code First Update by Stub Entity

Is it possible in EF Code-First to update without querying the entire row in db by using stub objects,...
e.g.
public class Dinner
{
public int DinnerID { get; set; }
public string Title { get; set; }
public DateTime EventDate { get; set; }
public string Address { get; set; }
public string Country { get; set; }
public string HostedBy { get; set; }
}
Dinner dinner = dinner.DinnerID = 1;
dinner.HostedBy = "NameChanged"
nerdDinners.SaveChanges();
will the code above create an Update Statement which will make the following columns null for the row of DinnerID 1 ?
Title, EventDate, Address, Country
Is there a way or method like "PropertyModified" = true, then the rest make them = false, so that HostedBy is the only one that will be updated?
I think you are looking for ApplyCurrentValues
public void UpdateDinner(Dinner existingDinner)
{
var stub = new Dinner { DinnerId = existingDinner.DinnerId };
ctx.Dinners.Attach(stub);
ctx.ApplyCurrentValues(existingDinner);
ctx.SaveChanges();
}
ApplyCurrentValues copies the scalar values from the existing object to the object in the graph (in the above case - the stub entity).
From the Remarks section on MSDN:
Any values that differ from the original values of the object are marked as modified.
Is that what your after?
To build on Paul's answer, the following will work when you are using EF Model or Database First:
context.ObjectStateManager.GetObjectStateEntry(dinner).SetModifiedProperty("HostedBy");
I think you are looking for the Attach() method.
Attaching and Detaching Objects
Try this maybe, it is specific to EF Code First which seems to do it differently than just EF.
var dinner = context.Dinners.Find(1);
context.Entry(dinner).Property(d => d.HostedBy).IsModified = true;
context.SaveChanges();
From ADO.NET team blog
"Marking a property as modified forces an update to be send to the database for the property when SaveChanges is called even if the current value of the property is the same as its original value."