BSonElement to c# Data Type - mongodb

I have an "BSonElement" in my DB and i have retry it with an standard Query.
The problem was that i can't Cast BsonDocument to Type.
Example:
UPDATE 1:
public partial class item_Stat
{
[BsonExtraElements]
public BsonDocument all_stat;
}
Basically, i have into my DB 10-15 property (field) that I can read with "BsonExtraElements". In this way, i can retry property without define it
in C#.
all_stat, can have 10-15-20 property that dinamically change.
C# is typed language, so I can't define this property in C# and i have used ExtraElements.
The problem is when i QUERY the Object from DB.
var item_db = myMongoCollection.find(theQuery); // find the OBJECT
item_db.all_stat // all the property hare HERE
// find the property "category_01"
var i = item_db.all_stat.Where(p => p.Name == "category_01").Single();
// ok, i have found the Category, so i can cast it to C# Data Type
var typed_value = (ItemStatSingle) i.Value // BsonElement to ItemStatSingle

Here's an example of what you can do, given a class from your domain model like:
public class Employee
{
public ObjectId Id { get; set; }
public string Name { get; set; }
}
You can use your class like this:
var collection = database.GetCollection<employee>("employees");
var employee = new Employee { Name = "John Smith" };
collection.Insert(employee);
employee = collection.FindOne();</employee>

BsonElement.Value is of type BsonValue. Use one of the As* method to convert appropriately. What is the type of the value here? Since you have a user defined type your best option is to retrieve as Barrie says above. If you want to customize the "mapping" refer to the Serialization tutorial http://www.mongodb.org/display/DOCS/CSharp+Driver+Serialization+Tutorial

You can't just cast a BsonDocument from your extra elements. You have to Deserialize it.
Suppose you have a class C
public class C
{
public int X;
}
and an extraDocuments variable (analogous to your item_db.all_stat property) initialized like this:
var extraElements = new BsonDocument();
var c = new C { X = 1 };
extraElements["c"] = c.ToBsonDocument();
Then you could extract the "c" value and deserialize it like this:
var r = BsonSerializer.Deserialize<C>(extraElements["c"].AsBsonDocument);
Console.WriteLine(r.ToJson());

Related

Save document with the member number instead the name with protobuf-net and MongoDB

I saw somewhere that with the Go MongoDB driver it is possible to save a document with the order number instead of the field name.
They end up with this in the database:
{
"3": "foo",
"10": 1,
"33": 123456
"107": {
"2": "bar",
"1": "foo"
}
}
I like the idea!
So, I tried to find a way to do the same with the MongoDB C# driver.
I have the code below but I am not sure what I should bring from the protobut-net to get the member order number.
var pack = new ConventionPack();
pack.AddMemberMapConvention("numbered", m => m.SetElementName( WHAT TO PUT HERE ));
ConventionRegistry.Register("numbered", pack, type => true);
The SetElementName takes a string parameter.
How can I grab the order number of a member from protobuf-net?
Something like ...Member.Order.ToString()
I don't know if this whole thing is a great idea but I want to test it.
Thanks
-- UPDATE --
Just to add more information. I am using inheritance for my models to use generics.
[BsonDiscriminator("Base", RootClass = true)]
[DataContract]
public abstract class Base
{
[BsonId]
[BsonRepresentation(BsonType.ObjectId)]
[ProtoMember(1)]
public string Id { get; set; }
[BsonDateTimeOptions]
[ProtoMember(2)]
public DateTime CreatedDate { get; private set; } = DateTime.UtcNow;
[BsonDateTimeOptions]
[ProtoMember(3)]
public DateTime UpdatedDate { get; set; } = DateTime.UtcNow;
}
[ProtoContract]
public class Todo : Base
{
[ProtoMember(10)]
public string Title { get; set; }
[ProtoMember(20)]
public string Content { get; set; }
[ProtoMember(30)]
public string Category { get; set; }
}
And I added this line as shown in the protobuf-net documentation:
RuntimeTypeModel.Default[typeof(Base)].AddSubType(42, typeof(Todo));
So with that and what Marc showed to get the member's number, I end up having a custom Convention Class in MongoDB with <T> so I can use it for other objects:
public class NumberedElementNameConvention<T> : ConventionBase, IMemberMapConvention where T : Base
{
public void Apply(BsonMemberMap memberMap)
{
var members = RuntimeTypeModel.Default[typeof(T)].GetFields();
foreach (var member in members)
{
memberMap.SetElementName(member.FieldNumber.ToString());
}
}
}
And the registration of this Convention is done like so:
var pack = new ConventionPack { new NumberedElementNameConvention<Todo>() };
ConventionRegistry.Register("NumberedName", pack, type => true);
After running this I get this error:
Grpc.AspNetCore.Server.ServerCallHandler[6]
Error when executing service method 'CreateOne'.
MongoDB.Bson.BsonSerializationException: The property 'UpdatedDate' of type 'Nnet.Models.Base' cannot use element name '30' because it is already being used by property 'CreatedDate'...
Also, when I run the code below I am expecting to get all members of the Todo object.
var members = RuntimeTypeModel.Default[typeof(Todo)].GetFields();
foreach (var member in members)
{
Console.WriteLine($"{member.FieldNumber}: {member.Member.Name}");
}
However, I am not getting those inherited from the Base object:
❯ dotnet run
10: Title
20: Content
30: Category
The field metadata for protobuf-net is available from the RuntimeTypeModel API, for example:
var members = RuntimeTypeModel.Default[yourType].GetFields();
foreach (var member in members)
{
Console.WriteLine($"{member.FieldNumber}: {member.Member.Name}");
}
The .FieldNumber gives the protobuf field-number, and .Member gives the MemberInfo of the corresponding field or property. You may want to do some level of caching if the m => m.SetElementName( WHAT TO PUT HERE ) is evaluated lots of times for the same m, so you don't perform unnecessary work - but: before you do, just add some logging to the lambda first, and see how often it gets called: if it isn't too often, maybe don't worry about it.
Note that there is also a lookup on MetaType that allows query by MemberInfo:
var member = RuntimeTypeModel.Default[yourType][memberInfo];
Re the edit; in this region:
var members = RuntimeTypeModel.Default[typeof(T)].GetFields();
foreach (var member in members)
{
memberMap.SetElementName(member.FieldNumber.ToString());
}
I believe you're meant to identify the relevant field from memberMap - i.e. in this context you're only talking about one field at the time; I suspect what is happening is that for each member in turn you're changing the element name multiple times, leaving it at the last protobuf field defined.
Separately, there's a complication of inheritance; protobuf-net doesn't implement inheritance in a flat way - instead, the base type is also expected to be a [ProtoContract] and is meant to define a [ProtoInclude(...)] for each derived type; the field numbers are type-specific, meaning: both the base type and the derived type can legally have a field 1. If you need to describe inheritance, and you are determined to use protobuf-net's model, then you would need to handle this; for example, you could use the [ProtoInclude(...)] number as a prefix on each, so Base.Id is "1", and if we imagine that Todo has field 5 in the [ProtoInclude(...)], then Todo.Title could be "5.10".
Alternatively: if you're not actively using protobuf-net: maybe just use your own attribute for the numbers? or there's usually an inbuilt attribute that the serializer you've chosen would use directly.
Okay now! So after a some investigation I end up with this simple way to do it with Marc's help. In MongoDB instead of using attributes to decorate models and its properties, it is possible to use code within BsonClassMap. Within that class I add the foreach loop that Marc provided and the right parameters, we can now have numbers instead names.
On the Client side and Server side it is this same code:
//Base Model ClassMap
BsonClassMap.RegisterClassMap<Base>(cm =>
{
cm.AutoMap();
foreach (var member in RuntimeTypeModel.Default[typeof(Base)].GetFields())
{
cm.MapMember(typeof(Base).GetMember(member.Member.Name)[0])
.SetElementName(member.FieldNumber.ToString())
.SetOrder(member.FieldNumber);
}
});
//Todo Model ClassMap
BsonClassMap.RegisterClassMap<Todo>(cm =>
{
cm.AutoMap();
foreach (var member in RuntimeTypeModel.Default[typeof(Todo)].GetFields())
{
cm.MapMember(typeof(Todo).GetMember(member.Member.Name)[0])
.SetElementName(member.FieldNumber.ToString())
.SetOrder(member.FieldNumber);
}
});
it's a little ugly but you can rework it.
One thing to note is that MongoDB has the control over the Id. In the database anything that represent the object id become _id. Same thing when you insert a new document in the database a _t field is added if you use Discriminator (I am not sure if it's full related). Basically, every member beginning with a underscore is reserved. See the image below after running de code:
You can refer to the question above in the update section to see if this result represent the models with the given orders (it does).
Here is the code I use for insertion and queries:
// INSERT
var client = channel.CreateGrpcService<IBaseService<Todo>>();
var reply = await client.CreateOneAsync(
new Todo
{
Title = "Some Title"
}
);
// FIND BY ID
var todoId = new UniqueIdentification { Id = "613c110a073055f0d87a0e27"};
var res = await client.GetById(todoId);
// FIND ONE BY QUERY FILTER REQUEST
...
var filter = Builders<Todo>.Filter.Eq("10", "Some Title");
var filterString = filter.Render(documentSerializer, serializerRegistry);
...
The last one above it's a query with the number ("10") of the property Title. But it's possible in the same way to query with the property name, like so:
// FIND ONE BY QUERY FILTER REQUEST
...
var filter = Builders<Todo>.Filter.Eq(e => e.Title, "Some Title");
var filterString = filter.Render(documentSerializer, serializerRegistry);
...
What is great with this approach is that these BsonClassMap are called once on the Client or/and Server when they are initiated.
I just realize that this might not be a good idea because it is going to be painful to prevent collision between numbers. The order numbers in the code below is possible:
[BsonDiscriminator("Base", RootClass = true)]
[DataContract]
public abstract class Base
{
[BsonId]
[BsonRepresentation(BsonType.ObjectId)]
[ProtoMember(1)]
public string Id { get; set; }
[BsonDateTimeOptions]
[ProtoMember(2)]
public DateTime CreatedDate { get; private set; } = DateTime.UtcNow;
[BsonDateTimeOptions]
[ProtoMember(3)]
public DateTime UpdatedDate { get; set; } = DateTime.UtcNow;
}
[ProtoContract]
public class Todo : Base
{
[ProtoMember(1)]
public string Title { get; set; }
[ProtoMember(2)]
public string Content { get; set; }
[ProtoMember(3)]
public string Category { get; set; }
}
but there is going to be three collisions if the foreach loop runs.
Yeah... :/
This is where Marc's second solution comes in, where you put a prefix... I am going to keep the name convention by default.
Cheers!

How to use AutoMapper to map an Enum Id to a queryable projection based on Enum Values?

I am trying to map an Enum value (Int16) to an IQueryable projection so that the string representation of the Enum value is used rather than the integer value for the database sorting.
The approach I used was taken from here.
Enum as follows:
public enum SafetyCategoryEnum : Int16 { High, Medium, Low }
See my AutoMapper mapping below:
config.CreateMap<SupplierContractEntity, SupplierContractEntityWrapper>()
.ForMember(dest => dest.SafetyCategoryEnumId,
opt => { opt.MapFrom(EnumerableExpressionHelper.CreateEnumToStringExpression((SupplierContractEntity e) => e.SafetyCategoryEnumId)); })
SupplierContractEntity is the EntityFramework Entity.
public class SupplierContractEntity : Entity
{
//Other properties removed for brevity
public Int16 SafetyCategoryEnumId { get; set; }
}
SupplierContractEntityWrapper is a custom business object:
public class SupplierContractEntityWrapper : EntityWrapper
{
//Other properties removed for brevity
public SafetyCategoryEnum? SafetyCategoryEnumId { get; set; }
}
Expression mapping is reversed in AutoMapper which is why the Entity maps to the business object
Implementation of CreateEnumToStringExpression:
public static class EnumerableExpressionHelper
{
public static Expression<Func<TSource, string>> CreateEnumToStringExpression<TSource, TMember>(Expression<Func<TSource, TMember>> memberAccess, string defaultValue = "")
{
var type = typeof(SafetyCategoryEnum);
var enumNames = Enum.GetNames(type);
var enumValues = (Int16[])Enum.GetValues(type);
var inner = (Expression)Expression.Constant(defaultValue);
var parameter = memberAccess.Parameters[0];
for (int i = 0; i < enumValues.Length; i++)
{
inner = Expression.Condition(
Expression.Equal(memberAccess.Body, Expression.Constant(enumValues[i])),
Expression.Constant(enumNames[i]), inner);
}
MyExpressionVisitor myExpressionVisitor = new MyExpressionVisitor();
var expression = Expression.Lambda<Func<TSource, string>>(inner, parameter);
myExpressionVisitor.Visit(expression);
return expression;
}
}
When performing a sort AutoMapper throws the following exception:
InvalidOperationException: Rewriting child expression from type 'System.Nullable`1[SafetyCategoryEnum]' to type 'System.String' is not allowed, because it would change the meaning of the operation.
If this is intentional, override 'VisitUnary' and change it to allow this rewrite.
Is there any way around this type issue?
Any help would be greatly appreciated!
This works for me with the latest AM, but you need to make the destination a string. I think you're not on the latest version and you've hit an already fixed bug.

Extract strongly-typed data context instance from arbitrary query

Background
We have a class library which has a grid (inherits from WPF DataGrid) with refresh functionality. The grid has a IQueryable Query property, which enables the refresh. Each grid's query is defined not in the class library, but in the referencing end-project:
var dg = new RefreshableDataGrid();
dg.Query = () => new ProjectDbContext().Persons;
Each grid also has a textbox for text filtering. When text is entered in the filter, an expression is generated which checks if any string property or string-convertible property (using SqlFunctions.StringConvert) contains the filter string. The expression is then appended to the original query as an argument to Where, and thus only the records containing matching strings are returned.
//within the class library
//pseudo-code -- this is actually done via reflection, because at compile time the
//actual type of the grid is not known, and there is no generic placeholder
this.ItemsSource = this.Query.Where(filterExpression)
In some cases, the filter logic is defined in end-projects, on the entity type. For example:
public interface IFilterable {
public Expression<Func<String, Boolean>> TextSearchExpression();
}
public class Email {
public int ID {get;set;}
public int PersonID {get;set;}
public string Address {get;set;}
}
public class Person : IFilterable
public int ID {get;set;}
public string LastName {get;set;}
public string FirstName {get;set;}
public Expression<Func<String, Boolean>> TextSearchExpression() {
Dim ctx = new ProjectDbContext();
return phrase => LastName.Contains(phrase) || FirstName.Contains(phrase) ||
ctx.Emails.Where(x => x.PersonID = ID && x.Address.Contains(prase).Any();
}
}
This expression tree uses an instance of the project-specific context, which is a different instance from that of the original query. Queries cannot use components from multiple contexts (at least not in Entity Framework). I can rewrite the expression tree to use a specific instance, but I need to extract the original instance from the query.
It seems obvious that the query holds some reference to the context instance, otherwise the query would not be able to return results.
I do not want to pass the context instance to the class library.
Hence:
Given a query, how can I get the strongly-typed DbContext instance used to create the query?
In other words, what goes in the body of this method:
DbContext GetDbContext<TSource>(IQueryable<TSource> qry) {
// ???
}
It seems obvious that the query holds some reference to the context instance, otherwise the query would not be able to return results.
That's true, but it's implementation specific detail and in EF is encapsulated inside internal members/classes/interfaces.
Also taking into account that DbContext is build on top of the ObjectContext, holding a reference to the DbContext is not strongly necessary. Fortunately that's not the case :)
The following uses reflection and implementation details of the latest EF6.1.3 (tested and working if you don't use some 3rd party extensions like LinqKit and similar that replace the query provider):
public static DbContext GetDbContext<TSource>(this IQueryable<TSource> query)
{
const BindingFlags flags = BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public;
var provider = query.Provider;
var internalContextProperty = provider.GetType().GetProperty("InternalContext", flags);
if (internalContextProperty == null) return null;
var internalContext = internalContextProperty.GetValue(provider, null);
if (internalContext == null) return null;
var ownerProperty = internalContext.GetType().GetProperty("Owner", flags);
if (ownerProperty == null) return null;
var dbContext = (DbContext)ownerProperty.GetValue(internalContext, null);
return dbContext;
}
I would recommend passing an instance of MyDataContext into your query function
public class DACPerson
{
public static IQueryable<Person> GetAllAsQueryable(MyDataContext db)
{
return db.People.AsQueryable();
}
}
This allows you to do the following in the calling function:
public List<Person> UpdateListofPeople(string term)
{
using(DataContext db = new DataContext())
{
var people = DACPerson.GetAllAsQueryable(db);
var result = people.Where(x=>x.Username.Contains(term)).
//call other Data Access Component FUnctions here using same DB....
}
}
i.e. you can bring the filtering to the layer above the data access class.
Some people may not like to do this. You may get the advice that its best to keep all entityframeowrk functionality within the same layer and just return the DTO. I like the above approach though. It depends on who will have to maintain each part of your application in the future.
Hope this helps at least

EF get dbset name in runtime from Type

Purpose:
I need to get the name of the dbset name of the entity
typeof(UserAccount) = "UserAccounts".
But in runtime I need a common type for the loop and therefor do not know example "UserAccount".
Only the "name" from typeof?
I have created an DbContext with some entities.
I have been googling for some time but it do not seem to be working for me because of the Type converting?
Please see my method GetDbSetName in the bottom of this description.
I am pretty new at this EF stuff - so please help med with my issue as described below ;-)
public class MyEntities : DbContext
{
public DbSet<UserAccount> UserAccounts { get; set;}
public DbSet<UserRole> UserRoles { get; set; }
public DbSet<UserAccountRole> UserAccountRoles { get; set; }
}
Defined a list of Type to control the output:
public static List<Type> ModelListSorted()
{
List<Type> modelListSorted = new List<Type>();
modelListSorted.Add(typeof(UserRole));
modelListSorted.Add(typeof(UserAccountRole));
modelListSorted.Add(typeof(UserAccount));
return modelListSorted;
}
The problem is below using Type - If I use "UserAccount" it Works and I get "UserAccounts".
But I do not have the "UserAccount" in runtime as I am in a loop with a serie of types.
I do only have the Type list giving the e
public static loopList()
{
List<Type> modelListSorted = ModelListSorted();
foreach (Type currentType in modelListSorted)
{
string s = DataHelper.GetDbSetName(currentType, db);
}
}
HERE IS THE METHOD GIVING ME THE CHALLANGES ;-)
Meaning not compiling.
saying I am missing a assembly?
I know it is pretty pseudo but can this be done smoothly?
public static string GetDbSetName(Type parmType, MyEntities db)
{
string dbsetname = (db as IObjectContextAdapter).ObjectContext.CreateObjectSet<parmType>().EntitySet.Name;
return dbsetname;
}
The challenge here is that two reflection steps are involved, one to invoke the generic CreateObjectSet method and one to get the EntitySet from the result. Here's a way to do this:
First, the method:
string GetObjectSetName(ObjectContext oc, MethodInfo createObjectSetMethodInfo,
Type objectSetType, Type entityType)
{
var objectSet = createObjectSetMethodInfo.MakeGenericMethod(entityType)
.Invoke(oc, null);
var pi = objectSetType.MakeGenericType(entityType).GetProperty("EntitySet");
var entitySet = pi.GetValue(objectSet) as EntitySet;
return entitySet.Name;
}
As you see, I first get the ObjectSet by invoking the MethodInfo representing the generic method CreateObjectSet<T>(). Then I find the PropertyInfo for the EntitySet property of the generic type ObectSet<T>. Finally, I get this property's value and the name of the obtained EntitySet.
To do this, I first get a MethodInfo for CreateObjectSet<>() (the one without parameters) and the ObjectSet<> type
var createObjectSetMethodInfo =
typeof(ObjectContext).GetMethods()
.Single(i => i.Name == "CreateObjectSet"
&& !i.GetParameters().Any());
var objectSetType = Assembly.GetAssembly(typeof(ObjectContext))
.GetTypes()
.Single(t => t.Name == "ObjectSet`1");
In GetObjectSetName their generic parameters are specified by a concrete entity type, which is done by these "MakeGeneric..." methods.
var oc = (dbContextInstance as IObjectContextAdapter).ObjectContext;
var entityType = typeof(UserRole);
var name = GetObjectSetName(oc, createObjectSetMethodInfo, objectSetType, entityType);
In EF 6 these should be the usings:
using System.Data.Entity.Core.Metadata.Edm
using System.Data.Entity.Core.Objects
using System.Data.Entity.Infrastructure
using System.Linq
using System.Reflection

What is "var" in Entity Framework

I would like to ask what is "var" in this statement.
var context = new MHC_CoopEntities();
var lists = (from c in context.InventLists
select new
{
c.InventID,
c.ItemName,
c.InventCategory.CategoryID,
c.UnitQty,
c.UnitPrice
}).ToList();
ListGridView.DataSource = lists;
ListGridView.DataBind();
I know that "var" can be any value. I am trying to create a helper class for this.
var has nothing to do with Entity Framework. It's a pure C# construct allowing you to define an implicitly typed object. It's explained in the documentation. Basically it allows the compiler to infer the actual type of the variable from the right handside of the assignment. This avoids you repeating the same type declaration twice. It is also necessary for anonymous types which do not have a name. For example:
var foo = new { Id = 123, Name = "anon" };
And this is exactly what happens in your example. In the select clause you are returning an anonymous type. So the only way is to use var.
In the first example:
var context = new MHC_CoopEntities();
it is equivalent to:
MHC_CoopEntities context = new MHC_CoopEntities();
because we know the type.
It will be a new anonymous type. If you need to pass it between functions, you should declare the custom type first (I've called it InventType):
public class InventType {
int InventId { set; get; }
string ItemName { set; get; }
int CategoryId { set; get; }
int UnitQty { set; get; }
decimal UnitPrice { set; get; }
}
var lists = (from c in context.InventLists
select new InventType
{
InventId = c.InventID,
ItemName = c.ItemName,
CategoryId = c.InventCategory.CategoryID,
UnitQty = c.UnitQty,
UnitPrice = c.UnitPrice
}).ToList();
Now var represents List<InventType>.