Custom deserialization - mongodb

I have collection with thousands of documents, in document there's field named Rate, problem is currently its type is string, so when it's not available, the old developer set it to "N/A". For now I want to change the type of this field to numeric in C# (set it to 0 when n/a), but if I do so I can't load the past data.
Can we customize the deserialization so it will convert N/A to 0?

You need to create an IBsonSerializer or SerializerBase<> and attach it to the property you wish to serialize using the BsonSerializerAttribute. Something like the following:
public class BsonStringNumericSerializer : SerializerBase<double>
{
public override double Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
{
var type = context.Reader.GetCurrentBsonType();
if (type == BsonType.String)
{
var s = context.Reader.ReadString();
if (s.Equals("N/A", StringComparison.InvariantCultureIgnoreCase))
{
return 0.0;
}
else
{
return double.Parse(s);
}
}
else if (type == BsonType.Double)
{
return context.Reader.ReadDouble();
}
// Add any other types you need to handle
else
{
return 0.0;
}
}
}
public class YourClass
{
[BsonSerializer(typeof(BsonStringNumericSerializer))]
public double YourDouble { get; set; }
}
If you don't want to use attributes you can create an IBsonSerializationProvider and register it using BsonSerializer.RegisterSerializationProvider.
Full documentation of MongoDB C# Bson serialization can be found here

Related

Can Kaitai Struct be used to describe TLV data without creating new types for each field?

I'm reverse engineering a file format that stores each field as TLV blocks (type, length, value).
The fields do not have to be in order, or even present at all. Their presence is denoted with a sentinel, which is a 16-bit type identifier and a 32-bit end offset. There are hundreds of unique identifiers, but a decent chunk of those are just single primitive values. aside from denoting the type, they can also identify what field the data should be stored in.
It is also worth noting that there will never be a duplicate id on a parent structure. The only time is can occur is if there are multiple of the same object type in an array/list.
I have successfully written a Kaitai definition for one of them:
meta:
id: struct_02ea
endian: le
seq:
- id: unk_00
type: s4
- id: fields
type: field_block
repeat: eos
types:
sentinel:
seq:
- id: id
type: u2
- id: end_offset
type: u4
field_block:
seq:
- id: sentinel
type: sentinel
- id: value
type:
switch-on: sentinel.id
cases:
0xF0: u1
0xF1: u1
0xF2: u1
0xF3: u1
0xF4: u4
0xF5: u4
size: sentinel.end_offset - _root._io.pos
Handling things this way does work, and I could likely map out the entire format like this. However, when it comes time to compiling this definition into another format, things get nasty.
Since I am wrapping each field in a field_block, the generated code stores these values in that type of object. This is incredibly inefficient when half of the generated field_block objects store a single integer. It would also require the consuming code to iterate through a list of each field block in order to get the actual field's value.
Ideally, I would like to define this structure so that the sentinels are only parsed while Kaitai is reading the data, and each value would be mapped to a field on the parent structure.
Is this possible? This technology is really cool, and I'd love to use it in my project, but I feel like the overhead that this is generating is a lot more trouble than it's worth.
Here's an example of the definition when compiled into C#:
using System.Collections.Generic;
namespace Kaitai
{
public partial class Struct02ea : KaitaiStruct
{
public static Struct02ea FromFile(string fileName)
{
return new Struct02ea(new KaitaiStream(fileName));
}
public Struct02ea(KaitaiStream p__io, KaitaiStruct p__parent = null, Struct02ea p__root = null) : base(p__io)
{
m_parent = p__parent;
m_root = p__root ?? this;
_read();
}
private void _read()
{
_unk00 = m_io.ReadS4le();
_fields = new List<FieldBlock>();
{
var i = 0;
while (!m_io.IsEof) {
_fields.Add(new FieldBlock(m_io, this, m_root));
i++;
}
}
}
public partial class Sentinel : KaitaiStruct
{
public static Sentinel FromFile(string fileName)
{
return new Sentinel(new KaitaiStream(fileName));
}
public Sentinel(KaitaiStream p__io, Struct02ea.FieldBlock p__parent = null, Struct02ea p__root = null) : base(p__io)
{
m_parent = p__parent;
m_root = p__root;
_read();
}
private void _read()
{
_id = m_io.ReadU2le();
_endOffset = m_io.ReadU4le();
}
private ushort _id;
private uint _endOffset;
private Struct02ea m_root;
private Struct02ea.FieldBlock m_parent;
public ushort Id { get { return _id; } }
public uint EndOffset { get { return _endOffset; } }
public Struct02ea M_Root { get { return m_root; } }
public Struct02ea.FieldBlock M_Parent { get { return m_parent; } }
}
public partial class FieldBlock : KaitaiStruct
{
public static FieldBlock FromFile(string fileName)
{
return new FieldBlock(new KaitaiStream(fileName));
}
public FieldBlock(KaitaiStream p__io, Struct02ea p__parent = null, Struct02ea p__root = null) : base(p__io)
{
m_parent = p__parent;
m_root = p__root;
_read();
}
private void _read()
{
_sentinel = new Sentinel(m_io, this, m_root);
switch (Sentinel.Id) {
case 243: {
_value = m_io.ReadU1();
break;
}
case 244: {
_value = m_io.ReadU4le();
break;
}
case 245: {
_value = m_io.ReadU4le();
break;
}
case 241: {
_value = m_io.ReadU1();
break;
}
case 240: {
_value = m_io.ReadU1();
break;
}
case 242: {
_value = m_io.ReadU1();
break;
}
default: {
_value = m_io.ReadBytes((Sentinel.EndOffset - M_Root.M_Io.Pos));
break;
}
}
}
private Sentinel _sentinel;
private object _value;
private Struct02ea m_root;
private Struct02ea m_parent;
public Sentinel Sentinel { get { return _sentinel; } }
public object Value { get { return _value; } }
public Struct02ea M_Root { get { return m_root; } }
public Struct02ea M_Parent { get { return m_parent; } }
}
private int _unk00;
private List<FieldBlock> _fields;
private Struct02ea m_root;
private KaitaiStruct m_parent;
public int Unk00 { get { return _unk00; } }
public List<FieldBlock> Fields { get { return _fields; } }
public Struct02ea M_Root { get { return m_root; } }
public KaitaiStruct M_Parent { get { return m_parent; } }
}
}
Affiliate disclaimer: I'm a Kaitai Struct maintainer (see my GitHub profile).
Since I am wrapping each field in a field_block, the generated code stores these values in that type of object. This is incredibly inefficient when half of the generated field_block objects store a single integer. It would also require the consuming code to iterate through a list of each field block in order to get the actual field's value.
I think that rather than trying to describe the entire format with an ultimate Kaitai Struct specification, it's better for you not to let the generated code parse all the fields automatically. Move the parsing control to your application code, where you use the type Struct02ea.FieldBlock that represents the individual field and basically replicate the "repeat until end of stream" loop that the generated code that you posted was doing:
_fields = new List<FieldBlock>();
{
var i = 0;
while (!m_io.IsEof) {
_fields.Add(new FieldBlock(m_io, this, m_root));
i++;
}
}
The advantage of doing so is that you can adjust the loop to fit your needs. To avoid the overhead you describe, you'll probably want to keep the Struct02ea.FieldBlock object in a local variable inside the loop body, pull only the values you care about (save them in your compact, consumer-friendly output structures) and let it leave the scope after the loop iteration ends. This will allow each original FieldBlock object to get garbage-collected once you process it, so the overhead they have will be limited to a single instance and not multiplied by the number of fields in the file.
The most straightforward and seamless way to prevent the Kaitai Struct-generated code parse fields (but otherwise keep everything the same) is to add if: false in the KSY specification, as #webbnh suggested in a GitHub issue:
seq:
- id: unk_00
type: s4
- id: fields
type: field_block
repeat: eos
if: false # add this
The if: false works better than omitting it from seq entirely, because the kaitai-struct-compiler has occasional troubles with unused types (when compiling the KSY spec with unused types, you may get an error "Unable to derive _parent type in ..." due to a compiler bug). But with this if: false trick, you can't run into them because the field_block type is no longer unused.

Error : Cannot find Facebook SDK Version

the SDK 7.x is not working on Unity 5.1.0f3 , i always got the error version not found .
does someone have see this error ?
Actually, it's just a warning. But you can fix it.
There are several places where facebook plugin calls
FBBuildVersionAttribute.GetVersionAttributeOfType(typeof(AbstractFacebook));
So first, you need to modify FBBuildVersionAttribute to this:
// we are going to apply this attribute to Class
// instead of Assembly
// also make it inheritable for all implementations
[AttributeUsage(AttributeTargets.Class, Inherited = true)]
public class FBBuildVersionAttribute : Attribute
{
private DateTime buildDate;
private string buildHash;
private string buildVersion;
private string sdkVersion;
public DateTime Date { get { return buildDate; } }
public string Hash { get { return buildHash; } }
public string SdkVersion { get { return sdkVersion; } }
public string BuildVersion { get { return buildVersion; } }
public FBBuildVersionAttribute(string sdkVersion, string buildVersion)
{
this.buildVersion = buildVersion;
var parts = buildVersion.Split('.');
buildDate = DateTime.ParseExact(parts[0], "yyMMdd", System.Globalization.CultureInfo.InvariantCulture);
buildHash = parts[1];
this.sdkVersion = sdkVersion;
}
public override string ToString()
{
return buildVersion;
}
public static FBBuildVersionAttribute GetVersionAttributeOfType(Type type)
{
foreach (FBBuildVersionAttribute attribute in getAttributes(type))
{
return attribute;
}
return null;
}
private static FBBuildVersionAttribute[] getAttributes(Type type)
{
if (type == null)
throw new ArgumentNullException("type");
// we want to get attributes from type instead of assmebly
return (FBBuildVersionAttribute[])(type.GetCustomAttributes(typeof(FBBuildVersionAttribute), false));
}
}
No you just need to add this attribute to AbstractFacebook:
[FBBuildVersionAttribute("7.0.1", "150604.98558e55096475c")]
public abstract class AbstractFacebook : MonoBehaviour
{
// ...
}
Note that 98558e55096475c part is trash string. It's not actual build hash, cause I don't have one.
Get the latest version of the FB Unity SDK. Change log says it's fixed now.
https://developers.facebook.com/docs/unity/change-log

Gtk.CellRendererText with format

I would like to write a custom CellRenderer that formats it's content. Instead of displaying -905.65000 it should display -905.65 €. I already could achieve this with a Gtk.TreeCellDataFunc but it would be more elegant with a custom CellRenderer.
Following code gives a segmentation fault:
class CellRendererTextFormat : Gtk.CellRendererText {
public new string text {
get { return text; }
set {
char[] a = new char[16];
text = double.parse(value).format(a,"%-.2f €");
}
}
public CellRendererTextFormat () {
GLib.Object ();
}
}
How should i do it instead?
There are two problems:
1) text is assigned the memory for a, which is freed. You need to assign it to memory that is durable beyond the scope of the set.
2) value is now a string (because you can only assign properties values of the same type). Do you actually have strings, or did you put in that parse just to make it work?
Instead:
class CellRendererEuroValue : Gtk.CellRendererText {
char[] euro_str = new char[16];
private double _euro_value;
public double euro_value {
get { return _euro_value; }
set {
_euro_value = value;
value.format(euro_str,"%-.2f €");
text = (string) euro_str;
}
}
public CellRendererTextFormat () {
GLib.Object ();
}
}

Setting Field Within Class using Reflection

This should be easy but it's managed to confound me a few times. I'm trying to set a Value within a class using Reflection.
public class EngineeringValueClass<T> {
public T Value { get { } set { } }
}
Then in a calling class I have:
public class MyClass {
EngineeringValueClass<double> Value1;
EngineeringValueClass<double> Value2;
// Along with many others.
public void SetValueByName(object FieldName,object FieldValue) {
// Get the "Value" field of a generic EngineeringValueClass<double>
PropertyInfo MyValuePropRef =
typeof(EngineeringValueClass<double>).GetProperty("Value");
// Get the field within this class I want to set.
FieldInfo MyNameFieldRef = typeof(MyClass).GetField(FieldName.ToString());
MyValuePropRef.SetValue(MyNameFieldRef.GetValue,
FieldValue,
null);
}
}
My goal is to have
SetValueByName("Value1",2.3);
set Value1's "Value" using the set accessor. I presume my problem is that MyNameFieldRef.GetValue doesn't return an object reference but rather a "Value", but I'm not sure how to work around that. I don't want to pass "this" because that's not right either.
Okay, I finally figured this out:
public void SetValueByName(object FieldName, object FieldValue) {
Type t = typeof(MyClass);
FieldInfo PrimaryField = t.GetField(FieldName.ToString());
object ValueField = PrimaryField.GetValue(this);
// To get the type of Value
MethodInfo TypeValueField = ValueField.GetType().GetMethod("GetValueType");
Type ValueType = (Type) TypeValueField.Invoke(ValueField, null);
// I added a "GetValueType () { return typeof(T); } to EngineeringValueClass
if (ValueType == typeof(Int16))
{
((EngineeringValueClass<Int16>)ValueField).Value = Int16.Parse(FieldValue.ToString());
}...
}

How can I dynamically make entity properties read-only?

I'm working with EF 4.5 and DbContext. At business rules layer level, I should implement checks to avoid change entity value properties in some entity scenarios. Sample: StartProjecteDate should be readonly if ProjectIsStarted but not in other status.
I follow DRY principle, for this reason, I should be able to inspect readonly properties list from context and also from UI.
My question:
Is there a DataAnnotation validator that can dynamically set properties as readonly?
(and if not, is there a different / better solution to this problem?)
Notice than I'm working with Web Forms (and Telerik) architecture, a clean and elegant pattern will be welcome.
I'm trying to set and get at run time EditableAttribute as Jesse Webb explains, but I'm not able to get dataannotation attributes from property, my code:
<EditableAttribute(False)>
<MaxLength(400, ErrorMessage:="Màxim 400 caracters")>
Public Property NomInvertebrat As String
Edited Nov 8 2013 after digging docs, it seems that dataanottions if for class but for instance object itself. Perhaps an iReadonlyableProperties interface may be a way.
I have a class containing extension methods that lets me read data annotations like this:
int maxRefLen = ReflectionAPI.GetProperty<Organisation, String>(x => x.Name)
.GetAttribute<StringLengthAttribute>()
.GetValueOrDefault(x => x.MaximumLength, 256);
So if you use it you should be able to do get the value of the EditableAttribute like this:
bool isEditable = ReflectionAPI.GetProperty<Foo, String>(x => x.NomInvertebrat)
.GetAttribute<EditableAttribute>()
.GetValueOrDefault(x => x.AllowEdit, true);
As for setting the data annotations at run-time, I haven't done it myself but I have read that there is a solution here: Setting data-annotations at runtime
Getting a list of all data annotations of a particular type I think would entail reading the entity framework metadata. Again I haven't tried this.
If you add that together I personally think it feels clunky rather than elegant, but you have asked for a solution using DataAnnotations and something more elegant would probably mean getting into your architecture.
I would be inclined to do this:
public bool StartDateIsReadOnly
{
//use this property client-side to disable the input
get{ return Project.IsStarted;}
}
//Implement IValidatable object to do server side validation
public IEnumerable<ValidationResult> Validate(ValidationContext validationContext
{
bool startdateIsChanged = // I'll leave you to work out this bit
var results = new List<ValidationResult>();
if(StartDateIsReadOnly && startdateIsChanged)
results.Add(new ValidationResult("Start Date cannot be changed after project is started");
}
Here is the ReflectionAPI class:
Please note that the class includes part of a hack that #JonSkeet posted and described as "evil". I personally think this bit ain't so bad, but you should read the following references:
Override a generic method for value types and reference types.
Evil code - overload resolution workaround
public static class ReflectionAPI
{
public static int GetValueOrDefault<TInput>(this TInput a, Func<TInput, int> func, int defaultValue)
where TInput : Attribute
//Have to restrict to struct or you get the error:
//The type 'R' must be a non-nullable value type in order to use it as parameter 'T' in the generic type or method 'System.Nullable<T>'
{
if (a == null)
return defaultValue;
return func(a);
}
public static Nullable<TResult> GetValueOrDefault<TInput, TResult>(this TInput a, Func<TInput, TResult> func, Nullable<TResult> defaultValue)
where TInput : Attribute
where TResult : struct
//Have to restrict to struct or you get the error:
//The type 'R' must be a non-nullable value type in order to use it as parameter 'T' in the generic type or method 'System.Nullable<T>'
{
if (a == null)
return defaultValue;
return func(a);
}
//In order to constrain to a class without interfering with the overload that has a generic struct constraint
//we need to add a parameter to the signature that is a reference type restricted to a class
public class ClassConstraintHack<T> where T : class { }
//The hack means we have an unused parameter in the signature
//http://msmvps.com/blogs/jon_skeet/archive/2010/11/02/evil-code-overload-resolution-workaround.aspx
public static TResult GetValueOrDefault<TInput, TResult>(this TInput a, Func<TInput, TResult> func, TResult defaultValue, ClassConstraintHack<TResult> ignored = default(ClassConstraintHack<TResult>))
where TInput : Attribute
where TResult : class
{
if (a == null)
return defaultValue;
return func(a);
}
//I don't go so far as to use the inheritance trick decribed in the evil code overload resolution blog,
//just create some overloads that take nullable types - and I will just keep adding overloads for other nullable type
public static bool? GetValueOrDefault<TInput>(this TInput a, Func<TInput, bool?> func, bool? defaultValue)
where TInput : Attribute
{
if (a == null)
return defaultValue;
return func(a);
}
public static int? GetValueOrDefault<TInput>(this TInput a, Func<TInput, int?> func, int? defaultValue)
where TInput : Attribute
{
if (a == null)
return defaultValue;
return func(a);
}
public static T GetAttribute<T>(this PropertyInfo p) where T : Attribute
{
if (p == null)
return null;
return p.GetCustomAttributes(false).OfType<T>().LastOrDefault();
}
public static PropertyInfo GetProperty<T, R>(Expression<Func<T, R>> expression)
{
if (expression == null)
return null;
MemberExpression memberExpression = expression.Body as MemberExpression;
if (memberExpression == null)
return null;
return memberExpression.Member as PropertyInfo;
}
}
.NET allows you to dynamically change structure of Class by implementing System.ComponentModel.ICustomTypeDescriptor. Most serializers support this interface.
// Sample Serialization
foreach(PropertyDescriptor p in TypeDescriptor.GetProperties(obj)){
string name = p.PropertyName;
object value = p.GetValue(obj);
}
Internally TypeDescriptor uses Reflection, but the implementation allows us to override reflection attributes easily.
Here are three steps of implementation,
// Implement System.ComponentModel.ICustomTypeDescriptor Interface on
// your Entity
public class MyEntity: System.ComponentModel.ICustomTypeDescriptor
{
....
// most methods needs only call to default implementation as shown below
System.ComponentModel.AttributeCollection
System.ComponentModel.ICustomTypeDescriptor.GetAttributes()
{
return TypeDescriptor.GetAttributes(this, true);
}
string System.ComponentModel.ICustomTypeDescriptor.GetClassName()
{
return TypeDescriptor.GetClassName(this, true);
}
string System.ComponentModel.ICustomTypeDescriptor.GetComponentName()
{
return TypeDescriptor.GetComponentName(this, true);
}
System.ComponentModel.TypeConverter System.ComponentModel.ICustomTypeDescriptor.GetConverter()
{
return TypeDescriptor.GetConverter(this, true);
}
System.ComponentModel.EventDescriptor System.ComponentModel.ICustomTypeDescriptor.GetDefaultEvent()
{
return TypeDescriptor.GetDefaultEvent(this, true);
}
System.ComponentModel.PropertyDescriptor System.ComponentModel.ICustomTypeDescriptor.GetDefaultProperty()
{
return TypeDescriptor.GetDefaultProperty(this, true);
}
object System.ComponentModel.ICustomTypeDescriptor.GetEditor(Type editorBaseType)
{
return TypeDescriptor.GetEditor(this, editorBaseType, true);
}
System.ComponentModel.EventDescriptorCollection System.ComponentModel.ICustomTypeDescriptor.GetEvents(Attribute[] attributes)
{
return TypeDescriptor.GetEvents(this, attributes, true);
}
System.ComponentModel.EventDescriptorCollection System.ComponentModel.ICustomTypeDescriptor.GetEvents()
{
return TypeDescriptor.GetEvents(this, true);
}
System.ComponentModel.PropertyDescriptorCollection System.ComponentModel.ICustomTypeDescriptor.GetProperties(Attribute[] attributes)
{
return TypeDescriptor.GetProperties(this, attributes, true);
}
object System.ComponentModel.ICustomTypeDescriptor.GetPropertyOwner(System.ComponentModel.PropertyDescriptor pd)
{
return this;
}
// The Only method that needs different implementation is below
System.ComponentModel.PropertyDescriptorCollection
System.ComponentModel.ICustomTypeDescriptor.GetProperties()
{
// ... you are supposed to create new instance of
// PropertyDescriptorCollection with PropertyDescriptor
PropertyDescriptorCollection pdc = new PropertyDescriptorCollection();
foreach(PropertyDescriptor p in TypeDescriptor.GetProperties(this,true)){
// if readonly..
AtomPropertyDescriptor ap = new AtomPropertyDescriptor(p, p.Name);
// or
AtomPropertyDescriptor ap = new AtomPropertyDescriptor(p, p.Name,
true,
new XmlIgnoreAttribute(),
new ScriptIgnoreAttribute(),
new ReadOnlyAttribute());
pdc.Add(ap);
}
return pdc;
}
}
// And here is the AtomPropertyDescriptorClass
public class AtomPropertyDescriptor : PropertyDescriptor
{
PropertyDescriptor desc;
bool? readOnly = null;
public AtomPropertyDescriptor(PropertyDescriptor pd, string name,
bool? readOnly, params Attribute[] attrs) :
base(name, attrs)
{
desc = pd;
this.readOnly = readOnly;
}
public override bool CanResetValue(object component)
{
return desc.CanResetValue(component);
}
public override Type ComponentType
{
get
{
return desc.ComponentType;
}
}
public override object GetValue(object component)
{
return desc.GetValue(component);
}
public override bool IsReadOnly
{
get
{
if (readOnly.HasValue)
return readOnly.Value;
return desc.IsReadOnly;
}
}
public override Type PropertyType
{
get { return desc.PropertyType; }
}
public override void ResetValue(object component)
{
desc.ResetValue(component);
}
public override void SetValue(object component, object value)
{
desc.SetValue(component, value);
}
public override bool ShouldSerializeValue(object component)
{
return desc.ShouldSerializeValue(component);
}
}
I think what you are looking for is a custom Annotation Attribute like this:
<DisableEditAttribute(this.IsProjectStarted)>
Public Property NomInvertebrat As String
public override bool IsValid(bool value)
{
bool result = true;
// Add validation logic here.
if(value)
{
//Compare Current Value Against DB Value.
}
return result;
}
See MSDN: http://msdn.microsoft.com/en-us/library/cc668224(v=vs.98).aspx