we are having DateTime field and we are reading from MongoDB collection and deserialising into that class. Lets say if it has null in DateTime DB field and MongoDriver is trying to set null to that datetime field which is non nullable type . It s throwing error.
.FindAll().ToList() => error in here.
Any help to overcome this problem ?
Note that : We can use nullable Datetime (DateTime?) . but we need Non nullable type only in domain model. So i just want to use non nullable DateTime while serializing
There are two possibilities for null in this case. You could have stored an actual null in your database:
{
_id:ObjectId(),
MyDateTime:null
}
Or you are not storing the field at all:
{
_id:ObjectId()
}
In the first case, you could handle this by creating your own serializer:
public class DateTimeSerializer : BsonBaseSerializer
{
public override object Deserialize(BsonReader bsonReader, Type nominalType, Type actualType, IBsonSerializationOptions options)
{
var bsonType = bsonReader.CurrentBsonType;
switch (bsonType)
{
case BsonType.Null:
bsonReader.ReadNull();
return new DateTime();
case BsonType.DateTime:
return bsonReader.ReadDateTime();
default:
var message = string.Format("DateTimeSerializer needs a DateTime not {0}.", bsonType);
throw new BsonSerializationException(message);
}
}
public override void Serialize(BsonWriter bsonWriter, Type nominalType, object value, IBsonSerializationOptions options)
{
if (value == null)
{
TimeSpan nowMs = DateTime.Now-new DateTime(1970,1,1);
bsonWriter.WriteDateTime((long)nowMs.TotalMilliseconds);
}
else
{
bsonWriter.WriteString((string)value);
}
}
}
(In this case, giving the current date whenever a null is serialized or de-serialized)
You would then need to register this as the serializer for the DateTime type:
BsonClassMap.RegisterClassMap<MyClass>(cm =>
{
cm.AutoMap();
cm.GetMemberMap(mc => mc.MyDateTime).SetSerializer(new DateTimeSerializer());
});
It has to be said that it would be easier to sanitize the data at the source, so that it didn't have nulls in the first place.
In the second case, this has been handled by the C# driver for MongoDB since 1.5, which version are you using? You could set the default value by registering your own class map, like the following, but as mentioned above it should no longer be required.
BsonClassMap.RegisterClassMap<MyClass>(cm =>
{
cm.AutoMap();
cm.GetMemberMap(mc => mc.MyDateTime).SetDefaultValue(new DateTime());
});
Related
Update 15/08/2020: Looks like Enum support was added on Jun 16. R2DBC commit.
Does H2DBC support PostgreSQL enums? I checked they git page but it doesn't mention anything about it. If it does, how enums could be used (INSERT, SELECT)?
Lets say PostgreSQL enum
CREATE TYPE mood AS ENUM ('UNKNOWN', 'HAPPY', 'SAD', ...);
Java class
#Data
public class Person {
private String name;
private Mood mood;
// ...
enum Mood{ UNKNOWN, HAPPY, SAD, ...}
}
I tried:
// insert
var person = ...;
client.insert()
.table("people")
.using(person)
.then()
.subscribe(System.out::println);
// select
var query = "SELECT * FROM people";
client.execute(query)
.as(Person.class)
.fetch().all()
.subscribe(System.out::println);
But I'm getting error messages:
# on insert
WARN [reactor-tcp-epoll-1] (Loggers.java:294) - Error: SEVERITY_LOCALIZED=ERROR, SEVERITY_NON_LOCALIZED=ERROR, CODE=42804, MESSAGE=column "mood" is of type mood but expression is of type character varying, HINT=You will need to rewrite or cast the expression., POSITION=61, FILE=parse_target.c, LINE=591, ROUTINE=transformAssignedExpr
# on select
ERROR [reactor-tcp-epoll-1] (Loggers.java:319) - [id: 0x8581acdb, L:/127.0.0.1:39726 ! R:127.0.0.1/127.0.0.1:5432] Error was received while reading the incoming data. The connection will be closed.
reactor.core.Exceptions$ErrorCallbackNotImplemented: org.springframework.data.mapping.MappingException: Could not read property private ...
I found similar post but without luck to solve my problem.. maybe I was applying it wrong..
Any help or tips are welcome.
Tested with org.springframework.data:spring-data-r2dbc:1.0.0.RELEASE and io.r2dbc:r2dbc-postgresql:0.8.1.RELEASE.
Kotlin version.
Define a enum class
enum class Mood {
UNKNOWN,
HAPPY,
SAD
}
Create a custom codec
class MoodCodec(private val allocator: ByteBufAllocator) : Codec<Mood> {
override fun canEncodeNull(type: Class<*>): Boolean = false
override fun canEncode(value: Any): Boolean = value is Mood
override fun encode(value: Any): Parameter {
return Parameter(Format.FORMAT_TEXT, oid) {
ByteBufUtils.encode(allocator, (value as Mood).name)
}
}
override fun canDecode(dataType: Int, format: Format, type: Class<*>): Boolean = dataType == oid
override fun decode(buffer: ByteBuf?, dataType: Int, format: Format, type: Class<out Mood>): Mood? {
buffer ?: return null
return Mood.valueOf(ByteBufUtils.decode(buffer))
}
override fun type(): Class<*> = Mood::class.java
override fun encodeNull(): Parameter =
Parameter(Format.FORMAT_TEXT, oid, Parameter.NULL_VALUE)
companion object {
// Get form `select oid from pg_type where typname = 'mood'`
private const val oid = YOUR_ENUM_OID
}
}
Registe the codec
You may need change runtimeOnly("io.r2dbc:r2dbc-postgresql") to implementation("io.r2dbc:r2dbc-postgresql")
#Configuration
#EnableR2dbcRepositories
class AppConfig : AbstractR2dbcConfiguration() {
override fun connectionFactory(): ConnectionFactory = PostgresqlConnectionConfiguration.builder()
.port(5432) // Add your config here.
.codecRegistrar { _, allocator, registry ->
registry.addFirst(MoodCodec(allocator))
Mono.empty()
}.build()
.let { PostgresqlConnectionFactory(it) }
}
I used the below for Spring boot 2.6.4 + r2dbc-postgresql 0.8.11 by adding a customizer rather than creating the connection factory myself.
Thanks #Hantsy for pointing EnumCodec out. I added it to a customizer therefore it can play nicely with existing autoconfigure procedure. Also, the spring-data keeps converting my enum to string until I added the converter.
Hopefully these can provide a little help to others.
Register EnumCodec to builder customizer as extensions
It is possible to register multiple enum, just repeat the withEnum() call.
/**
* Use the customizer to add EnumCodec to R2DBC
*/
#Bean
public ConnectionFactoryOptionsBuilderCustomizer connectionFactoryOptionsBuilderCustomizer() {
return builder -> {
builder.option(Option.valueOf("extensions"),
List.of(EnumCodec.builder()
.withEnum("enum_foo", FooEnum.class)
.withRegistrationPriority(RegistrationPriority.FIRST)
.build()));
logger.info("Adding enum to R2DBC postgresql extensions: {}", builder);
};
}
Implement spring data converter by extending EnumWriteSupport
public class FooWritingConverter extends EnumWriteSupport<Foo> {
}
Register converters so that spring data won't always convert enum to string.
This step is a slightly enhanced version of R2dbcDataAutoConfiguration in spring-boot-autoconfigure project.
/**
* Register converter to make sure Spring data treat enum correctly
*/
#Bean
public R2dbcCustomConversions r2dbcCustomConversions(DatabaseClient databaseClient) {
logger.info("Apply R2DBC custom conversions");
R2dbcDialect dialect = DialectResolver.getDialect(databaseClient.getConnectionFactory());
List<Object> converters = new ArrayList<>(dialect.getConverters());
converters.addAll(R2dbcCustomConversions.STORE_CONVERTERS);
return new R2dbcCustomConversions(
CustomConversions.StoreConversions.of(dialect.getSimpleTypeHolder(), converters),
List.of(
new FooWritingConverter()
));
}
Step 1 and 3 can be added to your application class or any other valid configuration.
Check my article about Postgres specific features supported in R2dbc.
There are two options.
use custom Postgres enum type and Java enum type, and register EnumCodec in the connection factory builder.
use a textual type as data type(such as varchar), and Java Enum type, Spring data r2dbc will convert them directly.
I have an EF model with a notification emails property. The notification emails are saved in the database as string separated by ';'. I added a conversion to retrieve the data as a ICollection in the model. This is working well except one thing: when the string is null the collection is also null, and I want to convert it to an empty collection instead. is it possible?
//This is my code
entity.Property(e => e.NotificationEmails)
.HasConversion(
v => string.Join(",", v.Select(s => s.Trim())),
v => v.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries));
I tried to add String.IsNullOrEmpty(v) but EF ignores it.
Currently, it isn't possible :
https://learn.microsoft.com/en-us/ef/core/modeling/value-conversions#configuring-a-value-converter
A null value will never be passed to a value converter. This makes the implementation of conversions easier and allows them to be shared amongst nullable and non-nullable properties.
It isn't elegant, but you can use a backing field :
public class Notification
{
private List<string> _emails = new List<string>();
public List<string> Emails
{
get => _emails;
set => _emails = value ?? new List<string>();
}
}
public class NotificationContext : DbContext
{
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Entity<Notification>().Property(d => d.Emails).HasConversion(
v => string.Join(",", v.Select(s => s.Trim())),
v => v.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries).ToList()
);
modelBuilder.Entity<Notification>()
.Property(b => b.Emails)
.HasField("_emails")
.UsePropertyAccessMode(PropertyAccessMode.Property);
}
}
Note : in where, a empty list will not be translated by null, but by a empty string.
Edit : This feature is available from EF Core 6, but bugged.
See this comment :
For anyone watching this issue: there are significant problems when executing queries that either convert nulls in the database to non-nulls in code or vice-versa. Therefore, we have marked this feature as internal for EF Core 6.0. You can still use it, but you will get a compiler warning. The warning can be disabled with a #pragma.
I have been working to setup Ormlite as the primary data access layer between a PostgreSQL database and Java application. Everything has been fairly straightforward, until I started messing with PostgreSQL's array types. In my case, I have two tables that make use of text[] array type. Following the documentation, I created a custom data persister as below:
public class StringArrayPersister extends StringType {
private static final StringArrayPersister singleTon = new StringArrayPersister();
private StringArrayPersister() {
super(SqlType.STRING, new Class<?>[]{String[].class});
}
public static StringArrayPersister getSingleton() {
return singleTon;
}
#Override
public Object javaToSqlArg(FieldType fieldType, Object javaObject) {
String[] array = (String[]) javaObject;
if (array == null) {
return null;
} else {
String join = "";
for (String str : array) {
join += str +",";
}
return "'{" + join.substring(0,join.length() - 1) + "}'";
}
}
#Override
public Object sqlArgToJava(FieldType fieldType, Object sqlArg, int columnPos) {
String string = (String) sqlArg;
if (string == null) {
return null;
} else {
return string.replaceAll("[{}]","").split(",");
}
}
}
And then in my business object implementation, I set up the persister class on the column likeso:
#DatabaseField(columnName = TAGS_FIELD, persisterClass = StringArrayPersister.class)
private String[] tags;
When ever I try inserting a new record with the Dao.create statement, I get an error message saying tags is of type text[], but got character varying... However, when querying existing records from the database, the business object (and text array) load just fine.
Any ideas?
UPDATE:
PostGresSQL 9.2. The exact error message:
Caused by: org.postgresql.util.PSQLException: ERROR: column "tags" is
of type text[] but expression is of type character varying Hint: You
will need to rewrite or cast the expression.
I've not used ormlite before (I generally use MyBatis), however, I believe the proximal issue is this code:
private StringArrayPersister() {
super(SqlType.STRING, new Class<?>[]{String[].class});
}
SqlType.String is mapped to varchar in SQL in the ormlite code, and so therefore I believe is the proximal cause of the error you're getting. See ormlite SQL Data Types info for more detail on that.
Try changing it to this:
private StringArrayPersister() {
super(SqlType.OTHER, new Class<?>[]{String[].class});
}
There may be other tweaks necessary as well to get it fully up and running, but that should get you passed this particular error with the varchar type mismatch.
I have a domain class:
class Book {
String id
Date dateCreated
Date lastUpdated
String title
boolean acceptPolicy // defines if the user accepts the terms and privacy poicy
String uploadImageId
static transients = [ "acceptPolicy" ]
static constraints = {
acceptPolicy( validator: { val, obj -> val == true} )
}
}
I choose "acceptPolicy" as a field which should not be stored in the database. When I create a book object everything works find. The problem occurs when I update the book instance. Every time I try to save I get a validation error for the "acceptPolicy". This should not be the case because "acceptPolicy" is not stored in the database. I also tried to set the
acceptPolicy = true
in a hidden input field but still this problem occurs. How can I fix it?
Change the validator and make it behave so that it only validates the acceptPolicy field when the entity is not yet persistent:
acceptPolicy( validator: { val, obj -> obj.id || val } )
In my domain, there's no important distinction between NULL and an empty string. How do I get EF to ignore the difference between the two and always persist an empty string as NULL?
Empty string is not default value for string property so it means your code is setting empty strings somewhere. In such case it is your responsibility to handle it.
If you are using code first with POCOs you can use custom setter:
private string _myProperty;
public string MyProperty
{
get { return _myProperty; }
set
{
if (value == String.Empty)
{
_myProperty = null;
}
else
{
_myProperty = value;
}
}
}
Here is a function I placed in my DbContext subclass that replaces empty or whitespace strings with null.
I still didn't optimize it so any performance hints will be very appreciated.
private const string StringType = "String";
private const EntityState SavingState = EntityState.Added | EntityState.Modified;
public override int SaveChanges()
{
var objectContext = ((IObjectContextAdapter)this).ObjectContext;
var savingEntries =
objectContext.ObjectStateManager.GetObjectStateEntries(SavingState);
foreach (var entry in savingEntries)
{
var curValues = entry.CurrentValues;
var fieldMetadata = curValues.DataRecordInfo.FieldMetadata;
var stringFields = fieldMetadata.Where(f =>
f.FieldType.TypeUsage.EdmType.Name == StringType);
foreach (var stringField in stringFields)
{
var ordinal = stringField.Ordinal;
var curValue = curValues[ordinal] as string;
if (curValue != null && curValue.All(char.IsWhiteSpace))
curValues.SetValue(ordinal, null);
}
}
return base.SaveChanges();
}
Optimization considerations:
Identify a string type property by different way other than string comparison I tried to look-up some enumeration of the built-in types but didn't find
Cache string fields for types (maybe is unnecessary, will have to decompile and see what the original impl does
Order result by entity type, backup iterated entity type, if next iterated entity is same type, use previous metadata, again, if the metadata is anyway there, performance is cheaper the way it is
Limit string length for whitespace check - i.e. if a string length > x, skip checking whether its a whitespace string or not
I'm using Silverlight and the TextBoxes in the UI set all the string properties to empty strings.
I tried setting:
<TextBox
Text="{Binding MyStringProperty,
Mode=TwoWay,
ValidatesOnDataErrors=True,
TargetNullValue=''}"/>
But it didn't help much.
That's not Entity Framework's job.
You should do it in your repository, or in the database with triggers.
Or do it at the start (e.g when the data comes in, UI, external source, etc)