I have a .NET DateTime value that I write to a SQL Server database "datetime" field (and trust me, I WISH we were just using "datetime2(7)" that matches .NET's DateTime precision exactly, but we're not).
Anyway, I write the entity to the database and that particular field ends up being '2016-03-03 08:55:19.560'.
It's a last processing time, and I'm looking for other records that were processed before that time. When I run an entity framework where clause, it ends up running a statement ending with "#p__linq__0='2016-03-03 08:55:19.5602354'" as the value it's comparing against, which ends up being slightly greater, even though these two values originate from the exact same DateTime instance.
I tried changing the time it's comparing against to an SqlDateTime, but then the lambda doesn't compile because it can't compare a DateTime? to a SqlDateTime. SqlDateTime has comparison methods, but I don't know whether entity framework recognizes the functions.
I can't even cast between the two in entity framework, which just gives the error "Unable to cast the type 'System.DateTime' to type 'System.Data.SqlTypes.SqlDateTime'. LINQ to Entities only supports casting EDM primitive or enumeration types."
I face the same problem. In my case I finally value the DateTime fields by using:
public static DateTime RoundedToMs(this DateTime dt) {
return new DateTime(dt.Ticks - (dt.Ticks % TimeSpan.TicksPerMillisecond), dt.Kind);
}
public static DateTime RoundedToMsForSql(this DateTime dt) {
DateTime n = dt.RoundedToMs();
return new DateTime(n.Year, n.Month, n.Day, n.Hour, n.Minute, n.Second, (n.Millisecond / 10) * 10);
}
And in the business code:
someEntity.SomeDate = dateValue.RoundedToMsForSql();
The point, in my case, is sql datetime has a 3ms precision, so I decided to remove millisecond unit.
Same extension may be used in the queries, well in fact to populate variables used in the queries.
var d = DateTime.Now.RoundedToMsForSql();
var q = from e in ctx.Entities where e.SomeDate <= d;
Related
reservationLogs = await this.dbContext.ReservationLogs
.Where(r => r.ProcessedAt == null)
.Where(r => r.PropertyId == validPropertyId)
.OrderBy(r => r.CreatedAt).ThenBy(r => r.Operation)
.Take(200)
.ToListAsync();
some times with the same query i get the error
' Can't cast database type timestamp without time zone to Instant'
note: CreatedAt nodaTime instane
i am trying to find the exact reason
The issue is that even though the date and time is clear, it is unclear whether or which timezone was in use. If I tell you that tomorrow at 5 P.M. I will go for a walk, then it will be unclear from your perspective what the exact time it will be, unless you know what timezone was I assuming while saying so.
You have the exact same type of confusion in your code and first, you need to install this plugin: https://www.npgsql.org/doc/types/nodatime.html
According to the docs, you need to add a dependency like this:
using Npgsql;
// Place this at the beginning of your program to use NodaTime everywhere (recommended)
NpgsqlConnection.GlobalTypeMapper.UseNodaTime();
// Or to temporarily use NodaTime on a single connection only:
conn.TypeMapper.UseNodaTime();
The docs go further in specifying how you can read and write values:
// Write NodaTime Instant to PostgreSQL "timestamp with time zone" (UTC)
using (var cmd = new NpgsqlCommand(#"INSERT INTO mytable (my_timestamptz) VALUES (#p)", conn))
{
cmd.Parameters.Add(new NpgsqlParameter("p", Instant.FromUtc(2011, 1, 1, 10, 30)));
cmd.ExecuteNonQuery();
}
// Read timestamp back from the database as an Instant
using (var cmd = new NpgsqlCommand(#"SELECT my_timestamptz FROM mytable", conn))
using (var reader = cmd.ExecuteReader())
{
reader.Read();
var instant = reader.GetFieldValue<Instant>(0);
}
Since you are not directly writing the query, but use the Entity Framework to do so you have another level of expression. But this is well-documented as well. You can safely and soundly declare types like this:
public LocalDate Date {get; set;}
Read this full article: https://www.davepaquette.com/archive/2019/03/26/using-noda-time-with-ef-core.aspx
You will need to find out exactly where the error occurs. It seems to me that the OrderBy is the culprit as well as the selection. You can change the type of the data member of your model.
You can do cast using Npgsql.EntityFrameworkCore.PostgreSQL.NodaTime package
protected override void OnConfiguring(DbContextOptionsBuilder builder)
{
builder.UseNpgsql("connection-string",
o => o.UseNodaTime());
}
or:
builder.Services.AddDbContext<ApplicationDbContext>(
options => options.UseNpgsql(
builder.Configuration.GetConnectionString("DefaultConection"),
o => o.UseNodaTime()));
source
I have simplified the following example from my code and hoping there's no obvious compilation errors because of it. Lets say I have the following entities (not what i actually have, please assume I have no EF or schema issues, this is just for example):
public class Company
{
public string GroupProperty {get;set;}
public virtual ICollection<PricingForm> PricingForms {get;set;}
}
public class PricingForm
{
public decimal Cost {get;set;}
}
And I want to query like so:
IQueryable DynamicGrouping<T>(IQueryable<T> query)
{
Expression<Func<Company, decimal?>> exp = c => c.PricingForms.Sum(fr => fr.Cost);
string selector = "new (it.Key as Key, #0(it) as Value)";
IQueryable grouping = query.GroupBy("it.GroupProperty", "it").Select(selector, exp);
return grouping;
}
I get the following error when calling the groupby/select line:
System.Linq.Dynamic.ParseException: 'Argument list incompatible with lambda expression'
What type is "it" when grouped? I have tried using other expressions that assume it is an IGrouping<string, Company>, or a IQueryable<Company>, same error. I've tried just selecting "Cost" and moving the Sum() aggregate into the selector string (i.e. Sum(#0(it)) as Value) and always seem to get the same error.
I eventually tried something along the lines of:
Expression<Func<IEnumerable<Company>, decimal?>> exp = l => l.SelectMany(c => c.PricingForms).Sum(fr => fr.Cost);
However this one, I get farther but when attempting to iterate through the results I got a different error.
The LINQ expression node type 'Invoke' is not supported in LINQ to Entities.
So, with this dynamic grouping and injecting my own select expression, what should I assume the datatype of 'it' is? Will this even work?
The type of it is IGrouping<TKey, TElement>, where TKey is dynamic based on the keySelector result type, and TElement is the element type of the input IQueryable. Luckily IGrouping<TKey, TElement> inherits (is a) IEnumerable<TElement>, so as soon as you know the input element type, you can safely base selector on IEnumerable<TElement>.
In other words, the last attempt based on Expression<Func<IEnumerable<Company>, decimal?>> is correct.
The new error you are getting is because #0(it) generates Expression.Invoke call which is not supported by EF. The easiest way to fix that is to use LINQKit Expand method:
Expression<Func<Company, decimal?>> exp = c => c.PricingForms.Sum(fr => fr.Cost);
string selector = "new (it.Key as Key, #0(it) as Value)";
IQueryable grouping = query.GroupBy("it.GroupProperty", "it").Select(selector, exp);
// This would fix the EF invocation expression error
grouping = grouping.Provider.CreateQuery(grouping.Expression.Expand());
return grouping;
I have these codes but then the value display in the edit box is "mm/dd/yyyy"
#Html.TextBoxFor(m => m.StartDate, new { #Value = Model.StartDate.ToString("MM/dd/yyyy"), #placeholder= Model.StartDate.ToString("MM/dd/yyyy"), #class = "form-control", #type="date" })
How can I achieve something like this where the displayed date is the value from the database and not "mm/dd/yyyy"
First, don't set the value attribute directly. Razor will pretty much ignore this anyways. The value for a bound field comes from ModelState, which is itself composed of values from Request, ViewBag/ViewData, and Model, in that order. So, for example, if you want StartDate to default to "today", then you would simply populate your model with that in the action before you return the view:
model.StartDate = DateTime.Today;
Or, better, you can change the property on your model class to default to today automatically:
private DateTime? startDate;
public DateTime StartDate
{
get { return startDate ?? DateTime.Today; }
set { startDate = value; }
}
Just bear in mind that if your action happens to take a param like startDate or you set something like ViewBag.StartDate, those values will always take precedence.
Second, you're utilizing an HTML5 date input type. In browsers that support the HTML5 input types, the supplied value for a datetime, date or time, must be in ISO format. For a date, that means YYYY-MM-DD. If the value is not supplied like that, then the browser considers it garbage and discards it, leaving the control as if no value was supplied.
Also, FWIW, you don't need to prefix every member of your anonymous object with #. It doesn't technically hurt anything, but it's code smell. The # prefix exists to escape language keywords. With class, for example, you can't use it directly since it's a keyword, so you have to use #class instead. However, things like placeholder and type are not keywords, and therefore don't need an # prefix.
When I save the float number into MongoDB using csharp driver it is not saved accurately. If my number is 1504.57 I expect the database will have the same number but for some reason it become 1504.56994628906 (with Double type in MongoDB). What's going on? How to keep data accurately?
My object keep all the field as object types and cast them on the fly depending on the type, for example:
this.Values[i] = float.Parse(this.Values[i].ToString());
Maybe is it the reason of this strange behavior? But after casting this.Values[i] is pretty accurate and it's spoiled only in database.
Thanks
Update.
The class that incapsulates data:
public class TransferredData
{
[BsonElement("_id")]
[ScriptIgnore]
public ObjectId Id { get; set; }
public class Data
{
public List<Object> Values { get; set; }
public DateTypes DataType { get; set; }
public void CastToType()
{
for (int i = 0; i < this.Values.Count; i++ )
{
if (this.DataType == DateTypes.Date)
{
DateTime dt = DateTime.Parse(this.Values[i].ToString());
this.Values[i] = dt.ToUniversalTime().Date;
}
else if (this.DataType == DateTypes.Other)
{
this.Values[i] = this.Values[i].ToString();
}
else if (this.DataType == DateTypes.Reading)
{
this.Values[i] = float.Parse(this.Values[i].ToString());
}
}
}
}
}
I use Object type because I don't know which actual type it could be. So, just before doing upsert I fill up this class by data and then call Cast method.
Then I save it into db:
data = new TransferredData();
...
data.Values[1] = "1504.57"; // Because the input is always string
data.CastToType(); // Here data.Values[1] = 1504.57 - cool
TransferredDataCollection.Save<TransferredData>(data, SafeMode.True);
After this moment, looking into database... it's 1504.56994628906
IEEE 754 floating point formats cannot represent every single number accurately. As such your issue cannot be avoided and works as intended.
You should not ever use floating point formats if you need absolute guarantees that input=output while you cannot guarantee that the input can be accurately represented by the used floating point format.
Most people run into these problems when they try to store monetary values in floats which is universaly accepted to be an extremely bad idea. If you need to store monetary values save it as an integer value, usually as cents.
i imagine this has something to do with the representatioh of 1504.57 as a double precision floating point #.
perhaps there are a couple options, one being to round off the excess digits and the other being to store cents instead of dollars if the number is a currency #.
I would echo all the above comments that floating point numbers often exhibit interesting rounding errors when converted from decimal to binary and back.
In your particular case, the rounding errors are being introduced by the C# language itself, not by the C# driver. MongoDB does not natively support single precision floating point numbers, so your C# float values are being converted by the C# driver to doubles before being sent to the database. It is during this conversion that the rounding errors occur.
Here's a small snippet of C# code that demonstrates how casting a float to a double introduces the rounding errors:
var f = (float)1504.57;
var d = (double)f;
Console.WriteLine(f);
Console.WriteLine(d);
As others have recommended, if you must have 100% precision you should use some other data type than float or double.
I use Oracle's specific data provider (11g), not the Microsoft provider that is being discontinued. The thing I've found about ODP.NET is how picky it is with data types. Where JDBC and other ADO providers just convert and make things work, ODP.NET will throw an invalid cast exception unless you get it exactly right.
Consider this code:
String strSQL = "SELECT DOCUMENT_SEQ.NEXTVAL FROM DUAL";
OracleCommand cmd = new OracleCommand(strSQL, conn);
reader = cmd.ExecuteReader();
if (reader != null && reader.Read()) {
Int64 id = reader.GetInt64(0);
return id;
}
Due to ODP.NET's pickiness on conversion, this doesn't work. My usual options are:
1) Retrieve into a Decimal and return it with a cast to an Int64 (I don't like this because Decimal is just overkill, and at least once I remember reading it was deprecated...)
Decimal id = reader.GetDecimal(0);
return (Int64)id;
2) Or cast in the SQL statement to make sure it fits into Int64, like NUMBER(18)
String strSQL = "SELECT CAST(DOCUMENT_SEQ.NEXTVAL AS NUMBER(18)) FROM DUAL";
I do (2), because I feel its just not clean pulling a number into a .NET Decimal when my domain types are Int32 or Int64. Other providers I've used are nice (smart) enough to do the conversion on the fly.
Any suggestions from the ODP.NET gurus?
This will work with your original SQL:
Int64 id = System.Convert.ToInt64(reader.GetValue(0));