Can't cast database type timestamp without time zone to Instant - postgresql

reservationLogs = await this.dbContext.ReservationLogs
   .Where(r => r.ProcessedAt == null)
   .Where(r => r.PropertyId == validPropertyId)
   .OrderBy(r => r.CreatedAt).ThenBy(r => r.Operation)
   .Take(200)
   .ToListAsync();
some times with the same query i get the error
' Can't cast database type timestamp without time zone to Instant'
note: CreatedAt nodaTime instane
i am trying to find the exact reason

The issue is that even though the date and time is clear, it is unclear whether or which timezone was in use. If I tell you that tomorrow at 5 P.M. I will go for a walk, then it will be unclear from your perspective what the exact time it will be, unless you know what timezone was I assuming while saying so.
You have the exact same type of confusion in your code and first, you need to install this plugin: https://www.npgsql.org/doc/types/nodatime.html
According to the docs, you need to add a dependency like this:
using Npgsql;
// Place this at the beginning of your program to use NodaTime everywhere (recommended)
NpgsqlConnection.GlobalTypeMapper.UseNodaTime();
// Or to temporarily use NodaTime on a single connection only:
conn.TypeMapper.UseNodaTime();
The docs go further in specifying how you can read and write values:
// Write NodaTime Instant to PostgreSQL "timestamp with time zone" (UTC)
using (var cmd = new NpgsqlCommand(#"INSERT INTO mytable (my_timestamptz) VALUES (#p)", conn))
{
cmd.Parameters.Add(new NpgsqlParameter("p", Instant.FromUtc(2011, 1, 1, 10, 30)));
cmd.ExecuteNonQuery();
}
// Read timestamp back from the database as an Instant
using (var cmd = new NpgsqlCommand(#"SELECT my_timestamptz FROM mytable", conn))
using (var reader = cmd.ExecuteReader())
{
reader.Read();
var instant = reader.GetFieldValue<Instant>(0);
}
Since you are not directly writing the query, but use the Entity Framework to do so you have another level of expression. But this is well-documented as well. You can safely and soundly declare types like this:
public LocalDate Date {get; set;}
Read this full article: https://www.davepaquette.com/archive/2019/03/26/using-noda-time-with-ef-core.aspx
You will need to find out exactly where the error occurs. It seems to me that the OrderBy is the culprit as well as the selection. You can change the type of the data member of your model.

You can do cast using Npgsql.EntityFrameworkCore.PostgreSQL.NodaTime package
protected override void OnConfiguring(DbContextOptionsBuilder builder)
{
builder.UseNpgsql("connection-string",
o => o.UseNodaTime());
}
or:
builder.Services.AddDbContext<ApplicationDbContext>(
options => options.UseNpgsql(
builder.Configuration.GetConnectionString("DefaultConection"),
o => o.UseNodaTime()));
source

Related

How to Parse an int in an EF Core 3 Query?

Upon upgrading to EF Core 3, I am getting the following error at the following code:
System.InvalidOperationException: 'The LINQ expression 'DbSet
.Max(c => Convert.ToInt32(c.ClaimNumber.Substring(c.ClaimNumber.Length - 6)))'
could not be translated. Either rewrite the query in a form that can
be translated, or switch to client evaluation explicitly by inserting
a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or
ToListAsync(). See https://go.microsoft.com/fwlink/?linkid=2101038 for
more information.'
var maxId = Db.Claims
.Select(c => c.ClaimNumber.Substring(c.ClaimNumber.Length - 6))
.Max(x => Convert.ToInt32(x));
I have also tried using int.Parse instead of Convert.ToInt32, and it produces the same error. I understand the error message. However, it's trivial to get SQL Server to parse a string to an int in T-SQL with CAST or CONVERT, I would hope there's a simple way to write the query so that it translates to a server-side operation right?
UPDATE After Claudio's excellent answer, I thought I should add some info for the next person who comes along. The reason I believed the parsing was the problem with the above code is because the following runs without error and produces the right result:
var maxId = Db.Claims
.Select(c => c.ClaimNumber.Substring(c.ClaimNumber.Length - 6))
.AsEnumerable()
.Max(x => int.Parse(x));
However, I dug deeper and found that this is the SQL query EF is executing from that code:
SELECT [c].[ClaimNumber], CAST(LEN([c].[ClaimNumber]) AS int) - 6
FROM [Claims] AS [c]
WHERE [c].[ClaimNumber] IS NOT NULL
That is clearly not doing anything like what I wanted, and therefore, Claudio is right that the call to Substring is, in fact, the problem.
Disclaimer: although feasable, I strongly recommed you do not use type conversion in your query, because causes heavy query performance degradation.
Fact is that Convert.ToInt(x) part is not the problem here. It is c.ClaimsNumber.Substring(c.ClaimNumber.Length - 6), that the EF Core translator isn't able to translate in T-SQL.
Despite RIGHT function exists in Sql Server, also, you won't able to use it with current versions of EF Core (last version is 3.1.2 at the moment I'm writing).
Only solution to get what you want is to create a Sql Server user function, map it with EF Core and use it in your query.
1) Create function via migration
> dotnet ef migrations add CreateRightFunction
In newly created migration file put this code:
public partial class CreateRightFunctions : Migration
{
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql(#"
CREATE FUNCTION fn_Right(#input nvarchar(4000), #howMany int)
RETURNS nvarchar(4000)
BEGIN
RETURN RIGHT(#input, #howMany)
END
");
}
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql(#"
DROP FUNCTION fn_Right
");
}
}
Then run db update:
dotnet ef database update
2) Map function to EF Core context
In your context class[DbFunction("fn_Right")]
public static string Right(string input, int howMany)
{
throw new NotImplementedException(); // this code doesn't get executed; the call is passed through to the database function
}
3) Use function in your query
var maxId = Db.Claims.Select(c => MyContext.Right(c.ClaimNumber, 6)).Max(x => Convert.ToInt32(x));
Generated query:
SELECT MAX(CONVERT(int, [dbo].[fn_Right]([c].[ClaimNumber], 6)))
FROM [Claims] AS [c]
Again, this is far from best practice, I think you should consider to add an int column to your table to store this "number", whatever it represents in your domain.
Also, first time last 6 characters of ClaimNumber contain a non-digit character, this won't work anymore. If the ClaimNumber is input by a human, sooner or later this will happen.
You should code and design your database and application for robustness, even if you're super sure that those 6 characters will always represent a number. They could not do it forever :)
Please change your code as below. It's working for me in Dotnet core 3.1 version
var maxId = Db.Claims.Select(c => c.ClaimNumber.Substring(c.ClaimNumber.Length - 6))
.Max(x => (Convert.ToInt32((x == null)? "0" : x.ToString())));

Comparing document timestamps in Firestore rules

I'm running into a weird problem while writing and testing my Firestore rules. Here's what I want to achieve:
When the application starts, the user gets logged in anonymously. The
user starts a new game.
I create a 'Session' that basically consists of just a timestamp.
The player plays the game, gets a certain highscore and goes to a screen where the score can be sent to the global highscore list. When the highscore is submitted, I check if there's an existing session for this player and if the time that has passed is long enough for the highscore to be considered valid.
On the client (javascript) I use the following line to send the timestamp in my documents:
firebase.firestore.FieldValue.serverTimestamp()
This is the current ruleset. You can see that a score can only be created when the createdAt of the new higscore is later than the createdAt of the session.
service cloud.firestore {
match /databases/{database}/documents {
function isValidNewScoreEntry() {
return request.resource.data.keys().hasOnly(['createdAt', 'name', 'score']) &&
request.resource.data.createdAt is timestamp &&
request.resource.data.name is string &&
request.resource.data.score is int &&
request.resource.data.name.size() <= 20
}
match /highscores/{entry} {
allow list: if request.query.limit <= 10;
allow get: if true;
allow create: if isValidNewScoreEntry() &&
request.resource.data.createdAt > get(/databases/$(database)/documents/sessions/$(request.auth.uid)).data.createdAt;
}
function isValidNewSession() {
return request.resource.data.keys().hasOnly(['createdAt']) &&
request.resource.data.createdAt is timestamp
}
match /sessions/{entry} {
allow list: if false;
allow get: if false;
allow create: if isValidNewSession();
allow update: if isValidNewSession();
}
}
}
When I simulate/test these rules, I get an error that says that I cannot compare a 'timestamp' to a 'map'. I don't know why the 'createdAt' value is a map, but it seems like the get() method returns something different than expected.
My question is: What would be the correct way to compare the property createdAt from the newly submitted entry to the property createdAt of the existing session document, like I'm trying to do in the rules described above.
This is what a'Score' entry look like
This is what a 'Session' entry looks like
EDIT:
I've done some more digging, and found that this line works:
if request.resource.data.createdAt.toMillis() > get(/databases/$(database)/documents/sessions/$(request.auth.uid)).data.createdAt.seconds * 1000;
This makes it pretty clear that not both createdAt are the same format. The last one seems to be a basic object with the properties 'seconds' and 'nanoseconds'. I'm sure it stems from the Timestamp interface, but it gets returned as a flat object since none of the methods found here exist and give an error when trying to call them. The property 'seconds' however does exists on the second timestamp, but is not accessible on the first one.
I've found out why the timestamp is not what I expected and got cast to a 'map'.
After digging through the documentation I found that the get() method returns a resource. resource has a property data: a map. So the get() method does not return a document as I expected but a flat JSON object that gives me all properties found in de database.
https://firebase.google.com/docs/reference/rules/rules.firestore
https://firebase.google.com/docs/reference/rules/rules.firestore.Resource

How to edit date value with date type format in bootstrap

I have these codes but then the value display in the edit box is "mm/dd/yyyy"
#Html.TextBoxFor(m => m.StartDate, new { #Value = Model.StartDate.ToString("MM/dd/yyyy"), #placeholder= Model.StartDate.ToString("MM/dd/yyyy"), #class = "form-control", #type="date" })
How can I achieve something like this where the displayed date is the value from the database and not "mm/dd/yyyy"
First, don't set the value attribute directly. Razor will pretty much ignore this anyways. The value for a bound field comes from ModelState, which is itself composed of values from Request, ViewBag/ViewData, and Model, in that order. So, for example, if you want StartDate to default to "today", then you would simply populate your model with that in the action before you return the view:
model.StartDate = DateTime.Today;
Or, better, you can change the property on your model class to default to today automatically:
private DateTime? startDate;
public DateTime StartDate
{
get { return startDate ?? DateTime.Today; }
set { startDate = value; }
}
Just bear in mind that if your action happens to take a param like startDate or you set something like ViewBag.StartDate, those values will always take precedence.
Second, you're utilizing an HTML5 date input type. In browsers that support the HTML5 input types, the supplied value for a datetime, date or time, must be in ISO format. For a date, that means YYYY-MM-DD. If the value is not supplied like that, then the browser considers it garbage and discards it, leaving the control as if no value was supplied.
Also, FWIW, you don't need to prefix every member of your anonymous object with #. It doesn't technically hurt anything, but it's code smell. The # prefix exists to escape language keywords. With class, for example, you can't use it directly since it's a keyword, so you have to use #class instead. However, things like placeholder and type are not keywords, and therefore don't need an # prefix.

How to compare a date in entity framework to avoid rounding errors?

I have a .NET DateTime value that I write to a SQL Server database "datetime" field (and trust me, I WISH we were just using "datetime2(7)" that matches .NET's DateTime precision exactly, but we're not).
Anyway, I write the entity to the database and that particular field ends up being '2016-03-03 08:55:19.560'.
It's a last processing time, and I'm looking for other records that were processed before that time. When I run an entity framework where clause, it ends up running a statement ending with "#p__linq__0='2016-03-03 08:55:19.5602354'" as the value it's comparing against, which ends up being slightly greater, even though these two values originate from the exact same DateTime instance.
I tried changing the time it's comparing against to an SqlDateTime, but then the lambda doesn't compile because it can't compare a DateTime? to a SqlDateTime. SqlDateTime has comparison methods, but I don't know whether entity framework recognizes the functions.
I can't even cast between the two in entity framework, which just gives the error "Unable to cast the type 'System.DateTime' to type 'System.Data.SqlTypes.SqlDateTime'. LINQ to Entities only supports casting EDM primitive or enumeration types."
I face the same problem. In my case I finally value the DateTime fields by using:
public static DateTime RoundedToMs(this DateTime dt) {
return new DateTime(dt.Ticks - (dt.Ticks % TimeSpan.TicksPerMillisecond), dt.Kind);
}
public static DateTime RoundedToMsForSql(this DateTime dt) {
DateTime n = dt.RoundedToMs();
return new DateTime(n.Year, n.Month, n.Day, n.Hour, n.Minute, n.Second, (n.Millisecond / 10) * 10);
}
And in the business code:
someEntity.SomeDate = dateValue.RoundedToMsForSql();
The point, in my case, is sql datetime has a 3ms precision, so I decided to remove millisecond unit.
Same extension may be used in the queries, well in fact to populate variables used in the queries.
var d = DateTime.Now.RoundedToMsForSql();
var q = from e in ctx.Entities where e.SomeDate <= d;

Laravel 5.1 Won't save date into database set as 00-00-00 00:00:00

I'm sending dates up using this Bootstrap datetimepicker with a format of MMM D, YYY created by MomentJS, which can't submit a different format than what is displayed to the user by default.
So on the server-side I added an mutator for the date in the model:
public function setStartDateAttribute($startDate) {
//dd(Carbon::parse($startDate)->toDateTimeString());
return Carbon::parse($startDate)->toDateTimeString();
}
Which when I dd the value looks like it should 2015-10-22 00:00:00, but it saves the date as 0000-00-00 00:00:00 or it just isn't setting the date at all, which I don't understand.
I don't want to change how the timestamps are formatted in the database so I didn't set $dateFormat and set 'start_date' in the $dates array of the Model. Thought an mutator seemed easier. The field is set to date() in the migration file, and I'm pretty sure I've done this before and it just worked. So I tried it in the my controller without the mutator and the same line works:
public function update(TournamentRequest $request, $tournamentId)
{
// Update a tournament with all fillable data, and persist to database
$tournament = Tournament::find($tournamentId)->fill($request->except('start_date'));
$tournament->start_date = Carbon::parse($request->start_date)->toDateTimeString();
$tournament->save();
return Redirect::route('dashboard.tournaments.index');
}
Why does the same code work inside the controller without the mutator setup in the Model, but not work when using the mutator?
This is how you make a setter method for your attribute. Note: you don't return anything, you just assign a value to the attribute:
public function setStartDateAttribute($startDate) {
$this->attributes['start_date'] = Carbon::parse($startDate)->toDateTimeString();
}
That's assuming you want to set a field named start_date in your table.