Here is my problem:
I want to calculate how long ago a record was updated in a DB.
The DB is in PostgreSQL, the update_time field is populated by a trigger that uses CURRENT_TIMESTAMP(2). The field is inflated to a DateTime object by DBIx::Class. I get the current time in my code using DateTime->now()
My problem is that when I retrieve the field value, it's off by 1 h (ie it's 1h ahead of DateTime->now()). I am in the CET time zone, so 1h ahead of UTC currently.
The right way to solve the problem is likely at the DB level. I have tried to replace CURRENT_TIMESTAMP with LOCALTIMESTAMP, to no avail.
I think actually a more robust solution (ie one that doesn't rely on getting the DB right) would be to get the current time stamp from the DB itself. I really just need the epoch, since that's what I use to compute the difference.
So the question is: is there a simple way to do this: get the current time from the DB using DBIx::Class?
A different way to get the DB and DateTime to agree on what the current time is would also be OK!
You can use dbh_do from your DBIx::Class::Storage to run arbitrary queries. With that, just SELECT the CURRENT_TIMESTAMP.
my ( $timestamp ) = $schema->storage->dbh_do(
sub {
my ($storage, $dbh) = #_;
$dbh->selectrow_array("SELECT CURRENT_TIMESTAMP");
},
);
I always recommend to do all date/time related things on the app server and not rely on the database server(s). Essentially that means to not use a trigger but pass the datetime on insert/update and make it mandatory (NOT NULL).
Besides that you should store datetimes in UTC and convert to your local or other required timezone in your code.
Your issue likely happens because of an incorrect or missing timezone configuartion in which case DateTime defaults to its floating timezone.
Related
I'm digging into how Postgres works and have decided that any date/time data in my database should be of datatype timestamptz.
The rules that govern how Postgres parses date/time information vary based on the server's timezone, the client session timezone, and/or the database timezone setting. I can't expect my developers to know all of this, so to avoid any ambiguity I would like to somehow procedurally require a timezone be specified in any INSERT or UPDATE to a timestamptz column, and for any UPDATES or INSERTS to fail when the input value for a timestamptz column doesn't explicity include a time zone. I've created a regex that I can use to match against the input value; I just don't know how to hook up the plumbing.
I first thought I could do this with a custom domain; however, it appears that the CHECK constraint on a domain is done after the input string has already been parsed, so that won't work. (By then, the server has already inferred the time zone for values where time zone wasn't explicitly included.)
I could use a custom data type, but that's a whole can of worms there and I'm not sure that doing so would preserve all of the operators and functions that would operate on the underlying timstamptz column.
I could use BEFORE INSERT and BEFORE UPDATE triggers, but doing so would require me to iterate over every column in the NEW record, determine its datatype, then check the value against a regex to ensure a time zone is specified.
Does the community have any ideas on how to accomplish this? I think the BEFORE INSERT/BEFORE UPDATE is likely the best place to do this work, but I don't know how to iterate over the new record and find the data type for each column.
Is there an easier way to accomplish this that I've missed?
I can't expect my developers to know all of this
I think that's your problem. If you want to use PostgreSQL and work with time zones, you need your developers to understand it.
It's all very simple: Only set the timezone parameter correctly for the client session, then everything will just work.
I have a table that I am using to store iso dates with timezones. I realize that dates should "always" be stored as utc but I have an exception to that rule. The timestamps aren't in any way related to the server they are running on. I want to be able to store an iso date like this:
2016-03-06T01:15:52-06:00
And regardless of the time zone of the server or anything else I want the timestamp returned as:
2016-03-06T01:15:52-06:00
Currently if I insert an iso date it automatically converts it to whatever the server timezone is. My above date gets converted to:
2016-03-06 07:15:52+00 (server is utc)
The only thing I can think of is storing the timezone offset in a separate column, storing my date as utc and then converting using the offset column, horribly messy. Surely there is a way to store my date in one column and get it out the way it was originally created?
Your proposed solution is correct. Or more precisely, it is one of several correct implementations. Any of the following would work:
Store the UTC timestamp in one field, store the offset in another.
Store the local timestamp in one field, store the offset in another.
Store the local date in one field, and store a time with time zone in another. (though time with time zone is generally discouraged...)
Store the UTC timestamps in one field and the local timestamp in another.
The easiest by far is the first one, which you already proposed.
I'd avoid against storing timestamps in text fields, as they tend not to be very efficiently searchable.
Also note - if you're coming from a SQL Server background, you might recall its datetimeoffset type, which stores the local datetime and offset in the field, and uses the UTC equivalent during indexing. It's common to think that Postgres and MySQL's timestamp with time zone would have the same behavior, but they don't. They simply use the session time zone to convert to/from UTC. SQL Server has no concept of a session time zone, and thus the discrepancy.
Be sure to read this part of the Postgres docs.
I used to insert timestamp into my PostgreSQL table in local time. Now I started to use UTC and I need to find a way to convert all dates/times already inserted to UTC time. How am I supposed to do that?
Pretty easy, in timestamp with time zone, all times are already stored in UTC, the conversion is done when you insert or select them (hours are added or subtracted).
The safest way to remove the "with time zone" attribute would be to set your timezone to UTC,
create a new field that is timestamp (without time zone) and set the data correctly, then drop the old field, re-create it without time zone, then copy the data back in, dropping the column.
Seems like a lot of work, but you could verify your results along the way, if you needed to do any updates or back out altogether, you'd know that there was no chance for data loss.
In a scala program, I receive from client side a specific date for instance:
2013-10-20T23:59:59.999Z
and I really want to keep this date when saving into DB and not convert to local, so this line:
debug("--sql timestamp: " + new Timestamp(reading.timestamp.getMillis()))
is printing out: 2013-10-21 02:59:59.999(I am in Romania).
Is there any way I can ignore timezone?
This is Timestamp.toString() behavior. java.sql.Timestamp extends java.util.Date and in its toString() method it uses, in particular, super.getHours(), which, according to javadoc, returns hours interpreted in local timezone - exactly as you observe.
However, internally Timestamp still holds correct timestamp value. There may be problems with storing it to the database, though. See this answer.
2013-10-20T23:59:59.999Z and 2013-10-21 02:59:59.999 are actually the same time: 2013-10-20T23:59:59.999Z is in the UTC time zone (Z), whereas the second one is relative, and expressed as your local time zone (UTC+3 then in Romania).
In PostgreSQL, you should store your timestamps as TIMESTAMP WITH TIME ZONE (TIMESTAMPTZ) in your database to handle this. You'll always be able to print it out later in the time zone you choose then (e.g. UTC). (You might be interested in this recent question to understand why the storage type matters.)
If you want to print out the timestamp in the UTC/Z time zone again, new DateTime(millis, DateTimeZone.UTC) should help (with Joda Time).
I have never used DBIx::Class until today, so I'm completely new at it.
I'm not sure if this is possible or not, but basically I have a table in my SQLite database that has a timestamp column in it. The default value for the timestamp column is "CURRENT_TIMESTAMP". SQLite stores this in the GMT timezone, but my server is in the CDT timeszone.
My SQLite query to get the timestamp in the correct timezone is this:
select datetime(timestamp, 'localtime') from mytable where id=1;
I am wondering if it is possible in my DBIx schema for "MyTable" to force it to apply the datetime function every time it is retrieving the "timestamp" field from the database?
In the cookbook it looks like it is possible to do this when using the ->search() function, but I am wondering if it's possible to make it so if I'm using search(), find(), all(), find_or_new(), or any function that will pull this column from the database, it will apply the datetime() SQLite function to it?
DBIx::Class seems to have great documentation - I think I'm just so new at it I'm not finding the right places/things to search for.
Thanks in advance!
I've used InflateColumn::DateTime in this way and with a timestamp, and I can confirm it works, but I wonder if you have this backward.
If your column is in UTC, mark the column UTC, and then it should be a UTC time when you load it. Then when you set_timezone on the DateTime (presumably that would be an output issue - it's at output that you care it's locally zoned) you can set it to local time and it will make the necessary adjustment.