We are moving BackEnd tables from a large MS Access application to postgresql.
In one table we have a field ErZei defined in postgres as
time(0) without time zone DEFAULT ('now'::text)::time(0) without time zone
Inside this field only the time value is stored, the day value has his own field.
The linked table inside Access shows the current day in front of the time value. Using the 24-hour format for displaying time values in Access looks fine. But by entering the field, the current day appears automatically.
If I call the time field by SQL-Query SELECT * from ..., I get the same result: everytime the current day value in front of the time value.
Inside postgres everything works fine.
Also a test with a ODBC query tool shows only the time.
How could I configure MS Access to show only the time value without the day in case of postgres time columns?
I can't find an option in the Access globals.
We use
MS Access 2016
ODBC: PostgreSQL Unicode 10.00.00 PSQLODBC35W.DLL
13.10.2017
PostgreSQL 10.1 as backend on ubuntu
Related
CURRENT SITUATION:
I have a table of wildfire incidents with a timestamp with time zone (timestamptz) field to track when the observation occurred.
Everything in the data creation process is in UTC: the incoming data from the source, the app server that inserts the data, the insert python code (appends a "Z" to the time), and the database server are all in UTC.
The incidents' geographic extent spans several time zones in the US, Canada, and Mexico.
PROBLEM:
I've been querying on a day's worth of data in UTC time, but need to extract out data relative to local time. The midnight to midnight range will be different in each time zone.
My use case now is one day, but I was asked to consider arbitrary time ranges. E.g.: find all incidents in the hottest part of the day (say 10:00 to 18:00) local time.
This table is quite large and I have an index on the timestamptz field right now. Any changes I make will need to work with an index.
Account for daylight saving time.
I have a method to get the time zone for each record, so I don't need help with that.
I created a test table for this with a timestamptz field ts_with and a varchar field for the time zone tz. The following query returns what I want, so I feel like I'm making progress.
SELECT
name, test_tz.ts_with, test_tz.tz,
TIMEZONE(test_tz.tz, test_tz.ts_with) as timezone_with
FROM fire_info.test_tz
WHERE TIMEZONE(test_tz.tz, test_tz.ts_with) BETWEEN
'2018-08-07 00:00:00' AND '2018-08-07 23:59:59';
QUESTIONS:
Will this use my index? I'm thinking the timezone function will avoid it. Any solution for that? I'm considering adding another condition to the where clause that selects on timestamptz buffered by a day on either side. That would use the index and then the timezone function isn't sorting through too much data (~6k records per day during fire season). Would PG figure that out?
The timezone function is giving me DST offsets (e.g.: Denver is currently UTC-06). I assume I'll get standard time after DST ends. If I run a query in December for data in August, will it apply standard time or DST?
thanks!!!
The way you wrote the query, it cannot use an index on ts_with.
To use an index, the condition would have to be of the form ts_with <operator> <constant>, and there is no way to rewrite the query in that fashion.
So you should create a second index on timezone(test_tz.tz, test_tz.ts_with).
First, I have read a lot about date types in Postgres using timezone or not. It seems that managing dates (timestamps) with timezone is the best to do in 99% of cases.
Currently I have built a sync engine using a Rails app set to UTC time by default (very important) and MariaDB managing dates (so UTC dates are written in the database but without seeing anything except the date: 2017-04-19 17:45:09). The devices that syncs to the server engine use local SQLite database with dates in UTC too. So for the same document, I can see the same date for a created_at field in the SQLite field and in the MariaDB field. It's working well.
Note that users never have to set a custom timezone in the server app and in the device apps. I want all UTC.
As I plan to migrate to PostgreSQL for many reasons, I'm quite sure I need to use Timestamp without timezone too, but I have some doubts after all readings.
What are your suggestions about this use case ?
EDIT:
So I made 3 tests using pgloader:
Test1: MariaDB => PostgreSQL using computer default timezone
Dates are the same but as you can see, sometimes there is +01, sometimes +02 ???
Test2: MariaDB => PostgreSQL using timezone after setting timezone to UTC in Postgres
Here there is +00, logical for UTC, but my dates were changed.
Test3: MariaDB => PostgreSQL using timestamp WITHOUT timezone
All the same there.
"With Timezone" simply tells postgres to convert from local time to UTC before storing the timestamp, and to convert back to local time when querying. The underlying data is stored in UTC and no timezone information is stored in the actual data, it's simply a flag the DB uses to determine how to convert the data for human use.
If your client connection is setting the PG timezone to UTC upon DB connection, there will be pretty much no difference. Conventional wisdom says to always use timestamps with timezone, as it'll make queries in other systems (postgresql client) a lot more flexible and easy to use in the future.
try with one of these in your config file
SET timezone= 'UTC'
SET client_timezone TO '+00:00'
my full settings were:
LOAD DATABASE
FROM mysql://usr#localhost/kk
INTO postgresql://usr:pwd#localhost/kk
WITH include drop,
create tables,
create indexes,
reset sequences,
workers = 8, concurrency = 1,
multiple readers per thread, rows per range = 50000
SET PostgreSQL PARAMETERS
maintenance_work_mem to '512MB',
work_mem to '64MB',
timezone to 'UTC'
SET MySQL PARAMETERS
net_read_timeout = '220',
net_write_timeout = '220'
SET timezone= 'UTC'
SET client_timezone TO '+00:00'
ALTER SCHEMA 'kk' RENAME TO 'public'
;
On all of my tables I have a modified column of type timestamptz and then a trigger that sets it to CURRENT_TIMESTAMP whenever the table is modified. If I'm running on my local box, which is in the PST timezone, and I make a modification, I see it saying 9:10pm my time. Now I log into a system in the EDT timezone (3 hours ahead) and query that same column. Instead of saying 12:10am it's still saying 9:10pm.
That tells me that it is in fact storing a timzeone and isn't just using the epoch value internally.
What's the right thing to be using to store a UTC time whenever this is updated?
My server side is a PHP back-end which grabs the time() output, which is supposed to be the epoch value, and then sends that back to the client on each call. When I call into the server I pass that specific value, and then my SQL query adds an "where modified > to_timestamp(?)" so that I'm only getting items which have been modified since the last time I talked to the server.
PostgreSQL 9.3 / postgresql-9.3-1100-jdbc41.jar
I have a table with a column of type timestamp without time zone, this generates my Object with the applicable java.util.Timestamp property.
What I'm seeing, during insert, is jOOQ's binding process converting a java.util.Timestamp into a date with local timezone offset.
eg for a unix timestamp 1421109419 (13 Jan 2015 00:36:59 GMT) the property is set with new Timestamp(1421109419 * 1000).
from the jOOQ logger I see:
2015-01-13 14:14:31,482 DEBUG [http-bio-8002-exec-4] org.jooq.tools.LoggerListener#debug:255 - -> with bind values : insert into "foo"."bar" ("start_date") values (timestamp '2015-01-13 13:36:59.0') returning "foo"."bar"."id"
2015-01-13 14:14:31,483 TRACE [http-bio-8002-exec-4] org.jooq.impl.DefaultBinding#trace:179 - Binding variable 3 : 2015-01-13 13:36:59.0 (class java.sql.Timestamp)
and sure enough in the record is the value "2015-01-13 13:36:59".
The software is running on a machine in NZDT which explains the +13 offset.
Given the time is being supplied in a TimeZone agnostic container (Timestamp) I would have expected that to be honoured when creating the insert statement.
How can I have jOOQ create timestamps NOT in local time?
Unfortunately you have a few things working against you:
The PostgreSQL JDBC driver sets the timezone to your JVM timezone in the Postgres session. So even if your Database Server is running in UTC a TIMESTAMP field will be inserted using the time zone of your JVM. When you insert or query data the database server will always use the JVM time zone.
You are using TIMESTAMP instead of TIMESTAMPTZ. The description of these types do not reflect their actually usage. TIMESTAMPTZ actually means time zone agnostic. Whatever value you insert it will be adjusted to UTC using the session timezone.
Because of these two issues, if you have two different JVMs -- one using Los Angeles time and the other using New York time -- whenever you write a TIMESTAMP with one JVM it will be a different "UTC time" in the other JVM. TIMESTAMP takes the adjusted value and just uses it as given. If you change your TIMESTAMP columns to be TIMESTAMPTZ then the same time in both JVMs will always be the same UTC time.
If you look at the Postgres JDBC Driver's ConnectionFactoryImpl#openConnectionImp you can see where it sets your local JVM's time zone as the time zone for the database server's session zone.
So the only sane way to deal with this is to only ever use TIMESTAMPTZ instead of TIMESTAMP. Here's some more information on this:
PostgreSQL/JDBC and TIMESTAMP vs. TIMESTAMPTZ
http://justatheory.com/computers/databases/postgresql/use-timestamptz.html
The following (very nasty) code works for me:
eventsRecord.setCreatedOn(new Timestamp(System.currentTimeMillis()
- TimeZone.getDefault().getOffset(new Date().getTime())));
Alas jOOQ simply uses the local timezone when saving into PostgreSQL "timestamp without timezone" or MySQL "datetime" fields. The source code evidence for this travesty is here, it does not specify the timezone nor have any facility for the user to override this functionality and specify a timezone. This renders usage of this very basic datatype from jOOQ completely useless, with a multitude of clients all with different timezones writing data to the same field without recording their timezone nor normalizing the data to UTC.
JDBC provides an extra three-argument setTimestamp where the user can specify what timezone is desired (UTC is basically the only value that makes sense). However jOOQ "abstracts" away from JDBC and does not offer this facility.
I'm managing small PostgreSQL database. Recently I created new date field in one of tables. I'm working on pgAdminIII and everything is fine, but one of the users just reported that while working on LibreOffice all dates are shifted by 10 days to the past.
I just installed LibreOffice to check it and they are shifted on my copy too but by 9 days.
This is simple date field without any default value or constraints. All dates are before year 1900 (1400-1500).
In pgAdmin and console psql client everything is fine.
What can cause such an issue? How to fix that?
EDIT:
Problem occurs in table view only. I can access correct date via Tools > SQL... menu while viewing table data or creating query produces invalid results. It may be format conversion error (from Y-M-D to D.M.Y).