Where are my timestamps getting "messed up"? - postgresql

Since I am already getting down voted I decided to give some more details to my problem.
My problem is that I want to store times. Times only for easy comparison. To me it makes sense to define it as following:
00:00:00 = (long) 0L
24:00:00 = (long) 24*60*60*1000L
The PostgreSQL documentation says about TIME WITHOUT TIME ZONE this:
time [ (p) ] [ without time zone ]
8 bytes | time of day (no date) | 00:00:00 - 24:00:00 | 1 microsecond / 14 digits
But for some reason, somewhere along the way from my database to my web application the timestamps are getting messed up.
In the following I want to show you that I am storing TIME WITHOUT TIME ZONE, mapped from LocalTime (jodatime), into my database and afterwards fetch it back.
Fetching it back from the database and mapping it back into a LocalTime object will give me something like 23 Feb 2016 08:00:00 GMT which is
1456214400000 and that is > 24*60*60*1000.
There are 3 to 4 options:
PostgresSQL stores actually the whole TIMESTAMP and shows HH:mm:ss just for presentation
jodatime is inventing things here that are not there.
(very unlikely) The mapper I am using does more than I tell him. But that is unlikely since the mapper does not touch anything.
(probably true) I myself mess things up somehow.
More details:
I decided to add some more details. I am creating a record for my table shop_times
private Long createShopTimes(Long shopId, DateTime dayFrom, DateTime dayTo, LocalTime timeFrom, LocalTime timeTo, DayOfWeek dayOfWeek, ShopTimesType shopTimesType) {
Long timePeriodId = this.ctx.insertInto(SHOP_TIMES)
.set(SHOP_TIMES.SHOP_ID, shopId)
.set(SHOP_TIMES.DAY_OF_WEEK_ID, dayOfWeek)
.set(SHOP_TIMES.SHOP_TIMES_TYPE_ID, shopTimesType)
.set(SHOP_TIMES.DAY_FROM, dayFrom)
.set(SHOP_TIMES.DAY_TO, dayTo)
.set(SHOP_TIMES.TIME_FROM, timeFrom)
.set(SHOP_TIMES.TIME_TO, timeTo)
.returning(SHOP_TIMES.ID)
.fetchOne().getValue(SHOP_TIMES.ID);
List<ShopTimesRecord> fetchInto = this.ctx.select(
SHOP_TIMES.TIME_FROM,
SHOP_TIMES.TIME_TO
)
.from(SHOP_TIMES)
.fetchInto(ShopTimesRecord.class);
for (ShopTimesRecord shopTimesRecord : fetchInto) {
if(shopTimesRecord.getTimeFrom().toDateTimeToday().getMillis() > 24*60*60*1000L) {
System.err.println("This should not happen..");
}
Date from = new Date(shopTimesRecord.getTimeFrom().toDateTimeToday().getMillis());
Date to = new Date(shopTimesRecord.getTimeTo().toDateTimeToday().getMillis());
System.out.println(from.toGMTString());
System.out.println(shopTimesRecord.getTimeFrom().toDateTimeToday().getMillis());
System.out.println(to.toGMTString());
System.out.println(shopTimesRecord.getTimeTo().toDateTimeToday().getMillis());
}
return timePeriodId;
}
As you can see, I am getting something that I would not expect at this point:
This should not happen..
23 Feb 2016 08:00:00 GMT
1456214400000
23 Feb 2016 20:00:00 GMT
1456257600000
This is the table shop_times that I am using to store timing information:
CREATE TABLE shop_times (
-- PRIMARY KEY
id BIGSERIAL PRIMARY KEY,
-- FOREIGN KEYS
shop_id BIGINT NOT NULL,
CONSTRAINT fk__shop_times__shop
FOREIGN KEY (shop_id)
REFERENCES shop(id),
shop_times_type_id BIGINT NOT NULL,
CONSTRAINT fk_shop_times__shop_times_type
FOREIGN KEY (shop_times_type_id)
REFERENCES shop_times_type(id),
day_of_week_id BIGINT NOT NULL,
CONSTRAINT fk__shop_times__day_of_week
FOREIGN KEY (day_of_week_id)
REFERENCES day_of_week(id),
-- ATTRIBUTES
day_from TIMESTAMP WITH TIME ZONE NOT NULL,
day_to TIMESTAMP WITH TIME ZONE NOT NULL,
time_from TIME WITHOUT TIME ZONE NOT NULL,
time_to TIME WITHOUT TIME ZONE NOT NULL,
-- CONSTRAINTS
CHECK(day_from < day_to),
CHECK(time_from < time_to)
);
The mapper I am using. But as you can see it is just taking the time it gets and passes it further;
public class TimeWithoutTzToJodaLocalTimeConverter implements Converter<Time, LocalTime> {
private static final long serialVersionUID = -2736422625956918206L;
#Override
public LocalTime from(Time timestamp) {
LocalTime dateTime = new LocalTime(timestamp.getTime());
return dateTime;
}
#Override
public Time to(LocalTime localTime) {
Time time = new Time(localTime.toDateTimeToday().getMillis());
return time;
}
#Override
public Class<Time> fromType() {
return Time.class;
}
#Override
public Class<LocalTime> toType() {
return LocalTime.class;
}
}

Sorry - don't believe you.
=> SELECT '24:00:00'::time without time zone;
time
----------
24:00:00
(1 row)
=> SELECT '24:00:01'::time without time zone;
ERROR: date/time field value out of range: "24:00:01"
LINE 1: SELECT '24:00:01'::time without time zone;

You're creating a full DateTime:
.toDateTimeToday()
http://joda-time.sourceforge.net/api-release/org/joda/time/TimeOfDay.html#toDateTimeToday%28%29

Related

how can i get the all data from postgresql thats from the current day, between the hours of 00:00:00 and 11:59:59?

very new to the backend as well as all things postgresql, at the moment all ive been able to do is
SELECT * FROM nutrition WHERE timestamp >= (extract(epoch from now())::bigint * 1000) - 86400000 AND timestamp <= (extract(epoch from now())::bigint * 1000) + 86400000
in the frontend using js, im using Date.now() to store the timestamp in the DB.
timestamp is a column in my db thats logging the unix time in bigint format in which the food was logged. I want to get all the data from the current day from the hours beteen 12 AM midnight, and 11:59 PM. thanks.
for example, the last item i logged was last night at 10pm (1663995295337 UNIX time) so the data shouldnt include it.
show timezone returns;
America/Los_Angeles
Solution below --------------------------------------------------------------------
const today = new Date();
const beginningOfDay = today.setUTCHours(7, 0, 0, 0);
const endOfDay = today.setUTCHours(30, 59, 59, 99);
switch (method) {
case "GET":
try {
const text = `
select
*
from
nutrition
where
timestamp
between ${beginningOfDay} and ${endOfDay}`
this was the solution i was looking for, thanks for the help. sorry if i wasnt descriptive enough.
Assuming that by Unix time you mean epoch.
select extract(epoch from now());
extract
-------------------
1664038032.392004
select to_timestamp(1664038032.392004);
to_timestamp
--------------------------------
09/24/2022 09:47:12.392004 PDT
select
*
from
some_table
where
to_timestamp(1664038032.392004)
between
current_date + '00:00:00'::time AND current_date + '23:59:59'::time
UPDATE
Using timestamptz field in Postgres and an ISO datetime string from Javascript in order to properly deal with time zone.
create table tsz_test(id integer, tsz_fld timestamptz);
--In Web frontend today = new Date().toISOString(); "2022-09-24T20:57:05.830Z"
insert into
tsz_test
values (1, '2022-09-24T20:57:05.830Z'), (2, '2022-09-25T08:57:05.830Z');
select * from tsz_test ;
id | tsz_fld
----+----------------------------
1 | 09/24/2022 13:57:05.83 PDT
2 | 09/25/2022 01:57:05.83 PDT
--Borrowing from #a_horse_with_no_name answer
select * from tsz_test where tsz_fld::date = '09/24/2022'::date;
id | tsz_fld
----+----------------------------
1 | 09/24/2022 13:57:05.83 PDT
You can convert your "unix timestamp" to a date, then compare it with "today":
where to_timestamp("timestamp")::date = current_date
This assumes that your column named "timestamp" is not really a timestamp
If the column doesn't actually store seconds (which would be a unix epoch), but milliseconds you need to_timestamp("timestamp"/1000)::date instead (another source of problems that wouldn't exist if you had used a proper timestamptz or at least timestamp data type).

How to query a Timestamptz with a Unix timestamp using Diesel?

I can easily get the system Unix millisecond timestamp, but how can I convert it into something I can query with Diesel? The table in my schema has a Timestampz field (timestamp with time zone) that I need to compare to the Unix timestamp.
use diesel::sql_types::Timestamptz;
fn main() {
let unix_time: i64 = 123456;
let time: Timestamptz = ???
}
[dependencies]
diesel = { version = "1.4.7", features = ["postgres"] }
Timestamptz can be created from a PgTimestamp, chrono::NaiveDateTime or chrono::DateTime according to the docs.
I don't think there is a direct way to generate a timestamptz, and it's supposed to be opaque in most instances. Instead where a queries requires a Timetamptz you can pass a NaiveDateTime and load a NaiveDateTime back.
Now we use NaiveDateTime as it's the only one that can be created from a unix timestamp. DateTime requires an explicit timezone and can be generated from a NaiveDateTime.
use chrono::NaiveDateTime;
fn main() {
let conn = get_connection();
let ndt = NaiveDateTime::from_timestamp(0, 42_000_000);
let (id, date) = users
// assume a "created" column being a Timestamptz
// you can just pass a NaiveDateTime.
.filter(created.gt(ndt))
.select((user_id, created))
// We load back a NaiveDateTime and never deal with Timestamptz directly.
.first::<(u64, NaiveDateTime)>(&conn)
.expect("query failed");
}

How to save time in the database in Go when using GORM and Postgresql?

I'm currently parsing a time string and saving it to the db (Postgresql):
event.Time, _ := time.Parse("3:04 PM", "9:00 PM")
// value of event.Time now is: 0000-01-01 21:00:00 +0000 UTC
db.Create(&event)
It's giving me this error: pq: R:"DateTimeParseError" S:"ERROR" C:"22008" M:"date/time field value out of range: \"0000-01-01T21:00:00Z\"" F:"datetime.c" L:"3540"
event.Time⁠⁠⁠⁠'s type is time.Time.
I also tried setting event.Time's type to string and using time data type in postgresql:
type Event struct {
Time string `gorm:"type:time
}
But now I'm getting an error when fetching records in the db:
sql: Scan error on column index 4: unsupported driver -> Scan pair: time.Time -> *string
Investigated this issue further. Currently, there's no support in GORM for any Date/Time types except timestamp with time zone
See this part of code from dialect_postgres.go:
case reflect.Struct:
if _, ok := dataValue.Interface().(time.Time); ok {
sqlType = "timestamp with time zone"
}
So basically I see two options for you:
Either use varchar(10) in DB, and string in Go, an simply save it as "9:00 PM" (where 10 is some number that suits you)
Or use timestamp with time zone in DB, time.Time in Go, and format your date part as a constant date, 01/01/1970, for example:
time.Parse("2006-01-02 3:04PM", "1970-01-01 9:00PM")
In that case you'll have to omit the date part in your presentation, but if you plan to select by date range, that could work better for you.
You can set an arbitrary database-specific type with Gorm using sql tag
type Event struct {
Time time.Time `sql:"type:timestamp without time zone"`
}
When updating the DATETIME field in SQL, the Go string must be in this format: time.Now().Format(time.RFC3339).
From Postgres perspective the error stems from there being no year 0000. If you don't the date you may just be able to add 1 year to the converted timestamp giving '0001-01-01T21:00:00+00' which is a valid Postgres timestamp.
select '0000-01-01T21:00:00+00'::timestamptz at time zone 'UTC'
ERROR: date/time field value out of range: "0000-01-01T21:00:00+00"
Gives he same error. And just as a demonstration 1 day before 0001-01-01 gives:
select '0001-01-01T21:00:00+00'::timestamptz at time zone 'UTC' - interval '1 day' "day_before_1/1/1";
--day_before_1/1/1
--0001-12-31 21:00:00 BC

can't insert date into postgresql via eclipselink

I have this table in postgresql
CREATE TABLE remise
(
id bigserial NOT NULL,
date_remise time without time zone
)
and this code to insert a row into it via eclipselink :
Remise rm = new Remise();
rm.setId(1L);
rm.setDateRemise(Calendar.getInstance().getTime()); //today
dao.begin();
dao.save(rm);
dao.commit();
in the Remise Entity class I have this field with it's getter and setter :
#Column(name = "date_remise")
#Temporal(TemporalType.TIME)
private Date dateRemise;
everything looks ok but the date I get inserted is like that with just time and no date !
in pgAdmin III : "03:16:03"
via my application : "Thu Jan 01 03:16:03 WET 1970"
I'm using : Postgresql 9.3, 9.1-901-1.jdbc4, eclipselink 2.5.2.
Need your help to fix this problem. thanks.
You need to use timestamp without time zone.
time without time zone only stores time portion.
thank you araqnoon for the correct response, just one precision for the mapping of the field it must look like :
#Column(name = "date_remise")
#Temporal(TemporalType.TIMESTAMP)
private Date dateRemise;

problems to get the full DATE info from Oracle DB (dd/mm/yyyy hh/mm/ss)

I have a column in a table in which we are storing date in DATETIME format. (DD-MON-RRRR HH24:MI:SS) - Database Oracle 11g.
Data Type of a column is DATE, and storing date in 01-01-2012 01:00 PM (i.e. jan 1, 2012) format.
entity
#NotNull
#Column(name = "dateColumnName")
#Temporal(TemporalType.TIMESTAMP)
private Date sampleDate;
I am fetching all data by passing date
SAMPLE_QUERY = "select * from TableA tab where tab.dateWithTime = :sampleDate order by tab.dateWithTime ASC "
singleDate is "Tue Jan 24 00:00:00 IST 2012" , fasttime :
1327343400000
The problem is I am passing only date in the query, though Date through which records are being fetched is in DATE TIME format i.e 01-01-2012 01:00 PM.
How can i change my query so that it fetches all the records in ascending order of DateTime.
If you want to fetch all times for that day, then change your query to be more like
SELECT ... WHERE dateField >= :lowerParam AND dateField < :upperParam
Oracle has no DATE TIME datatype. The DATE datatype contains both a date and a time component, down to the second. TIMESTAMPS get a bit more complicated.
If your dateWithTime column is indeed a DATE datatype, the ORDER BY dateWithTime ASC clause should order your results in ascending order.
You may not be displaying the time component of your date. You can convert a date to a string in that format with TO_CHAR( dateWithTime, 'dd/mm/yyyy hh24/mm/ss' ) or whatever format you want.
Edit:
Oh, do you mean you want to find the cases where the date component of the DATE matches, but you don't care about the time component? That can be handled in the where clause with something like:
WHERE TRUNC( tab.dateWithTime ) = TRUNC( :sampledate )
TRUNC by default truncates a date to the beginning of the day.