I have a script that will insert data into a postgres table the issue is the data has a date format of DD/MM/YYYY and this date style Isn't accepted - I cant change the date style as that only lasts a session ( is what I read). I have this function and trigger to change the date format but I get the error date/time field value out of range: "DD/MM/YYY"
could anyone help on what is wrong with the function/trigger to get the date format to work I've tried a few iterations of this and all give the same error
CREATE OR REPLACE FUNCTION format_date()
RETURNS TRIGGER
LANGUAGE plpgsql
AS
$$
BEGIN
NEW.Invoice_Date = to_char(NEW.Invoice_Date, 'YYYY-MM-DD')::date;
NEW.Due_Date = to_char(NEW.Due_Date, 'YYYY-MM-DD')::date;
NEW.Paid_Date = to_char(NEW.Paid_Date, 'YYYY-MM-DD')::date;
RETURN NEW ;
END;
$$
CREATE TRIGGER format_date
BEFORE INSERT ON table_name
FOR EACH ROW
EXECUTE PROCEDURE format_date();
I've tried with to_date with and without the new. within the to_char and I've tried forcing it to date with ::date on the column. Any help will be much appreciated
Related
I have imported a csv file in my postgresql database. The file contains a column of dates. I have imported that column in 'varchar' form. the column contains dates that are not in one particular format, meaning some dates are dd/mm/yyyy and some in mm/dd/yyyy. Now I'm trying to cast these strings into date format, but Im unable to do. postgresql gives error 'field value out of range'.
What should I do? Should I make date format consistent in csv file or is there any method in postgres?
Sample data ;
project_date 1. '23/10/2022'
2. '12/31/2022'
I am trying to get answers from the fellows here
The first thing you need to do is get a specific date format (all dates the same format), preferably ISO-8601 (yyyy-mm-dd). If you cannot do that then you can build a function that tests the date formats permitted with the to_date and returns the first successful conversion. Something like:
create or replace function unscrew_date(should_be_a_date text)
returns date
language plpgsql
as $$
declare
k_date_formats constant text[] = array['dd/mm/yyyy','mm/dd/yyyy'];
dt_fmt text;
begin
foreach dt_fmt in array k_date_formats
loop
begin
return to_date(should_be_a_date,dt_fmt);
exception
when datetime_field_overflow then null;
when invalid_datetime_format then null;
when others then raise exception 'Unexpected error: %, %',sqlstate, sqqlerrm;
end;
end loop;
return null;
end;
$$;
However this still leaves a whole set of dates the cannot be known correct; any day less than 13. For example '12/08/2020', is that December 8, or August 12. The above function will always return August 12,2020.
The only valid workable solution is standardize the date format. Then deal with that format.
I Have some difficulties with my data base.
i have uploaded data from multiple excel file, each file has a spécific date Format. some time DD/MM/YYYY and some time YYYY/MM/DD the column is character varring.
i want to make them YYYY/MM/DD.
A simple solution:
select regexp_replace('05/01/2019', '(\d\d)/(\d\d)/(\d\d\d\d)', '\3/\2/\1')
regexp_replace
----------------
2019/01/05
(1 row)
You could update the table with
update my_table
set date_column = regexp_replace(date_column, '(\d\d)/(\d\d)/(\d\d\d\d)', '\3/\2/\1')
However, you should basically store dates in a column of type date. Use the function to convert differently formatted texts to dates:
create or replace function iso_date(text)
returns date language sql immutable as $$
select case
when $1 like '__/__/____' then to_date($1, 'DD/MM/YYYY')
when $1 like '____/__/__' then to_date($1, 'YYYY/MM/DD')
end
$$
The above is an example, you can modify the function if you have more different formats. Now you can alter the column type in this way:
alter table my_table alter date_column type date using iso_date(date_column);
Read more about Data Type Formatting Functions and POSIX Regular Expressions in the documentation.
This is basically a duplicate of this with s/mysql/postgresql/g.
I created a table that has a timestamp TIMESTAMP column and I am trying to import data from CSV files that have Unix timestamped rows.
However, when I try to COPY the file into the table, I get errors to the tune of
2:1: conversion failed: "1394755260" to timestamp
3:1: conversion failed: "1394755320" to timestamp
4:1: conversion failed: "1394755800" to timestamp
5:1: conversion failed: "1394755920" to timestamp
Obviously this works if I set the column to be INT.
In the MySQL variant, I solved with a trick like
LOAD DATA LOCAL INFILE 'file.csv'
INTO TABLE raw_data
fields terminated by ','
lines terminated by '\n'
IGNORE 1 LINES
(#timestamp, other_column)
SET timestamp = FROM_UNIXTIME(#timestamp),
third_column = 'SomeSpecialValue'
;
Note two things: I can map the #timestamp variable from the CSV file using a function to turn it into a proper DATETIME, and I can set extra columns to certain values (this is necessary because I have more columns in the database than in the CSV).
I'm switching to postgresql because mysql lacks some functions that make my life so much easier with the queries I need to write.
Is there a way of configuring the table so that the conversion happens automatically?
I think you could accomplish this by importing the data as-is, creating a second column with the converted timestamp and then using a trigger to make sure any time a row is inserted it populates the new column:
alter table raw_table
add column time_stamp timestamp;
CREATE OR REPLACE FUNCTION raw_table_insert()
RETURNS trigger AS
$BODY$
BEGIN
NEW.time_stamp = timestamp 'epoch' + NEW.unix_time_stamp * interval '1 second';
return NEW;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
CREATE TRIGGER insert_raw_table_trigger
BEFORE INSERT
ON raw_table
FOR EACH ROW
EXECUTE PROCEDURE raw_table_insert();
If the timestamp column can be modified, then you will want to make sure the trigger applies to updates as well.
Alternatively, you can create a view that generates the timestamp on the fly, but the advantages/disadvantages depend on how often you search on the column, how large the table is going to be and how much DML you expect.
This question already has answers here:
PostgreSQL - how to render date in different time zone?
(2 answers)
Closed 5 years ago.
I have a "timestamp with time zone" field in a table.
I need to return is as iso 8601 string in a give time zone.
The closest thing that I managed to do is this:
select to_char(crtdate, 'YYYY-MM-DD"T"HH24:MI:SS.MSOF:"00"') from t1
But it returns a timestamp in a system default time zone (UTC) i.e. something like this:
2017-07-12T02:46:26.194+00:00
while I need it be formatted for a specific time zone.
E.g.
2017-07-12T14:46:26.194+12:00
for "Pacific/Auckland".
Can someone please advise how this can be achieved?
we are on PG v 9.6.
Thank you,
You can play with GUC parameters datestyle and timezone inside a function to get what you want. Here is an example (however, it returns microseconds, so probably you'll need to tune it a bit):
create or replace function timestamp_iso8601(ts timestamptz, tz text) returns text as $$
declare
res text;
begin
set datestyle = 'ISO';
perform set_config('timezone', tz, true);
res := ts::timestamptz(3)::text;
reset datestyle;
reset timezone;
return replace(res, ' ', 'T') || ':00';
end;
$$ language plpgsql volatile;
Results:
test=# select timestamp_iso8601(now()::timestamptz, 'Europe/Moscow');
timestamp_iso8601
-------------------------------
2017-07-12T08:56:58.692985+03:00
test=# select timestamp_iso8601(now()::timestamptz, 'Pacific/Auckland');
timestamp_iso8601
-------------------------------
2017-07-12T17:59:05.863483+12:00
(1 row)
Update: edited. You can use timestamptz(3), specifying the precision (by default, it will go with microseconds, while 3 will keep only milliseconds). Alternatively, you can use res := to_char(ts::timestamptz, 'IYYY-MM-DDT HH24:MI:SS:MSOF'); instead of ::timestamptz(3)::text conversion chain, and in this case (3) will not be needed.
I'm trying to create an index on the cast of a varchar column to date. I'm doing something like this:
CREATE INDEX date_index ON table_name (CAST(varchar_column AS DATE));
I'm getting the error: functions in index expression must be marked IMMUTABLE But I don't get why, the cast to date doesn't depends on the timezone or something like that (which makes a cast to timestamp with time zone give this error).
Any help?
Your first error was to store a date as a varchar column. You should not do that.
The proper fix for your problem is to convert the column to a real date column.
Now I'm pretty sure the answer to that statement is "I didn't design the database and I cannot change it", so here is a workaround:
CAST and to_char() are not immutable because they can return different values for the same input value depending on the current session's settings.
If you know you have a consistent format of all values in the table (which - if you had - would mean you can convert the column to a real date column) then you can create your own function that converts a varchar to a date and is marked as immutable.
create or replace function fix_bad_datatype(the_date varchar)
returns date
language sql
immutable
as
$body$
select to_date(the_date, 'yyyy-mm-dd');
$body$
ROWS 1
/
With that definition you can create an index on the expression:
CREATE INDEX date_index ON table_name (fix_bad_datatype(varchar_column));
But you have to use exactly that function call in your query so that Postgres uses it:
select *
from foo
where fix_bad_datatype(varchar_column) < current_date;
Note that this approach will fail badly if you have just one "illegal" value in your varchar column. The only sensible solution is to store dates as dates,
Please provide the database version, table ddl, and some example data.
Would making your own immutable function do what you want, like this? Also look into creating a new cast in the docs and see if that does anything for you.
create table emp2 (emp2_id integer, hire_date VARCHAR(100));
insert into emp2(hire_date)
select now();
select cast(hire_date as DATE)
from emp2
CREATE FUNCTION my_date_cast(VARCHAR) RETURNS DATE
AS 'select cast($1 as DATE)'
LANGUAGE SQL
IMMUTABLE
RETURNS NULL ON NULL INPUT;
CREATE INDEX idx_emp2_hire_date ON emp2 (my_date_cast(hire_date));