I am importing a csv file into a Postgres Table. The file has the following format:
2019/12/13, 14:56:02, 3172.50, 3174.25, 3172.50, 3172.50, 1, 1, 1, 0
The table is defined as:
CREATE TABLE tablename (
date date,
time time,
v1 numeric,
v2 numeric,
v3 numeric,
v4 numeric,
v5 integer,
v6 integer,
v6 integer,
v7 integer,
PRIMARY KEY(date, time)
);
There is an issue with the time field. In some cases, milliseconds are added for precision:
14:56:02.1
14:56:02.9
14:56:02.10
Unfortunately, Postgres seems to drop the trailing zero, which causes it to mark below two values as duplicates:
14:56:02.1
14:56:02.10
ERROR: duplicate key value violates unique constraint "tablename_pkey"
DETAIL: Key (date, "time")=(2019-12-13, 14:56:02.1) already exists.
CONTEXT: COPY input_file, line 1584
Is there a way to instruct psql not to drop trailing zeroes? I tried time(4) to enforce 4 digit precision, with no difference.
Thanks!
Postgres is not doing anything wrong here. It took me a moment to realize that the issue is with the data.
.1 and .10 are equal. In the data, the timestamp was used creatively, i.e. in this case .1 means "1st record within this second" and .10 means "10th record within this second", so the millisecond component didn't make sense from timestamp's point of view.
Related
Postgres (V11.3, 64bit, Windows) truncates trailing zeros for timestamps. So if I insert the timestamp '2019-06-12 12:37:07.880' into the table and I read it back as text postgres returns '2019-06-12 12:37:07.88'.
Table date_test:
CREATE TABLE public.date_test (
id SERIAL,
"timestamp" TIMESTAMP WITHOUT TIME ZONE NOT NULL,
CONSTRAINT pkey_date_test PRIMARY KEY(id)
)
SQL command when inserting data:
INSERT INTO date_test (timestamp) VALUES( '2019-06-12 12:37:07.880' )
SQL command to retrieve data:
SELECT dt.timestamp ::TEXT FROM date_test dt
returns '2019-06-12 12:37:07.88'
Do you consider this a bug or a feature?
My real issue is: I´m running queries from a C++ program and I have to convert the data returned from the database to appropriate data types. Since the protocol is text-based everything I read from the database is plain text. When parsing timestamps I first tokenize the string and then convert each token to integer. And because the millisecond part is truncated, the last token is "88" instead of "880", and converting "88" yields another value that converting "880" to integer.
That's the default display format when using a cast to text.
If you want to see all three digits, use to_char()
SELECT to_char(dt.timestamp,'yyyy-mm-d hh24:mi:ss.ms')
FROM date_test dt;
will return 2019-06-12 12:37:07.880
It’s a matter of presentation only.
First note that 07.88 seconds and 07.880 seconds is the same amount of time (also 7.88 and 07.880000000 for that matter).
PostgreSQL internally represents a timestamp in a way that we shouldn’t be concerned about as long as it’s an unambiguous representation. When you retrieve the timestamp, it is formatted into some string. This is where PostgreSQL apparently chooses not to print redundant trailing zeros. So it’s probably not even correct to say that it truncates anything. It just refrains from generating that 0.
I think that the nice solution would be to modify your parser in C++ to accept any number of decimals and parse them correctly with and without trailing zeroes. Another solution that should work is given in the answer by a_horse_with_no_name.
I've come across a requirement to create a sequence in Postgres for generating a code (in string) which is expected to generate a unique code to increment by one for each new row and it should follow a six digit pattern.
For instance,
AC0001
AC0040
AC0201
AC3421
where the first two letters are chars and the remaining are integers.
I have created a sequence first,
CREATE SEQUENCE code_sequence START WITH 1
INCREMENT BY 1
CACHE 1;
Then, created a table,
CREATE TABLE account
(
code VARCHAR NOT NULL DEFAULT 'AC'||nextval('code_sequence'::regclass)::VARCHAR,
desc VARCHAR
);
This generates the code as AC1, AC2 etc. But, I want to have the code like AC0001, AC0002. Trying to "pad" zero's just after the 'AC'.
I would appreciate, if any one suggest a solution or idea for this problem.
Use to_char() to format the number:
CREATE TABLE account
(
code VARCHAR NOT NULL DEFAULT 'AC'||to_char(nextval('code_sequence'), 'FM0000'),
"desc" VARCHAR
);
Try the LPAD function.
CREATE TABLE account
(
code VARCHAR NOT NULL DEFAULT 'AC' || LPAD(nextval('code_sequence'::regclass), 4, '0')::VARCHAR,
desc VARCHAR
);
create table Department
(Dname varchar(255) NOT NULL, Dnumber int NOT NULL PRIMARY KEY, Mgr_SSN char(9) NOT NULL, Mgr_start_Date DATE);
insert into Department values('HR', '1', '11001', '2012-04-05 10:15:00');
I am getting the error "not valid month".
Should we define date format when we create the table?
I am using Oracle11g.
When you have a DATE column, you should always insert a DATE, not a VARCHAR2. Relying on implicit casting to correctly convert the string is a bad idea-- it is very easy for different sessions to have different NLS settings and, thus, to do the implicit conversion differently (either resulting in a different DATE or an error). The easiest way to do that is to use the to_date function.
insert into Department( dname,
dnumber,
mgr_ssn,
mgr_start_date )
values('HR',
1,
'11001',
to_date( '2012-04-05 10:15:00', 'yyyy-mm-dd hh24:mi:ss') );
I also modified the statement to list the columns, which is generally a good practice since it ensures that you don't have to look up the physical order of columns in the table every time and since it allows the INSERT statement to work in the future if you add new columns to the table. Since dnumber is a NUMBER, I also changed the INSERT statement to insert a number rather than inserting a string (again, don't rely on implicit conversion if there is no need to do so). I did not correct the apparent bug that you have a CHAR(9) column representing a social security number for which you are inserting a 5 character string.
I'm working with postgresql-9.1 recently.
For some reason I have to use a tech which does not support data type numeric but decimal. Unfortunately, the data type of columns which I've assigned decimal to them in my Postgresql are always numeric. I tried to alter the type, but it did not work though I've got the messages just like "Query returned successfully with no result in 12 ms".
SO, I want to know how can I get the columns to be decimal.
Any help will be highly appreciate.
e.g.
My creating clauses:
CREATE TABLE IF NOT EXISTS htest
(
dsizemin decimal(8,3) NOT NULL,
dsizemax decimal(8,3) NOT NULL,
hidentifier character varying(10) NOT NULL,
tgrade character varying(10) NOT NULL,
fdvalue decimal(8,3),
CONSTRAINT htest_pkey PRIMARY KEY (dsizemin , dsizemax , hidentifier , tgrade )
);
My altering clauses:
ALTER TABLE htest
ALTER COLUMN dsizemin TYPE decimal(8,3);
But it does not work.
In PostgreSQL, "decimal" is an alias for "numeric" which poses some problems when your app thinks it expects a type called "decimal" from the database. As Craig noted above, you can't even create a domain called "decimal"
There is no good workaround in the database side. The only thing you can do is change the application to expect a numeric data type back.
Use Numeric (precision, scale) to store decimals
precision represents the total number of expected digits on either side of the decimal point. scale is the number decimals you wish to store.
This Numeric (5,5) would imply you only want numbers less than 1 (negative or positive) with 5 decimal points. Debug, it may be Numeric (6,5) if the postgre sql errors out because it things the leading 0 is a decimal.
0.12345 would be an example of the above.
1.12345 would need a field Numeric (6,5)
100.12345 would need a field Numeric (8,5)
-100.12345 would need a field Numeric (8,5)
When you write a select statement to see the decimals, it rounds to 2; but if you do something like Select 100 * [field] from [table], then extra decimals should start appearing....
Thanks to a last minute client request a integer field in our database now needs to be a decimal, to two points.A value of 23 should become 23.00.
Is there a nice way I can convert the table and cast the data across?
I'll freely admit, I haven't done anything like this with PostgreSQL before so please be gentle with me.
Something like this should work:
alter table t alter column c type decimal(10,2);
Edit:
As #Oli stated in the comments; the first number is the entire length of the number (excluding the point) so the maxval for (10,2) would be 99999999.99
alter table table_name alter column columname type decimal(16,2);
for converting data type from int to decimal. with precession after decimal point 2 values it will come for example 10.285 then it will be 10.23.or in decimal place we can use numeric also it will work.