Well consider a table created like this:
CREATE TABLE public.test
(
id integer NOT NULL DEFAULT nextval('user_id_seq'::regclass),
name text,
PRIMARY KEY (id)
)
So the table has a unique 'id' column that auto generates default values using a sequence.
Now I wish to import data from a csv file, extending this table. However "obviously" the ids need to be unique, and thus I wish to let the database itself generate the ids, the csv file itself (coming from a complete different source) has hence an "empty column" for the ids:
,username
,username2
However if I then import this csv using psql:
\copy public."user" FROM '/home/paul/Downloads/test.csv' WITH (FORMAT csv);
The following error pops up:
ERROR: null value in column "id" violates not-null constraint
So how can I do this?
The empty colum from the CSV file is interpreted as SQL NULL, and inserting that value overrides the DEFAULT and leads to the error.
You should omit the empty column from the file and use:
\copy public."user"(name) FROM '...' (FORMAT 'csv')
Then the default value will be used for id.
Related
I am new in PostgreSQL and I am working with this database.
I got a file which I imported, and I am trying to get rows with a certain ID. But the ID is not defined, as you can see it in this picture:
so how do I access this ID? I want to use an SQL command like this:
SELECT * from table_name WHERE ID = 1;
If any order of rows is ok for you, just add a row number according to the current arbitrary sort order:
CREATE SEQUENCE tbl_tbl_id_seq;
ALTER TABLE tbl ADD COLUMN tbl_id integer DEFAULT nextval('tbl_tbl_id_seq');
The new default value is filled in automatically in the process. You might want to run VACUUM FULL ANALYZE tbl to remove bloat and update statistics for the query planner afterwards. And possibly make the column your new PRIMARY KEY ...
To make it a fully fledged serial column:
ALTER SEQUENCE tbl_tbl_id_seq OWNED BY tbl.tbl_id;
See:
Creating a PostgreSQL sequence to a field (which is not the ID of the record)
What you see are just row numbers that pgAdmin displays, they are not really stored in the database.
If you want an artificial numeric primary key for the table, you'll have to create it explicitly.
For example:
CREATE TABLE mydata (
id integer GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
obec text NOT NULL,
datum timestamp with time zone NOT NULL,
...
);
Then to copy the data from a CSV file, you would run
COPY mydata (obec, datum, ...) FROM '/path/to/csvfile' (FORMAT 'csv');
Then the id column is automatically filled.
I am writing a PLPGSQL function, that needs to import files into a table.
I have created a temporary table with 4 columns
CREATE TABLE IF NOT EXISTS tmp_ID_Customer (
ID int4 NULL,
Name varchar(2000) NULL,
CodeEx varchar(256) NULL,
AccountID varchar(256) NULL
)ON COMMIT DROP;
I am then trying to copy a file into this table, with the following
EXECUTE format('COPY tmp_ID_Customer FROM %L (FORMAT CSV, HEADER TRUE, DELIMITER(''|''))', _fileName);
The issue I have is some of these files only contain the first 3 columns.
So I am receiving an error saying
extra data after last expected column
I've tried specifying the columns, but as the final column doesn't always exist. I get an error.
Specify the columns you are copying:
COPY tmp_ID_Customer(id, name, codex) FROM ...
When trying to use the COPY command via SQL in Postgres 9.5.1 in a simple example database…
I am getting this error:
ERROR: invalid input syntax for integer: "Sally"
CONTEXT: COPY customer_, line 2, column id_: "Sally"
********** Error **********
ERROR: invalid input syntax for integer: "Sally"
SQL state: 22P02
Context: COPY customer_, line 2, column id_: "Sally"
…when importing this data in CSV (comma-separated value):
"first_name_","last_name_","phone_","email_"
"Sally","Jones","425.555.1324","s.jones#acme.com"
"Jarrod","Barkley","206.555.3454","j.barkley#example.com"
"Wendy","Melvin","415.555.2343","wendy#wendyandlisa.com"
"Lisa","Coleman","425.555.7282","lisa#wendyandlisa.com"
"Jesse","Johnson","507.555.7865","j.j#guitar.com"
"Jean-Luc","Martin","212.555.2244","jean-luc.martin#example.com"
…being imported via the following SQL executed in pgAdmin:
COPY customer_
FROM '/home/parallels/Downloads/customer_.csv'
CSV
HEADER
;
…into this table:
-- Table: public.customer_
-- DROP TABLE public.customer_;
CREATE TABLE public.customer_
(
id_ integer NOT NULL DEFAULT nextval('customer__id__seq'::regclass),
first_name_ text NOT NULL,
last_name_ text NOT NULL,
phone_ text NOT NULL DEFAULT ''::text,
email_ text NOT NULL DEFAULT ''::text,
CONSTRAINT pkey_customer_ PRIMARY KEY (id_)
)
WITH (
OIDS=FALSE
);
ALTER TABLE public.customer_
OWNER TO postgres;
COMMENT ON TABLE public.customer_
IS 'Represents a person whose pets visit our clinic.';
So it seems the first row containing the names of the columns is being processed successfully. The failure point is with the first data value in the first data line of the CSV. None of my imported data is of integer type, so the I am befuddled by the error message. The only integer is the id_ primary key, auto-incrementing SERIAL.
I did read the Question page on PG COPY error: invalid input syntax for integer. But that question did involve integer values, and the lack thereof in an empty quoted string being interpreted as a NULL. In my case here we have no integer values in the data; the only integer is the primary key SERIAL column with a DEFAULT generated value (not in the data being imported).
I also found the Question, PostgreSQL ERROR: invalid input syntax for integer. But it seems irrelevant.
Try specifying the columns . . . without the primary key:
COPY customer_ (first_name_ text, last_name_ text, phone_ text, email_ text)
FROM '/home/parallels/Downloads/customer_.csv'
CSV
HEADER
;
Without the column list, it is looking for a value for id_.
The import data file’s first row of column names are not used for mapping to the table columns. The HEADER flag merely tells Postgres to skip over that first line, as documented:
HEADER
Specifies that… on input, the first line is ignored. …
COPY table_name FROM 'C:\path\file.csv' DELIMITERS ',' CSV header;
This wasn't the OP's problem, but posting because this is one of the top results when I google the error message.
I was trying to import a .csv file with no header. Adding a header to the file and changing COPY ... CSV to COPY ... CSV HEADER in the sql command fixed the problem for me.
for example i need to export mytbl as csv
CREATE TABLE public.mytbl
(
id integer,
product character varying(20),
patent character varying(50)
)
WITH (
OIDS = FALSE
)
;
and i use the following query to export the mytbl into csv
copy(select * from mytbl) to 'D:\mytbl.csv' with csv header
and using COPY mytbl FROM 'D:\mytbl.csv' CSV HEADER this will inserts from csv
but i need to delete the existing data in mytbl before importing it from mytbl.csv,
when i deletes getting error
ERROR: update or delete on table "mytbl" violates foreign key constraint "mytblX_forinkey_productid" on table "mytblX"
how to overcome this ?
On PostgreSQL 9.2
It appears that your mytblX has a FK to mytbl. Before you can drop your mytbl you should ALTER TABLE mytblX DROP CONSTRAINT mytblX_forinkey_productid. Then you can copy the data back in and issue ALTER TABLE mytblX ADD your_table_constraint.
Note that FK constraints are based on an index so you should create the appropriate index on the newly copied in data before you recreate the FK constraint. Also note that the new data may not meet the requirements set by mytblX data; i.e. if that references a productid which is not in the data you copy into the database then you will have problems that need to be solved first (usually manually and tediously).
You can set the constraint as deferrable, and then defer it. This will let you delete the contents of the original table and reload it from the file within a single transaction. But if the file doesn't contain all the rows it needs to satisfy the constraint, then you will get an error on COMMIT.
I am following an PostgreSQL book, and had to import a CSV file into a table census.lu_tracts.
Problem: When performing the INSERT query as shown below, I get the error:
ERROR: duplicate key value violates unique constraint "pk_lu_tracts"
DETAIL: Key (tract_id)=(25001010800) already exists.
How did the key becomes duplicate? SELECT * from lu_tracs shows 0 rows.
CREATE SCHEMA census;
set search_path=census;
CREATE TABLE lu_tracts(tract_id varchar(11), tract_long_id varchar(25)
, tract_name varchar(150)
, CONSTRAINT pk_lu_tracts PRIMARY KEY (tract_id));
INSERT INTO lu_tracts( tract_id, tract_long_id, tract_name)
SELECT geo_id2, geo_id, geo_display
FROM staging.factfinder_import
WHERE geo_id2 ~ '^[0-9]+';
The right answer is DISTINCT ON (geo_id2) which will select only one row per geo_id2 (more in the manual), it should be accompanied by an ORDER BY clause that will specify what row will be choosen.