If I pg_restore --data-only into a table like this:
CREATE TABLE foo (
id int4 NOT NULL GENERATED ALWAYS AS IDENTITY,
bar varchar
);
what happens to the id column? Is the GENERATED ALWAYS ignored and the ids from the pg_dump file inserted, or are new ids generated?
The pg_restore docs say, under the --data-only flag:
Table data, large objects, and sequence values are restored, if present in the archive.
Does "sequence values" here include GENERATED ALWAYS AS IDENTITY?
I'm using Postgres 11 but it would be interesting to know if this behaviour has been the same since all generally supported versions (>=9.5).
With INSERT, you need to use OVERRIDING SYSTEM VALUE if you want to overrule the default value from the identity column, but with COPY there are no such restrictions.
That makes it simple for pg_dump. Here is an example of what a dump of a table with identity column looks like:
/* section = pre-data */
CREATE TABLE laurenz.identity (
id integer NOT NULL,
value text NOT NULL
);
ALTER TABLE laurenz.identity ALTER COLUMN id ADD GENERATED ALWAYS AS IDENTITY (
SEQUENCE NAME laurenz.identity_id_seq
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1
);
/* section = data */
COPY laurenz.identity (id, value) FROM stdin;
1 one
2 two
3 three
-10 weird
-5 also weird
\.
/* section = post-data */
SELECT pg_catalog.setval('laurenz.identity_id_seq', 3, true);
Related
In database dump created with pg_dump, some tables have DEFAULTs in the CREATE TABLE statement, i.e.:
CREATE TABLE test (
f1 integer DEFAULT nextval('test_f1_seq'::regclass) NOT NULL
);
But others have an additional ALTER statement:
ALTER TABLE ONLY test2 ALTER COLUMN f1 SET DEFAULT nextval('test2_f1_seq'::regclass);
What is the reason of this? All sequential fields were created with type SERIAL, but in the dump they look different, and I can't guess any rule for this.
The difference must be that in the first case, the sequence is “owned” by the table column.
You can specify this dependency using the OWNED BY clause when you create a sequence. A sequence that is owned by a column will automatically be dropped when the column is.
If a sequence is implicitly created by using serial, it will be owned by the column.
I am new in PostgreSQL and I am working with this database.
I got a file which I imported, and I am trying to get rows with a certain ID. But the ID is not defined, as you can see it in this picture:
so how do I access this ID? I want to use an SQL command like this:
SELECT * from table_name WHERE ID = 1;
If any order of rows is ok for you, just add a row number according to the current arbitrary sort order:
CREATE SEQUENCE tbl_tbl_id_seq;
ALTER TABLE tbl ADD COLUMN tbl_id integer DEFAULT nextval('tbl_tbl_id_seq');
The new default value is filled in automatically in the process. You might want to run VACUUM FULL ANALYZE tbl to remove bloat and update statistics for the query planner afterwards. And possibly make the column your new PRIMARY KEY ...
To make it a fully fledged serial column:
ALTER SEQUENCE tbl_tbl_id_seq OWNED BY tbl.tbl_id;
See:
Creating a PostgreSQL sequence to a field (which is not the ID of the record)
What you see are just row numbers that pgAdmin displays, they are not really stored in the database.
If you want an artificial numeric primary key for the table, you'll have to create it explicitly.
For example:
CREATE TABLE mydata (
id integer GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
obec text NOT NULL,
datum timestamp with time zone NOT NULL,
...
);
Then to copy the data from a CSV file, you would run
COPY mydata (obec, datum, ...) FROM '/path/to/csvfile' (FORMAT 'csv');
Then the id column is automatically filled.
I have a table with existing data. Is there a way to add a primary key without deleting and re-creating the table?
(Updated - Thanks to the people who commented)
Modern Versions of PostgreSQL
Suppose you have a table named test1, to which you want to add an auto-incrementing, primary-key id (surrogate) column. The following command should be sufficient in recent versions of PostgreSQL:
ALTER TABLE test1 ADD COLUMN id SERIAL PRIMARY KEY;
Older Versions of PostgreSQL
In old versions of PostgreSQL (prior to 8.x?) you had to do all the dirty work. The following sequence of commands should do the trick:
ALTER TABLE test1 ADD COLUMN id INTEGER;
CREATE SEQUENCE test_id_seq OWNED BY test1.id;
ALTER TABLE test1 ALTER COLUMN id SET DEFAULT nextval('test_id_seq');
UPDATE test1 SET id = nextval('test_id_seq');
Again, in recent versions of Postgres this is roughly equivalent to the single command above.
ALTER TABLE test1 ADD COLUMN id SERIAL PRIMARY KEY;
This is all you need to:
Add the id column
Populate it with a sequence from 1 to count(*).
Set it as primary key / not null.
Credit is given to #resnyanskiy who gave this answer in a comment.
To use an identity column in v10,
ALTER TABLE test
ADD COLUMN id { int | bigint | smallint}
GENERATED { BY DEFAULT | ALWAYS } AS IDENTITY PRIMARY KEY;
For an explanation of identity columns, see https://blog.2ndquadrant.com/postgresql-10-identity-columns/.
For the difference between GENERATED BY DEFAULT and GENERATED ALWAYS, see https://www.cybertec-postgresql.com/en/sequences-gains-and-pitfalls/.
For altering the sequence, see https://popsql.io/learn-sql/postgresql/how-to-alter-sequence-in-postgresql/.
I landed here because I was looking for something like that too. In my case, I was copying the data from a set of staging tables with many columns into one table while also assigning row ids to the target table. Here is a variant of the above approaches that I used.
I added the serial column at the end of my target table. That way I don't have to have a placeholder for it in the Insert statement. Then a simple select * into the target table auto populated this column. Here are the two SQL statements that I used on PostgreSQL 9.6.4.
ALTER TABLE target ADD COLUMN some_column SERIAL;
INSERT INTO target SELECT * from source;
ALTER TABLE test1 ADD id int8 NOT NULL GENERATED ALWAYS AS IDENTITY;
I've recently started developing apps with PostgreSQL as backend DB (imposed on me) with no previous experience of Postgres. So far it hasn't been too bad, but now I run into a problem to which I cannot find answer for.
I created a batch scripts that runs a pg_dump command for a particular database on the server. This batch file is executed on schedule by the pgAgent.
The pg_dump itself seems to work ok. All the database structure and data are dumped to a file. However the sequences are all set to 1. For example for table tbl_departments the sequence dump looks like this:
CREATE SEQUENCE "tbl_departments_iID_seq"
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;
ALTER TABLE "tbl_departments_iID_seq" OWNER TO postgres;
ALTER SEQUENCE "tbl_departments_iID_seq" OWNED BY tbl_departments."iID";
In this particular example the sequence should be set to start with 8, since the last inserted record has iID = 7.
How do I make the pg_dump set the sequence starting number the next one available for each table?
The command for dump is:
%PGBIN%pg_dump -h 192.168.0.112 -U postgres -F p -b -v --inserts -f "\\192.168.0.58\PostgresDB\backup\internals_db.sql" Internals
EDIT:
I think I have found the issue, although I still don't know how to resolve this:
If I open pgAdmin and generate CREATE script for tbl_departments, it look like this:
CREATE TABLE tbl_departments
(
"iID" serial NOT NULL, -- id, autoincrement
"c150Name" character varying(150) NOT NULL, -- human readable name for department
"bRetired" boolean NOT NULL DEFAULT false, -- if TRUE that it is no longer active
"iParentDept" integer NOT NULL DEFAULT 0, -- ID of the parent department
CONSTRAINT tbl_departments_pkey PRIMARY KEY ("iID")
)
The pg_dump statement is:
CREATE TABLE tbl_departments (
"iID" integer NOT NULL,
"c150Name" character varying(150) NOT NULL,
"bRetired" boolean DEFAULT false NOT NULL,
"iParentDept" integer DEFAULT 0 NOT NULL
);
ALTER TABLE tbl_departments OWNER TO postgres;
COMMENT ON TABLE tbl_departments IS 'list of departments';
COMMENT ON COLUMN tbl_departments."iID" IS 'id, autoincrement';
COMMENT ON COLUMN tbl_departments."c150Name" IS 'human readable name for department';
COMMENT ON COLUMN tbl_departments."bRetired" IS 'if TRUE that it is no longer active';
COMMENT ON COLUMN tbl_departments."iParentDept" IS 'ID of the parent department';
CREATE SEQUENCE "tbl_departments_iID_seq"
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;
ALTER TABLE "tbl_departments_iID_seq" OWNER TO postgres;
ALTER SEQUENCE "tbl_departments_iID_seq" OWNED BY tbl_departments."iID";
INSERT INTO tbl_departments VALUES (1, 'Information Technologies', false, 0);
INSERT INTO tbl_departments VALUES (2, 'Quality Control', false, 0);
INSERT INTO tbl_departments VALUES (3, 'Engineering', false, 0);
INSERT INTO tbl_departments VALUES (5, 'Quality Assurance', false, 0);
INSERT INTO tbl_departments VALUES (6, 'Production', false, 2);
ALTER TABLE ONLY tbl_departments
ADD CONSTRAINT tbl_departments_pkey PRIMARY KEY ("iID");
SELECT pg_catalog.setval('"tbl_departments_iID_seq"', 1, false);
the pg_dump sets the iID column to integer rather than serial, which disabled the auto-incrementation. The setval is also set to 1 rather than 7 as one would expect.
When I open the front-end application and go to add new department it fails because all I am providing is: name of new department, active/disabled (true/false), ID of parent dept. (0 if no parent).
I am expecting for the new record primary key iID to be created automatically by the DB, which as far as I know is an expected basic feature of any RDBMS.
because the pg_dump converts the serials to integers the auto-incrementation stops.
There is no reason for concern.
The generated SQL file will restore current values of sequences.
Open the file with an editor and look for setval.
There should be lines like this:
SELECT pg_catalog.setval('test_id_seq', 1234, true);
If you cannot find them it means that INSERT commands set the proper value of a sequence.
As Craig noticed, the current value of the sequence had to be equal to 1 at the time of dump of the original database. You have probably inserted iID values directly, not using default. In that case the sequence is not used.
Therefore I suggest start from the beginning, but in two databases:
make an sql dump like in the question,
create a new database,
run the sql script in the new database,
check whether corresponding serial columns have the same declaration in both databases,
compare current values of corresponding sequences in both databases.
the pg_dump sets the iID column to integer rather than serial, which disabled the auto-incrementation.
That's normal. See the manual.
SERIAL is basically just shorthand for CREATE SEQUENCE and then an integer column that makes that sequence its default for nextval('seq_name').
The setval is also set to 1 rather than 7 as one would expect.
I can only explain that one by assuming that the sequence start point is 1 in the DB. Perhaps due to a prior attempt at running DDL that altered it, such as a setval or alter sequence?
setval it to the start point you expect. Then, so long as you don't run other setval commands, alter sequence commands, etc, you'll be fine.
Or maybe the app inserted values directly, without using the sequence?
SELECT setval(pg_get_serial_sequence('public.table', 'id'), max(id)+1) FROM public.table;
I thought this would be simple, but I can't seem to use AUTO_INCREMENT in my db2 database. I did some searching and people seem to be using "Generated by Default", but this doesn't work for me.
If it helps, here's the table I want to create with the sid being auto incremented.
create table student(
sid integer NOT NULL <auto increment?>
sname varchar(30),
PRIMARY KEY (sid)
);
Any pointers are appreciated.
You're looking for is called an IDENTITY column:
create table student (
sid integer not null GENERATED ALWAYS AS IDENTITY (START WITH 1 INCREMENT BY 1)
,sname varchar(30)
,PRIMARY KEY (sid)
);
A sequence is another option for doing this, but you need to determine which one is proper for your particular situation. Read this for more information comparing sequences to identity columns.
You will have to create an auto-increment field with the sequence object (this object generates a number sequence).
Use the following CREATE SEQUENCE syntax:
CREATE SEQUENCE seq_person
MINVALUE 1
START WITH 1
INCREMENT BY 1
CACHE 10
The code above creates a sequence object called seq_person, that starts with 1 and will increment by 1. It will also cache up to 10 values for performance. The cache option specifies how many sequence values will be stored in memory for faster access.
To insert a new record into the "Persons" table, we will have to use the nextval function (this function retrieves the next value from seq_person sequence):
INSERT INTO Persons (P_Id,FirstName,LastName)
VALUES (seq_person.nextval,'Lars','Monsen')
The SQL statement above would insert a new record into the "Persons" table. The "P_Id" column would be assigned the next number from the seq_person sequence. The "FirstName" column would be set to "Lars" and the "LastName" column would be set to "Monsen".
hi If you are still not able to make column as AUTO_INCREMENT while creating table. As a work around first create table that is:
create table student(
sid integer NOT NULL
sname varchar(30),
PRIMARY KEY (sid)
);
and then explicitly try to alter column bu using the following
alter table student alter column sid set GENERATED BY DEFAULT AS
IDENTITY
Or
alter table student alter column sid set GENERATED BY DEFAULT
AS IDENTITY (start with 100)
Added a few optional parameters for creating "future safe" sequences.
CREATE SEQUENCE <NAME>
START WITH 1
INCREMENT BY 1
NO MAXVALUE
NO CYCLE
CACHE 10;