Postgresql lost the autoincrement feature after a restore. My database was created on Windows 10 (v 10.1) and I restored it to Postgresql on Ubuntu (v 9.6). Now that I posted the question I saw that the versions are different. I didn't use any obscure feature, only tables, functions, and columns with serials. Also, the restore process didn't complain about anything. I checked the dump options but I couldn't find anything that caused the problem.
With Pgadmin right-clicking the table > scripts > create a script on my original table gives this:
CREATE TABLE public.produto
(
produto_id integer NOT NULL DEFAULT nextval('produto_produto_id_seq'::regclass),
...
);
In my server, the restored database. It seems it lost the feature.
CREATE TABLE public.produto
(
produto_id integer NOT NULL,
...
);
You didn't check for errors during restore of the database; there should have been a few.
A dump of a table like yours will look like this in PostgreSQL v10 (this is 10.3 and it looks slightly different in 10.1, but that's irrelevant to this case):
CREATE TABLE public.produto (
produto_id integer NOT NULL
);
ALTER TABLE public.produto OWNER TO laurenz;
CREATE SEQUENCE public.produto_produto_id_seq
AS integer
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;
ALTER TABLE public.produto_produto_id_seq OWNER TO laurenz;
ALTER SEQUENCE public.produto_produto_id_seq
OWNED BY public.produto.produto_id;
ALTER TABLE ONLY public.produto
ALTER COLUMN produto_id
SET DEFAULT nextval('public.produto_produto_id_seq'::regclass);
Now the problem is that AS integer was introduced to CREATE SEQUENCE in PostgreSQL v10, so that statement will fail with a syntax error in 9.6.
What is the consequence?
The table is created like in the first statement.
The third statement creating the sequence fails.
All the following statements that require the sequence will also fail.
Note: It is not supported to downgrade PostgeSQL with dump and restore.
The solution is to manually edit the dump until it works, in particular you'll have to remove the AS integer or AS bigint clause in CREATE SEQUENCE.
In database dump created with pg_dump, some tables have DEFAULTs in the CREATE TABLE statement, i.e.:
CREATE TABLE test (
f1 integer DEFAULT nextval('test_f1_seq'::regclass) NOT NULL
);
But others have an additional ALTER statement:
ALTER TABLE ONLY test2 ALTER COLUMN f1 SET DEFAULT nextval('test2_f1_seq'::regclass);
What is the reason of this? All sequential fields were created with type SERIAL, but in the dump they look different, and I can't guess any rule for this.
The difference must be that in the first case, the sequence is “owned” by the table column.
You can specify this dependency using the OWNED BY clause when you create a sequence. A sequence that is owned by a column will automatically be dropped when the column is.
If a sequence is implicitly created by using serial, it will be owned by the column.
I use knex to create a postgres table as following:
knex.schema.createTable('users', table => {
table.bigIncrements('user_id');
....
})
But after the table was created, the column user_id is a integer not the serial as expected.
The sql get by the pgAdmin is as following:
CREATE TABLE public.users
(
user_id bigint NOT NULL DEFAULT nextval('users_user_id_seq'::regclass),
....
)
And the consequence is that when I do insert statement, the user_id won't auto increment as expected.
Any gives?
====================
Currently I just changed to mysql connection, and the inserting works well. But if I changed the database back to postgresql, then inserting would fail due to the duplication of user_id. The code can be found here: https://github.com/buzz-buzz/buzz-service
serial and bigserial are not real types they are just shorthand for what pgAdmin is showing.
You will also find that a sequence has been created with the name users_user_id_seq when you look under sequences in pgAdmin.
I'm trying to migrate my sqlite database to postgresql.
I'm new to postgresql but stuck on this simple issue.
I fist used the pgloader tool to migrate to postgresql and have a working database. I see it created the table this way:
CREATE TABLE "AOrder"
(
"OrderNumber" bigserial NOT NULL,
"BuyerName" text,
etc
)
WITH (
OIDS=FALSE
);
ALTER TABLE "AOrder"
OWNER TO postgres;
Using pgadmin3 if I run the SQL query:
select
*
from AOrder
This returns the column names plus all the data as expected.
However, the following:
select
*
from "AOrder"
This returns just the column names and no data. Where's the data?
So it's not a problem with capitalization. Is there a setting in postgresql that is making this happen?
(This is ultimately the root problem for me since I am then using Sqlachemy which inserts the double quotes in queries. I could not figure out a way to change that behavior.)
Thanks!
I've recently started developing apps with PostgreSQL as backend DB (imposed on me) with no previous experience of Postgres. So far it hasn't been too bad, but now I run into a problem to which I cannot find answer for.
I created a batch scripts that runs a pg_dump command for a particular database on the server. This batch file is executed on schedule by the pgAgent.
The pg_dump itself seems to work ok. All the database structure and data are dumped to a file. However the sequences are all set to 1. For example for table tbl_departments the sequence dump looks like this:
CREATE SEQUENCE "tbl_departments_iID_seq"
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;
ALTER TABLE "tbl_departments_iID_seq" OWNER TO postgres;
ALTER SEQUENCE "tbl_departments_iID_seq" OWNED BY tbl_departments."iID";
In this particular example the sequence should be set to start with 8, since the last inserted record has iID = 7.
How do I make the pg_dump set the sequence starting number the next one available for each table?
The command for dump is:
%PGBIN%pg_dump -h 192.168.0.112 -U postgres -F p -b -v --inserts -f "\\192.168.0.58\PostgresDB\backup\internals_db.sql" Internals
EDIT:
I think I have found the issue, although I still don't know how to resolve this:
If I open pgAdmin and generate CREATE script for tbl_departments, it look like this:
CREATE TABLE tbl_departments
(
"iID" serial NOT NULL, -- id, autoincrement
"c150Name" character varying(150) NOT NULL, -- human readable name for department
"bRetired" boolean NOT NULL DEFAULT false, -- if TRUE that it is no longer active
"iParentDept" integer NOT NULL DEFAULT 0, -- ID of the parent department
CONSTRAINT tbl_departments_pkey PRIMARY KEY ("iID")
)
The pg_dump statement is:
CREATE TABLE tbl_departments (
"iID" integer NOT NULL,
"c150Name" character varying(150) NOT NULL,
"bRetired" boolean DEFAULT false NOT NULL,
"iParentDept" integer DEFAULT 0 NOT NULL
);
ALTER TABLE tbl_departments OWNER TO postgres;
COMMENT ON TABLE tbl_departments IS 'list of departments';
COMMENT ON COLUMN tbl_departments."iID" IS 'id, autoincrement';
COMMENT ON COLUMN tbl_departments."c150Name" IS 'human readable name for department';
COMMENT ON COLUMN tbl_departments."bRetired" IS 'if TRUE that it is no longer active';
COMMENT ON COLUMN tbl_departments."iParentDept" IS 'ID of the parent department';
CREATE SEQUENCE "tbl_departments_iID_seq"
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;
ALTER TABLE "tbl_departments_iID_seq" OWNER TO postgres;
ALTER SEQUENCE "tbl_departments_iID_seq" OWNED BY tbl_departments."iID";
INSERT INTO tbl_departments VALUES (1, 'Information Technologies', false, 0);
INSERT INTO tbl_departments VALUES (2, 'Quality Control', false, 0);
INSERT INTO tbl_departments VALUES (3, 'Engineering', false, 0);
INSERT INTO tbl_departments VALUES (5, 'Quality Assurance', false, 0);
INSERT INTO tbl_departments VALUES (6, 'Production', false, 2);
ALTER TABLE ONLY tbl_departments
ADD CONSTRAINT tbl_departments_pkey PRIMARY KEY ("iID");
SELECT pg_catalog.setval('"tbl_departments_iID_seq"', 1, false);
the pg_dump sets the iID column to integer rather than serial, which disabled the auto-incrementation. The setval is also set to 1 rather than 7 as one would expect.
When I open the front-end application and go to add new department it fails because all I am providing is: name of new department, active/disabled (true/false), ID of parent dept. (0 if no parent).
I am expecting for the new record primary key iID to be created automatically by the DB, which as far as I know is an expected basic feature of any RDBMS.
because the pg_dump converts the serials to integers the auto-incrementation stops.
There is no reason for concern.
The generated SQL file will restore current values of sequences.
Open the file with an editor and look for setval.
There should be lines like this:
SELECT pg_catalog.setval('test_id_seq', 1234, true);
If you cannot find them it means that INSERT commands set the proper value of a sequence.
As Craig noticed, the current value of the sequence had to be equal to 1 at the time of dump of the original database. You have probably inserted iID values directly, not using default. In that case the sequence is not used.
Therefore I suggest start from the beginning, but in two databases:
make an sql dump like in the question,
create a new database,
run the sql script in the new database,
check whether corresponding serial columns have the same declaration in both databases,
compare current values of corresponding sequences in both databases.
the pg_dump sets the iID column to integer rather than serial, which disabled the auto-incrementation.
That's normal. See the manual.
SERIAL is basically just shorthand for CREATE SEQUENCE and then an integer column that makes that sequence its default for nextval('seq_name').
The setval is also set to 1 rather than 7 as one would expect.
I can only explain that one by assuming that the sequence start point is 1 in the DB. Perhaps due to a prior attempt at running DDL that altered it, such as a setval or alter sequence?
setval it to the start point you expect. Then, so long as you don't run other setval commands, alter sequence commands, etc, you'll be fine.
Or maybe the app inserted values directly, without using the sequence?
SELECT setval(pg_get_serial_sequence('public.table', 'id'), max(id)+1) FROM public.table;