I want to make one field in my database nullable.
ALTER TABLE answers ALTER COLUMN author_id DROP NOT NULL;
The problem is that I have a lot of data in the database and I'm not sure if the alter command will block the table for write operations.
I know that changing the size or type of the column will cause the row exclusive lock https://www.postgresql.org/docs/current/static/explicit-locking.html#LOCKING-TABLES.
Will the alter I lock the table for writes too or it's safe to use?
Yes, the ALTER will require an exclusive lock on the table (including blocking write access to the table)
However, the duration of the lock is extremely short and does not depend on the size of the table. It is essentially only an update the internal system tables.
Related
Need help figuring this out. Tried googling, but found no specific answer.
Does altering a table in PostgreSQL like this will lock the table?
ALTER TABLE public.account ALTER COLUMN type acc_type VARCHAR(16) USING acc_type::VARCHAR(16);
Yes, that has to take an access exclusive lock, because the metadata change. Depending on what data type it was before that would be a very fast operation (e. g., it was varchar(10) before) or a very slow operation (if the previous type is not binary compatible).
I want to change a column to NOT NULL:
ALTER TABLE "foos" ALTER "bar_id" SET NOT NULL
The "foos" table has almost 1 000 000 records. It does fairly low volumes of writes, but quite constantly. There are a lot of reads.
In my experience, changing a column in a big table to NOT NULL like this can cause downtime in the app, presumably because it leads to (b)locks.
I've yet to find a good explanation corroborating this, though.
And if it is true, what can I do to avoid it?
EDIT: The docs (via this comment) say:
Adding a column with a DEFAULT clause or changing the type of an existing column will require the entire table and its indexes to be rewritten.
I'm not sure if changing NULL counts as "changing the type of an existing column", but I believe I did have an index on the column the last time I saw this issue.
Perhaps removing the index, making the column NOT NULL, and then adding the index back would improve things?
I think you can do that using a check constraint rather then set not null.
ALTER TABLE foos
add constraint id_not_null check (bar_id is not null) not valid;
This will still require an ACCESS EXCLUSIVE lock on the table, but it is very quick because Postgres doesn't validate the constraint (so it doesn't have to scan the entire table). This will already make sure that new rows (or changed rows) can not put a null value into that column
Then (after committing the alter table!) you can do:
alter table foos validate constraint id_not_null;
Which does not require an ACCESS EXCLUSIVE lock and still allows access to the table.
I have run into a case such that Pg always preferring into a sequential scan for a table that has around 70M rows. (Index scan is ideal for that query and i have confirmed it by setting enable_seq_scan=off, speed improved by 200x)
So, in order to help Pg understand my data better i executed this
ALTER TABLE tablename ALTER COLUMN columnname SET STATISTICS 1000;
Unfortunately this requires Update Exclusive lock which locks the entire table (too much lock).
Is there a solution to avoid locking for this statement ?
Data sharding is done for this table based on Primary Key Range, so I would like Pg to even understand my Pk better so that it knows which User has got large data. Will it be of use if i increase the statistics of PrimaryKey column as well ?
From the very docs you linked
SET STATISTICS
This form sets the per-column statistics-gathering target for subsequent ANALYZE operations. The target can be set in the range 0 to 10000; alternatively, set it to -1 to revert to using the system default statistics target (default_statistics_target). For more information on the use of statistics by the PostgreSQL query planner, refer to Section 14.2.
SET STATISTICS acquires a SHARE UPDATE EXCLUSIVE lock.
And, on the docs for Explicit Locking
SHARE UPDATE EXCLUSIVE
Conflicts with the SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, and ACCESS EXCLUSIVE lock modes. This mode protects a table against concurrent schema changes and VACUUM runs.
Acquired by VACUUM (without FULL), ANALYZE, CREATE INDEX CONCURRENTLY, and ALTER TABLE VALIDATE and other ALTER TABLE variants (for full details see ALTER TABLE).
So you can't change the schema, or vacuum while analytics are happening. So what? They should happen very fast. Almost instantly.
What type of table lock is acquired when you move a table (or rather a partition) from one tablespace to another in PostgreSQL 9.1?
Should I execute NO INHERIT first to detach it from the master table?
That will take an ACCESS EXCLUSIVE lock on the table (and its toast table and toast index, if they exist).
It does not matter if the table inherits from another table or not.
If the table has any indexes and you want to move those too, you'll have to explicitly move them with ALTER INDEX ... SET TABLESPACE ....
I think I read somewhere that running an ALTER TABLE foo ADD COLUMN baz text on a postgres database will not cause a read or write lock. Setting a default value causes locking, but allowing a null default prevents a lock.
I can't find this in the documentation, though. Can anyone point to a place that says, definitively, if this is true or not?
The different sorts of locks and when they're used are mentioned in the doc in
Table-level Locks. For instance, Postgres 11's ALTER TABLE may acquire a SHARE UPDATE EXCLUSIVE, SHARE ROW EXCLUSIVE, or ACCESS EXCLUSIVE lock.
Postgres 9.1 through 9.3 claimed to support two of the above three but actually forced Access Exclusive for all variants of this command. This limitation was lifted in Postgres 9.4 but ADD COLUMN remains at ACCESS EXCLUSIVE by design.
It's easy to check in the source code because there's a function dedicated to establishing the lock level needed for this command in various cases: AlterTableGetLockLevel in src/backend/commands/tablecmds.c.
Concerning how much time the lock is held, once acquired:
When the column's default value is NULL, the column's addition should be very quick because it doesn't need a table rewrite: it's only an update in the catalog.
When the column has a non-NULL default value, it depends on PostgreSQL version: with version 11 or newer, there is no immediate rewriting of all the rows, so it should be as fast as the NULL case. But with version 10 or older, the table is entirely rewritten, so it may be quite expensive depending on the table's size.
Adding new null column will lock the table for very very short time since no need to rewrite all data on disk. While adding column with default value requires PostgreSQL to make new versions of all rows and store them on the disk. And during that time table will be locked.
So when you need to add column with default value to big table it's recommended to add null value first and then update all rows in small portions. This way you'll avoid high load on disk and allow autovacuum to do it's job so you'll not end up doubling table size.
http://www.postgresql.org/docs/current/static/sql-altertable.html#AEN57290
"Adding a column with a non-null default or changing the type of an existing column will require the entire table and indexes to be rewritten."
So the documentation only specifies when the table is not rewritten.
There will always be a lock, but it will be very short in case the table is not to be rewritten.